Site Access - Issues

[AUbicycles]

Currently all of the worlds bots are crawling the site and when it happens simultaneously it is a real drain and affects access.

I have adjusted some things so that 'real' people should have good access, though something to keep an eye one. If you spot any issues, let me know. From my side - the news content on the right side is not working as it should be which is a caching issue and hopefully resolves shortly.

Christopher

[AUbicycles]

A list of bots I have been seeing:

googlebot
archive.org bot
linkdex
bingbot
msnbot
baidu
Yahoo Slurp
exabot

And this is just at a glance and only the ones who call themselves a bot (as there are plenty of hidden / bad bots).

[cj7hawk]

I was getting error code 500 - Page cannot be accessed. However, the apache server on which the web site sits was performing correctly and delivering pages without issue, so it might have been a CGI based failure rather than a webserver failuer.

I got the 500 errors last night and the night before -

Regards
David

[AUbicycles]

Should be fixed now. When the site got bombarded by multiple crawlers - I adjusted some settings to try an alleviate the load.

One of those website settings (for the main BNA site) started making pages disappear so part of my trouble-shooting was to identify and fix that. If you get any more however - please let me know.

At times, access may still be shaky - but I am preparing for the new host which will magically fix all problems.