Site Access - Issues

User avatar
AUbicycles
Site Admin
Site Admin
Posts: 12528
Joined: Tue Aug 23, 2005 2:14 am
Location: Sydney
Contact:

Site Access - Issues

Postby AUbicycles » Mon May 09, 2016 12:25 pm

Currently all of the worlds bots are crawling the site and when it happens simultaneously it is a real drain and affects access.

I have adjusted some things so that 'real' people should have good access, though something to keep an eye one. If you spot any issues, let me know. From my side - the news content on the right side is not working as it should be which is a caching issue and hopefully resolves shortly.

Christopher

User avatar
AUbicycles
Site Admin
Site Admin
Posts: 12528
Joined: Tue Aug 23, 2005 2:14 am
Location: Sydney
Contact:

Re: Site Access - Issues

Postby AUbicycles » Mon May 09, 2016 1:29 pm

A list of bots I have been seeing:

googlebot
archive.org bot
linkdex
bingbot
msnbot
baidu
Yahoo Slurp
exabot

And this is just at a glance and only the ones who call themselves a bot (as there are plenty of hidden / bad bots).

cj7hawk
Posts: 1144
Joined: Sun Jan 18, 2015 9:00 pm

Re: Site Access - Issues

Postby cj7hawk » Thu May 12, 2016 9:37 am

I was getting error code 500 - Page cannot be accessed. However, the apache server on which the web site sits was performing correctly and delivering pages without issue, so it might have been a CGI based failure rather than a webserver failuer.

I got the 500 errors last night and the night before -

Regards
David

User avatar
AUbicycles
Site Admin
Site Admin
Posts: 12528
Joined: Tue Aug 23, 2005 2:14 am
Location: Sydney
Contact:

Re: Site Access - Issues

Postby AUbicycles » Thu May 12, 2016 9:47 am

Should be fixed now. When the site got bombarded by multiple crawlers - I adjusted some settings to try an alleviate the load.

One of those website settings (for the main BNA site) started making pages disappear so part of my trouble-shooting was to identify and fix that. If you get any more however - please let me know.

At times, access may still be shaky - but I am preparing for the new host which will magically fix all problems.

Who is online

Users browsing this forum: No registered users