BNA Server Down - Connection Issues

News from Bicycles Network Australia and the Australian Cycling Forums

Re: BNA Server Down - Connection Issues

Postby timgws » Fri Mar 27, 2009 11:54 am

Hi everyone,

My name is Tim, and I work at Digital Pacific. I thought I might just come and debunk a few of the myths that were displayed on this forum.

Chanboy wrote:I checked out the Digital Pacific website and some of their other clients, all seem to be performing ok.


This is true. No other clients of ours were getting any serious performance losses apart from bicycles.net.au. At Digital Pacific, we monitor all sites that have more then 500MB worth of traffic a day, this is on top of per-server monitoring. With this site monitoring, we are aiming to identify sites that are getting high load(s), and if necessary move the site to a more powerful server.

Image

This is the graph that was shown from our per-site monitoring for bicycles.net.au. To provide you all with a bit of history, the server before around 7:30PM (AEST / GMT+10) was moved already to three servers in different configurations in one day to find out what configurations were working or not.

Chanboy wrote:Software configuration - more likely what my guess would be.


Well, at this moment the team working on the performance issues on Wednesday are still not sure exactly where the issue was, but to narrow it down, it has something to do with the visualization technology "Virtuozzo". When the site was moved to any server that did not have Virtuozzo, the site started performing well.

Chanboy wrote:I would be investigating the setup of the server and also the forum software to see if any such configuration has been overlooked.


This is what Chris on Tuesday evening asked us to do, and for twelve hours on Wednesday, this is what four staff members (including me) were doing. On the phone while speaking to Chris, I told him that it could take anywhere between ten minutes to three days to find out where the bottleneck was. By 4pm that day, we had the website working on a server at a very acceptable speed. When we found that this configuration worked, we put the server in datacentre, and did some internal load testing. At 8pm, when we were sure that everything was working well, we decided to do the move to the new server.

Aushiker wrote:G'day

Using the information provided Chanboy I have filed a job with Digital Pacific, job no. OKJ-584202. Don't know if it will help or not but at least they know they have upset users.


Andrew (and others), we already knew that customers of bicycles.net.au were being affected, and we were working as hard as possible to fix these issues. At the end of the day, it was a mix of different things (both our fault - in the case of Virtuozzo) and the fault of the software (such as Typo3) that were causing serious performance issues.

Rockford wrote:Looks like they have moved you to your own server. I can't find any other domains being hosted on that IP 203.19.59.80

But if the other domains still hosted on .81 are having issues it could be a Datacentre issue or a switch/router issue.


Ever since the upgrade, bicycles.net.au has been on a Virtuozzo virtual environment, and the only site hosted on the IP was bicycles.net.au. The issue was never related to any networking issues.

Since the move to a dedicated server without any Virtuozzo technology running in the background, the site will now be running smoothly. I have made a few changes to the Typo3 config to ensure that there are no issues with MySQL connections all being taken up, as what was happening previously.

The virtuozzo container that Bicycles had did not have any limits to memory, disk or CPU usage, and the new dedicated server has a very similar configuration. Memory utilization on the old container was never more then 50%, so memory as well was never an issue.

The technical team here at Digital Pacific will be working hard to find out why when the website is hosted in the Parallels Virtuozzo environment the site is slow to load, and file a bug report with Parallels, the company who provides the software.

I hope that you understand that we have been working hard for a whole week to ensure that response times for the website were improved. We knew about the delays. Digital Pacific had a team of four working Wednesday for twelve hours to fix the issues that everyone have been seeing for the past week.

Hope this helps you all understand the issues that were being faced,
- Tim
timgws
 
Posts: 4
Joined: Fri Mar 27, 2009 9:35 am

by BNA » Fri Mar 27, 2009 12:12 pm

BNA
 

Re: BNA Server Down - Connection Issues

Postby sogood » Fri Mar 27, 2009 12:12 pm

Good job Tim/Digital Pacific, and thanks for the hard work! 8)
Bianchi, Ridley, Montague, GT, Garmin and All things Apple :)
RK wrote:And that is Wikipedia - I can write my own definition.
User avatar
sogood
 
Posts: 16929
Joined: Thu Aug 31, 2006 7:31 am
Location: Sydney AU

Re: BNA Server Down - Connection Issues

Postby Chanboy » Fri Mar 27, 2009 12:26 pm

nice and unexpected update. Thanks!
Chanboy
 
Posts: 690
Joined: Mon Jan 12, 2009 8:51 am
Location: Sydney

Re: BNA Server Down - Connection Issues

Postby casual_cyclist » Fri Mar 27, 2009 12:29 pm

sogood wrote:Good job Tim/Digital Pacific, and thanks for the hard work! 8)


+1. It's great the forum seems to be working fine now.
<removed by request>
User avatar
casual_cyclist
 
Posts: 7312
Joined: Fri Feb 20, 2009 10:41 am
Location: Kewdale

Re: BNA Server Down - Connection Issues

Postby Boognoss » Fri Mar 27, 2009 12:30 pm

Chanboy wrote:nice and unexpected update. Thanks!


+2. Followup service like this is all too uncommon these days (especially acknowledgement that there was a problem before it mysteriously fixed itself). Well done.
n = 3 for now

Image
User avatar
Boognoss
Super Mod
Super Mod
 
Posts: 5770
Joined: Sat Nov 10, 2007 10:45 am
Location: Castle Hill, NSW

Re: BNA Server Down - Connection Issues

Postby sharktamin » Fri Mar 27, 2009 8:11 pm

sogood wrote:Good job Tim/Digital Pacific, and thanks for the hard work! 8)


+2
Three cheers for Christopher and for tim and staff at DP.
User avatar
sharktamin
 
Posts: 737
Joined: Sat Jan 31, 2009 4:53 pm

Re: BNA Server Down - Connection Issues

Postby Chooky78 » Fri Mar 27, 2009 8:16 pm

sogood wrote:Bug? Search function can't find posts from earlier time frames ie. Before the move.


Same here... Still can't search any historical posts...
2005 Yamaha R6 (R46 Yellow/Black)
2006 Giant Alias MTB (Grey)
2007 Giant XTC C1 - Project MTB (Carbon)
2007 Teschner Euro Pro - Custom Road (Carbon/Red)
User avatar
Chooky78
 
Posts: 117
Joined: Sat Apr 26, 2008 3:04 am
Location: Toowoomba

Re: BNA Server Down - Connection Issues

Postby Mulger bill » Fri Mar 27, 2009 9:43 pm

G'Day Tim, welcome outside. :wink:

Another prop from me, and thanks. Most of the tech was beyond me, but I can smell bloody hard work when I read it.

We owe you and the team a beer. Each :wink:

Shaun
...whatever the road rules, self-preservation is the absolute priority for a cyclist when mixing it with motorised traffic.
London Boy 29/12/2011
User avatar
Mulger bill
Super Mod
Super Mod
 
Posts: 25681
Joined: Sun Sep 24, 2006 2:41 pm
Location: Sunbury Vic

Re: BNA Server Down - Connection Issues

Postby AUbicycles » Sat Mar 28, 2009 6:06 am

Hello,
Thank you Tim for your post. After speaking with Digital Pacific after all of the issues they organised a team to attack and solve the issue.

Also, thank you for members for such a positive reaction - it is certainly not everyday that a host will communicate directly.

Cheers
Christopher
User avatar
AUbicycles
Site Admin
Site Admin
 
Posts: 9069
Joined: Tue Aug 23, 2005 2:14 am
Location: Sydney

Re: BNA Server Down - Connection Issues

Postby philip » Sat Mar 28, 2009 4:55 pm

+another thanks Tim. I retract my "incompetent" comment, it was just very frustrating at the time as we love our bicycle talk :) cheers, and yes, please pass on thanks to the team that identified/solved the problems.
User avatar
philip
 
Posts: 1604
Joined: Wed Sep 17, 2008 1:57 pm
Location: Lane Cove, Sydney

Re: BNA Server Down - Connection Issues

Postby }SkOrPn--7 » Sun Mar 29, 2009 11:17 am

I gotta say this is the first time that this forum now flies and is so responsive it's great not having to have other sites open reading while this site once took for ever too load.
}SkOrPn--7
 
Posts: 1986
Joined: Sun Oct 12, 2008 6:15 pm

Re: BNA Server Down - Connection Issues

Postby Postman Pat » Sun Mar 29, 2009 2:08 pm

}SkOrPn--7 wrote:I gotta say this is the first time that this forum now flies and is so responsive it's great not having to have other sites open reading while this site once took for ever too load.

It has been awhile and things are starting to settle down. While I didn't like the new format to start with it is Growing on me as I am finding the buttons I regularly use :lol: and I wish to add my thanks for all the work you and the Moderators Put in to keeping the site running. :D :mrgreen:
Image
User avatar
Postman Pat
 
Posts: 246
Joined: Sat Mar 22, 2008 1:14 pm
Location: Tamworth

Re: BNA Server Down - Connection Issues

Postby AUbicycles » Tue Mar 31, 2009 5:28 am

I am posting this in a few threads that have dealt with the look and feel (covering all bases).
You can now change the look and feel of the forum - there are 3 styles to choose from - quick and easy.

Here is the thread with details and instructions >
User avatar
AUbicycles
Site Admin
Site Admin
 
Posts: 9069
Joined: Tue Aug 23, 2005 2:14 am
Location: Sydney

Re: BNA Server Down - Connection Issues

Postby Deanj » Thu Apr 02, 2009 5:41 am

Just had 5-10 mins of not being able to get in.

"Think" the error message was 403-You are forbidden to access to the server. Clicked on other sites before I read it fully not thinking :roll:
Deanj
 
Posts: 1698
Joined: Mon Aug 06, 2007 8:03 pm
Location: Adelaide

Re: BNA Server Down - Connection Issues

Postby AUbicycles » Thu Apr 02, 2009 7:09 am

Hi,
I changed a setting - killed the site - and got a prompt phone call from one of my programmers who was not impressed.
No more pressing buttons for me... well, maybe just the submit button.
User avatar
AUbicycles
Site Admin
Site Admin
 
Posts: 9069
Joined: Tue Aug 23, 2005 2:14 am
Location: Sydney

Re: BNA Server Down - Connection Issues

Postby Deanj » Thu Apr 02, 2009 5:49 pm

:lol: Thought I was banned for a few minutes.
Deanj
 
Posts: 1698
Joined: Mon Aug 06, 2007 8:03 pm
Location: Adelaide

Re: BNA Server Down - Connection Issues

Postby timgws » Fri Apr 03, 2009 9:53 am

While Digital Pacific do a few updates to the server this server will come and go. In total the server should only be down for two minutes, just giving the heads up.

We will be updating the version of PHP (as to Chris' orders ;-) ) to the latest PHP.

Thanks,
-- Tim
timgws
 
Posts: 4
Joined: Fri Mar 27, 2009 9:35 am

Re: BNA Server Down - Connection Issues

Postby RICHARDH » Fri Apr 03, 2009 10:01 am

Just got a Forbidden message come up a few minutes ago, got worried and thought my post regarding Dean Brogans 1 match ban had upset the Essendon supporting Mod. :shock:
We are stuck with technology when what we really want is just stuff that works (Douglas Adams)
User avatar
RICHARDH
 
Posts: 1306
Joined: Tue Sep 16, 2008 2:46 pm
Location: adelaide

Re: BNA Server Down - Connection Issues

Postby timgws » Fri Apr 03, 2009 10:10 am

RICHARDH wrote:Just got a Forbidden message come up a few minutes ago, got worried and thought my post regarding Dean Brogans 1 match ban had upset the Essendon supporting Mod. :shock:


Heh, can you please try it again and tell us if it still happens? Thanks.

EDIT: The server is now updated, and seems to be working OK :)
timgws
 
Posts: 4
Joined: Fri Mar 27, 2009 9:35 am

Re: BNA Server Down - Connection Issues

Postby AUbicycles » Fri Apr 03, 2009 8:44 pm

Thanks Tim,

-
With the new server, there are still a few more bits and pieces to do - now mainly functionality for the main BNA site.
In completing these we expect no major outages - though am happy for problems or issues to be reported.

Cheers
Christopher
User avatar
AUbicycles
Site Admin
Site Admin
 
Posts: 9069
Joined: Tue Aug 23, 2005 2:14 am
Location: Sydney

Previous

Return to Info / announcements

Who is online

Users browsing this forum: AUbicycles, reefer



Popular Bike Shops
Torpedo 7 Torpedo7 AU
Ground Effect Ground Effect NZ
Chain Reaction Cycles CRC UK
Wiggle Wiggle UK
Ebay Ebay AU

“Bicycles BNA Twitter
“Bicycles BNA Facebook
“Google+ BNA Google+
“Bicycles BNA Newsletter

> FREE BNA Stickers
> BNA Cycling Kit