BNA Server Down - Connection Issues

[timgws]

Hi everyone,

My name is Tim, and I work at Digital Pacific. I thought I might just come and debunk a few of the myths that were displayed on this forum.

I checked out the Digital Pacific website and some of their other clients, all seem to be performing ok.

This is true. No other clients of ours were getting any serious performance losses apart from bicycles.net.au. At Digital Pacific, we monitor all sites that have more then 500MB worth of traffic a day, this is on top of per-server monitoring. With this site monitoring, we are aiming to identify sites that are getting high load(s), and if necessary move the site to a more powerful server.



This is the graph that was shown from our per-site monitoring for bicycles.net.au. To provide you all with a bit of history, the server before around 7:30PM (AEST / GMT+10) was moved already to three servers in different configurations in one day to find out what configurations were working or not.

Software configuration - more likely what my guess would be.

Well, at this moment the team working on the performance issues on Wednesday are still not sure exactly where the issue was, but to narrow it down, it has something to do with the visualization technology "Virtuozzo". When the site was moved to any server that did not have Virtuozzo, the site started performing well.

I would be investigating the setup of the server and also the forum software to see if any such configuration has been overlooked.

This is what Chris on Tuesday evening asked us to do, and for twelve hours on Wednesday, this is what four staff members (including me) were doing. On the phone while speaking to Chris, I told him that it could take anywhere between ten minutes to three days to find out where the bottleneck was. By 4pm that day, we had the website working on a server at a very acceptable speed. When we found that this configuration worked, we put the server in datacentre, and did some internal load testing. At 8pm, when we were sure that everything was working well, we decided to do the move to the new server.

G'day

Using the information provided Chanboy I have filed a job with Digital Pacific, job no. OKJ-584202. Don't know if it will help or not but at least they know they have upset users.

Andrew (and others), we already knew that customers of bicycles.net.au were being affected, and we were working as hard as possible to fix these issues. At the end of the day, it was a mix of different things (both our fault - in the case of Virtuozzo) and the fault of the software (such as Typo3) that were causing serious performance issues.

Looks like they have moved you to your own server. I can't find any other domains being hosted on that IP 203.19.59.80

But if the other domains still hosted on .81 are having issues it could be a Datacentre issue or a switch/router issue.

Ever since the upgrade, bicycles.net.au has been on a Virtuozzo virtual environment, and the only site hosted on the IP was bicycles.net.au. The issue was never related to any networking issues.

Since the move to a dedicated server without any Virtuozzo technology running in the background, the site will now be running smoothly. I have made a few changes to the Typo3 config to ensure that there are no issues with MySQL connections all being taken up, as what was happening previously.

The virtuozzo container that Bicycles had did not have any limits to memory, disk or CPU usage, and the new dedicated server has a very similar configuration. Memory utilization on the old container was never more then 50%, so memory as well was never an issue.

The technical team here at Digital Pacific will be working hard to find out why when the website is hosted in the Parallels Virtuozzo environment the site is slow to load, and file a bug report with Parallels, the company who provides the software.

I hope that you understand that we have been working hard for a whole week to ensure that response times for the website were improved. We knew about the delays. Digital Pacific had a team of four working Wednesday for twelve hours to fix the issues that everyone have been seeing for the past week.

Hope this helps you all understand the issues that were being faced,
- Tim

[sogood]

Good job Tim/Digital Pacific, and thanks for the hard work!

[Chanboy]

nice and unexpected update. Thanks!

[casual_cyclist]

Good job Tim/Digital Pacific, and thanks for the hard work!

+1. It's great the forum seems to be working fine now.

[Boognoss]

nice and unexpected update. Thanks!

+2. Followup service like this is all too uncommon these days (especially acknowledgement that there was a problem before it mysteriously fixed itself). Well done.

[sharktamin]

Good job Tim/Digital Pacific, and thanks for the hard work!

+2
Three cheers for Christopher and for tim and staff at DP.

[Chooky78]

Bug? Search function can't find posts from earlier time frames ie. Before the move.

Same here... Still can't search any historical posts...

[Mulger bill]

G'Day Tim, welcome outside.

Another prop from me, and thanks. Most of the tech was beyond me, but I can smell bloody hard work when I read it.

We owe you and the team a beer. Each

Shaun

[AUbicycles]

Hello,
Thank you Tim for your post. After speaking with Digital Pacific after all of the issues they organised a team to attack and solve the issue.

Also, thank you for members for such a positive reaction - it is certainly not everyday that a host will communicate directly.

Cheers
Christopher

[philip]

+another thanks Tim. I retract my "incompetent" comment, it was just very frustrating at the time as we love our bicycle talk cheers, and yes, please pass on thanks to the team that identified/solved the problems.

[}SkOrPn--7]

I gotta say this is the first time that this forum now flies and is so responsive it's great not having to have other sites open reading while this site once took for ever too load.

[Postman Pat]

I gotta say this is the first time that this forum now flies and is so responsive it's great not having to have other sites open reading while this site once took for ever too load.

It has been awhile and things are starting to settle down. While I didn't like the new format to start with it is Growing on me as I am finding the buttons I regularly use and I wish to add my thanks for all the work you and the Moderators Put in to keeping the site running.

[AUbicycles]

I am posting this in a few threads that have dealt with the look and feel (covering all bases).
You can now change the look and feel of the forum - there are 3 styles to choose from - quick and easy.

Here is the thread with details and instructions >

[Deanj]

Just had 5-10 mins of not being able to get in.

"Think" the error message was 403-You are forbidden to access to the server. Clicked on other sites before I read it fully not thinking

[AUbicycles]

Hi,
I changed a setting - killed the site - and got a prompt phone call from one of my programmers who was not impressed.
No more pressing buttons for me... well, maybe just the submit button.

[Deanj]

Thought I was banned for a few minutes.

[timgws]

While Digital Pacific do a few updates to the server this server will come and go. In total the server should only be down for two minutes, just giving the heads up.

We will be updating the version of PHP (as to Chris' orders ) to the latest PHP.

Thanks,
-- Tim

[RICHARDH]

Just got a Forbidden message come up a few minutes ago, got worried and thought my post regarding Dean Brogans 1 match ban had upset the Essendon supporting Mod.

[timgws]

Just got a Forbidden message come up a few minutes ago, got worried and thought my post regarding Dean Brogans 1 match ban had upset the Essendon supporting Mod.

Heh, can you please try it again and tell us if it still happens? Thanks.

EDIT: The server is now updated, and seems to be working OK

[AUbicycles]

Thanks Tim,

-
With the new server, there are still a few more bits and pieces to do - now mainly functionality for the main BNA site.
In completing these we expect no major outages - though am happy for problems or issues to be reported.

Cheers
Christopher