Connectivity issues (updated)
![[something went wrong]](http://www.twofo.co.uk/pics/explosion_sm.jpg "[something went wrong]")
There are presently a couple of issues with the server. Network connectivity appeared to be lost at around 12:40 today (Wednesday 30th) and DNS is still not working. For twofo users, this will primarily affect emails being sent out from the server, e.g. for forum use. I'm awaiting a response from my ISP as to whether something's gone wrong at their end.
Update - 18:00 - Details received from hosting provider
Today at 12:46 there was an unscheduled outage that affected all of our clients in the RHC data centre.
This was caused by a UPS failure. During routine testing one of the parallel UPS units failed, and it left the parallel UPS cluster in an ungraceful fashion. This caused a "brown out" of the protected power supply, which meant full voltage was not supplied to servers. All servers and network equipment rebooted at this point; power was restored immediately. As the network equipment and servers were rebooting clients would have been unable to access their services for a few minutes. Some clients whose servers did not successfully come back up would have seen longer outages.
The knock-on effect of the servers rebooting meant that our own external DNS resolving servers rebooted at the same time. On booting back up, all of our external DNS servers did not start the DNS daemon. This meant that clients who use our DNS service to resolve external domains could not do so for a short time. We quickly replaced these servers with alternative servers and will investigate why they did not come back up automatically further at a later date. Our primary and secondary DNS services were not affected, it was only the external resolvers.
The outage lasted less than 5 minutes.
Date: 30/04/2008 Time: 12:46 Duration: < 5 minutes
There are presently a couple of issues with the server. Network connectivity appeared to be lost at around 12:40 today (Wednesday 30th) and DNS is still not working. For twofo users, this will primarily affect emails being sent out from the server, e.g. for forum use. I'm awaiting a response from my ISP as to whether something's gone wrong at their end.
Update - 18:00 - Details received from hosting provider
Today at 12:46 there was an unscheduled outage that affected all of our clients in the RHC data centre.
This was caused by a UPS failure. During routine testing one of the parallel UPS units failed, and it left the parallel UPS cluster in an ungraceful fashion. This caused a "brown out" of the protected power supply, which meant full voltage was not supplied to servers. All servers and network equipment rebooted at this point; power was restored immediately. As the network equipment and servers were rebooting clients would have been unable to access their services for a few minutes. Some clients whose servers did not successfully come back up would have seen longer outages.
The knock-on effect of the servers rebooting meant that our own external DNS resolving servers rebooted at the same time. On booting back up, all of our external DNS servers did not start the DNS daemon. This meant that clients who use our DNS service to resolve external domains could not do so for a short time. We quickly replaced these servers with alternative servers and will investigate why they did not come back up automatically further at a later date. Our primary and secondary DNS services were not affected, it was only the external resolvers.
The outage lasted less than 5 minutes.
Date: 30/04/2008 Time: 12:46 Duration: < 5 minutes
This entry was posted on April 30th, 2008 at 23:00 by cocodude and is filed under twofo.