Outage: Power Event [02/11/2008]


Tony

Recommended Posts

Today around 12:34 EDT our Venus, Mars and Mercury servers all went off line for about 8 minutes as a result of a power issue within the datacenter we use. Here is the official notice we received from the datacenter:

At approximately 12:34 EDT a PDU (power distribution unit) feeding 880 servers in Server Rooms 01, 02, and 03 tripped a main breaker causing interruption to all electrical devices downstream. Electricians on site determined that a short in power strip attached to a downstream circuit caused the upstream breaker to trip. They isolated this circuit and sequentially began to restore power to the affected infrastructure while monitoring its integrity. Datacenter Engineers then replaced the shorted power strip and restored power to the remaining devices.

Engineers are currently working to verify that all servers return to their normal state of operation, and will continue to closely monitor this circuit for additional issues over the next 24 hours.

Date: 02/11/2008

Start time (EDT): 12:34

End time (EDT): 12:42

Services affected: power

Device: limited subsets of sr01, 02, 03

Location: dal01 Datacenter

Duration: 8 minutes

Link to comment
Share on other sites