Tony Posted May 24, 2009 Report Share Posted May 24, 2009 Date: Sunday, May 24th, 2009 (05/24/2009) Start time: 07:45 PDT End time: 08:15 PDT Location: SEA01 Services Affected: Seattle Public Network Datacenter Engineers were made aware of loss of public connectivity in the Seattle datacenter at approximately 7:45 AM PDT. While the root cause of this issue is still being determined, engineers were able to restore service to the public network at approximately 8:15 AM PDT. A RFO will be forthcoming once the cause of the outage has been determined. Link to comment Share on other sites More sharing options...
Tony Posted May 24, 2009 Author Report Share Posted May 24, 2009 Date: 05-24-2009 Start time (CDT): 9:41am End time (CDT): 10:15am Services Affected: Public network connectivity to portions of SEA01.SR01 and SR02 Device: FCR01.SEA01 Location: SEA01 Duration: 30 minutes Description: At approximately 9:41am CDT, a datacenter technician in the sea01 facility failed to properly follow procedures when dealing with a standard administrative/maintenance issue related to the fcr01.sea01 network device. Immediately upon realizing that the mistake had caused a shutdown of the device, network teams reacted to the incident and worked to restore service. Future Mitigation: Employment of the technician in question has been terminated. In addition, procedures will be reviewed and all related staff will go through additional training on a regular basis. Additional safeguards are being reviewed for future implementation. Link to comment Share on other sites More sharing options...
Recommended Posts