Tony

This has been completed.

The verify/fix resolved any pending issues. We'll be monitoring this closely over the next 48 hours.

Well here's what we have thus far. The raid card thought it had a drive with bad data so it proceeded to scrub it and rebuild it. While doing this the server encountered an i/o error then bus errors on when connecting to the file system thus resulting in it being unable to do anything besides what was in memory. We are currently running a verify/fix on the raid to see if there are still some problems. If there are potentially bad drives they will be replaced.

Machine should be back up at this point we're investigating the exact cause still.

The machine has been giving bus errors so what we're doing is rebooting it and having a technician be on site on the console. If anything needs to be replaced such as the raid card they'll be able to do it.

We're currently investigating a hardware problem with the Jedi node which is causing instability with the machine. Once we have more information it'll be posted here.

Date: Sunday, June 13, 2009 (06/13/2009) Start time: 03:00 EDT End time: 04:00 EDT Duration: 1 hour Service Affected: WDC01 Public Network Reason: IOS Upgrade Device: cer01.wdc01, cer02.wdc01 Location: wdc01 ================================================== Datacenter Engineers will be conducting maintenance on the Customer Edge Routers in Washington, DC. During this time, each one will be upgraded and reloaded. We expect no impact to customers; however, you may notice some higher than normal latency as routes re-converge to redundant CER routers. Each reload should take approximately 15 minutes. Customers should not expect a service outage as a result of this work. ==================================================

Date: Saturday, June 6, 2009 (06/06/2009) Start time: 12:00 PDT End time: 01:00 PDT Duration: 1 hour Service Affected: SEA01 Public Network Reason: IOS Upgrade Device: cer01.sea01, cer02.sea01 Location: sea01 ================================================== Datacenter Engineers will be conducting maintenance on the Customer Edge Routers in Seattle, WA. During this time, each one will be upgraded and reloaded. We expect no impact to customers; however, you may notice some higher than normal latency as routes re-converge to redundant CER routers. Each reload should take approximately 15 minutes. Customers should not expect a service outage as a result of this work. ==================================================

Date: Sunday, May 31, 2009 (05/31/2009) Start time: 02:00 CDT End time: 05:00 CDT Duration: 3 hours Service Affected: DAL01 Public Network Reason: IOS Upgrade Device: cer01.dal01, cer02.dal01, cer03.dal01 Location: dal01 ================================================== Datacenter Engineers will be conducting maintenance on the Customer Edge Routers in Dallas, TX. During this time, each one will be upgraded and reloaded. We expect no impact to customers; however, you may notice some higher than normal latency as routes re-converge to redundant CER routers. Each reload should take approximately 15 minutes. Customers should not expect a service outage as a result of this work. ==================================================

Date: Saturday, May 30, 2009 (05/30/2009) Start time: 02:00 CDT End time: 03:00 CDT Duration: 1 hour Service Affected: DAL01 Public Network Reason: IOS Upgrade Device: dar01.dal01, dar02.dal01 Location: dal01 ================================================== Datacenter Engineers will be conducting maintenance on the Distribution Access Routers in Dallas, TX. During this time, each one will be upgraded and reloaded. We expect no impact to customers; however, you may notice some higher than normal latency as routes re-converge to the redundant DAR router. Each reload should take approximately 15 minutes. Customers should not expect a service outage as a result of this work. ==================================================

This has been completed with no service impact.

Date: 05/29/2009 Start time (CDT): 01:00:00 AM End time (CDT): 04:00:00 AM Services affected: DAL01 Internap 10G Link Location: DAL01 Duration: up to 3 hours ================================================== Internap will be replacing the router chassis to which our Data center connects. Customer Impact: Customers normally routed on this link may experience latency and/or packet-loss as traffic is rerouted out the Data centers other, redundant links. ==================================================

Yep you can.

We do not restrict content in any way other than asking that it be legal content. It's just not feasible for us to say no this or that so we simply do not.

Everything was repaired around 8:00PM CDT the same day sorry about the lack of update on this.

This has already happened but we did not pick up on it as there was no actual down time so we were not aware of any issues as we had also not had any users contact us about problems. There is still news to post so here's the information that has already been posted: Date: Monday, May 25th, 2009 (05.25.2009) Start time (CDT): 12:00 Services Affected: SEA01 Public / Private Device: TSN Network Location: sea01 Duration: Emerging Issue Description: The Datacenter Emergency Response Team is currently working on the issue affecting network connectivity. Engineers have located the problem and are currently working to resolve the issue. At this time, no ETA has been determined for service restoration. As resolution nears, details surrounding the service interruption will be posted inside the public and private portals. We appreciate your continued support. An official RFO will be available after complete resolution and post mortem meetings have concluded. -- Update 12:15 PM CDT -- We've determined that our Dallas to Seattle 10G circuit has been cut. This cut has also taken down our connectivity to a number of carriers in the Seattle facility (Verio, Level 3, Comcast, and the SIX fabric). Due to the sudden cut to all carriers involved, customers would have noticed a longer than average reroute time as all these providers must update their routing tables. This would have effected connectivity from Dallas and WDC to Seattle as well. -- Update 12:30 PM CDT -- Connectivity to Seattle from our other datacenters has been corrected. We're still working with providers to resolve the circuit cuts; however, as the case with any physical fiber cut, this could take time. If you're still experiencing issues, please open a ticket to have them looked at.

We're using whmcs currently as our own in house system is not nearly ready to go live.

If it's for another hosting account sure you can use the coupon.

/usr/bin/convert is where imagemagick is always located. The problem is some security we have in place to prevent binary access for PHP. So that is why you'd need to make a ticket for us to give you access to it.

At approximately 2:08pm CDT, Datacenter engineers were alerted to a power failure in portions of sr02 and sr03 in the dal01 facility. Datacenter engineers immediately began to diagnose the root cause of the issue and to restore power to the affected areas. A full datacenter emergency response team was called in to manage the event and to ensure that all affected servers were brought back online. The root cause of the issue has been determined to be a power surge in the circuit due to a failed capacitor in a UPS. The UPS in question was immediately removed from service as the datacenter staff reset main breakers in the affected PDU's that service the datacenter floor. Future mitigation: Service technicians, electricians and manufacturer engineers worked throughout the day and night to replace all of the failed components of the UPS. After replacement, the UPS was put through a full series of tests to verify operational capacity.

Date: 05-24-2009 Start time (CDT): 9:41am End time (CDT): 10:15am Services Affected: Public network connectivity to portions of SEA01.SR01 and SR02 Device: FCR01.SEA01 Location: SEA01 Duration: 30 minutes Description: At approximately 9:41am CDT, a datacenter technician in the sea01 facility failed to properly follow procedures when dealing with a standard administrative/maintenance issue related to the fcr01.sea01 network device. Immediately upon realizing that the mistake had caused a shutdown of the device, network teams reacted to the incident and worked to restore service. Future Mitigation: Employment of the technician in question has been terminated. In addition, procedures will be reviewed and all related staff will go through additional training on a regular basis. Additional safeguards are being reviewed for future implementation.

Date: Sunday, May 24th, 2009 (05/24/2009) Start time: 07:45 PDT End time: 08:15 PDT Location: SEA01 Services Affected: Seattle Public Network Datacenter Engineers were made aware of loss of public connectivity in the Seattle datacenter at approximately 7:45 AM PDT. While the root cause of this issue is still being determined, engineers were able to restore service to the public network at approximately 8:15 AM PDT. A RFO will be forthcoming once the cause of the outage has been determined.

The machine is back online refer to http://www.hawkhost.com/forums/showthread.php?t=418 for information about what caused it once we have that available.

We're back up but we have nothing else to report as to the cause or anything of that nature.

Update from datacenter: Datacenter engineers have detected power anomalies in server room 2 (sr02.dal01). The Emergency Response Team has been notified and engineers are working to identify and resolve the issue. As the situation unfolds, more information will be available via the emergency communication center inside the public and private portal. Due to the high volume of tickets and calls that may arise during this issue, delays may occur as engineers work to quickly and efficiently communicate all information. We appreciate your patience and cooperation. Updated information will be posted as soon as it becomes available.