Tony Posted March 22, 2009 Report Share Posted March 22, 2009 One of the drives on the raid-10 array of Yoda has failed. This was causing large amounts of i/o wait on the node as the drive had not yet been dropped. Once the drive was dropped from the raid array due to inconsistencies the i/o wait has come back down. We are already preparing to replace the bad drive as our machines support hot-swap which means we can replace the drive on the fly. We do not have a time as of yet as we're in the process of coordinating with datacenter staff. We will update this thread once we have an exact time on when the bad drive is being replaced. Date: 03/21/2008 Start time (EST): 10:00pm EST End time (EST): Ongoing Estimated Down Time: None Duration: Unknown Mineeamappy, enquilkinsepe, Joxofssofsdip and 2 others 5 Link to comment Share on other sites More sharing options...
Tony Posted March 22, 2009 Author Report Share Posted March 22, 2009 We will be replacing the drive within the next 30 minutes. After that it'll be a matter of the rebuild which may cause slightly higher i/o wait but nothing major to slow things to a crawl. Link to comment Share on other sites More sharing options...
Tony Posted March 22, 2009 Author Report Share Posted March 22, 2009 The drive has been replaced. It's just rebuilding now. Link to comment Share on other sites More sharing options...
Tony Posted March 22, 2009 Author Report Share Posted March 22, 2009 It just finished rebuilding things should be getting back to normal load wise now. There were a few spikes near the end of the rebuild. There was not much we could do about it as keeping data safe out ways a few minutes of less then stellar performance. Link to comment Share on other sites More sharing options...
Recommended Posts