Tony Posted August 4, 2009 Report Posted August 4, 2009 At about 1am PDT on August 4th we were alerted to bad blocks being reported on the Titan servers raid array. The server currently shows an optimal state on the array but bad blocks are something that should be investigated. We're going to run a verify on the raid array right now at 12pm PDT to verify the integrity of the data. We're hoping this reveals a drive to be bad which we will then replace. Date: 08/04/2009 Start time (PDT): 12:00pm End time (PDT): Unknown Duration: Unknown Estimated Down Time: None Currently
Tony Posted August 4, 2009 Author Report Posted August 4, 2009 We're going to replace the raid card it's acting very strange as if it's going to fail reporting all sorts of things that make no sense. We're going to do this ASAP and it'll require the machine be off line for hopefully no more than 20 minutes.
Tony Posted August 4, 2009 Author Report Posted August 4, 2009 The server should be going off line in the next 10 minutes to replace the card. Once replaced we'll re-check the array and hopefully find nothing wrong and it was just the raid card messing with us.
Tony Posted August 4, 2009 Author Report Posted August 4, 2009 Raid card has been replaced and machine is back online.
Tony Posted August 4, 2009 Author Report Posted August 4, 2009 Verify on the array is now being ran to check for any bad data or drives.
Tony Posted August 4, 2009 Author Report Posted August 4, 2009 That has been completed. That should be the end of this now hopefully.
Recommended Posts