Hi,
At approximately 0442Z today, 2011-04-05, I was alerted that several
VPSes on
barbar.bitfolk.com were not responding.
On investigation, the host itself was displaying kernel errors
related to Xen, and was also unresponsive. At this point I decided
to power cycle it.
When it came back up several of its RAID devices would not mount. It
looked like this was just because of the power cycle; a write had
managed to hit one of the disks but not any of the others, and a
mount could be forced. I didn't want to rush into anything though
due to the possibility of data loss.
After some further investigation, I was able to find a way to sync
the arrays and mount cleanly, and boot progressed. As of
approximately 0521Z VPSes have been starting up again on barbar and
this has just completed.
I am not seeing any problems at the moment, though there is high
disk contention due to many VPSes trying to fsck at once and
a background verify of RAID integrity taking place. barbar had
previously been up for nearly 2 years so many VPSes on it require
fsck.
For now we'll be keeping a close eye on the server and later today
will investigate further.
Please accept my apologies for the disruption.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting