[bitfolk] Unscheduled reboot of obstler.bitfolk.com

Hi, At approximately 0047Z this morning I was alerted that a customer VPS on obstler could not be started. I investigated and was unable to start the VPS myself. I attempted to start a test VPS on this server and this failed in the same way. At this point it seemed inevitable that a server reboot would be required, so I tried restarting a few Xen-related services that I normally wouldn't touch on a production server, in the hope that this might fix things. Unfortunately it did not and only led to an inability to cleanly shut down the VPSes that were still running. So, at approximately 0110Z I rebooted obstler, and all VPSes on it will have experienced equivalent of power off and boot again. Boot and start of all VPSes was completed by 0123Z, but a few VPSes were still running fsck at that point. As of now, BitFolk's Nagios shows no more red than it was before the reboot. I still do not know the root cause of the problem, though it seems likely to be a bug in Xen. The server obstler is in need of upgrade to a newer version of Xen; this is scheduled for some time in January, and I would think it unlikely that this will reoccur before then (last reboot for this server was in July, under similar circumstances). Apologies for the disruption. Cheers, Andy -- http://bitfolk.com/ -- No-nonsense VPS hosting