Hello, On Wed, Jul 21, 2010 at 08:06:10PM +0000, Andy Smith wrote: Those watching the tweets will have seen all of this already, but here's the summary. The faustino maintenance was necessary because Telehouse want the remaining rack tenants in TFM1 to move to new racks so they can finish refitting the suite. This was planned months ago. On the night, it turned out that Telehouse had managed to allocate a new rack location that didn't actually exist. Everyone thought it did exist until the point where the floor tiles were to be lifted and power switched on, at which point there wasn't anything to turn on. There was no possibility of rescheduling, and given the late hour it took a long time to get to the point where the move could be made. This delayed the start of the maintenance from 2245Z to around 0119Z and made it slightly longer than it was intended to be (approximately 40 minutes, instead of the announced maximum of 30 minutes). This went according to plan. urquell was down between about 0009Z and about 0027Z. In both the faustino and urquell cases this morning I suspended the VMs to disk and restored them afterwards. This appears to have caused clocks to go a bit crazy, since they saw no ticks while they were suspended. On restore, ntpd will have noted the massive skew and killed itself, like this: Jul 24 00:22:16 spamd3 ntpd[13408]: synchronized to 209.237.247.192, stratum 3 Jul 24 00:22:16 spamd3 ntpd[13408]: time correction of 1365 seconds exceeds sanity limit (1000); set clock manually to the correct UTC time. Those with VPSes on faustino or urquell may wish to check that their time is correct and that ntpd is running. In future would you prefer if we never try to do this suspend/restore thing and instead just do a normal shutdown/boot? By the way, if you ask for it we can monitor your ntpd at no charge, which would alert you if it wasn't running or if it didn't have sync. Email to support(a)bitfolk.com if you want that. Cheers, Andy -- http://bitfolk.com/ -- No-nonsense VPS hosting