Hi,
At approximately 1106Z today, suite TFM1 in Telehouse lost power for
around 2 minutes. The only live BitFolk server in TFM1 at present
is dunkel.
Servers are set to not automatically start every VPS, as if they have
been rebooted it is usually for maintenance purposes. What should
have happened is that I should have been alerted, and then I would
have sorted things out.
A significant part of the internal monitoring is currently running
on a VPS on dunkel, but there is external monitoring for situations
such as this.
However, this has not worked and it wasn't until I received a phone
call on the urgent support number that I was aware of the problem.
As I type this, all VPSes on dunkel have now been restarted -- some
may still be fscking.
I am waiting on Telehouse for more information on the power outage
but obviously the real failing here is one of BitFolk's monitoring.
This is the second time there has been a monitoring failure; the
system had actually been changed since the first time, although that
was a different class of failure. I will be looking into why the
external monitoring failed, and changing how it's done if necessary.
In the meantime I will be adding a day of free service to all those
on dunkel who were affected.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
"I'd be happy to buy all variations of sex to ensure I got what I wanted."
-- Gary Coates (talking about cabling)