18 Oct
2019
18 Oct
'19
4:51 p.m.
Hi,
Some security issues have been found in the hypervisor software we
use, which we have to fix as they can theoretically allow privilege
escalation.
They are under embargo until Thursday 31 October, so we will most
likely do the work in the early hours of the morning (UK time) on
29, 30, and 31 October.
As usual this will entail a clean shutdown of your guest and then a
boot again 20–30 minutes later after the patching is done. Some time
next week an email will go out telling you of the two hour window in
which this work will take place for each of your VMs.
If the assigned window is unacceptable to you, we can most likely
move your VM to an already-patched host at a time of your choosing
before 31 October. When the direct email comes to let you know of
your maintenance window, if it's not acceptable then you can reply
to it to open a support ticket and we will work it out.
As usual, if you have opted in to suspend/restore then your guest
will be suspended to disk and restored again instead of shutdown and
booted. More info on that:
https://tools.bitfolk.com/wiki/Suspend_and_restore
For our own maintenance work we like to give more than 2 weeks of
notice. Unfortunate when dealing with security issues there is an
agreed embargo process and notice periods are much shorter. It is
preferable that there is ~2 weeks of notice rather than a "0-day"
exploit being unleashed.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
23 Oct
23 Oct
8:11 p.m.
New subject: [bitfolk] Reboots will be required for security patching, most likely 29/30/31 October
Hello,
Individual notification emails regarding the below maintenance
windows have now been sent out. If yours hasn't arrived, please
check spam folders etc.
Do note that the times are all in UTC. If you're in UK you are
currently at UTC+1, but British Summer Time ends on Sunday 27
October, so you will be back on UTC by the time of the work.
Cheers,
Andy
On Fri, Oct 18, 2019 at 04:51:53PM +0000, Andy Smith wrote:
Hi,
Some security issues have been found in the hypervisor software we
use, which we have to fix as they can theoretically allow privilege
escalation.
They are under embargo until Thursday 31 October, so we will most
likely do the work in the early hours of the morning (UK time) on
29, 30, and 31 October.
As usual this will entail a clean shutdown of your guest and then a
boot again 20–30 minutes later after the patching is done. Some time
next week an email will go out telling you of the two hour window in
which this work will take place for each of your VMs.
If the assigned window is unacceptable to you, we can most likely
move your VM to an already-patched host at a time of your choosing
before 31 October. When the direct email comes to let you know of
your maintenance window, if it's not acceptable then you can reply
to it to open a support ticket and we will work it out.
As usual, if you have opted in to suspend/restore then your guest
will be suspended to disk and restored again instead of shutdown and
booted. More info on that:
https://tools.bitfolk.com/wiki/Suspend_and_restore
For our own maintenance work we like to give more than 2 weeks of
notice. Unfortunate when dealing with security issues there is an
agreed embargo process and notice periods are much shorter. It is
preferable that there is ~2 weeks of notice rather than a "0-day"
exploit being unleashed.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
--
https://bitfolk.com/ -- No-nonsense VPS hosting
31 Oct
31 Oct
3:29 a.m.
New subject: [bitfolk] Reboots will be required for security patching, most likely 29/30/31 October
Hello,
This maintenance has now been completed without incident.
The security issues that were fixed will come out of embargo after
12:00Z today, so you can find details about them here after that
time:
https://xenbits.xen.org/xsa/
In addition, all CPU firmware was updated to the latest available
and SMT (hyperthreading) was disabled to better protect you against
existing and future CPU side channel attacks.
We may be able to revisit that decision when Xen's "Core Scheduling"
feature is available:
https://www.youtube.com/watch?v=qLWc2QvsU0A
https://www.slideshare.net/xen_com_mgr/xpdds19-core-scheduling-in-xen-jrgen…
This would mean that all the vCPUs you see in your VM would not
share a core with other VMs, as a lot of the side channel attacks
involve fetching data out of resources that are shared by all
threads on the same CPU core.
I am not sure we will ever get to the stage where that is safe,
however. For example, processes within your VM may still be able to
use that to get data from shared resources and leak across security
domains.
Disabling SMT has raised apparent CPU usage to a level which is
slightly alarming but not, I think, actually detrimental. You can
see that here:
https://bitfolk.com/techspec.html
where it is very obvious where the disabling of SMT happened for
each server (except "limoncello" and "talisker" which always had it
disabled)
I am not entirely sure that the old stats were valid. They assume
that every CPU thread could provide the same amount of compute
power, even when half of them are on the same core as the other
half. If a server showed as 40% loaded before, was it really, or was
there in reality a lower cap to what could possibly be used?
Anyway, there are two largely unused servers and I'm going to
rebalance heaviest users onto them to reduce overall CPU load.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
1657
days inactive
1670
days old
2 comments
1 participants
participants (1)
-
Andy Smith