Hi,
We were recently made aware that our Cacti¹ bandwidth graphs for a
particular customer were dramatically different from reality.
On investigation I realised that it was a bug in the Linux kernel
where it wasn't using 64-bit counters for Xen backend network
devices. As a result for readings above ~228Mbit/s the counter was
wrapping twice and reporting incorrect values on an SNMP read (as
used by Cacti).
This is a fairly minor issue because we do not use SNMP counters for
billing. It does mean that if your VPS has ever done more than about
228Mbit/s average in a 5 minute period that Cacti won't be showing
it properly.
The kernel bug has since been fixed but deploying a fixed version
would involve using a self-compiled² backports kernel. I am not
going to do this because I haven't tested it enough yet.
Instead I have identified the 30 or so customers that have ever
recorded that much bandwidth use in the last 12 months and am adding
new bandwidth graphs for you, using 1-minute polling. Also new
customers will have the 1-minute resolution graphs. That should be
safe to about 1.1Gbit/s.
So, if you are looking at Cacti and see you now have two bandwidth
graphs with one cutting off where the other began, this is the
reason why.
I wrote a blog article about this at:
http://strugglers.net/~andy/blog/2017/09/03/when-is-a-64-bit-counter-not-a-…
Cheers,
Andy
¹ https://tools.bitfolk.com/cacti/ - Log in with your usual BitFolk credentials
² We already use self-compiled kernels based on Debian kernel
packages, because some security patches have not yet made it into
Debian's packages. Building the kernel isn't the problem, it's
testing it well enough.
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
Unfortunately some more serious security issues have been uncovered
in Xen which affect versions and configurations which we have
deployed.
These were pre-disclosed today, with full public disclosure coming
on Tuesday 15 August as normal.
So, we're going to have to patch everything and reboot before then.
This will very likely be taking place over three nights starting in
the early hours (BST) of Saturday 12 August, but we will be sending
out an individual email to every customer confirming when they will
be affected.
For those unaware of what this entails, it means that at
some point within an hour-long maintenance window we will shut your
VPS down cleanly as the machine it's on is shut down, and then boot
it again once the machine has booted up. It typically takes 5-10
minutes.
Unlike last time which required an upgrade of major version, this
time you should be able to opt for your VPS to be suspended to and
restored from SSD if you don't like losing program state:
https://tools.bitfolk.com/wiki/Suspend_and_restore
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
I've updated our Xen Shell to support installation of the new Debian
stable release, version 9 (stretch) and Debian testing (buster).
This requires you to be running version v1.48bitfolk45. You can see
the version of the running Xen Shell by typing "help". If you're
running an older version then type "exit" until you are logged out
then log in again.
That is for clean (re)installs using the "install" command. More
info here:
https://tools.bitfolk.com/wiki/Using_the_self-serve_net_installer
Debian does of course also support in-place upgrade.
If instead you would like a new VPS account free for two weeks to
migrate your services into then we can do that:
https://tools.bitfolk.com/wiki/Migrating_to_a_new_VPS
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
Unfortunately during the work for the next Xen release, developers
have found a bunch of serious security issues which affect versions
and configurations we have deployed.
These were pre-disclosed today, with full public disclosure coming
on Tuesday 20 June as normal.
So, we're going to have to patch everything and reboot before then.
This will very likely be taking place over three nights starting in
the early hours of Saturday 17 June, but we will be sending out an
individual email to every customer confirming when they will be
affected.
For those unaware of what this entails, it means that at some point
within an hour-long maintenance window we will shut your VPS down
cleanly as the machine it's on is shut down, and then boot it again
once the machine has booted up. It typically takes 5-10 minutes.
You can opt for your VPS to be suspended to and restored from SSD if
you don't like losing program state:
https://tools.bitfolk.com/wiki/Suspend_and_restore
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
At around 06:52Z today we lost IPv6 connectivity for about 12
minutes. This was due to one of our colo provider's routers
crashing.
They run a redundant network so it should failover, and did for
IPv4, but we're experiencing some problems with IPv6 failover which
meant it took longer than we would like or find acceptable.
The same thing happened last Thursday and that time initially due to
a miscommunication we believed that v6 was down because of the
router failure, when in fact it was not expected under those
circumstances.
Since then I have been working to identify why v6 failover takes so
long and to improve it, but we are not there yet.
The router that died today is the same router which died last
Thursday. It had been up for over 4 years and so its failure last
week was deemed an aberration and it was power cycled.
I have now flipped routes around so the other router is first choice
for v6, but will continue to work on v6 failover.
Apologies for the disruption!
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
TL;DR version
=============
We're now in the position to be able to provide you with double your
current amount of memory for the same price.
Please contact support to ask for your upgrade and it will happen,
though your VPS may need to be moved between servers. If you're not
bothered then just do nothing and it will still happen eventually.
Long version
============
Unfortunately, while there is enough total capacity to do this, some
individual servers do not have enough spare memory to do it
immediately. Some customers will need to have their VPSes moved in
order to get the upgrade.
Experience has shown that this can be a lengthy process, so although
I am strongly opposed to offering new customers something that
existing customers can't have, I also do not want to spend that
whole time being less competitive.
So, those who want to have their upgrade just need to ask and you
can have it immediately, but we may need to move your VPS between
hosts to do it.
If you don't care then you can just do nothing and eventually when
enough moves have been done, the upgrade can happen all at once for
everyone who didn't get it yet.
Here's some more information:
https://tools.bitfolk.com/wiki/Memory_upgrade,_May_2017
If you have any questions that aren't answered there then please
ask, and the answer will be added there.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
Two bugs¹ have been discovered in Xen which have serious security
implications, therefore we must patch and reboot every host before
the security advisory comes out of embargo at 12:00Z on 2 May.
We'll be sending out notifications shortly to let you know the exact
window in which the reboot will take place for your VPS(es), but
this is just a short note to let you know that the work will most
likely be taking place in the early hours (UK) of 30 April, 1 May
and 2 May.
A reminder that if you wish you can have your VPS suspended to and
then restored from storage, instead of shut down and booted:
https://panel.bitfolk.com/account/config/#prefshttps://tools.bitfolk.com/wiki/Suspend_and_restore
Cheers,
Andy
¹ XSA-213 and XSA-215 - http://xenbits.xen.org/xsa/
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
Approximately 8 hours ago we were made aware that Cross-Site Request
Forgery (CSRF) could be used to trick a logged-in user of the
BitFolk Panel at https://panel.bitfolk.com/ into carrying out
changes that could allow their account to be compromised.
As there was no checking that requests actually came from forms
generated on the panel site, if a logged-in user was tricked into
submitting an HTTP request from elsewhere then they could
change sensitive details about their account such as:
- Adding SSH keys for console access
- Altering contact email address
- Invalidating/Disabling two factor authentication
- Enabling email password reset, if it was disabled
We have no evidence that any of these actions have ever been carried
out maliciously, but aside from reports we would have no way of
knowing, so we would advise that all customers log in to their Panel
account and check that the list of SSH keys is as they expect.
All of the forms on the sensitive pages, which include everything
under:
* https://panel.bitfolk.com/account/security/
* https://panel.bitfolk.com/account/contacts/
were today secured against CSRF so there is now no way to use this
technique to compromise an account. The vulnerability would have
been there ever since the Panel site existed, or the relevant
features were added.
The remaining forms, which only cover fairly trivial informational
items, will be fixed as soon as possible. You can track that work at
our tracker:
* https://tools.bitfolk.com/redmine/issues/156
Thanks must go to Dominic Cleal <https://m0dlx.com/> who responsibly
disclosed the problem to us today and has assisted with testing of
our fixes.
More general information about CSRF:
* https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)
If you have any further questions please do let us know by replying
to the users list (users(a)lists.bitfolk.com) or to
support(a)bitfolk.com if you need to discuss anything specific to your
account.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
A bug¹ has been discovered in Xen which has serious security
implications, therefore we must patch and reboot every host before
the security advisor comes out of embargo on 4 April.
We'll be sending out notifications shortly to let you know the exact
window in which the reboot will take place for your VPS(es), but
this is just a short note to let you know that the work will most
likely be taking place in the early hours (UK) of 30 March, 31 March
and 1 April.
A reminder that if you wish you can have your VPS suspended to and
then restored from storage, instead of shut down and booted:
https://panel.bitfolk.com/account/config/#prefshttps://tools.bitfolk.com/wiki/Suspend_and_restore
Cheers,
Andy
¹ XSA-212 - http://xenbits.xen.org/xsa/
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
Ubuntu 12.04.x LTS (Precise Pangolin) goes end of life in about 6
weeks, but today I attempted to boot its installer and found its
netboot kernel¹ didn't boot.
With no useful diagnostics it's probably not something I will be
able to get fixed in the remaining time so I have removed it from
our offering a bit early.
If this severely affects anyone then let me know and I'll see if I
can spend a bit more effort on it…
The ones for 14.04.x LTS and 16.04.x LTS still work fine.
Cheers,
Andy
¹ As downloaded from
http://gb.archive.ubuntu.com/ubuntu/dists/precise/main/installer-amd64/curr…
--
https://bitfolk.com/ -- No-nonsense VPS hosting