Dear customers,
== TL;DR
- Our colo provider is moving out of Telehouse to another datacentre close by,
and we are going to be moving with them.
- It will mean one short (2h or less) outage for each of your VMs as our
servers are physically moved on multiple dates (to be decided) in
December and January.
- Nothing about your service will change. IPs, specs, prices will all remain
the same. Our colo provider will retain the same transit providers and will
still peer at LINX.
== Background
Since we started in 2006 BitFolk has been hosted with the same colo provider in
the same datacentre: Telehouse London. Telehouse have decided to not renew our
colo provider's contract with them and so they will be moving almost all of
their infrastructure to another datacentre; the nearby IP House.
https://www.ip-house.co.uk/
Given that we have much less than a single rack of infrastructure ourselves,
our options here are to either move with our current colo provider or find
another colo provider. Staying exactly where we are is not an available option.
In light of the good relationship we have had with our colo provider since
2006, we have decided to move with them. This must take place before the middle
of January 2024.
== Planning
At this early stage only broad decisions have been made. The main reason I'm
writing at this time is to give you as much notice as possible of the
relatively short outage you will experience in December or January. As soon as
more detailed plans are made I will communicate them to you.
As soon as the infrastructure is available at IP House we plan to move some of
our servers there - ones with no customer services on them. We will schedule
the physical movements of our servers that have customers VMs on them across
multiple dates in December and January. As IP House is only about 500 metres
from Telehouse, we don't expect an outage of more than about 2 hours for each
server that is moved.
If you can not tolerate an outage like that, please open a support ticket by
emailing support(a)bitfolk.com as soon as possible. We will do our best to
schedule a live migration of your service from hardware in Telehouse to
hardware in IP House, resulting in only a few seconds of unreachability. You
can contact us about that now, or you can wait until the exact date of your
move is communicated to you, but there are a limited number of customers we can
do this for. So please only ask for this if it's really important to you. If it
is, please ask for it as soon as you can to avoid disappointment.
== Answers To Anticipated Questions
=== Will anything about my service change?
No. It will be on the same hardware, with the same IP addresses and
same specification for the same price.
We're aware that we're well overdue for a hardware refresh and we hope to be
tackling that as soon as the move is done. That will result in a higher
specification for the same price.
=== Will the network connectivity change?
No. Our colo provider will retain the same mix of transit providers that it
currently does, and it will still peer at LINX.
=== When exactly will outages affect me?
We have not yet planned exactly when we will move our servers. As soon as we do
we'll contact all affected customers. We need to co-ordinate work with our colo
provider who are also busy planning the movement of all of their infrastructure
and other customers, and installing new infrastructure at IP House.
We expect there to be several dates with one or more servers moving on each
date. All customers on those servers wil experience a short outage, but in
total we would expect only the one outage per VM.
=== Will I be able to change the date/time of my outage?
No. As the whole server that your VM is on will be moving at the specified
time, your options will be to either go with that or seek to have your service
live migrated ahead of that date. Please contact support if you want one or
more of your VMs to be live migrated.
If you have multiple VMs on different servers it is possible that they will be
affected at the same time, i.e. if the two servers that your VMs are on are
both to be relocated in the same maintenance window. If that is undesirable,
again one or all of your VMs will need to be live migrated to servers already
present in IP House.
=== What is live migration?
It's where we snapshot your running VM and its storage, ship the data across to
a new host and resume execution again. It typically results in only a few
seconds of unreachability, and probably TCP connections still stay alive.
More information: https://tools.bitfolk.com/wiki/Suspend_and_restore
== Thanks
Thanks for your custom. Despite the upheaval I am looking forward to
a new chapter in a new datacentre!
Andy Smith
Director
BitFolk Ltd
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
After a recent update of the cloud-init package on Ubuntu 22.04
something appears to be going wrong and cloud-init is being run at
every boot - WHICH WILL LEAD TO YOUR NETWORKING CONFIG BEING
DELETED.
In addition, the password of the "ubuntu" user is locked and the SSH
host keys are regenerated.
I do not yet know whether this is the result of some misuse of
cloud-init on our part, or some bug in cloud-init. The outcome is so
bad that I have to warn you about this as soon as possible, before
I've fully understood the issue.
It is safe — and at this stage recommended — to simply remove the
cloud-init package which serves no purpose at BitFolk after first
boot.
$ sudo apt remove cloud-init
If it is too late for you and you already did reboot and are now
wondering why your VM has no network and is trying to DHCP for one,
here's how to fix things.
1. Connect to your Xen Shell with
ssh accountname(a)accountname.console.bitfolk.com
More info: https://tools.bitfolk.com/wiki/Xen_Shell
1. Work out how you're going to get root access from a console log
in prompt. If you have a user other than the initial "ubuntu" one,
you'll use that. If you don't, you'll need to reset the password for
"ubuntu" as it has now been locked.
If you have a login already, use "console" command and log in to
your VM at its console.
If you need to reset the "ubuntu" password:
a) Make sure VM is shut down
xen shell> shutdown
b) Follow these instructions substituting "ubuntu" for "root":
https://tools.bitfolk.com/wiki/Resetting_root_password
then "boot" your VM again and log in as "ubuntu".
2. At this point you're logged in as "ubuntu" on a VM with no
network.
Put /etc/netplan/50-cloud-init.yaml back to how it was. Here is
an example file:
https://tools.bitfolk.com/wiki/Ubuntu#Migrate_to_netplan
The "gateway" statements are deprecated but will still work.
Make sure to "chmod go= /etc/netplan/50-cloud-init.yaml" so it
has correct permissions
$ sudo netplan generate
$ sudo netplan apply
Your networking should now work again
3. sudo apt remove cloud-init
It will now be safe to reboot in future.
As I say I am still looking into where the problem lies here and the
best way to fix it.
The example netplan config linked above has some deprecated
statements in it which I will also fix (if it doesn't have
"gateway4" etc in it any more then I did already by the time you
read this), but it does (still) work.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
