BitFolk Announcements

announce@mailman.bitfolk.com

1 participants
259 discussions

The impending release of Ubuntu 24.04 (Noble Numbat)

by Andy Smith

Hi, As you may be aware, the next LTS release of Ubuntu is supposed to be ready in a couple of days. I've tested a do-release-upgrade from a basic 22.04 cloud image (what you get when you install 22.04 at BitFolk) and it seemed to go fine. As usual with this sort of thing though, all the complexity is in the packages you have installed, so that is no promise that it would be plain sailing for you. We will try to get a Xen Shell installer option added for 24.04 as soon after release as we can, but in the mean time just installing 22.04 and then typing "sudo do-release-upgrade -d" should get you there. I *think* it is the case that you need the "-d" as do-release-upgrade normally doesn't like doing it until the first point release. Thanks, Andy Ubuntu 24.04 LTS debtest1.vps.bitfolk.space hvc0 debtest1 login: ubuntu Password: Welcome to Ubuntu 24.04 LTS (GNU/Linux 6.8.0-31-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/pro System information as of Tue Apr 23 13:31:00 UTC 2024 System load: 0.07 Memory usage: 6% Processes: 131 Usage of /: 14.6% of 19.20GB Swap usage: 0% Users logged in: 0 Expanded Security Maintenance for Applications is not enabled. 0 updates can be applied immediately. Enable ESM Apps to receive additional future security updates. See https://ubuntu.com/esm or run: sudo pro status ubuntu@debtest1:~$ uname -a Linux debtest1.vps.bitfolk.space 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 00:40:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 day, 14 hours

Updated renewable energy statement for IP House

by Andy Smith

Hi, Since BitFolk moved datacentre from Telehouse to IP House, and since some customers had previously actually asked about a renewable energy statement, I've just updated it. TL;DR: it's 59% now and they aim for it to be 100% by February 2025. More detail: https://tools.bitfolk.com/wiki/Renewable_energy_statement Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

2 months, 1 week

Network disruption today between about 2307 and 2314

by Andy Smith

Hi, Between approximately 23:07Z and 23:14Z today, due to an error in some work our colo provider was undertaking, two servers suffered a total network outage and our remaining servers a partial outage. Apologies for the disruption. It is not expected to re-occur. The two worst affected servers were "clockwork" and "macallan". Our servers have a pair of public network interfaces and are connected to two separate switches. The error took out one of the switches but left our ports enabled there, so it was a blackhole for such traffic. At the moment we use network bonding in active-backup mode. This didn't fail over because the link state didn't go down, so the two servers that had the misconfigured switch as their "active" interface experienced the longer outage. We are in the middle of transitioning away from a bonded setup and having separate interfaces do BGP with our colo provider and use BGP for such redundancy. We are already doing the BGP part but have yet to disable the bonding and split the interfaces back out. When that is complete — which I would hope to have done in a timescale of weeks, not months — this failure mode won't exist. Apologies again, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

2 months, 2 weeks

Datacentre move - advance notice of short expected outage in December or January

by Andy Smith

Dear customers, == TL;DR - Our colo provider is moving out of Telehouse to another datacentre close by, and we are going to be moving with them. - It will mean one short (2h or less) outage for each of your VMs as our servers are physically moved on multiple dates (to be decided) in December and January. - Nothing about your service will change. IPs, specs, prices will all remain the same. Our colo provider will retain the same transit providers and will still peer at LINX. == Background Since we started in 2006 BitFolk has been hosted with the same colo provider in the same datacentre: Telehouse London. Telehouse have decided to not renew our colo provider's contract with them and so they will be moving almost all of their infrastructure to another datacentre; the nearby IP House. https://www.ip-house.co.uk/ Given that we have much less than a single rack of infrastructure ourselves, our options here are to either move with our current colo provider or find another colo provider. Staying exactly where we are is not an available option. In light of the good relationship we have had with our colo provider since 2006, we have decided to move with them. This must take place before the middle of January 2024. == Planning At this early stage only broad decisions have been made. The main reason I'm writing at this time is to give you as much notice as possible of the relatively short outage you will experience in December or January. As soon as more detailed plans are made I will communicate them to you. As soon as the infrastructure is available at IP House we plan to move some of our servers there - ones with no customer services on them. We will schedule the physical movements of our servers that have customers VMs on them across multiple dates in December and January. As IP House is only about 500 metres from Telehouse, we don't expect an outage of more than about 2 hours for each server that is moved. If you can not tolerate an outage like that, please open a support ticket by emailing support(a)bitfolk.com as soon as possible. We will do our best to schedule a live migration of your service from hardware in Telehouse to hardware in IP House, resulting in only a few seconds of unreachability. You can contact us about that now, or you can wait until the exact date of your move is communicated to you, but there are a limited number of customers we can do this for. So please only ask for this if it's really important to you. If it is, please ask for it as soon as you can to avoid disappointment. == Answers To Anticipated Questions === Will anything about my service change? No. It will be on the same hardware, with the same IP addresses and same specification for the same price. We're aware that we're well overdue for a hardware refresh and we hope to be tackling that as soon as the move is done. That will result in a higher specification for the same price. === Will the network connectivity change? No. Our colo provider will retain the same mix of transit providers that it currently does, and it will still peer at LINX. === When exactly will outages affect me? We have not yet planned exactly when we will move our servers. As soon as we do we'll contact all affected customers. We need to co-ordinate work with our colo provider who are also busy planning the movement of all of their infrastructure and other customers, and installing new infrastructure at IP House. We expect there to be several dates with one or more servers moving on each date. All customers on those servers wil experience a short outage, but in total we would expect only the one outage per VM. === Will I be able to change the date/time of my outage? No. As the whole server that your VM is on will be moving at the specified time, your options will be to either go with that or seek to have your service live migrated ahead of that date. Please contact support if you want one or more of your VMs to be live migrated. If you have multiple VMs on different servers it is possible that they will be affected at the same time, i.e. if the two servers that your VMs are on are both to be relocated in the same maintenance window. If that is undesirable, again one or all of your VMs will need to be live migrated to servers already present in IP House. === What is live migration? It's where we snapshot your running VM and its storage, ship the data across to a new host and resume execution again. It typically results in only a few seconds of unreachability, and probably TCP connections still stay alive. More information: https://tools.bitfolk.com/wiki/Suspend_and_restore == Thanks Thanks for your custom. Despite the upheaval I am looking forward to a new chapter in a new datacentre! Andy Smith Director BitFolk Ltd -- https://bitfolk.com/ -- No-nonsense VPS hosting

3 months, 2 weeks

Do not upgrade Debian 12 (bookworm) to the current point release yet

by Andy Smith

Hi, A bug sneaked into the upstream Linux kernel and was included in the latest Debian stable kernel release. As the point release to Debian 12.3 happened yesterday, if you upgrade to that kernel and boot into it you will be exposed to a data corruption bug in ext4. So do not install linux-image-6.1.0-14-amd64 version 6.1.64-1. Wait for 6.1.66-1 which contains the fix. https://micronews.debian.org/2023/1702150551.html Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

4 months, 2 weeks

Investigating issues with host "macallan"

by Andy Smith

Hi, I'm currently investigating problems with host "macallan" which started having issues around 12:57 as far as I can see. I will keep you updated when I know more, Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

5 months, 3 weeks

URGENT: Ubuntu 22.04 customers risk config reset at reboot

by Andy Smith

Hi, After a recent update of the cloud-init package on Ubuntu 22.04 something appears to be going wrong and cloud-init is being run at every boot - WHICH WILL LEAD TO YOUR NETWORKING CONFIG BEING DELETED. In addition, the password of the "ubuntu" user is locked and the SSH host keys are regenerated. I do not yet know whether this is the result of some misuse of cloud-init on our part, or some bug in cloud-init. The outcome is so bad that I have to warn you about this as soon as possible, before I've fully understood the issue. It is safe — and at this stage recommended — to simply remove the cloud-init package which serves no purpose at BitFolk after first boot. $ sudo apt remove cloud-init If it is too late for you and you already did reboot and are now wondering why your VM has no network and is trying to DHCP for one, here's how to fix things. 1. Connect to your Xen Shell with ssh accountname(a)accountname.console.bitfolk.com More info: https://tools.bitfolk.com/wiki/Xen_Shell 1. Work out how you're going to get root access from a console log in prompt. If you have a user other than the initial "ubuntu" one, you'll use that. If you don't, you'll need to reset the password for "ubuntu" as it has now been locked. If you have a login already, use "console" command and log in to your VM at its console. If you need to reset the "ubuntu" password: a) Make sure VM is shut down xen shell> shutdown b) Follow these instructions substituting "ubuntu" for "root": https://tools.bitfolk.com/wiki/Resetting_root_password then "boot" your VM again and log in as "ubuntu". 2. At this point you're logged in as "ubuntu" on a VM with no network. Put /etc/netplan/50-cloud-init.yaml back to how it was. Here is an example file: https://tools.bitfolk.com/wiki/Ubuntu#Migrate_to_netplan The "gateway" statements are deprecated but will still work. Make sure to "chmod go= /etc/netplan/50-cloud-init.yaml" so it has correct permissions $ sudo netplan generate $ sudo netplan apply Your networking should now work again 3. sudo apt remove cloud-init It will now be safe to reboot in future. As I say I am still looking into where the problem lies here and the best way to fix it. The example netplan config linked above has some deprecated statements in it which I will also fix (if it doesn't have "gateway4" etc in it any more then I did already by the time you read this), but it does (still) work. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

6 months, 2 weeks

CentOS Stream 9

by Andy Smith

Hi, CentOS Stream 9 is now available for self-install and new installs. https://tools.bitfolk.com/wiki/Using_the_self-serve_net_installer/CentOS_St… We haven't yet sorted out installers for Alma Linux, Rocky Linux or any of the other CentOS-like distributions and although that would be pretty simple I'm not sure that we will do that. It depends upon demand. I believe it's the case that as with CentOS Stream 8.x, you can convert from it to Alma, Rocky or even RHEL without reinstall using a script, so that might have to be the BitFolk-recommended way to do that. We are also going to consult about how much demand there is for RHEL itself. Although that does require a Red Hat subscription, an individual can get a no-cost subscription for personal use on up to 16 systems. We do have to run these VMs under the kernel-lt or kernel-ml kernels from ELRepo though, because Red Hat disables Xen support in its kernels. Therefore such a VM may not be eligible for any form of support from Red Hat which may result in there being no customer demand to do so. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

8 months, 1 week

SSHd was upgraded on xen shell servers, may cause older clients to fail to connect

by Andy Smith

Hi, Back at the start of June the version of OpenSSH that we run on the Xen Shell hosts was updated in order to provide support for ecdsa-sk and ed25519-sk keys. These are used with "security key" devices which support FIDO/U2F and was done after customer request. At the same time this version of SSH disables the ssh-rsa signature scheme. Older ssh clients may fail to negotiate an SSH connection to the Xen Shell hosts (i.e. when you do "ssh username(a)username.console.bitfolk.com") due to this. If you see an error that reads something like this: Couldn't agree a key exchange algorithm (available curve25519-sha256,curve25519-sha256(a)libssh.org,ecdh-sha2-nistp256, ecdh-sha2-nistp384,ecdh-sha2-nistp512,diffie-hellman-group16-sha512, diffiehellman-group18-sha512,diffie-hellman-group14-sha256) then this is the problem. You do not need to change your authentication keys (if any) because that is not the problem. You need to upgrade your SSH client. The above message was from PuTTY 0.64; version 0.78 and above are known to work. ssh-rsa was deprecated since version 8.2 of the server in February 2020: https://www.openssh.com/txt/release-8.2 Future deprecation notice ========================= It is now possible[1] to perform chosen-prefix attacks against the SHA-1 hash algorithm for less than USD$50K. For this reason, we will be disabling the "ssh-rsa" public key signature algorithm that depends on SHA-1 by default in a near-future release. [1] "SHA-1 is a Shambles: First Chosen-Prefix Collision on SHA-1 and Application to the PGP Web of Trust" Leurent, G and Peyrin, T (2020) https://eprint.iacr.org/2020/014.pdf It was then disabled from version 8.8 in September 2021: https://www.openssh.com/txt/release-8.8 Potentially-incompatible changes ================================ This release disables RSA signatures using the SHA-1 hash algorithm by default. This change has been made as the SHA-1 hash algorithm is cryptographically broken, and it is possible to create chosen-prefix hash collisions for <USD$50K. For most users, this change should be invisible and there is no need to replace ssh-rsa keys. OpenSSH has supported RFC8332 RSA/SHA-256/512 signatures since release 7.2 and existing ssh-rsa keys will automatically use the stronger algorithm where possible. If you are unsure whether this affects you, just verify that you can connect to your Xen Shell host. If you can't, and you can't find a way to upgrade your client (or doing so is ineffective), please let us know at support(a)bitfolk.com. Again, this not about rsa authentication keys. You do not need to abandon ssh-rsa public keys: https://ikarus.sg/rsa-is-not-dead/ Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

9 months

Debian 12 "bookworm" is now available for self-install

by Andy Smith

Hi, Debian 12 "bookworm" which was released on 10 June 2023 is now available for self-install from our Xen Shell: https://tools.bitfolk.com/wiki/Using_the_self-serve_net_installer The command: xen shell> install debian_testing also now leaves you with an install of testing, but aside from the code names in /etc/apt.sources.list that is currently pretty much exactly the same as bookworm. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

10 months, 2 weeks

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BitFolk Announcements