BitFolk Announcements

announce@mailman.bitfolk.com

265 discussions

Do not upgrade Debian 12 (bookworm) to the current point release yet

by Andy Smith

Hi, A bug sneaked into the upstream Linux kernel and was included in the latest Debian stable kernel release. As the point release to Debian 12.3 happened yesterday, if you upgrade to that kernel and boot into it you will be exposed to a data corruption bug in ext4. So do not install linux-image-6.1.0-14-amd64 version 6.1.64-1. Wait for 6.1.66-1 which contains the fix. https://micronews.debian.org/2023/1702150551.html Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year

Investigating issues with host "macallan"

by Andy Smith

Hi, I'm currently investigating problems with host "macallan" which started having issues around 12:57 as far as I can see. I will keep you updated when I know more, Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 1 month

URGENT: Ubuntu 22.04 customers risk config reset at reboot

by Andy Smith

Hi, After a recent update of the cloud-init package on Ubuntu 22.04 something appears to be going wrong and cloud-init is being run at every boot - WHICH WILL LEAD TO YOUR NETWORKING CONFIG BEING DELETED. In addition, the password of the "ubuntu" user is locked and the SSH host keys are regenerated. I do not yet know whether this is the result of some misuse of cloud-init on our part, or some bug in cloud-init. The outcome is so bad that I have to warn you about this as soon as possible, before I've fully understood the issue. It is safe — and at this stage recommended — to simply remove the cloud-init package which serves no purpose at BitFolk after first boot. $ sudo apt remove cloud-init If it is too late for you and you already did reboot and are now wondering why your VM has no network and is trying to DHCP for one, here's how to fix things. 1. Connect to your Xen Shell with ssh accountname(a)accountname.console.bitfolk.com More info: https://tools.bitfolk.com/wiki/Xen_Shell 1. Work out how you're going to get root access from a console log in prompt. If you have a user other than the initial "ubuntu" one, you'll use that. If you don't, you'll need to reset the password for "ubuntu" as it has now been locked. If you have a login already, use "console" command and log in to your VM at its console. If you need to reset the "ubuntu" password: a) Make sure VM is shut down xen shell> shutdown b) Follow these instructions substituting "ubuntu" for "root": https://tools.bitfolk.com/wiki/Resetting_root_password then "boot" your VM again and log in as "ubuntu". 2. At this point you're logged in as "ubuntu" on a VM with no network. Put /etc/netplan/50-cloud-init.yaml back to how it was. Here is an example file: https://tools.bitfolk.com/wiki/Ubuntu#Migrate_to_netplan The "gateway" statements are deprecated but will still work. Make sure to "chmod go= /etc/netplan/50-cloud-init.yaml" so it has correct permissions $ sudo netplan generate $ sudo netplan apply Your networking should now work again 3. sudo apt remove cloud-init It will now be safe to reboot in future. As I say I am still looking into where the problem lies here and the best way to fix it. The example netplan config linked above has some deprecated statements in it which I will also fix (if it doesn't have "gateway4" etc in it any more then I did already by the time you read this), but it does (still) work. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 2 months

CentOS Stream 9

by Andy Smith

Hi, CentOS Stream 9 is now available for self-install and new installs. https://tools.bitfolk.com/wiki/Using_the_self-serve_net_installer/CentOS_St… We haven't yet sorted out installers for Alma Linux, Rocky Linux or any of the other CentOS-like distributions and although that would be pretty simple I'm not sure that we will do that. It depends upon demand. I believe it's the case that as with CentOS Stream 8.x, you can convert from it to Alma, Rocky or even RHEL without reinstall using a script, so that might have to be the BitFolk-recommended way to do that. We are also going to consult about how much demand there is for RHEL itself. Although that does require a Red Hat subscription, an individual can get a no-cost subscription for personal use on up to 16 systems. We do have to run these VMs under the kernel-lt or kernel-ml kernels from ELRepo though, because Red Hat disables Xen support in its kernels. Therefore such a VM may not be eligible for any form of support from Red Hat which may result in there being no customer demand to do so. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 4 months

SSHd was upgraded on xen shell servers, may cause older clients to fail to connect

by Andy Smith

Hi, Back at the start of June the version of OpenSSH that we run on the Xen Shell hosts was updated in order to provide support for ecdsa-sk and ed25519-sk keys. These are used with "security key" devices which support FIDO/U2F and was done after customer request. At the same time this version of SSH disables the ssh-rsa signature scheme. Older ssh clients may fail to negotiate an SSH connection to the Xen Shell hosts (i.e. when you do "ssh username(a)username.console.bitfolk.com") due to this. If you see an error that reads something like this: Couldn't agree a key exchange algorithm (available curve25519-sha256,curve25519-sha256(a)libssh.org,ecdh-sha2-nistp256, ecdh-sha2-nistp384,ecdh-sha2-nistp512,diffie-hellman-group16-sha512, diffiehellman-group18-sha512,diffie-hellman-group14-sha256) then this is the problem. You do not need to change your authentication keys (if any) because that is not the problem. You need to upgrade your SSH client. The above message was from PuTTY 0.64; version 0.78 and above are known to work. ssh-rsa was deprecated since version 8.2 of the server in February 2020: https://www.openssh.com/txt/release-8.2 Future deprecation notice ========================= It is now possible[1] to perform chosen-prefix attacks against the SHA-1 hash algorithm for less than USD$50K. For this reason, we will be disabling the "ssh-rsa" public key signature algorithm that depends on SHA-1 by default in a near-future release. [1] "SHA-1 is a Shambles: First Chosen-Prefix Collision on SHA-1 and Application to the PGP Web of Trust" Leurent, G and Peyrin, T (2020) https://eprint.iacr.org/2020/014.pdf It was then disabled from version 8.8 in September 2021: https://www.openssh.com/txt/release-8.8 Potentially-incompatible changes ================================ This release disables RSA signatures using the SHA-1 hash algorithm by default. This change has been made as the SHA-1 hash algorithm is cryptographically broken, and it is possible to create chosen-prefix hash collisions for <USD$50K. For most users, this change should be invisible and there is no need to replace ssh-rsa keys. OpenSSH has supported RFC8332 RSA/SHA-256/512 signatures since release 7.2 and existing ssh-rsa keys will automatically use the stronger algorithm where possible. If you are unsure whether this affects you, just verify that you can connect to your Xen Shell host. If you can't, and you can't find a way to upgrade your client (or doing so is ineffective), please let us know at support(a)bitfolk.com. Again, this not about rsa authentication keys. You do not need to abandon ssh-rsa public keys: https://ikarus.sg/rsa-is-not-dead/ Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 4 months

Debian 12 "bookworm" is now available for self-install

by Andy Smith

Hi, Debian 12 "bookworm" which was released on 10 June 2023 is now available for self-install from our Xen Shell: https://tools.bitfolk.com/wiki/Using_the_self-serve_net_installer The command: xen shell> install debian_testing also now leaves you with an install of testing, but aside from the code names in /etc/apt.sources.list that is currently pretty much exactly the same as bookworm. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 6 months

Debian 12 bookworm - a reminder about eth0 → enX0

by Andy Smith

Hi, As you may be aware, the latest stable release Debian 12 bookworm was released on 10 June. It is available for new orders and you can of course upgrade your existing Debian VMs to this release, but we haven't yet updated the web site to reflect this nor the Xen Shell to allow a clean self-install. That will happen in the next couple of days. A reminder though, that in this new release udev has learned about Xen network interfaces and therefore will rename your eth0 to enX0. This was previously discussed here: https://mailman.bitfolk.com/mailman/hyperkitty/list/users@mailman.bitfolk.c… Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 6 months

The SPF record for bitfolk.com was tightened from ~all to -all on 3 June 2023

by Andy Smith

Hi, After a minor incident of some phishing emails being sent purporting to be from bitfolk.com, on 3 June I tightened the SPF record of bitfolk.com from ~all to -all. This basically says that ONLY the hosts listed in the SPF record are permitted to use a bitfolk.com envelope sender on email, and that any other host trying to do so should be rejected. I have since noticed that some customers are using traditional server-side forwarding, e.g. on role addresses, to send BitFolk emails to a group of people, and some of those recipients are doing as asked and rejecting the email. More are probably silently discarding or filing the mails away in spam/junk folders. This happens because when your mail server forwards an email from e.g. billing(a)bitfolk.com through role(a)yourdomain.co.uk and out to joe.bloggs(a)example.com, your mail server is pretending to be billing(a)bitfolk.com. Since you do not match bitfolk.com's SPF record, the mail server for example.com rejects the email (unless configured otherwise). Unfortunately we can't go back on this configuration. It's just the way that email works in the 21st century. Server-side forwarding of email in this way is not something that can be expected to work any more, unless you control both the recipient address and every address it expands out to. So what can you do if you currently do forwarding of addresses like this? - The generic answer is Snder Rewriting Scheme: https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme - A more limited answer is to run a real mailing list of some sort, so that, for example, billing(a)yourdomain.co.uk goes through a real mailing list manager like majordomo or Mailman, is rewritten and sent out to the people who should receive it. - If you control or have strong influence on all recipients, you can configure them to allowlist particular senders. - You can set up real mailboxes for your role accounts and have interested parties download the email by IMAP or POP. I'm sorry that this change has broken some previously-working forwarding setups. It isn't something we can revert though (and indeed, it will have to get stricter, with additional DKIM and DMARC to come). Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 6 months

Secondary nameservers changed from PowerDNS to BIND - beware your expire timers

by Andy Smith

Hello, If you do not make use of BitFolk's secondary DNS service then you can safely skip this email. Over the last few days we've upgraded the servers used for our secondary DNS service and also switched software from PowerDNS to BIND. On the whole nothing changes, but there is one thing I would like to draw your attention to. BIND actually pays attention to the expire timers that you set in your SOA records whereas PowerDNS does not. An SOA record looks like this: $ dig +multi +noall +answer -t soa bitfolk.com bitfolk.com. 86383 IN SOA a.authns.bitfolk.co.uk. hostmaster.bitfolk.com. ( 2023042101 ; serial 14400 ; refresh (4 hours) 7200 ; retry (2 hours) 1209600 ; expire (2 weeks) 43200 ; minimum (12 hours) ) The "expire" timer tells authoritative DNS servers how long the records they hold are valid for, if they have not been able to contact the primary nameservers. In the above example, should the primary nameserver be unreachable, any secondary nameservers that are still responding will serve the zone content for a further two weeks. After that time they will respond with SERVFAIL. Compliant DNS client behaviour is to retry any other servers when that happens. PowerDNS does not implement these "expire" semantics and always answers queries. In watching logs carefully over the last few days I have seen that some of you have extremely short expire timers. I'm not sure whether you intend for that to be the case. For example, there are many zones currently on BitFolk's servers with an expire time of 300 seconds. That means that you are indicating that the entire zone should not be served 5 minutes¹ after your primary server stops responding. It doesn't seem likely to me that you really want all authoritative servers for your domain to stop working 5 minutes into any sort of outage. Since the secondary servers will now really believe you on this, I urge you to review your expire timers. If in doubt please put your domain name into this: https://zonemaster.net/en/run-test and it'll advise you if any of those timers seem wrong. Cheers, Andy ¹ Different DNS server implementations actually behave slightly differently here. As mentioned, PowerDNS doesn't handle expire timers at all. BIND has a minimum of 300 seconds, or the refresh+retry timers, whichever is larger. So in fact the shortest expire behaviour I can see at the moment is 600 seconds (10 minutes). Which still seems unusually short. See: https://jpmens.net/2022/01/14/fun-with-the-dns-soa-expire-field/ for more info about how different implementations treat the expire timer. -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 8 months

2023-04-19: b.authns maintenance

by Andy Smith

Hi, Today and tonight (UK time) we're going to be doing some work on b.authns.bitfolk.com (secondary DNS service). I'm going to disable alerts for it and stop it from responding (so it can't give any incorrect answers). The other servers will remain up and working; I'm just letting you know in case you notice it is intermittently down. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

1 year, 8 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BitFolk Announcements