BitFolk Announcements

announce@mailman.bitfolk.com

1 participants
265 discussions

A bit behind on billing due to payment provider breakage

by Andy Smith

Hi, It appears that our Direct Debit payment provider turned off another bit of their legacy API¹ on 20 December and as a result we haven't been issuing (any) invoices or requesting Direct Debit payments since then as I needed to fix up some things. I've only just had chance to look into it, but now it's all flowing again so if you have today received invoices which talk about service running from a date that is before today, that is why. Apologies for any confusion caused. Cheers, Andy ¹ They in theory turned it off at the end of October 2017, but did in fact leave it running so that customers set up using it could still be billed. Apparently some of that is no longer the case. At the moment it seems that this will just mean that I have to ask a few people to authorise new mandates. -- https://bitfolk.com/ -- No-nonsense VPS hosting

5 years, 10 months

Please log in to the Panel web site to upgrade your password hash

by Andy Smith

Hi, == Short version Please log in to https://panel.bitfolk.com/ to facilitate an upgrade of the secure hash function used to protect your BitFolk password. == Longer version I've recently performed a major upgrade on our LDAP servers (skipping multiple OS releases), and have taken the opportunity to review what algorithms are used for protecting your BitFolk account passwords. We have been using the same LDAP directory since the beginning of BitFolk in 2007 and the options for hash functions were a bit more limited back then. Although I have changed the default algorithm several times as servers were upgraded, once a given algorithm is used for a user it is not changed until the user changes their password. So, worst case, if you have been a customer for 10 years and have never changed your account password, it is currently hashed with an algorithm which would today be considered obsolete for that purpose. Sadly we cannot just upgrade everyone's hash scheme because we don't know your passwords! I have added functionality to the Panel web site to check which hash algorithm is in use when you log in and if it isn't the new default it sets your password to the same as the one you just supplied at login. This upgrades you to the new default hash algorithm. Therefore it would be good if you could take the time to log in to https://panel.bitfolk.com/ If you do a password reset then it will also upgrade when you follow the link to log in with the randomly-generated password. Also if you change your password it will upgrade. There is no visual feedback of this change happening, so you'll just have to trust that it has. It is logged though, so I can tell you if you want to know. I'm afraid you will need to do this with each account you have, if you have multiple VPSes. === What are cryptographic hash functions? They're algorithms which map arbitrary data (your password) into an output string of a fixed size, in a way which is not feasible to reverse. We store the output in our password database, and if our database (the LDAP directory) is ever leaked or stolen then we will have some small comfort that the attacker would not be able to immediately read your passwords in clear text. === Why do they need to change? Computers get faster and flaws are discovered in algorithms. What was once considered too difficult to brute-force can now be done in seconds with a single computer. The ability to spin up a set of servers with 1,000 GPUs is within the reach of many people. === Are you telling us to do this because there has been a compromise? No. It's just that there was a major upgrade in the last week which everyone would benefit from; particularly the longest-standing customers who will be stuck on algorithms older than the previous default. The other option would have been to invalidate everyone's passwords, forcing a mass password change. That seemed likely to foster speculation of compromise, and be a real annoyance besides. === Which algorithms are in use? What's the new default? I'm afraid I would rather not comment on the specific algorithms involved, but it's already public knowledge that we use OpenLDAP. Here is documentation showing the options: http://www.openldap.org/doc/admin24/guide.html#Password%20Storage …and this article gives some more modern (2016) recommendations: https://www.redpill-linpro.com/techblog/2016/08/16/ldap-password-hash.html === I'm confused; is something going to break if I don't take action? No. You can do nothing and your password will stay working and protected with the same algorithm that was the default the last time you changed your password (or when your account was created). Next time you log in to the Panel (or do a password reset, or change your password) it will be upgraded. I just thought that I'd let people know they can influence things sooner if they want to. Some customers have never logged in to the Panel site, so their password hashes might never be upgraded otherwise. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

5 years, 10 months

Networking on 18.04 and beyond is configured by netplan

by Andy Smith

Hi, Briefly: The Ubuntu 18.04 installer now configures networking properly using netplan¹, and later versions of Ubuntu will continue to do so as long as the default in Ubuntu remains netplan. More detail: There was a recent thread² on the "users" list where a customer had upgraded to Ubuntu 18.04 and was having problems configuring their networking using ifupdown (as configured by /etc/network/interfaces) like they always had. During the course of that discussion it was noted³ that as of Ubuntu 18.04 (Bionic Beaver), Ubuntu now uses netplan to configure networking. BitFolk has been generating an /etc/network/interfaces file using a post-install script in the Ubuntu installer, but this file is no longer consulted for configuring networking in Ubuntu. The installer has now been made to generate a correct netplan config in /etc/netplan/01-netcfg.yaml. We recommend that customers upgrading to Ubuntu 18.04 and beyond switch to netplan for a simpler life. netplan can't currently do everything that ifupdown can, so if you have a complicated networking setup you should look into this carefully. A lot can be done in systemd-networkd hooks instead of ifupdown hooks. The "IPv6" article on the wiki has been updated for the relevant netplan configurations: https://tools.bitfolk.com/wiki/IPv6 Cheers, Andy ¹ https://netplan.io/ ² https://lists.bitfolk.com/lurker/message/20181109.144005.13377548.en.html ³ https://lists.bitfolk.com/lurker/message/20181113.170055.93f45da1.en.html -- https://bitfolk.com/ -- No-nonsense VPS hosting

5 years, 10 months

New monitoring system coming online soon

by Andy Smith

Hi, I'm in the final stages of putting a new monitoring system in place and retiring Nagios. I've already added ping checks for everyone that had them before, and am now going through and porting the more complicated checks that people have. I won't cut it over or turn alert notifications on until everything is being checked by both systems, so for a while you are going to see service checks from: - 85.119.80.238 - 85.119.80.244 - 2001:ba8:1f1:f040::/64 - 2001:ba8:1f1:f25d::/64 If you have firewalled your services to only allow checks from 85.119.80.244 then now would be a good time to allow the other IPs too. At the moment if you don't have any monitoring set up you have to ask for it in a support ticket. If you'd like that to be set up for you, please send a mail to support(a)bitfolk.com. Hopefully shortly after the new system is in place I will be able to add some sort of checkbox to the web panel to enable it, as I already have it so the simple ping tests and alerting contacts are built from the customer database. It will probably be some time before more complex checks are self-service like that, but don't be afraid to ask for something in the mean time. Many things can be monitored without an agent, more still by NRPE or SNMP. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

5 years, 11 months

Host "hen" unexpectedly rebooted 2018-11-26 22:24

by Andy Smith

Hi, At approximately 22:24Z, host "hen" rebooted itself unexpectedly. It all appears to be back up again now, and I am investigating to try to determine the cause. Apologies for the disruption. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

5 years, 11 months

Security reboot needed, likely to be weekend of 17/18/19 November

by Andy Smith

Hello, Unfortunately some security flaws have been discovered in the hypervisor software we use (Xen) so we will need to patch and reboot everything before the public release of the details on 20 November. I expect we will do this work in the early hours of the morning (UK time) between Saturday 17 and Monday 19 November, but customers will soon receive an individual email confirming the hour-long maintenance window specific to their server(s). The actual work for each server is expected to take 10–30 minutes within that window. As before we should be able to do a suspend/restore of your guest if you have enabled that at: <https://panel.bitfolk.com/account/config/> Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting > The optimum programming team size is 1. Has Jurassic Park taught us nothing? — pfilandr

6 years

Latest kernel update seems broken on 32-bit Ubuntu 18.04 (Bionic Beaver)

by Andy Smith

Hi, If you aren't running 32-bit Ubuntu then you can probably skip this message. A customer reported that they'd updated their 32-bit Ubuntu 18.04 VPS and now the kernel doesn't boot. I had a look and indeed the latest 32-bit linux-image-generic immediately kernel panics. This means that if you're on 32-bit and you try to boot into it, your VPS won't boot. Also it means that currently 32-bit Ubuntu 18.04 installs aren't possible because the installer kernel won't boot. I haven't yet had any reports of the same thing happening in other versions of Ubuntu, but it might do since presumably there is some security patch being pushed out (I'm going to take a wild stab at 32-bit KPTI protection) that is broken, so it might hit any supported version of 32-bit Ubuntu. The only quick way I have found to work around this at present is to run a 64-bit kernel. Here's how you'd do that, assuming that your VPS is currently unbootable. xen-shell> rescue (boot and log in to rescue VM) $ sudo mount /dev/xvda1 /mnt $ sudo mount --bind /dev /mnt/dev $ sudo mount --bind /sys /mnt/sys $ sudo mount --bind /proc /mnt/proc $ sudo chroot /mnt /bin/bash # dpkg --add-architecture amd64 # apt update # apt install linux-image-generic:amd64 # exit $ sudo halt xen-shell> arch x86_64 xen-shell> boot This: 1) Enables multi-arch on your VPS and says that amd64 architecture packages are acceptable 2) Installs a 64-bit kernel 3) Boots into it You will then be running a 64-bit kernel with a 32-bit user land. It should work fine and continue offering you updated 64-bit kernel packages and updates 32-bit packages for everything else. If you were brave you could completely cross-grade to amd64 but it is a complicated, risky and unsupported procedure. Another workaround may be to boot into rescue and do a chroot as above, but then downgrade the kernel package. I will attempt to replicate the problem and report it to Ubuntu. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

6 years, 1 month

Security reboot needed, likely to be weekend of 11/12/13 August

by Andy Smith

Hello, I'm afraid there's another round of security updates needed as a flaw has been discovered in some (Xen-specific) parts of the Linux kernel. It's not the hypervisor this time but it still needs a kernel upgrade on our side so the effect is basically the same, so there will be another reboot of all servers, likely in the early hours (UK time) of 11–13 August. Customers will soon receive an individual email confirming the hour-long maintenance window specific to their server(s). The actual work is expected to take 10–30 minutes within that window. As before we should be able to do a suspend/restore if you have enabled that at: <https://panel.bitfolk.com/account/config/> Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

6 years, 3 months

Latest Debian stretch (stable) kernel update (4.9.110-1) seems to be broken under Xen

by Andy Smith

Hi, It looks like the recent Debian stretch updated kernel doesn't boot under Xen: <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=903767> This also means that right now you cannot self-install Debian stable. If you reboot and experience this, you will need to boot back into the previous kernel. I am investigating now if there is any quick fix or workaround. In the mean time those affected should follow that bug to know when it is resolved. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

6 years, 3 months

Security reboot needed, likely to be weekend of 23/24/25 June

by Andy Smith

Hello, I'm afraid there's another round of security updates needed, and so there will be another reboot of all servers, likely in the early hours (UK time) of 23–25 June. Customers will soon receive an individual email confirming the hour-long maintenance window specific to their server(s). The actual work is expected to take 10–30 minutes within that window. As before we should be able to do a suspend/restore if you have enabled that at: <https://panel.bitfolk.com/account/config/> Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

6 years, 4 months

← Newer
1
...
7
8
9
10
11
12
13
...
27
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BitFolk Announcements