Oops, I've just noticed that this only went to users@, not announce@. Sorry about the extra copy some of you are now getting. :(
Cheers,
Andy
----- Forwarded message from Andy Smith <andy(a)bitfolk.com> -----
Date: Fri, 21 Dec 2018 14:49:34 +0000
From: Andy Smith <andy(a)bitfolk.com>
To: users(a)lists.bitfolk.com
Subject: Re: [bitfolk] Host "hen" crashed again (Was: Re: Host "hen" unexpectedly rebooted 2018-11-26 22:24)
User-Agent: Mutt/1.5.23 (2014-03-12)
Reply-To: users(a)lists.bitfolk.com
Hi,
This happened again a few minutes ago.
IPv6 failover appears to have happened correctly due to some fixes
since last time - hence no v6 outage until manual intervention.
hen has now booted with some settings which it is hoped will avoid
the problem in future.
I'm not in front of a computer at the moment so later today I'll
review what was logged.
Apologies for the ongoing hassle.
Cheers,
Andy
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
== Short version
Please log in to https://panel.bitfolk.com/ to facilitate an upgrade
of the secure hash function used to protect your BitFolk password.
== Longer version
I've recently performed a major upgrade on our LDAP servers
(skipping multiple OS releases), and have taken the opportunity to
review what algorithms are used for protecting your BitFolk account
passwords.
We have been using the same LDAP directory since the beginning of
BitFolk in 2007 and the options for hash functions were a bit more
limited back then. Although I have changed the default algorithm
several times as servers were upgraded, once a given algorithm is
used for a user it is not changed until the user changes their
password.
So, worst case, if you have been a customer for 10 years and have
never changed your account password, it is currently hashed with an
algorithm which would today be considered obsolete for that purpose.
Sadly we cannot just upgrade everyone's hash scheme because we don't
know your passwords!
I have added functionality to the Panel web site to check which hash
algorithm is in use when you log in and if it isn't the new default
it sets your password to the same as the one you just supplied at
login. This upgrades you to the new default hash algorithm.
Therefore it would be good if you could take the time to log in to
https://panel.bitfolk.com/
If you do a password reset then it will also upgrade when you follow
the link to log in with the randomly-generated password. Also if you
change your password it will upgrade.
There is no visual feedback of this change happening, so you'll just
have to trust that it has. It is logged though, so I can tell you if
you want to know.
I'm afraid you will need to do this with each account you have, if
you have multiple VPSes.
=== What are cryptographic hash functions?
They're algorithms which map arbitrary data (your password) into an
output string of a fixed size, in a way which is not feasible to
reverse. We store the output in our password database, and if our
database (the LDAP directory) is ever leaked or stolen then we will
have some small comfort that the attacker would not be able to
immediately read your passwords in clear text.
=== Why do they need to change?
Computers get faster and flaws are discovered in algorithms. What
was once considered too difficult to brute-force can now be done in
seconds with a single computer. The ability to spin up a set of
servers with 1,000 GPUs is within the reach of many people.
=== Are you telling us to do this because there has been a
compromise?
No. It's just that there was a major upgrade in the last week which
everyone would benefit from; particularly the longest-standing
customers who will be stuck on algorithms older than the previous
default.
The other option would have been to invalidate everyone's passwords,
forcing a mass password change. That seemed likely to foster
speculation of compromise, and be a real annoyance besides.
=== Which algorithms are in use? What's the new default?
I'm afraid I would rather not comment on the specific algorithms
involved, but it's already public knowledge that we use OpenLDAP.
Here is documentation showing the options:
http://www.openldap.org/doc/admin24/guide.html#Password%20Storage
…and this article gives some more modern (2016) recommendations:
https://www.redpill-linpro.com/techblog/2016/08/16/ldap-password-hash.html
=== I'm confused; is something going to break if I don't take
action?
No. You can do nothing and your password will stay working and
protected with the same algorithm that was the default the last time
you changed your password (or when your account was created). Next
time you log in to the Panel (or do a password reset, or change your
password) it will be upgraded.
I just thought that I'd let people know they can influence things
sooner if they want to. Some customers have never logged in to the
Panel site, so their password hashes might never be upgraded
otherwise.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
It appears that our Direct Debit payment provider turned off another
bit of their legacy API¹ on 20 December and as a result we haven't
been issuing (any) invoices or requesting Direct Debit payments
since then as I needed to fix up some things.
I've only just had chance to look into it, but now it's all flowing
again so if you have today received invoices which talk about
service running from a date that is before today, that is why.
Apologies for any confusion caused.
Cheers,
Andy
¹ They in theory turned it off at the end of October 2017, but did
in fact leave it running so that customers set up using it could
still be billed. Apparently some of that is no longer the case. At
the moment it seems that this will just mean that I have to ask a
few people to authorise new mandates.
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
Briefly: The Ubuntu 18.04 installer now configures networking
properly using netplan¹, and later versions of Ubuntu will continue
to do so as long as the default in Ubuntu remains netplan.
More detail:
There was a recent thread² on the "users" list where a customer had
upgraded to Ubuntu 18.04 and was having problems configuring their
networking using ifupdown (as configured by /etc/network/interfaces)
like they always had.
During the course of that discussion it was noted³ that as of Ubuntu
18.04 (Bionic Beaver), Ubuntu now uses netplan to configure
networking. BitFolk has been generating an /etc/network/interfaces
file using a post-install script in the Ubuntu installer, but this
file is no longer consulted for configuring networking in Ubuntu.
The installer has now been made to generate a correct netplan config
in /etc/netplan/01-netcfg.yaml.
We recommend that customers upgrading to Ubuntu 18.04 and beyond
switch to netplan for a simpler life. netplan can't currently do
everything that ifupdown can, so if you have a complicated
networking setup you should look into this carefully. A lot can be
done in systemd-networkd hooks instead of ifupdown hooks.
The "IPv6" article on the wiki has been updated for the relevant
netplan configurations:
https://tools.bitfolk.com/wiki/IPv6
Cheers,
Andy
¹ https://netplan.io/
² https://lists.bitfolk.com/lurker/message/20181109.144005.13377548.en.html
³ https://lists.bitfolk.com/lurker/message/20181113.170055.93f45da1.en.html
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
At approximately 22:24Z, host "hen" rebooted itself unexpectedly. It
all appears to be back up again now, and I am investigating to try
to determine the cause.
Apologies for the disruption.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi all,
I remember a while back there were some emails about issues with Ubuntu 16.04 on Xen, so it was advised to not upgrade at the time. Have these issues been resolved now, is it safe to upgrade?
Kind regards,
Paul
Hi,
I'm in the final stages of putting a new monitoring system in place
and retiring Nagios.
I've already added ping checks for everyone that had them before,
and am now going through and porting the more complicated checks
that people have.
I won't cut it over or turn alert notifications on until everything
is being checked by both systems, so for a while you are going to
see service checks from:
- 85.119.80.238
- 85.119.80.244
- 2001:ba8:1f1:f040::/64
- 2001:ba8:1f1:f25d::/64
If you have firewalled your services to only allow checks from
85.119.80.244 then now would be a good time to allow the other IPs
too.
At the moment if you don't have any monitoring set up you have to
ask for it in a support ticket. If you'd like that to be set up for
you, please send a mail to support(a)bitfolk.com.
Hopefully shortly after the new system is in place I will be able to
add some sort of checkbox to the web panel to enable it, as I
already have it so the simple ping tests and alerting contacts are
built from the customer database.
It will probably be some time before more complex checks are
self-service like that, but don't be afraid to ask for something in
the mean time. Many things can be monitored without an agent, more
still by NRPE or SNMP.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi all,
Sorry if this doesn't belong here, I'm having a bizarre problem with Apache
(2.4.18) on an ubuntu 16.04.5 bitfolk VPS, just wondering if anyone might
be able to shed any light as I'm at a complete loss. Any pointers would be
much appreciated.
Over the past week or so my Apache installation has an issue where it
just.. stops responding and requests just time out until the server is
restarted, 12 hours later same thing happens, lather rinse repeat. Top /
Htop show the expected number of apache processes / threads running (<10)
but clearly they aren't doing anything, memory is 60% utilised, < 0.1 load
avg, 30% disk usage.
I'm using apache2 + mod-php7 (not ideal, but it's something I plan to
change later) just for a few low traffic Wordpress sites. Using sane
prefork settings for a low(ish) memory VPS.
Nothing obvious in the apache logs other than they just seem to go silent
reflecting the dead server, entries resume again after restarting the
server. dmesg has no hints either.
Thought it might be a DOS / brute force scan attack and maybe apache child
processes are hanging for some reason and not being terminated after
maxRequestsPerChild, or the server is saturated and not responding because
of a backlog but there is no load.. After apache2 restart everything works
perfectly again just for a while.
Not sure how else to diagnose the issue :(
Cheers,
Luke T
