***This has been solved, post for others who may experience it.***
Late yesterday evening I returned hone and had a desperate message from the
secretary of the local cancer support charity for whom I provide email
facilities and web support. The email server seemed to be refusing emails
in or out.
I investigated and saw that the VPS had run out of storage space. My first
act was to try to uninstall some programs thet have been there since Adam
was a boy. But apt would not play ball. No storage space meant it could not
do it.
So I firstly contacted Andy to request some more storage space, but he
would not see it, of course, till the morning.
Next I trawled through manually hunting larger text or data files I could
happily lose. Interestingly fail2ban logs and the latest archive of them
were very large, they were sacrificed. The emailbox that is used to collect
confirmation of DKIM was busting at the seams, they all went. I will change
the config so that I wont get confirmations anymore. 6 months of it working
seems to be sufficient. Various other gains were made which should have
provided enough space but nothing was happening and I was falling asleep at
the keyboard so gave up until bright and early today.
This morning Andy had already activated extra storage for me (as well as
giving loads of tips), but it needed activation at my end, which required
installing parted which I could not do as apt could not access any storage.
So I decided to try rebooting. (always the first thing recommended) It went
down OK but would not reboot. Dead as a dodo. Undaunted, I fired up a Xen
terminal and rebooted from there. It worked a dream. and the recovered
space was there, so I could now use apt to remove old kernel modules and do
an autoremove, install parted, get the new atorage recognised and start on
the slow process of tidying up the disc space and so on. I fully intend to
ask Andy for the Icinga monitoring service so I don't get this again.
What a fun day, but a day of learning
Keith
--
Leighton Linslade Cancer Support Group <https://leighton-linslade-csg.org>
CVE-2024-6387 details a flaw in OpenSSH that could *potentially* give an
attacker a root shell in "6-8 hours"
It's not in MITRE yet, but Qualys have named it "regreSSHion" and you can
read about it on their site
There's an updated package in Debian already, but it looks like the
information's still embargoed (even the openssh package changelog is
404ing) so I can only *assume* they've fixed it but can't tell anyone yet
(it wasn't on security.debian.org just now either
This is probably an update you don't want to be sleeping on
Hi,
An unauthenticated remote root exploit has been discovered in SSH,
including in versions shipped by Debian stable and newer, and most
other up to date Linux distributions.
https://security-tracker.debian.org/tracker/CVE-2024-6387
Please make sure you have applied the necessary upgrades.
If for some reason you are unable to apply an upgrade, the issue can
be mitigated by setting LoginGraceTime to 0 in /etc/ssh/sshd_config.
This will make it easier for people to tie up all connection slots,
denying access to legitimate connections, but does avoid the remote
root exploit.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi folks
I'm running mail-in-a-box on a Bitfolk VPS.
https://mailinabox.email/
It's making the following complaint:
"This box's reverse DNS is currently aquitaine.richardskingdom.net
(IPv4) and 2001-ba8-1f1-f037-0-0-0-2.autov6rev.bitfolk.space (IPv6), but
it should be aquitaine.richardskingdom.net. Your ISP or cloud provider
will have instructions on setting up reverse DNS for this box."
This is with reverse DNS set to "automatic" in the Bitfolk panel.
The only other panel option seems to be to delegate the reverse IPv6
zones to my name server.
I'm using the mail-in-a-box built-in name server, however, and
delegating to that produces the following result:
"This box's reverse DNS is currently aquitaine.richardskingdom.net
(IPv4) and [Not Set] (IPv6), but it should be
aquitaine.richardskingdom.net ..."
I infer that mail-in-a-box name server is not setting reverse IPv6
records for itself.
There doesn't appear to be a way to tell mail-in-a-box to set the
reverse DNS correctly via its GUI - there is no option to add a custom
PTR record (other record types can be added).
I don't know what name server software is running under the hood, and
I'm loath to make config changes except via the GUI in case they get
overwritten when mail-in-a-box updates.
Can anyone advise me how to set my IPv6 reverse DNS to
aquitaine.richardskingdom.net?
I should note the mail server works (I am sending this message through
it) so this is only to make the error message go away (and possibly to
get IPv6 mail transportation working correctly).
If this sounds like an ignorant / nonsense request, congratulations, you
have detected successfully that I have no idea what I'm doing with IPv6...
Thanks in advance
Richard.
>
> I will do some more investigation of this failure mode but in light of
> doing away with bonding being the direction we are already going, I don't
> think I want to alter how bonding is done on what will soon be a legacy
> setup.
Shouldn't this failure mode have been caught by LACPDUs?
--
Maria Blackmore
I am responsible for several VPSes, here and elsewhere. Five of them are
running Ubuntu 22.04, three Debians.
The script I use to update them checks, at the end, for the existence of
/var/run/reboot-required
If it finds it, it offers to reboot the VPS.
It does this happily on all but one VPS, one of the Ubuntu ones here. The
Ubuntu version of apt-get on all of the Ubuntu ones recognises that a
reboot is required after a kernel update etc and will popup a message
saying so, but it looks like only on this single machine, that file doesn't
exist afterwards.
I have no idea why not. Anyone got any ideas?
Ian
Andy said on the announce list:
"As usual with this sort of thing though, all the complexity is in the
packages you have installed, so that is no promise that it would be plain
sailing for you."
Having just tried, I am left with a VPS with no external networking.
Looking at the launchpad bug tracker, there have been problems with this.
I have tried one of the solutions there via logging in through the xen
console without getting it to work, but it is complicated by trying this on
my phone in terms of sorting it out.
Fortunately, it wasn't doing anything much - almost no additional packages
were installed - so at the moment, I have turned it off and will wait for
the installer to do a fresh install.
Ian
Hi,
As you may be aware, the next LTS release of Ubuntu is supposed to
be ready in a couple of days.
I've tested a do-release-upgrade from a basic 22.04 cloud image
(what you get when you install 22.04 at BitFolk) and it seemed to go
fine. As usual with this sort of thing though, all the complexity is
in the packages you have installed, so that is no promise that it
would be plain sailing for you.
We will try to get a Xen Shell installer option added for 24.04 as
soon after release as we can, but in the mean time just installing
22.04 and then typing "sudo do-release-upgrade -d" should get you
there.
I *think* it is the case that you need the "-d" as
do-release-upgrade normally doesn't like doing it until the first
point release.
Thanks,
Andy
Ubuntu 24.04 LTS debtest1.vps.bitfolk.space hvc0
debtest1 login: ubuntu
Password:
Welcome to Ubuntu 24.04 LTS (GNU/Linux 6.8.0-31-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
System information as of Tue Apr 23 13:31:00 UTC 2024
System load: 0.07 Memory usage: 6% Processes: 131
Usage of /: 14.6% of 19.20GB Swap usage: 0% Users logged in: 0
Expanded Security Maintenance for Applications is not enabled.
0 updates can be applied immediately.
Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
ubuntu@debtest1:~$ uname -a
Linux debtest1.vps.bitfolk.space 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat
Apr 20 00:40:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
In light of the recent XZ/lzma backdoor we should perhaps think
harder about how complex sshd is and the wisdom of exposing that to
the entire Internet.
At BitFolk, this currently needs to be exposed to the Internet so
that you can connect to your Xen Shell (console).
More info on the Xen Shell for those new to it:
https://tools.bitfolk.com/wiki/Xen_Shell
When thinking about ways to tighten this up, they are all going to
involve taking away your ability to just SSH to
you(a)you.console.bitfolk.com.
There's a couple of ways this could go:
1. Remove direct SSH capability, replace with web
All current Xen Shell features to be put on a web interface.
Console to be used from a web interface.
I don't relish this, but I have seen some pretty nifty web
terminals so maybe it wouldn't be that bad.
2. Firewall off SSH from the Internet, poke holes temporarily
You'd have to supply to the Panel some list of net blocks that
you will SSH from and then there'd be a button to punch holes in
the firewall for SSH from those net blocks for 6 hours (for
example).
There would have to be a limit on the size of the net blocks.
Let's say a /16 for IPv4 and a /32 for IPv6.
Option (2) is lots easier to implement so could happen fairly
quickly, but it is more fiddly to use: You're in an arbitrary place
and you suddenly need to connect to your Xen Shell; you've then got
to log in to Panel, work out your IP address¹, add it to the allow
list, then hit the button. Finally you can SSH.
Option (1) will require a lot of work but maybe it's less friction
to use a web-based terminal if you have to visit a web site anyway?
Option (2) does not preclude option (1). Providing a web-based
terminal that doesn't need the "allow SSH" button to be pressed can
be implemented after option (2) is done, as a sort of stretch goal.
Or maybe the web terminal is such an attractive and required feature
that it should be done first, before option (2) would be enacted?
I am disregarding the idea of flipping the default to having
password auth disabled. It is currently by default enabled but you
can disable it on a per-VM basis. I do recommend that you do that,
but what I am really more concerned about is an(other) exploit in
sshd, and that doesn't really have to be anything to do with
authentication (the XZ thing wasn't). Yes it would be a very bad day
for you if someone got in to your Xen Shell, but that's as far as
that goes, so I am still okay with leaving that determination up to
the customer.
I can't see a way of avoiding there having to be a list of net
blocks. With 50+ VMs on a host, if SSH is opened up from anywhere
for any one of them then it is open for all so really even with a
timeout it would be open a lot of the time.
I guess something interesting could be done for those not on legacy
Internet: assign unique IPv6 console address that can only be used
for connecting to that VM's Xen Shell. 😀
I'd be interested in hearing your thoughts on this.
Thanks,
Andy
¹ Which you may not know because you might be behind one or more
layers of NAT.
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
We had a customer ask about backups of images of their VM disk. This
is not currently a service that we offer.
It has some advantages, so it could be a service that we offer.
The main advantage being:
- High degree of confidence that the VM is just able to be one-click
restored and is then bootable and working, as it's an exact image
of the disk
It does of course also have a heap of disadvantages such as:
- Granularity of backup is "the entire disk". No way to pick and
choose what is backed up except by putting things on different OS
disks.
- It's based off a disk snapshot so nothing in memory gets
preserved; a restore event will be similar to a "boot after
unexpected power cut" event to the restored VM. Though doing the
snapshot while the VM is powered off for a few seconds would be
an option.
- This obviously uses a lot of disk space which we'd charge for at
our archive storage rate: £0.40+VAT/50GB/month. If preferred this
could go onto third party cloud storage but bear in mind that
Amazon S3 is about 3 times the cost of BitFolk's archive storage.
- Limited differential backups are possible though I'm yet to fully
understand how well that works. All I can say right now is that
a differential would use less, but I'm not sure really how much
less.
So, there would be "some charge" for the service itself and
definitely a charge at archive storage rates for the actual amount
of storage involved.
If this sounds like a service that you think you'd actually want,
please get in touch off-list and we'll see if we can develop it.
As it's more the BitFolk way for customers to do this sort of thing
themselves I don't really expect this to be widely desired, but I am
working on it for one customer anyway so if I can make it more
generally applicable then I will.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
