Hi,
If you do not use BitFolk's Entropy service and have no interest in
doing so then this email will be of little interest to you can be
safely ignored.
If you haven't heard about the Entropy service before, please see:
https://tools.bitfolk.com/wiki/Entropy
If you *do* use the Entropy service though, I'm interested to know
what software you have that actually uses /dev/random (and not
/dev/urandom).
Some background to this question:
To provide the Entropy service we use hardware entropy generators,
currently exclusively a pair of EntropyKeys manufactured by a UK
company called Simtec Electronics Ltd.
Despite the fact that these were extremely popular little devices
(compared to other fairly niche little gadgets), Simtec always had a
supply problem and then Simtec imploded as a company, so as far as I
know these are now impossible to obtain, the IP is lost forever etc.
Although I have one spare EntropyKey ready to put in service should
one of the two in service ever die (I've not experienced that yet),
that left me slightly worried as to what I'd do if I needed to get
more.
Then I saw the OneRNG kickstarter, and decided to pledge. So now I
have 5 of (the internal USB version of) these:
http://onerng.info/
I've not yet gone any further than verifying that they keep the
entropy pool full on the machine they're plugged into, but that's
good enough for now. Could be a decade before one of my existing
EntropyKeys dies.
I have since heard that this device proved far more popular than its
manufacturer expected (sense a theme?) and they're now extremely
difficult to get hold of because they need to get a new batch made
in China. I've had multiple people contacting me on the basis of a
tweet I did about getting these, asking me to sell them mine (which
I would, but they didn't want internal USB).
The point I'm trying to make here is that the world of hardware
random number generators is not one with reliable supply lines,
unless you want to spend a fortune on some black box.
So when I came across:
http://www.2uo.de/myths-about-urandom/
I was sad that the nerdery that is the Entropy service may be
misguided, but also happy with the possibility that I might never
have to source a hardware RNG again.
Let's just take the argument posited by the article, that all
(Linux) software should just learn to love /dev/urandom¹, as true.
If you don't agree with this claim, you are disagreeing with some
pretty big names in crypto. The Hacker News commentary on the
article may also prove of interest:
https://news.ycombinator.com/item?id=10149019
At the very least, I feel the Entropy article on the BitFolk Wiki
needs an update in light of this. To justify the service's
existence, if nothing else.
Going further, the question becomes, well, what software is there in
existence that forces use of /dev/random with no configuration that
would allow otherwise? Because even if we agree that all software
*should* be using urandom, if some popular software *refuses* to
without recompile, then we're still going to have to provide an
Entropy service, because doing so is easier than running
non-packaged software.
So Entropy service users, what have you got that uses /dev/random?
Cheers,
Andy
¹ A more correct summary of it is probably, "urandom is fine all the
time except for on initial boot when a small amount of entropy
from outside the CSPRNG is desirable."
On shutdown all fairly modern Linuxes save the current entropy
pool to the filesystem and load it up from there on boot, so it's
only essential on first boot.
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
Some BitFolks are planning a trip to Sweeney & Todd's pie shop
(restaurant) in Reading, UK. If you would like to come, please fill
in this Doodle poll to say when you can:
http://doodle.com/poll/nh9abghg58g6fh6u
We have been several times before. They do nice pies. There is only
1 vegetarian option which probably isn't actually vegetarian
(cheese), and neither is the gravy. We will probably meet
in a pub first, and again afterwards.
http://www.sweeneyandtodd.co.uk/pies/
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
What will become Debian stretch (9.x) is going to be frozen on
5 February:
https://wiki.debian.org/DebianStretch
and presumably released relatively soon after that, so we've added
stretch to our self-installer.
If you are keen on testing a new install of it you should now be
able to use that. Bear in mind it still is Debian "testing" though,
not a release yet, so there may be issues you will need to report to
Debian.
More info on the self-installer:
https://tools.bitfolk.com/wiki/Using_the_self-serve_net_installer
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hello,
Ubuntu 12.04.x LTS (Precise Pangolin) goes end of life in about 6
weeks, but today I attempted to boot its installer and found its
netboot kernel¹ didn't boot.
With no useful diagnostics it's probably not something I will be
able to get fixed in the remaining time so I have removed it from
our offering a bit early.
If this severely affects anyone then let me know and I'll see if I
can spend a bit more effort on it…
The ones for 14.04.x LTS and 16.04.x LTS still work fine.
Cheers,
Andy
¹ As downloaded from
http://gb.archive.ubuntu.com/ubuntu/dists/precise/main/installer-amd64/curr…
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
As previously discussed on the users list:
https://is.gd/wvBvEn
…everyone's booting method has now been changed to pvgrub2.
The short summary is that it will look and behave more like GRUB now
(because it *is* GRUB), and you can expect a typical GRUB2
configuration in /boot/grub/grub.cfg (/boot/grub2/grub.cfg on CentOS
7.x) to work now.
More details at:
https://tools.bitfolk.com/wiki/Booting
If you think you may have reinstalled your VPS using the rescue VM
and changed its architecture in the process, it is important that
you check that we know the correct architecture. You can see what we
think your architecture is by:
- Typing "arch" in the Xen Shell.
- Looking at https://panel.bitfolk.com/account/ where it is listed
next to what we think your operating system is.
If your architecture is set incorrectly then your VPS will not boot.
If the bootloader appears to detect the correct configuration file
and kernel file but still says it can't load it, then check the
architecture first.
As far as I am aware the only way you could have got a VPS with a
different architecture to what we think you have is if you installed
it manually through the rescue VM. The supported installers only
install the matching architecture.
If you checked that and it's correct but your VPS still does not
boot, do not panic. Contact support and we can revert you to pygrub
until we figure out what is wrong. But having tested many
configurations of VPS now, the only failures encountered so far have
been trying to boot a 64-bit kernel with an i686 bootloader or vice
versa.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hello,
As you may be aware, it's long been on the BitFolk TODO list to
implement a better way (than pygrub) to boot your VPSes.
Unfortunately I have to explain some rather lengthy history so you
can know how things arrived at where they are now, and what the
options are for the future.
If you want to skip all that then please do at least skip down to
the heading "== Now ==" so you are aware of what I am proposing to
do. Without reading the history though it may not make sense.
== Ancient times ==
Kernels and initrds had to live outside of the guest in dom0. Users
couldn't update their own kernel but had to find some means to sync
their kernel and initrd to some place in the dom0 (the privileged
gust that controls everything else). People came up with various
hacks to allow users to do that, the disadvantage being that quite a
lot of that had to run in dom0.
== Still rather a long time ago ==
Someone invented pygrub, a Python implementation of something that
looked a but like GNU GRUB 0.x. It was capable of looking inside an
image file, block device or partitions on same, to try to find a
file called menu.lst.
If it found that file it would try to parse it like GRUB would,
display a menu emulating GRUB's menu, and pass the chosen kernel,
initrd and boot arguments to Xen.
Advantages:
- Users could install their own kernel packages, since these would
maintain a menu.lst file in their guest, and pygrub would then
pick up the changes when they next booted.
- Users could control their own kernel command line since this also
was in the menu.lst file.
Disadvantages:
- It still all runs in dom0. pygrub is opening an actual filesystem
that is supplied by the user, in the context of a user in dom0,
and that is not a particularly safe thing to be doing.
- It isn't actual GRUB, it's just trying to emulate it. That means
it doesn't behave quite the same and doesn't support every single
thing that is valid in a real GRUB menu.lst file.
- As a further consequence of the above, pygrub is relying on
userspace filesystem access libraries that aren't part of actual
GRUB. This sometimes lead to surprising discrepancies between what
GRUB was expected to support and what pygrub would support, such
as XFS filesystems, XZ-compressed initrds, etc.
- GRUB 0.x is now referred to as GRUB-legacy. GRUB moved on to 1.x,
it got a lot more complicated, and it now puts its configuration
in /boot/grub/grub.cfg. Many Linux distributions gave up on
GRUB-legacy and only support grub.cfg now, so users of such are
left to maintain their own menu.lst file.
On Debian and Ubuntu you can still install the package grub-legacy
and it will maintain menu.lst, but it's getting increasingly
persistent about wanting you to move to grub.cfg.
This is the boot method that BitFolk has almost always been using.
Up until a couple of months ago there was still one customer who for
various reasons had to have a kernel hard-coded in their dom0 config
file, but now there's none. Everyone's on pygrub.
== Recent-ish history ==
Around 2010 in light of the above disadvantages, a member of the Xen
project ported GRUB 0.x to boot as a paravirtual guest under Xen.
That's the same sort of thing that your VPSes are. So you get a VM
started, and the VM runs actual GRUB. Once GRUB knows what it wants
to boot, it chain loads to that.
This was known as "pvgrub", an awful name really since it differs
from pygrub in only the descender of one letter. Should have passed
that one by the marketing department first.
Advantages:
- Runs only inside a VM.
- Still allows user control.
Disadvantages:
- Part of the Xen code tree, not really packaged anywhere.
- Still only GRUB-legacy support.
Due to it still only supporting GRUB-legacy, which wasn't any
different from what pygrub supports, BitFolk did not pursue it.
We ran into a couple of sticky situations such as when XZ-compressed
kernels became popular in Debian and pygrub didn't support that, but
we worked around it¹.
Still it became more pressing to support a better boot system
because it would only be a matter of time before more distributions
cease to support menu.lst or do something else that pygrub doesn't
support.
== Just 3¼ years ago! ==
A GNU GRUB committer added PV-booting support to upstream GRUB 2:
https://lists.xen.org/archives/html/xen-devel/2013-11/msg01216.html
What this means is that using only the upstream GRUB 2 binaries you
can generate a GRUB-as-kernel image that can be booted as Xen PV
virtual machine, which can then do everything that GRUB 2 can
normally do. Namely look inside your block devices for GRUB configs
and boot your own kernels.
It took a couple of years for this to filter down to being supported
in Debian stable and to iron out some bugs, so, fast forward to…
== Now ==
My recent experimentations with GRUB indicate that it may now be a
good time to switch to this method of booting instead of pygrub.
Here's how I am thinking of doing it.
- BitFolk's 64-bit or 32-bit GRUB2 image is booted as your kernel,
in your VM.
- It checks for existence of:
- /boot/xen/pvboot-x86_64.elf
- /xen/pvboot-x86_64.elf
- /boot/xen/pvboot-i386.elf
- /xen/pvboot-i386.elf
on any of your block devices/partitions.
If any of those are found then a menu entry for chainloading to
your own bootloader at that path is created.
- It checks for existence of:
- /boot/grub/grub.cfg
- /grub/grub.cfg
on any of your block devices/partitions.
If either of those are found then a menu entry for reading your
GRUB 2 config from that path is created.
- It checks for existence of:
- /boot/grub/menu.lst
- /grub/menu.lst
on any of your block devices/partitions.
If either of those are found then a menu entry for reading a
GRUB-legacy config from that path is created.
- The available menu entries are displayed for 5 seconds. If no
selection is made then a boot attempt is made for each menu
entry, starting at the top and going in order.
- Selection of a menu entry (or allowing it to time out and be
selected by default) will result in some text being displayed for
2 seconds explaining what is going to happen, e.g.:
Loading your GRUB config at (xen/xvda,msdos1)/boot/grub/grub.cfg...
would indicate that a GRUB2 config was found on the first MSDOS
partition of the Xen disk called xvda.
The message and 2 second wait could be cancelled by pressing
Escape though obviously you'd have to be looking at the console to
do that.
As you can see, the order of boot method is:
1. Chainload to your own bootloader.
2. Use your GRUB2 config.
3. Use your GRUB-legacy config.
Although there is some limited interactivity in this proposal (you
get to select a menu item if you're looking at your console),
obviously servers do need to boot automatically, so assuming no
intervention then after 5 seconds it's going to pick the first
available from the above and then after 2 seconds more it will do
it.
Unfortunately the configuration for how GRUB will behave has to be
baked into the binary image itself so I can't think of an easy way
to allow you to select your own timeouts or remove them altogether,
not on a per-user basis anyway.
It does mean a 7 second delay to booting, or more if for some reason
one of the earlier steps fails. A step can fail if the image or
config is incorrect. BitFolk's GRUB will only be checking for the
existence of a file path, not that what is there is valid. So for
example if you have GRUB2 installed so there is a
/boot/grub/grub.cfg file present but then you also touch the file
/boot/xen/pvboot-x86_64.elf, it will try to chainload to that first.
If that fails then it will fall back to parsing /boot/grub/grub.cfg,
which will add another 2 second pause.
Perhaps it would be acceptable to reduce the initial 5 second
countdown to 2 seconds? Pressing any key aborts the countdown and
gives you as long as you like to choose a menu entry, as usual with
GRUB.
The chainloading option hasn't been mentioned yet. In practice I
don't expect many people will be interested in it, but it does
provide for running your own bootloader.
At the moment although you may have GRUB-legacy installed in order
to maintain a menu.lst file, your bootloader is never actually
executed, only its config file is parsed, by pygrub. I am proposing
to move to using GRUB2 to parse your grub.cfg, but still this is
BitFolk's GRUB2 binary that is being run, not yours.
If for some reason you did need to run your own bootloader, perhaps
because you need to boot off of something that is not supported by
the GRUB binary that BitFolk is using, then you could install your
own bootloader that supports the PV boot protocol.
As far as I am aware only Debian has distribution support for this.
As of Debian jessie, installing grub-xen in your guest installs a
GRUB-as-PV-kernel image in /boot/xen/pvboot-$ARCH.elf. Using this
option, the last thing that runs before your VPS's kernel would be
your VPS's bootloader.
As I say, I expect for most people that will be unneeded
complication, as you don't really care whose bootloader is being
run, just that it boots. But the option would be there should you
need it.
I am not intending for the switching to pvgrub from pygrub to be
optional or reversible. I have put some effort into trying to ensure
that pvgrub will boot from a guest's menu.lst and all customers
currently have a menu.lst, so I am hoping I do not need to build in
a setting for this. Naturally if we switch and then someone finds
out it doesn't work then we can switch them back to pygrub until the
problem is worked out.
At the moment I have tested the following configurations:
- 64-bit Debian jessie guest, single root filesystem on xvda1:
- Chainload to GRUB, parse /boot/grub/grub.cfg
- Parse /boot/grub/grub.cfg
- Parse /boot/grub/menu.lst
- i686 Debian jessie guest, single root filesystem on xvda1:
- Parse /boot/grub/grub.cfg
- Parse /boot/grub/menu.lst
- i686 Debian jessie guest, single root filesystem directly on xvda:
- Parse /boot/grub/menu.lst
If you have a planned reboot coming up, it would be useful to me if
you would let me do the reboot for you. I would take the opportunity
to switch you to pvgrub booting to check it works in your
configuration. If it didn't then I would just revert you back to
pygrub and leave you be.
So, that is what I am thinking of doing and how I am proposing it
will work. If you have any comments about this, especially if you
think it will not work for you, or you have suggestions about how it
could be done better, I really want to hear about it.
Thanks for reading this immense email!
Cheers,
Andy
¹ You may recall, we modified pygrub to detect an XZ-compressed
kernel and unpack it using the actual xz utility, until a newer
version of pygrub could be installed which supported those
kernels.
--
https://bitfolk.com/ -- No-nonsense VPS hosting
> I'd be interested to hear any (even two word) reviews of their sofas…
Provides seating.
— Andy Davidson
