13 Mar
2016
13 Mar
'16
1:56 a.m.
Hi,
If you do not use BitFolk's Entropy service and have no interest in
doing so then this email will be of little interest to you can be
safely ignored.
If you haven't heard about the Entropy service before, please see:
https://tools.bitfolk.com/wiki/Entropy
If you *do* use the Entropy service though, I'm interested to know
what software you have that actually uses /dev/random (and not
/dev/urandom).
Some background to this question:
To provide the Entropy service we use hardware entropy generators,
currently exclusively a pair of EntropyKeys manufactured by a UK
company called Simtec Electronics Ltd.
Despite the fact that these were extremely popular little devices
(compared to other fairly niche little gadgets), Simtec always had a
supply problem and then Simtec imploded as a company, so as far as I
know these are now impossible to obtain, the IP is lost forever etc.
Although I have one spare EntropyKey ready to put in service should
one of the two in service ever die (I've not experienced that yet),
that left me slightly worried as to what I'd do if I needed to get
more.
Then I saw the OneRNG kickstarter, and decided to pledge. So now I
have 5 of (the internal USB version of) these:
http://onerng.info/
I've not yet gone any further than verifying that they keep the
entropy pool full on the machine they're plugged into, but that's
good enough for now. Could be a decade before one of my existing
EntropyKeys dies.
I have since heard that this device proved far more popular than its
manufacturer expected (sense a theme?) and they're now extremely
difficult to get hold of because they need to get a new batch made
in China. I've had multiple people contacting me on the basis of a
tweet I did about getting these, asking me to sell them mine (which
I would, but they didn't want internal USB).
The point I'm trying to make here is that the world of hardware
random number generators is not one with reliable supply lines,
unless you want to spend a fortune on some black box.
So when I came across:
http://www.2uo.de/myths-about-urandom/
I was sad that the nerdery that is the Entropy service may be
misguided, but also happy with the possibility that I might never
have to source a hardware RNG again.
Let's just take the argument posited by the article, that all
(Linux) software should just learn to love /dev/urandom¹, as true.
If you don't agree with this claim, you are disagreeing with some
pretty big names in crypto. The Hacker News commentary on the
article may also prove of interest:
https://news.ycombinator.com/item?id=10149019
At the very least, I feel the Entropy article on the BitFolk Wiki
needs an update in light of this. To justify the service's
existence, if nothing else.
Going further, the question becomes, well, what software is there in
existence that forces use of /dev/random with no configuration that
would allow otherwise? Because even if we agree that all software
*should* be using urandom, if some popular software *refuses* to
without recompile, then we're still going to have to provide an
Entropy service, because doing so is easier than running
non-packaged software.
So Entropy service users, what have you got that uses /dev/random?
Cheers,
Andy
¹ A more correct summary of it is probably, "urandom is fine all the
time except for on initial boot when a small amount of entropy
from outside the CSPRNG is desirable."
On shutdown all fairly modern Linuxes save the current entropy
pool to the filesystem and load it up from there on boot, so it's
only essential on first boot.
--
http://bitfolk.com/ -- No-nonsense VPS hosting
14 Mar
14 Mar
9:46 a.m.
New subject: Dear Entropy service users, what software uses /dev/random?
On 13/03/16 01:56, Andy Smith wrote:
Going further, the question becomes, well, what
software is there in
existence that forces use of /dev/random with no configuration that
would allow otherwise? Because even if we agree that all software
*should* be using urandom, if some popular software *refuses* to
without recompile, then we're still going to have to provide an
Entropy service, because doing so is easier than running
non-packaged software.
So Entropy service users, what have you got that uses /dev/random?
My information might be very out of date, so perhaps somebody will
correct me, but Exim's TLS support (via gnutls) would read a lot from
/dev/random to set up connections - far more than OpenSSL did.
I think this was the reason I started using the entropy service, but
I've not checked for a long time whether the issue's still valid. A few
Debian bug reports suggest it might be fixed.
The other place I've seen it recently is when generating a new key pair
with GnuPG, though that was in a test suite rather than on my VPS.
--
Dominic Cleal
dominic(a)computerkb.co.uk
10:22 a.m.
New subject: Dear Entropy service users, what software uses /dev/random?
Hi Dominic,
On Mon, Mar 14, 2016 at 09:46:04AM +0000, Dominic Cleal wrote:
On 13/03/16 01:56, Andy Smith wrote:
Interesting, thanks. I'll now do:
$ sudo strace -o open -p $(pgrep exim4) 2>&1 | grep random
and see what it says…
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
So Entropy service users, what have you got that
uses /dev/random?
My information might be very out of date, so perhaps somebody will
correct me, but Exim's TLS support (via gnutls) would read a lot from
/dev/random to set up connections - far more than OpenSSL did. I'd be interested to hear any (even two word)
reviews of their sofas…
Provides seating. — Andy Davidson
11:23 a.m.
Hi,
I listened to some of the ubuntu podcast on the way in to work this
morning and they mentioned the util "fatrace". Turns out you can't use
the fanotify functions with /dev, but I've managed to figure out a
good way of doing this.
Assuming you've got SystemTap (kernel probing functionality, see at
the end of the email) installed, then try:
sudo stap random_read.stp
where random_read.stp looks like:
probe kernel.function("random_read").call
{
printf("%s[%d] len:%d\n", execname(), tid(), $nbytes)
}
This will print out the executable name, process id and number of
bytes requested each time a process reads from /dev/random. You can
verify it with e.g.
dd if=/dev/random of=/dev/null count=1
FWIW, whilst I was testing I was using urandom_read instead and exim
was reading from there, not from random_read. ymmv. The only thing
I've seen so far is "dd" :) I'll leave it running and report back if I
spot anything.
This is the first time I've played with systemtap and I may have
missed something. I'm not sure that the ".call" should be there for
example.
Cheers,
Roger
Installation notes for ubuntu:
apt-get install systemtap
# Install kernel debug symbols, this is less optimal than it could be.
See https://wiki.ubuntu.com/Kernel/Systemtap#Where_to_get_debug_symbols_for_ker…
codename=$(lsb_release -c | awk '{print $2}')
sudo tee /etc/apt/sources.list.d/ddebs.list << EOF
deb http://ddebs.ubuntu.com/ ${codename} main restricted universe
multiverse
deb http://ddebs.ubuntu.com/ ${codename}-security main restricted
universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-updates main restricted
universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-proposed main restricted
universe multiverse
EOF
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys ECDCAD72428D7C01
sudo apt-get update
sudo apt-get install linux-image-$(uname -r)-dbgsym
On Mon, Mar 14, 2016 at 10:24 AM, Andy Smith <andy(a)bitfolk.com> wrote:
On Mon, Mar 14, 2016 at 10:22:28AM +0000, Andy Smith
wrote:
$ sudo strace -o open -p $(pgrep exim4)
2>&1 | grep random
Hmm, maybe need a -ff on that to follow forks…
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEAREDAAYFAlbmkVMACgkQIJm2TL8VSQuUkgCfdFHtYhq/iJsa3HXykeVA73GH
gDMAn13IDR+rDx63BHFGp2HnGuJbTgEE
=j5zJ
-----END PGP SIGNATURE-----
_______________________________________________
users mailing list
users(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/users
1:50 p.m.
New subject: Dear Entropy service users, what software uses /dev/random?
On Mon, Mar 14, 2016 at 11:23:04AM +0000, Roger Light wrote:
I listened to some of the ubuntu podcast on the way in
to work this
morning and they mentioned the util "fatrace". Turns out you can't use
the fanotify functions with /dev, but I've managed to figure out a
good way of doing this.
Assuming you've got SystemTap (kernel probing functionality, see at
the end of the email) installed, then try:
sudo stap random_read.stp
where random_read.stp looks like:
probe kernel.function("random_read").call
{
printf("%s[%d] len:%d\n", execname(), tid(), $nbytes)
}
This will print out the executable name, process id and number of
bytes requested each time a process reads from /dev/random. You can
verify it with e.g.
[snipped]
SystemTap is cool, but I can strongly recommend sysdig as a
user-friendly alternative to it. It works on the same principle,
i.e. using a kernel module to insert probes into the running system,
but you don't need to know how to write any code. Instead the
invocation is very much like wireshark / tshark filters (no
coincidence, since both projects share the same founders). For
example:
$ sudo sysdig evt.type=open and fd.name=/dev/random
1911991 13:44:51.773593245 0 dd (21147) < open fd=3(<f>/dev/random)
name=/dev/random flags=1(O_RDONLY) mode=0
As you can imagine, system-wide events can be sliced and diced in
countless different ways, and much more quickly than with SystemTap as
a result of not needing to write a new probe for each inspection
type. There's even an interactive UI. For more info see
http://www.sysdig.org/
16 Mar
16 Mar
3:22 p.m.
Hi,
I've not had any output from the systemtap script so I don't believe
anything has accessed /dev/random since my last email. I'm running
exim4, spamd, php-fpm, sshd, nginx amongst others.
Same caveat as before - I might have missed something important.
Cheers,
Roger
On Mon, Mar 14, 2016 at 11:23 AM, Roger Light <roger(a)atchoo.org> wrote:
Hi,
I listened to some of the ubuntu podcast on the way in to work this
morning and they mentioned the util "fatrace". Turns out you can't use
the fanotify functions with /dev, but I've managed to figure out a
good way of doing this.
Assuming you've got SystemTap (kernel probing functionality, see at
the end of the email) installed, then try:
sudo stap random_read.stp
where random_read.stp looks like:
probe kernel.function("random_read").call
{
printf("%s[%d] len:%d\n", execname(), tid(), $nbytes)
}
This will print out the executable name, process id and number of
bytes requested each time a process reads from /dev/random. You can
verify it with e.g.
dd if=/dev/random of=/dev/null count=1
FWIW, whilst I was testing I was using urandom_read instead and exim
was reading from there, not from random_read. ymmv. The only thing
I've seen so far is "dd" :) I'll leave it running and report back if I
spot anything.
This is the first time I've played with systemtap and I may have
missed something. I'm not sure that the ".call" should be there for
example.
Cheers,
Roger
Installation notes for ubuntu:
apt-get install systemtap
# Install kernel debug symbols, this is less optimal than it could be.
See https://wiki.ubuntu.com/Kernel/Systemtap#Where_to_get_debug_symbols_for_ker…
codename=$(lsb_release -c | awk '{print $2}')
sudo tee /etc/apt/sources.list.d/ddebs.list << EOF
deb http://ddebs.ubuntu.com/ ${codename} main restricted universe
multiverse
deb http://ddebs.ubuntu.com/ ${codename}-security main restricted
universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-updates main restricted
universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-proposed main restricted
universe multiverse
EOF
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys ECDCAD72428D7C01
sudo apt-get update
sudo apt-get install linux-image-$(uname -r)-dbgsym
On Mon, Mar 14, 2016 at 10:24 AM, Andy Smith <andy(a)bitfolk.com> wrote:
> On Mon, Mar 14, 2016 at 10:22:28AM +0000, Andy Smith wrote:
>> $ sudo strace -o open -p $(pgrep exim4) 2>&1 | grep random
>
> Hmm, maybe need a -ff on that to follow forks…
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEAREDAAYFAlbmkVMACgkQIJm2TL8VSQuUkgCfdFHtYhq/iJsa3HXykeVA73GH
> gDMAn13IDR+rDx63BHFGp2HnGuJbTgEE
> =j5zJ
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> users mailing list
> users(a)lists.bitfolk.com
> https://lists.bitfolk.com/mailman/listinfo/users
> 
18 Mar
18 Mar
5:37 p.m.
I also have had nothing access /dev/random in the last 48 hours.
I wrote a simple script to log available entropy every 10 seconds and ran
it for 36 hours. I had a maximum of 2043 and a minimum of 132, the graph
being quite erratic.
I suppose the question really is what's a sensible minimum level to be
happy?
On 16 March 2016 at 15:22, Roger Light <roger(a)atchoo.org> wrote:
Hi,
I've not had any output from the systemtap script so I don't believe
anything has accessed /dev/random since my last email. I'm running
exim4, spamd, php-fpm, sshd, nginx amongst others.
Same caveat as before - I might have missed something important.
Cheers,
Roger
On Mon, Mar 14, 2016 at 11:23 AM, Roger Light <roger(a)atchoo.org> wrote:
--
Robert Gauld
http://www.robertgauld.co.uk
Hi,
I listened to some of the ubuntu podcast on the way in to work this
morning and they mentioned the util "fatrace". Turns out you can't use
the fanotify functions with /dev, but I've managed to figure out a
good way of doing this.
Assuming you've got SystemTap (kernel probing functionality, see at
the end of the email) installed, then try:
sudo stap random_read.stp
where random_read.stp looks like:
probe kernel.function("random_read").call
{
printf("%s[%d] len:%d\n", execname(), tid(), $nbytes)
}
This will print out the executable name, process id and number of
bytes requested each time a process reads from /dev/random. You can
verify it with e.g.
dd if=/dev/random of=/dev/null count=1
FWIW, whilst I was testing I was using urandom_read instead and exim
was reading from there, not from random_read. ymmv. The only thing
I've seen so far is "dd" :) I'll leave it running and report back if I
spot anything.
This is the first time I've played with systemtap and I may have
missed something. I'm not sure that the ".call" should be there for
example.
Cheers,
Roger
Installation notes for ubuntu:
apt-get install systemtap
# Install kernel debug symbols, this is less optimal than it could be.
See
https://wiki.ubuntu.com/Kernel/Systemtap#Where_to_get_debug_symbols_for_ker…
codename=$(lsb_release -c | awk '{print $2}')
sudo tee /etc/apt/sources.list.d/ddebs.list << EOF
deb http://ddebs.ubuntu.com/ ${codename} main restricted universe
multiverse
deb http://ddebs.ubuntu.com/ ${codename}-security main restricted
universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-updates main restricted
universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-proposed main restricted
universe multiverse
EOF
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
ECDCAD72428D7C01
sudo apt-get update
sudo apt-get install linux-image-$(uname -r)-dbgsym
On Mon, Mar 14, 2016 at 10:24 AM, Andy Smith <andy(a)bitfolk.com> wrote:
> On Mon, Mar 14, 2016 at 10:22:28AM +0000, Andy Smith wrote:
>> $ sudo strace -o open -p $(pgrep exim4) 2>&1 | grep random
>
> Hmm, maybe need a -ff on that to follow forks…
>
_______________________________________________
users mailing list
users(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/users
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEAREDAAYFAlbmkVMACgkQIJm2TL8VSQuUkgCfdFHtYhq/iJsa3HXykeVA73GH
> gDMAn13IDR+rDx63BHFGp2HnGuJbTgEE
> =j5zJ
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> users mailing list
> users(a)lists.bitfolk.com
> https://lists.bitfolk.com/mailman/listinfo/users
>
5:46 p.m.
New subject: Dear Entropy service users, what software uses /dev/random?
Hi Robert,
On Fri, Mar 18, 2016 at 05:37:22PM +0000, Robert Gauld wrote:
I wrote a simple script to log available entropy every
10 seconds and ran
it for 36 hours. I had a maximum of 2043 and a minimum of 132, the graph
being quite erratic.
I suppose the question really is what's a sensible minimum level to be
happy?
Not really; a key argument of the article
(http://www.2uo.de/myths-about-urandom/) is that measurements of
available entropy are meaningless, because (a) there is really no way to
know, and (b) the CSPRNG behind /dev/urandom can always provide you
more and you should be using that.
*Anything* that is reading from /dev/random is a concern because it
could potentially block.
So far it seems we are not finding anything now that uses
/dev/random, although I suspect that gpg may well still do so when
generating a new key. I haven't tested that yet.
It's looking like the entropy service wiki article at the very least
needs rewriting to stress:
- urandom is good enough; try to make your software use that
- don't configure this just because you measure a low entropy pool,
do check exactly what software is blocking on /dev/random
- let us know what software that is
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
5:55 p.m.
I don't know if it's related but:
openssl dhparam -out dhparam.pem 4096
was taking forever on amazon and was certainly faster on bitfolk.
Worth to check with/without entropy generators.
On Fri, Mar 18, 2016 at 1:46 PM, Andy Smith <andy(a)bitfolk.com> wrote:
Hi Robert,
On Fri, Mar 18, 2016 at 05:37:22PM +0000, Robert Gauld wrote:
I wrote a simple script to log available entropy
every 10 seconds and ran
it for 36 hours. I had a maximum of 2043 and a minimum of 132, the graph
being quite erratic.
I suppose the question really is what's a sensible minimum level to be
happy?
Not really; a key argument of the article
(http://www.2uo.de/myths-about-urandom/) is that measurements of
available entropy are meaningless, because (a) there is really no way to
know, and (b) the CSPRNG behind /dev/urandom can always provide you
more and you should be using that.
*Anything* that is reading from /dev/random is a concern because it
could potentially block.
So far it seems we are not finding anything now that uses
/dev/random, although I suspect that gpg may well still do so when
generating a new key. I haven't tested that yet.
It's looking like the entropy service wiki article at the very least
needs rewriting to stress:
- urandom is good enough; try to make your software use that
- don't configure this just because you measure a low entropy pool,
do check exactly what software is blocking on /dev/random
- let us know what software that is
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEAREDAAYFAlbsPv4ACgkQIJm2TL8VSQuHwQCgyDT2rpmPyfgliTmRfZhde7kn
Qe8AnjxS1meNgk0+CqhUki43RE93Wq7v
=surv
-----END PGP SIGNATURE-----
_______________________________________________
users mailing list
users(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/users
26 Apr
26 Apr
4:09 p.m.
On Fri, Mar 18, 2016 at 5:46 PM, Andy Smith <andy(a)bitfolk.com> wrote:
So far it seems we are not finding anything now that
uses
/dev/random, although I suspect that gpg may well still do so when
generating a new key. I haven't tested that yet.
I happened to need to test this today. It seems to open both
/dev/urandom and /dev/random, but block on /dev/random
$ strace -e file,select gpg --gen-key
...
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, use the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
open("/home/graham/.gnupg/random_seed", O_RDONLY) = 4
open("/dev/urandom", O_RDONLY) = 4
select(5, [4], NULL, NULL, {3, 0}) = 1 (in [4], left {2, 999994})
open("/dev/random", O_RDONLY) = 5
select(6, [5], NULL, NULL, {3, 0}) = 1 (in [5], left {2, 999995})
select(6, [5], NULL, NULL, {3, 0}) = 0 (Timeout)
This isn't something I need to run on bitfolk. However, if anyone does
know how to persuade gpg to use /dev/urandom and/or whether it's safe
to do so in a docker container shortly after startup, I'd be curious
to hear the answer.
From my reading of
http://www.2uo.de/myths-about-urandom/ the ideal
solution would be for gpg to use
getrandom(2) for this.
8:05 p.m.
New subject: Dear Entropy service users, what software uses /dev/random?
Hi Graham,
On Tue, Apr 26, 2016 at 05:09:50PM +0100, Graham Bleach wrote:
On Fri, Mar 18, 2016 at 5:46 PM, Andy Smith
<andy(a)bitfolk.com> wrote:
[…]
So far it seems we are not finding anything now
that uses
/dev/random, although I suspect that gpg may well still do so when
generating a new key. I haven't tested that yet.
I happened to need to test this today. It seems to open both
/dev/urandom and /dev/random, but block on /dev/random
$ strace -e file,select gpg --gen-key This isn't something I need to run on bitfolk.
However, if anyone does
know how to persuade gpg to use /dev/urandom and/or whether it's safe
to do so in a docker container shortly after startup, I'd be curious
to hear the answer.
(I know you know this but just to make the general point…) Best
practice of course would be to never generate a long-lived key like
a PGP key on networked hardware belonging to someone else, as the
risk of some lower level of infrastructure being compromised is too
great for a key that is expected to be in use for years.
Anyway, from what I understand, urandom is fine forever as long as
the PRNG has been initially seeded with entropy. "Initially" being
"on first boot", since every modern Linux distro saves the PRNG
state to a file on shut down and loads it back on boot.
I don't know what Docker would be doing here. Presumably the
container just has (u)random devices that make use of the host's
PRNG so as long as Docker isn't doing this shortly after the first
ever boot of the host then it would be fine. But not fine soon after
first boot, and sadly no way to tell if it is safe yet other than
first reading from /dev/random and then switching to urandom
afterwards.
From my reading of
http://www.2uo.de/myths-about-urandom/ the ideal
solution would be for gpg to use getrandom(2) for this.
I agree; according to https://lwn.net/Articles/605828/ it will block
(or return EAGAIN) during the problematic initial early first boot
period.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
29 Jan
29 Jan
6:33 p.m.
On 13 March 2016 at 01:56, Andy Smith <andy(a)bitfolk.com> wrote:
To provide the Entropy service we use hardware entropy generators,
currently exclusively a pair of EntropyKeys manufactured by a UK
company called Simtec Electronics Ltd.
Apologies for resurrecting this thread, but I do occasionally look at
devices like this and wondered if you had kept up-to-date with the
market for them and, if so, had any thoughts on any of the
alternatives?
30 Jan
30 Jan
1:18 a.m.
New subject: Dear Entropy service users, what software uses /dev/random?
Hi Gerald,
On Mon, Jan 29, 2018 at 06:33:30PM +0000, Gerald Davies wrote:
Apologies for resurrecting this thread, but I do
occasionally look at
devices like this and wondered if you had kept up-to-date with the
market for them and, if so, had any thoughts on any of the
alternatives?
I haven't really kept up, I'm afraid.
At the point where the manufacturer of the EntropyKey folded, I had
two of them in production and one spare. It seems clear that even a
single EntropyKey would keep up with demand, so I'm not overly
concerned.
Nevertheless I did pick up a couple of OneRNG devices a few years
ago:
<http://onerng.info/>
Aside from verifying that they work I haven't put any into
production use and I'm not sure I ever will because probably the
EntropyKeys will last a very long time.
I was convinced by <https://www.2uo.de/myths-about-urandom/> that
/dev/urandom is good enough on modern Linux and any software that
demands /dev/random is deficient. The purpose of this thread was to
see if people were aware of such stubborn software was in wide use.
It got a few replies but it seems that insistence on /dev/random is
rightly fading away so my interest in hardware RNG has faded.
In my opinion BitFolk's wiki article on the Entropy service needs
rewriting to stress that software should be using urandom.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
2526
days inactive
3214
days old
13 comments
8 participants
participants (8)
-
Adam Spiers -
Alex Sergeyev -
Andy Smith -
Dominic Cleal -
Gerald Davies -
Graham Bleach -
Robert Gauld -
Roger Light