On 5 Nov 2019, at 15:13, Andy Smith <andy(a)bitfolk.com> wrote:
>
> On Tue, Nov 05, 2019 at 03:02:39PM +0000, Chris Smith wrote:
>> I’m also a mild -1 for much the same reasons already mentioned, but out of interest do you know why it doesn’t work great for those people? I can understand that people might think a mailing list a bit old fashioned, but I’m a bit baffled at how someone could try to use one and fail in the attempt.
>
> Well, did you intend to send this to me alone? If not, that's one
> example of how someone can fail. Perhaps you did intend to send it
> only to me, but 4 different people this past week have done that and
> did not intend to send it just to me, so that's one thing.
Ha! What a brilliant example. I only wish it was deliberate. Yes, I did intend to reply to the list, and I’ve corrected that now so that everyone else can have a quick giggle at my expense.
> Most of the replies I see on the list are more like forum responses
> not emails. I find them hard to read as emails. The top posting,
> terribly broken quoting. I'm seeing less and less point in expecting
> clear emails from people who don't see value in sending clear emails.
I’m with you on that. My mail client has no option to turn off top posting so I have to go to the effort of formatting manually, and I only go to that effort for technical mailing lists because those are the only people who care.
> I have used email since 1994, and FidoNet for a couple of years
> prior to that. But I wonder how far the tide is turning.
Sadly I think the only thing keeping email alive is the fact that almost every non-email platform requires an email address for user identification and authentication. There’s a big shift now for linking to Facebook and Google etc. for doing that, but those are obviously very specific and platform dependent mechanisms, and even those require an email address. Sooner or later there will be a common, platform-agnostic mechanism that will become dominant, and at that point I suspect email die.
Chris
—
Chris Smith <space.dandy(a)icloud.com>
Hi,
I've added a warning to the Xen Shell "install" command that new
installs of 32-bit guests are now deprecated, and pointing you to:
https://tools.bitfolk.com/wiki/64-bit_guests
Those of you with existing 32-bit guests, which I know is many of
you, should be okay for a few years yet. Even after the Xen
hypervisor officially ditches support for 32-bit guests we will
ensure they are still bootable. But I would encourage you to choose
64-bit at your next install or operating system upgrade for an
easier life.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hello,
I've been noticing SYN flooding¹ on various TCP services going on for
weeks now, not just against BitFolk hosts but against all hosts I
have access to, worldwide. Others have noticed it too.
Today I also noticed that some customer DNS and HTTP servers were
occasionally taking a long time (10+ seconds) to respond and this was
generating an alert from our monitoring, which would then clear, and
then trigger again.
I've done a tcpdump against two such customer services so far and I
see stuff like this:
$ sudo tcpdump -vpni v-[redacted] 'tcp and dst port 53'
00:53:33.622739 IP (tos 0x8, ttl 79, id 47798, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.150.52945 > 85.119.[redacted].53: Flags [S], cksum 0x2dca (correct), seq 3396968370, win 29200, length 0
00:53:34.451820 IP (tos 0x8, ttl 75, id 64961, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.137.42540 > 85.119.[redacted].53: Flags [S], cksum 0x74cf (correct), seq 1691936512, win 29200, length 0
00:53:34.636166 IP (tos 0x0, ttl 62, id 38397, offset 0, flags [DF], proto TCP (6), length 60)
85.119.80.238.42534 > 85.119.[redacted].53: Flags [S], cksum 0x144a (correct), seq 3351471429, win 29200, options [mss 1460,sackOK,TS val 2923360440 ecr 0,nop,wscale 7],
length 0
00:53:34.792156 IP (tos 0x8, ttl 55, id 51802, offset 0, flags [DF], proto TCP (6), length 40)
185.90.118.36.58121 > 85.119.[redacted].53: Flags [S], cksum 0x5b25 (correct), seq 3802350695, win 29200, length 0
00:53:35.659295 IP (tos 0x0, ttl 62, id 38398, offset 0, flags [DF], proto TCP (6), length 60)
85.119.80.238.42534 > 85.119.[redacted].53: Flags [S], cksum 0x134a (correct), seq 3351471429, win 29200, options [mss 1460,sackOK,TS val 2923360696 ecr 0,nop,wscale 7],
length 0
00:53:37.247124 IP (tos 0x8, ttl 68, id 41159, offset 0, flags [DF], proto TCP (6), length 40)
185.90.117.12.41829 > 85.119.[redacted].53: Flags [S], cksum 0xf1bb (correct), seq 805085492, win 29200, length 0
00:53:37.675366 IP (tos 0x0, ttl 62, id 38399, offset 0, flags [DF], proto TCP (6), length 60)
85.119.80.238.42534 > 85.119.[redacted].53: Flags [S], cksum 0x1152 (correct), seq 3351471429, win 29200, options [mss 1460,sackOK,TS val 2923361200 ecr 0,nop,wscale 7],
length 0
00:53:37.756095 IP (tos 0x8, ttl 78, id 5747, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.142.65402 > 85.119.[redacted].53: Flags [S], cksum 0xf3b5 (correct), seq 2604980314, win 29200, length 0
00:53:37.760805 IP (tos 0x8, ttl 73, id 5112, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.61.62352 > 85.119.[redacted].53: Flags [S], cksum 0x5be0 (correct), seq 636416449, win 29200, length 0
00:53:37.860091 IP (tos 0x8, ttl 55, id 53714, offset 0, flags [DF], proto TCP (6), length 40)
185.90.118.42.47449 > 85.119.[redacted].53: Flags [S], cksum 0x3f4d (correct), seq 2136599859, win 29200, length 0
00:53:39.582555 IP (tos 0x8, ttl 73, id 11769, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.251.54065 > 85.119.[redacted].53: Flags [S], cksum 0xfd05 (correct), seq 2936203048, win 29200, length 0
00:53:40.357845 IP (tos 0x8, ttl 66, id 24626, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.143.49296 > 85.119.[redacted].53: Flags [S], cksum 0x6e13 (correct), seq 3886043274, win 29200, length 0
00:53:40.369750 IP (tos 0x8, ttl 62, id 39030, offset 0, flags [DF], proto TCP (6), length 40)
185.90.118.213.43688 > 85.119.[redacted].53: Flags [S], cksum 0x2a92 (correct), seq 231047561, win 29200, length 0
00:53:41.477175 IP (tos 0x8, ttl 53, id 47222, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.92.32796 > 85.119.[redacted].53: Flags [S], cksum 0x2417 (correct), seq 2911442245, win 29200, length 0
00:53:41.562028 IP (tos 0x8, ttl 54, id 46361, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.136.63986 > 85.119.[redacted].53: Flags [S], cksum 0x6088 (correct), seq 3811125553, win 29200, length 0
00:53:41.839400 IP (tos 0x0, ttl 62, id 38400, offset 0, flags [DF], proto TCP (6), length 60)
85.119.80.238.42534 > 85.119.[redacted].53: Flags [S], cksum 0x0d42 (correct), seq 3351471429, win 29200, options [mss 1460,sackOK,TS val 2923362240 ecr 0,nop,wscale 7],
length 0
00:53:41.839884 IP (tos 0x0, ttl 62, id 38401, offset 0, flags [DF], proto TCP (6), length 52)
85.119.80.238.42534 > 85.119.[redacted].53: Flags [.], cksum 0x1d5b (correct), ack 3745901387, win 229, options [nop,nop,TS val 2923362241 ecr 41403076], length 0
00:53:41.839934 IP (tos 0x0, ttl 62, id 38402, offset 0, flags [DF], proto TCP (6), length 52)
85.119.80.238.42534 > 85.119.[redacted].53: Flags [F.], cksum 0x1d5a (correct), seq 0, ack 1, win 229, options [nop,nop,TS val 2923362241 ecr 41403076], length 0
00:53:41.842751 IP (tos 0x0, ttl 62, id 0, offset 0, flags [DF], proto TCP (6), length 52)
85.119.80.238.42534 > 85.119.[redacted].53: Flags [.], cksum 0x1d58 (correct), ack 2, win 229, options [nop,nop,TS val 2923362241 ecr 41403077], length 0
00:53:41.938989 IP (tos 0x8, ttl 56, id 57551, offset 0, flags [DF], proto TCP (6), length 40)
185.90.118.99.40999 > 85.119.[redacted].53: Flags [S], cksum 0xeefb (correct), seq 137547173, win 29200, length 0
00:53:42.234411 IP (tos 0x8, ttl 60, id 59687, offset 0, flags [DF], proto TCP (6), length 40)
185.90.117.200.49353 > 85.119.[redacted].53: Flags [S], cksum 0x875f (correct), seq 216731778, win 29200, length 0
00:53:43.192600 IP (tos 0x8, ttl 79, id 8563, offset 0, flags [DF], proto TCP (6), length 40)
185.90.116.208.47026 > 85.119.[redacted].53: Flags [S], cksum 0xd35e (correct), seq 1815703363, win 29200, length 0
00:53:43.355109 IP (tos 0x8, ttl 80, id 37254, offset 0, flags [DF], proto TCP (6), length 40)
185.90.118.237.43294 > 85.119.[redacted].53: Flags [S], cksum 0xd9af (correct), seq 848473872, win 29200, length 0
So I'm seeing a lot of TCP packets with the SYN flag set ("[S]")
coming from hosts all over 185.90.116.0/22, but they never actually
establish a connection and exchange data. I think that's a SYN
flood.
If you are seeing flapping alerts for your TCP-based services, can
you have a look to see if same is happening to you?
If using tcpdump you'd want something like:
# tcpdump -vpni eth0 'tcp and dst port 53'
(For looking at TCP traffic to your DNS server)
The source IPs are probably going to change rapidly, so I'm not sure
what configuration if any is needed in the OS or DNS server to
prevent this from starving out legit connections, e.g. the ones from
our monitoring!
Against BitFolk's own infrastructure I do see this too, but I'm
maybe not using the same software as you because it's not starving
out legit connections. So if you are affected by this and you do
find a way to block this or increase your limits or whatever, I'd
appreciate you letting me know what you're running and what you
needed to do.
Cheers,
Andy
¹ https://en.wikipedia.org/wiki/SYN_flood
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
Some security issues have been found in the hypervisor software we
use, which we have to fix as they can theoretically allow privilege
escalation.
They are under embargo until Thursday 31 October, so we will most
likely do the work in the early hours of the morning (UK time) on
29, 30, and 31 October.
As usual this will entail a clean shutdown of your guest and then a
boot again 20–30 minutes later after the patching is done. Some time
next week an email will go out telling you of the two hour window in
which this work will take place for each of your VMs.
If the assigned window is unacceptable to you, we can most likely
move your VM to an already-patched host at a time of your choosing
before 31 October. When the direct email comes to let you know of
your maintenance window, if it's not acceptable then you can reply
to it to open a support ticket and we will work it out.
As usual, if you have opted in to suspend/restore then your guest
will be suspended to disk and restored again instead of shutdown and
booted. More info on that:
https://tools.bitfolk.com/wiki/Suspend_and_restore
For our own maintenance work we like to give more than 2 weeks of
notice. Unfortunate when dealing with security issues there is an
agreed embargo process and notice periods are much shorter. It is
preferable that there is ~2 weeks of notice rather than a "0-day"
exploit being unleashed.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi users!
I've been a happy Bitfolk customer for some time and now I'm looking at
adding some more storage to my VPS.
In the past I've always taken the 5GiB packages. This time I see that there
is now Archival Storage in 50GiB packages.
I've read the docs at https://tools.bitfolk.com/wiki/Archive_storage but am
writing here to see if anyone has any practical experience with these
packages.
Whatever I choose, I'll keep all my existing packages: I'm just looking to
add new ones.
Having been a Bitfolk customer for so long, am I correct in assuming that
the disks that power these packages are similar technology to what my
entire VPS ran on a few years ago?
I'm definitely in two minds. I want the storage for IMAP... and as everyone
knows, IMAP and RDBMS are all about the number of spindles. ...but if I
only have a small handful of users (albeit ones who all have their phones
connected almost all the time), would everything stick nicely in the page
cache and spare me noticing the performance of the underlying storage?
So, if anyone has any direct experience reports of the performance of
Archival Storage, especially under an IMAP workload, I'd love to hear them!
Thanks!
(If I take the Archival Storage, I'll move my own mailbox first and try it
out for a while. If I take the 5GiB package I'll just add it to the
existing device and grow the volume.)
Best wishes,
@ndy
--
andyjpb(a)ashurst.eu.org
http://www.ashurst.eu.org/
0x7EBA75FF
Hi,
A few customers have been testing this for a while now, and it's
been a while since the last issues were addressed, so now seems like
a good time to announce it.
We're going to be retiring our Cacti instance¹ in favour of the new
setup which can be found at:
https://tools.bitfolk.com/grafana/
You all already have access to it.
Those who are familiar with Prometheus and Grafana may be a little
disappointed: this is not intended to be a full hosted instance,
only a fairly locked-down replacement for what Cacti provides. I'm
satisfied that it goes beyond the functionality and usability of
Cacti, but it isn't like having your own setup and isn't intended to
be.
Everyone has a default dashboard exposing graphs similar to those
provided by Cacti, plus a few more besides.
The offer was always open for more of your metrics to be graphed by
Cacti, but as of today only one customer was making use of that. The
offer is still open for us to graph extra metrics from you if you
wish. To do that you'll first need to install Node Exporter² and
then send a support ticket. You'll then get an additional
dashboard that looks like a bit like this:
https://tools.bitfolk.com/grafana/dashboard/snapshot/fysbHKJGqJm3Fq6KmtqlRJ…
Over the next week or two a wiki article for our Grafana will appear
and any references to Cacti on our web sites and docs will start to
disappear, except for a pointer to historical Cacti graphs. Update
of Cacti graphs is going to be disabled very soon.
Feedback on the service is still welcome of course, though the
general approach is by now pretty much decided.
Cheers,
Andy
¹ https://tools.bitfolk.com/cacti/
² Available as your usual kind of single Go binary from here:
https://github.com/prometheus/node_exporter
but also available in modern Debian (at least) as a package.
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
I've been working towards replacing our aging and creaky Cacti¹
installation with something more modern, based on Prometheus and
Grafana.
I think it's nearing the stage of being able to take over from Cacti
now, so I would like some volunteers to have a go at using it, give
feedback etc.
If you would be willing to do so, please drop me an email off-list
and I'll enable you to log in to it (with your usual BitFolk
credentials). Cacti will continue running in the meantime so there
is nothing to lose, only some time spent using it.
The goal here is not to offer a full hosted Grafana/Prometheus
setup, as that would be rather complicated and there are companies
that already do that as their entire paid service offering. I'm just
trying to replace the functionality of our Cacti, which for
customers basically amounts only to bandwidth and CPU graphs.
We can do a little better than this — in particular, block device
graphs have been pretty easy to add — but that's the sort of scope I
am looking at for right now.
Once we're happy with it, Cacti is going to stop gathering any
further measurements.
After you've had a play for a bit, feedback to this thread is
welcome, or to me personally if you'd rather.
Cheers,
Andy
¹ https://tools.bitfolk.com/cacti/
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
Another serious bug has been found in Exim, which is installed by
default on Debian and some other Linux distributions:
https://seclists.org/oss-sec/2019/q3/253
The impact is remote execution as an unprivileged user, although
it cannot be ruled out that there might be other routes to the same
code running in a privileged context.
If your distribution is still under security support then I expect
they will push out new packages in the next few days.
If not then you will need to upgrade it or rebuild the package. It's
quite a simple fix.
There's been no embargo this time, so attacks could be out in the
wild already.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
TL;DR: Read this to learn how to install CentOS 8
https://tools.bitfolk.com/wiki/Installing_CentOS_8
Unabridged edition:
Given that CentOS 8 was released a few days ago I had a look at
adding its installer.
Unfortunately it seems that CentOS 8 has dropped kernel support for
PV-mode Xen guests, which are the only type of guests that BitFolk
currently supports. It is therefore not possible to use the official
CentOS installer or core kernel package at the moment.
We are in the process of moving to PVH mode¹ guests, but that's not
ready yet. It all works; the main difficulty now is supporting both
modes without it being a terribly confusing user experience.
In the meantime, it is pretty simple to install CentOS 8 from
another Linux. This could be any distribution including an earlier
version of CentOS, though I would suggest that doing it from the
BitFolk Rescue VM² makes most sense as it's always available and
runs from RAM.
As the core kernel package of CentOS 8 also does not support PV mode
guests, it is also necessary to enable ELRepo³ and install the
kernel-ml package.
Here is a transcript of me installing CentOS 8 from scratch by this
method with full explanation of every step.
https://tools.bitfolk.com/wiki/Installing_CentOS_8
Don't be put off by the massive amount of text here; the vast majority
of it is command output which I have only included so you know what
to expect.
The only issue I have found with this method are some odd 1–2 minute
pauses around creating initramfs / bootloader config. This only
happens inside the install chroot and is probably something trying
to probe and timing out. It appears to be harmless, just irritating.
If you know what that is about or have any other improvements to
make, please do edit the page⁴; it is a wiki.
Cheers,
Andy
¹ https://wiki.xen.org/wiki/Xen_Project_Software_Overview#PVH_.28x86.29
² https://tools.bitfolk.com/wiki/Rescue
³ https://elrepo.org/tiki/kernel-ml
⁴ I would suggest refraining from adding purely optional things that
are a matter of taste though, as otherwise the page will become
incredibly long and opinionated.
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
