Hi folks,
Short version:
Two of our servers appear to be subject to a now fixed kernel bug
affecting IPv6, and require a reboot for kernel upgrade. Host
bellini.bitfolk.com will be rebooted on Monday 20th February at
2200Z.
Provided that does fix the problem, host
president.bitfolk.com will
be similarly rebooted the following day, Tuesday 21st February also
at 2200Z.
Longer version:
While investigating some recent reports of poor IPv6 performance, it
seems that both
bellini.bitfolk.com and
president.bitfolk.com are
affected by a bug in the igb Intel gigabit Ethernet driver as
described here:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630730
The symptoms are very poor IPv6 performance in the region of a
maximum of 100kilobit/sec. On doing a tcpdump you will see packets
of length greater than 1500 bytes, followed by an ICMP6 "packet too
big" message coming from our server, and then a retransmit.
These hosts have been up for 138 days (bellini) and 223 days
(president), and unfortunately neither we nor any customers noticed
any problems until recently. On casual inspection IPv6 works. It's
only noticeable when trying to do a larger data transfer.
Since the current impact is so low, I am not going to rush to reboot
these hosts. I would rather give plenty of notice to those who need
it.
When it's time for the reboot we will shut down all VPSes on these
servers cleanly, reboot the machine and then begin booting them up
again. Downtime is expected to be in the region of 15 minutes. If
you are in any doubt as to whether your VPS starts cleanly with all
required services running, you should test this ahead of time.
I am fairly confident that the problems observed are caused by that
bug and therefore that a kernel upgrade will fix it, but
unfortunately we do not have any other hardware that uses the igb
driver. If doing the upgrade on bellini does not resolve the issue
then we will have to consider our options.
In the mean time, if your VPS is hosted on bellini or president then
you may wish to set your VPS to prefer IPv4 DNS results ahead of
IPv6 results:
https://tools.bitfolk.com/wiki/IPv6#Preferring_IPv4_over_IPv6
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting