Hi,
On Fri, Jul 31, 2009 at 02:38:00AM +0000, Andy Smith wrote:
Hi,
On Fri, Jul 31, 2009 at 01:57:13AM +0000, Andy Smith wrote:
Hi,
A couple of hours ago the server
kwak.bitfolk.com stopped being able
to start new VPSes or do networking to itself and an emergency
reboot is required, which is happening now. Apologies for the
disruption.
As of about 0220Z all VPSes on kwak are either up or still in the
process of booting. I will be finishing checking things over and
will post more info later in the morning.
This problem appears to be the same thing that has affected obstler
a couple of times: someone shuts down a VPS and then the host seems
to get stuck all the while sending "unregister_netdevice: waiting
for <interface> to become free. Usage count = 1" to the console.
New network traffic to the host itself is disrupted and it stops
being possible to start new VPSes.
It seems to be related to IPv6, a bug where deleting an interface
(such as when a VPS is shut down) while it still has IPv6 addresses
active can sometimes cause a reference count in the kernel to not be
decremented. It's a race condition and happens very rarely.
It's unclear whether it's actually been fixed in the latest kernels
(all the search results I find just say "does this still happen?")
and as very few people actually use IPv6, I think the best thing to
do is change configuration to attempt to avoid this situation. I've
already don't it on obstler, the machine which originally started
being affected by this.
So on Monday I'm going to disable radvd everywhere and not have any
global IPv6 addresses on the interfaces on the BitFolk host side. As a
consequence auto-configured IPv6 will stop working. If you want
IPv6 to continue working you will need to set it up statically,
which generally involves manually adding addresses inside your /64
and adding a default IPv6 route via fe80::fcff:ffff:feff:ffff/64.
If you want to keep working IPv6 but don't know what to do, please
respond either on or off-list and I will try to help.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting