2020-12-27 ~00:00 – ~00:45 internal packet loss and alerts regarding "clockwork" and "limoncello"

Hi, As of about 0000Z we started receiving alerts of packet loss and began investigation. It was found to be an internal issue between hosts "clockwork" and "limoncello" only. That is, everything on both hosts was reachable from outside our network and also from inside as long as it wasn't between those two hosts. As there is a monitoring node on "limoncello", a number of alerts were sent out regarding customer services on "clockwork" that it considered to be down, but they weren't actually down - unless you happened to be hosted on "limoncello", anyway, and vice versa. I tracked the issue to one of the two bonded switch ports for "clockwork"; bringing that interface down and up again appears to have cleared it. That happened at about 0045Z. If the problem reoccurs we can down the interface and have it run on one interface until the port or switch can be changed. If the problem is actually in the NIC of the server itself things will be more tricky, but we'll cross that bridge if we come to it. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting