24 May
2024
24 May
'24
11:40 p.m.
I will do some more investigation of this failure mode but in light of
doing away with bonding being the direction we are already going, I don't
think I want to alter how bonding is done on what will soon be a legacy
setup.
Shouldn't this failure mode have been caught by LACPDUs?
--
Maria Blackmore
25 May
25 May
9:22 a.m.
New subject: 2024-05-23 ~20:25Z – ~20:50Z: Networking issue on host "jack"
Hi Maria,
On Sat, May 25, 2024 at 12:40:31AM +0100, Maria Blackmore via BitFolk Users wrote:
Hmm, I'm not sure.
Here's the configuration:
auto bond0
iface bond0 inet static
bond_mode 1
slaves eth0 eth1
bond_miimon 100
$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v5.10.0-0.deb10.16-amd64
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth1
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 0c:c4:7a:76:c6:8c
Slave queue ID: 0
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: 0c:c4:7a:76:c6:8d
Slave queue ID: 0
But, during the incident the MII Status remained up (and "ip link"
showed as up). With tcpdump I could see LACP traffic arriving but no
traffic at all going out of eth1.
This is an old post but I can't see any improvements (unless you
count BPF…) for finding out what is going on with LACP:
https://serverfault.com/a/823865/71071
But also, as far as I understand, mode 1 (active-backup) doesn't
require any LACP support in the switches so maybe doesn't care about
LACPDU and only uses link state?
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
I will do some more investigation of this failure mode but in light of
doing away with bonding being the direction we are already going, I don't
think I want to alter how bonding is done on what will soon be a legacy
setup.
Shouldn't this failure mode have been caught by LACPDUs? 
10:41 a.m.
New subject: 2024-05-23 ~20:25Z – ~20:50Z: Networking issue on host "jack"
On Sat 25 May 2024 09:22:08 GMT, Andy Smith via BitFolk Users wrote:
Hi Maria,
On Sat, May 25, 2024 at 12:40:31AM +0100, Maria Blackmore via BitFolk Users wrote:
Hmm, I'm not sure.
I’m pretty sure it would have, it’s the whole purpose of active checks.
nas-hp ~ # cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v6.6.16-gentoo-dist
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0
802.3ad info
LACP active: on
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 66:44:af:1d:d5:9b
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 2
Actor Key: 9
Partner Key: 1076
Partner Mac Address: c0:4a:00:e4:09:a0
Slave Interface: eno1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:1d:48:c7:a3:08
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: 66:44:af:1d:d5:9b
port key: 9
port priority: 255
port number: 1
port state: 61
details partner lacp pdu:
system priority: 32768
system mac address: c0:4a:00:e4:09:a0
oper key: 1076
port priority: 32768
port number: 2
port state: 60
Slave Interface: eno2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:1d:48:c7:a3:09
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: 66:44:af:1d:d5:9b
port key: 9
port priority: 255
port number: 2
port state: 61
details partner lacp pdu:
system priority: 32768
system mac address: c0:4a:00:e4:09:a0
oper key: 1076
port priority: 32768
port number: 1
port state: 60
--
Alarig
I will do some more investigation of this failure mode but in light of
doing away with bonding being the direction we are already going, I don't
think I want to alter how bonding is done on what will soon be a legacy
setup.
Shouldn't this failure mode have been caught by LACPDUs?
11:03 a.m.
New subject: 2024-05-23 ~20:25Z – ~20:50Z: Networking issue on host "jack"
Hello,
On Sat, May 25, 2024 at 12:41:16PM +0200, Alarig Le Lay via BitFolk Users wrote:
On Sat 25 May 2024 09:22:08 GMT, Andy Smith via
BitFolk Users wrote:
Right, but what I'm saying is, I'm not using a link aggregated mode
am I so I'm not sure bond mode 1 is even meant to check this.
I can change the mode but I really want to get rid of the bond…
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
On Sat, May 25, 2024 at 12:40:31AM +0100, Maria
Blackmore via BitFolk Users wrote:
I’m pretty sure it would have, it’s the whole purpose of active checks. Shouldn't this failure mode have been caught
by LACPDUs?
Hmm, I'm not sure.
12:06 p.m.
New subject: 2024-05-23 ~20:25Z – ~20:50Z: Networking issue on host "jack"
On Sat 25 May 2024 11:03:26 GMT, Andy Smith via BitFolk Users wrote:
I can change the mode but I really want to get rid of
the bond…
Two BGP sessions are indeed easier to manage that bondings :)
--
Alarig
11:39 a.m.
New subject: 2024-05-23 ~20:25Z – ~20:50Z: Networking issue on host "jack"
Hi Andy
On Sat, 25 May 2024, 10:22 Andy Smith via BitFolk Users, <
users(a)mailman.bitfolk.com> wrote:
require any LACP support in the switches so
maybe doesn't care about
> LACPDU and only uses link state?
This isn't using LACP, so it doesn't benefit from any of the monitoring and
control that LACP brings, and is essentially "blind" and as you say relies
purely on the link state. If this was configured using mode 4, with fast
LACP, then the LACPDUs are exchanged once every second, allowing rapid
detection of this and other issues where the interface remains up but
doesn't forward traffic. This is far faster than relying on BGP for link
detection, and mode 4 is the only configuration I trust for production use,
since failover occurs on the order of seconds rather than a minute or more.
Thanks
Maria
Here's the configuration:
auto bond0
iface bond0 inet static
bond_mode 1
slaves eth0 eth1
bond_miimon 100
You're correct; bond mode 1 doesn't use LACP
But also, as far as I understand, mode 1 (active-backup) doesn't
12:08 p.m.
New subject: 2024-05-23 ~20:25Z – ~20:50Z: Networking issue on host "jack"
On Sat 25 May 2024 12:39:19 GMT, Maria Blackmore via BitFolk Users wrote:
doesn't forward traffic. This is far faster than
relying on BGP for
link detection, and mode 4 is the only configuration I trust for
production use, since failover occurs on the order of seconds rather
than a minute or more.
I also had issues with active-passive links in production, so I always
do active-active now.
--
Alarig
146
days inactive
147
days old
6 comments
3 participants
participants (3)
-
Alarig Le Lay -
Andy Smith -
Maria Blackmore