I have 3 servers with 2 port Melanox ConnectX-4 Lx 25Gbit adapters and two Mikrotok CRS504-4XQ-IN. Port 1 of all adapters connected to Switch 1, port 2 connected to switch 2. Switch 1 and Switch 2 are not connected to each other.
Ubuntu 24 is installed on all servers.
Ideal solution: all switches and links online - all the servers have 50Gbit connection between each other.
If one link to any server becomes offline - its connection becomes 25Gbit, two others have 50Gbit. If one switch goes down - all servers will have 25Gbit.
Suitable solution: all switches and links online - all the servers have 25Gbit connection between each other.
If one link to any server becomes offline, it works via the second port. If one switch goes down - all servers will have a network connection via their second port.
What I have now. Bonding is configured on each server in active-backup mode. Port1 on all bonding interfaces is set to primary. I have 25Gbit connection between the servers. If one of the switches goes down - it works. But if one active link goes down - the server becomes offline. If I ping any other server - I see incoming traffic on the inactive slave interface (via tcpdump), but bonding interface ignores it.
I tried to connect switches via ether1 port and set up MLAG, but it requires a dedicated port.
What other options could there be?