Community discussions

MikroTik App
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Sat May 06, 2023 1:06 pm

Hello, I have a WORKING configuration in MLAG with 4 switches CRS326-24S+2Q+ and ROS 7.6 which I will call SW1 - SW2 - SW3 - SW4

SW1/SW2 are in MLAG with each other via the 40GBps port
SW3/SW4 are in MLAG with each other via the 40GBps port

SW1 does LACP with SW3/SW4 via two SFP ports, one going to SW3 and the other going to SW4

Same thing goes for SW2

In practice, the two pairs of switches do MLAG and then BONDING with the corresponding ones using MLAG.

The problem:

With ROS 7.9 nothing works anymore, the MAC Addresses on the ARP table continue to switch between the MLAG port and the LACP one driving the network completely crazy.

So I downgraded to 7.7 which I see has several fixes on BRIDGE/RSTP etc.. The problem also occurs with 7.7 so I think the malfunction starts from this version..

As soon as I downgrade to 7.6 the network starts working fine..

I am attaching the network diagram: Image
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Sat May 06, 2023 1:40 pm

#[SUP-115519]
 
colonelbeefies
just joined
Posts: 4
Joined: Mon Jul 13, 2020 1:40 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Wed May 10, 2023 1:45 pm

Thanks for sharing! I started with 7.9 and nothing was working. I moved to 7.6 and everything is working. Have you heard anything from support?
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Wed May 10, 2023 2:57 pm

Then you also confirm the problem.. No, support never replied to the ticket
 
markdutton
newbie
Posts: 47
Joined: Fri Sep 24, 2010 4:59 am

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Wed May 10, 2023 4:24 pm

I have the same issue. Very annoying on a mission critical network when they give you 30 minutes downtime.
I noticed with 7.9, when I did a show lacp neighbour on a connected Cisco switch, it was showing the port address of one of the MTs for both neighbours, not the bond address. When I downgraded to 7.6 it showed the bond address of one of the MTs. The MT bond interfaces have different macs, so I assume the one advertised is effectively virtualised across both MTs?
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Wed May 10, 2023 4:33 pm

The problem starts from v7.7, if you notice it contains many changelogs related to the Bridge:

*) bridge - added support for static MDB entries;
*) bridge - disallow port-controller while the bridge has MSTP enabled;
*) bridge - fixed "edge=yes" setting for MSTP;
*) bridge - fixed MSTP compatibility with STP;
*) bridge - fixed R/M/STP bridge identifier on protocol-mode change;
*) bridge - fixed RSTP BCP with bridged PPP interfaces;
*) bridge - fixed STP blocking state on port-controller;
*) bridge - fixed host moving with fast-path;
*) bridge - fixed incorrect root port blocking for MSTP;
*) bridge - fixed master port conversion;
*) bridge - fixed mst-override port priority for MSTP;
*) bridge - fixed port priority for STP and RSTP;
*) bridge - improved port-controller system stability;
*) bridge - improved system stability when using MSTP and many VLAN mappings;
*) bridge - removed "age" monitoring property from the host table;

Something broke MLAG/Bridge...

Also open ticket to MT sent email to support@mikrotik.com

They haven't replied to my ticket dated 05/05/2023 yet
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Mon May 15, 2023 3:09 pm

I have performed extensive tests in the laboratory and I confirm that if the configuration is MLAG on 2 switches everything works correctly even on version 7.9 (7.7 and 7.8)

If MLAG is done between 4 switches (as per schema) the functioning is compromised from version 7.7 (including 7.8 and 7.9)

Since the 7.7 changelog there are several changes to the bridge
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Thu Jun 08, 2023 2:25 pm

Hi, Mikrotik Support has reproduced and confirmed the bug:

Hello,
Thank you for the report!
We have managed to reproduce the issue locally in our labs and look forward to fixing it on upcoming RouterOS versions, unfortunately, I cannot provide a release date now.
Unfortunately, I cannot suggest any known workarounds.
Best regards,
Edgars P.
 
toto4ds
just joined
Posts: 13
Joined: Fri Dec 03, 2021 10:39 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Thu Jun 08, 2023 7:24 pm

Good news, everyone!
 
wiseroute
Member
Member
Posts: 352
Joined: Sun Feb 05, 2023 11:06 am

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Fri Jun 09, 2023 5:16 am

good morning folks,

@ connectlife

i haven't lab your diagram yet - but just having some thought from basic stp point of view.

from your diagram- let us just view them in basic stp operation minus the lacp/mlag operation - each sw1, 2, 3 and 4 ---> from those 3 interface each on the switch, basic stp will only allow each of the switch to have 1 port activated.

am i correct?

ok, now we will have a look at lacp/bonding/mlag point of view - which is their operation state need to *modify* in certain way from that basic stp operation ---> so that their bonding ports could become activated at the same time ie. rr, master-slave etc.

from above assumption, did you see any port seemed to be still in disabled state in your lab/diagram? 🤔

if yes, then I would say the problem might be in m/r/stp software?

if not, then maybe we need to adapt/adjust this problem to a working MT diagram?

just a thought 🤔
 
User avatar
Repo
just joined
Posts: 1
Joined: Tue Jun 13, 2023 10:49 am
Location: Madrid

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Tue Jun 13, 2023 11:16 am

good morning everyone,

Good thing they confirm it - I was going crazy.
I will install version 7.6 to see if it is stable on my network and in case of new instabilities - I will unmount MLAG until it is stable. I already knew I had to test it in my lab first... but "rush" is what they have

And I will wait for them to confirm the resolution of the problem in later versions - before updating ANYTHING
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Sun Jun 18, 2023 10:00 pm

good morning everyone,

Good thing they confirm it - I was going crazy.
I will install version 7.6 to see if it is stable on my network and in case of new instabilities - I will unmount MLAG until it is stable. I already knew I had to test it in my lab first... but "rush" is what they have

And I will wait for them to confirm the resolution of the problem in later versions - before updating ANYTHING
Hi on version 7.10 it should be fixed
 
DmitriiS
just joined
Posts: 1
Joined: Wed Jun 21, 2023 3:21 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Wed Jun 21, 2023 3:28 pm

Hi everyone!
We have configured the similar HA setup - two pairs of MLAGGed switches.
But we connected them with ONE LACP bond of 4 lines. Generally it works good, guests connected to switches works as expected.
But when we try to connect to switches manage IP across this 4-lines bond - connection lags/freezes (but not hanging completely).
As we can see - packets jumping to the destination host from ICPP interlink to bond and back again constantly.

Any ideas what it could be/how it could be fixed?
 
BSCC
just joined
Posts: 3
Joined: Thu May 25, 2023 10:06 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Wed Jun 21, 2023 9:48 pm

Is MLAG working on 7.10 for you?
 
toto4ds
just joined
Posts: 13
Joined: Fri Dec 03, 2021 10:39 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Sat Jun 24, 2023 12:53 pm

Is MLAG working on 7.10 for you?
viewtopic.php?p=1008334&hilit=mlag#p1008334
 
connectlife
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 99
Joined: Tue Sep 01, 2020 10:20 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Sun Jun 25, 2023 10:20 pm

hi Dimitri, can you share the configuration of your switches? in our configuration we do not have the problem you encountered
 
Kalata
just joined
Posts: 2
Joined: Sat Jul 01, 2023 11:09 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Sat Jul 01, 2023 11:20 pm

hi Dimitri, can you share the configuration of your switches? in our configuration we do not have the problem you encountered
Mine not working too:

/interface bonding
add mlag-id=101 mode=802.3ad name=bond1 slaves=sfp-sfpplus23 transmit-hash-policy=layer-2-and-3

/interface bridge
add name=bridge priority=0x1000 vlan-filtering=yes

/interface bridge mlag
set bridge=bridge peer-port=qsfpplus1-1

/interface bridge port
add bridge=bridge interface=qsfpplus1-1 pvid=777
add bridge=bridge frame-types=admit-only-vlan-tagged interface=bond1

/interface bridge vlan
add bridge=bridge comment="10 - Internet" tagged=qsfpplus1-1,bridge,bond1 vlan-ids=10


> interface/bridge/mlag/monitor
status: connected
system-id: 48:A9:8A:B4:D9:C9
active-role: primary


] > interface/bonding/monitor bond1
mode: 802.3ad
active-ports: sfp-sfpplus23
inactive-ports:
lacp-system-id: 48:A9:8A:B4:D9:C9
lacp-system-priority: 65535
lacp-partner-system-id: 84:39:8F:40:9F:1F




factory-firmware: 7.8
current-firmware: 7.10
upgrade-firmware: 7.6 < ---------------- It was upgradet to latest version 7.10, but after that I downgrade it to 7.6
Last edited by Kalata on Sat Jul 01, 2023 11:34 pm, edited 3 times in total.
 
ernieball17
just joined
Posts: 16
Joined: Thu Jan 28, 2021 9:55 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Mon Jul 31, 2023 9:00 pm

I have performed extensive tests in the laboratory and I confirm that if the configuration is MLAG on 2 switches everything works correctly even on version 7.9 (7.7 and 7.8)

If MLAG is done between 4 switches (as per schema) the functioning is compromised from version 7.7 (including 7.8 and 7.9)

Since the 7.7 changelog there are several changes to the bridge
Hi connectlife, one quick question, after a reboot of one of the switches the traffic is normally balanced between them? Or just one is working with the traffic and the rebooted works as backup?

Regards
 
toto4ds
just joined
Posts: 13
Joined: Fri Dec 03, 2021 10:39 pm

Re: MLAG Bridge not work in ROS 7.7 - 7.8 - 7.9, OK in ROS 7.6

Mon Jul 31, 2023 9:22 pm

hi Dimitri, can you share the configuration of your switches? in our configuration we do not have the problem you encountered
Mine not working too:

/interface bonding
add mlag-id=101 mode=802.3ad name=bond1 slaves=sfp-sfpplus23 transmit-hash-policy=layer-2-and-3

/interface bridge
add name=bridge priority=0x1000 vlan-filtering=yes

/interface bridge mlag
set bridge=bridge peer-port=qsfpplus1-1

/interface bridge port
add bridge=bridge interface=qsfpplus1-1 pvid=777
add bridge=bridge frame-types=admit-only-vlan-tagged interface=bond1

/interface bridge vlan
add bridge=bridge comment="10 - Internet" tagged=qsfpplus1-1,bridge,bond1 vlan-ids=10


> interface/bridge/mlag/monitor
status: connected
system-id: 48:A9:8A:B4:D9:C9
active-role: primary


] > interface/bonding/monitor bond1
mode: 802.3ad
active-ports: sfp-sfpplus23
inactive-ports:
lacp-system-id: 48:A9:8A:B4:D9:C9
lacp-system-priority: 65535
lacp-partner-system-id: 84:39:8F:40:9F:1F




factory-firmware: 7.8
current-firmware: 7.10
upgrade-firmware: 7.6 < ---------------- It was upgradet to latest version 7.10, but after that I downgrade it to 7.6
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bonding.mlag slaves=qsfpplus1-1.mlag,qsfpplus2-1.mlag
add mlag-id=101 mode=802.3ad name=bonding.pve1 slaves=sfp-sfpplus1.s_pve1
add mlag-id=102 mode=802.3ad name=bonding.pve2 slaves=sfp-sfpplus2.s_pve2
add mlag-id=103 mode=802.3ad name=bonding.pve3 slaves=sfp-sfpplus3.s_pve3

/interface bridge
add admin-mac=XX:XX:XX:XX:AA:BB auto-mac=no dhcp-snooping=yes name=bridge priority=0x3000 vlan-filtering=yes

/interface bridge mlag
set bridge=bridge peer-port=bonding.mlag

/interface bridge port
add bridge=bridge disabled=yes interface=qsfpplus1-1.mlag
add bridge=bridge disabled=yes interface=qsfpplus2-1.mlag
add bpdu-guard=yes bridge=bridge edge=yes interface=bonding.pve1
add bpdu-guard=yes bridge=bridge edge=yes interface=bonding.pve2
add bpdu-guard=yes bridge=bridge edge=yes interface=bonding.pve3
add bridge=bridge edge=no interface=bonding.mlag pvid=99 trusted=yes

/interface bridge vlan
add bridge=bridge tagged=bonding.mlag vlan-ids=1
add bridge=bridge comment=mgmt tagged=bridge,bonding.mlag vlan-ids=13

/interface/bridge/mlag/monitor
status: connected
system-id: XX:XX:XX:XX:AA:BB
active-role: primary

/interface/bonding/monitor bonding.pve1
mode: 802.3ad
active-ports: sfp-sfpplus1.s_pve1
inactive-ports:
lacp-system-id: XX:XX:XX:XX:AA:F6
lacp-system-priority: 65535
lacp-partner-system-id: DE:4F:59:AA:F7:C5

/interface/bonding/monitor bonding.pve2
mode: 802.3ad
active-ports: sfp-sfpplus2.s_pve2
inactive-ports:
lacp-system-id: XX:XX:XX:XX:AA:F7
lacp-system-priority: 65535
lacp-partner-system-id: 6E:B1:61:51:91:AD

/interface/bonding/monitor bonding.pve3
mode: 802.3ad
active-ports: sfp-sfpplus3.s_pve3
inactive-ports:
lacp-system-id: XX:XX:XX:XX:AA:F8
lacp-system-priority: 65535
lacp-partner-system-id: BA:EF:E0:8A:D8:E7

current-firmware: 7.10
upgrade-firmware: 7.10

Who is online

Users browsing this forum: Ahrefs [Bot], ameliask, Bing [Bot], miks and 69 guests