Strange MSTP behaviour, CRS328, RB760iGS, RBM33G, hAP AC^2 - MMIPS-specific?

RouterOS General
1

Hello,
I have a weird issue with spanning tree on RouterOS 7.3.1: I have a bridge set up on a CRS328, and a bridge on an RB760iGS (hEX S). The CRS328 has ether1 linked to the RB760’s ether1.

CRS328 bridge:

/interface bridge
add frame-types=admit-only-vlan-tagged name=unibridge priority=0 protocol-mode=mstp vlan-filtering=yes
/interface bridge port
add bridge=unibridge frame-types=admit-only-vlan-tagged interface=sfp-sfpplus1
add bridge=unibridge frame-types=admit-only-vlan-tagged interface=ether1 pvid=100
/interface bridge vlan
add bridge=unibridge tagged=unibridge,sfp-sfpplus1 vlan-ids=1
add bridge=unibridge tagged=sfp-sfpplus1,ether1 vlan-ids=5
add bridge=unibridge tagged=sfp-sfpplus1,ether1 vlan-ids=100

RB760 bridge:

/interface bridge
add frame-types=admit-only-vlan-tagged name=unibridge priority=0x1000 protocol-mode=mstp vlan-filtering=yes
/interface bridge port
add bridge=unibridge frame-types=admit-only-vlan-tagged interface=ether1 pvid=100
add bridge=unibridge frame-types=admit-only-untagged-and-priority-tagged interface=ether2 pvid=5
/interface bridge vlan
add bridge=unibridge tagged=unibridge,ether1 vlan-ids=100
add bridge=unibridge tagged=ether1 untagged=ether2 vlan-ids=5

When attached to the RB760, I lose connectivity after a few seconds with the following in the CRS328’s log (when topics=stp is enabled):

03:16:03 bridge,stp ether1:0 learning 
03:16:03 bridge,stp ether1:0 discarding 
03:16:05 bridge,stp ether1:0 learning 
03:16:05 bridge,stp ether1:0 discarding 
03:16:07 bridge,stp ether1:0 learning 
03:16:07 bridge,stp ether1:0 discarding

If I swap out the RB760 for an hAP-AC^2 with an identical bridge config, everything works as expected. I’ve erased, netinstalled, reset and reconfigured the RB760 so many times now I’m not being objective and probably missing something blindingly obvious. I’ve consulted the block diagrams and as far as I can tell, this should work.

If I disable MSTP on the RB760, connectivity returns and the CRS328 logs the following:

 03:16:09 bridge,stp ether1:0 learning
 03:16:11 bridge,stp ether1:0 forwarding

If I swap out the RB760 for an hAP-AC^2 and leave MSTP enabled, the issue goes away and MSTP works fine.

I’m rummaging around now to see if I have another MMIPS board with a switch chip to see if it’s architecture-specific. In the mean time, can anyone spot my obvious mistake?

(Some while later): Tried with an RBM33G and the exact same bridge config - this is also causing the CRS328 to spit out the learning, discarding, learning, discarding loop, so based on a sample size of two, is this MMIPS-specific?

EDIT: Is this endian-specific? Next test this evening is to try the two MMips boards against each other and see if they work…

Thanks!

2

which version of RouterOS are you running??

i have an MSTP simple implementation involving an hEX S without problem using 6.48.6

in your config i don’t see your configured parameters of MSTP and bridge related to that

consider the possibility of some port flapping on the network causing that

3

Hi,

7.3.1 as stated. Routerboard fIrmware is also updated on all devices.


As I say, I may have it wrong, but when defaults are enabled in 7.3.1, MSTP works fine simply by including the “protocol mode” and “priority” parameters in the bridge config.This is also working as shown on at least 6 other switches in a different setup with them running 7.3.1. So far, leaving edge port definitions as auto has also worked fine, although that’s now something I’ll check. Also, if identical config works on one model but not another, curtious why this would change (although not doubting it might!)

Config line that introduces MSTP:

/interface bridge
add frame-types=admit-only-vlan-tagged name=unibridge priority=0 protocol-mode=mstp vlan-filtering=yes

Thanks!

4

In recent versions of routeros 7 hex s received updates to run bridge vlan filtering by hardware, maybe You need to check if disabling bridge hardware offload solves they situation

5

That’s a great idea. I’d just found the chip feature compatibility matrix in the new documentation. I’d just figured that if it was an incompatible feature it’d disable hardware offload anyway, but perhaps not. Thanks!

6

OK - so this has got to be a bug. No changes to config, MMIPS to MMIPS - works fine (and with hardware acceleration, which is a bonus). ARM32 to ARM32 - works fine. ARM32 to MMIPS - no workie.

Endianness in a calc somewhere, or something more simple?

The test rig is getting a little out-of-hand now…

7

nice lab you have !!!

8

Thank you! It all started with an RB750 about 15 years ago. Can’t believe I actually get paid to fiddle with this stuff :grin:

9

Disabling hardware acceleration has fixed the issue.

I’ve raised an issue with Mikrotik support and they’ve duplicated it in their labs, with a comment of …“we are looking forward to fixing it in the future RouterOS versions” - which is encouraging :slight_smile: Marking your post as the fix as ultimately this workaround works until the hardware fix is in place. Cheers!

EDIT: Same (software bridge working, hardware offload enabled does not work) in 7.4 also.

10

Hi,

See the same thing on 24x CRS328-24P-4S+ running v7.4.

Its not just the local MSTP instance that suffers, edge events are propagated downstream as well. Same result except now every edge event results in all switches in the region quitting forwarding for a short while. The downstream propagation only happens if the switch has more than one path. Disable your alternate paths and the edge propagations go away.

Local instance still bounces each time. I have an open ticket on the issue. But it seems you already have an answer, they know and will fix.

11

Wow, OK - I was only one switch deep at the time in a lab environment so didn’t see anything of that scale. Feel like I’ve dodged something more substatial now though!

12

Back to rSTP for awhile :slight_smile:

13

Are there any Updates on this topic?

We’ve got the same problem at our campus.
18x CRS326-24S+2Q+ on ROS 7.5 as core switches and 45x Dell N1100/N3048 as access switches.
Back to RSTP would be a bit much work… :confused:

14

FYI
From the mikrotik support:

Hello,
Thank you for the report!
We have managed to reproduce the issue locally in our labs and look forward to fixing it on upcoming RouterOS versions, unfortunately, I cannot provide a release date now.
Sadly there is no workaround available, other than running RSTP.
Best regards,