RSTP fails to set link as alternate resulting in loop

I have a very basic, fully-connected 3 device topology (see picture):

  • ccr2116 acting as a router and connected to a cable modem for WAN interface
  • crs312 for non-POE 10gbe clients
  • qnap qsw-m2116p for 2.5gbe/POE clients

The MT devices are running ROS 7.4.1. Note that the picture shows when ccr2116 has the link manually disabled between it and qsw-m2116p to prevent the loop.

When all links are active the crs312 fails to setup the link between it and the qsw-m2116p as alternate which creates a loop. If loop protection is disabled on the qsw-m2116p the network becomes unstable. If loop protection is enabled one of the links gets completely disconnected and retested every few minutes to break the loop.

  • ccr2116 has priority 0x1000 and is identified as the root bridge by all 3 devices
  • all links have cost 2000 → qsw-m2116p has fixed cost so I changed all other devices to match
  • heartbeat time and other STP parameters match across all 3 devices
  • all other links are setup correctly. root ports on the path towards the ccr2116p. designated ports on all other links to switches. qsw-m2116p correctly sets up the link between it and crs312 as alternate.

I have tried the following and it still fails:

  • use different physical links on all 3 devices
  • replace the qsw-m2116p with a crs309. the crs309 sets up alternate, but crs312 still fails.
  • replace the crs312 with a crs309 leaving the qsw-m2116p in the network. the crs309 now fails.

In the packet dump crs312 doesn’t seem to account for the last received BPDU frame identifying the link as alternate or the missing heart beat. It never stops sending heart beats and sets up the link as designated. I filed a ticket with more info, SUP-89438, 2 weeks ago but haven’t received a reply which is unusual. They are usually more responsive. crs312 config attached. combo1 and combo2 are used to connect to ccr2116 and qsw-m2116p, respectively.

Has anyone seen a problem like this before with RSTP on ROS 7.x?
rstp_topology.PNG
rstp_switch1_crs312_incorrect_link.PNG

aug/25/2022 00:00:01 by RouterOS 7.4.1

software id = HAZK-1ULR

model = CRS312-4C+8XG

serial number = xxx

/interface bridge
add admin-mac= auto-mac=no comment=defconf name=bridge
vlan-filtering=yes
/interface ethernet
set [ find default-name=combo1 ] l2mtu=10218 mtu=9000 rx-flow-control=auto
tx-flow-control=auto
set [ find default-name=combo2 ] l2mtu=10218 mtu=9000 rx-flow-control=auto
tx-flow-control=auto
set [ find default-name=combo3 ] l2mtu=10218 mtu=9000 rx-flow-control=auto
tx-flow-control=auto
set [ find default-name=combo4 ] l2mtu=10218 mtu=9000 rx-flow-control=auto
tx-flow-control=auto
set [ find default-name=ether1 ] l2mtu=10218 mtu=9000
set [ find default-name=ether2 ] l2mtu=10218 mtu=9000
set [ find default-name=ether3 ] l2mtu=10218 mtu=9000
set [ find default-name=ether4 ] l2mtu=10218 mtu=9000
set [ find default-name=ether5 ] l2mtu=10218 mtu=9000
set [ find default-name=ether6 ] l2mtu=10218 mtu=9000
set [ find default-name=ether7 ] l2mtu=10218 mtu=9000
set [ find default-name=ether8 ] l2mtu=10218 mtu=9000
set [ find default-name=ether9 ] mtu=1598
/disk
set usb1 disabled=no name=disk1
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=combo1
internal-path-cost=2000 path-cost=2000
add bridge=bridge comment=defconf ingress-filtering=no interface=combo2
internal-path-cost=2000 path-cost=2000
add bridge=bridge comment=defconf ingress-filtering=no interface=combo3
add bridge=bridge comment=defconf ingress-filtering=no interface=combo4
add bridge=bridge comment=defconf ingress-filtering=no interface=ether1
internal-path-cost=2000 path-cost=2000
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7
add bridge=bridge comment=defconf ingress-filtering=no interface=ether8
add bridge=bridge comment=defconf ingress-filtering=no interface=ether6
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface bridge vlan
add bridge=bridge tagged=combo1,combo2,combo3,ether1 vlan-ids=10
add bridge=bridge tagged=combo1,combo2,combo3,ether1 vlan-ids=20
add bridge=bridge tagged=combo1,combo2,combo3,ether1 vlan-ids=30
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=192.168.88.1/24 comment=defconf disabled=yes interface=bridge
network=192.168.88.0
/ip dhcp-client
add interface=bridge
/ip service
set telnet disabled=yes
set ftp disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/snmp
set enabled=yes trap-interfaces=all trap-target=0.0.0.0
/system clock
set time-zone-name=xxx`
/system identity
set name=RouterOS
/system ntp client
set enabled=yes
/system ntp client servers
add address=xxx
/system routerboard settings
set auto-upgrade=yes boot-os=router-os
/tool sniffer
set file-name=rstp_crs312.pcap filter-interface=combo2 filter-mac-address=
01:80:C2:00:00:00/FF:FF:FF:FF:FF:FF filter-operator-between-entries=and
memory-limit=1000KiB streaming-server=10.0.1.15

Looks like I misunderstood the RSTP protocol. Only one end of the link is supposed to be marked alternate and it’s responsible for discarding all packets both local and across the link. This is happening correctly on my network. When I shift the bridge priority around so a MT device has the alternate port it is stable. I need to figure out why the network is unstable if qsw-2116p has the alternate port.