I have a scenario with two CRS326 switches that have a dual wireless connections (for redundancy).
CRS326 #1 → port 1 → 60GHz PTP link → port 1 → CRS326 #2
root bridge → port 2 → 24GHZ PTP link → port 2
CRS326 #1 and #2 have IP address on their bridge port (all ports of switches are on the bridge on each unit)
When I enable port 2’s I see the proper RSTP topology formed, BUT I lose connectivity to CRS326 #2.. I can no longer ping it.. If I MAC telnet into CRS326 #2, I can disable port 2 and I can reach it again..
Something about turning on RSTP causes the non root-bridge port to lose connectivity (only to itself, it is passing traffic correctly over the lower path cost redundant link).
Sounds like a forgotten L2 (bridge or switch) filter rule, or port 2 missing in /interface bridge vlan rule for the native VLAN (as you say the IP is attached to the bridge interface itself), or ingress filtering set to yes with forbidden tagless frames on port 2 (you mention mac-telnet, not mac-winbox, so it may find its way through some other VLAN than the native one). So when port 2 goes to forwarding state and the other one goes to blocking, the tagless frames carrying that VLAN/subnet cannot pass through. STP only controls which ports will be forwarding and which will be blocking but doesn’t care about permitted VLANs and frame types on them.
The issue may be on the machine to which port 2 of the affected machine is connected.
If you find nothing, post config exports of both devices. One often needs a fresh pair of eyes in these cases.
thanks for the response. Some things to be clear about.
No VLAN’s at all in this setup.
root and alternation ports are selected by the switches (non root switches) correctly based on path cost..
pinging stops working INTERMITTENTLY to 10.0.1.10 or 10.0.1.133,etc any switch OTHER than the ROOT switch, whether it’s an IP on the port of the switch or the bridge on the switch from the PC in the attached diagram.
from the PC I can ping to the core ROOT bridge switch (10.0.1.11) perfeclty.. no loss.. If I disable the redundant port on either of the NON root bridge switches I do not have this problem.. It seems to be directly related to having RSTP takeover.
Here’s the config for Remote Tower 2 (a CRS326 switch), and Root Switch (also a CRS326).. I just want the 60GHz to be the priority link and when that goes down, make the 24GHz the failover link.
Remote Tower 2
# jul/22/2019 16:27:35 by RouterOS 6.45.2
# software id = ZNC9-8KAI
#
# model = CRS326-24G-2S+
# serial number = 94560942CC92
/interface bridge add name=bridge1 priority=0x2000
/interface ethernet set [ find default-name=ether1 ] name=ether1-ptp-dad-24GHz speed=100Mbps
/interface ethernet set [ find default-name=ether2 ] name=ether2-ptp-dad-60GHz speed=100Mbps
/interface ethernet set [ find default-name=ether3 ] name=ether3-ap-ne speed=100Mbps
/interface ethernet set [ find default-name=ether4 ] speed=100Mbps
/interface ethernet set [ find default-name=ether5 ] name=ether5-ptp-hce2-wt1-5GHz speed=100M
bps
/interface ethernet set [ find default-name=ether6 ] speed=100Mbps
/interface ethernet set [ find default-name=ether7 ] name=ether7-ap-nw speed=100Mbps
/interface ethernet set [ find default-name=ether8 ] speed=100Mbps
/interface ethernet set [ find default-name=ether9 ] speed=100Mbps
/interface ethernet set [ find default-name=ether10 ] speed=100Mbps
/interface ethernet set [ find default-name=ether11 ] speed=100Mbps
/interface ethernet set [ find default-name=ether12 ] speed=100Mbps
/interface ethernet set [ find default-name=ether13 ] speed=100Mbps
/interface ethernet set [ find default-name=ether14 ] speed=100Mbps
/interface ethernet set [ find default-name=ether15 ] speed=100Mbps
/interface ethernet set [ find default-name=ether16 ] speed=100Mbps
/interface ethernet set [ find default-name=ether17 ] speed=100Mbps
/interface ethernet set [ find default-name=ether18 ] speed=100Mbps
/interface ethernet set [ find default-name=ether19 ] speed=100Mbps
/interface ethernet set [ find default-name=ether20 ] speed=100Mbps
/interface ethernet set [ find default-name=ether21 ] speed=100Mbps
/interface ethernet set [ find default-name=ether22 ] speed=100Mbps
/interface ethernet set [ find default-name=ether23 ] speed=100Mbps
/interface ethernet set [ find default-name=ether24 ] speed=100Mbps
/interface ethernet set [ find default-name=sfp-sfpplus1 ] name=sfp-sfpplus1-castle speed=10Gbps
/interface ethernet set [ find default-name=sfp-sfpplus2 ] name=sfp-sfpplus2-comcast speed=10Gbps
/interface wireless security-profiles set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile set [ find default=yes ] html-directory=flash/hotspot
/interface bridge port add bridge=bridge1 interface=ether23
/interface bridge port add bridge=bridge1 interface=ether24 priority=0x70
/interface bridge port add bridge=bridge1 interface=ether8
/interface bridge port add bridge=bridge1 interface=ether1-ptp-dad-24GHz path-cost=50
/interface bridge port add bridge=bridge1 interface=ether2-ptp-dad-60GHz
/interface bridge port add bridge=bridge1 interface=ether3-ap-ne
/interface bridge port add bridge=bridge1 interface=ether4
/interface bridge port add bridge=bridge1 interface=ether5-ptp-hce2-wt1-5GHz
/interface bridge port add bridge=bridge1 interface=ether6
/interface bridge port add bridge=bridge1 interface=ether7-ap-nw
/interface bridge port add bridge=bridge1 interface=ether9
/interface bridge port add bridge=bridge1 interface=ether10
/interface bridge port add bridge=bridge1 interface=ether11
/interface bridge port add bridge=bridge1 interface=ether12
/interface bridge port add bridge=bridge1 interface=ether13
/interface bridge port add bridge=bridge1 interface=ether14
/interface bridge port add bridge=bridge1 interface=ether15
/interface bridge port add bridge=bridge1 interface=ether16
/interface bridge port add bridge=bridge1 interface=ether17
/interface bridge port add bridge=bridge1 interface=ether18
/interface bridge port add bridge=bridge1 interface=ether19
/interface bridge port add bridge=bridge1 interface=ether20
/interface bridge port add bridge=bridge1 interface=ether21
/interface bridge port add bridge=bridge1 interface=ether22
/interface bridge port add bridge=bridge1 interface=sfp-sfpplus1-castle
/interface bridge port add bridge=bridge1 interface=sfp-sfpplus2-comcast
/interface bridge settings set allow-fast-path=no
/ip settings set allow-fast-path=no
/ip address add address=10.0.1.10/24 interface=bridge1 network=10.0.1.0
/ip dns set servers=10.0.1.1
/ip route add distance=1 gateway=10.0.1.1
/ip service set telnet disabled=yes
/ip service set ftp disabled=yes
/ip service set www disabled=yes
/ip ssh set forwarding-enabled=remote
/system clock set time-zone-name=America/Denver
/system identity set name=CORE-SWITCH-CASTLE
/system logging add disabled=yes topics=stp,debug
/system routerboard settings set boot-os=router-os
/tool sniffer set filter-interface=ether23
Config of Root Bridge
# jul/22/2019 16:37:22 by RouterOS 6.45.2
# software id = VIDI-2QZJ
#
# model = CRS326-24G-2S+
# serial number = 945609FBC879
/interface bridge add name=bridge1 priority=0x1000
/interface ethernet set [ find default-name=ether1 ] name=ether1-ptp-hce1-wt1 speed=100Mbps
/interface ethernet set [ find default-name=ether2 ] speed=100Mbps
/interface ethernet set [ find default-name=ether3 ] name=ether3-ptp-castle-24GHz speed=100Mbps
/interface ethernet set [ find default-name=ether4 ] speed=100Mbps
/interface ethernet set [ find default-name=ether5 ] name=ether5-ptp-castle-60GHz speed=100Mbps
/interface ethernet set [ find default-name=ether6 ] speed=100Mbps
/interface ethernet set [ find default-name=ether7 ] speed=100Mbps
/interface ethernet set [ find default-name=ether8 ] speed=100Mbps
/interface ethernet set [ find default-name=ether9 ] name=ether9-ptp-TH-60GHz speed=100Mbps
/interface ethernet set [ find default-name=ether10 ] speed=100Mbps
/interface ethernet set [ find default-name=ether11 ] speed=100Mbps
/interface ethernet set [ find default-name=ether12 ] speed=100Mbps
/interface ethernet set [ find default-name=ether13 ] speed=100Mbps
/interface ethernet set [ find default-name=ether14 ] speed=100Mbps
/interface ethernet set [ find default-name=ether15 ] speed=100Mbps
/interface ethernet set [ find default-name=ether16 ] speed=100Mbps
/interface ethernet set [ find default-name=ether17 ] speed=100Mbps
/interface ethernet set [ find default-name=ether18 ] speed=100Mbps
/interface ethernet set [ find default-name=ether19 ] speed=100Mbps
/interface ethernet set [ find default-name=ether20 ] speed=100Mbps
/interface ethernet set [ find default-name=ether21 ] speed=100Mbps
/interface ethernet set [ find default-name=ether22 ] speed=100Mbps
/interface ethernet set [ find default-name=ether23 ] name=ether23-ptp-TH-24GHz speed=100Mbps
/interface ethernet set [ find default-name=ether24 ] speed=100Mbps
/interface ethernet set [ find default-name=sfp-sfpplus1 ] name=sfp-sfpplus1-shop speed=10Gbps
/interface ethernet set [ find default-name=sfp-sfpplus2 ] speed=10Gbps
/interface wireless security-profiles set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile set [ find default=yes ] html-directory=flash/hotspot
/interface bridge port add bridge=bridge1 interface=ether1-ptp-hce1-wt1
/interface bridge port add bridge=bridge1 interface=ether3-ptp-castle-24GHz
/interface bridge port add bridge=bridge1 interface=ether23-ptp-TH-24GHz
/interface bridge port add bridge=bridge1 interface=sfp-sfpplus1-shop
/interface bridge port add bridge=bridge1 interface=sfp-sfpplus2
/interface bridge port add bridge=bridge1 interface=ether5-ptp-castle-60GHz
/interface bridge port add auto-isolate=yes bridge=bridge1 interface=ether9-ptp-TH-60GHz restricted-role=yes
/interface bridge port add bridge=bridge1 interface=ether2
/interface bridge port add bridge=bridge1 interface=ether4
/interface bridge port add bridge=bridge1 interface=ether6
/interface bridge port add bridge=bridge1 interface=ether7
/interface bridge port add bridge=bridge1 interface=ether8
/interface bridge port add bridge=bridge1 interface=ether10
/interface bridge port add bridge=bridge1 interface=ether11
/interface bridge port add bridge=bridge1 interface=ether12
/interface bridge port add bridge=bridge1 interface=ether13
/interface bridge port add bridge=bridge1 interface=ether14
/interface bridge port add bridge=bridge1 interface=ether15
/interface bridge port add bridge=bridge1 interface=ether16
/interface bridge port add bridge=bridge1 interface=ether17
/interface bridge port add bridge=bridge1 interface=ether18
/interface bridge port add bridge=bridge1 interface=ether19
/interface bridge port add bridge=bridge1 interface=ether20
/interface bridge port add bridge=bridge1 interface=ether21
/interface bridge port add bridge=bridge1 interface=ether22
/interface bridge port add bridge=bridge1 interface=ether24
/interface bridge settings set allow-fast-path=no
/ip settings set allow-fast-path=no
/ip address add address=10.0.1.11/24 interface=bridge1 network=10.0.1.0
/ip dhcp-client add dhcp-options=hostname,clientid interface=bridge1
/ip dns set servers=10.0.1.1
/ip route add distance=1 gateway=10.0.1.1
/ip ssh set forwarding-enabled=remote
/system clock set time-zone-name=America/Denver
/system identity set name=CORE-SWITCH-DAD
/system logging add topics=stp
/system routerboard settings set boot-os=router-os
So - in my mind I think it is an issue with the ROOT BRIDGE not disabling the backup port.. see attached image.
In the image below of the ROOT BRIDGE (/interfaces bridge port print) list you can see the paired ports
Remote Tower 2 (in our previous example)
ether3-ptp-castle-24GHz
ether5-ptp-castle-60GHz
Remote Tower 1 (in our previous example)
ether2-ptp-TH-24GHz
ether9-ptp-TH-60GHz
They all show a Role of designated port.. Shouldn’t the port that is disabled show up as “backup” role instead of all of them being active and designated? Remember this is the display on the root bridge. It’s as if the root bridge has no idea which port is active, and perhaps tries to send traffic out both, and this may explain my issues of erratic behavior?
I’ve tried modifying port priority or path-cost to get the ROOT BRIDGE to prefer specific ports (as we had done on the NON ROOT BRIDGES), but there is no change in the ROLE type.
Nope, a root bridge can’t have a backup port. It’s on the other bridges to turn ports into backup.
I guess the 60GHz link is MikroTik?
What is the 24GHz link? Is it an AirFiber? Is WDS enabled on it?
Is any STP-flavor definitely disabled on the w/l links?
I had some similar issues a while ago when STP (MSTP in a Cisco-only deployment with a w60g link as a backup in my case) went totally nuts once both links were active.
In my case (which is about 18 months (and hence a lot of routerOS releases) ago) my solution was to disable neighbor discovery on the wap60g and adding bridge filters that block input and output from/to mac 01:80:C2:00:00:00, but lets it through in forward chain).
I never dug deeper in it but that solved it for that situation.
And, as long as you’re still debugging STP, I’d avoid setting path-costs and port priorities manually. That can be done as soon as STP is working.
Indeed the 60GHz link is a Mikrotik and the 24GHz link is a Mimosa B24.. I don’t see any fine control options for STP for the Mimosa, but I can defintely try those things on the Mikrotik 60GHz links.. I’ll try and report back. I did notice that RSTP was ON, for the 60GHz links
