I have some mikrotiks setup as switches using RSTP. Basically I bridge all physical Ethernet ports and turn RSTP on. On the main switch at the router I set the priority lower than the default of 8000 to 7000 so it becomes the root bridge. All works well, I can even get failover working with this setup.

The problem: Network loops (broadcast storms) can still destroy the network. When ever this happens it's from a customer that is not natted an exists on a vlan. The loop will take down everything plugged into that switch. Shouldn't it be stopped by RSTP?

Should I be creating a separate bridge with RSTP for the VLANS? Or should all Ethernet ports bridged be sufficient?

From what I understand about your description you don't have RSTP on your vlans which is why the broadcast storms are persisting, you will need to put a RSTP filter on the vlan at a point which can break the bridge loop apart. Putting RSTP on your ethernet ports only stops a non tagged broadcast, since RSTP does its detection with BPDU's and not from the data itself the vlan loop will persist.

Can you put up a simplistic network map so I can better understand how you have it setup?

Ryan

Ok, don't make fun of my drawing I don't have visio. In it's simplest form this would be the layout. The loop was created by the customer on VLAN 100 on Switch 2. It took down everything plugged into Switch 2 and Switch 1. I could log into Switch 1 when it was happening because I was on the other side of the router, but I could not see anything on Switch 1 or Switch 2 until we unplugged the customer from the network.

The problem turned out to be a damaged cat-5 cable at the customer that had transmit and receive wires shorting out together. All the customers that are on untagged vlans sit behind a natted router and cannot create loops. Customers on vlan 100 are bridged.

If I create a 2nd bridge for all the Vlan 100 ports, do I still need to Bridge all the Ethernet ports for the untagged traffic or would I create a bridge for Vlan 0 in that case?

If I create a 2nd bridge for all the Vlan 100 ports, do I still need to Bridge all the Ethernet ports for the untagged traffic or would I create a bridge for Vlan 0 in that case?

From what I understand of your diagram you would need two bridges on each router. I am expecting ether1 on each switch links with the other here. This also doesn't take into account any IP's or gateway configuration.

Switch1

/int vlan add interface=untagged vlan-id=100 comment=taggedfromswitch2 name=vlan100

/int bridge add name=vlan100
/int bridge port add interface=vlan100 bridge=vlan100
/int bridge port add interface=ether4 bridge=vlan100

/int bridge add name=untagged
/int bridge port add interface=ether1 bridge=untagged
/int bridge port add interface=ether2 bridge=untagged
/int bridge port add interface=ether3 bridge=untagged

Switch2

/int vlan add interface=untagged vlan-id=100 comment=taggedfromswitch1 name=vlan100

/int bridge add name=vlan100
/int bridge port add interface=vlan100 bridge=vlan100
/int bridge port add interface=ether4 bridge=vlan100

/int bridge add name=untagged
/int bridge port add interface=ether1 bridge=untagged
/int bridge port add interface=ether2 bridge=untagged
/int bridge port add interface=ether3 bridge=untagged

If I am off base here let me know where I went wrong and I will update this.

Ryan

Ok I'm going to try this and run some experiments but I'm a bit confused. If I currently have just 1 bridge with all physical Ethernet ports on that bridge, how do the vlans even cross that bridge without utilizing the spanning tree? I would think that all traffic on the bridge would be bound to the spanning tree rules if all physical ports are bridged regardless if it was tagged or not.

Ok I'm going to try this and run some experiments but I'm a bit confused. If I currently have just 1 bridge with all physical Ethernet ports on that bridge, how do the vlans even cross that bridge without utilizing the spanning tree? I would think that all traffic on the bridge would be bound to the spanning tree rules if all physical ports are bridged regardless if it was tagged or not.

VLANs by definition does not need to utilize any Spanning tree. The STP is only used for creating logically non-looping topology.
What Ryan is suggesting, which I also recommend is to create a bridge for every Vlan in your LAN and than associate the bridge with the ports where this vlan needs to be enabled.
Working with Vlans is possible in RouterOS.. but it's a bit unlogical to me. I come from the world of Cisco IOS where the concept of Vlan configuration is completely different.
There's also a possibility to use the submenu /interface ethernet switch where you can configure tagged and untagged ports. As far as i understand the configuration through that submenu directly utilizes the switch chip on your device, if you have one.

One question... do you have any physical redundancy in your network? I don't see any on your topology... If you don't have any physical redundancy, whether you run or not STP doesn't really make any difference whatsoever....


Boyan

Hmm, maybe this is my problem. I do use RSTP in other parts of the network for redundancy links, but not in this case. I've been expecting it to stop network loops if they occur however. Maybe I am mis-using this. In my diagram above, if a customer on switch 2 creates a network loop at his location, shouldn't RSTP stop that storm from going beyond the port on switch 2 that he is plugged into? Or does it not work this way.

R/STP will only detect L2 loops - it will not stop broadcast storms coming on a single port (say a loop on a downstream unmanaged switch).

Does ROS support storm control or PPS limit on a port?

Hmm, maybe this is my problem. I do use RSTP in other parts of the network for redundancy links, but not in this case. I've been expecting it to stop network loops if they occur however. Maybe I am mis-using this. In my diagram above, if a customer on switch 2 creates a network loop at his location, shouldn't RSTP stop that storm from going beyond the port on switch 2 that he is plugged into? Or does it not work this way.

That's a fair point you make. I'm not sure how the Mikrotik device would react when you have a local loop (patch two local ports for example).
On the Cisco switches I've noticed that they use a slightly modified version of the very old LOOP protocol. So if the switch detects it's own LOOP frame which it constantly sends on each active port - the port where this frame is received is disabled to prevent a local loop.
STP in general is designed to exchange signaling information between switches. So in redundant topology the switches can decide which ports will be forwarding traffic, which ports will be blocked. But by blocked I mean, blocked only for user traffic, all of the ports continuously exchange STP frames, by default every 5 seconds.
If I have time, I'll test how does a Routerboard react when I loop two port... I haven't done this...

R/STP will only detect L2 loops - it will not stop broadcast storms coming on a single port (say a loop on a downstream unmanaged switch).

Does ROS support storm control or PPS limit on a port?

You can set a limit per port with mangle by setting the limit option and applying it directly to a port which may or may not work if the port is in a bridge. If it doesn't work you need to mark the traffic coming in the interfaces and then have mangle limit it.

I just got back from vacation (yay cancun!) and finally got around to playing with this. I ran a test and an upstream loop does not get blocked by RSTP. So I'd like to play with mangle a bit and see if I can create storm control on these Mikrotiks. Any body have any examples?

I know a real switch would be better but I have 70 existing towers with Mikrotiks as switches and upgrading is just not an option for me right now. (I don't decide where the money goes).