The 2 Cisco switches (trunk ports) are each plugged into the Mikrotik with 1 cable each. They are not plugged into each other. There are no loops.
Everything generally works okay, but sometimes 1 of the Cisco switches will lose connection.
1 of the cisco switches is designated port in stp, and the other cisco switch is root (for whatever reason). The 2 ports on the Mikrotik properly show the opposite roles for each port. So that’s good.
The STP status on the cisco switch at that time of the disconnection is “P2p Dispute”, which means that cisco receives a bpdu when it is the designated port so it stops forwarding.
So… WHY is Mikrotik sending bpdu packets over that port? Are the bpdu packets from the cisco which is designated as root getting leaked through the mikrotik to the other cisco somehow?
As a side note, if I change the stp protocol inside the mikrotik bridge to stp (instead of rstp), that seems to fix it … but is not ideal.
well bpdus generally use multicast mac address which means they will get propagated throughout the network, be received by all hosts, and processed only by those that are “interested” in them. So basically yeah your suspicion is true: the Mikrotik does indeed forward PVST+ BPDUS (or Rapid PVST+) to the other Cisco, the Cisco processes it, does STP elections and you end up with a Cisco that is the root bridge and a Cisco that is not a root bridge and tries to point its designated ports to the root bridge. I believe this has something to do with the proposal-agreement mechanism RSTP follows when there is a topology change detected so that’s why when changing to 802.1D on the mikrotik the problem is solved (now there is a scenario on my head but I don’t have the time to think it through clearly so I won’t write it down). Now there are 3 solutions to the problem starting from the best:
Move over all switches to MSTP which is supported by all brands. You can even use a single MST for all vlans which means it will work identically to RSTP. This will future-proof your design as it will allow you to create redundant topologies with whatever vendor by just tweaking the MSTP. And of course for the problem at hand, you will not have to worry about silly issues happening due to incompatibility
Disable stp altogether since this is a flat network topology. For mikrotik you would just set protocol=none on bridge settings. On Cisco it will be a bit more complicated: https://community.cisco.com/t5/switching/how-to-disable-stp-on-port/td-p/1283874. But bear in mind that doing this will make layer 2 loops more likely to occur and it will add configuration burden if you ever want to introduce redundancy in the network.
You describe a lot of things. Just to make a few points:
Mikrotik and CISCO stuff are fully compatible on the RSTP front. STP nowadays should not be used. MSTP is simply unnecessary. PVST and its variants are not supported by Mikrotik. Stick with RSTP.
That a root bridge is selected is normal. If you have any sort of nontrivial setup, you should play attention to which bridge you would want to have as the root. In simple setups (like yours) all you have to do is set the bridge priorities correctly. Not setting these up should not and will not lead to the “flapping” scenario described. Nevertheless it’s good practice to set it up correctly.
If RSTP (or any other STP) is enabled (and the port is not an edge port), BPDUs are exchanged regardless of forwarding status. They are used to propagate topology information, which is necessary for STP to function.
Managed (proper) switches don’t forward BPDU frames. (They propagate the contained information when sending BPDUs of their own, but that is not the same as forwarding the packets themselves.) Dumb switches often forward these. Mikrotik has a special “dumb switch” mode, where STP protocol mode is set to “none” and “forward restricted” is explicitly enabled. If you don’t have some highly specific use for this, don’t use this.
The p2p mismatch error means that one side considers the connection point-to-point, while the other one does not. Mikrotik allows configuring this. The default setting of “auto” results in p2p=yes if the port if full duplex. On Cisco this is not the default mode. Either change Mikrotik to p2p=no, or the Cisco to p2p=yes. But they should match. (p2p=yes is in the strict sense the correct option.) Your port flapping should disappear.
you’re absolutely right on your points and thanks for correcting me. I was wondering though about the interoperation between R-PVST+ and RSTP? How do you make it work exactly? They are sending bpdu to a different multicast destination mac address and PVST+ is also sending bpdus tagged with 802.1Q. I get that vlan 1 bpdus are sent to the 802.1d multicast mac address 01:80:c2:00:00:00 but what about the rest of the bpdus that are sent to 01:00:0c:cc:cc:cd and tagged, won’t these be propagated downstream (routeros doesn’t know what to do with them)? What would happen in case there is a multi-redundant (and multivendor) environment where cisco pvst+ is configured to choose different uplinks based on the per-vlan spanning trees? I get that simple, loop-free setups will be easy to configure but what happens in complex setups?
This is basically why I’m suggesting MSTP. Yes, it’s more of a headache configuration-wise but you don’t have to think what will happen if you add more vendors beyond cisco down the line. Thanks again for your input!
Basically PVST and PVST+ (and R-PVST+) in general were designed as a better alternative to the STP-RSTP protocols. In that they succeeded. MSTP coming about muddled things a bit; while achieving the same general goal (per-VLAN spanning trees), it does so very differently. (MSTP does not address redundancy.)
PVST provides limited support and compatibility with RSTP. Cisco actually has a really nice KB article on this, which I haven’t been able to find in the 30 seconds I dedicated to it Main points:
PVST per-VLAN traffic does not conflict with RSTP traffic - and a switch may speak both simultaneously, which Cisco implements
PVST traffic specifically uses MAC addresses that are forwarded in the STP domain (the STP domain is “flooded”)
PVST switches are willing to speak RSTP over native VLAN 1
The intended use case for this is making a transition from an RSTP setup to a PVST setup gradually. Therefore PVST supports having a PVST-enabled region and a neighboring RTSP-enabled one, and also two PVST regions separated by an RSTP-only zone. Cisco documentation repeatedly warns that this is not “good design”, but rather a forced interim measure.
As to MSTP. While MSTP can be setup (and its setup is only a little bit more involved than RSTP) and I have nothing specifically against it, in simple scenarios I prefer RSTP, because it is very widely supported even in multi-vendor scenarios (especially in case that involves budget devices). Also, in simple setups, what do we actually gain by using it? In terms of being ubiquitous, MSTP is also getting there. In the end it’s a matter of preference.
Cisco’s p2p defaults are the same as Mikrotik, according to the Cisco website: “The link type is automatically derived from the duplex mode of a port. A port that operates in full-duplex is assumed to be point-to-point, while a half-duplex port is considered as a shared port by default.”
So they are both already p2p yes by default.
The error is not “p2p mismatch”. The error is “p2p dispute”. The definition of “p2p dispute” is “cisco receives a bpdu when it is the designated port so it stops forwarding”. Therefore, I still think that this is an issue of leaking bdpu’s through the Mikrotik bridge for some reason instead of a p2p enable/disable mismatch.
I’m going to try lowering the priority number of the mikrotik bridge to force it to be the root. Then, I think that neither cisco should be root and that neither cisco should be sending bpdu packets.
Main points: