I had a topology like this:
SW1 --- Mikrotik A ----- Mikrotik C --- SW3
| |
SW2 --- Mikrotik B ---------+
The network was actually a bit more complex than this - but it’s the general idea - the two Mikrotiks (A & B) on the left were intended to be redundant. I wanted to use VPLS to bridge SW1/SW2 with SW3. There is an obvious loop, and the obvious way to avoid problems would be to enable spanning tree.
However, something between SW1 and SW2 ate spanning tree packets, and I eventually gave up on the spanning tree packets getting through. So I came up with this ugly hack which might be useful to someone else - that said, it still carries some risks, so use with care. It’s definitely not the right way to do this.
Basically, I disable the VPLS BGP entry on either Mikrotik A or Mikrotik B when the other is active. I can figure out which one is active based on the VRRP state. So if Mikrotik A is the master in VRRP, it will have active VPLS tunnels, while Mikrotik B will have them disabled.
The heart of it is this script:
# Bring VPLS up or down based on VRRP state
# We look for VPLS interfaces in bridges by looking for /interface vpls bgp_vpls
# entries where bridge-horizon=1, so use a different bridge horizon for entries
# you don't want impacted by this script.
:local vrrp;
:set vrrp "vrrp209"
:if ( [ /interface vrrp find where master=yes running=yes name=$vrrp ] != "" ) do={
# We are *UP*
/interface vpls bgp-vpls enable [ find where bridge-horizon=1 disabled=yes ]
} else={
# We are *DOWN*
/interface vpls bgp-vpls disable [ find where bridge-horizon=1 disabled=no ]
};
Basically, if this script is executed and vrrp209 is up, it will enable everything with a bridge-horizon=1 in /interface vpls bgp-vpls. Otherwise, it disables everything with a bridge-horizon=1 there. I can create entries with bridge-horizon=2 (for instance) if I don’t want them impacted by this script.
This script is executed at startup, and whenever VRRP goes up or down (via an on-master or on-backup in the /int vrrp entry). The VRRP is really just used as a semaphore to determine which router is “VPLS Master”. I also run this script every minute or so just to have sort of a belt-and-suspenders approach, because bad things are going to happen if VPLS is enabled on both Mikrotik A and Mikrotik B!
The bridge that the VPLS goes into on Mikrotik A and Mikrotik B is running spanning tree with a forwarding delay of 20s (the default). This gives me a little bit of slack on a router reboot, if the VPLS was up when the system was shut down - hopefully my on-start script executes before the bridge comes out of that state.
Now, if for some reason both routers thought they were the VRRP master (like, for instance, VRRP packets were being eaten somewhere), then bad things would happen and you would have a loop and likely some sort of switching meltdown. Spanning tree - assuming your switches don’t eat spanning tree - is probably the right way to do this (or, better yet, layer 3 VPN). I’d be very careful putting this in production somewhere.
It would be kind of nice if, one day, Mikrotik supported an alternative loop detection method for layer 2 - something like what HP does with theirs, where basically the switch can send a broadcast packet, and, if it hears that packet on any port, it can then shut down that port. I imagine they implemented that for similar reasons that I implemented this atrocity - some switches aren’t transparent to spanning tree and/or don’t support it. You can also get into issues with large diameters with spanning tree, if you have a need to cable enough switches together (not good practice, but it also happens).