Hi,

I have two core routers setup to distribute routes for a VRF using the BGP VRF function. nothing complicated and routes do load on both routers but they flash on and off a few times before eventually disappearing on 1 of the routers and loading as secondary on the other (what I want them both to do).

The idea here is this VRF is the routing domain for customers and all VPLS tunnels terminate into it. This VRF then gets upstream routes from our internet routers (just a default route and a few others). I want both core routers to load the routes so that if core 1 loses its link to its upstream those routes will drop off and the secondary routes via the other core will go active and forward traffic to core 2 who will then forward it upstream to internet router 2.

Have I done anything wrong here in my config as its all simple and about as straight forward as I can make it I'm just new to Mikrotik.

Image of core 2 with routes as expected (cust-isp mark and blue secondary routes are the ones from core 1)


Image of core 1 with routes after they disappear (they do some up then go them come up then go a few times before they go for good)


FYI this is all in an EVE-NG lab for now as I have to get this correct before I implement into a live network.

What does your MPLS config look like?

What does your MPLS config look like?

This is the MPLS config from a core router: (happy to post the whole lab if preferred)
The basic setup is all routers run OSPF for backbone with loopback interfaces for MPLS. MPLS LDP is setup for MPLS to work and BGP VPLS is used to build tunnels from the towers back to the cores. The core routers have a VRF for doing customer traffic routing (all VPLS tunnels and internet facing bridges are in the VRF) (design has both shared internet VRF and customer dedicated VRF) and these VRF's do BGP to our edge routers for internet connectivity. All I want to do is have the VRF's exchange routes so that if internet feed 1 goes down on core 1 but core 1 is still the VRRP master for the customer CPE gateway then core 1 should re-route traffic to core 2 and use it's internet feed.

It works perfectly if I create a new BGP instance tied to the VRF routing mark then run standard BGP between the two VRF instances (add an interface into the VRF and use a mangle rule to get around the VRF to VRF bug in MT code).

But if I try to use the BGP VRF and sync routes between them enabling vpn4 distribution in the peer I just get the routes flashing up and disappearing form both routers as I mentioned until eventually they stick only on 1 router. The other shows no VPN4 routes and nothing learned in the route list. Sometimes it will even go half and half over both but never full on both.

Any chance you can grab a packet capture from BGP / MPLS when the routes are going in and out? Might provide a clue as to what it's unhappy about

Any chance you can grab a packet capture from BGP / MPLS when the routes are going in and out? Might provide a clue as to what it's unhappy about

Tried to get a packet capture using the internal tool but when it is running the flapping does not happen the routes come up on a single router and remain that way.

I did an external capture and it seems to be normal BGP. Further testing shows what happens is:

both core 1 and core 2 refuse to hold both routes at the same time almost like some sort of loop prevention. They load on core 1 then on core 2, once core 2 has them loaded core 1 drops them. core 1 then loads them again and core 2 drops them. core 2 then loads them and core 1 drops them. This repeats until BGP internal flap counter triggers and the routes are removed from 1 of the routers (if MK do have a BGP flap counter).

If I load the routers with anything like packet capture then the bounce back and forward does not happen it goes straight to one of the routers holding the routes and the other ignoring them completely.

I have rebuilt the lab and am unable to make this work with BGP VRF at all every scenario I try gives the same result. Going to see if a version change effects it. Will also try physical routers to see if it still happens on them.

Found the fault. It was using the same Route Distinguisher on both VRF instances (thinking was these are redundant versions of the same network)

So I changed it to be:

Core 1

RD 1:50
Import 2:50
Export 1:50

Core 2

RD:2:50
Import 1:50
Export 2:50

Once that was done both come online and load the backup routes and everything works as expected. My bad all along (read the manual).