Please review my IP routing table below this post.
The Mikrotik has two PPPoE clients, and both clients add a default route to the routing table. One with a higher distance value set than the other.
On top of the PPPoE sessions, two BGP sessions are also established, both receiving a default route. One with the same default distance (20), both with a different LOCAL_PREF BGP attribute value.
Since the Mikrotik has two uplinks for its Internet connection, routing marks are used to direct response traffic to the appropriate interface from where packets first came from.
We had a situation where the PPPoE interface was removed from server side, but not from the Mikrotik.
Because we forgot to check the keepalive interval checkbox in the pppoe-out-Eurofiber_LINK#1 PPPoE client, the default route created by the PPPoE client was still active.
After the expiration of the hold timer, the BGP session over the pppoe-out-Eurofiber_LINK#1 PPPoE client was removed, thus removing the default route learned from BGP.
This means that, looking at the distance value of 20 of a default route learned from BGP, that the default route learned from the BGP session over the pppoe-out-Ziggo_LINK#2 should win.
However, this was not the case.
The pppoe-out-Eurofiber_LINK#1 PPPoE clients default route was still preferred over the pppoe-out-Ziggo_LINK#2 BGP default route.
Would anyone be able to explain why this could have happened?
Reproducing attempts of the situation were unsuccessful so far.
At all times, the pppoe-out-Ziggo_LINK#2 BGP default route was preferred over any other route.