Community discussions

MikroTik App
 
alexvdk
just joined
Topic Author
Posts: 5
Joined: Mon Jan 24, 2022 8:46 am

Router not withdrawing BGP routes from unrechable peer

Thu Jun 16, 2022 4:48 am

Greetings.

Models 951Ui-2HnD and 1100AHx2.
Running different versions of RouterOS from 6.42.7 to 6.48.6

I have found an issue with BGP on a lot of Mikrotik routers.
Mikrotik router on site is building GRE Tunnels with two border routers in Data-Center.
eBGP peering is established via this GRE Tunnels with each of two border routers in DC.
Image

Two default routes are installed into routing table of Mikrotik router on site. Routes are received via eBGP.
Here how routing table does look like. One of two default routes is picked via "Lowest Router-ID BGP tiebreaker".
Image

The issue is:
When GRE Tunnel 1, which leads to Border1 and from which bgp peer default route is "active", goes down second default route from Border2 does never become active. Even when first route from Border1 is withdrawn from routing table due to BGP peering is not "established" anymore and there is only one default route from Border2 leaft in routing table.
Default route from Border2 becomes active only if I disable and reanable Border2 bgp peer.

I tried to reproduce this issue in EVE NG lab, but had no success. All works as expected and bgp route is withdrawn quickly after GRE Tunnel falls.

Thanks in advance!
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7042
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Router not withdrawing BGP routes from unrechable peer

Thu Jun 16, 2022 10:34 am

v6 does not trigger bgp selection when one bgp route goes down. This is solved in v7 with the new routing engine.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10195
Joined: Mon Jun 08, 2015 12:09 pm

Re: Router not withdrawing BGP routes from unrechable peer

Thu Jun 16, 2022 11:55 am

Why doesn't the BGP peer connection go down when the GRE tunnel is down? it should do that at least after the hold time.
Make sure you have configured TTL=1 and proper "update source" (either 172.17.1.2/172.18.1.2 or the GRE interface name) as well.
I use such configurations a lot and it switches over reliably with these settings. I never enable GRE keepalive.
(we do use BFD for quicker response, BGP hold time is normally 180 so it will take 3 minutes for BGP to notice the tunnel is down, BFD makes that 1 second)
 
alexvdk
just joined
Topic Author
Posts: 5
Joined: Mon Jan 24, 2022 8:46 am

Re: Router not withdrawing BGP routes from unrechable peer

Fri Jun 17, 2022 4:43 am

Why doesn't the BGP peer connection go down when the GRE tunnel is down? it should do that at least after the hold time.
Make sure you have configured TTL=1 and proper "update source" (either 172.17.1.2/172.18.1.2 or the GRE interface name) as well.
I use such configurations a lot and it switches over reliably with these settings. I never enable GRE keepalive.
(we do use BFD for quicker response, BGP hold time is normally 180 so it will take 3 minutes for BGP to notice the tunnel is down, BFD makes that 1 second)
BGP on "dead" GRE tunnel does go down after hold timer expires and route is withdrawn from routing table, but "backup" route never becomes "active".
TTL=1 is in place, "update source" is not needed as it is pure eBGP. There is no other options to sent packet with "source ip" different from GRE tunnel address.
May be BFD solves this issue, but I cannot enable it on 1000+ site GRE tunnels on Border routers as they do not have BFD hardware offload and enabling "software" BFD will crush them.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10195
Joined: Mon Jun 08, 2015 12:09 pm

Re: Router not withdrawing BGP routes from unrechable peer

Fri Jun 17, 2022 11:02 am

Did you check that the BGP peer indeed goes down? The recommendations for TTL and update source I made are to make sure that the peer does not remain connected via another path. Update source in RouterOS is also used as the source address for the BGP peer connect. With these settings you are sure it can only connect the BGP peer over its own tunnel so the BGP connection is assured to go down when the tunnel goes down.

In our network we have many connections like this (eBGP routers connected via tunnels and wireless links) and I never noticed such behavior!
We use BFD for quick failure detection but it is not required, as long as you are sure that the BGP connection to a peer goes down when its tunnel goes down.
(i.e. it does not fall back to some multihop connection)
 
alexvdk
just joined
Topic Author
Posts: 5
Joined: Mon Jan 24, 2022 8:46 am

Re: Router not withdrawing BGP routes from unrechable peer

Tue Jun 21, 2022 11:13 am

Did you check that the BGP peer indeed goes down? The recommendations for TTL and update source I made are to make sure that the peer does not remain connected via another path. Update source in RouterOS is also used as the source address for the BGP peer connect. With these settings you are sure it can only connect the BGP peer over its own tunnel so the BGP connection is assured to go down when the tunnel goes down.

In our network we have many connections like this (eBGP routers connected via tunnels and wireless links) and I never noticed such behavior!
We use BFD for quick failure detection but it is not required, as long as you are sure that the BGP connection to a peer goes down when its tunnel goes down.
(i.e. it does not fall back to some multihop connection)
I double checked BGP connection to go down. This is confirmed.
Default route from "dead" BGP peer is withdrawn from routing table. This is confirmed.
Second default route from "active"="established" peer is in routing table, but it is never active until I diasble and enable BGP peer which this route is received from.

I did a little test today.
Platform - hAP AC^2, arm architecture, software 6.48.6 long term.
And it is operating as excpected! BGP routes work as expected, removed when GRE Tunnel is down and second route goes active.
Conclusion. There is some strage BUG with hardware paltform - 951Ui-2HnD.
We have 1500+ this platforms and BGP is just bugged on 'em.

Some words from Mikrotik support?
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7042
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Router not withdrawing BGP routes from unrechable peer

Tue Jun 21, 2022 11:30 am

As I already mentioned above, ROS v6 does not trigger bgp best path selection when one of the BGP routes goes down.

* if you have multiple BGP routes as well as other routes to the same destination form other protocols, then after the active BGP route is removed or becomes inactive next best will be considered route from other protocols with lowest distance.
* if you have only multiple BGP routes to that destination, then you can get active next BGP route.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10195
Joined: Mon Jun 08, 2015 12:09 pm

Re: Router not withdrawing BGP routes from unrechable peer

Tue Jun 21, 2022 12:09 pm

I use those 951Ui-2HnD routers in some places and I never saw a different behavior on them.
(if anything, I see bugs on the ARM platform that I do not see on the MIPSBE and TILE platforms, but they are not BGP related)

I am still confused about what mrz is trying to tell us with "ROS v6 does not trigger bgp best path selection when one of the BGP routes goes down". In our network, routes (all received via BGP) perfectly switchover when a BGP peer (over tunnel or wifi) goes down.

Who is online

Users browsing this forum: No registered users and 12 guests