In RouterOS v7 it seems that when two or more peers have a BGP session to the same local IP, and one of them closes, they all are closed. The log says "Idle" for those sessions.
Usually they are quickly re-established by the peer and remain up from that time.
In my config this occurs for a number of peers connected via L2TP/IPsec. I have a loopback device with a /24 RFC1918 subnet on it, and the router has one address in it. All the L2TP/IPsec clients connect to the router, get their IP (via a fixed "remote address" in the PPP secrets list), and BGP is (statically) configured between those remote addresses and the local router address. I know it can be done more automatically in v7, but part of the clients are still v6.
Those peers are often over LTE, so regularly they go down/up because the provider has disconnected them after a while and issued a new public IP on re-connection. What I see is that those BGP sessions with the same local address go down at that moment. But other BGP sessions, that are over GRE tunnels and thus have a different local IP on the central router, remain up.
Does anyone else see this? Is this "normal" for v7 or should I search for something in the config that indirectly triggers this?
It doesn't happen with the same config on v6.
Should it be considered a bug? Or is this something that is specified deep in the BGP specs and just wasn't implemented in v6?
Re: BGP sessions close when another session to the same IP closes
Observing more closely it does not seem to be related to "session to the same IP".
It is important that on the central router the connections are listen=yes connect=no (i.e. "passive").
All connections are running (session is active), then one of them closes for whatever reason, it re-connects and then another one goes to Idle state waiting for a new connection.
It is important that on the central router the connections are listen=yes connect=no (i.e. "passive").
All connections are running (session is active), then one of them closes for whatever reason, it re-connects and then another one goes to Idle state waiting for a new connection.
Re: BGP sessions close when another session to the same IP closes
I think I am closing in on the problem...
It turns out that the L2TP client addresses of the different routers get sent around using BGP, mostly due to the different way that BGP networks and filtering work in v7 relative to v6.
Whenever an L2TP link closes, that info gets sent around the entire network, and although I have no explanation for that it seems to cause a routing flap or whatever for the other clients, causing them to close as well.
I have now changed the routing filters so these individual addresses are no longer accepted (by limiting the dst-len to at most 29) and after a couple of days it appears the situation has improved much.
It is always a bit debatable whether you want to distribute routes for tunnels through the network, instead of keeping them private to the peers. It can be a good idea when "preferred source address" is not carefully configured in routes and route filters, because the router would select the tunnel address as source when it has to send something. It also may be convenient for monitoring.
But for the L2TP/IPsec dummy bridge it is sufficient to send a route for the entire subnet, not for each individual address.
It turns out that the L2TP client addresses of the different routers get sent around using BGP, mostly due to the different way that BGP networks and filtering work in v7 relative to v6.
Whenever an L2TP link closes, that info gets sent around the entire network, and although I have no explanation for that it seems to cause a routing flap or whatever for the other clients, causing them to close as well.
I have now changed the routing filters so these individual addresses are no longer accepted (by limiting the dst-len to at most 29) and after a couple of days it appears the situation has improved much.
It is always a bit debatable whether you want to distribute routes for tunnels through the network, instead of keeping them private to the peers. It can be a good idea when "preferred source address" is not carefully configured in routes and route filters, because the router would select the tunnel address as source when it has to send something. It also may be convenient for monitoring.
But for the L2TP/IPsec dummy bridge it is sufficient to send a route for the entire subnet, not for each individual address.
Re: BGP sessions close when another session to the same IP closes
No, unfortunately that wasn't it... I still see sequences like this:
Router n0211 closes the connection (which is L2TP/IPsec), and as a result two other connections over L2TP/IPsec are also closed and immediately re-established. But why?
Code: Select all
2024-05-17T14:37:50+02:00 N0280 n0211-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.12} Connection closed
2024-05-17T14:37:50+02:00 N0280 n0211-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.12} Idle
2024-05-17T14:37:50+02:00 N0280 n0211-1 {l_addr: 172.22.32.126:179, r_addr: 172.22.32.12:38505} Entering OpenSent state
2024-05-17T14:37:50+02:00 N0280 n0211-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.12} Starter {openOk: false} Entering OpenConfirm state
2024-05-17T14:37:50+02:00 N0280 n0211-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.12} Starter {openOk: true} Entering Established state
2024-05-17T14:37:50+02:00 N0280 n0211-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.12} Established
2024-05-17T14:37:50+02:00 N0280 n0343-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.15} Idle
2024-05-17T14:37:50+02:00 N0280 n0344-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.16} Idle
2024-05-17T14:37:50+02:00 N0211 1 {l_addr: 172.22.32.12, r_addr: 172.22.32.126} Established
2024-05-17T14:37:51+02:00 N0343 n0280-l-1 {l_addr: 172.22.32.15, r_addr: 172.22.32.126} Connection closed
2024-05-17T14:37:51+02:00 N0343 n0280-l-1 {l_addr: 172.22.32.15, r_addr: 172.22.32.126} Idle
2024-05-17T14:37:51+02:00 N0344 n0280-l-1 {l_addr: 172.22.32.16, r_addr: 172.22.32.126} Connection closed
2024-05-17T14:37:51+02:00 N0344 n0280-l-1 {l_addr: 172.22.32.16, r_addr: 172.22.32.126} Idle
2024-05-17T14:37:51+02:00 N0343 n0280-l-1 {l_addr: 172.22.32.15:37289, r_addr: 172.22.32.126:179} Entering OpenSent state
2024-05-17T14:37:51+02:00 N0280 n0343-l-1 {l_addr: 172.22.32.126:179, r_addr: 172.22.32.15:37289} Entering OpenSent state
2024-05-17T14:37:51+02:00 N0280 n0343-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.15} Starter {openOk: false} Entering OpenConfirm state
2024-05-17T14:37:51+02:00 N0280 n0343-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.15} Starter {openOk: true} Entering Established state
2024-05-17T14:37:51+02:00 N0344 n0280-l-1 {l_addr: 172.22.32.16:36369, r_addr: 172.22.32.126:179} Entering OpenSent state
2024-05-17T14:37:51+02:00 N0343 n0280-l-1 {l_addr: 172.22.32.15, r_addr: 172.22.32.126} Starter {openOk: false} Entering OpenConfirm state
2024-05-17T14:37:51+02:00 N0343 n0280-l-1 {l_addr: 172.22.32.15, r_addr: 172.22.32.126} Starter {openOk: true} Entering Established state
2024-05-17T14:37:51+02:00 N0280 n0343-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.15} Established
2024-05-17T14:37:51+02:00 N0280 n0344-l-1 {l_addr: 172.22.32.126:179, r_addr: 172.22.32.16:36369} Entering OpenSent state
2024-05-17T14:37:51+02:00 N0280 n0344-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.16} Starter {openOk: false} Entering OpenConfirm state
2024-05-17T14:37:51+02:00 N0280 n0344-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.16} Starter {openOk: true} Entering Established state
2024-05-17T14:37:51+02:00 N0343 n0280-l-1 {l_addr: 172.22.32.15, r_addr: 172.22.32.126} Established
2024-05-17T14:37:51+02:00 N0344 n0280-l-1 {l_addr: 172.22.32.16, r_addr: 172.22.32.126} Starter {openOk: false} Entering OpenConfirm state
2024-05-17T14:37:51+02:00 N0344 n0280-l-1 {l_addr: 172.22.32.16, r_addr: 172.22.32.126} Starter {openOk: true} Entering Established state
2024-05-17T14:37:51+02:00 N0280 n0344-l-1 {l_addr: 172.22.32.126, r_addr: 172.22.32.16} Established
2024-05-17T14:37:51+02:00 N0344 input: in:l2tp out:(unknown 0), connection-state:invalid proto TCP (RST), 172.22.32.126:179->172.22.32.16:33881, len 40
2024-05-17T14:37:51+02:00 N0344 n0280-l-1 {l_addr: 172.22.32.16, r_addr: 172.22.32.126} EstablishedRe: BGP sessions close when another session to the same IP closes
Upgraded that router to 7.15.1 but the issue remains the same...
Re: BGP sessions close when another session to the same IP closes
Upgraded to 7.16 and now it has become much worse...
When a peer on L2TP/IPsec disconnects because their public IP has changed and they re-establish the L2TP/IPsec session, I have observed several times that all BGP sessions (15 total) go to Idle state and have to re-connect.
When a peer on L2TP/IPsec disconnects because their public IP has changed and they re-establish the L2TP/IPsec session, I have observed several times that all BGP sessions (15 total) go to Idle state and have to re-connect.
Re: BGP sessions close when another session to the same IP closes
I am seeing the same thing. GUYS, LETS GET THIS FIXED PLEASE!
Re: BGP sessions close when another session to the same IP closes
@mblfone - This is just a user forum. Please open a bug report with Mikrotik support.
Re: BGP sessions close when another session to the same IP closes
I have a ticket open since Jul 23 with ID SUP-159987 so you can also refer to that.
Who is online
Users browsing this forum: mikrotik4kvm3, sindy and 65 guests