Community discussions

MikroTik App
  4. RouterOS
  5. Forwarding Protocols

BGP/OSPF interaction weiredness (simple lab setup)

Post Reply
 
User avatar
peterh
just joined
Topic Author
Posts: 21
Joined: Tue Dec 11, 2018 7:39 pm

BGP/OSPF interaction weiredness (simple lab setup)

Sat Jan 05, 2019 4:08 pm

Another day, another BGP/OSPF interaction weirdness under RouterOS 6.43.8. Very simple lab setup:
Code: Select all
[Router R1 AS65000]---eBGP---[Router R2 AS65010]---iBGP/OSPF---[Router R3 AS65010]
    |                            |                                 |
[1.1.1.0/24]                 [2.2.2.0/24]                      [3.3.3.0/24]
Everything working fine, all three routers can see each other's connected networks:
1.1.1.0/24
2.2.2.0/24
3.3.3.0/24
:)
### R1 ###
Code: Select all
# jan/ 5/2019 13:31:44 by RouterOS 6.43.8
# software id = 13FD-KY09
#
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADC  1.1.1.0/24         1.1.1.1         br1                       0
 1 ADb  2.2.2.0/24                         10.1.2.2                 20
 2 ADb  3.3.3.0/24                         10.1.2.2                 20
 3 ADC  10.1.2.0/24        10.1.2.1        ether1                    0
 4 ADC  192.168.99.1/32    192.168.99.1    brlo                      0
### R2 ###
Code: Select all
# jan/ 5/2019 13:32:16 by RouterOS 6.43.8
# software id = P955-524K
#
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  1.1.1.0/24                         10.1.2.1                 20
 1 ADC  2.2.2.0/24         2.2.2.1         br2                       0
 2 ADb  3.3.3.0/24                         192.168.99.3            200
 3 ADC  10.1.2.0/24        10.1.2.2        ether1                    0
 4 ADC  10.2.3.0/24        10.2.3.1        ether2                    0
 5 ADC  192.168.99.2/32    192.168.99.2    brlo                      0
 6 ADo  192.168.99.3/32                    10.2.3.2                110
### R3 ###
Code: Select all
# jan/ 5/2019 13:33:23 by RouterOS 6.43.8
# software id = 6CCJ-2JSK
#
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  1.1.1.0/24                         192.168.99.2            200
 1 ADb  2.2.2.0/24                         192.168.99.2            200
 2 ADC  3.3.3.0/24         3.3.3.0         br3                       0
 3 ADC  10.2.3.0/24        10.2.3.2        ether1                    0
 4 ADo  192.168.99.2/32                    10.2.3.1                110
 5 ADC  192.168.99.3/32    192.168.99.3    brlo                      0
...
...
:lol:Now, let's enter an evil magic spell on router R3:
Code: Select all
/routing ospf network add network=3.3.3.0/24 area=backbone
:lol:
....
OK, simple stuff, that evil spell. We are just advertising 3.3.3.0/24 not only through BGP (as we already did before), but now additionally via OSPF. More is more, right? Two paths to the same destination to chose from - one advertised via BGP, one via OSPF. No problem with that, should have no influence on the reachability of any of the three networks, right?
1.1.1.0/24
2.2.2.0/24
3.3.3.0/24

Wrong! eBGP on R2 stops advertising 3.3.3.0/24! Apparently because the OSPF route is the active one (lower admin distance), which causes the iBGP route to become inactive. And R2 does not seem to like inactive iBGP routes - at least it stops advertising them, even though it is a perfectly valid path! :(

### R1 - missing 3.3.3.0/24 :( ####
Code: Select all
# jan/ 5/2019 13:48:17 by RouterOS 6.43.8
# software id = 13FD-KY09
#
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADC  1.1.1.0/24         1.1.1.1         br1                       0
 1 ADb  2.2.2.0/24                         10.1.2.2                 20
 2 ADC  10.1.2.0/24        10.1.2.1        ether1                    0
 3 ADC  192.168.99.1/32    192.168.99.1    brlo                      0
### R2 - does no longer advertise 3.3.3.0/24 via BGP because the OSPF route has become active :( ####
Code: Select all
# jan/ 5/2019 13:48:25 by RouterOS 6.43.8
# software id = P955-524K
#
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  1.1.1.0/24                         10.1.2.1                 20
 1 ADC  2.2.2.0/24         2.2.2.1         br2                       0
 2 ADo  3.3.3.0/24                         10.2.3.2                110
 3  Db  3.3.3.0/24                         192.168.99.3            200
 4 ADC  10.1.2.0/24        10.1.2.2        ether1                    0
 5 ADC  10.2.3.0/24        10.2.3.1        ether2                    0
 6 ADC  192.168.99.2/32    192.168.99.2    brlo                      0
 7 ADo  192.168.99.3/32                    10.2.3.2                110
### R3 - normal ####
Code: Select all
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  1.1.1.0/24                         192.168.99.2            200
 1 ADb  2.2.2.0/24                         192.168.99.2            200
 2 ADC  3.3.3.0/24         3.3.3.0         br3                       0
 3 ADC  10.2.3.0/24        10.2.3.2        ether1                    0
 4 ADo  192.168.99.2/32                    10.2.3.1                110
 5 ADC  192.168.99.3/32    192.168.99.3    brlo                      0
...
Works as designed?
Top
 
TheCiscoGuy
Frequent Visitor
Frequent Visitor
Posts: 51
Joined: Fri Jun 22, 2018 8:32 am

Re: BGP/OSPF interaction weiredness (simple lab setup)

Sun Jan 06, 2019 4:58 am

Can you provide the bgp config for router 2
Top
 
User avatar
peterh
just joined
Topic Author
Posts: 21
Joined: Tue Dec 11, 2018 7:39 pm

Re: BGP/OSPF interaction weiredness (simple lab setup)

Sun Jan 06, 2019 1:31 pm

Full config of all 3 routers:

### R1 ###
Code: Select all
# jan/04/2019 23:28:08 by RouterOS 6.43.8
# software id = 13FD-KY09
#
#
#
/interface bridge
add name=br1
add name=brlo
/routing bgp instance
set default as=65000 router-id=192.168.99.1
/ip address
add address=192.168.99.1 interface=brlo network=192.168.99.1
add address=1.1.1.1/24 interface=br1 network=1.1.1.0
add address=10.1.2.1/24 interface=ether1 network=10.1.2.0
/ip cloud
set update-time=no
/routing bgp network
add network=1.1.1.0/24
/routing bgp peer
add name=r2 remote-address=10.1.2.2 remote-as=65010 update-source=ether1
/system identity
set name=r1
### R2 ###
Code: Select all
# jan/04/2019 23:28:18 by RouterOS 6.43.8
# software id = P955-524K
#
#
#
/interface bridge
add name=br2
add name=brlo
/routing bgp instance
set default as=65010 router-id=192.168.99.2
/routing ospf instance
set [ find default=yes ] router-id=192.168.99.2
/ip address
add address=192.168.99.2 interface=brlo network=192.168.99.2
add address=2.2.2.1/24 interface=br2 network=2.2.2.0
add address=10.1.2.2/24 interface=ether1 network=10.1.2.0
add address=10.2.3.1/24 interface=ether2 network=10.2.3.0
/ip cloud
set update-time=no
/routing bgp network
add network=2.2.2.0/24
/routing bgp peer
add name=r1 remote-address=10.1.2.1 remote-as=65000 update-source=ether1
add name=r2 nexthop-choice=force-self remote-address=192.168.99.3 remote-as=\
    65010 update-source=brlo
/routing ospf network
add area=backbone network=192.168.99.2/32
add area=backbone network=10.2.3.0/24
/system identity
set name=r2
### R3 ####
Code: Select all
# jan/04/2019 23:28:26 by RouterOS 6.43.8
# software id = 6CCJ-2JSK
#
#
#
/interface bridge
add name=br3
add name=brlo
/routing bgp instance
set default as=65010
/routing ospf instance
set [ find default=yes ] router-id=192.168.99.3
/ip address
add address=192.168.99.3 interface=brlo network=192.168.99.3
add address=3.3.3.0/24 interface=br3 network=3.3.3.0
add address=10.2.3.2/24 interface=ether1 network=10.2.3.0
/ip cloud
set update-time=no
/routing bgp network
add network=3.3.3.0/24
/routing bgp peer
add name=r2 remote-address=192.168.99.2 remote-as=65010 update-source=brlo
/routing ospf network
add area=backbone network=192.168.99.3/32
add area=backbone network=10.2.3.0/24
add area=backbone network=3.3.3.0/24
/system identity
set name=r3
Top
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:
Contact StubArea51

Re: BGP/OSPF interaction weiredness (simple lab setup)

Sun Jan 06, 2019 6:43 pm

Nothing weird about this. R2 is going to pick the best route for prefix 3.3.3.0/24 to put into the FIB and it has two choices.

1) An iBGP route with an admin distance of 200
2) An OSPF intra-area route with an admin distance of 110

It's going to pick the OSPF route which means there will be no active route in BGP to announce to the eBGP peer. This is normal and intended behavior - you'll see the same behavior with three routers in Cisco or Juniper set up in the same fashion. The admin distance will always win when the prefix lengths are the same and if there is no active BGP route for a prefix and no network statement or redistribution to inject 3.3.3.0/24 from OSPF, then it won't be advertised.

Read up in the wiki about route selection

https://wiki.mikrotik.com/wiki/Manual:I ... _selection
Top
 
User avatar
peterh
just joined
Topic Author
Posts: 21
Joined: Tue Dec 11, 2018 7:39 pm

Re: BGP/OSPF interaction weiredness (simple lab setup)

Sun Jan 06, 2019 10:53 pm

Implemented the same setup using Vyos. It does NOT show the RouterOS behaviour:

### R1 has all routes to 1.1.1.0/24, 2.2.2.0/24, 3.3.3.0/24 ###
Code: Select all
vyos@r1:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route

C>* 1.1.1.0/24 is directly connected, br1
B>* 2.2.2.0/24 [20/1] via 10.1.2.2, eth0, 00:02:06
B>* 3.3.3.0/24 [20/0] via 10.1.2.2, eth0, 00:02:06
C>* 10.1.2.0/24 is directly connected, eth0
C>* 127.0.0.0/8 is directly connected, lo
C>* 192.168.99.1/32 is directly connected, lo

### R2 - OSPF route for 3.3.3.0/24 is active/selected, BGP route is inactive - BUT 3.3.3.0/24 is still advertised via eBGP! ###
Code: Select all
vyos@r2:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route

B>* 1.1.1.0/24 [20/1] via 10.1.2.1, eth0, 00:04:23
C>* 2.2.2.0/24 is directly connected, br2
O>* 3.3.3.0/24 [110/20] via 10.2.3.2, eth1, 00:10:49
B   3.3.3.0/24 [200/1] via 192.168.99.3 (recursive via 10.2.3.2), 00:12:33
C>* 10.1.2.0/24 is directly connected, eth0
O   10.2.3.0/24 [110/10] is directly connected, eth1, 00:19:10
C>* 10.2.3.0/24 is directly connected, eth1
C>* 127.0.0.0/8 is directly connected, lo
O   192.168.99.2/32 [110/10] is directly connected, lo, 00:19:10
C>* 192.168.99.2/32 is directly connected, lo
O>* 192.168.99.3/32 [110/20] via 10.2.3.2, eth1, 00:18:24

vyos@r2:~$ show ip bgp neighbors 10.1.2.1 advertised-routes 
BGP table version is 0, local router ID is 192.168.99.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2.2.2.0/24       10.1.2.2                 1         32768 i
*> 3.3.3.0/24       10.1.2.2                      100      0 i

Total number of prefixes 2

### R3 - is "poisoned" with an "set protocol ospf area 0 network 3.3.3.0/24" - Still everything works fine! ###
Code: Select all
vyos@r3:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route

B>* 1.1.1.0/24 [200/1] via 192.168.99.2 (recursive via 10.2.3.1), 00:07:49
B>* 2.2.2.0/24 [200/1] via 192.168.99.2 (recursive via 10.2.3.1), 00:20:35
O   3.3.3.0/24 [110/10] is directly connected, br3, 00:14:21
C>* 3.3.3.0/24 is directly connected, br3
O   10.2.3.0/24 [110/10] is directly connected, eth0, 00:22:36
C>* 10.2.3.0/24 is directly connected, eth0
C>* 127.0.0.0/8 is directly connected, lo
O>* 192.168.99.2/32 [110/20] via 10.2.3.1, eth0, 00:21:46
O   192.168.99.3/32 [110/10] is directly connected, lo, 00:22:36
C>* 192.168.99.3/32 is directly connected, lo
Whose implementation is correct?
Top
 
User avatar
peterh
just joined
Topic Author
Posts: 21
Joined: Tue Dec 11, 2018 7:39 pm

Re: BGP/OSPF interaction weiredness (simple lab setup)

Mon Jan 07, 2019 12:55 am

It's going to pick the OSPF route which means there will be no active route in BGP to announce to the eBGP peer. This is normal and intended behavior - you'll see the same behavior with three routers in Cisco or Juniper set up in the same fashion.

I also checked it now under Cisco IOS. IOS does by default not care which route has been selected as active. It still advertises the BGP route even if the OSPF one has been selected as active.

### R1 - Sees all routes to 1.1.1.0/24, 2.2.2.0/24, 3.3.3.0/24 via eBGP learned from R2 ###
Code: Select all
r1#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default, U - per-user static route, o - ODR
       P - periodic downloaded static route

Gateway of last resort is not set

     1.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       1.1.1.0/24 is directly connected, GigabitEthernet0/2
L       1.1.1.1/32 is directly connected, GigabitEthernet0/2
     2.0.0.0/24 is subnetted, 1 subnets
B       2.2.2.0/24 [20/0] via 10.1.2.2, 00:00:00
     3.0.0.0/24 is subnetted, 1 subnets
B       3.3.3.0/24 [20/0] via 10.1.2.2, 00:00:00
     10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       10.1.2.0/24 is directly connected, GigabitEthernet0/0
L       10.1.2.1/32 is directly connected, GigabitEthernet0/0
     192.168.99.0/32 is subnetted, 1 subnets
C       192.168.99.1/32 is directly connected, Loopback0

### R2 - OSPF route for 3.3.3.0/24 is selected, yet eBGP still announces 3.3.3.0/24 to R1 ####
Code: Select all
r2#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default, U - per-user static route, o - ODR
       P - periodic downloaded static route

Gateway of last resort is not set

     1.0.0.0/24 is subnetted, 1 subnets
B       1.1.1.0/24 [20/0] via 10.1.2.1, 00:00:00
     2.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       2.2.2.0/24 is directly connected, GigabitEthernet0/2
L       2.2.2.1/32 is directly connected, GigabitEthernet0/2
     3.0.0.0/24 is subnetted, 1 subnets
O       3.3.3.0/24 [110/2] via 10.2.3.2, 00:00:09, GigabitEthernet0/1
     10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
C       10.1.2.0/24 is directly connected, GigabitEthernet0/0
L       10.1.2.2/32 is directly connected, GigabitEthernet0/0
C       10.2.3.0/24 is directly connected, GigabitEthernet0/1
L       10.2.3.1/32 is directly connected, GigabitEthernet0/1
     192.168.99.0/32 is subnetted, 2 subnets
C       192.168.99.2/32 is directly connected, Loopback0
O       192.168.99.3/32 [110/2] via 10.2.3.2, 00:00:09, GigabitEthernet0/1

It is apparently possible to make IOS behave like RouterOS, by issuing "bgp suppress-inactive". But by default, Cisco IOS advertises BGP routes which have NOT been been selected as "active" in the RIB due to competing routes with better administrative distance.

https://www.cisco.com/c/en/us/support/d ... bgp-s.html
It has been demonstrated that by default, BGP continues to advertise routes in RIB-Failure condition. Those are routes received via BGP and that are not installed in the Routing Table. The bgp suppress-inactive command is available to modify this behavior.
https://blog.ipspace.net/2007/12/what-i ... ilure.html
The BGP routes that are not used due to higher administrative distance are still advertised to all BGP peers (contrary to what most other distance-vector routing protocols do), unless you configure bgp suppress-inactive (introducted in 12.2T and 12.0(26)S).
https://mellowd.co.uk/ccie/?p=371
Note that rib failure prefixes are still advertised to BGP neighbours. This is not like EIGRP and RIP who will not. You can however prevent BGP from advertising rib-failure routes by configuring bgp suppress-inactive under the BGP process.
Top
Post Reply

Who is online

Users browsing this forum: No registered users and 15 guests
  4. RouterOS
  5. Forwarding Protocols