BGP prefering local preference over routing distance

RouterOS Forwarding Protocols
1

Hello,

I came across a strange issue on Mikrotik Cloud Core routers CCR1036. Setup is quite simple, I use 2 routers on one location and 2 on the second, there are iBGP sessions between routers on same location, and eBGP sessions between 2 pairs of routers on remote locations. In that setup one router should preferably reach a network on remote location via eBGP, and in case of failure via iBGP and then the other link.
What actually happens is that router receives prefix via iBGP with default local preference set to 100 and ROUTER THEN PREFERS iBGP route (distance 200 over distance 20).
I tested this with route filtering and chaning announced local preference to 0, and it worked - router now prefers smaller distance parameter.

Below is the output with example of two subnets, one being active with greater distance (10.116.21.0), and the other being active as it should be after my intervention (10.116.23.0/24)

69 ADb dst-address=10.116.21.0/24 gateway=10.116.3.146 gateway-status=10.116.3.146 reachable via ether8-iBGP distance=200 scope=40 target-scope=30 bgp-local-pref=100
bgp-origin=igp received-from=MT-ZGALT-CORE2

70 Db dst-address=10.116.21.0/24 gateway=172.30.2.1 gateway-status=172.30.2.1 recursive via 10.116.4.22 sfp1G-Metronet_dark_fiber_ZG1 distance=20 scope=40 target-scope=30
bgp-as-path=“65002” bgp-origin=incomplete received-from=MT-ZG-CORE1


77 ADb dst-address=10.116.23.0/24 gateway=172.30.2.1 gateway-status=172.30.2.1 recursive via 10.116.4.22 sfp1G-Metronet_dark_fiber_ZG1 distance=20 scope=40 target-scope=30
bgp-as-path=“65002” bgp-origin=igp received-from=MT-ZG-CORE1

78 Db dst-address=10.116.23.0/24 gateway=10.116.3.146 gateway-status=10.116.3.146 reachable via ether8-iBGP distance=200 scope=40 target-scope=30 bgp-local-pref=0 bgp-origin=ig>
received-from=MT-ZGALT-CORE2


Is this an issue with Mikrotiks implementation of BGP, or am I missing something?

2

That’s how BGP is supposed to work.

Think of BGP routing in a similar way to how you might choose an airline ticket - you may drive to an airport that is much farther away if you like that flight better by some criteria (time, duration, cost, airline, layovers, etc). Once the flight is chosen, it is at a specific time at a specific airport (i.e. specific interface on a specific router) and then the only consideration is how to get to the airport. Of course the trouble of getting to the airport might be too much, so you also take that into account when choosing the flight - if you had two flights that were otherwise equally good, you’d choose the more convenient airport.

So in very broad terms, Local preference is how your whole network can be set to follow a certain policy (e.g. “send traffic to YouTube through Level3, but use Hurricane Electric as the default”).

Local Preference is the most important metric in the BGP path selection algorithm-
Except Cisco created an even higher priority metric, weight, which Mikrotik also uses. Weight is strictly internal to one particular router (there’s no “weight” attribute in BGP messages on the wire)

So among equal-length prefixes, BGP will chose:
1: highest weight
2: highest LOCAL_PREF (this value is passed along in your IBGP so it’s basically system-wide)
3: prefer originated routes over aggregated routes
4: AS Path length
5: route source = interior routing, exterior routing, unknown source
6: lowest MED (distance to target within the next network - only avail for multi-homing to same provider)
7: EBGP > IBGP
8: closest next-hop
9: older advertisements > newer advertisements
10: tiebreaker: lowest router_id that announced the route
11: tiebreaker: lowest peer IP address (if peering to same router over multiple connections)

As you can see, AS_PATH is far from being the most important criteria.

In the big leagues, routing policy is done with LOCAL_PREF and this can be tweaked by sending and receiving agreed-upon communities. Networks can send communities that mean things like “this is one of my customer’s networks” or “this is one of my transit carrier’s networks” etc - so your in-filter for a neighbor might set a high LOCAL_PREF for “customer networks” from carrier A, and a low LOCAL_PREF for their upstream routes, so that you use the link to ISP1 when going to their customers, but otherwise, they’re a backup… And you can send communities to them as well.

They often have communites that mean they should AS_PREPEND a number of times when passing your route along to customers/providers/peers, or that they shouldn’t pass your advertisements to other networks (private peering, for instance).

In general, most carriers’ routing policies I’ve read try to follow the basic idea that they send traffic to customers first, peers second, and paid transport providers last. So - if carrier X receives your advertisement from two paths, one being a short AS-PATH but from a peer network, and the other being heavily AS-prepended but from a customer path, then that’s going to win. Of course, other networks on the other side of this carrier would see the heavily-prepended AS path and probably choose not to route via this carrier, but being a carrier, they’re going to have lots of customers, and those customers are going to go straight to you, no matter how much you as-prepend.

I know this has been a long post, but I hope I’ve helped shed some light on how BGP routing policy is done.

3

Thank you for your answer, but I know how BGP inbound and outbound route manipulations work. I actually asked why does MT prefer routes with worse administrative distance..

See that eBGP has better (smaller) distance 20 than iBGP, but
Route Source Default Distance Values
Connected interface 0
Static route 1
Enhanced Interior Gateway Routing Protocol (EIGRP) summary route 5
External Border Gateway Protocol (BGP) 20
Internal EIGRP 90
IGRP 100
OSPF 110
Intermediate System-to-Intermediate System (IS-IS) 115
Routing Information Protocol (RIP) 120
Exterior Gateway Protocol (EGP) 140
On Demand Routing (ODR) 160
External EIGRP 170
Internal BGP 200

4

Look at BGP best path selection algorithm
http://wiki.mikrotik.com/wiki/Manual:BGP_Best_Path_Selection_Algorithm

  1. Prefer the path with the lowest ORIGIN type.
    Interior Gateway Protocol (IGP) is lower than Exterior Gateway Protocol (EGP), and EGP is lower than INCOMPLETE

in other words IGP < EGP < INCOMPLETE

…

  1. Prefer eBGP over iBGP paths

Origin is compared before iBGP, eBGP path selection

5

ok,

Maybe I wasn’t clear enough in my question..

let’s say you have several routes learned from various dynamic routing protocols:
a router should prefer the one with the lowest distance, right?

so if I have OSPF route (110) it’s better than IS-IS (115). ok?

so when I have eBGP route with distance of 20 it should be always used as best candidate as long as it’s valid and prefer it over an iBGP learned one (as it has distance of 200)

Hope I made myself clear now..

6

Gotcha - basically, BGP picks the winning route and puts it in the routing table.
If the winning route happened to come from iBGP, then the route goes into the routing table with a distance of 200.
If the winning route happened to come from eBGP, then the route goes into the routing table with a distance of 20.

The BGP routes must compete with each other first. Once in the routing table, the AD determines whether the BGP route wins or some other protocol like OSPF wins. (in other words, Administrative distance is the RIB’s metric, and isn’t considered at all by BGP)

Consider Router1 has an ebgp peer with a poor route to some destination X, and Router2 has a great route to destination X. Router 1 would have two routes to X, one with distance 20 via the local ebgp peer, and one with distance 200 via ibgp from router2 → peer2… Since peer2’s route is better in the BGP world, that is the route that wins, regardless of AD.

If destination X is also in OSPF, then router1 should end up using the OSPF route for X as long as BGP prefers router2 because the AD of OSPF > the AD of iBGP.

EDIT: Being primarily a Cisco guy, the Mikrotik method of using the RIB for everything (as opposed to a separate BGP table) has confused me a few times like this, but when I stop and think that in Cisco world, I would never see inferior BGP paths in the RIB - it helped me.

7

Man this is soo true :slight_smile:. It took me a little while as well when I was first learning the differences between Cisco and MikroTik BGP to understand the MikroTik RIB/FIB vs. Cisco RIB/FIB. I was also mostly a Cisco engineer until a few years ago and when MikroTIk routers would advertise prefixes that weren’t in the routing table it drove me nuts for a while :slight_smile:

8

ok, thank you all for comments! I am working for almost a year with Mikrotik (after few years of Cisco, HP and Juniper) and it doesn’t stop surprising me :smiley:

9

Is Cisco, Juniper method better than what our current implementation?

10

No offense - I love Mikrotik but honestly, yes I believe their (Cisco, Juniper, etc.) implementation is better. RouterOS has some subtle glitches that make things broken.

For example: Try originating default route into OSPF based on a floating static backup route on router B. It won’t release the floating static if the primary comes from router A via OSPF.

Another is how ROS won’t repeat a default route prefix in iBGP, but forces you to originate a new one. This can lead to routing loops that repeating the original one from eBGP wouldn’t cause.

11

@ZeroByte Your mentioned problems are not related to current topic. These problems will be solved in ROSv7.

My question was is it better that distance is compared first? Any example where our method is not acceptable?

12

I’ve never seen anything wrong with the ROS bgp path selection algorithm. In fact, distance should never be considered by BGP. That’s a RIB metric and doesn’t exist in BGP. You’ll note that IPANetEngineer and I are both explaining to the OP why it’s normal for a distance 200 route to beat a distance 20 route when they’re both BGP routes.

We’re just saying that seeing the losers from the BGP table sitting in the RIB as inactive routes can be confusing.

13

Hi all,

I don’t get it. I have two defaul routes, one form OSPF and one form eBGP, RouterOS selects the OSPF one why?

[fulvio.allegretti@rtr-mad-01] /routing ospf lsa> /ip route print detail where dst-address=0.0.0.0/0
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
0 ADo dst-address=0.0.0.0/0 gateway=a.b.c.d gateway-status=a.b.c.d reachable via PMI-MAD1 distance=110 scope=20 target-scope=10 route-tag=1000 ospf-metric=10 ospf-type=external-type-2

1 Db dst-address=0.0.0.0/0 gateway=x.y.z.t gateway-status=x.y.z.t reachable via ISP_BRKOUT distance=20 scope=40 target-scope=10 bgp-as-path=“12357” bgp-local-pref=910 bgp-origin=igp
received-from=ISP_INET

Surely eBGP with distance of 20 should be selected. What am I missing?

14

update and reboot?

15

:frowning: live environment, but yes I had the same reaction.
As it happens I brought up a new BGP peer, internal, and the router pushed the eBGP def route into the routing table, I would say having to go through the selection alorithm again made it realised it had a better distance default route. Of course this is all speculation which it seems to me it’s all one can do with MT sometime

16

@mrz: can you please clarify?

So the admin distance is compared first and then parameters within the same routing protocol?

If I have the same route from OSPF / iBGP (local pref 200) and eBGP; following the logic above it eBGP would win the admin distance comparison and then iBGP would win the BGP fight so I would end up with iBGP.

@ZeroByte - your explanation Thu Dec 17, 2015 5:13 pm

Sounds like Cisco logic: compare parameters within the routing protocol and puts (I suppose Cisco presents it to the RIB) the winner to the RIB; a second competition starts in the RIB based on admin distance; best admin distance wins. I am failing to see the difference in the logic; I understand the implementation is different as Cisco has multiple tables, one per protocol

17

That’s indeed how it should work according to RFC 4271. RouterOS however does not care for the IGP metrics to the next hop:

https://wiki.mikrotik.com/wiki/Manual:BGP_Best_Path_Selection_Algorithm

As such you cannot use RouterOS in any environment with redundant paths from BGP point of view (multiple next hops for same prefix) since RouterOS cannot fall back on the IGP metrics to chose the best next hop.

18

Actually RouterOS is not RFC4271 compliant since it does not take IGP metrics into account in its BGP path selection algorithm. BGP normally requires to perform a recursive lookup of the distance to multiple alternative next-hops and to chose the shortest path according to the IGP.

https://tools.ietf.org/html/rfc4271
"9.1.2.2. Breaking Ties (Phase 2)

In its Adj-RIBs-In, a BGP speaker may have several routes to the same
destination that have the same degree of preference. The local
speaker can select only one of these routes for inclusion in the
associated Loc-RIB. […]

e) Remove from consideration any routes with less-preferred
interior cost. The interior cost of a route is determined by
calculating the metric to the NEXT_HOP for the route using the
Routing Table. […]"