IPv6 recursive nexthops via iBGP

RouterOS Forwarding Protocols
1

Hi,

We recently added a new RouterOS device to our network and as we use iBGP and eBGP, this was the first instance where we have had to use multihop iBGP (all devices were directly connected to each other at this point; this new device is only reachable via routed IPv4/IPv6).

All devices are running RouterOS v4.10.

The problem that I’m seeing is that IPv4 multihop iBGP works absolutely fine as long as the ‘target-scope’ for a route is set in the incoming filter for that session; routes appear in the routing table as I would expect them to:

‘a.a.a.a recursive via b.b.b.b ether1’

Before I figured out that ‘target-scope’ was the issue, I was seeing a.a.a.a unreachable’ instead - nevertheless, IPv4 multihop iBGP is working fine now so I am a happy bunny.

Unfortunately, IPv6 multihop iBGP has got me well and truly stumped.

The same tweak for ‘target-scope’ on the incoming filter for the v6-only iBGP session does not have the same effect; the filter is definitely working as I am seeing the changes in the ‘Attributes’ tab in Winbox for all of the routes learnt via that session.

Is anyone out there in RouterOS land willing to share any tips/suggestions/example config that might help me solve this ?

I am happy to post config details if this will help.

Thanks in advance!

Regards,
Terry Froy
Spilsby Internet Solutions

2

try setting the next hop or gateway using routing filters to override whats coming in.

3

please print one of the IPv6 unresolved routes.

4

That worked - once I had sorted out some quick VRRP to ensure that the nexthop wasn’t a specific routers’ IPv6 address; no point in doing BGP if it can’t failover to an alternative route!

Reverting back to previous config; I get the following:

[admin@tcw-gw1.man.spilsby.net.uk] > /ipv6 route print where dst-address is 2a01:568:3000::/64
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable 
 #      DST-ADDRESS              GATEWAY                  DISTANCE
[admin@tcw-gw1.man.spilsby.net.uk] > /ipv6 route print where dst-address is "2a01:568:3000::/64"
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable 
 #      DST-ADDRESS              GATEWAY                  DISTANCE
[admin@tcw-gw1.man.spilsby.net.uk] >

Going by the above, the route doesn’t actually appear via the console but it does appear within Winbox (fields and values copied below):

Dst. Address: 2a01:568:3000::/64
Gateway: 2a01:568:fff:f003:20c:42ff:fe43:739c (unreachable)
Check Gateway: (empty)
Type: unicast

Distance: 200
Scope: 40
Target Scope: 255 (this is being set via incoming route filter like its’ IPv4 cousin)

Received From: hex-ipv6-int-gw1

BGP Local Pref.: 100
BGP Origin: EGP (now this confuses me; both routers and their single BGP instances each specify the same AS number - 43950 - so why this is being considered as EGP, I don’t know - this should be igp, right ?)

Both routers can ping each other and I can confirm there are currently no traffic filters between them; these only get put in place after issues like this are fixed otherwise you end up diagnosing problems that may not actually exist, etc, etc.

Enabling ‘Check Gateway’ with method ‘ping’ doesn’t seem to provide any other useful information either :frowning:

Although, just to assure you that I’m not going mad:

[admin@tcw-gw1.man.spilsby.net.uk] > /ping 2a01:568:fff:f003:20c:42ff:fe43:739c
2a01:568:fff:f003:20c:42ff:fe43:739c 64 byte ping: ttl=63 time=8 ms
2a01:568:fff:f003:20c:42ff:fe43:739c 64 byte ping: ttl=63 time=8 ms
2a01:568:fff:f003:20c:42ff:fe43:739c 64 byte ping: ttl=63 time=8 ms
2a01:568:fff:f003:20c:42ff:fe43:739c 64 byte ping: ttl=63 time=8 ms
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 8/8.0/8 ms
[admin@tcw-gw1.man.spilsby.net.uk] > /tool traceroute 2a01:568:fff:f003:20c:42ff:fe43:739c
     ADDRESS                                    STATUS
   1 2a01:568:fff:f001:20c:42ff:fe20:7ee4 1ms 1ms 1ms 
   2 2a01:568:fff:f003:20c:42ff:fe43:739c 9ms 9ms 9ms

… and peer config:

[admin@tcw-gw1.man.spilsby.net.uk] > /routing bgp peer export
<snip countless other IPv4/IPv6 peers>
add address-families=ipv6 as-override=no comment="" default-originate=never \
    disabled=no hold-time=3m in-filter=spilsby-tcw-ipv6-multihop-ibgp-in \
    instance=default interface=ether1 multihop=yes name=hex-ipv6-int-gw1 \
    nexthop-choice=force-self out-filter="" passive=no remote-address=\
    2a01:568:fff:f003:20c:42ff:fe43:739c remote-as=43950 remove-private-as=no \
    route-reflect=no tcp-md5-key="" ttl=default use-bfd=no

Config on the remote router is identical (except remote peer address of course!) and a full iBGP mesh is configured between all routers on this AS (all of which are RouterOS 4.10 devices - either RB1000 or RB1100).

Regards,
Terry Froy
Spilsby Internet Solutions

5

Replying to my own post…

VRRP wasn’t actually necessary once I realized that a combination of multiple IPv6 nexthops could be set and ‘Check Gateway’ → ‘ping’ would handle any failover automatically.

Quite impressed that it actually worked :wink:

Regards,
Terry Froy
Spilsby Internet Solutions[/quote]

6

Anyone manage to solve this?

Looks like theres no communication between OSPFv3 and BGP so BGP isnt seeing the interior route?

[admin@hostname] /ipv6 route> print detail where distance=200
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, o - ospf, b - bgp, U - unreachable
0 Db dst-address=2000:2000::/32 gateway=2403:9000::1:0:0:1000:3
gateway-status=2403:9000::1:0:0:1000:3 unreachable distance=200
scope=40 target-scope=30 bgp-as-path=“AS_PATH” bgp-local-pref=1000
bgp-med=0 bgp-origin=igp received-from=IPv6-ACCESS01

1 Db dst-address=2000:2000::/32 gateway=2403:9000::1:0:0:1000:3
gateway-status=2403:9000::1:0:0:1000:3 unreachable distance=200
scope=40 target-scope=30 bgp-as-path=“AS_PATH” bgp-local-pref=1000
bgp-med=0 bgp-origin=igp received-from=IPv6-ACCESS02

21 ADo dst-address=2403:9000::1:0:0:1000:0/124 gateway=fe80::222:90ff:fe0c:25f1%vlan0621
gateway-status=fe80::222:90ff:fe0c:25f1%vlan0621 reachable distance=110 scope=20 target-scope=10
ospf-metric=10

7

This is a known bug.
Recursive lookup is not working if gateway is link local address. To make BGP routes work you need static route with global address as gateway.

8

Do you plan to fix it or introduce an official workaround for it in an upcoming release?

9

You can’t announce anything with link local addresses in them, they are after all, link local.
Use non link-locals for routing (even for gateway).

10

Hello sten,

Can you provide me with an example as to how to do that? Try as I might, I cannot force my MTs to accept a global unique next hop over a link-local one…

11

Hi,

Is there any update on this? I just faced this bug, in 5.18! This is really basic routing stuff, working on every other system I know. What’s the problem that RouterOS cannot handle this? OSPFv3 nexthops are always link-local so there is no workaround at all I guess.

It’s best practice to use loopback addresses for iBGP, routing the loopback addresses with OSPFv3.

when will this bug be fixed? This is actually a total no-go for MikroTik as BGP Routers (at least if you have more than 1 router).

thanks
marco

12

Hi,

I have the same problem (running 5.17)

OSPFv3 routes have link-local nexthops (as per RFC) and are not used to recursively resolve nexthops of iBGP routes!

Route filters (using set-in-nexthop-ipv6 in ospf-in chain) are a workaround only if you have no more than one OSPF neighbor because the filter can’t determine from which neighbor the route came from and set the nexthop accordingly.

Can we hope this is fixed soon?

The iBGP route with the unresolved nexhop:

35  Db  dst-address=2001:db8::/32 gateway=fc00::2 
        gateway-status=fc00::2 unreachable distance=200 scope=40 
        target-scope=30 bgp-as-path="65000" bgp-local-pref=100 bgp-origin=igp 
        received-from=ibgp_router2

The route from OSPFv3 that should be used to resolve the nexthop but is not:

32 ADo  dst-address=fc00::2/128 gateway=fe80::a61:702%sit4-vr1 
        gateway-status=fe80::a61:702%sit4-vr1 reachable distance=110 scope=20 
        target-scope=10 ospf-metric=10
13

Hi,

Months later, routeros 6.0 is released and this problem is still not solved. Did anyone find a good workaround besides adding static routes? static routes is not really a workaround because static routes simply do not work in a redundant network layout!

I would really appriciate to know when this problem is fixed. It’s actually a total no-go for BGP dual-stacked backbones with MikroTIK.

thanks
Marco

14

I also am having big troubles with RouterOS and iBGP with IPv6 due to this issue. Please Mikrotik, consider fixing it!

15

You may also write an e-Mail to MT Support about this issue.

It’s really sad that such basic stuff does not work. MikroTik needs to change thinking about IPv6, it’s nothing new, it’s there, years ago and every ISP has or is impelmenting it, now. MikroTik can only be a full alternative to cisco, etc. in the ISP branch when it has at least the simple and basic routing availible and working (and yes, ipv6 is simple).
Please fix it.

thanks

16

I already mailed them a couple of months ago, when I replaced some cisco backbone infrastructure with CCR routers (~ 25 CCR). They answered that it will be fixed in the future.

Actually I am having really big troubles. I started to deploy IPv6 to customers in 2008, and now I have a network with all backbone links and router loopbacks distributed with OSPF and customers routes into BGP. The problem is that it’s impossible to use IBGP with IPv6. I can’t even redistribute BGP routes into OSPFv3 because OSPFv3 lacks of route-filtering. Actually I had to put a huge number of static routes (!!!) in order to make the iBGP route reachable, and this sucks because static routes are indeed STATIC and it’s a nightmare to do maintenance on it.

Probably Mikrotik is underevaluating the importance of this issue. I just hope that they will fix it soon.

17

The problem still exists in RouterOS 6.3. Upgrading to 6.4 later in the week.

18

Still occurs with 6.5rc1. I emailed support@ and was told that I simply need to tweak my scope and target-scope values. Not sure what I’m supposed to do in that regard.

19

You do not have problem discussed in this topic. Recursive lookup doesn’t work only when nexthop is link-local. You had global nexthops.

20

I might be interpreting things wrong, but it looks like the next hops are link local.

 0  Db  dst-address=2001:470:1f10:9a7::/64 gateway=2001:470:c334::b
        gateway-status=2001:470:c334::b unreachable distance=200 scope=40 target-scope=30
        bgp-local-pref=100 bgp-origin=igp received-from=fhl-r3

 1 ADC  dst-address=2001:470:c334::9/128 gateway=loopback gateway-status=loopback reachable
        distance=0 scope=10

 2 ADo  dst-address=2001:470:c334::a/128 gateway=fe80::20c:42ff:feff:5c7d%ether5
        gateway-status=fe80::20c:42ff:feff:5c7d%ether5 reachable distance=110 scope=20
        target-scope=10 ospf-metric=20

 3 ADo  dst-address=2001:470:c334::b/128 gateway=fe80::d6ca:6dff:fe35:d345%ether1
        gateway-status=fe80::d6ca:6dff:fe35:d345%ether1 reachable distance=110 scope=20
        target-scope=10 ospf-metric=20

 4 ADo  dst-address=2001:470:c334::13/128 gateway=fe80::20c:42ff:feff:5c7d%ether5
        gateway-status=fe80::20c:42ff:feff:5c7d%ether5 reachable distance=110 scope=20
        target-scope=10 ospf-metric=30

 5 ADC  dst-address=2001:470:c334:1::/64 gateway=ether1 gateway-status=ether1 reachable distance=0
        scope=10

 6 ADC  dst-address=2001:470:c334:4::/64 gateway=ether5 gateway-status=ether5 reachable distance=0
        scope=10

 7 ADo  dst-address=2001:470:c334:6::/64 gateway=fe80::20c:42ff:feff:5c7d%ether5
        gateway-status=fe80::20c:42ff:feff:5c7d%ether5 reachable distance=110 scope=20
        target-scope=10 ospf-metric=20

 8 ADo  dst-address=2001:470:c334:7::/64 gateway=fe80::20c:42ff:feff:5c7d%ether5
        gateway-status=fe80::20c:42ff:feff:5c7d%ether5 reachable distance=110 scope=20
        target-scope=10 ospf-metric=20

 9 ADo  dst-address=2001:470:c334:b::/64 gateway=fe80::d6ca:6dff:fe73:25ce%ether2
        gateway-status=fe80::d6ca:6dff:fe73:25ce%ether2 reachable distance=110 scope=20
        target-scope=10 ospf-metric=20

10 ADC  dst-address=2001:470:c334:c::/64 gateway=ether2 gateway-status=ether2 reachable distance=0
        scope=10

line 0 is received via iBGP. It’s gateway is set to the router’s loopback address, 2001:470:c334::b. Line 3 then contains the routing entry for how to get to the router loopback. The gateway is fe80::d6ca:6dff:fe35:d345%ether1, a link-local address. These two routing entries seem a lot like those in igwana’s post