Community discussions

 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: IPv6 recursive nexthops via iBGP

Thu Mar 03, 2016 3:50 pm

@paoloaga and @nz_monkey

Same boat here.....we've gotten by with peering on the subnet instead of on a /128 loopback until it's fixed. We have a ton of customer IPv6 migrations we are working on that need this since the IPv4 topology is recursion based with BGP and OSPF.

RouterOS v7 #thelongwait :shock:
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4048
Joined: Wed May 11, 2011 6:08 pm

Re: IPv6 recursive nexthops via iBGP

Thu Mar 03, 2016 10:44 pm

RouterOS v7 #thelongwait :shock:
I keep expecting to see this every time there's a new MUM....

There's so much that's been promised in ROSv7.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: IPv6 recursive nexthops via iBGP

Thu Mar 03, 2016 10:49 pm

RouterOS v7 #thelongwait :shock:
I keep expecting to see this every time there's a new MUM....

There's so much that's been promised in ROSv7.
Maybe we'll get lucky and it will be Dallas in a month and a half :-)
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Sat Mar 05, 2016 12:20 am

It is no trivial task developing a new routing engine, adding major new features and testing all the corner cases, all the while maintaining the existing code base and back porting drivers so you can release new hardware.


I'm sure Mikrotik are just as desperate as we are to get v7 out, so they can focus on one code base and a newer kernel.

Dallas does seem to be the likely place we will see v7 announced [WHITE SMILING FACE]
 
User avatar
paoloaga
Member Candidate
Member Candidate
Posts: 221
Joined: Tue Mar 08, 2011 2:52 am
Location: Vaprio d'Agogna (NO) - Italy
Contact:

Re:

Sun Mar 06, 2016 10:38 am

It is no trivial task developing a new routing engine, adding major new features and testing all the corner cases, all the while maintaining the existing code base and back porting drivers so you can release new hardware.
What we are asking is not an exotic feature, it's basic IPv6 functionality...
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4048
Joined: Wed May 11, 2011 6:08 pm

Re: Re:

Sun Mar 06, 2016 4:51 pm

It is no trivial task developing a new routing engine, adding major new features and testing all the corner cases, all the while maintaining the existing code base and back porting drivers so you can release new hardware.
What we are asking is not an exotic feature, it's basic IPv6 functionality...
Monkey's statement was directed at the comments on here hoping that ROSv7 should come out soon, and how long it's been said by Mikrotik that "such-and-such a feature will be in next version of ROS" What he was referring to is the fact that they're apparently re-writing the routing engine for ROSv7....

I've found several "misbehaviors" of ROS in both OSPF and BGP that are about this same level - they don't stop the system from being a viable platform for routing, but a few common "best practice" designs are broken due to these oddities. (The one which irritates me most is how a backup OSPF default-gw router won't drop its announcement if the original default-gw is also an OSPF route.)
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: IPv6 recursive nexthops via iBGP

Thu Mar 10, 2016 2:20 pm

v7 Teaser alert, recursive ipv6 works :D
[admin@tested] > /routing/route/print 
Flags: D - dynamic, X - disabled, I - inactive, A - active, 
C - connect, S - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, Y - synthetic, T - terminal, 
> - best, ' - candidate, U - updated, F - filtered 
        DST-ADDRESS        GATEWAY            DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW                          
 AS >'   0.0.0.0/0          10.5.130.1                1    30           10 10.5.130.1%ether1                     
DAC >'   10.5.130.0/24      ether1                    0    10              ether1                                
 AS >'   ;;; router-test
        2001:2001::/64     fe80::29:dfff:f...        1    15           10 fe80::29:dfff:fe2b:14d0%ether2        
 AS >'   ;;; router-test
        2001:2002::/64     2001:2001::1              1    30           15 fe80::29:dfff:fe2b:14d0%ether2   
 
hedele
Member
Member
Posts: 338
Joined: Tue Feb 24, 2009 11:23 pm

Re: IPv6 recursive nexthops via iBGP

Thu Mar 10, 2016 2:27 pm

v7 Teaser alert, recursive ipv6 works :D
[admin@tested] > /routing/route/print 
ooops.. looks like CLI will change a lot?
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8292
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: IPv6 recursive nexthops via iBGP

Thu Mar 10, 2016 3:57 pm

C - connect, S - static, r - rip, b - bgp, o - ospf
and the very first v7 bug report :D a typo: should be 'connected', I think :)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
patrick7
Member Candidate
Member Candidate
Posts: 298
Joined: Sat Jul 20, 2013 2:40 pm

Re: IPv6 recursive nexthops via iBGP

Thu Mar 10, 2016 5:53 pm

@mrz More informations about v7 please :D Even it would be a fully broken beta, I'd be happy to see the news ;-)
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: IPv6 recursive nexthops via iBGP

Fri Mar 11, 2016 12:34 am

v7 Teaser alert, recursive ipv6 works :D
[admin@tested] > /routing/route/print 
Flags: D - dynamic, X - disabled, I - inactive, A - active, 
C - connect, S - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, Y - synthetic, T - terminal, 
> - best, ' - candidate, U - updated, F - filtered 
        DST-ADDRESS        GATEWAY            DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW                          
 AS >'   0.0.0.0/0          10.5.130.1                1    30           10 10.5.130.1%ether1                     
DAC >'   10.5.130.0/24      ether1                    0    10              ether1                                
 AS >'   ;;; router-test
        2001:2001::/64     fe80::29:dfff:f...        1    15           10 fe80::29:dfff:fe2b:14d0%ether2        
 AS >'   ;;; router-test
        2001:2002::/64     2001:2001::1              1    30           15 fe80::29:dfff:fe2b:14d0%ether2   

What I see, I like :)

Particularly the "> - best, ' - candidate, U - updated, F - filtered " feature. This will save us a lot of troubleshooting time.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
User avatar
paoloaga
Member Candidate
Member Candidate
Posts: 221
Joined: Tue Mar 08, 2011 2:52 am
Location: Vaprio d'Agogna (NO) - Italy
Contact:

Re: IPv6 recursive nexthops via iBGP

Fri Mar 11, 2016 5:49 pm

Wonderful news. Hope we can play with it very soon.

Now there is another question that bugs my mind: if the CLI syntax is new, I will have to refactor all the automated software (and it is complex!): Is it possible to access to an early documentation or have an early version to install on a lab box so we can start working on the management software and have it ready for when v7 will be released???
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4048
Joined: Wed May 11, 2011 6:08 pm

Re: IPv6 recursive nexthops via iBGP

Fri Mar 11, 2016 5:57 pm

Wonderful news. Hope we can play with it very soon.

Now there is another question that bugs my mind: if the CLI syntax is new, I will have to refactor all the automated software (and it is complex!): Is it possible to access to an early documentation or have an early version to install on a lab box so we can start working on the management software and have it ready for when v7 will be released???
I noticed that too - if the command syntax requires slashes now, that's a huge change that will require everyone to modify their scripts and so forth. Hopefully they're going to go through a deprecation period where older syntax is still supported but gives deprecation warnings if you use it in an interactive shell, or something like that. I don't personally use scripts much, but many do. I wonder how that's going to unfold.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8292
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: IPv6 recursive nexthops via iBGP

Fri Mar 11, 2016 10:02 pm

I can't see any reason to use slashes instead of spaces, moreover slash has different positions on different keyboards — I don't believe they will change current command style...
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: IPv6 recursive nexthops via iBGP

Sat Mar 12, 2016 2:17 am

Mikrotik have used this different CLI syntax in previous early test releases for new systems.

I don't think what is shown above will be in the final release.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: IPv6 recursive nexthops via iBGP

Tue Mar 15, 2016 3:22 pm

Mikrotik have used this different CLI syntax in previous early test releases for new systems.

I don't think what is shown above will be in the final release.
Well that's good because my brain can't digest yet another CLI syntax without imploding. On a daily basis, I'm already juggling Cisco [IOS, NX-OS, ASA and Wireless Controller], Juniper, HP, Dell, ZyXEL, AdTran, IBM, Lenovo, etc...as well as MikroTik.

#nonewcli :-)
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: IPv6 recursive nexthops via iBGP

Fri Mar 18, 2016 1:58 am

Old menu without slashes will be available, so scripts will not break.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: IPv6 recursive nexthops via iBGP

Fri Mar 18, 2016 7:30 am

Old menu without slashes will be available, so scripts will not break.
Thanks for the confirmation Maris.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8292
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: IPv6 recursive nexthops via iBGP

Fri Mar 18, 2016 11:21 am

Just available? So, new style will stay here in release version? %)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
hknet
Frequent Visitor
Frequent Visitor
Posts: 88
Joined: Sun Jul 17, 2016 6:05 pm
Location: Vienna, Austria
Contact:

Re: IPv6 recursive nexthops via iBGP

Sun Aug 07, 2016 6:30 pm

after reading this thread I'm still trying to make sense, so I conclude:

OSPFv3 and Loopback-bridge-interfaces with /128 IPv6 addresses assigned in RouterOS will only be shown reachable if one sets an admin-mac to the bridge (named eg Loopback0).
well, that's not 100% intuitive, but I guess that's something I can live with.

to receive those Looback IPs (/128) via OSPFv3 one has to remove the LA bit somehow (simple workaround suggested by MikroTik is to simply import those Loopback IPs as external routes), well this is also not perfectly straight forward, but one can work with this.

but on the iBGP case I read something about setting static routes on the RouterOS side to make the connection to Loopback IPs of other iBGP speakers...
well, tried that, but I'm with no luck in getting any routes received from a routereflector installed as active.

currently tested with RouterOS v6.36

any advice greatly appreciated :)
 
savage
Forum Guru
Forum Guru
Posts: 1191
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:

Re: IPv6 recursive nexthops via iBGP

Thu May 25, 2017 12:59 am

7 years later, and still not fixed :roll:

Thank you MT. You pretty much put the final nail in the coffin as far as using Mikrotik goes. I (like many others), can not continue to wait 'indefinitely' until the mythical v7 finally appears.
Regards,
Chris
 
savage
Forum Guru
Forum Guru
Posts: 1191
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:

Re: IPv6 recursive nexthops via iBGP

Thu May 25, 2017 1:01 am

OSPFv3 and Loopback-bridge-interfaces with /128 IPv6 addresses assigned in RouterOS will only be shown reachable if one sets an admin-mac to the bridge (named eg Loopback0).
well, that's not 100% intuitive, but I guess that's something I can live with.
What you perhaps don't know, and can't live with... If Loopback0 (in your example) is in a VRF for IPv4, IPv6 won't work completely. The connected route never becomes active, and OSPF never injects the route - ANOTHER MT bug...
Regards,
Chris
 
User avatar
guilhermeramires
Trainer
Trainer
Posts: 56
Joined: Fri Jan 22, 2010 9:06 pm

Re: IPv6 recursive nexthops via iBGP

Tue Aug 29, 2017 4:18 pm

Use ULAs instead of LL. No need to use filters or static this way.
Mikrotik Training Partner
MPLS for the Masses
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4048
Joined: Wed May 11, 2011 6:08 pm

Re: IPv6 recursive nexthops via iBGP

Tue Aug 29, 2017 4:55 pm

And how exactly does one cause OSPFv3 to use ULAs as the next hop address instead of LL addresses?
If it takes work-arounds to use ULAs, you may as well use the global-unique (public) addresses instead of having a third layer on there.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
User avatar
hknet
Frequent Visitor
Frequent Visitor
Posts: 88
Joined: Sun Jul 17, 2016 6:05 pm
Location: Vienna, Austria
Contact:

Re: IPv6 recursive nexthops via iBGP

Tue Aug 29, 2017 5:39 pm

Hi
saw this thread reactivated :)

The only workaround we have seen so far for iBGP IPv6 routes to get active is to add a static ipv6 route for the loopback IP for the next-hop delivered through the route-reflectors.
(again tested with RouterOS 6.40.2)

If there is any workaround, I'd be glad to hear about it.

Dear MikroTik it would be nice to know how static routes work in this case and how eg. ospfv3 routes fail or maybe there is the possibility for a script run every 30 seconds or so to get the /128 routes (probably loopbacks) out of the ospfv3 table and into static routes and cleaning up those routes that were lost since the last run. I'm not savvy enough in terms of MikroTik scripting to come up with a script that does that.

thx
hk
 
JimmyNyholm
Member Candidate
Member Candidate
Posts: 249
Joined: Mon Apr 25, 2016 2:16 am
Location: Sweden

Re: IPv6 recursive nexthops via iBGP

Sun Sep 03, 2017 1:31 pm

Hi. RR's need not to be in data path (most often aren't) so please consider your own setup before fiddeling with above statement.

@Mikrotik please fix IPV6 it is 2017 after all and Ipv4 is getting more and more expensive.
 
User avatar
hknet
Frequent Visitor
Frequent Visitor
Posts: 88
Joined: Sun Jul 17, 2016 6:05 pm
Location: Vienna, Austria
Contact:

Re: IPv6 recursive nexthops via iBGP

Sun Sep 03, 2017 3:03 pm

Hi. RR's need not to be in data path (most often aren't) so please consider your own setup before fiddeling with above statement.
ahem, the nexthop delivered by RRs was not implying the nexthop in fact is the RR, in fact the nexthop is usually the IP set by "next-hop self" (or similar) by BGP-routers talking to the RRs, but as long as those next-hop IPs are delivered via ospf (which is pretty common imho) routes are not getting active, therefore static routes to those IPs set by BGP-routers as "next hop" are the only workaround I'm aware of, but I'd be happy to hear any other solution.

Regards
hk
 
JimmyNyholm
Member Candidate
Member Candidate
Posts: 249
Joined: Mon Apr 25, 2016 2:16 am
Location: Sweden

Re: IPv6 recursive nexthops via iBGP

Sun Sep 03, 2017 8:39 pm

Hi. RR's need not to be in data path (most often aren't) so please consider your own setup before fiddeling with above statement.
ahem, the nexthop delivered by RRs was not implying the nexthop in fact is the RR, in fact the nexthop is usually the IP set by "next-hop self" (or similar) by BGP-routers talking to the RRs, but as long as those next-hop IPs are delivered via ospf (which is pretty common imho) routes are not getting active, therefore static routes to those IPs set by BGP-routers as "next hop" are the only workaround I'm aware of, but I'd be happy to hear any other solution.

Regards
hk
"Nexthop self" in an bgp environment is an abomination. I agree that it has some good corner cases but a good setup IGP for distributing nexthops (ie: ospf with loopbacks) and a non nexthop modifying ebgp and ibgp. Depending on number of routers you have fullmesh ibgp is fast responsive but explodes in management if you are constantly growing. This is where clustered RR's is a must to keep peering sessions manageable. It's in this space I just claimed that a RR is most of the time not in the packets Datapath and it need/should not to be it should only RR your network.

Having Mikrotiks they can not be pure reflecting as they only reflect active routes. This is a limitation on the MT but then again the good IGP takes care of this even here. The request for being true RR mode has been asked here before and we are told to expect that in the unicorn version 7.
I think that we speak of the same thing but may from different perspective.

As for IPV6 my statement is even more clear from my perspective. MT Please fix IPV6.
Yours J
 
User avatar
hknet
Frequent Visitor
Frequent Visitor
Posts: 88
Joined: Sun Jul 17, 2016 6:05 pm
Location: Vienna, Austria
Contact:

Re: IPv6 recursive nexthops via iBGP

Sun Sep 03, 2017 9:17 pm

Well
"Nexthop self" in an bgp environment is an abomination.
It should not be needed - agreed, but real world teaches us:
One looses an external interface and therefore the nexthop is removed from the IGP.
In a small network this converges fast and causes virtually no service disruption.
Think about lots of BGP-peers and lots (fulltable times N) of routes all handled with policies and communities and so on.
IGP (eg. ospf) still converges within seconds and removes the route to the previously valid next-hop.
Now all BGP-routes that learned these routes have to reevaluate their situation on where to go next. If you loose say 500k routes from this single event this can take a moment, packets are still forwarded to the now unreachable next-hop and get lost.
Here a forward to a still existing loopback-IP is a much less disruptive solution: packets are still forwarded to the loopback, but the router owning it knows its interface is down and can decide where to go next with those packets as it has probably already marked all routes for removal that were formerly learned from this external peer. You may see some funny packetflows, but usually don't loose as much packets.

Cheers,
hk
 
wsftech
just joined
Posts: 22
Joined: Wed Jun 10, 2015 12:19 am

Re: IPv6 recursive nexthops via iBGP

Thu Sep 07, 2017 9:40 am

Hi
The only workaround we have seen so far for iBGP IPv6 routes to get active is to add a static ipv6 route for the loopback IP for the next-hop delivered through the route-reflectors.
(again tested with RouterOS 6.40.2)
hk
Let me recap, and please correct me if I'm wrong.

1. Under no circumstances does ROS6 recursively resolve BGP IPv6 routes using link-local addresses; or OSPF-populated loopback IPs using either link-local, ULA, or static ('public') IPv6 addresses
2. ROS can resolve IPv6 routes that recurse to *static* routes with appropriate scope/target scope values.
3. It is impossible to use a modern network design of OSPF-populated loopbacks and multihop BGP peering for IPv6 routes.
4. We are stuck in the dark ages of static addressing.

I've tested this on 6.38 and 6.39, and I can only assume from this thread that <6.41 doesn't fix this either.

I assume that one could simulate an OSPF-like failover behavior by using multiple static routes, ping gateway, and distance, but yuck!

Is the only alternative for, say, IGP, to use OSPF only?
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4048
Joined: Wed May 11, 2011 6:08 pm

Re: IPv6 recursive nexthops via iBGP

Thu Sep 07, 2017 5:43 pm

Pretty much correct.

For those following along at home, here is a summary of the issue:
OSPFv3 installs the link-local address of its neighbor as the next-hop address when it places routes into the routing table.
eBGP will use the peer's address as the next-hop. Note that this is directly-attached eBGP peering (which is the standard methodology)
iBGP preserves the next hop address of the prefixes shared.

example:
Two EBGP neighbors link up over a /126 connection (2001:db8:ffff::0/124 with the external peer being ::1
The local EBGP router learns 2001:db8:cafe::/48 via 2001:db8:ffff::1 and installs this into the local routing table. Everything is a-okay so far.
There is some arbitrary topology within the local ASN.
Somewhere inside the ASN is an iBGP peer of the EBGP border router.

The BGP strategy is to use the IGP (OSPF in this case) to determine the best next hop for reaching the egress point chosen by BGP.
So in the RIB (live routing table), the next hop will be the OSPF-determined best next hop to reach the peering address.
Since OSPFv3 uses link-local, this is where the problem comes from.

example:
So the iBGP peer has some set of paths to reach the eBGP router above.
The eBGP router will also be advertising its peering link into the local IGP (ospf) so that other routers will know how to reach this peering point.
So 2001:db8:ffff::/0 will be in the OSPF database.
the iBGP router will have a route with dst=2001:db8:ffff::/0 gateway=fe80::abcd -- where fe80::abcd%ether1 is the link-local IPv6 address of the OSPF neighbor through which the best path leads.
The iBGP router learns prefix 2001:db8:cafe::/48 from the eBGP router. The next hop for this prefix will be 2001:db8:ffff::1
The iBGP router must now perform a recursive lookup on 2001:db8:ffff::1 in order to set the next-hop for 2001:db8:cafe::/48

In ROS, this is where the problem happens because the next hop is going to be fe80::abcd%ether1. The iBGP recursive lookup bug prevents this route from being installed as an active route in the routing table. The work-around is to use a static route to reach 2001:db8:ffff::/124 where the next hop address is not a link-local address.

e.g. if the router with link-local address fe80::abcd%eth1 is also using 2001:db8:beef::1 as a global public address on the link to iBGP router, then on iBGP router would need a static route as follows:
dst=2001:db8:ffff::/124 gateway=2001:db8:beef::1

This is obviously quite problematic.
Apparently the root cause is something deep enough in the routing engine code that Mikrotik has stated this will only be fixed in RouterOS v7

In a nutshell, it is impossible to deploy a standard OSPF+iBGP ASN for IPv6 using Mikrotik. If your network does not use iBGP, then you can use IPv6 in your core w/o worrying about this issue.

You may be able to make a work-around with GRE tunnels directly between all bgp-speaking routers and some next-hop-self tweaks, but this seems very undesirable to me as well.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
wsftech
just joined
Posts: 22
Joined: Wed Jun 10, 2015 12:19 am

Re: IPv6 recursive nexthops via iBGP

Fri Sep 08, 2017 2:58 am

ZeroByte, thanks for the detailed comments.

After further testing, I have reached the same conclusions you have. I would rather not tunnel — although I did briefly contemplate a full-mesh MPLS/VPLS for directly peering each IPv6 router. (Traffic queue management would be just one problem with that design.)

I settled on sticking with OSPFv3, as that at least works, and can use link-local, so not actually very painful to implement. In addition I found out today that my upstream provider (a publicly traded company) also uses OSPFv3 as IGP, and they're a Juniper shop… go figure.

For eBGP, no problem is encountered, as next-hops are rewritten anyway upon further route advertisement.

<sigh>
 
aplvmn
just joined
Posts: 1
Joined: Sun Sep 24, 2017 7:50 pm

Re: IPv6 recursive nexthops via iBGP

Sun Sep 24, 2017 7:53 pm

I just started deploying IPv6 and found this issue, going over this post made me feel disappointed, I think I have spent money for nothing on this boxes, I have found so many bugs with other technologies in MK (Radius, PPP, and an endless list), I guess that is why is so cheap compared to Cisco or Juniper.
 
SplitHorizon
just joined
Posts: 11
Joined: Tue Feb 09, 2016 10:56 am

Re: IPv6 recursive nexthops via iBGP

Thu Oct 05, 2017 10:46 am

Just marking the register on this issue.. it has caused me enough grief..
 
patrick7
Member Candidate
Member Candidate
Posts: 298
Joined: Sat Jul 20, 2013 2:40 pm

Re: IPv6 recursive nexthops via iBGP

Thu Oct 05, 2017 12:28 pm

I think there will be no fix in the near future.
 
zimage
just joined
Posts: 21
Joined: Wed Jul 10, 2013 9:48 pm

Re: IPv6 recursive nexthops via iBGP

Thu Oct 05, 2017 6:05 pm

I gave up on mikrotik when we moved to a dual stack network because of this bug. You can find new Juniper SRX routers pretty cheaply if you look hard. Don’t pay more than 25% of the list cost, though.
 
savage
Forum Guru
Forum Guru
Posts: 1191
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:

Re: IPv6 recursive nexthops via iBGP

Thu Oct 05, 2017 7:09 pm

I gave up on mikrotik when we moved to a dual stack network because of this bug. You can find new Juniper SRX routers pretty cheaply if you look hard. Don’t pay more than 25% of the list cost, though.
I'm in the same boat. Can't use MT in my core / borders. MT is definitely not aware of the actual seriousness of this issue, especially on larger networks.

There's actually PLENTY of IPv6 broken, especially on the PPP side (filters, address-lists, IP assignments, the list goes on). BGP not working, is a HUGE issue.
Regards,
Chris
 
JimmyNyholm
Member Candidate
Member Candidate
Posts: 249
Joined: Mon Apr 25, 2016 2:16 am
Location: Sweden

Re: IPv6 recursive nexthops via iBGP

Fri Aug 24, 2018 8:46 pm

Passing Into Late 2018 And still this is big issue when @Mikrotik WHEN will recursive routing work in routeros. Installed V6 routes that have reachables nexthops (recursivly that is) will never be active due to something broken. FIX NOW. IPV4 days are over and we must deploy ipv6.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: IPv6 recursive nexthops via iBGP

Wed Aug 29, 2018 6:25 pm

We could really use an update on this MikroTik. :-)

We are seeing IPv6 adoption move at a much faster pace in 2018 and are having to modify the routing architecture or use other brand routers for our clients to solve this problem.

This has been an issue for a long time but we could really use a fix
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
patrick7
Member Candidate
Member Candidate
Posts: 298
Joined: Sat Jul 20, 2013 2:40 pm

Re: IPv6 recursive nexthops via iBGP

Wed Aug 29, 2018 6:55 pm

@IPANetEngineer If it would be important for them, they would have fixed this issue years ago.
Just proceed with FRRouting :-) It's better anyways.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: IPv6 recursive nexthops via iBGP

Wed Aug 29, 2018 7:48 pm

@IPANetEngineer If it would be important for them, they would have fixed this issue years ago.
Just proceed with FRRouting :-) It's better anyways.
Depends on your use case. I like FRR and talk to a number of the developers at FRR on a regular basis. However, it's still software that's go to go on bare metal or a VM and there are a number of operational challenges with that if you're doing something outside of BGP Full table or RR.

Would really just like to see this fixed. :-)
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
wsftech
just joined
Posts: 22
Joined: Wed Jun 10, 2015 12:19 am

Re: IPv6 recursive nexthops via iBGP

Wed Aug 29, 2018 7:52 pm

It's not possible to use Mikrotik in a real production-quality dual-stack service provider network without working BGP.

I'd hate to see the CCR series, for example, be left out in the cold, if we have to wait for an entirely new generation of hardware to get ROS7. Not to spread FUD, but I don't have much to go on.
 
sep
just joined
Posts: 11
Joined: Thu Nov 28, 2013 2:34 pm

Re: IPv6 recursive nexthops via iBGP

Thu Sep 20, 2018 11:25 pm

is there any updates on this incredibly critical part of the puzzle ? is there no other option that just avoiding using mikrotik for routing ?
there is nothing we deployed without ipv6 nowadays. But plenty that we deploy as ipv6 only. and CCR would be a excellent fit is so many cases.

but this is a brick wall show stopper for mikrotik.

kind regards
Ronny Aasen
 
bbs2web
Member Candidate
Member Candidate
Posts: 197
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: IPv6 recursive nexthops via iBGP

Sat Sep 22, 2018 11:07 pm

Getting decent IPv4 performance without packet loss on CCR routers has required us to use raw firewall rules to stop connection tracking on forwarded traffic and building a distributed core network where P routers talk only IPv4 OSPF and PE routers exchange routes via BGP route reflectors so that they reference the remote router's loopback IP. This way traffic ingressing at a PE router will be switched to the destination router via MPLS.

IPv6 OSPFv3 is reported as being the culprit but without IPv6 MPLS support we would be back at P routers having to route each packet. Setting up a BGP signaled VPLS using split horizon bridging allows one to set the bridge protocol mode to none, whilst avoiding bridging loops. We then assigned IPv6 /128 loopback IPs and assigned the same IP with a /64 subnet to the VPLS bridge interfaces.

Is my thinking in this regard wrong or is the IPv6 routing problem on RouterOS actually a recursive routing issue at heart?

Topology:
Image


Summary:
  • IPv4 OSPF with MPLS using loopback IPs, where only the default gateway is redistributed on R1
  • RR1 has two IPv4 BGP sessions to R1 and R5. RR1 reflects routes and both R1 and R5 set the nexthop as their loopback interface IPs
  • BGP peering sessions are running with IPv4, IPv6 and L2VPN address families
  • Everything establishes without errors and I can ping from the test PC connected to R5 to 8.8.8.8 using IPv4 (MPLS switched through R4, R3 and R2)
  • I can ping IPv6 gateways and routes appear to install correctly but I can not ping remote IPv6 addresses

R1:
Image


R5:
Image


Configuration:
R1:
/interface bridge
add name=bridge-ipv6-mpls protocol-mode=none
add name=lo
/interface ethernet
set [ find default-name=ether2 ] mtu=2026
/routing bgp instance
set default redistribute-connected=yes redistribute-static=yes router-id=10.0.0.1
/routing ospf instance
set [ find default=yes ] distribute-default=if-installed-as-type-1 router-id=10.0.0.1
/interface vpls bgp-vpls
add bridge=bridge-ipv6-mpls bridge-horizon=1 export-route-targets=1:1 import-route-targets=1:1 name=bgp-vpls1 route-distinguisher=1:1
/ip address
add address=10.0.0.1 interface=lo
add address=172.16.254.5/30 interface=ether2
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
/ipv6 address
add address=2001:db8::1/128 advertise=no interface=lo
add address=2001:db8:1000::1/48 advertise=no interface=ether1
add address=2001:db8::1 advertise=no interface=bridge-ipv6-mpls
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.1 transport-address=10.0.0.1
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether2
/mpls local-bindings
add label=impl-null
/routing bgp peer
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=RR1 nexthop-choice=force-self remote-address=10.0.0.6 remote-as=65530 ttl=default update-source=lo
/routing ospf interface
add authentication=md5 authentication-key=PHi02OnUPN1gzENt dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.1/32
add area=backbone network=172.16.254.4/30
/system identity
set name=R1

R2:
/interface bridge
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
set [ find default-name=ether2 ] mtu=2026
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.2
/ip address
add address=10.0.0.2 interface=lo
add address=172.16.254.9/30 interface=ether1
add address=172.16.254.6/30 interface=ether2
/ip dns
set servers=1.1.1.1,1.0.0.1
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.2 transport-address=10.0.0.2
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
add hello-interval=1s hold-time=10s interface=ether2
/routing ospf interface
add authentication=md5 authentication-key=PHi02OnUPN1gzENt dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
add authentication=md5 authentication-key=3r5MTWTfAD4dDMeH dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.2/32
add area=backbone network=172.16.254.4/30
add area=backbone network=172.16.254.8/30
/system identity
set name=R2

R3:
/interface bridge
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
set [ find default-name=ether2 ] mtu=2026
set [ find default-name=ether3 ] mtu=2026
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.3
/ip address
add address=10.0.0.3 interface=lo
add address=172.16.254.13/30 interface=ether2
add address=172.16.254.10/30 interface=ether1
add address=172.16.254.21/30 interface=ether3
/ip dns
set servers=1.1.1.1,1.0.0.1
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.3 transport-address=10.0.0.3
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
add hello-interval=1s hold-time=10s interface=ether2
add hello-interval=1s hold-time=10s interface=ether3
/routing ospf interface
add authentication=md5 authentication-key=3r5MTWTfAD4dDMeH dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
add authentication=md5 authentication-key=ZxOnGXB2ZLsCr0dJ dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
add authentication=md5 authentication-key=5bnwqgfAomwbU9B3 dead-interval=10s hello-interval=1s interface=ether3 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.3/32
add area=backbone network=172.16.254.12/30
add area=backbone network=172.16.254.8/30
add area=backbone network=172.16.254.20/30
/system identity
set name=R3

R4:
/interface bridge
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
set [ find default-name=ether2 ] mtu=2026
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.4
/ip address
add address=10.0.0.4 interface=lo
add address=172.16.254.17/30 interface=ether2
add address=172.16.254.14/30 interface=ether1
/ip dns
set servers=1.1.1.1,1.0.0.1
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.4 transport-address=10.0.0.4
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
add hello-interval=1s hold-time=10s interface=ether2
/routing ospf interface
add authentication=md5 authentication-key=ZxOnGXB2ZLsCr0dJ dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
add authentication=md5 authentication-key=A54RUkZukh3lDRBb dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.4/32
add area=backbone network=172.16.254.16/30
add area=backbone network=172.16.254.12/30
/system identity
set name=R4

R5:
/interface bridge
add name=bridge-ipv6-mpls protocol-mode=none
add name=lo
/interface ethernet
set [ find default-name=ether2 ] mtu=2026
/ip pool
add name=DHCP ranges=192.168.248.50-192.168.248.199
/ip dhcp-server
add address-pool=DHCP disabled=no interface=ether1 lease-time=12h name=ether1
/routing bgp instance
set default redistribute-connected=yes redistribute-static=yes router-id=10.0.0.5
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.5
/interface vpls bgp-vpls
add bridge=bridge-ipv6-mpls bridge-horizon=1 export-route-targets=1:1 import-route-targets=1:1 name=bgp-vpls1 route-distinguisher=1:1 site-id=5
/ip address
add address=10.0.0.5 interface=lo
add address=172.16.254.18/30 interface=ether2
add address=192.168.248.1/24 interface=ether1
/ip dhcp-server network
add address=192.168.248.0/24 gateway=192.168.248.1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ipv6 address
add address=2001:db8::5/128 advertise=no interface=lo
add address=2001:db8:5000::1/48 advertise=no interface=ether1
add address=2001:db8::5 advertise=no interface=bridge-ipv6-mpls
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.5 transport-address=10.0.0.5
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether2
/routing bgp peer
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=RR1 nexthop-choice=force-self remote-address=10.0.0.6 remote-as=65530 ttl=default update-source=lo
/routing ospf interface
add authentication=md5 authentication-key=A54RUkZukh3lDRBb dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.5/32
add area=backbone network=172.16.254.16/30
/system identity
set name=R5

RR1:
/interface bridge
add name=bridge-ipv6-mpls protocol-mode=none
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
/routing bgp instance
set default redistribute-connected=yes redistribute-static=yes router-id=10.0.0.6
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.6
/interface vpls bgp-vpls
add bridge=bridge-ipv6-mpls bridge-horizon=1 export-route-targets=1:1 import-route-targets=1:1 name=bgp-vpls1 route-distinguisher=1:1 site-id=6
/ip address
add address=10.0.0.6 interface=lo
add address=172.16.254.22/30 interface=ether1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ipv6 address
add address=2001:db8::6/128 advertise=no interface=lo
add address=2001:db8::6 advertise=no interface=bridge-ipv6-mpls
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.6 transport-address=10.0.0.6
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
/routing bgp peer
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=R1 remote-address=10.0.0.1 remote-as=65530 route-reflect=yes ttl=default
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=R5 remote-address=10.0.0.5 remote-as=65530 route-reflect=yes ttl=default
/routing ospf interface
add authentication=md5 authentication-key=5bnwqgfAomwbU9B3 dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
/routing ospf network
add area=backbone network=172.16.254.20/30
add area=backbone network=10.0.0.6/32
/system identity
set name=RR1
 
mducharme
Trainer
Trainer
Posts: 788
Joined: Tue Jul 19, 2016 6:45 pm

Re: IPv6 recursive nexthops via iBGP

Sun Sep 23, 2018 12:07 am

We then assigned IPv6 /128 loopback IPs and assigned the same IP with a /64 subnet to the VPLS bridge interfaces.
I'm guessing that might be the cause of your issue. Use different IPs with a /64 subnet for the VPLS bridge interfaces than you are using for the loopback. I expect the ping above would then work.
 
bbs2web
Member Candidate
Member Candidate
Posts: 197
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: IPv6 recursive nexthops via iBGP

Sun Sep 23, 2018 6:59 am

The point is to get IPv6 ingressing at a PE switched across P routers using MPLS. You also missed the fact that I can ping R5's IPv6 loopback from R1 and vice versa, so the gateways are reachable.
We then assigned IPv6 /128 loopback IPs and assigned the same IP with a /64 subnet to the VPLS bridge interfaces.
I'm guessing that might be the cause of your issue. Use different IPs with a /64 subnet for the VPLS bridge interfaces than you are using for the loopback.
 
mducharme
Trainer
Trainer
Posts: 788
Joined: Tue Jul 19, 2016 6:45 pm

Re: IPv6 recursive nexthops via iBGP

Sun Sep 23, 2018 7:06 am

The point is to get IPv6 ingressing at a PE switched across P routers using MPLS. You also missed the fact that I can ping R5's IPv6 loopback from R1 and vice versa, so the gateways are reachable.
I did not miss this, I can see quite well that the gateways are reachable. We offer IPv6 over VPLS tunnels and it works quite well. But given the routing tables you have shown, there is no other reason I can see (short of a bug) why the ping would fail. I expect that the duplication of IPs from the loopback to the VPLS bridge interface is not impacting the direct ping, but may be preventing the recursive routing from functioning properly. For instance, your routing tables show that the nexthop for 2001:db8::1 is 2001:db8::1, the nexthop for 2001:db8::5 is 2001:db8::5, and the nexthop for 2001:db8::6 is 2001:db8::6. This would seem to me to create a loop that can not be recovered from, hence not being reachable when it comes to recursive routing, but reachable when it comes to a direct ping response.
 
bbs2web
Member Candidate
Member Candidate
Posts: 197
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: IPv6 recursive nexthops via iBGP

Sun Sep 23, 2018 8:28 am

IPv6 appears extremely unreliable in the GNS3 virtual lab I put together. The following initially only worked in one direction (R1 -> R5) until I restarted R5, after which it worked in both.

Added IPv6 prefix filter to the route reflector (RR1):
/routing filter add chain=bgp-in address-family=ipv6 prefix=2001:db8::/64 prefix-length=64-128 action=discard
/routing filter add chain=bgp-out address-family=ipv6 prefix=2001:db8::/64 prefix-length=64-128 action=discard
/routing bgp peer set [ find ] in-filter=bgp-in out-filter=bgp-out


R1:
Image


R5:
Image
Last edited by bbs2web on Sun Sep 23, 2018 9:49 am, edited 1 time in total.
 
mducharme
Trainer
Trainer
Posts: 788
Joined: Tue Jul 19, 2016 6:45 pm

Re: IPv6 recursive nexthops via iBGP

Sun Sep 23, 2018 8:39 am

IPv6 appears extremely unreliable in the GNS3 virtual lab I put together. The following initially only worked in one direction (R1 -> R5) until I restarted R5, after which it worked in both.
Again, what I see here as a common link is that you are assigning the same IPv6 address to the loopback interface as a /128 as you are to the VPLS interface (/64). I strongly suspect that this is the cause of your issues.

I have found that RouterOS does not always treat interfaces properly with IPv6. For instance, if two OSPFv3 neighbors share the same link local address (even though their interface IDs are different), the OSPFv3 process will get confused and not allow one of the two neighbors to become an OSPFv3 neighbor.
 
bbs2web
Member Candidate
Member Candidate
Posts: 197
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: IPv6 recursive nexthops via iBGP

Sun Sep 23, 2018 9:37 am

My intention with this lab exercise was to find a solution to efficiently switch IPv6 packets between provider edge (PE) routers R1 and R5, through an IPv4 MPLS core, using iBGP. RouterOS 6.43.2 can not be used to recursively resolve IPv6 iBGP nexthop using OSPFv3 and running OSPFv3 without IPv6 MPLS in the core is not optimal anyway and would unnecessarily duplicate administrative work. The lab essentially attempts to emulates Cisco 6PE where IPv6 PE routers communicate with each other over an IPv4 MPLS core.

Routing lookups should reference the best match so 2001:db8::5 from R1 matches the 2001:db8::/64 route referencing the BGP signaled VPLS bridge interface and transmit it out there. RouterOS is Linux based so all IPs are present in table 255 without a netmask, which is hard coded as the first lookup rule when processing sent or received packets. One can run 'ip rule list; ip route show table 255' on any Linux system to observe this behaviour (the same reason why RouterOS can not fully isolate VRFs).

The environment does now appear to be working consistently, the problem was due to IPv6 addresses associated with the VPLS bridge interfaces having been set as 'advertise=no'. 2nd amendment to the lab environment:
R1 + R5 + RR1:
/ipv6 address set [ find interface=bridge-ipv6-mpls ] advertise=yes

Edit: This was working randomly, see further comment.


You mention having numerous production networks doing just this, I have no production experience and would appreciate it if you would be so kind as to share your knowledge in this regard.
Last edited by bbs2web on Tue Sep 25, 2018 1:05 am, edited 1 time in total.

Who is online

Users browsing this forum: No registered users and 7 guests