I keep expecting to see this every time there's a new MUM....RouterOS v7 #thelongwait
Maybe we'll get lucky and it will be Dallas in a month and a halfI keep expecting to see this every time there's a new MUM....RouterOS v7 #thelongwait
There's so much that's been promised in ROSv7.
What we are asking is not an exotic feature, it's basic IPv6 functionality...It is no trivial task developing a new routing engine, adding major new features and testing all the corner cases, all the while maintaining the existing code base and back porting drivers so you can release new hardware.
Monkey's statement was directed at the comments on here hoping that ROSv7 should come out soon, and how long it's been said by Mikrotik that "such-and-such a feature will be in next version of ROS" What he was referring to is the fact that they're apparently re-writing the routing engine for ROSv7....What we are asking is not an exotic feature, it's basic IPv6 functionality...It is no trivial task developing a new routing engine, adding major new features and testing all the corner cases, all the while maintaining the existing code base and back porting drivers so you can release new hardware.
[admin@tested] > /routing/route/print
Flags: D - dynamic, X - disabled, I - inactive, A - active,
C - connect, S - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, Y - synthetic, T - terminal,
> - best, ' - candidate, U - updated, F - filtered
DST-ADDRESS GATEWAY DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW
AS >' 0.0.0.0/0 10.5.130.1 1 30 10 10.5.130.1%ether1
DAC >' 10.5.130.0/24 ether1 0 10 ether1
AS >' ;;; router-test
2001:2001::/64 fe80::29:dfff:f... 1 15 10 fe80::29:dfff:fe2b:14d0%ether2
AS >' ;;; router-test
2001:2002::/64 2001:2001::1 1 30 15 fe80::29:dfff:fe2b:14d0%ether2
ooops.. looks like CLI will change a lot?v7 Teaser alert, recursive ipv6 worksCode: Select all[admin@tested] > /routing/route/print
and the very first v7 bug report a typo: should be 'connected', I thinkCode: Select allC - connect, S - static, r - rip, b - bgp, o - ospf
v7 Teaser alert, recursive ipv6 worksCode: Select all[admin@tested] > /routing/route/print Flags: D - dynamic, X - disabled, I - inactive, A - active, C - connect, S - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, Y - synthetic, T - terminal, > - best, ' - candidate, U - updated, F - filtered DST-ADDRESS GATEWAY DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW AS >' 0.0.0.0/0 10.5.130.1 1 30 10 10.5.130.1%ether1 DAC >' 10.5.130.0/24 ether1 0 10 ether1 AS >' ;;; router-test 2001:2001::/64 fe80::29:dfff:f... 1 15 10 fe80::29:dfff:fe2b:14d0%ether2 AS >' ;;; router-test 2001:2002::/64 2001:2001::1 1 30 15 fe80::29:dfff:fe2b:14d0%ether2
I noticed that too - if the command syntax requires slashes now, that's a huge change that will require everyone to modify their scripts and so forth. Hopefully they're going to go through a deprecation period where older syntax is still supported but gives deprecation warnings if you use it in an interactive shell, or something like that. I don't personally use scripts much, but many do. I wonder how that's going to unfold.Wonderful news. Hope we can play with it very soon.
Now there is another question that bugs my mind: if the CLI syntax is new, I will have to refactor all the automated software (and it is complex!): Is it possible to access to an early documentation or have an early version to install on a lab box so we can start working on the management software and have it ready for when v7 will be released???
Well that's good because my brain can't digest yet another CLI syntax without imploding. On a daily basis, I'm already juggling Cisco [IOS, NX-OS, ASA and Wireless Controller], Juniper, HP, Dell, ZyXEL, AdTran, IBM, Lenovo, etc...as well as MikroTik.Mikrotik have used this different CLI syntax in previous early test releases for new systems.
I don't think what is shown above will be in the final release.
Thanks for the confirmation Maris.Old menu without slashes will be available, so scripts will not break.
What you perhaps don't know, and can't live with... If Loopback0 (in your example) is in a VRF for IPv4, IPv6 won't work completely. The connected route never becomes active, and OSPF never injects the route - ANOTHER MT bug...OSPFv3 and Loopback-bridge-interfaces with /128 IPv6 addresses assigned in RouterOS will only be shown reachable if one sets an admin-mac to the bridge (named eg Loopback0).
well, that's not 100% intuitive, but I guess that's something I can live with.
ahem, the nexthop delivered by RRs was not implying the nexthop in fact is the RR, in fact the nexthop is usually the IP set by "next-hop self" (or similar) by BGP-routers talking to the RRs, but as long as those next-hop IPs are delivered via ospf (which is pretty common imho) routes are not getting active, therefore static routes to those IPs set by BGP-routers as "next hop" are the only workaround I'm aware of, but I'd be happy to hear any other solution.Hi. RR's need not to be in data path (most often aren't) so please consider your own setup before fiddeling with above statement.
"Nexthop self" in an bgp environment is an abomination. I agree that it has some good corner cases but a good setup IGP for distributing nexthops (ie: ospf with loopbacks) and a non nexthop modifying ebgp and ibgp. Depending on number of routers you have fullmesh ibgp is fast responsive but explodes in management if you are constantly growing. This is where clustered RR's is a must to keep peering sessions manageable. It's in this space I just claimed that a RR is most of the time not in the packets Datapath and it need/should not to be it should only RR your network.ahem, the nexthop delivered by RRs was not implying the nexthop in fact is the RR, in fact the nexthop is usually the IP set by "next-hop self" (or similar) by BGP-routers talking to the RRs, but as long as those next-hop IPs are delivered via ospf (which is pretty common imho) routes are not getting active, therefore static routes to those IPs set by BGP-routers as "next hop" are the only workaround I'm aware of, but I'd be happy to hear any other solution.Hi. RR's need not to be in data path (most often aren't) so please consider your own setup before fiddeling with above statement.
Regards
hk
It should not be needed - agreed, but real world teaches us:"Nexthop self" in an bgp environment is an abomination.
Let me recap, and please correct me if I'm wrong.Hi
The only workaround we have seen so far for iBGP IPv6 routes to get active is to add a static ipv6 route for the loopback IP for the next-hop delivered through the route-reflectors.
(again tested with RouterOS 6.40.2)
hk
I'm in the same boat. Can't use MT in my core / borders. MT is definitely not aware of the actual seriousness of this issue, especially on larger networks.I gave up on mikrotik when we moved to a dual stack network because of this bug. You can find new Juniper SRX routers pretty cheaply if you look hard. Don’t pay more than 25% of the list cost, though.
Depends on your use case. I like FRR and talk to a number of the developers at FRR on a regular basis. However, it's still software that's go to go on bare metal or a VM and there are a number of operational challenges with that if you're doing something outside of BGP Full table or RR.@IPANetEngineer If it would be important for them, they would have fixed this issue years ago.
Just proceed with FRRouting It's better anyways.
/interface bridge
add name=bridge-ipv6-mpls protocol-mode=none
add name=lo
/interface ethernet
set [ find default-name=ether2 ] mtu=2026
/routing bgp instance
set default redistribute-connected=yes redistribute-static=yes router-id=10.0.0.1
/routing ospf instance
set [ find default=yes ] distribute-default=if-installed-as-type-1 router-id=10.0.0.1
/interface vpls bgp-vpls
add bridge=bridge-ipv6-mpls bridge-horizon=1 export-route-targets=1:1 import-route-targets=1:1 name=bgp-vpls1 route-distinguisher=1:1
/ip address
add address=10.0.0.1 interface=lo
add address=172.16.254.5/30 interface=ether2
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
/ipv6 address
add address=2001:db8::1/128 advertise=no interface=lo
add address=2001:db8:1000::1/48 advertise=no interface=ether1
add address=2001:db8::1 advertise=no interface=bridge-ipv6-mpls
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.1 transport-address=10.0.0.1
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether2
/mpls local-bindings
add label=impl-null
/routing bgp peer
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=RR1 nexthop-choice=force-self remote-address=10.0.0.6 remote-as=65530 ttl=default update-source=lo
/routing ospf interface
add authentication=md5 authentication-key=PHi02OnUPN1gzENt dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.1/32
add area=backbone network=172.16.254.4/30
/system identity
set name=R1
/interface bridge
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
set [ find default-name=ether2 ] mtu=2026
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.2
/ip address
add address=10.0.0.2 interface=lo
add address=172.16.254.9/30 interface=ether1
add address=172.16.254.6/30 interface=ether2
/ip dns
set servers=1.1.1.1,1.0.0.1
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.2 transport-address=10.0.0.2
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
add hello-interval=1s hold-time=10s interface=ether2
/routing ospf interface
add authentication=md5 authentication-key=PHi02OnUPN1gzENt dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
add authentication=md5 authentication-key=3r5MTWTfAD4dDMeH dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.2/32
add area=backbone network=172.16.254.4/30
add area=backbone network=172.16.254.8/30
/system identity
set name=R2
/interface bridge
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
set [ find default-name=ether2 ] mtu=2026
set [ find default-name=ether3 ] mtu=2026
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.3
/ip address
add address=10.0.0.3 interface=lo
add address=172.16.254.13/30 interface=ether2
add address=172.16.254.10/30 interface=ether1
add address=172.16.254.21/30 interface=ether3
/ip dns
set servers=1.1.1.1,1.0.0.1
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.3 transport-address=10.0.0.3
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
add hello-interval=1s hold-time=10s interface=ether2
add hello-interval=1s hold-time=10s interface=ether3
/routing ospf interface
add authentication=md5 authentication-key=3r5MTWTfAD4dDMeH dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
add authentication=md5 authentication-key=ZxOnGXB2ZLsCr0dJ dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
add authentication=md5 authentication-key=5bnwqgfAomwbU9B3 dead-interval=10s hello-interval=1s interface=ether3 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.3/32
add area=backbone network=172.16.254.12/30
add area=backbone network=172.16.254.8/30
add area=backbone network=172.16.254.20/30
/system identity
set name=R3
/interface bridge
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
set [ find default-name=ether2 ] mtu=2026
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.4
/ip address
add address=10.0.0.4 interface=lo
add address=172.16.254.17/30 interface=ether2
add address=172.16.254.14/30 interface=ether1
/ip dns
set servers=1.1.1.1,1.0.0.1
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.4 transport-address=10.0.0.4
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
add hello-interval=1s hold-time=10s interface=ether2
/routing ospf interface
add authentication=md5 authentication-key=ZxOnGXB2ZLsCr0dJ dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
add authentication=md5 authentication-key=A54RUkZukh3lDRBb dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.4/32
add area=backbone network=172.16.254.16/30
add area=backbone network=172.16.254.12/30
/system identity
set name=R4
/interface bridge
add name=bridge-ipv6-mpls protocol-mode=none
add name=lo
/interface ethernet
set [ find default-name=ether2 ] mtu=2026
/ip pool
add name=DHCP ranges=192.168.248.50-192.168.248.199
/ip dhcp-server
add address-pool=DHCP disabled=no interface=ether1 lease-time=12h name=ether1
/routing bgp instance
set default redistribute-connected=yes redistribute-static=yes router-id=10.0.0.5
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.5
/interface vpls bgp-vpls
add bridge=bridge-ipv6-mpls bridge-horizon=1 export-route-targets=1:1 import-route-targets=1:1 name=bgp-vpls1 route-distinguisher=1:1 site-id=5
/ip address
add address=10.0.0.5 interface=lo
add address=172.16.254.18/30 interface=ether2
add address=192.168.248.1/24 interface=ether1
/ip dhcp-server network
add address=192.168.248.0/24 gateway=192.168.248.1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ipv6 address
add address=2001:db8::5/128 advertise=no interface=lo
add address=2001:db8:5000::1/48 advertise=no interface=ether1
add address=2001:db8::5 advertise=no interface=bridge-ipv6-mpls
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.5 transport-address=10.0.0.5
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether2
/routing bgp peer
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=RR1 nexthop-choice=force-self remote-address=10.0.0.6 remote-as=65530 ttl=default update-source=lo
/routing ospf interface
add authentication=md5 authentication-key=A54RUkZukh3lDRBb dead-interval=10s hello-interval=1s interface=ether2 network-type=point-to-point
/routing ospf network
add area=backbone network=10.0.0.5/32
add area=backbone network=172.16.254.16/30
/system identity
set name=R5
/interface bridge
add name=bridge-ipv6-mpls protocol-mode=none
add name=lo
/interface ethernet
set [ find default-name=ether1 ] mtu=2026
/routing bgp instance
set default redistribute-connected=yes redistribute-static=yes router-id=10.0.0.6
/routing ospf instance
set [ find default=yes ] router-id=10.0.0.6
/interface vpls bgp-vpls
add bridge=bridge-ipv6-mpls bridge-horizon=1 export-route-targets=1:1 import-route-targets=1:1 name=bgp-vpls1 route-distinguisher=1:1 site-id=6
/ip address
add address=10.0.0.6 interface=lo
add address=172.16.254.22/30 interface=ether1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ipv6 address
add address=2001:db8::6/128 advertise=no interface=lo
add address=2001:db8::6 advertise=no interface=bridge-ipv6-mpls
/mpls interface
set [ find default=yes ] mpls-mtu=1992
/mpls ldp
set distribute-for-default-route=yes enabled=yes lsr-id=10.0.0.6 transport-address=10.0.0.6
/mpls ldp interface
add hello-interval=1s hold-time=10s interface=ether1
/routing bgp peer
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=R1 remote-address=10.0.0.1 remote-as=65530 route-reflect=yes ttl=default
add address-families=ip,ipv6,l2vpn in-filter=bgp-in multihop=yes name=R5 remote-address=10.0.0.5 remote-as=65530 route-reflect=yes ttl=default
/routing ospf interface
add authentication=md5 authentication-key=5bnwqgfAomwbU9B3 dead-interval=10s hello-interval=1s interface=ether1 network-type=point-to-point
/routing ospf network
add area=backbone network=172.16.254.20/30
add area=backbone network=10.0.0.6/32
/system identity
set name=RR1
I'm guessing that might be the cause of your issue. Use different IPs with a /64 subnet for the VPLS bridge interfaces than you are using for the loopback. I expect the ping above would then work.We then assigned IPv6 /128 loopback IPs and assigned the same IP with a /64 subnet to the VPLS bridge interfaces.
I'm guessing that might be the cause of your issue. Use different IPs with a /64 subnet for the VPLS bridge interfaces than you are using for the loopback.We then assigned IPv6 /128 loopback IPs and assigned the same IP with a /64 subnet to the VPLS bridge interfaces.
I did not miss this, I can see quite well that the gateways are reachable. We offer IPv6 over VPLS tunnels and it works quite well. But given the routing tables you have shown, there is no other reason I can see (short of a bug) why the ping would fail. I expect that the duplication of IPs from the loopback to the VPLS bridge interface is not impacting the direct ping, but may be preventing the recursive routing from functioning properly. For instance, your routing tables show that the nexthop for 2001:db8::1 is 2001:db8::1, the nexthop for 2001:db8::5 is 2001:db8::5, and the nexthop for 2001:db8::6 is 2001:db8::6. This would seem to me to create a loop that can not be recovered from, hence not being reachable when it comes to recursive routing, but reachable when it comes to a direct ping response.The point is to get IPv6 ingressing at a PE switched across P routers using MPLS. You also missed the fact that I can ping R5's IPv6 loopback from R1 and vice versa, so the gateways are reachable.
/routing filter add chain=bgp-in address-family=ipv6 prefix=2001:db8::/64 prefix-length=64-128 action=discard
/routing filter add chain=bgp-out address-family=ipv6 prefix=2001:db8::/64 prefix-length=64-128 action=discard
/routing bgp peer set [ find ] in-filter=bgp-in out-filter=bgp-out
Again, what I see here as a common link is that you are assigning the same IPv6 address to the loopback interface as a /128 as you are to the VPLS interface (/64). I strongly suspect that this is the cause of your issues.IPv6 appears extremely unreliable in the GNS3 virtual lab I put together. The following initially only worked in one direction (R1 -> R5) until I restarted R5, after which it worked in both.
/ipv6 address set [ find interface=bridge-ipv6-mpls ] advertise=yes
Strange - advertise=yes shouldn't make a difference, you should not need router advertisements for this to work. They must be somehow working around an underlying issue.The environment does now appear to be working consistently, the problem was due to IPv6 addresses associated with the VPLS bridge interfaces having been set as 'advertise=no'. 2nd amendment to the lab environment:
R1 + R5 + RR1:Code: Select all/ipv6 address set [ find interface=bridge-ipv6-mpls ] advertise=yes
I didn't quite say that. I said we offer IPv6 over VPLS tunnels, but there is no BGP involved in the way we provision services - we bridge the customer at one end to a VPLS tunnel, and the other end of the tunnel has their gateway IP bound. We do L2VPN, not L3VPN.You mention having numerous production networks doing just this, I have no production experience and would appreciate it if you would be so kind as to share your knowledge in this regard.
/interface bridge set [ find name=bridge-ipv6-mpls ] auto-mac=no admin-mac=00:14:F2:00:00:01
/interface bridge set [ find name=bridge-ipv6-mpls ] auto-mac=no admin-mac=00:14:F2:00:00:05
/interface bridge set [ find name=bridge-ipv6-mpls ] auto-mac=no admin-mac=00:14:F2:00:00:06
Strange - advertise=yes shouldn't make a difference, you should not need router advertisements for this to work. They must be somehow working around an underlying issue.The environment does now appear to be working consistently, the problem was due to IPv6 addresses associated with the VPLS bridge interfaces having been set as 'advertise=no'. 2nd amendment to the lab environment:
R1 + R5 + RR1:Code: Select all/ipv6 address set [ find interface=bridge-ipv6-mpls ] advertise=yes
Yes, I have seen issues like this as well. In cases where a port leaves a bridge with auto-mac and rejoins, the link local shown in IPv6->Addresses doesn't always match the link-local that the router responds on, which leads to all kinds of other issues. Setting an admin MAC is the safest workaround.As you state the advertise option is not needed and was most probably only effecting a change by it flapping the IPv6 address when applying the change. Problem resurfaces if the layer 2 VPLS tunnels re-establish and automatically get removed and added to the bridge, thereby changing its MAC address. I assume a deeper bug relating to the link local address not updating properly to track the interface's MAC so I simply set a static MAC address on the 'bridge-ipv6-mpls' interfaces to avoid the quirk.