ROSv6 to v7 VPNv4 VRF Routes

RouterOS Forwarding Protocols
1

First time poster long time lurker.

We ordered new cloud core CCR2004-16G-2S+ that require ROSv7. We are having trouble getting our existing ROSv6 VPN4 configurations to work. We strongly feel our underlying MPLS (LDP) network isn’t the issue as we have ROSv6 devices and juniper ACX receiving iBGP and VPNv4 routes from our PE and have been for a few years.

OSPF, LDP, iBGP and VPNv4 routes come up on ROSv7 CCR2004-16G-2S+ without any issue. Our PE router (Juniper MX) sees VPNv4 /30 block advertised from an attached interface within a customers VRF (vrf-internet) on CCR2004. We can even route all the way to the CCR2004 /30 block. However there are no routes installed into the customers VRF vrf-internet other then the statically added /30 block attached to the vrf-internet on CCR2004.

Below is our running configuration on CCR2004-16G-2S running ROSv7.9. Is there anything that seems off to anyone in our config below?

/interface bridge
add name=PUBGWBR
/interface bridge settings
set allow-fast-path=no
/ip address
add address=100.64.34.37 comment=CE-Loopback interface=LOOP1 network=100.64.34.37
add address=100.64.20.37/22 comment=CE-MPLS-p2p interface=ether1 network=100.64.20.0
add address=xxx.xxx.79.141/30 interface=PUBGWBR network=xxx.xxx.79.140
/ip vrf
add interfaces=PUBGWBR name=vrf-internet

/mpls interface
add input=yes interface=ether1 mpls-mtu=2000
/mpls ldp
add afi=ip disabled=no lsr-id=100.64.34.37 transport-addresses=100.64.34.37
/mpls ldp accept-filter
add accept=yes comment=lo0.p disabled=no prefix=100.64.34.2/32
add accept=yes comment=lo0.pe disabled=no prefix=100.64.34.1/32
add accept=no disabled=no
/mpls ldp advertise-filter
add advertise=yes comment=LOOP1 prefix=100.64.34.37/32
add advertise=no
/mpls ldp interface
add afi=ip disabled=no hello-interval=2s interface=ether1 transport-addresses=100.64.34.37

/routing bgp template
set default address-families=ip,vpnv4 as=65101 disabled=no hold-time=1m30s keepalive-time=30s output.redistribute=static router-id=100.64.34.37 routing-table=main
/routing bgp connection
add address-families=ip,vpnv4 as=65101 disabled=no hold-time=1m30s keepalive-time=30s local.address=100.64.34.37 .role=ibgp multihop=no name=lo0.pe output.no-client-to-client-reflection=yes .redistribute=static remote.address=100.64.34.1/32 \
    .as=65101 router-id=100.64.34.37 routing-table=main
/routing bgp vpn
add disabled=no export.redistribute=connected,static .route-targets=65101:11 import.route-targets=65101:11 .router-id=100.64.34.37 label-allocation-policy=per-vrf name=bgp-mpls-vpn-1 route-distinguisher=65101:11 vrf=vrf-internet

I want to be clear that we do see VPNv4 routes on the CCR2004-16G-2S+ from our PE router but the VRF (vrf-internet) can not reach them.

routing/route/print  where afi=vpn4
Flags: A - ACTIVE; b, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, AFI, DISTANCE, SCOPE, TARGET-SCOPE, IMMEDIATE-GW
   DST-ADDRESS                         GATEWAY               AFI   DISTANCE  SCOPE  TARGET-SCOPE  IMMEDIATE-GW
Ay xxx.xxx.79.140/30&65101:11          PUBGWBR@vrf-internet  vpn4       200     40            10  PUBGWBR
Ab xxx.xxx.79.250:1                    100.64.34.1           vpn4       200     40            30  100.64.20.1%ether1
Ab 100.64.34.3/32&xxx.xxx.79.250:1     100.64.34.1           vpn4       200     40            30  100.64.20.1%ether1
Ab xxx.xxx.65.0/24&xxx.xxx.79.250:1    100.64.34.1           vpn4       200     40            30  100.64.20.1%ether1
Ab xxx.xxx.79.250/31&xxx.xxx.79.250:1  100.64.34.1           vpn4       200     40            30  100.64.20.1%ether1
Ab xxx.xx.12.0/23&xxx.xxx.79.250:1     100.64.34.1           vpn4       200     40            30  100.64.20.1%ether1

Is anyone else having an issue like this or are we just missing something? We have a ticket open with MikroTik (SUP-113133).

2

downgrade to 7.8 and try again

3

Have you verified that fast path is disabled in ip settings?

4

I do have fast path disabled in the ip settings as well. I did not include that in my first post.

 
[mikrotik@DEVL] > ip/settings/export
# may/09/2023 15:56:28 by RouterOS 7.9
#
# model = CCR2004-16G-2S+
/ip settings
set allow-fast-path=no
5

Just downgrade to ROS 7.8
i have exactly same issue . Downgrade solve it .

6

You are right… That fixed it, everything came up with the exact configuration.

So is the solution to run on 7.8 for VPNv4 VRF and not stable 7.9?

7

Better create ticket in their support . i created but if it will be more people i presume it will get more priority.

PS: today i got a weird answer from support where they say something like that now router decide to which VRF routes have to be installed using RD instead of RT.
Due to it . It is look like that they now broken route exchange between different vrfs and now not possible to exchange routes usign RT -

8

Got the same answer in a ticket opened on 4th May, in which I reported the same behaviour: routes being filtered by route-distinguisher before being imported into a VRF.
This behaviour was introduced in RouterOS 7.9, as the changelog indicates: “bgp - improved BGP VPN selection”.
However, after I pointed out RFC 4364, Mikrotik Support came back yesterday, 10th May, and said that will change this behaviour in the next beta.
Hope they can sort this out accordingly with the RFCs and the industry standards.
For now, L3 MPLS VPNs should work in RouterOS 7.8. At least from my lab testing point of view, all seems OK regarding L3 MPLS VPNs in RouterOS 7.8.

9

Nope VPRN still working really bad. For example in ROS 7.8 if you add on PE router bridge interface to VRF and start to ping it over VPRN you will see that it inbound interface is “unknown” and destination interface is also “unknown”. I was forced to dowgrade to ROS 6.

10

We are stuck in a hard place here. We ordered a few of CCR2004-16G-2S+ (ARM64) because CCR1009-7G-1C-1S (TILE) are longer available and we need the 10G SFP+ port for some of our customers. The CCR2004-16G-2S+ (ARM64) does not have the ability to downgrade to ROSv6. We would’ve done that in a heart beat if we could.

In our lab using the posted configuration it appears the L3 MPLS VPNs work on 7.8. It’s been running for almost 2 days and we’ve rebooted the unit 5 + times. I feel this is fairly basic VPNv4 configuration.

So it sounds like our options here are (please correct me if I am wrong)

  1. Run CCR2004-16G-2S+ (ARM64) on ROS 7.8
  2. Find second hand (TILE) and run ROS 6
  3. User alternative hardware (Juniper ACX)
11

I usually don’t use bridging at all (production and lab environment), so I didn’t have the chance to hit the bug you pointed out.
It’s good to know that bridging is buggy with VPRN in ROS 7; for sanity, I’ll also include this test in the qualification policy.
Unfortunately, I cannot downgrade to ROS 6, as the platform I’m using (CCR2004-1G-12S+2XS) has support starting with ROS7. Good thing I don’t have a need for bridging in this setup.

12

Same here.
We also stuck that it is almost not possible to buy anymore CCR1036. We have only 3 routers on stock . Tried 2004 but because ot ROS 7 which is still not stable with VPRN we started to move 2004 to P role and old CCR 1036 which now P role to PE role and ROS6. As i see true MPLS work acceptable on ROS7. Only problem i got was with aggregated MPLS routes but i not had a lot of time to reserach it properly and create ticket. So i not use aggregation anymore between zones with MPLS enabled routes.

13

I just wanted to post a quick update. It appears the upgrading to 7.10 fixed the VPNv4 issues we had.

14

Nope not everything fixed. Traffic to bridge interface which is member of VRF still don`t hit mangle prerouting and also not possible to ping it from another side of tunnel.

15

apparently this still is the “at the moment way to go” in routeros v7

https://help.mikrotik.com/docs/pages/viewpage.action?pageId=328206
at least after version 7.8 where the import/export at least happens