To start, here is the design. It is all virtualized CHRs running 7.8stable. The general idea is that there are two regions being simulated (region 1 in blue on the left, region 2 in purple on the right).
All of the routers in each region have a full mesh with each other and comprise an OSPFv3 area. One router in each region serves as an inter-region backbone router (R1 in region 1 and R2 in region 2) and is an OSPFv3 ABR between the backbone area (in pink) and the OSPFv3 areas in each region.
From every non-backbone router (R3, R4, R5, R6) to RR1 and RR2, there are Wireguard tunnels and static routes to loopbacks configured. Between the loopbacks, the routers run iBGP with RR1 and RR2 acting as route reflectors. No OSPFv3 is configured as RR1 and RR2 are never intended to handle traffic. There is a backdoor link (gre3000) between R4 and R5 but it can be safely ignored as it is not currently configured.
MPLS and LDP is configured on all of the routers (not RR1 and RR2, obviously). The "customer" LANs (light green at the bottom) are redistributed into BGP and carried to every PE router (R3, R4, R5, R6) but not the P routers (R1, R2). Other than some oddities like the full mesh and the use of tunnels, this is a pretty canonical BGP-free MPLS core design. All of the OSPFv3 and LDP configs appear to work.
A trivial test case is to ensure that R3's loopback can ping R6, as that only requires OSPFv3 to be working correctly. Everything works and the path is what we expect:
Code: Select all
[admin@R3-region1] > routing route print det where ospf and fd63:eef6:907::6 in dst-address
Flags: X - disabled, F - filtered, U - unreachable, A - active;
c - connect, s - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, m - modem, a - ldp-address, l - ldp-mapping, y - copy, a - slaac; H - hw-offloaded;
+ - ecmp, B - blackhole
Ao afi=ip6 contribution=active dst-address=fd63:eef6:907::6/128 routing-table=main gateway=fe80::2:8000:302%gre1000 immediate-gw=fe80::2:8000:302%gre1000
distance=110 scope=20 target-scope=10 belongs-to="tyrr-ospfv3"
ospf.metric=3 .type=inter
debug.fwp-ptr=0x20242540
[admin@R3-region1] > ping src-address=fd63:eef6:907::3 address=fd63:eef6:907::6
SEQ HOST SIZE TTL TIME STATUS
0 fd63:eef6:907::6 56 62 4ms653us echo reply
1 fd63:eef6:907::6 56 62 16ms735us echo reply
2 fd63:eef6:907::6 56 62 17ms185us echo reply
sent=3 received=3 packet-loss=0% min-rtt=4ms653us avg-rtt=12ms857us max-rtt=17ms185us
[admin@R3-region1] > tool traceroute fd63:eef6:907::6
Columns: ADDRESS, LOSS, SENT, LAST, AVG, BEST, WORST, STD-DEV
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV
1 fd63:eef6:907::1 0% 1 1ms 1 1 1 0
2 fd63:eef6:907::2 0% 1 2ms 2 2 2 0
3 fd63:eef6:907::6 0% 1 3.5ms 3.5 3.5 3.5 0
No problem. Now to inspect LDP. The neighbor is up and we see labels that can get us to the destination loopback (fd63:eef6:907::6).
Code: Select all
[admin@R3-region1] > mpls exp
# may/13/2023 17:48:24 by RouterOS 7.8
# software id =
#
/mpls interface
add interface=tyrr-area1-region1
/mpls ldp
add afi=ipv6 lsr-id=172.17.0.3 transport-addresses=fd63:eef6:907::3
/mpls ldp interface
add interface=tyrr-area1-region1
[admin@R3-region1] > interface list memb pr
Columns: LIST, INTERFACE
# LIST INTERFACE
0 tyrr-area1-region1 lo0
1 tyrr-area1-region1 gre1000
[admin@R3-region1] > mpls ldp neigh pr det
Flags: X - disabled, D - dynamic, I - inactive; O - operational, C - active-connect, W - passive-wait, T - throttled; t - sending-targeted-hello; v - vpls;
p - passive; d - on-demand
0 DO transport=fd63:eef6:907::1 peer=172.17.0.1:0 local-transport=fd63:eef6:907::3
addresses=2001:db8:0:1::2,fd63:eef6:907::1,fd63:eef6:907:1:5200:ff:fe07:1,fe80::2:8000:202,fe80::2:8000:302,fe80::200:5efe:8000:102,
fe80::1cb1:e4ff:fe1f:dd3e,fe80::5200:ff:fe07:0,fe80::5200:ff:fe07:1,fe80::5200:ff:fe07:2,fe80::5200:ff:fe07:3
path-vector-limit=0 on-demand=no used-afi=ipv6
[admin@R3-region1] > mpls ldp remote-mapping pr where dst-address=fd63:eef6:907::6
Flags: I - INACTIVE; D - DYNAMIC
Columns: VRF, DST-ADDRESS, LABEL, PEER
# VRF DST-ADDRESS LABEL PEER
5 ID main fd63:eef6:907::6 17 172.17.0.1:0
[admin@R3-region1] > mpls forwarding-table print where prefix=fd63:eef6:907::6
Flags: L, V - VPLS
Columns: LABEL, VRF, PREFIX, NEXTHOPS
# LABEL VRF PREFIX NEXTHOPS
2 L 18 main fd63:eef6:907::6 { nh=fe80::2:8000:302%gre1000; interface=gre1000 }
The meaningful test case here is whether R3 can ping from its "customer" facing address (fd63:eef6:907:3::1) across to R6's "customer" facing address (fd63:eef6:907:6::1). To do this, R3 would need to rely on BGP and I would expect it would synthesize an MPLS forwarding table entry towards the next-hop PE router.
Let's check BGP first:
Code: Select all
[admin@R3-region1] > routing route print where bgp
Flags: A - ACTIVE; b, a - SLAAC
Columns: DST-ADDRESS, GATEWAY, AFI, DISTANCE, SCOPE, TARGET-SCOPE, IMMEDIATE-GW
DST-ADDRESS GATEWAY AFI DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW
Ab fd63:eef6:907:6::/64 fd63:eef6:907::6 ip6 200 40 30 fe80::2:8000:302%gre1000
Looks like we see the remote route and the route reflector preserved the next-hop (fd63:eef6:907::6) like it should have. Unfortunately, we still don't see anything in the forwarding table for this route:
Code: Select all
[admin@R3-region1] > mpls forwarding-table print
Flags: L, V - VPLS
Columns: LABEL, VRF, PREFIX, NEXTHOPS
# LABEL VRF PREFIX NEXTHOPS
0 L 16 main fd63:eef6:907::1 { nh=fe80::2:8000:302%gre1000; interface=gre1000 }
1 L 17 main fd63:eef6:907::2 { nh=fe80::2:8000:302%gre1000; interface=gre1000 }
2 L 18 main fd63:eef6:907::6 { nh=fe80::2:8000:302%gre1000; interface=gre1000 }
I would expect to see an entry with fd63:eef6:907:6::/64, label 18 (same as the BGP next-hop fd63:eef6:907::6), and an identical link next-hop of { nh=fe80::2:8000:302%gre1000; interface=gre1000 } towards the OSPF next-hop that leads to fd63:eef6:907::6.
Predictably, the ping fails with a timeout:
Code: Select all
[admin@R3-region1] > ping src-address=fd63:eef6:907:3::1 address=fd63:eef6:907:6::1
SEQ HOST SIZE TTL TIME STATUS
0 fd63:eef6:907:6::1 timeout
1 fd63:eef6:907:6::1 timeout
sent=2 received=0 packet-loss=100%
[admin@R3-region1] > routing route print where fd63:eef6:907:6::1 in dst-address
Flags: A - ACTIVE; s, b, l, a - SLAAC; H - HW-OFFLOADED
Columns: DST-ADDRESS, GATEWAY, AFI, DISTANCE, SCOPE, TARGET-SCOPE, IMMEDIATE-GW
DST-ADDRESS GATEWAY AFI DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW
lH ::/0 ip6 0
As ::/0 2001:db8:0:3::1 ip6 1 30 10 2001:db8:0:3::1%ether1-internet
Ab fd63:eef6:907:6::/64 fd63:eef6:907::6 ip6 200 40 30 fe80::2:8000:302%gre1000
The packet is being sent via traditional IP routing towards the next-hop which then encounters the BGP-free core which, lacking a route, cannot get it further.
Have I forgotten to configure something with MPLS for the BGP routes to have MPLS forwarding entries created? Is it a bug? Is IPv6 on MPLS not supported correctly (I know VPNv6 isn't yet)? Full configs for all six involved routers (not R4 and R5) are attached if you are suspicious and want to look.
Thanks all