Hello,

I have CCR1036 with OS 6.47.10 in production which was configured with separate bridges for the WAN uplink and LAN site, For IPv4 it works now, but I have been asked to add support for IPv6, but I have issues with routing IPv6 between LAN and WAN bridges.

- From br_wan I can reach the ISP router via IPv6 LL
- From br_wan I can reach the ISP router if set the GUA address on br_wan (via GUA addresses)
- From br_lan I can not reach br_wan via IPv6 LL, but I expect this as LL is link-local(bridge limited), and if bridges as separated then would not reach it.
- From br_lan I can not reach br_wan via GUA if both bridges are configured with the GUA address
- IPv6 is enabled (as LL is generated by Mikrotik)
- IPv6 firewall is cleaned
- IGMP Snooping / Proxy disabled

Ping from Server to link-local address of br_wan interface is as follows (looks like multicast neighbour solicitations is not forwarded from br_lan to br_wan)

/tool/sniffer/quick ipv6-address=fe80::7ec2:55ff:fe69:f2af interface=vlan1177
Columns: INTERFACE, TIME, NUM, DIR, SRC-MAC, DST-MAC, SRC-ADDRESS, DST-ADDRESS, PROTOCOL, SIZE, CPU
INTERFACE TIME NUM DIR SRC-MAC DST-MAC SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU
vlan1177 0.384 1 <- 7C:C2:55:69:F2:AF 33:33:FF:F3:C1:76 fe80::7ec2:55ff:fe69:f2af ff02::1:fff3:c176 ipv6:icmpv6 86 31
vlan1177 1.408 2 <- 7C:C2:55:69:F2:AF 33:33:FF:F3:C1:76 fe80::7ec2:55ff:fe69:f2af ff02::1:fff3:c176 ipv6:icmpv6 86 31
vlan1177 2.433 3 <- 7C:C2:55:69:F2:AF 33:33:FF:F3:C1:76 fe80::7ec2:55ff:fe69:f2af ff02::1:fff3:c176 ipv6:icmpv6 86 31
vlan1177 3.456 4 <- 7C:C2:55:69:F2:AF 33:33:FF:F3:C1:76 fe80::7ec2:55ff:fe69:f2af ff02::1:fff3:c176 ipv6:icmpv6 86 31

Any idea if this scenario is supported by Mikrotik(reach ISP from br_lan)?


Scenario as below

link local addresses, as the name suggests, are only valid within a layer2 broadcast domain.

You say "From br_lan I can not reach br_wan via GUA if both bridges are configured with the GUA address" - you should assign different GUA addresses to each otherwise routing will not work. Typically you would be assigned or use an autoconfigured (SLAAC) or DHCP address (/64) on the WAN and use routed static or DHCP delegated prefix addresses for any other interfaces.

link local addresses, as the name suggests, are only valid within a layer2 broadcast domain.

You say "From br_lan I can not reach br_wan via GUA if both bridges are configured with the GUA address" - you should assign different GUA addresses to each otherwise routing will not work. Typically you would be assigned or use an autoconfigured (SLAAC) or DHCP address (/64) on the WAN and use routed static or DHCP delegated prefix addresses for any other interfaces.

ISP just provide a pool /48 which I then then split internally. ISP just takes one address 2a02:XXXX:A::1/64 on their router

I have assigned GUA addresses for br_wan and br_lan (different)

ISP Router has - 2a02:XXXX:A::1/64
br_wan - 2a02:XXXX:A::2/128
br_lan - 2a02:XXXX:A::3/128

From br_lan I can not ping br_wan via GUA

/ipv6/address> /ping 2a02:XXXX:A::2 interface=br_lan
SEQ HOST SIZE TTL TIME STATUS
0 no route to host
1 no route to host
2 no route to host
3 no route to host
sent=4 received=0 packet-loss=100%

ICMP via GUA from br_wan works to ISP but does not work from br_lan

The br_wan address should be /64, and the ISP router should be configured to route the /48 to this address.
The br_lan address should again be /64 and also a different subnet.

The br_wan address should be /64, and the ISP router should be configured to route the /48 to this address.
The br_lan address should again be /64 and also a different subnet.

It does not matter what I configure on br_wan and br_lan as IPv6 routing between br_lan and br_wan does not work

Example as you wanted:

2a02:a3XX:8::1 - ISP Router
2a02:a3XX:8:1::1/64 - br_wan (separate /64)
2a02:a3XX:8:2::1/64 - br_lan (separate /64)

Ping from br_lan to br_wan does not work

/ping 2a02:a3XX:8:1::1 interface=br_lan
SEQ HOST SIZE TTL TIME STATUS
0 no route to host
1 no route to host
2 no route to host

Ping from br_lan to ISP router does not work (as it can not reach br_wan so also can not reach uplink to ISP interface which is a member of br_wan)

/ping 2a02:a3XX:8::1 interface=br_lan
SEQ HOST SIZE TTL TIME STATUS
0 no route to host
1 no route to host
2 no route to host

br_wan can reach ISP router as uplink to ISP router is member of this bridge and default GW set to LL of ISP router ( ::/0 fe80:1bff:fe4b:c444%br_wan)

/ping 2a02:a3XX:8::1 interface=br_wan
SEQ HOST SIZE TTL TIME STATUS
0 2a02:a3XX:8::1 56 64 20ms386us echo reply
1 2a02:a3XX:8::1 56 64 10ms729us echo reply

The br_wan address should be /64, and the ISP router should be configured to route the /48 to this address.
The br_lan address should again be /64 and also a different subnet.

It does not matter what I configure on br_wan and br_lan as IPv6 routing between br_lan and br_wan does not work

Example as you wanted:

2a02:a3XX:8::1 - ISP Router
2a02:a3XX:8:1::1/64 - br_wan (separate /64)

You have configured br_wan to be in a different subnet, for the same subnet as the ISP router it would be 2a02:a3XX:8::2/64

If the ISP just presents 2a02:a3XX:8::/48 on their router interface you will not be able to make it work. Some vendors implement NDproxy, equivalent to proxy-arp in IPv4, but Mikrotik don't.

A typical setup would be the ISP provides a /64 just for the WAN link and a /48 routed to your address on that link. Using part of the /48 is also possible, for example:
ISP router - IP 2a02:a3XX:8::1/64, route 2a02:a3XX:8::/48 via 2a02:a3XX:8::2
Your router - IP 2a02:a3XX:8::2/64, blackhole route 2a02:a3XX:8::/48 (this prevents routing loops), other /64s assigned to interfaces as required.

Ping from br_lan to br_wan does not work

/ping 2a02:a3XX:8:1::1 interface=br_lan

With the command quoted you told ROS to "ping said address, but use br_lan as egress interface" ... which is overriding routing decission. And IPv6 address of br_wan is not accessible via br_lan.

In short: you can not test routing between different IPv6 subnets by using router alone.

A typical setup would be the ISP provides a /64 just for the WAN link and a /48 routed to your address on that link.

Another option is for ISP to provide /48 (or /56) via DHCPv6 prefix delegation and routing (etc.) via RAs. It can be a "statically assigned" prefix, just like "static IPv4 address" (usually bound to MAC address of CPE's WAN interface which also determines IPv6 ULA).
I guess that ISP's DHCPv6 server configures ISP's router to route delegated prefix via DHCPv6 client's ULA. So no need for GUA on CPE's WAN interface (but if CPE does have GUA on WAN interface, it's result of SLAAC).

It does not matter what I configure on br_wan and br_lan as IPv6 routing between br_lan and br_wan does not work

Example as you wanted:

2a02:a3XX:8::1 - ISP Router
2a02:a3XX:8:1::1/64 - br_wan (separate /64)

You have configured br_wan to be in a different subnet, for the same subnet as the ISP router it would be 2a02:a3XX:8::2/64

If the ISP just presents 2a02:a3XX:8::/48 on their router interface you will not be able to make it work. Some vendors implement NDproxy, equivalent to proxy-arp in IPv4, but Mikrotik don't.

A typical setup would be the ISP provides a /64 just for the WAN link and a /48 routed to your address on that link. Using part of the /48 is also possible, for example:
ISP router - IP 2a02:a3XX:8::1/64, route 2a02:a3XX:8::/48 via 2a02:a3XX:8::2
Your router - IP 2a02:a3XX:8::2/64, blackhole route 2a02:a3XX:8::/48 (this prevents routing loops), other /64s assigned to interfaces as required.

If I configure 2a02:a3XX:8::2 for br_wan, it does not help. I have already tried this.

For example
br_wan - 2a02:a3XX:8::2/64
br_lan - 2a02:a3XX:8::3/64

Mikrotik in my opinion should be able to route between those GUA addresses as those are internally assigned and GUA must be routed, but it does not.

I can not ping br_wan using GUA from br_lan, and can not reach ISP router (from br_lan)

I think it is a little bit around what ISP expects, as they probably would like to reach them via LL

- ISP Router LL is reachable from br_wan
- Default route added (::/) using LL to ISP
- At this point, I believed that if I configured the default route on the Server to br_lan LL, Mikrotik would internally route it to default GW (GUA) but it does not.

A typical setup would be the ISP provides a /64 just for the WAN link and a /48 routed to your address on that link.

Another option is for ISP to provide /48 (or /56) via DHCPv6 prefix delegation and routing (etc.) via RAs. It can be a "statically assigned" prefix, just like "static IPv4 address" (usually bound to MAC address of CPE's WAN interface which also determines IPv6 ULA).
I guess that ISP's DHCPv6 server configures ISP's router to route delegated prefix via DHCPv6 client's ULA. So no need for GUA on CPE's WAN interface (but if CPE does have GUA on WAN interface, it's result of SLAAC).

DHCPv6 is not enabled from ISP.

From their point of view, we should reach them via LL and set the default route on them that's it. They just forwarded to us /48 prefix. Golden solution would be to use ONLY LL but as I see it is not possible to use while having multiple bridges (I did not configure them some else did for some reason)

For example
br_wan - 2a02:a3XX:8::2/64
br_lan - 2a02:a3XX:8::3/64

Mikrotik in my opinion should be able to route between those GUA addresses as those are internally assigned and GUA must be routed, but it does not.

No. This doesn't just apply to Mikrotik, addresses in the same subnet are only reachable within the same layer2 network, they are never routed.

The equivalent to what you are attempting in IPv4, which also would not work, would be
br_wan - 192.168.1.2/24
br_lan - 192.168.1.3/24

They just forwarded to us /48 prefix.

Forwarded to what address? This is different to the interface on their gateway being given a /48 subnet mask.

A few ISPs seem clueless about this. I suggest reading https://www.ripe.net/publications/docs/ripe-690/, in particular section 4.1

For example
br_wan - 2a02:a3XX:8::2/64
br_lan - 2a02:a3XX:8::3/64

Mikrotik in my opinion should be able to route between those GUA addresses as those are internally assigned and GUA must be routed, but it does not.

No. This doesn't just apply to Mikrotik, addresses in the same subnet are only reachable within the same layer2 network, they are never routed.

The equivalent to what you are attempting in IPv4, which also would not work, would be
br_wan - 192.168.1.2/24
br_lan - 192.168.1.3/24

As mentioned in post #5 tested also different /64 on both br_wan and br_lan, they can not reach each other

Ok, I get traffic through between bridges.

DAc 2a02:aXXX:8::/64 br_wan 0
DAc 2a02:aXXX:8:1::/64 br_lan 0

But the default route is not working, I still can only reach IPv6 GUA address on 2a02:aXXX:8::/64 prefix (br_wan) but for some reason other GUA addresses which are not member of routing are not forwarded to default

I am adding the default as follows

add dst-address=::/0 gateway=2a02:aXXX:8::1%br_wan

As mentioned in post #5 tested also different /64 on both br_wan and br_lan, they can not reach each other

As mentioned in post #7 above, your testing is flawed.

I am adding the default as follows

add dst-address=::/0 gateway=2a02:aXXX:8::1%br_wan

You should set gateway IPv6 address to address of upstream (i.e. ISP's) router. Not IPv6 address of your WAN interface. If you don't know GUA of ISP's router, then it may be possible to use it's ULA in route definition.

I am adding the default as follows

add dst-address=::/0 gateway=2a02:aXXX:8::1%br_wan

You should set gateway IPv6 address to address of upstream (i.e. ISP's) router. Not IPv6 address of your WAN interface. If you don't know GUA of ISP's router, then it may be possible to use it's ULA in route definition.

2a02:aXXX:8::1 is an address of ISP router

I think the problem is with Neighbout Solicitation, not sure if can forward it between bridges.

When pinging ISP router from br_lan it sends NS but does not get a reply as multicast packet is not forwarded between br_wan and br_lan to host

When pinging ISP router from br_lan it sends NS but does not get a reply as multicast packet is not forwarded between br_wan and br_lan to host

Again: how exactly are you pinging "from br_lan"? I pointed out in post #7 why it is generally flawed due to misunderstanding the meaning of interface property. You did not ack that you've seen it so far ... so you may still be doing the wrong thing.

I think the problem is with Neighbout Solicitation, not sure if can forward it between bridges.

When pinging ISP router from br_lan it sends NS but does not get a reply as multicast packet is not forwarded between br_wan and br_lan to host

No it can't, see post #6.

The ISP should be routing the /48 to you via a /64 (can be part of the /48 or a completely separate subnet), not just attaching the entire /48 to the interface on their router.

I think the problem is with Neighbout Solicitation, not sure if can forward it between bridges.

When pinging ISP router from br_lan it sends NS but does not get a reply as multicast packet is not forwarded between br_wan and br_lan to host

No it can't, see post #6.

The ISP should be routing the /48 to you via a /64 (can be part of the /48 or a completely separate subnet), not just attaching the entire /48 to the interface on their router.

I have asked my IT for technical contract details and got confirmation that ISP configured 2a02:aXXX:8::1/48 on their router.

When pinging ISP router from br_lan it sends NS but does not get a reply as multicast packet is not forwarded between br_wan and br_lan to host

Again: how exactly are you pinging "from br_lan"? I pointed out in post #7 why it is generally flawed due to misunderstanding the meaning of interface property. You did not ack that you've seen it so far ... so you may still be doing the wrong thing.

I am pinging from server connected to br_lan, my gateway is br_lan IPv6 LL and I can ping GUA address of br_wan, but can not ping ISP GUA. When I dump packets to wireshark from interface going to ISP, ICMPv6 is seen from server but ISP router send NS via multicast to my host, which is not responding as it is in a different bridge (multicast is limited by bridge) and it does not work

ISP Router has 2a02:aXXX:8::1/48
br_wan - 2a02:aXXX:8::2/48 (or I can commit it if does not want to have GUA)
br_lan - 2a02:aXXX:8:1:1/64
Server connected to br_lan - 2a02:aXXX:8:1:2/64

From server I can ping
a) br_lan (obviously)
b) br_wan if any GUA address is configured (from different prefix)
c) CAN not ping ISP Router but my traffic is reaching them (can see ICMPv6 from server on the interface to ISP)

So routing is not a problem now, but as IPv6 works via multicast to get neighbours and multicast is not forwarded from br_wan to br_lan

It seems to me that there's a major error in config ... perhaps on ISP's side.

The setup you have is similar to this one from IPv4:

ISP router: 1.1.0.1/16
your router WAN: 1.1.0.2/16
your router LAN: 1.1.1.1/24
server in LAN: 1.1.1.2/24

It's clear that ISP's router expects it's able to deliver packet with destination address 1.1.1.2 directly as it's inside network 1.1.0.0/16 ... but it can't because your router is in the way. The problem can be solved In both cases (IPv4 and IPv6) by properly configuring link between ISP and your router and by adding a route on ISP router.

If ISP doesn't want to do it properly, then in IPv4 the problem can be worked around by using proxy ARP. And in IPv6 similar solution is to use ND proxy which doesn't seem to be supported by ROS.

It seems to me that there's a major error in config ... perhaps on ISP's side.

The setup you have is similar to this one from IPv4:

ISP router: 1.1.0.1/16
your router WAN: 1.1.0.2/16
your router LAN: 1.1.1.1/24
server in LAN: 1.1.1.2/24

It's clear that ISP's router expects it's able to deliver packet with destination address 1.1.1.2 directly as it's inside network 1.1.0.0/16 ... but it can't because your router is in the way. The problem can be solved In both cases (IPv4 and IPv6) by properly configuring link between ISP and your router and by adding a route on ISP router.

If ISP doesn't want to do it properly, then in IPv4 the problem can be worked around by using proxy ARP. And in IPv6 similar solution is to use ND proxy which doesn't seem to be supported by ROS.

I get it working by moving link to br_wan bridge, now I can use whatever prefix I want. The problem was with IPv6 multicast packets (Neighout Solicitation/Advertisement ) they could not be routered between bridges and the host could not communicate

Topic may be closed. Issue fixed by operator.

Added dedicated /64 prefix between ISP Router and Mikrotik
Routed /48 prefix via /64 to us

Added dedicated /64 prefix between ISP Router and Mikrotik
Routed /48 prefix via /64 to us

@kafart: Did you need to rework your config to contain only one bridge? I'm in a similar situation where I have the uplink VLAN on an "uplink" bridge, and internal VLANs on another, "internal" bridge (for VMs) on a CHR. IPv6 traffic is flowing fine between the VMs over multiple VLANs through the CHR, but it stops packets between the uplink bridge and the internal bridge for IPv6. Only the router has global IPv6 connectivity this way. IPv4 has no problems either way. My ISP is providing a single /48 prefix, where the very first address is their default route for v6, similarly to your setup in OP.