IPv6 connection lost between to MT-routers

I have been pulling my hair out over this problem since it popped up last Sunday, but I have no clue anymore where to look. Most likely I have done something stupid myself, but I can’t figure out what it could be. I didn’t think I made any (big) configuration changes prior to when the problems started.

At one site I have two MT-routers running: a 750GL as border router and an 1100AHX2. The two are directly connected, routing both IPv4 and IPv6. Both are running RouterOS 6.35.4 since last Wednesday. Last Sunday the IPv6 connection suddenly broke down and it just won’t come up anymore. It’s just as if both routers can’t “see” eachother anymore (through IPv6, that is; IPv4 is no problem whatsoever).

Even a ping to the link-local address of each interface results in a time-out.

[admin@border] /ipv6 firewall filter> /tool torch interface=ether4-school src-address=0.0.0.0/0 src-address6=::/0 ip-protocol=any
MAC-PROTOCOL    IP-PROT... SRC-ADDRESS                                       TX
ip              tcp        10.64.0.121                                  5.4kbps
ip              udp        10.1.0.2                                        0bps
ipv6            udp        fe80::4e5e:cff:fee3:7f20                        0bps
                                                                        5.4kbps

[admin@border] /ipv6 firewall filter> :ping count=1 fe80::4e5e:cff:fee3:7f20%ether4-school
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0 fe80::4e5e:cff:fee3:7f20                                timeout
    sent=1 received=0 packet-loss=100%



[admin@router] /ipv6 route> /tool torch interface=ether11-uplink src-address=0.0.0.0/0 src-address6=::/0 ip-protocol=any
MAC-PROTOCOL    IP-PROT... SRC-ADDRESS                                       TX
ip              tcp        10.1.0.1                                     1056bps
ip              tcp        xxx.xxx.xx.xx                               10.7kbps
ip              udp        10.1.0.1                                        0bps
ipv6            udp        fe80::d6ca:6dff:fe38:f904                       0bps
                                                                       11.8kbps

[admin@router] /ipv6 route> :ping count=1 fe80::d6ca:6dff:fe38:f904%ether11-uplink
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0 fe80::d6ca:6dff:fe38:f904                               timeout
    sent=1 received=0 packet-loss=100%

I have checked the configurations of both routers more than once, but I must be overlooking something trivial.

Does anybody recognize these symptoms? Any suggestions?

Ipv6 firewall is not blocking traffic?

Ok, I think I have finally figured it out. It appears that this single line in the IPv6 mangle rules is the culprit:

add action=change-dscp chain=output new-dscp=48 passthrough=no

I’m doing some QoS at the WAN-side, with a limited set of DSCP-values. So why not give a high value to outgoing traffic of the routers themselves? At the IPv4-side the same rules doesn’t give any trouble, so why does it result in this strange behaviour under IPv6?

There must be something else or some other rules that rely on DSCP filtering. Because just setting DSCP on IPV6 packet doe snot make it drop.

I have been able to reproduce this behaviour in a test environment.

This is the setup:

  • Router 1: 751G-2HnD running RouterOS 6.34.6
  • Router 2: 951Ui-2HnD running RouterOS 6.35.4

Both routers have been installed with the following (active) packages: advanced-tools, dhcp, ipv6, security and system.

Connections Router 1:

  • ether1: “WAN”-side
  • ether2: to Router 2
  • ether5: LAN

Connections Router 2:

  • ether1: to Router 1
  • ether2 to ether5: bridged like in default config

Connect your PC/laptop to Router 2.

Configuration of Router 1:

/interface ethernet
set [ find default-name=ether1 ] name=ether1-uplink
set [ find default-name=ether5 ] name=ether5-locmgt
/ip pool
add name=pool88 ranges=192.168.88.192/27
add name=pool-eth2 ranges=10.2.0.192/27
/ip dhcp-server
add address-pool=pool88 disabled=no interface=ether5-locmgt lease-time=2h \
    name=server88
add address-pool=pool-eth2 disabled=no interface=ether2 lease-time=2h name=\
    server-eth2
/ip settings
set allow-fast-path=no rp-filter=strict
/ip address
add address=192.168.88.1/24 interface=ether5-locmgt network=192.168.88.0
add address=10.2.0.1/24 interface=ether2 network=10.2.0.0
/ip cloud
set update-time=no
/ip dhcp-client
add default-route-distance=0 dhcp-options=hostname,clientid disabled=no \
    interface=ether1-uplink
/ip dhcp-server config
set store-leases-disk=never
/ip dhcp-server network
add address=10.2.0.0/24 dns-server=10.2.0.1 gateway=10.2.0.1
add address=192.168.88.0/24 dns-server=192.168.88.1 gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes cache-max-ttl=8h
/ip firewall mangle
add action=change-dscp chain=output new-dscp=48 passthrough=no
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1-uplink
/ip route
add distance=1 dst-address=10.0.0.0/8 type=unreachable
add distance=1 dst-address=10.2.1.0/24 gateway=10.2.0.6
/ipv6 address
add address=2001:xxxx:yyyy:zzz0::cafe advertise=no interface=ether1-uplink
add address=2001:xxxx:yyyy:zzz5::1 interface=ether5-locmgt
add address=2001:xxxx:yyyy:zzz2::1 interface=ether2
/ipv6 firewall filter
add chain=input protocol=icmpv6
add chain=input connection-state=established
add chain=input connection-state=related
add action=drop chain=input connection-state=invalid
add action=reject chain=input in-interface=ether1-uplink reject-with=\
    icmp-admin-prohibited
/ipv6 firewall mangle
add action=change-dscp chain=output comment="set dscp to 48" disabled=yes \
    new-dscp=48 passthrough=no
/ipv6 route
add distance=1 gateway=fe80::4e5e:cff:fe27:42e9%ether1-uplink
add distance=1 dst-address=2001:xxxx:yyyy:zzz0::/60 type=unreachable
add distance=1 dst-address=2001:xxxx:yyyy:zzza::/64 gateway=\
    fe80::4e5e:cff:fefe:ed%ether2
/system clock
set time-zone-autodetect=no time-zone-name=Europe/Amsterdam
/system identity
set name=router1
/system ntp client
set enabled=yes
/system package update
set channel=bugfix

Configuration of Router 2:

/interface bridge
add admin-mac=4C:5E:0C:FE:00:EE auto-mac=no name=bridge
/interface ethernet
set [ find default-name=ether2 ] name=ether2-master
set [ find default-name=ether3 ] master-port=ether2-master
set [ find default-name=ether4 ] master-port=ether2-master
set [ find default-name=ether5 ] master-port=ether2-master
/ip neighbor discovery
set ether1 discover=no
/ip pool
add name=pool-10-2-1 ranges=10.2.1.192/27
/ip dhcp-server
add address-pool=pool-10-2-1 disabled=no interface=bridge name=server-10-2-1
/interface bridge port
add bridge=bridge comment=defconf interface=ether2-master
add bridge=bridge comment=defconf
/ip address
add address=10.2.1.1/24 interface=bridge network=10.2.1.0
add address=10.2.0.6/24 interface=ether1 network=10.2.0.0
/ip cloud
set update-time=no
/ip dhcp-server config
set store-leases-disk=never
/ip dhcp-server network
add address=10.2.1.0/24 dns-server=10.2.1.1 gateway=10.2.1.1
/ip dns
set allow-remote-requests=yes servers=10.2.0.1
/ip firewall mangle
add action=change-dscp chain=output comment="set dscp to 48" new-dscp=48 \
    passthrough=no
/ip route
add distance=1 gateway=10.2.0.1
/ipv6 address
add address=2001:xxxx:yyyy:zzz2::6 advertise=no interface=ether1
add address=2001:xxxx:yyyy:zzza::1 interface=bridge
/ipv6 firewall filter
add chain=input protocol=icmpv6
add chain=input connection-state=established
add chain=input connection-state=related
add action=drop chain=input connection-state=invalid
add action=reject chain=input in-interface=ether1 reject-with=\
    icmp-admin-prohibited
/ipv6 firewall mangle
add action=change-dscp chain=output comment="set dscp to 48" disabled=yes \
    new-dscp=48 passthrough=no
/ipv6 route
add distance=1 gateway=fe80::d6ca:6dff:fe52:60d3%ether1
/system clock
set time-zone-autodetect=no time-zone-name=Europe/Amsterdam
/system identity
set name=router2
/system ntp client
set enabled=yes server-dns-names=europe.pool.ntp.org
/system routerboard settings
set init-delay=0s silent-boot=yes
/tool mac-server
set [ find default=yes ] disabled=yes
add interface=bridge
/tool mac-server mac-winbox
set [ find default=yes ] disabled=yes
add interface=bridge

Find and replace 2001:xxxx:yyyy:zzz with a /60 of your own environment. Adjust the references to MAC- and link local addresses according to your own equipment.

Notice the lines

add action=change-dscp chain=output comment="set dscp to 48" \
    new-dscp=48 passthrough=no

in both IPv4 and IPv6 firewall mangle rules.

When you enable this in IPv6 in both routers, your IPv6-connectivity will go down pretty quickly. (Actually, this happens within a minute, so I don’t understand why it took me earlier so long to find the cause.) Your IPv4-connectivity will continue to work normally.

I don’t see why it would be a good idea to change the DSCP of all output.
Add some matching to that… Now you change the DSCP of ICMPv6 output and I bet that is what is making it fail.

Admittedly, in hindsight it probably wasn’t a good idea, but that is not the point. As far as I know it should be working, regardless of what I do to the DSCP.