IPv6 over vlan issues

CaioPet · January 28, 2021, 11:32pm

Hi guys!

Unfortunately, I’m facing some issues with my routerboard while trying to connect my network using IPv4/IPv6 via VLAN set up by my ISP.

Before I go into the details I’ll post below a diagram that shows my topology:
Simple IPv6.png
Okay, so basically I’ve tried this before on another network, and it works!
The only catch is that in the working network, the IPv6 P2P is not configured via VLAN. It’s the same config posted below (except for the VLAN part of course).
Here is the config:

Interfaces, IPv4 (firewall ommited) and IPv6:

/interface bridge
add fast-forward=no name="WAN Bridge"
/interface ethernet
set [ find default-name=ether1 ] name="eth1 - WAN"
/interface vlan
add interface="WAN Bridge" name=IPv4 vlan-id=3789
add interface="WAN Bridge" name=IPv6 vlan-id=33
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge="WAN Bridge" interface="eth1 - WAN"
/interface list member
add disabled=yes interface="WAN Bridge" list=LAN
add disabled=yes interface="eth1 - WAN" list=WAN
# IPv4
/ip dhcp-server
add address-pool=dhcp_pool29 bootp-support=dynamic interface=bridge name=dhcp1 relay=192.168.88.1
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=189.xx.xx.x6/28 interface=IPv4 network=189.xx.xx.x2
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1 netmask=24
/ip dns
set allow-remote-requests=no servers=8.8.8.8,8.8.4.4,2001:4860:4860::8888,2001:4860:4860::8844
/ip dns static
add address=192.168.88.1 name=router.lan
/ip neighbor discovery-settings
set discover-interface-list=LA
/ip route
add distance=1 gateway=189.xx.xx.x3
# IPv6
/ipv6 dhcp-server
add address-pool=pool1 interface=bridge name=server1
/ipv6 pool
add name=P2P prefix=2001:DB8:ffff:fffe::108/126 prefix-length=126
add name=pool1 prefix=2001:DB8:3::/48 prefix-length=56
/ipv6 address
add address=::2/126 advertise=no from-pool=P2P interface=IPv6
add address=2001:DB8:3::1 interface=bridge
/ipv6 route
add disabled=yes distance=1 dst-address=2000::/3 gateway=2001:DB8:ffff:fffe::109
/ipv6 nd
set [ find default=yes ] hop-limit=64 managed-address-configuration=yes other-configuration=yes ra-interval=30s-1m30s
/ipv6 nd prefix default
set preferred-lifetime=1m30s valid-lifetime=1m30s
/ipv6 settings
set accept-router-advertisements=no
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
add address=::224.0.0.0/100 comment="defconf: other" list=bad_ipv6
add address=::127.0.0.0/104 comment="defconf: other" list=bad_ipv6
add address=::/104 comment="defconf: other" list=bad_ipv6
add address=::255.0.0.0/104 comment="defconf: other" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=forward in-interface=IPv6 src-address=fe00::/7
add action=accept chain=input comment="Router - Allow IPv6 ICMP" protocol=icmpv6
add action=accept chain=input comment="Router - Accept established connections" connection-state=established
add action=accept chain=input comment="Router - Accept related connections" connection-state=related
add action=drop chain=input comment="Router - Drop invalid connections" connection-state=invalid
add action=accept chain=input comment="Router- UDP" protocol=udp
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept UDP traceroute" port=33434-33534 protocol=udp
add action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp src-address=fe80::/16
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec

Ok, so it does not work on VLAN… But what doesn’t?

If I try to ping my ISP gateway which is “2001:DB8:ffff:fffe::109” it gets a timeout…
So I investigated some more and I found that the ND is not finding my neighbor and also, traffic goes only in the TX direction on the VLAN interface (?)

I’ve posted in the attachments some of my tests.

Pinging
Packet Sniffing
ND List
Torch to explore VLAN IDs

I’ve tried to reason with the ISP and they’ve been accommodating. Unfortunately, we haven’t yet found the core problem.

Znevna · January 29, 2021, 10:46am

I donno about the VLAN config but your whole IPv6 config just looks wrong.
Shouldn’t you have 2001:DB8:ffff:fffe::8/126 set on your IPv6 VLAN Interface? What is that ::2 doing there? which is outside the said /126.
I see no purpose for that P2P pool.
And where is your gateway? You’ve mentioned 2001:DB8:ffff:fffe::9 as the gateway but there’s no sign of it in the config. Just one disabled route.
Shouldn’t you have a default route to that gateway?

CaioPet · January 29, 2021, 12:31pm

Znevna, “2001:DB8:ffff:fffe::9” its just to explain and mask my real IPv6 ::2 referees to the second IP in the pool range for instance…
if 2001:DB8:ffff:fffe::8/126 is the pool then MK will assign 2001:DB8:ffff:fffe::10a/126 to the interface. It’s just notation stuff.

Znevna · January 29, 2021, 12:54pm

That’s not how it works, and that ::10a is outside that specified /126, again.
But since you seem to know better, fix it yourself.
You’re still missing a proper default route/gateway.
And if your ISP routed that /48 through that ::8 they gave you, you have to use that ::8, not whatever you want.
Cheers.

Sob · January 29, 2021, 2:09pm

What exactly you got from ISP? Because if nothing else, what you do on “IPv6” interface, which if I understand it correctly is your WAN port for IPv6, looks completely wrong. If you got some /126 connecting subnet for that, it should be just static address and gateway, and no need to do anything with pool.

CaioPet · January 29, 2021, 2:59pm

My ISP routes a /48 to my router using a static IP in the /126 subnet, the gateway is xxxx:xxxx:ffff:fffe::109 so I route all the traffic to that IP, my border interface is supposed to be xxxx:xxxx:ffff:fffe::10a both are in the /126 address space…
It’s all static, no PPPoE or DHCPv6 in the WAN side.. As I’ve mentioned, this config works when the IPv6 border interface is not through VLAN. I find that very odd…

My ISP routes ( xxxx:xxxx:x::/48 → xxxx:xxxx:ffff:fffe::10a) and I route (2000::/3 gateway → xxxx:xxxx:ffff:fffe::109)
Hope that clears stuff up!

CaioPet · January 29, 2021, 3:05pm

Znevna I’m sorry about that, I’ve mistyped the config:
the address space is supposed to be: 2001:DB8:ffff:fffe::108/126 (since I’m masking the real IP here I’ve edited manually, but I’ve checked the address space before)

Sob · January 29, 2021, 3:15pm

Some of it is typo. It’s either ..::10a and ..::109, or ..::a and ..::9. So you want (if it’s the first one):

/ipv6 address
add address=xxxx:xxxx:ffff:fffe::10a/126 advertise=no interface=IPv6
/ipv6 route
add dst-address=2000::/3 gateway=xxxx:xxxx:ffff:fffe::109

CaioPet · January 29, 2021, 3:36pm

Some of it is typo. It’s either ..::10a and ..::109, or ..::a and ..::9. So you want (if it’s the first one):
/ipv6 address
add address=xxxx:xxxx:ffff:fffe::10a/126 advertise=no interface=IPv6
/ipv6 route
add dst-address=2000::/3 gateway=xxxx:xxxx:ffff:fffe::109

Okay so I double-checked the config and I still get the same result:

Znevna · January 29, 2021, 3:53pm

Did you change your VLAN config again? now a bridge appeared! Why?
Add those VLANs to Ethernet1 directly.

CaioPet · January 29, 2021, 4:26pm

I didn’t change my config, here it is:

/interface bridge
add fast-forward=no name="WAN Bridge"
/interface ethernet
set [ find default-name=ether1 ] name="eth1 - WAN"
/interface vlan
add interface="WAN Bridge" name=IPv4 vlan-id=3789
add interface="WAN Bridge" name=IPv6 vlan-id=33
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge="WAN Bridge" interface="eth1 - WAN"
/interface list member
add disabled=yes interface="WAN Bridge" list=LAN
add disabled=yes interface="eth1 - WAN" list=WAN

I’ve tried to remove the WAN bridge and set the VLANs to eth1 directly… unfortunately, the same result…
I’m wondering, MikroTik uses the same mac address for both VLANs, my ISP is running a Cisco-based topology, isn’t there a chance that Cisco can’t recognize the same MAC in different VLANs? (I might be tripping here, I have no idea what they do in there).

But seams odd to me that my router is sending ff02 packets and my neighbour is simply not responding, MK just marks the xxxx:xxxx:ffff:fffe::109 as failed in ND.

Sob · January 29, 2021, 4:50pm

You can also get rid of pool. It shouldn’t be breaking anything, when you get the right address from it. But it doesn’t add anything useful.

Next step I’d take is playing with packet sniffer. Catch what happens on parent interface (not VLAN, so you’d be able to see even things like wrong VLAN number), feed it to Wireshark or something, and hopefully you’ll see something useful. Use e.g. some online traceroute to get incoming packets to your xxxx:xxxx:ffff:fffe::10a.