LeMoi
September 22, 2024, 2:03pm
1
Hi everyone,
Got a situation where I can ping / communicate while inside a network, but there’s a bug in my routing that denies any communication inter-LAN/WAN.
Firewall:
has top AllowAll rules for all interfaces and directions for debugging
ICMPv6, v6-Delegation and ICMP are explicitly allowed
IPv4:
has no internet access (running IPv6 only at the moment)
two networks (bridge1 LAN and eth5 MGMT)
IPv6
two GUA networks (bridge1 LAN and eth5 MGMT)
two ULA networks (bridge1 LAN and eth5 MGMT)
WAN
IPv6 route looks like this
[Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, d - DHCP, v - VPN; + - ECMP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
DST-ADDRESS GATEWAY DISTANCE
DAd+ ::/0 fe80::bac2:53ff:fe34:73%pppoe-out1_DGN 1
DAv+ ::/0 pppoe-out1_DGN 1
DAd 2a01:41e3:5001:ae00::/56 1
DAc 2a01:41e3:5001:ae00::/64 bridge1 0
DAc 2a01:41e3:5001:ae01::/64 ether5_mgmt 0
DAc fd21:6584:3b21:1216::/64 ether5_mgmt 0
DAc fd21:6584:3b21:1314::/64 bridge1 0
DAc fe80::%ether1_wan/64 ether1_wan 0
DAc fe80::%ether5_mgmt/64 ether5_mgmt 0
DAc fe80::%bridge1/64 bridge1 0
DAc fe80::%pppoe-out1_DGN/64 pppoe-out1_DGN 0
DAc fe80::%007_DGN/64 007_DGN 0
But the system behaves as if no routes were defined…
LeMoi
September 22, 2024, 3:06pm
2
Additional Info:
/ipv6/settings> print
disable-ipv6: no
forward: yes
accept-redirects: yes-if-forwarding-disabled
accept-router-advertisements: yes-if-forwarding-disabled
max-neighbor-entries: 4096
LeMoi
September 22, 2024, 3:07pm
3
Interfaces
Flags: D - dynamic; X - disabled, R - running; S - slave; P - passthrough
0 R name="ether1_wan" default-name="ether1" type="ether" mtu=1500 actual-mtu=1500
l2mtu=1596 max-l2mtu=2026 mac-address=DC:2C:6E:57:EB:2D ifname="eth0" ifindex=7 id=1
last-link-up-time=2024-09-22 11:19:10 link-downs=0
1 RS name="ether2" default-name="ether2" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1596
max-l2mtu=2026 mac-address=DC:2C:6E:57:EB:2E ifname="eth1" ifindex=8 id=2
last-link-up-time=2024-09-22 11:19:10 link-downs=0
2 S name="ether3" default-name="ether3" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1596
max-l2mtu=2026 mac-address=DC:2C:6E:57:EB:2F ifname="eth2" ifindex=9 id=3
link-downs=0
3 S name="ether4" default-name="ether4" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1596
max-l2mtu=2026 mac-address=DC:2C:6E:57:EB:30 ifname="eth3" ifindex=10 id=4
link-downs=0
4 R name="ether5_mgmt" default-name="ether5" type="ether" mtu=1500 actual-mtu=1500
l2mtu=1596 max-l2mtu=2026 mac-address=DC:2C:6E:57:EB:31 ifname="eth4" ifindex=11 id=5
last-link-up-time=2024-09-22 11:19:10 link-downs=0
5 R name="007_DGN" type="vlan" mtu=1500 actual-mtu=1500 l2mtu=1592
mac-address=DC:2C:6E:57:EB:2D ifname="vlan11" ifindex=13 id=11
last-link-up-time=2024-09-22 11:19:10 link-downs=0
6 R name="bridge1" type="bridge" mtu=auto actual-mtu=1500 l2mtu=1596
mac-address=DC:2C:6E:57:EB:2E ifname="br0" ifindex=12 id=6
last-link-up-time=2024-09-22 11:18:59 link-downs=0
7 R name="pppoe-out1_DGN" type="pppoe-out" mtu=1492 actual-mtu=1492 ifname="ppp0"
ifindex=14 id=7 last-link-down-time=2024-09-22 11:19:12
last-link-up-time=2024-09-22 11:19:12 link-downs=1
LeMoi
September 22, 2024, 3:08pm
4
DHCPv6 Client
Flags: D - dynamic; X - disabled, I - invalid
0 interface=pppoe-out1_DGN status=bound duid="0x00030001dc2c6e57eb2d"
dhcp-server-v6=fe80::bac2:53ff:fe34:73 request=prefix add-default-route=yes
default-route-distance=1 use-peer-dns=yes dhcp-options=OPTION_ORO
pool-name="pool-pppoe-pd" pool-prefix-length=64 prefix-hint=::/0
dhcp-options=OPTION_ORO prefix=2a01:41e3:5001:ae00::/56, 22h19m55s
LeMoi
September 22, 2024, 3:08pm
5
IPv6 Pools
/ipv6/pool> print detail
Flags: D - dynamic
0 D id=5 name="pool-pppoe-pd" prefix=2a01:41e3:5001:ae00::/56 prefix-length=64
expires-after=22h20m53s
LeMoi
September 22, 2024, 3:10pm
6
ND settings
/ipv6/nd> print
Flags: X - disabled, I - invalid; * - default
0 * interface=bridge1 ra-interval=3m20s-10m ra-delay=3s mtu=unspecified
reachable-time=unspecified retransmit-interval=unspecified ra-lifetime=30m
ra-preference=medium hop-limit=unspecified advertise-mac-address=yes advertise-dns=yes
managed-address-configuration=no other-configuration=no dns="" pref64=""
1 interface=ether5_mgmt ra-interval=3m20s-10m ra-delay=3s mtu=unspecified
reachable-time=unspecified retransmit-interval=unspecified ra-lifetime=30m
ra-preference=medium hop-limit=unspecified advertise-mac-address=yes advertise-dns=yes
managed-address-configuration=no other-configuration=no dns="" pref64=""
and Prefixes
Flags: X - disabled, I - invalid; D - dynamic
0 D prefix=fd21:6584:3b21:1314::/64 6to4-interface=none interface=bridge1 on-link=yes
autonomous=yes valid-lifetime=4w2d preferred-lifetime=1w
1 D prefix=fd21:6584:3b21:1216::/64 6to4-interface=none interface=ether5_mgmt on-link=yes
autonomous=yes valid-lifetime=4w2d preferred-lifetime=1w
2 D prefix=2a01:41e3:2561:b00::/64 6to4-interface=none interface=bridge1 on-link=yes
autonomous=yes valid-lifetime=4w2d preferred-lifetime=1w
3 D prefix=2a01:41e3:2561:b01::/64 6to4-interface=none interface=ether5_mgmt on-link=yes
autonomous=yes valid-lifetime=4w2d preferred-lifetime=1w
LeMoi
September 22, 2024, 3:11pm
7
and Addresses
Flags: X - disabled, I - invalid, D - dynamic; G - global, L - link-local
0 G address=2a01:41e3:2561:b00:de2c:6eff:fe57:eb2e/64 from-pool=pool-pppoe-pd
interface=bridge1 actual-interface=bridge1 eui-64=yes advertise=yes no-dad=no
1 G address=2a01:41e3:2561:b01:de2c:6eff:fe57:eb31/64 from-pool=pool-pppoe-pd
interface=ether5_mgmt actual-interface=ether5_mgmt eui-64=yes advertise=yes no-dad=no
2 G address=fd21:6584:3b21:1216:de2c:6eff:fe57:eb31/64 from-pool="" interface=ether5_mgmt
actual-interface=ether5_mgmt eui-64=yes advertise=yes no-dad=no
3 G address=fd21:6584:3b21:1314:de2c:6eff:fe57:eb2e/64 from-pool="" interface=bridge1
actual-interface=bridge1 eui-64=yes advertise=yes no-dad=no
4 DL address=fe80::de2c:6eff:fe57:eb2e/64 from-pool="" interface=bridge1
actual-interface=bridge1 eui-64=no advertise=no no-dad=no
5 DL address=fe80::de2c:6eff:fe57:eb31/64 from-pool="" interface=ether5_mgmt
actual-interface=ether5_mgmt eui-64=no advertise=no no-dad=no
6 DL address=fe80::de2c:6eff:fe57:eb2d/64 from-pool="" interface=ether1_wan
actual-interface=ether1_wan eui-64=no advertise=no no-dad=no
7 DL address=fe80::de2c:6eff:fe57:eb2d/64 from-pool="" interface=007_DGN
actual-interface=007_DGN eui-64=no advertise=no no-dad=no
8 DL address=fe80::60f4:f35a:0:7/64 from-pool="" interface=pppoe-out1_DGN
actual-interface=pppoe-out1_DGN eui-64=no advertise=no no-dad=no
LeMoi
September 22, 2024, 3:30pm
8
ah, and DNS seems to work, as a try to reach a website via FQDN leads to “UDP connect: no route to host” and not "can’t be resolved.
another thing that appears odd:
tdw
September 22, 2024, 4:42pm
9
Providing the print output from a random set of sections isn’t particularly helpful, post the output of /export in a code block (the
The usual errors with setting up IPv6 is not setting add-default-route=no to the DHCPv6 client, and not setting accept-router-advertisements=yes (although with a PPPoE WAN connection this is likely unnecessary).
Unfortunately there are many different ways ISPs provide IPv6, does yours provide any information as to how their implementation operates?
LeMoi
September 22, 2024, 4:55pm
10
Thanks tdw, info as requested follows.
not setting add-default-route=no to the DHCPv6 client
→ had this set, has been corrected.
not setting accept-router-advertisements=yes
→ set to “yes-if-forwarding-disabled” and “forward=yes”, so could be an issue.
LeMoi
September 22, 2024, 4:58pm
11
/Export
# 2024-09-22 18:54:29 by RouterOS 7.13.5
# software id = YS0Q-E2S9
#
# model = RB750Gr3
# serial number = CC##########
/interface bridge
add admin-mac=DC:2C:6E:57:EB:2E auto-mac=no igmp-snooping=yes name=bridge1
/interface ethernet
set [ find default-name=ether1 ] name=ether1_wan
set [ find default-name=ether5 ] name=ether5_mgmt
/interface vlan
add interface=ether1_wan name=007_DGN vlan-id=7
/interface pppoe-client
add add-default-route=yes allow=pap,chap,mschap2 disabled=no interface=007_DGN max-mru=1492 \
max-mtu=1492 name=pppoe-out1_DGN use-peer-dns=yes user=########@dgn.digital
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip dhcp-client option
add code=55 name=req_6rd value=0x010306d4
add code=6 name=req_dns_aftr value=0x00170040
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=mgmt_dhcp_pool1 ranges=192.168.88.101-192.168.88.200
add name=bridge1_pool1 ranges=172.21.201.101-172.21.201.200
/ipv6 dhcp-client option
add code=64 name=OPTION_ORO value=0x0040
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge1 interface=ether2 pvid=201
add bridge=bridge1 interface=ether3 pvid=201
add bridge=bridge1 interface=ether4 pvid=201
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add interface=bridge1 list=LAN
add interface=ether5_mgmt list=LAN
add interface=pppoe-out1_DGN list=WAN
add interface=ether4 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
/ip address
add address=192.168.88.1/24 interface=ether5_mgmt network=192.168.88.0
add address=172.21.201.1/24 interface=bridge1 network=172.21.201.0
/ip dhcp-server
add address-pool=mgmt_dhcp_pool1 interface=ether5_mgmt lease-time=10m name=mgmt_dhcpv4 \
parent-queue=*FFFFFFFF
add address-pool=bridge1_pool1 interface=bridge1 name=bridge1_dhcpv4 parent-queue=*FFFFFFFF
/ip dhcp-server network
add address=172.21.201.0/24 dns-server=172.21.201.1 gateway=172.21.201.1 netmask=24
add address=192.168.88.0/24 dns-server=192.168.88.1 gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes servers="8.8.8.8,1.1.1.1,9.9.9.9,2001:4860:4860::8888,2001:4860:\
4860::4444,2606:4700:4700::1111,2606:4700:4700::1001,2620:0:ccc::2,2620:0:ccd::2"
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" \
connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="accept ICMP from WAN" in-interface=pppoe-out1_DGN \
protocol=icmp
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" \
dst-address=127.0.0.1
add action=accept chain=input comment="allow WinBox from WAN" in-interface=pppoe-out1_DGN \
port=8291 protocol=tcp
add action=accept chain=input comment="allow SSH from WAN" in-interface=pppoe-out1_DGN port=\
22 protocol=tcp
add action=drop chain=input comment="drop anything else from WAN" in-interface=pppoe-out1_DGN
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=\
in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=\
out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=\
established,related hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" \
connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" \
connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none \
out-interface-list=WAN
/ip service
set telnet disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ipv6 address
add address=::xxxx:xxxx:xxxx:xxzz eui-64=yes from-pool=pool-pppoe-pd interface=bridge1
add address=::xxxx:xxxx:xxxx:yyzz eui-64=yes from-pool=pool-pppoe-pd interface=ether5_mgmt
add address=fdxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx eui-64=yes interface=ether5_mgmt
add address=fdxx:xxxx:xxxx:xxyy:xxxx:xxxx:xxxx:xxxx eui-64=yes interface=bridge1
/ipv6 dhcp-client
add dhcp-options=OPTION_ORO dhcp-options=OPTION_ORO interface=pppoe-out1_DGN pool-name=\
pool-pppoe-pd request=prefix
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment="DEBUG ONLY - IN AllowAll" in-interface-list=all
add action=accept chain=forward comment="DEBUG ONLY - FWD AllowAll" in-interface-list=all \
out-interface-list=all
add action=accept chain=output comment="DEBUG ONLY - OUT AllowAll" out-interface-list=all
add action=accept chain=input comment="defconf: accept established,related,untracked" \
connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" dst-port=33434-33534 \
protocol=udp
add action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." \
dst-port=546 protocol=udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept all that matches ipsec policy" \
ipsec-policy=in,ipsec
add action=drop chain=input comment="defconf: drop everything else not coming from LAN" \
in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept established,related,untracked" \
connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" \
src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" \
dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 \
protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" \
ipsec-policy=in,ipsec
add action=accept chain=forward comment="accept UDP traceroute" dst-port=33434-33534 port="" \
protocol=udp src-port=""
add action=drop chain=forward comment="defconf: drop everything else not coming from LAN" \
in-interface-list=!LAN
/ipv6 firewall raw
add action=accept chain=prerouting comment="DEBUG ONLY - AcceptAll"
/ipv6 nd
set [ find default=yes ] interface=bridge1
add interface=ether5_mgmt
/routing rule
add action=lookup disabled=no interface=bridge1 table=main
add action=lookup disabled=no interface=ether5_mgmt table=main
/system clock
set time-zone-name=Europe/Berlin
/system note
set show-at-login=no
/system ntp client
set enabled=yes
/system ntp server
set enabled=yes
/system ntp client servers
add address=ptbtime1.ptb.de
add address=ptbtime2.ptb.de
add address=ptbtime3.ptb.de
add address=ptbtime4.ptb.de
/tool bandwidth-server
set enabled=no
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
LeMoi
September 22, 2024, 5:07pm
12
…many different ways ISPs provide IPv6, does yours provide any information…
The ISP is not very open in his communication, they try force you to use Fritz!Boxes provided by them.
So, my first aim was to get the whole IPv6 side up and running, and later trying to figure out the scripts for the Option… or booking a DualStack.
LeMoi
September 23, 2024, 10:23am
13
Issue solved. IPv6 routing now works after setting add-default-route=no on the DHCPv6 client that’s attached to the WAN-side PPPoE-interface.
Thanks again to tdw
And side note that this “how to” leads to the bug, just in case someone else uses it as reference:https://administrator.de/tutorial/ipv6-mittels-prefix-delegation-bei-pppoe-mikrotik-632633.html add-default-route=yes on both PPPoE and DHCPv6 client.
tdw
September 23, 2024, 10:44am
14
I was just about to post suggesting setting add-default-route=no to the DHCPv6 client so the PPPoE default route is used.
What are the /routing rule entries for?
I’ve made posts previously regarding DS-Lite setup. These used AFTR provided by DNS, they would need some modification to use the DHCPv6 option.
LeMoi
September 23, 2024, 11:23am
15
The routing rules are redundant, I think. They were added while searching for the cause of the missing routing info.
Had a chat with my ISP 2nd level support today. He’s forwarded my questions regarding the AFTR and how it’s provided to the 3rd level. Waiting for reply…
LeMoi
September 23, 2024, 3:54pm
16
tdw
September 23, 2024, 4:59pm
17
LeMoi
September 23, 2024, 8:47pm
18
Mhm. I’ll follow that up… seems like I’ll need to learn scripting a bit
