Hi,
I have 2 global IPV6 prefixes in my network and my goal is to use netwatch to update the distance of the two gateway and also “gently” disable the prefix thats down. I wanted to use the command /ipv6 advertise=no but that is not recognized.
What is a good way to achieve that? note: i do not want to use NPT or NAT for IPv6, i want to use the full delegated network as it is actually intended by the IPv6 standard, so that the clients will select the best route/prefix themselves over SLAAC.
the ressources on this topic are very limited as it seems, everybody wants to mess around with NAT for IPv6, which is just wrong.
IPv6-RA seems to only work when forcing the removal of a prefix, below script is working in this regard, even though the clients are still clinging on to the other prefix but seem to deprecate them quickly enough.
As Im the first one attempting this (I cannot find any other postings in the forum with a successful working failover config) I will post my current setup, which works flawlessly as of now. Note: Im using 8.8.8.8, 1.1.1.1, 4.2.2.1 as monitoring IP, but not for actual DNS resolving, Im using quad9 and 1.1.1.2 for that (maleware block). Important: for check-gateway and netwatch to work, those IPs must be unreachable during WAN-failover. Blackhole routes are the only means to do that. There are scripts in the forum using recursive routing and route-marking, thats incorrect and does not work. Blackholing is actually meant to be used for this.
Some more sidenotes:
- only one prefix can be active, if two prefixes are active, clients will choose them randomly for sending packets. As this is a failover configuration only one WAN is routed at a time. If packets leaving WAN1 are using WAN2 prefix, the ISP will just drop them. Therefore the netwatch script must make sure that the other prefix will be removed during failover
- eui-64 must be disabled for the gateway address, as this will otherwise dynamically generate more IPv6 addresses on the interface from the currently inactive prefix
- Windows and Linux do not support DNSv6 in any form, it is impossible to assign the ULA of the router over RDNSS or DHCPv6, therefore DNS queries are intercepted and rewritten to use the internal resolver, which is also a security benefit
- as this is a failover configuration, there is no script for down of WAN2, if WAN2 is not working during a WAN1 failover… then were just offline.
- ra-lifetime and valid lifetime for ipv6 nd must be set to a very low value, otherwise the prefixes will be stuck forever and can cause hickups during failover as the clients use SLAAC and decide for themselves when they switch
- Im using 3 different DNS servers for netwatch and a single script will evaluate if all of them are down. Note that global variables do NOT work with netwatch, therefore we are using files to save the status of the 6 hosts in total and evaluate that in a single script that is running with the scheduler (otherwise there would be race conditions if the netwatch scripts would each call this single script)
- the script will switch when either IPv6 or IPv4 is down or both
- the scripts are all written in one line each. This is due to the fact that ROS cant handle multiline scripts well. But there are easy ways to beautify this, ChatGPT for example.
here is the > ROS 7.18 config:
# Set interface names
/interface set [find default-name=ether2] name=WAN1
/interface set [find default-name=ether3] name=WAN2
[.... VLAN and DHCP configuration ... ]
# DNS setup
/ip dns set servers=9.9.9.9,1.1.1.2,2620:fe::9,2606:4700:4700::1112 allow-remote-requests=yes
# WAN DHCP clients (IPv4 and IPv6)
/ip dhcp-client add interface=WAN1 add-default-route=no disabled=no use-peer-dns=no
/ip dhcp-client add interface=WAN2 add-default-route=no disabled=no use-peer-dns=no
/ipv6 dhcp-client add interface=WAN1 add-default-route=no pool-name=WAN1-pool prefix-hint=::/62 request=address,prefix disabled=no use-peer-dns=no
/ipv6 dhcp-client add interface=WAN2 add-default-route=no pool-name=WAN2-pool prefix-hint=::/62 request=address,prefix disabled=no use-peer-dns=no
# IPv6 Address assignment for VLAN50
/ipv6 address add address=fd37:443d:252b:0::1/64 interface=main-network advertise=no comment="ULA generated from ether5 address of this hEX router"
/ipv6 address add from-pool=WAN1-pool interface=main-network address=::1 advertise=yes eui-64=no comment="Prefix from WAN1"
# ipv4 default routes and special routes for online-check
/ip route add dst-address=0.0.0.0/0 distance=1 gateway=192.168.1.1 check-gateway=ping comment="Primary WAN"
/ip route add dst-address=0.0.0.0/0 distance=2 gateway=192.168.2.1 check-gateway=ping comment="Secondary WAN"
/ip route add dst-address=1.1.1.1/32 gateway=192.168.1.1 distance=1 comment="OnlineCheck - Cloudflare DNS"
/ip route add dst-address=8.8.8.8/32 gateway=192.168.1.1 distance=1 comment="OnlineCheck - Google DNS"
/ip route add dst-address=4.2.2.1/32 gateway=192.168.1.1 distance=1 comment="OnlineCheck - Level 3 DNS"
/ip route add dst-address=1.1.1.1/32 blackhole distance=10 comment="Blackhole Cloudflare DNS"
/ip route add dst-address=8.8.8.8/32 blackhole distance=10 comment="Blackhole Google DNS"
/ip route add dst-address=4.2.2.1/32 blackhole distance=10 comment="Blackhole Level 3 DNS"
# ipv6 default routes and special routes for online-check
/ipv6 route add dst-address=::/0 distance=1 gateway=fe80::%WAN1 check-gateway=ping comment="Default IPv6 Gateway"
/ipv6 route add dst-address=2001:4860:4860::8888/128 gateway=fe80::%WAN1 distance=1 comment="Onlinecheck Google DNSv6"
/ipv6 route add dst-address=2606:4700:4700::1111/128 gateway=fe80::%WAN1 distance=1 comment="Onlinecheck Cloudflare DNSv6"
/ipv6 route add dst-address=2620:0:ccc::2/128 gateway=fe80::%WAN1 distance=1 comment="Onlinecheck Cisco DNSv6"
/ipv6 route add dst-address=2001:4860:4860::8888/128 blackhole distance=10 comment="Blackhole Google DNSv6"
/ipv6 route add dst-address=2606:4700:4700::1111/128 blackhole distance=10 comment="Blackhole Cloudflare DNSv6"
/ipv6 route add dst-address=2620:0:ccc::2/128 blackhole distance=10 comment="Blackhole Cisco DNSv6"
# Failover-Script
/system script add name="WAN-up" dont-require-permissions=yes source=":delay 75; /ip route set [find where dst-address=0.0.0.0/0 and gateway=192.168.1.1] distance=1; /ipv6 address remove [find where from-pool=\"WAN2-pool\"]; /ipv6 route set [find where dst-address=::/0] gateway=fe80::%WAN1; :if ([:len [/ipv6 address find where interface=\"main-network\" and from-pool=\"WAN1-pool\"]]=0) do={ /ipv6 address add from-pool=WAN1-pool interface=main-network address=::1 advertise=yes eui-64=no comment=\"Prefix from WAN1\";}; /ipv6 nd set [find interface=main-network] disabled=yes; /ipv6 nd set [find interface=main-network] disabled=no; /log warning \"Failover event: WAN1 is now UP\""
/system script add name="WAN-down" dont-require-permissions=yes source=":delay 2; /ip route set [find where dst-address=0.0.0.0/0 and gateway=192.168.1.1] distance=10; /ipv6 address remove [find where from-pool=\"WAN1-pool\"]; /ipv6 route set [find where dst-address=::/0] gateway=fe80::%WAN2; :if ([:len [/ipv6 address find where interface=\"main-network\" and from-pool=\"WAN2-pool\"]]=0) do={ /ipv6 address add from-pool=WAN2-pool interface=main-network address=::1 advertise=yes eui-64=no comment=\"Prefix from WAN2\";}; /ipv6 nd set [find interface=main-network] disabled=yes; /ipv6 nd set [find interface=main-network] disabled=no; /log error \"Failover event: WAN1 is now DOWN\""
/system script add name="failover-check" dont-require-permissions=yes source=":global currentState; :global lastSwitch; :local now [:tonum [/system resource get uptime]]; :if ([:typeof $currentState] = \"nil\") do={:set currentState \"down\";}; :if ([:typeof $lastSwitch] = \"nil\") do={:set lastSwitch \$now;}; :local v4ups 0; :local v6ups 0; :local hosts {\"v4_cf\"; \"v4_gg\"; \"v4_lv\"; \"v6_gg\"; \"v6_cf\"; \"v6_ci\"}; :local status \"\"; :if (\$now - \$lastSwitch < 100) do={:log warning \"Debounce active, waiting 100s\"; :quit}; :foreach h in=\$hosts do={:if ([/file get [/file find name=\$h] size] > 1) do={:set status (\$status . \$h . \" 1 \"); :if ([:find \$h \"v6\"] >= 0) do={:set v6ups (\$v6ups + 1)} else={:set v4ups (\$v4ups + 1)}} else={:set status (\$h . \" 0 \")}}; :log debug \"Failovercheck, host states: \$status | IPv4 up: \$v4ups | IPv6 up: \$v6ups\"; :local newState \"down\"; :if (\$v4ups > 0 && \$v6ups > 0) do={:set \$newState \"up\"}; :if (\$currentState != \$newState) do={:set currentState \$newState; :set lastSwitch \$now; :if (\$newState = \"up\") do={:log warning \"All interfaces online, resuming back to WAN1\"; /system script run WAN-up} else={:log error \"connection to WAN1 lost, failover to WAN2\"; /system script run WAN-down}}"
/file add name=v4_cf
/file add name=v4_gg
/file add name=v4_lv
/file add name=v6_cf
/file add name=v6_ci
/file add name=v6_gg
/tool netwatch add host=1.1.1.1 interval=4s timeout=2s down-script="/file set [/file find name=\"v4_cf\"] contents=\"\"" up-script="/file set [/file find name=\"v4_cf\"] contents=\"up\""
/tool netwatch add host=8.8.8.8 interval=4s timeout=2s down-script="/file set [/file find name=\"v4_gg\"] contents=\"\"" up-script="/file set [/file find name=\"v4_gg\"] contents=\"up\""
/tool netwatch add host=4.2.2.1 interval=4s timeout=2s down-script="/file set [/file find name=\"v4_lv\"] contents=\"\"" up-script="/file set [/file find name=\"v4_lv\"] contents=\"up\""
/tool netwatch add host=2001:4860:4860::8888 interval=4s timeout=2s down-script="/file set [/file find name=\"v6_gg\"] contents=\"\"" up-script="/file set [/file find name=\"v6_gg\"] contents=\"up\""
/tool netwatch add host=2606:4700:4700::1111 interval=4s timeout=2s down-script="/file set [/file find name=\"v6_cf\"] contents=\"\"" up-script="/file set [/file find name=\"v6_cf\"] contents=\"up\""
/tool netwatch add host=2620:0:ccc::2 interval=4s timeout=2s down-script="/file set [/file find name=\"v6_ci\"] contents=\"\"" up-script="/file set [/file find name=\"v6_ci\"] contents=\"up\""
/system scheduler add name="failover-check-scheduler" interval=5s on-event="/system script run failover-check"
# IPv6 Router Advertisement configuration
/ipv6 settings set forward=yes
/ipv6 nd prefix default set preferred-lifetime=1m valid-lifetime=5m
/ipv6 nd add interface=main-network ra-lifetime=5m ra-delay=60s ra-interval=60s-100s other-configuration=no advertise-dns=yes dns=fd37:443d:252b:0::1 ra-preference=high
/ipv6 nd add interface=IoT-network disabled=yes
/ipv6 nd add interface=all disabled=yes
# NAT for IPv4, the first rule shall catch all bogos DNS queries and redirect to the local router
/ip firewall nat add chain=dstnat in-interface=main-network protocol=udp dst-port=53 action=redirect to-ports=53 comment="Force IPv4 DNS to router"
/ip firewall nat add chain=dstnat in-interface=main-network protocol=tcp dst-port=53 action=redirect to-ports=53 comment="Force IPv4 DNS to router"
/ip firewall nat add chain=srcnat action=masquerade out-interface=WAN1
/ip firewall nat add chain=srcnat action=masquerade out-interface=WAN2
# Firewall rules for ICMP and inter-VLAN communication and FastTrack
/ip firewall filter add chain=forward action=fasttrack-connection connection-state=established,related hw-offload=yes
/ip firewall filter add chain=forward action=accept connection-state=established,related
[other ipv4 rules...]
/ip firewall filter add chain=forward action=drop connection-state=invalid
#Firewall rules to catch all bogos DNSv6 queries that want to leave our network, this router is the only DNS server but as of now Windows and Linux have no working DNSv6 implementation therefore we need to force them
/ipv6 firewall nat add chain=dstnat in-interface=main-network protocol=udp dst-port=53 action=dst-nat to-address=fd37:443d:252b:0::1 comment="Force IPv6 DNS to router"
/ipv6 firewall nat add chain=dstnat in-interface=main-network protocol=tcp dst-port=53 action=dst-nat to-address=fd37:443d:252b:0::1 comment="Force IPv6 DNS to router"
#Firewall for ipv6 fasttrack
/ipv6 firewall filter add chain=forward action=fasttrack-connection connection-state=established,related comment="fasttrack6"
/ipv6 firewall filter add chain=forward action=accept connection-state=established,related
I hope my script is helping some. With the upcoming ROS 7.18 there should be a drastic improvement in terms of performance (thanks to Fasttrack). I will update the config if needed. If you encounter any issue, feel free to ask here.
I have updated the script as promised and there are some more tweaks, this setup runs very stable for some weeks now in my scenario.
My hEX refresh manages up to 800Mbit of IPv6 traffic and the fail-overs are generally not noticable.