Hi,
I have suffered from an unexpected latency when failing over to a second WAN based on a simple recursive routing setup. Both WANs are NATed. My application would take 1-3 minutes to start using the new route while if restarted it would pick up the new connection immediately
The issue was that within the iOS app it looked like 3-4 ports were connection-pooled for smaller requests. So the packets would continue flowing using the old public ip address but via the new route. Ouch. BTW, there seems to be no firewall rule to catch such packets, as the source address even in the postrouting table is still the local lan address, even though the packet is SNATed already.
So, I have taken a night and came up with this connection mark setup that works quite well, resetting the connections immediately upon seeing new packets after the failover switch.
I am also utilizing the new 6.39 feature - dhcp client script. However the same rules can be used to configure the setup manually or from other trigger.
Comments are appreciated.
To use the script simply set the $distance parameter as needed per interface.
The script makes use of connection marks same as interface names, plus one connection mark “reset”
:local distance 1
:local checkIp ("4.2.2." . $distance)
:local tag ("WAN " . $interface)
:local tagAny ("Any WAN")
/ip route remove [find comment~("^" . $tag)]
/ip firewall nat remove [find comment~("^" . $tag)]
/ip firewall mangle remove [find comment~("^" . $tag)]
:if ($bound=1) do={
log info ("DHCP Client (" . $interface .")" . " received new lease address " . $"lease-address" . " from server " . $"server-address")
:local gatewayAddress [/ip dhcp-client get [find dhcp-server=$"server-address"] gateway]
log info ("DHCP Client (" . $interface .")" . " received new gateway address " . $gatewayAddress)
/ip firewall nat add chain=srcnat out-interface=$interface action=masquerade comment=($tag . ": masquerade outgoing packets")
/ip route add dst-address=$checkIp gateway=$gatewayAddress scope=5 comment=($tag . ": marker route")
/ip route add gateway=$checkIp distance=$distance check-gateway=ping comment=($tag . ": recursive default route via marker " . $checkIp)
# below is a set of rules to reset any tracked connection which starts flowing
# via the new route but still uses the old SNAT address
:local notInterface ("!" . $interface)
/ip firewall mangle add chain=postrouting out-interface=$interface connection-mark=$notInterface action=mark-connection new-connection-mark=reset comment=($tag . ": reset alien connections")
/ip firewall mangle add chain=postrouting out-interface=$interface connection-state=new action=mark-connection new-connection-mark=$interface comment=($tag . ": mark forwarded")
/ip firewall mangle add chain=output out-interface=$interface connection-state=new action=mark-connection new-connection-mark=$interface comment=($tag . ": mark output")
/ip firewall mangle add chain=input in-interface=$interface connection-state=new action=mark-connection new-connection-mark=$interface comment=($tag . ": mark input")
# this remove is here since it is a common rule, and we don't want to delete it
# when one of the interfaces goes down - it may still be needed by another interface which is still up
# deleting here means that the rules will be recreated immediately
/ip firewall filter remove [find comment~("^" . $tagAny)]
/ip firewall filter add action=reject place-before=0 chain=forward connection-mark=reset protocol=tcp reject-with=tcp-reset comment=($tagAny . ": RST TCP connections marked as 'reset'")
/ip firewall filter add action=reject place-before=0 chain=forward connection-mark=reset reject-with=icmp-admin-prohibited comment=($tagAny . ": ICMP unreachable other connections marked as 'reset'")
}