Advanced Routing Failover without Scripting

here is my exact failover script that i use and so on as i tested it works well… any comments? i use distance 1 in both wan cause i mark ports in magle and i do routing mark. LB in torrents and failover seems that working fine.

/ip route
add check-gateway=ping distance=1 gateway=1.0.0.2 routing-mark=to_WAN2
add check-gateway=ping distance=1 gateway=1.1.1.2 routing-mark=to_WAN1
add check-gateway=ping distance=1 gateway=1.1.1.2
add check-gateway=ping distance=1 gateway=1.0.0.2
add check-gateway=ping distance=1 dst-address=1.0.0.2/32 gateway=192.168.2.1 scope=10
add distance=5 dst-address=1.0.0.2/32 type=blackhole
add check-gateway=ping distance=1 dst-address=1.1.1.2/32 gateway=192.168.0.1 scope=10
add distance=5 dst-address=1.1.1.2/32 type=blackhole

Looks like there’s no mangle rules in your script…

I just posted my routing rules, here r my magle rules…


/ip firewall mangle
add action=mark-routing chain=prerouting comment=______Guest_to_WAN2 \
    new-routing-mark=to_WAN2 passthrough=no src-address=10.10.20.0/24
add action=mark-routing chain=prerouting comment=_______Remotes&Games_to_WAN1 \
    new-routing-mark=to_WAN1 passthrough=no port=1320,17771,5000-5500,7985 \
    protocol=udp src-address=10.157.138.0/24
add action=mark-routing chain=prerouting dst-port="" new-routing-mark=to_WAN1 \
    passthrough=no port=1320,12975,32976,4899,5938,48377 protocol=tcp \
    src-address=10.157.138.0/24
add action=mark-routing chain=prerouting comment=_______ZLO_to_WAN1 \
    dst-address-list=Zlo_Games new-routing-mark=to_WAN1 passthrough=no \
    src-address=10.157.138.100/31
add action=mark-routing chain=prerouting comment=\
    "_______Torrents_to_WAN1 or WAN2" disabled=yes new-routing-mark=to_WAN2 \
    passthrough=no port=8999-65535 protocol=tcp src-address=10.157.138.100/31
add action=mark-routing chain=prerouting disabled=yes new-routing-mark=\
    to_WAN2 passthrough=no port=8999-65535 protocol=udp src-address=\
    10.157.138.100/31
add action=mark-connection chain=input comment=\
    _______Load_Balance_Mark_IN-OUT in-interface=WAN1 new-connection-mark=\
    WAN1_conn passthrough=no
add action=mark-connection chain=input in-interface=WAN2 new-connection-mark=\
    WAN2_conn passthrough=no
add action=mark-routing chain=output connection-mark=WAN1_conn \
    new-routing-mark=to_WAN1 passthrough=no
add action=mark-routing chain=output connection-mark=WAN2_conn \
    new-routing-mark=to_WAN2 passthrough=no
add action=accept chain=prerouting comment=\
    _______Load_Balance_Accept_All_WANS dst-address=192.168.0.0/24 \
    in-interface=Bridge
add action=accept chain=prerouting dst-address=192.168.2.0/24 in-interface=\
    Bridge
add action=mark-routing chain=prerouting comment="_______HTTP-S_Routing mark" \
    new-routing-mark=to_WAN1 passthrough=no port=80,443 protocol=tcp \
    src-address=10.157.138.0/24
add action=mark-routing chain=prerouting new-routing-mark=to_WAN1 \
    passthrough=no port=80,443 protocol=udp src-address=10.157.138.0/24
add action=mark-connection chain=prerouting comment=\
    "_______Load_Balance_Divider&Routing mark" dst-address-type=!local \
    in-interface=Bridge new-connection-mark=WAN1_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:2/0
add action=mark-connection chain=prerouting dst-address-type=!local \
    in-interface=Bridge new-connection-mark=WAN2_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:2/1
add action=mark-routing chain=prerouting connection-mark=WAN1_conn \
    in-interface=Bridge new-routing-mark=to_WAN1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=WAN2_conn \
    in-interface=Bridge new-routing-mark=to_WAN2 passthrough=no
add action=change-dscp chain=prerouting comment=_______DSCP_63_ICMP new-dscp=\
    63 passthrough=no protocol=icmp
add action=change-dscp chain=postrouting comment=_______DSCP_63_ICMP \
    new-dscp=63 passthrough=no protocol=icmp
add action=change-dscp chain=prerouting comment=\
    _______DSCP_63_DNS-REMOTES-GAMES new-dscp=63 passthrough=no port=\
    53,1320,17771,5000-5500,48377 protocol=udp
add action=change-dscp chain=postrouting comment=\
    _______DSCP_63_DNS-REMOTES-GAMES new-dscp=63 passthrough=no port=\
    53,1320,17771,5000-5500,48377 protocol=udp
add action=change-dscp chain=prerouting comment=\
    _______DSCP_56_HTTP-S_SMALL-REMOTES connection-bytes=0-500000 new-dscp=56 \
    passthrough=no port=80,443,8080,1320,12975,32976,4899,5938,7062 protocol=\
    tcp
add action=change-dscp chain=postrouting comment=\
    _______DSCP_56_HTTP-S_SMALL-REMOTES connection-bytes=0-500000 new-dscp=56 \
    passthrough=no port=80,443,8080,1320,12975,32976,4899,5938,7062 protocol=\
    tcp
add action=change-dscp chain=prerouting comment=_______DSCP_24_HTTP_S_LARGE \
    new-dscp=24 passthrough=no port=80,443,8080 protocol=tcp
add action=change-dscp chain=postrouting comment=_______DSCP_24_HTTP_S_LARGE \
    new-dscp=24 passthrough=no port=80,443,8080 protocol=tcp
add action=change-dscp chain=prerouting comment=_______DSCP_0_Torrents \
    new-dscp=0 passthrough=no port=8999-65355 protocol=tcp
add action=change-dscp chain=postrouting comment=_______DSCP_0_Torrents \
    new-dscp=0 passthrough=no port=8999-65355 protocol=tcp
add action=change-dscp chain=prerouting comment=_______DSCP_0_Torrents \
    new-dscp=0 passthrough=no port=8999-65355 protocol=udp
add action=change-dscp chain=postrouting comment=_______DSCP_0_Torrents \
    new-dscp=0 passthrough=no port=8999-65355 protocol=udp

Hi Note,
If you have a rule that marks a connection, and then a rule to mark a route then you must have passthrough = YES on the mark connection. That way, the processing can drop onto the route mark rule.

Hi Dave and thnx for ur participation,

I do not know what exactly u mean by that, but to have that working well i had to put the ports that i mark in the beginning and the LB rules after and only the dividers with passthrough yes. Otherwise i had issues.

Just a note: you don’t need to mark connections in your setup, as you mark connection for every packet from LAN, and then mark routing for every packet from LAN using connection-mark you just set. You can mark routing directly. Unless you’re using those marks in Filter or NAT for some reason…

Anyway, I’d like to see basic diagnostics when WAN1 is unavailable. Traceroute, for example. Because generally everything looks good.

At ping 8.8.8.8 -t, i dont even have one request timeout when i disable first wan1 then enable and then disable wan2. The echo reply is consecutive.

You mean, now everything works as expected?..

Exactly… i have also set blackhole distance=3

I know what you’re talking about. 1 packet loss every time. Just 1 literally. And it happens on LAN traffic as well.

I’ve narrowed down the problem to PCC load balancing itself, I’ve reduced it by using a destination address list for LAN traffic to exclude it from marking. But the 1 packet loss still occurs. It has nothing to do with the recursive routes.

Maybe someone else knows why this happens.

In my opinion it looks like a RouterOS bug.

I am marking connections because I cannot get any traffic to flow without conn marks.
What kind of mangle do I need here?

Mark routing on Prerouting?
Mark routing on Output?

I am convinced the reason this doesn’t work is the mangle.
My current config has the connections in Route, switching over when a link dies, so that is good. But no traffic flows.

I am using SRCNAT. Is that correct? should I be using Masquerade?

 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  ;;; DEFAULT route for WAN1 devices to WAN1
        0.0.0.0/0                          8.8.8.8                   1
 1   S  ;;; backup route for WAN1 devices to WAN2
        0.0.0.0/0                          8.8.4.4                   2
 2 A S  ;;; DEFAULT route for WAN2 devices to WAN2
        0.0.0.0/0                          8.8.4.4                   1
 3   S  ;;; backup route for WAN2 devices to WAN1
        0.0.0.0/0                          8.8.8.8                   2
 5 A S  ;;; Ping target 2 on WAN2
        8.8.4.4/32                         192.168.15.1              1
 6 A S  ;;; Ping target 1 on WAN1
        8.8.8.8/32                         192.168.10.1              1
 7 ADC  192.168.10.0/24    192.168.10.10   ether7                    0
 8 ADC  192.168.15.0/24    192.168.15.254  ether6                    0
 9  DC  192.168.40.0/24    192.168.40.1    sfp-sfpplus1            255
10 ADC  192.168.50.0/24    192.168.50.1    bridge1                   0
11  DC  192.168.51.0/24    192.168.51.1    ether5                  255
12  DC  192.168.80.0/24    192.168.80.1    ether8                  255

Route

/ip route
add check-gateway=ping comment="DEFAULT route for WAN1 devices to WAN1" distance=1 gateway=8.8.8.8 routing-mark=to_WAN1 scope=10
add check-gateway=ping comment="backup route for WAN1 devices to WAN2" distance=2 gateway=8.8.4.4 routing-mark=to_WAN1 scope=10
add check-gateway=ping comment="DEFAULT route for WAN2 devices to WAN2" distance=1 gateway=8.8.4.4 routing-mark=to_WAN2 scope=10
add check-gateway=ping comment="backup route for WAN2 devices to WAN1" distance=2 gateway=8.8.8.8 routing-mark=to_WAN2 scope=10
add comment="Ping target 2 on WAN2" distance=1 dst-address=8.8.4.4/32 gateway=192.168.15.1 scope=10
add comment="Ping target 1 on WAN1" distance=1 dst-address=8.8.8.8/32 gateway=192.168.10.1 scope=10

Mangle (EDIT - this is not correct, see next post)

add action=mark-connection chain=prerouting comment="Anything to Local 192.168.50.0/24 set NO MARK drop-out mangle" dst-address-list=MyLocalLAN \
    in-interface=bridge1 new-connection-mark=no-mark passthrough=no
add action=mark-connection chain=prerouting comment=WAN1 dst-address-list=!MyLocalWAN2 new-connection-mark=WAN1conn passthrough=yes src-address-list=\
    to_WAN1list
add action=mark-connection chain=prerouting connection-mark=!WAN2conn in-interface=ether7 new-connection-mark=WAN1conn passthrough=no
add action=mark-connection chain=prerouting dst-address-list=MyLocalWAN1 new-connection-mark=WAN1conn passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN1conn new-routing-mark=to_WAN1 passthrough=no
add action=mark-routing chain=output connection-mark=WAN1conn new-routing-mark=to_WAN1 passthrough=no
add action=mark-connection chain=prerouting comment=WAN2 dst-address-list=!MyLocalWAN1 new-connection-mark=WAN2conn passthrough=yes src-address-list=\
    to_WAN2list
add action=mark-connection chain=prerouting connection-mark=!WAN1conn in-interface=ether6 new-connection-mark=WAN2conn passthrough=no
add action=mark-connection chain=prerouting dst-address-list=MyLocalWAN2 new-connection-mark=WAN2conn passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2conn new-routing-mark=to_WAN2 passthrough=no
add action=mark-routing chain=output connection-mark=WAN2conn new-routing-mark=to_WAN2 passthrough=no

MyLocalWan1 = 192.168.10.0/24
MyLocalWan2 = 192.168.15.0/24
MyLocalLan = 192.168.50.0/24

I had another go at doing the mangle without conn marks and I think that worked.

add action=mark-routing chain=prerouting comment=WAN1 dst-address-list=!to_WAN2list new-routing-mark=to_WAN1 passthrough=no src-address-list=to_WAN1list
add action=mark-routing chain=prerouting comment=WAN2 dst-address-list=!to_WAN1list new-routing-mark=to_WAN2 passthrough=no src-address-list=to_WAN2list

With SRCNAT There is a delay of maybe 10 seconds but the sessions do not appear to recover when switching over. Pages already trying to load then stall.
With Masquerade NAT there is the same delay but things don’t properly load for about 10 seconds, then after that all is ok. I do get some page stall but appears less pronounced.

So I guess my question is which NAT type is best?

/ip firewall nat
add action=src-nat chain=srcnat comment=WAN1 disabled=yes out-interface=ether7 src-address=192.168.50.0/24 to-addresses=192.168.10.10
add action=masquerade chain=srcnat comment=WAN1 out-interface=ether7
add action=src-nat chain=srcnat comment=WAN2 disabled=yes out-interface=ether6 src-address=192.168.50.0/24 to-addresses=192.168.15.254
add action=masquerade chain=srcnat comment=WAN2 out-interface=ether6

Masqurade is meant for PPPoE or DHCP Client with dynamic IP.

Try removing the “src-address” completely. And test it again. See how it fares.

I removed the Source address and it made no difference.
I don’t know if I’m imagining it but now I have a simple Mangle on Prerouting, it appears that some web pages are stalling. Is it correct to simply have a single prerouting mangle rule covering the lan (for each WAN)?

add action=mark-routing chain=prerouting comment=WAN1 dst-address-list=!to_WAN2list new-routing-mark=to_WAN1 passthrough=no src-address-list=to_WAN1list
add action=mark-routing chain=prerouting comment=WAN2 dst-address-list=!to_WAN1list new-routing-mark=to_WAN2 passthrough=no src-address-list=to_WAN2list

Should I not have something covering WAN inputs too?

Is it possible to have a single NAT but attach 2x WAN to it?

I would need to have a failover link in my setup.
Reading through this thread, I’m a little bit confused and I was unable to use this in my setup.
Can someone help me? Is this a good way to go?

What I would need:
I have 2 uplinks (ether) and I would need if the first (main) goes down to route all traffic to the second (failover) uplink. When the main link is up, don’t route anything to the failover link.
I have tried setting the routes as described in the first post, but it did not work. Later from the thread I realized that I would need to setup mangle rules for this to work.

What would be the best in my scenario? Setup mangle rules to mark packets or something else? Can anyone help me setting up the mangle rules?

Thanks!

You don’t need routing marks at all:

/ip route
add dst-address=CheckingHost gateway=GW_MAIN_IP scope=10
add distance=1 gateway=CheckingHost check-gateway=ping
add distance=10 gateway=GW_FAILOVER_IP

Thanks I will try this!

I forgot to mention that I use DHCP client because I don’t have static public IPs. Only dynamic ones. I assume that this works there also, I just need to remove the default route from dhcp-client right?

If your gateways are static (I didn’t see any situations where they are not), just disable adding the default route. If they are not, you may use DHCP Client Script to update your routes with correct gateways.

Gateways are static, I have 2 dedicated uplink gateway, but it’s IP addresses are not static.

If I add this to my route list:

add dst-address=CheckingHost gateway=GW_MAIN_IP scope=10

Like this:

1 A S  8.8.8.8/32                         ether1                    1

I can’t reach 8.8.8.8 from my network. Why?

The only thing that might be wrong that I did not add IP to gateway, but a port. However I can’t add IP here, because I don’t have a static IP.

“gateway=etherN” works not the same as with point-to-point interfaces, and definitely not as you expect. Don’t use this.

Gateways are static

So use gateway IPs in gateway= parameter, that’s exactly what you need.