Dual wan - only one active at a time

whzzz28 · August 23, 2017, 12:05pm

Hi,

I am attempting to setup dual wan with an active/active setup, with a primary route and a fail over route.
I also wish to route specific ports (port forwarding) via the secondary WAN interface.

Port forwarding aside, I seem to have a more fundamental problem with routes.
When one WAN interface is online, the other does not pass any traffic. Even with a static route to a specific IP, it does not route.
I have two connections:

PPPoE on ether1, distance 2
DHCP on ether6, distance 1

If the PPPoE connection has a distance of 0, all traffic is routed via this interface, as expected. Ether6 is showing as online, the DHCP client shows an IP address and gateway, however using the ping tool in winbox, and selecting ether6 as interface results in no ping responses.
I added a default route

add distance=1 dst-address=8.8.8.8/32 gateway=ether6

.
However I can still not ping 8.8.8.8. It times out.
Trace isnt helpful either.

[admin@Kepler] /tool> traceroute 8.8.8.8
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS                                                                                                                    
 1                                  100%    1 timeout                                                                                                                                                           
 2                                  100%    1 timeout

If I disable ether1, ether6 kicks in (distance of 1) and all traffic flows via this interface. And it works, so the internet connection is fine.
If I repeat the above, try to ping 8.8.8.8 via the PPPoE connection now, it times out.
While ether6 is active, the DS route for the PPPoE connection shows that the gateway is unreachable.

It seems that I can only have 1 active connection at a time. I assume this shouldn’t be the case, so I probably have a config issue. I am unable to find it though.

Routes:

Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreacha
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  ;;; Routing mark route through TPG
        0.0.0.0/0                          pppoe-tpg                 1
 1 A S  ;;; Routing mark route through ABB
        0.0.0.0/0                          ether6                    1
 2 ADS  0.0.0.0/0                          180.150.12.1              0
 3  DS  0.0.0.0/0                          10.20.22.167              2
 4 X S  ;;; TPG failover, distance 2
        0.0.0.0/0                          pppoe-tpg                 2
 5 X S  ;;; ABB primary, distance 1
        0.0.0.0/0                          ether6                    1
 6 A S  8.8.8.8/32                         pppoe-tpg                 1
 7 ADC  10.7.1.0/24        10.7.1.1        bridge                    0
 8 ADC  10.20.22.167/32    203.213.114.239 pppoe-tpg                 0
 9 ADC  180.150.12.0/23    180.150.12.94   ether6                    0

add comment="Routing mark route through TPG" distance=1 gateway=pppoe-tpg routing-mark=to_WAN1
add comment="Routing mark route through ABB" distance=1 gateway=ether6 routing-mark=to_WAN2
add comment="TPG failover, distance 2" disabled=yes distance=2 gateway=pppoe-tpg
add comment="ABB primary, distance 1" disabled=yes distance=1 gateway=ether6

Any thoughts on where I should be looking?
Cheers.

pukkita · August 23, 2017, 1:08pm

Have a look at this presentation by Tomas Kirnak.

On dual WAN setups means should be deployed so that traffic entering by a given wan leaves by the same one.

Double check you’re actually masquerading on the secondary WAN too, when it’s the default route hosts from internet should ping too.

whzzz28 · August 25, 2017, 6:58am

Thanks.
I have followed this and have dual wan setup, I believe. Routes are showing that both gateways for each WAN interface are reachable.
However I am not having any luck sending any traffic out of the secondary WAN interface while the other is active.

What I am attempting to do:

All traffic goes via WAN2.
Specific port forwards setup (NAT) go via WAN1.

All traffic via WAN2 is working fine. Port forwarding via WAN2 works fine.
Port forwarding via WAN1 does not.

My mangle looks like this:

/ip firewall mangle
add action=accept chain=prerouting comment="Connected Networks" src-address-list=local-lan
add action=mark-connection chain=input comment="WAN -> ROS" connection-mark=no-mark in-interface=pppoe-tpg new-connection-mark=WAN1 passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark in-interface=ether6 new-connection-mark=WAN2 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN1 new-routing-mark=WAN1_route passthrough=yes
add action=mark-routing chain=output connection-mark=WAN2 new-routing-mark=WAN2_route passthrough=yes
add action=mark-connection chain=forward comment="WAN -> LAN" connection-mark=no-mark in-interface=pppoe-tpg new-connection-mark=WAN1->LAN passthrough=yes
add action=mark-connection chain=forward connection-mark=no-mark in-interface=ether6 new-connection-mark=WAN2->LAN passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2->LAN new-routing-mark=WAN2_route passthrough=yes src-address-list=local-lan
add action=mark-routing chain=prerouting connection-mark=WAN1->LAN new-routing-mark=WAN1_route passthrough=yes src-address-list=local-lan
add action=mark-connection chain=prerouting comment=LAN->WAN connection-mark=no-mark dst-address-list=!local-lan dst-address-type=!local new-connection-mark=LAN->WAN passthrough=yes src-address-list=\
    local-lan
add action=mark-routing chain=prerouting comment="load balancing here" connection-mark=LAN->WAN new-routing-mark=WAN1_route passthrough=yes src-address-list=local-lan
add action=mark-connection chain=prerouting comment="Sticky WAN1" connection-mark=LAN->WAN new-connection-mark=Sticky_WAN1 passthrough=yes routing-mark=WAN1_route
add action=mark-connection chain=prerouting comment="Sticky WAN2" connection-mark=LAN->WAN new-connection-mark=Sticky_WAN2 passthrough=yes routing-mark=WAN2_route
add action=mark-routing chain=prerouting comment="Sticky WAN1 - route" connection-mark=Sticky_WAN1 new-routing-mark=WAN1_route passthrough=yes src-address-list=local-lan
add action=mark-routing chain=prerouting comment="Sticky WAN2 - route" connection-mark=Sticky_WAN2 new-routing-mark=WAN2_route passthrough=yes src-address-list=local-lan

Routes look like this:

/ip route
add comment="Routing mark route through TPG" distance=1 gateway=pppoe-tpg routing-mark=WAN1_route
add comment="Routing mark route through ABB" distance=1 gateway=ether6 routing-mark=WAN2_route
add check-gateway=ping comment="ABB primary, distance 1" distance=1 gateway=ether6
add comment="TPG failover, distance 2" distance=2 gateway=pppoe-tpg
add distance=1 dst-address=8.8.8.8/32 gateway=pppoe-tpg

NAT rules:

/ip firewall nat
/ip firewall nat
add action=dst-nat chain=dstnat comment="" dst-port=35761 in-interface=pppoe-tpg protocol=tcp src-address-list=local-lan to-addresses=10.7.1.20 to-ports=61327
add action=dst-nat chain=dstnat comment="" dst-address-type="" dst-port=37245 in-interface=pppoe-tpg log=yes log-prefix="[HH]" protocol=tcp to-addresses=10.7.1.20 to-ports=37245
add action=dst-nat chain=dstnat comment="" dst-port=52268 in-interface=pppoe-tpg protocol=tcp to-addresses=10.7.1.20 to-ports=52268
add action=dst-nat chain=dstnat comment="" dst-port=52269 in-interface=pppoe-tpg protocol=tcp to-addresses=10.7.1.20 to-ports=52269
add action=masquerade chain=srcnat comment="SRC NAT - TPG" out-interface=pppoe-tpg src-address-list=local-lan
add action=masquerade chain=srcnat comment="SRC NAT - ABB" out-interface=ether6 src-address-list=local-lan

I have tried connection marking the nat rules, but this was no help.
As mentioned, port forwarding works via WAN2, but not via WAN1 (pppoe-tpg). There is no traffic flow.

whzzz28 · August 25, 2017, 7:02am

A bit of probing with logging on for one of the port forwards, I can see requests externally hitting and triggering the port forward.
So I suspect my issue is LAN → WAN.

Possibly connections are not being tagged LAN → WAN and possibly even WAN → LAN, for port forwards.

pukkita · August 25, 2017, 8:11am

You’re not doing any load balancing, and steering all LAN - > Internet traffic towards WAN1:

add action=mark-connection chain=prerouting comment=LAN->WAN connection-mark=no-mark dst-address-list=!local-lan dst-address-type=!local \
new-connection-mark=LAN->WAN passthrough=yes src-address-list=local-lan

add action=mark-routing chain=prerouting comment="load balancing here" connection-mark=LAN->WAN new-routing-mark=WAN1_route passthrough=yes src-address-list=local-lan

In fact, that rule sends all LAN traffic via WAN1, that’s why it doesn’t use WAN2 unless WAN1 is disabled.

So if you want WAN2 to be used instead of WAN1 you need to change that to:

add action=mark-connection chain=prerouting comment=LAN->WAN connection-mark=no-mark dst-address-list=!local-lan dst-address-type=!local \
new-connection-mark=LAN->WAN2 passthrough=yes src-address-list=local-lan

add action=mark-routing chain=prerouting comment="load balancing here" connection-mark=LAN->WAN2 new-routing-mark=WAN2_route passthrough=no src-address-list=local-lan

whzzz28 · August 25, 2017, 9:51am

Hi,

Yes I wish to steer all LAN traffic to WAN1, except for traffic directed through ports that are forwarded.
However I have seen an issue with doing this. So instead I am going to simplify.

My plan now is to simply forward all traffic from the specific LAN host our WAN1.
All other traffic is to go via WAN2.

Reading up on this it seems straight forward - add a srcnat with the src address of the host and the out interface as WAN1.
However this doesnt appear to work. All traffic still flows via WAN2.

/ip firewall nat
add action=masquerade chain=srcnat log-prefix="[tpg]" out-interface=pppoe-tpg src-address=10.7.1.20
add action=masquerade chain=srcnat comment="SRC NAT - TPG" out-interface=pppoe-tpg src-address-list=local-lan
add action=masquerade chain=srcnat comment="SRC NAT - ABB" out-interface=ether6 src-address-list=local-lan

I will still need to mangle. I followed the post by fabianobonin here: http://forum.mikrotik.com/t/problems-with-dual-wan-and-incoming-connections/52625/1

add action=mark-connection chain=input comment="in wan1, out wan1" in-interface=pppoe-tpg new-connection-mark=wan1_conn passthrough=yes
add action=mark-routing chain=output comment="in wan1, out wan1" connection-mark=wan1_conn new-routing-mark=WAN1_route passthrough=no
add action=mark-connection chain=input comment="in wan2, out wan2" in-interface=ether6 new-connection-mark=wan2_conn passthrough=yes
add action=mark-routing chain=output comment="in wan2, out wan2" connection-mark=wan2_conn new-routing-mark=WAN2_route passthrough=no
add action=mark-connection chain=forward comment="pfw wan1, out wan1" connection-state=new in-interface=pppoe-tpg new-connection-mark=wan1_pfw passthrough=no
add action=mark-routing chain=prerouting comment="pfw wan1, out wan1" connection-mark=wan1_pfw in-interface=bridge new-routing-mark=WAN1_route passthrough=yes
add action=mark-connection chain=forward comment="pfw wan2, out wan2" connection-state=new in-interface=ether6 new-connection-mark=wan2_pfw passthrough=no
add action=mark-routing chain=prerouting comment="pfw wan2, out wan2" connection-mark=wan2_pfw in-interface=bridge new-routing-mark=WAN2_route passthrough=no

And I am right back where I started. The host goes via WAN2 rather than WAN1, despite the srcnat rule.
I added a mangle to mark connections from 10.7.1.20 to mark them as “wan1_pwf”, but it again went via wan2.

pukkita · August 25, 2017, 11:00am

NAT is the last step in traffic manipulation, unless routing steers traffic via a specific WAN, where NAT is set, NAT is not going to happen.

Please post

/ip route print detail
/ip address print
/ip firewall address-list print