Multiple WAN, multiple WAN

RouterOS General
1

At a customer I have a CCR with 3 x WAN and 3 x LAN. In this situation I would like to link each LAN to an own WAN IP address.

Public IPs:
2.2.2.2/29 - ether10
4.4.4.4/29 - ether11
6.6.6.6/29 - ether12

Internal IPs:
10.0.0.1/16 - ether1
10.10.0.1/16 - vlan10
10.20.0.1/16 - vlan20

NAT:
10.0.0.1/16 → 4.4.4.4/29
10.10.0.1/16 → 2.2.2.2/29
10.20.0.1/16 → 6.6.6.6/29

Now I want to make sure that 10.0.0.1/16 goes out on ether11, 10.10.0.1/16 on ether 10 and 10.20.0.1/16 on ether12, all with NAT to the public IP. I have already created a configuration that ensures this. So if you are on vlan10 for example, you have public IP address 2.2.2.2.

Now the problem is: on the 10.0.0.1/16 subnet there are several servers. This servers need to be reachable from the other subnets. The strange this is, from 10.20.0.1/16 I can reach the 10.0.0.1/16 subnet without any problems. But from the 10.10.0.1/16 any request to the 10.0.0.1/16 subnet is routed to the public WAN gateway.

/ip address
add address=10.0.0.1/16 interface=ether1 network=10.0.0.0
add address=10.10.0.1/16 interface=vlan10 network=10.10.0.0
add address=10.20.0.1/16 interface=vlan20 network=10.20.0.0
add address=2.2.2.2/29 interface=ether10 network=2.2.2.0
add address=4.4.4.4/29 interface=ether11 network=4.4.4.0
add address=6.6.6.6/29 interface=ether12 network=6.6.6.0

/ip firewall mangle
add action=mark-connection chain=input in-interface=ether11 \
    new-connection-mark=WAN1_conn
add action=mark-connection chain=input in-interface=ether12 \
    new-connection-mark=WAN2_conn
add action=mark-connection chain=input in-interface=ether10 \
    new-connection-mark=WAN3_conn
add action=mark-routing chain=output connection-mark=WAN1_conn \
    new-routing-mark=to_WAN1
add action=mark-routing chain=output connection-mark=WAN2_conn \
    new-routing-mark=to_WAN2
add action=mark-routing chain=output connection-mark=WAN3_conn \
    new-routing-mark=to_WAN3
add chain=prerouting dst-address=6.6.6.0/29 in-interface=vlan20
add chain=prerouting dst-address=4.4.4.0/29 in-interface=ether1
add chain=prerouting dst-address=2.2.2.0/29 in-interface=vlan10
add action=mark-connection chain=prerouting dst-address-type=!local \
    in-interface=ether1 new-connection-mark=WAN1_conn
add action=mark-connection chain=prerouting dst-address-type=!local \
    in-interface=vlan10 new-connection-mark=WAN3_conn
add action=mark-connection chain=prerouting dst-address-type=!local \
    in-interface=vlan20 new-connection-mark=WAN2_conn
add action=mark-routing chain=prerouting connection-mark=WAN1_conn \
    in-interface=ether1 new-routing-mark=to_WAN1
add action=mark-routing chain=prerouting connection-mark=WAN3_conn \
    in-interface=vlan10 new-routing-mark=to_WAN3
add action=mark-routing chain=prerouting connection-mark=WAN2_conn \
    in-interface=vlan20 new-routing-mark=to_WAN2
	
/ip firewall nat
add action=masquerade chain=srcnat
add action=masquerade chain=srcnat out-interface=ether10
add action=masquerade chain=srcnat out-interface=ether11
	
/ip route
add distance=1 gateway=4.4.4.1 routing-mark=to_WAN1
add distance=1 gateway=6.6.6.1 routing-mark=to_WAN2
add distance=1 gateway=2.2.2.1 routing-mark=to_WAN3
add distance=1 gateway=4.4.4.1
add distance=2 gateway=6.6.6.1
add distance=3 gateway=2.2.2.1
2

You need to accept connections between subnets before you are marking them. This way they will not be marked and will be processed normally in the routing table:

/ip firewall mangle
add chain=prerouting src-address=10.0.0.0/16 dst-address=10.10.0.0/16 action accept
add chain=prerouting src-address=10.0.0.0/16 dst-address=10.20.0.0/16 action accept
add chain=prerouting src-address=10.10.0.0/16 dst-address=10.0.0.0/16 action accept
add chain=prerouting src-address=10.10.0.0/16 dst-address=10.20.0.0/16 action accept
add chain=prerouting src-address=10.20.0.0/16 dst-address=10.0.0.0/16 action accept
add chain=prerouting src-address=10.20.0.0/16 dst-address=10.10.0.0/16 action accept

The above rules have to stand before the marking process starts, rule number 10 in your post.

Also, i don’t understand these rules on your post

add chain=prerouting dst-address=6.6.6.0/29 in-interface=vlan20
add chain=prerouting dst-address=4.4.4.0/29 in-interface=ether1
add chain=prerouting dst-address=2.2.2.0/29 in-interface=vlan10

there is no action defined for them.

You have three masquerade rules, but only the first one is actually working since it grabs all the traffic and so the traffic is not processed any further below, I believe.

3

Thank you Caci99, this does the trick!

Also, i don’t understand these rules on your post

When I look in Winbox, these have the action Accept, don’t know why it is exported like this.