NAT with load balancing - understanding order of operation

RouterOS General
1

Hi,

New to Mikrotik, I've gotten the router configured to use two ISP's, everything on the LAN side is working fine, outbound connections work. I'm trying to setup inbound connections for servers and services inside the network and have just begun testing. I've read through Stephen Dischers book - RouterOS by Example, and have tried to setup the NAT. I'd like to know how I can identify and test where my rules are may be failing. I've tried turning on logging, but it's not showing me what I think I need to see. I'm sure the router is dropping the packets, I can see the packet count go up on the drop rule, what I can't figure out is how I can determine why my rules aren't matching. I'll post the export here, an answer would be great, but learning how or why I'm not able to determine where my packets aren't matching the rules is what I'm after. TIA.

jan/22/2020 14:40:59 by RouterOS 6.46.1

software id = 8R12-JUFA

model = CCR1009-7G-1C-1S+

serial number = 91500A03CCCA

/interface ethernet
set [ find default-name=combo1 ] disabled=yes
set [ find default-name=ether1 ] comment="WAN1 CMCSA Comcast" speed=100Mbps
set [ find default-name=ether2 ] comment="WAN2 ftr frontier fios" speed=100Mbps
set [ find default-name=ether3 ] comment=dot5 speed=100Mbps
set [ find default-name=ether4 ] comment=dot100 speed=100Mbps
set [ find default-name=ether5 ] disabled=yes speed=100Mbps
set [ find default-name=ether6 ] disabled=yes speed=100Mbps
set [ find default-name=ether7 ] disabled=yes speed=100Mbps
set [ find default-name=sfp-sfpplus1 ] advertise=10M-full,100M-full,1000M-full disabled=yes
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface list member
add interface=ether1 list=WAN
add interface=ether4 list=LAN
add interface=ether3 list=LAN
add interface=ether2 list=WAN
/ip address
add address=10.1.10.10/24 comment="CMCSA Comcast/Xfinity Business" disabled=yes interface=ether1 network=10.1.10.0
add address=192.168.5.1/24 comment="LAN POE" interface=ether3 network=192.168.5.0
add address=192.168.100.1/24 comment="LAN Blade" interface=ether4 network=192.168.100.0
add address=184.18.219.182/30 comment="FTR Frontier FIOS" interface=ether2 network=184.18.219.180
add address=162.17.15.193/30 comment="CMCSA Comcast/Xfinity Business" interface=ether1 network=162.17.15.192
/ip dns
set servers=8.8.8.8,8.8.4.4
/ip firewall address-list
add address=192.168.5.0/24 list=MyLAN
add address=192.168.100.0/24 list=MyLAN
add address=162.17.15.0/30 list=ConnectedNetworks
add address=184.18.219.0/30 list=ConnectedNetworks
add address=192.168.100.0/24 list=ConnectedNetworks
add address=192.168.5.0/24 list=ConnectedNetworks
add address=192.168.100.0/24 list=MasqueradedNetworks
add address=192.168.5.0/24 list=MasqueradedNetworks
/ip firewall filter
add action=drop chain=input connection-state=invalid
add action=drop chain=forward connection-state=invalid
add action=accept chain=input src-address-list=MyLAN
add action=accept chain=input connection-state=established
add action=drop chain=input
add action=accept chain=forward connection-state=new src-address-list=MyLAN
add action=accept chain=forward connection-state=related
add action=accept chain=forward connection-state=established
add action=drop chain=forward
/ip firewall mangle
add action=accept chain=prerouting comment="Allow connected networks to exit Mangle chain so we don't load balance to
our connected networks. Put all LAN and WAN connected networks in the address list [ConnectedNetworks]"
dst-address-list=ConnectedNetworks
add action=mark-connection chain=prerouting comment=
"Create Mangle rules that will sort the traffic into streams WAN1" connection-mark=no-mark dst-address-type=
!local in-interface=ether3 new-connection-mark=WAN1 passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting comment=
"Create Mangle rules that will sort the traffic into streams WAN2" connection-mark=no-mark dst-address-type=
!local in-interface=ether3 new-connection-mark=WAN2 passthrough=yes per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting comment=
"Create the mangles to add the routing marks to the packets basedon the connection mark in the PREROUTING CHAIN"
connection-mark=WAN1 in-interface=ether3 new-routing-mark=ether1-mark passthrough=yes
add action=mark-routing chain=prerouting comment=
"Create the mangles to add the routing marks to the packets basedon the connection mark in the PREROUTING CHAIN"
connection-mark=WAN2 in-interface=ether3 new-routing-mark=ether2-mark passthrough=yes
add action=mark-routing chain=output comment=
"This rule ensures traffic from the router itself returns through the proper interface" connection-mark=WAN1
new-routing-mark=ether1-mark passthrough=yes
add action=mark-routing chain=output comment=
"This rule ensures traffic from the router itself returns through the proper interface" connection-mark=WAN2
new-routing-mark=ether2-mark passthrough=yes
add action=mark-connection chain=prerouting comment=
"Identify which WAN interface the traffic came in and mark the connections appropriately" connection-mark=no-mark
in-interface=ether1 new-connection-mark=WAN1 passthrough=yes
add action=mark-connection chain=prerouting comment=
"Identify which WAN interface the traffic came in and mark the connections appropriately" connection-mark=no-mark
in-interface=ether2 new-connection-mark=WAN2 passthrough=yes
add action=mark-routing chain=output comment="Mark traffic from the router" new-routing-mark=ether1-mark src-address=
162.17.15.193
add action=mark-routing chain=output comment="Mark traffic from the router" new-routing-mark=ether2-mark src-address=
184.18.219.182
/ip firewall nat
add action=masquerade chain=srcnat comment="Masquerade traffic out WAN1 from [MasqueradeNetworks]" out-interface=
ether1 src-address-list=MasqueradedNetworks
add action=masquerade chain=srcnat comment="Masquerade traffic out WAN2 from [MasqueradeNetworks]" out-interface=
ether2 src-address-list=MasqueradedNetworks
add action=masquerade chain=srcnat out-interface-list=WAN
add action=log chain=dstnat disabled=yes dst-address=162.17.15.193 log-prefix=" dstnat" protocol=tcp
add action=accept chain=dstnat dst-port=123 in-interface-list=WAN protocol=tcp to-addresses=192.168.5.100 to-ports=22
add action=dst-nat chain=dstnat dst-address=162.17.15.193 dst-port=123 in-interface-list=WAN protocol=tcp
to-addresses=192.168.5.100 to-ports=22
/ip route
add check-gateway=ping comment="Default router WAN1, marked" distance=1 gateway=162.17.15.194 routing-mark=
ether1-mark
add check-gateway=ping comment="Default router WAN2, marked" distance=1 gateway=184.18.219.181 routing-mark=
ether3-mark
add distance=1 gateway=10.1.10.1
add comment="Default router WAN2, unmarked" distance=1 gateway=184.18.219.181
add comment="Default router WAN1, unmarked" distance=1 gateway=162.17.15.194
/lcd
set time-interval=daily
/system clock
set time-zone-name=America/Chicago
/system ntp client
set enabled=yes server-dns-names=us.pool.ntp.org

2

So, I found the packet trace feature and logging. Awesome. I can see my inbound packets from the WAN with their destination ip address and port being correct, I can see the 3 syn attempts trying to start the connection. What I can’t seem to figure out is where this packet is being dropped in my rules. What should my next steps be?

3

Hi, you said “inbound connections”, but the only forward I see for new connections is:
add action=accept chain=forward connection-state=new src-address-list=MyLAN
This is not covering WAN src addresses as inbound connection, isn’t it?

4

This is what was killing me:

add action=drop chain=forward

With the nat rule enabled, and the drop disabled I can connect. Now to find out what else passes this path.

Thanks.

5

It’s very easy, you can allow all forwarded ports using connection-nat-state=dstnat matcher. Also the order of rules could be more efficient:

/ip firewall filter
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward src-address-list=MyLAN
add action=accept chain=forward connection-nat-state=dstnat
add action=drop chain=forward

It’s because usually vast majority of packets belong to established connections, so it’s better when they are accepted by first rule. The same rule can allow related and untracked (you want that). And if you deal with invalid next, then everything else is new, so you don’t need to worry about connection state in following rules. The same is true for input.