Page 1 of 1

Feature request: connection nat mismatch detection

Posted: Thu Jul 11, 2019 12:33 pm
by sebastia
Hi

When operating a router with wan fail-over, when NAT is applied to both links, (ex two residential ISP connection), it is possible that "ip leakage" can occur.
This is only relevant for networks bound to specific ranges, such as for residential ISP. This doesn't apply to situation when dynamic routing protocols are used to route ip traffic (with peering) over best path possible.

In such a residential configuration, on fail-over event packets with wrong ip (=ip of the other ISP) will be forwarded over the fail-over interface as long as the original connection hasn't timed out.
This is something else than private ip range leakage, as addressed in this presentation: https://mum.mikrotik.com/presentations/ ... 042977.pdf, slides 27-31, video: https://youtu.be/D80_a_O86jc.

To replicate and analyse the behaviour I've setup a test environment in GNS3:
Network.png
C-1: "client" simulated by Tik instance, with ssh client
R-1: client side router, with fail-over connections and natting on both up-links
R-2: "internet"
S-1: "server" simulated by Tik, with ssh server enabled

Config info of R-1:
* natting of both uplinks, ether1 is primary, ether5 secondary
* ether1 is src-nat-ing, ether5 masquerading
* filter:forward drop invalid is set

Procedure to replicate issue:
* start capture on R-1:eth5<->R-2:eth5 link
* start ssh session from C-1 to S-1 (over primary link R-1:eth1<->R-2:eth1)
* suspend R-1:eth1<->R-2:eth1
* wait and observe leakage on secondary link R-1:eth5<->R-2:eth5

Observation: R-1 transmits packets with ip of eth1 on eth5. On residential ISP connection, this doesn't make any sense...


Work around:
Looking at the flow (https://wiki.mikrotik.com/wiki/Manual:Packet_Flow_v6), after "Src-Nat" step in Postrouting, actual packet header can't be consulted, so explicit verification is not possible.
But indirectly could be possible: on state=new in mangle:forward (after routing) mark the connection to indicate chosen route/outgoing interface.
On state=established (for outbound packets only) verify in filter:forward that the outgoing interface still matches the connection mark. If it doesn't then it's a case of wrong src ip as observed.

Additional config R-1
/ip firewall mangle
add action=mark-connection chain=forward connection-state=new new-connection-mark=ether1 out-interface=ether1
add action=mark-connection chain=forward connection-state=new new-connection-mark=ether5 out-interface=ether5

/ip firewall filter
add action=passthrough chain=forward connection-mark=!ether1 connection-state=established log=yes log-prefix="Leak: " out-interface=ether1
add action=passthrough chain=forward connection-mark=!ether5 connection-state=established log=yes log-prefix="Leak: " out-interface=ether5
Log entries:
10:35:04 firewall,info Leak:  forward: in:ether3 out:ether5, src-mac 0c:27:06:71:89:02, proto TCP (ACK,PSH), 192.168.42.11:34753->192.168.43.2:22, NAT (192.168.42.11:34753->192.168.41.2:34753)->192.168.43.2:22, len 156 
10:35:32 firewall,info Leak:  forward: in:ether3 out:ether5, src-mac 0c:27:06:71:89:02, proto TCP (ACK,PSH), 192.168.42.11:34753->192.168.43.2:22, NAT (192.168.42.11:34753->192.168.41.2:34753)->192.168.43.2:22, len 156 
Notice the "NAT (192.168.42.11:34753->192.168.41.2:34753)":
* connection tracking knows that NAT is needed
* connection tracking knows what is the associated outing ip
* so verifying that outgoing NAT ip belongs to the used outing interface should be easy to do

Feature request:
Please expose such "connection-nat-ipmismatch" function, so implementing such filtering doesn't have to go through roundabout way over mangling (with all complications of that, ex when using mangling for QOS).

Thank you
Sebastian

Re: Feature request: connection nat mismatch detection

Posted: Fri Jul 12, 2019 12:28 pm
by sindy
@Sebastia, depending on the balance between damage done by address leakage and damage caused by CPU load, you can make the actual WAN interface (unless it is some kind of L3-only one) a member port of a dedicated /interface bridge, migrate the IP configuration from the actual WAN to the bridge, and use /interface bridge filter rules to block incompatible IP addresses in chain=output.

Re: Feature request: connection nat mismatch detection

Posted: Sat Jul 13, 2019 2:22 pm
by sebastia
Thank you for your feedback.

Tried the suggestion:

Additional config:
/interface bridge
add name=bridgeE5 protocol-mode=none
/interface bridge filter
add action=passthrough chain=output log=yes log-prefix="Bridge rule: " mac-protocol=ip src-address=!192.168.45.2/32
/interface bridge port
add bridge=bridgeE5 interface=ether5
# + adjustment of ether5 -> bridgeE5 in filter/mangle/nat

This gives me:
11:04:18 firewall,info Leak:  forward: in:ether3 out:bridgeE5, src-mac 0c:27:06:71:89:02, proto TCP (ACK,PSH), 192.168.42.11:34109->192.168.43.2:22, NAT (192.168.42.11:34109->192.168.41.2:34109)->192.168.43.2:22, len 156 

11:04:18 firewall,info Bridge rule:  output: in:(unknown 0) out:ether5, src-mac 0c:27:06:62:23:04, dst-mac 0c:27:06:94:2a:04, eth-proto 0800, TCP (ACK,PSH), 192.168.41.2:34109->192.168.43.2:22, NAT (192.168.42.11:34109->192.168.41.2:34109)->192.168.43.2:22, len 156 

Then there is indeed the question of load impact...
In my case, it's "low" throughput, manually capped to 20mbit, LTE connection functioning as fail-over.
Since it's a metered (limited volume) link, I care more about the leak traffic, which is significant from my main link, than the extra load, which is low due to limited throughput.

The added bonus of this work-around is that I can keep on using "FastTrack"-ing, and don't have to mangle everything. So win-win ...

Regards
Sebastian

Re: Feature request: connection nat mismatch detection

Posted: Sat Jul 13, 2019 4:47 pm
by Sob
Please expose such "connection-nat-ipmismatch" function, so implementing such filtering doesn't have to go through roundabout way over mangling ...
It does sound useful, but it's not always easy to tell what address is wrong.

For example, with hairpin NAT, I usually use srcnat with to-addresses=<public address>, where public address is on different interface, so that would be a mismatch. Or it's possible to use srcnat with addresses that router doesn't have at all (this is also hairpin NAT, which can be seen as special case, but the same can be used for any other transparent network mapping). And I didn't even start inventing something crazy. ;)

But as optional matcher that would simply check addresses on outgoing interface, which would work for your simple (and probably most common) case, why not.