"pure" ipsec, how to deal with MTU?

nostromog · Sat Sep 21, 2019 10:35 pm

Hi, I set up in my home router a "pure" ipsec VPN, experimentally before I set it up in my company. It is currently ikev1 with xauth, something like

/ip address
add address=192.168.90.1/24 interface=bridge network=192.168.90.0
/ip ipsec mode-config
add address-pool=vpn2 name=RW-cfg split-include=0.0.0.0/0
add address-pool=vpn2 name=RWsplit-cfg split-include=192.168.88.0/24,192.168.89.0/24,192.168.90.0/24
/ip ipsec policy group
add name=RWsplit
add name=RWsecure
/ip ipsec profile
add name=rw
/ip ipsec peer
add name=RoadWarrior passive=yes profile=rw
/ip ipsec identity
add auth-method=pre-shared-key-xauth generate-policy=port-strict mode-config=RW-cfg peer=RoadWarrior policy-template-group=\
    RWsecure username=usr
add auth-method=pre-shared-key-xauth generate-policy=port-strict mode-config=RWsplit-cfg peer=RoadWarrior policy-template-group=\
    RWsplit username=usr2
add auth-method=pre-shared-key-xauth generate-policy=port-strict mode-config=RWsplit-cfg peer=RoadWarrior policy-template-group=\
    RWsplit username=usr3
/ip ipsec policy
add dst-address=192.168.90.0/24 group=RWsplit src-address=192.168.88.0/24 template=yes
add dst-address=192.168.90.0/24 group=RWsplit src-address=192.168.89.0/24 template=yes
add dst-address=192.168.90.0/24 group=RWsplit src-address=192.168.90.0/24 template=yes
add dst-address=192.168.90.0/24 group=RWsecure src-address=0.0.0.0/0 template=yes
/ip pool
add name=vpn2 ranges=192.168.90.2-192.168.90.254

You can see that different users can either have some split policies sent through the router, or send the whole traffic, which is tricky in the client side.

I'm planning to use this setting for teleworkers that have a small mikrotik router configured straight away. In those the configuration would be similar to:

> /ip ipsec export hide-sensitive     
/ip ipsec peer
add address=my.server name=Home send-initial-contact=no
/ip ipsec identity
add auth-method=pre-shared-key-xauth generate-policy=port-strict mode-config=request-only my-id=fqdn:usr.mine peer=Home \
    remote-id=fqdn:my.server username=usr
/ip route
add check-gateway=ping distance=1 gateway=192.168.90.1
add comment="to:my.server gateway" distance=1 dst-address=<myip>/32 gateway=<wan-gateway>

Now, when I use the connection from the client router, obviously, the MTU is not 1500; rather 1406 when using NAT-T or 1422 when not:

> :ping 1.1.1.1 size=1422 count=1 do-not-fragment 
  SEQ HOST                                     SIZE TTL TIME  STATUS                                                               
    0 1.1.1.1                                  1422  57 32ms 
    sent=1 received=1 packet-loss=0% min-rtt=32ms avg-rtt=32ms max-rtt=32ms 
> :ping 1.1.1.1 size=1423 count=1 do-not-fragment  
  SEQ HOST                                     SIZE TTL TIME  STATUS                                                               
    0                                                         packet too large and cannot be fragmented                            
    sent=1 received=0 packet-loss=100%

Now, how am I supposed to deal with it?

In the router there is no interface that I can use to trim the MTU
The clients just connect to the router...

Is there any "best practice" way to deal with this in the Mikrotik side?

Sat Sep 21, 2019 11:22 pm

You don't need to do anything about it. Just make sure you do not blindly block the ICMP traffic so PMTUD over your tunnels works.

nostromog · Mon Sep 23, 2019 8:21 pm

You don't need to do anything about it. Just make sure you do not blindly block the ICMP traffic so PMTUD over your tunnels works.

It is not working.

It is working locally, i.e. the router at the "client" side of the ipsec tunnel will give the error I posted, but
it is not working for clients of the given router. For them (example is a linux machine):

$ ping -s 1 -c 1 -s 1410 -M do 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 1410(1438) bytes of data.
1418 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=206 ms

--- 1.1.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 206.688/206.688/206.688/0.000 ms
$ ping -s 1 -c 1 -s 1411 -M do 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 1411(1439) bytes of data.

--- 1.1.1.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

This example has a 1438 MTU, the MTU changes depending on how the connection is done to the main router.

It is also working for clients of the router that have the split-include=192.168.88.0/24,192.168.89.0/24,192.168.90.0/24 configuration.

It is not working for clients of the router that have the split-include=0.0.0.0/0 configuration. The router does not send the fragmentation needed packet.

At least this is what I'm seeing, both with stable (6.45.6) and testing (6.46beta44) configuration in the clients and server. I have fully deactivated the firewall of the client to ensure that I was not blocking the icmp packet.

Sob · Mon Sep 23, 2019 9:31 pm

Check what the router really sends or not directly on router, add logging rule in output for icmp and destination address of client, and you'll see.

I'd expect the opposite for split-include configs, i.e. that 0.0.0.0/0 would work, but selected subnets would not. Because if the router sends icmp packet from wrong address not covered by policy, tunnel won't take it. But there's no wrong address with 0.0.0.0/0.

nostromog · Mon Sep 23, 2019 9:59 pm

Check what the router really sends or not directly on router, add logging rule in output for icmp and destination address of client, and you'll see.

This is what I did, using /tool sniffer and some logging, and I have seen.

I'd expect the opposite for split-include configs, i.e. that 0.0.0.0/0 would work, but selected subnets would not. Because if the router sends icmp packet from wrong address not covered by policy, tunnel won't take it. But there's no wrong address with 0.0.0.0/0.

Well, my laptop is sending a icmp echo from 192.168.21.251 to (say) 1.1.1.1. This is masqueraded at the router to 192.168.90.253 -> 1.1.1.1, and then it falls into the policy, so it is encrypted, but it is too big. The error is not visible anywhere. My hypothesis is that the policy "eats" the error before nat can set the source address back to 192.168.21.1, as the policy is 192.168.90.253 -> 0.0.0.0/0.

When the split-include configs are used everything works. My hypothesis is that the return packet being outside of the split-include (192.168.21.N) it is correctly returned, and what I see in linux:

From 192.168.90.253 icmp_seq=1 Frag needed and DF set (mtu = 1438)

Reinforces the hypothesis, as it is sourced from the natted network, the same subject to the policy (harmless in this case)

This would point to a bug in Routeros (or even the underlying linux machinery), I think errors during ipsec encapsulation should not be subject to ipsec policies.

And I can't find a workaround because there is no firewall action to reject-with-fragmentation-needed that I could use before nat+ipsec traps the packet...

Sob · Mon Sep 23, 2019 10:17 pm

Can't it be your srcnat rules touching something they should not? Because unless I'm lost in what's connected where, if the icmp response should go to 192.168.21.251, then 192.168.90.253 as source doesn't look right.

nostromog · Mon Sep 23, 2019 11:13 pm

Can't it be your srcnat rules touching something they should not? Because unless I'm lost in what's connected where, if the icmp response should go to 192.168.21.251, then 192.168.90.253 as source doesn't look right.

Well, I don't choose the source of an ICMP error packet generated by the kernel/network machinery. I'm sending traffic and masquerading it, it is the masqueraded packet that generates the error.

I just have a "masquerade" rule for all traffic to WAN

/ip firewall nat add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

This error packet is NOT going to WAN, 192.168.21.0 is LAN.

I think the bug or misbehaviour is that the error packet inherits the source address or the original packet instead of using the routing machinery to select the right source, which would be 192.168.21.1. Let's see if anyone from Mikrotik that knows the guts can confirm or I should send a standard bug report and wait forever.

"pure" ipsec, how to deal with MTU?

"pure" ipsec, how to deal with MTU?

Re: "pure" ipsec, how to deal with MTU?

Re: "pure" ipsec, how to deal with MTU?

Re: "pure" ipsec, how to deal with MTU?

Re: "pure" ipsec, how to deal with MTU?

Re: "pure" ipsec, how to deal with MTU?

Re: "pure" ipsec, how to deal with MTU?

Who is online