Hi,
I have an RB750Gr3 hex router and created a LAN to LAN IKEv2 IPSEC tunnel between two sites and subnets. The tunnel comes up and works fine, I have even created a mangle rule to clamp MSS to PMTU. I have a Cisco 2802 AP with mobility express setup on one end and a Microsoft NPS radius server on the other end of the IPSEC tunnel. The problem is that using EAP-TLS certificate based wifi authentication the large UDP packets arrive fragmented to the Mikrotik’s LAN interface (1514 bytes + a smaller 350 bytes fragment) but they are not sent over the IPSEC tunnel , just dropped by the router. The DF bit is not set in these packets so I would expect it to be reassembled and then fragmented again to the mtu size of the IPSEC tunnel. The same works fine when I use a Draytek Vigor router, that does this fragmentation before sending in IPSEC tunnel. I have read a lots of forum topics here but no luck so far. Like the Mikrotik does not handle these correctly.
Any idea what config trick could help?
Thanks
Hi @Ishe and welcome to the forum!
Are you running ROS v6?
I’m not entirely sure, but I think ROS v6 had some issues with IP reassembly back in the day. Also (and this is just speculation) I vaguely remember reading somewhere that if fast-path was enabled, fragmented UDP packets might get dropped or skipped in some cases, like when they don’t match a full connection. Might’ve been a v6 thing too.
You could try disabling fast-path temporarily with:
/ip settings set allow-fast-path=no
Another thing you might do is add logging rules to see if fragments are being silently dropped. Just make sure the rules are placed high enough in the list so they actually gets triggered.
/ip firewall filter add action=log chain=forward protocol=udp fragment=yes log-prefix=FWD_FRAG_DROP
/ip firewall filter add action=log chain=input protocol=udp fragment=yes log-prefix=INP_FRAG_DROP
Hi Larsa,
I am using Router OS v6 (6.49.18), however I have tested it with v7 too and faced the same issue. I just reverted it back to v6.
I have tested what you suggested, disabled fast path but it did not help. The firewall logs you specified did not catch any packets, even though I have moved them up to the top of the rules (under the first passthrough rule). It looks like those fragments never reach the firewall module? I can see them when in switch I mirror the port of the WLC (wlan AP) and I use a PC with wireshark. I have done a quick packet sniff in Mikrotik cli and I can see the large packet coming in on ether5 (where my AP is connected) using size 1514 and its fragment using size 351.
ether5 8.014 21 ← 00:00:5E:00:01:01 F4:1E:57:18:DD:0C 10.110.7.5:46565 10.110.2.11:1812 (radius) ip:udp 1514 2 no
bridge 8.014 22 ← 00:00:5E:00:01:01 F4:1E:57:18:DD:0C 10.110.7.5:46565 10.110.2.11:1812 (radius) ip:udp 1514 2 no
ether5 8.014 23 ← 00:00:5E:00:01:01 F4:1E:57:18:DD:0C 10.110.7.5:12593 10.110.2.11:13369 ip:udp 351 2 no
bridge 8.014 24 ← 00:00:5E:00:01:01 F4:1E:57:18:DD:0C 10.110.7.5:12593 10.110.2.11:13369 ip:udp 351 2 no
the IPSEC tunnel is between 10.110.2.0/24 and 10.110.7.0/24. It is like this fragment is not reassembled and sent to ipsec. However if it needs to be sent out on the WAN interface (ether1) then it is sent properly (of course the MTU there is 1500 too).
One thing you could try is adding a RAW rule to see if those fragments even hit the router’s CPU. RAW is processed before conntrack or filters, so it might be a good way to check if something’s silently dropping them early.
Use something like this: “/ip firewall raw add chain=prerouting fragment=yes action=log log-prefix=FRAG_RAW”
If you spot anything in the logs, it means the fragments at least reaching the first chain.
EDIT
This is probably a pretty dumb question at this point, but is the NPS server on the Mikrotik or the Cisco side? Just trying to figure out if this is an inbound or outbound issue from the Mikrotik’s point of view.
Hi,
Ok I have tried this raw log rule and I can see hits in the log.
FRAG_RAW prerouting: in:bridge out:(unknown 0), src-mac 00:00:5e:00:01:01, proto UDP, 10.110.7.5->10.110.2.11, len 337
That out:unknown worries me though. Also I can only see len 337 packets and not the large ones. Like it is the second fragment only?
To give some more details about my setup:
I have a mikrotik router and a Cisco 2802 wlan AP with Cisco Mobility Express (WLC) connected to it. This Cisco is a radius client of a Microsoft NPS server which is on the other end of the ipsec tunnel. The Ipsec tunnel is between a Draytek Vigor and the Mikrotik. The NPS server policy is configured to send a framed-mtu=1100 value and I can see that NPS honors it but the Windows wireless client not much. The windows client uses 1486 framed mtu for EAP packets and that is which does not go through.
On my home setup I have a pfsense firewall as router and ipsec tunnel to the same Draytek Vigor. If I connect this Cisco Mobility Express AP to my LAN then it works fine, all packets go through.