I am debugging a very strange issue experienced in our HAMNET network. Prerequisites:
- GRE or GRE6 tunnels transporting IPv4 traffic.
- DF set to “inherit”.
- Traffic with DF flag, but smaller than MTU.
- Traffic forwarded to destination beyond the router.
Observation: WinBox sessions run very sluggish. I experienced this quite a while ago and experimented a bit, in the end I found that adding IPsec to the tunnel removes the issue. But today I went looking again and I finally found what is going on.
The router is sending “ICMP: Fragmentation needed” replies to traffic, and from a wireshark trace it appears that the payload of the ICMP packet indicates a wrong length field in the inner IP header, and that is likely the reason why it was returned. But the original packet as received from the internet had a correct length field! It is being corrupted somewhere.
Sending traffic without DF prevents the issue from happening (I checked using a mangle rule on the remote router to “clear DF” on winbox traffic). Setting “DF: no” on the GRE tunnel on the remote end also fixes it.
So apparently it only happens when both the inner and outer header have DF set. And for some reason it only happens on plain GRE and GRE6 tunnels, not when IPsec is used. That may be why it is seldomly seen.
When doing traffic between the endpoint routers of the GRE tunnel, e.g. a bandwidth test, everything runs normal. The issue is only visible when forwarding traffic.
Has anyone observed this or does it ring a bell?
I have created SUP-222764 including traces and supout, but I have 4 other open issues "waiting for support" so it seems better to ask here as well...
I think the issue has started somewhere in the 7.20.x versions but it still occurs in 7.24.1 (only the version at the receiving end matters, the remote is 6.49.20).