Would you help me to understand this MTU issue ?

RouterOS General
1

I’m running a GRE tunnel with IPSEC to connect two different lan subnets , 192.168.0.0/24 site A and 192.168.1.0/24 site B.
Site B lan, has no direct internet connection, so it uses site A internet ISP router as gateway.

All is working fine but now customer need to change connectivity between sites, so a new pair of routers by a different provider are now installed.
I have copied the same gre/ipsec configuration to estabilish a new tunnel (except new peers ip address).
New tunnel is up and running and traffic has been switched over it by changing existing static routes: applications like VoIP and file sharing, etc etc are working fine but B site https internet navigation and B site machine accessing A site database server are failing.
No MTU was ever set on GRE tunnels , the “actual MTU” on both tunnels is 1426

With old tunnel , I’ve checked MTU from a B site windows machine to HTTPS internet website or A site server :

ping -f -l 1399(and bigger) <host_ip> :
“Packet needs to be fragmented but DF set”

ping -f -l 1398(and smaller) <host_ip> :
reply from host…

all is fine.

Same check with new tunnel :

ping -f -l 1399(and bigger) <host_ip> :
“Packet needs to be fragmented but DF set”

ping -f -l (1391 to 1398) <host_ip> :
timeout (packet loss)

ping -f -l 1390(and smaller) <host_ip> :
reply from host…

So, just for test I have forced a MTU of 1400 on new tunnel and all is working fine also on this tunnel. Now:

ping -f -l 1373(and bigger) <host_ip> :
“Packet needs to be fragmented but DF set”

ping -f -l 1372(and smaller) <host_ip> :
reply from host…

For information the new connectivity is supplied with CISCO 877 routers while the old one was supplied with AETHRA routers.
Mainly, Mikrotik interface (eth) facing old router has a public static ip address while the eth facing new router has a private ip address and cisco router wan ip address is totally forwarded (natted) to mikrotik private ip.

Is there anybody who can explain this behaviour and how can I choose the right MTU to be set on new tunnel ??

Thank you very much.

2

It seems that the new path has a max MTU that is 8 bytes smaller then the previous path. Your MTU tests show that between 1390 and 1398, packets are disappearing. That’s cos they are rejected because the packet is not completely transferred (between 1 and 8 bytes are sent but never arrive as it’s chopped off somewhere en route). If you try an MTU of 1418, things should work as expected.
You might ask “but why does ping tell me the MTU is 1390?” Well, that’s due to the 20 bytes overhead of the IP header and 8 bytes overhead from ICMP (to arrive at a total of 28 bytes, the difference between 1390 and 1418).

Keep in mind when looking at MTU that each protocol has it’s own overhead. TCP for instance adds 20 bytes of overhead.

You might find this article helpful.

3

With the default MTU of an internet link of 1500, and using GRE over IPsec Transport (what you get when you just create a GRE Tunnel in MikroTik and set an IPsec secret for it), without a key on the GRE tunnel, the actual MTU is 1434.
MikroTik sets an MTU of 1426, as you observed. This leaves an 8-byte room for an additional GRE option like a tunnel key.

What can happen (unfortunately) is that the ISP uses PPPoE on the connection to a client, which often results in an MTU of 1492 instead of 1500. This means all MTUs in the system go down by 8 bytes. Maybe this has happened at the other end.
(when it happens local to you it should be detected by the MikroTik)

Note that the -l option of ping does not set the MTU but the length of the payload portion of the ping packet.
Use tracepath when you want to easily see the MTU on the path through the network.

4

Thank you for replies and clarifications.

Yes , with an MTU of 1418 all is working fine , probably dued in fact to ppoe 8 byte less…

Let’s see how it works…

Regards