I have a router in the middle with 2 EoIP tunnels running to two different sites. Lets say Site A,B,and C. Both A and C have a tunnel back to B.

I have set up the config for the EoIP tunnel the same on all router with the exception of the tunnel ID and IP addresses. When pinging from A to B I can ping with a 1500 byte packet, no problem. However, when I ping from B to C, the largest packet I can get through is 1458. Every once in a while a ping goes through at 1500.

Neither router has IP firewall turned on.
I enabled connection tracking on all routers.
Tried doing IP Packing because of this post - viewtopic.php?t=20440
Built new tunnels not in a bridge - assigned IP directly on interface. Still could not get 1500 between B and C
All routers are running 6.40.1. All routers are 3011s

Can layer 2 effect EoIP packets?
Anyone have any Ideas.

Set MTU on your EOIP tunnels interfaces to 1500.

I have done that. And from tunnel A to B it has fixed it, from tunnel B to C I am still unable to get the 1500

Make sure that on both EoIP tunnel end points MTU is the same and do-not-fragment=no.

Below is a capture of all 4 tunnel settings. Between A and B work but between B and C do not.

I have tried between B and C with and without a defined local address.

Because it is working between A and B i assume it is an issue with the settings in router C but I have rebuilt it and triple checked everything.

Should I open a ticket and submit .sup files?

Should work fine. A point of interest is that if the underlying network doesn't support 1500 you're just changing the point at which fragmentation occurs.

posting.php?mode=quote&f=2&p=596498

A post I wrote showing how a packet is structured. It shows EoIP and GRE.

As long as MTU is 576 IPv4 (top of head, on phone) and 1280 for IPv6 you will be fine. Packets should be fragmented as necessary through normal mechanisms.

Nothing is wrong with a sub 1500 ping MTU wise. The only way to get a ping above 1500 without fragmentation is to have transport large enough to handle the 1500 EoIP packet. By setting MTU to 1500 without large enough transport unnecessary fragmentation will occur for IPv6 traffic because ICMPv6 would tell the hosts to adjust. In IPv4 the fragmentation will occur at the tunnel driver instead of within the tunnel as we would normally expect.

I can ping with 1500 bytes outside of the tunnel, but inside I cannot. Even without don't fragment checked. I'm stumped on this one.. I run eoip tunnels lots of places.. I will see if I can get the circuit setup to pass jumbo frames. Maybe something layer 2 is blocking it

I am also using ipv4 not 6

Sniff it and wireshark it.

Just for the sake of clarity, you are aware that you can't get a full 1500 byte payload inside of a 1500 byte MTU right? Your payload still has to fit in minus whatever headers you have wedged in there, IP, EoIP, whatever.

Also, as was said above, does it really matter that much? If it's being fragmented somewhere what is the point?

For example:

$ ping -c 1 -s 1472 192.168.88.1
PING 192.168.88.1 (192.168.88.1) 1472(1500) bytes of data.
1480 bytes from 192.168.88.1: icmp_seq=1 ttl=64 time=0.336 ms

--- 192.168.88.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.336/0.336/0.336/0.000 ms

$ ping -c 1 -s 1473 192.168.88.1
PING 192.168.88.1 (192.168.88.1) 1473(1501) bytes of data.

--- 192.168.88.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

Every other tunnel i have I can ping 1500 bytes.. Except this one tunnel. It actually looks like it is not letting the packets fragment.

I guarantee you that if you have a 1500 IP MTU on a tunnel interface, then something somewhere is doing fragmentation / reassembly because something's gotta give.

Native un-tunneled IP packets typically expect to have 1500 bytes to play with - tax, tag, and title. This means that the L2 MTU needs to have the extra 18 bytes needed for the ethernet header (in the case of ethernet layer2 links).

so if you're doing tunneling, then the outer IP packet has the normal 1500 bytes of MTU space, but the INNER interface loses X bytes - X being the header size of your particular tunnel protocol's header. In the case of EoIP, the header size is 42 bytes.

It's got to come from SOMEWHERE.

If you set the inner interface's MTU to 1500, then that means the space goes "outward" - i.e. the outer IP MTU must be 1542 in order to carry this packet w/o fragmentation, and then add another 18 bytes for ethernet links - so the ethernet interface's L2MTU needs to be at LEAST 1560 to carry your 1500-byte inner packet without fragmentation. If any link in the chain between B and C cannot carry the associated frame size (1560) or packet size (1542) then that device will do something based on the situation.

A: If it's a switch w/ a l2mtu less than 1560, it's going to silently discard the frame and bump its giants counter on the interface.
B: If it's a router (or L3 switch) with the next-hop interface's IP MTU being less than 1542 then one of two things will happen:
- 1) if the EoIP packet has the DF bit set, then the router will discard the oversized packet and return an ICMP mtu exceeded message to your tunnel Mikrotik which sent the EoIP packet
- 2) if the EoIP packet has no DF bit set, then the router will fragment the EoIP packet and send the chunks along their way. Your receiving tunnel router will get the two fragments (hopefully) and re-assemble them into the original 1542-byte packet, decapsulate the inner payload, which is an ethernet frame containing an IP packet - which will be 1518 bytes - decode THAT and then handle the resulting 1500 byte inner IP packet.

Note that situation A looks exactly like your situation.
Note that situation B1 looks exactly like yours as well, except that you would be able to detect ICMP packet too large messages coming from wherever it is bottle-necking. (assuming that nobody's filtering the ICMP messages on their path back to you)

Probably in your case, the EoIP packets are getting fragmented/reassembled on "all the other tunnels" but getting discarded somewhere along the path from B to C.

The clean solution is to determine the true path MTU of the inner packets, and set your EoIP interfaces' IP MTUs to that value. Your own network will be able to use path mtu discovery to properly utilize all of the links. (assuming that you're not being daft and filtering ICMP messages)

I've had similar problem, where Mikrotik will not obey configured 1500 MTU on EoIP. One day it was working great with 1500 MTU, router rebooted and after that i couldn't get packets bigger than 1458 through EoIP, even though no other changes in network were made. I fixed my problem after updating RouterOS (in my case problem appeared on 6.38.7 and it was solved after updating to 6.40.1) and that also happened one other time but I don't remember software versions. Anyway, in both cases updating RouterOS solved my problem.

It appears to have been a layer 2 issue with the fiber carrier.

Seems there is a MTU on EoIP bug in 6.37 - 6.39. Updating to 6.40.3 fixed the issue for me.