IPIP Tunnel MTU Problem

RouterOS General
1

Have 3 devices:

DEVICE_1:
WAN PPP interface with static external IP 1.1.1.1 & real MTU/MRU 1500

DEVICE_2:
WAN PPPoE interface with static external IP 2.2.2.2 & real MTU/MRU 1492

DEVICE_3:
WAN ethernet interface with static external IP 3.3.3.3 & real MTU 1500

All devices with ROS 6.33.5


From DEVICE 1 established IPIP tunnels to DEVICE 2 & to DEVICE 3

Tunnels parameters are absolutely identical for all devices (except for the IP address & MTU).

While examined the best possible MTU for all tunnels set it all to 1500 and started Ping with DF flag from DEVICE_1 to tunnel IP address of DEVICE_2.

Found that, contrary to the logic, tunnel from D_1 to D_2 passes packets up to 1500 bytes without fargmentation, but tunnel from D_1 to D_3 passes packets up to 1480 bytes, which in general is normal. But I am confused by the behavior of the tunnel from D_1 to D_2. I thought that this tunnel dynamically fragmenting packets at the time of encapsulation to be assembled on the end. Set tunnel MTU to 1600 only confirmed my assumption (and even delighted). But tunnel between D_1 and D_3 still passed whole packets maximum of 1480 bytes :confused:

After reading a lot of documentation I thought possible provider of D_3 just removes the DF flag from incapsulated packets (it happens), but assumption has not been confirmed because packets more than 1600 bytes are discarded anyway.

Could not find anything about automatic fragmentation at the documentation about IPIP tunnels.
It is also not clear what is the reason that this mechanism (if any) does not work for the tunnel between D_1 and D_3.

How did it actually?

Trying to examine the document but think not quickly able to understand it, and not the fact that it will help: http://www.cisco.com/c/en/us/support/docs/ip/generic-routing-encapsulation-gre/25885-pmtud-ipfrag.html

Next, I try to put all the settings that can be involved.

2

DEVICE_1:

/interface ppp-client
add add-default-route=no apn=yota.ru dial-on-demand=no disabled=no info-channel=1 name=ppp-yota port=usb1 profile=YOTA

/interface ipip
add !keepalive local-address=1.1.1.1 mtu=1600 name=ipip-to-home-vladlink2 remote-address=2.2.2.2
add !keepalive local-address=1.1.1.1 mtu=1600 name=ipip-to-home-vtc remote-address=3.3.3.3
    
/ip address
add address=192.168.60.1/30 interface=ipip-to-home-vtc network=192.168.60.0
add address=192.168.60.18/30 interface=ipip-to-home-vladlink2 network=192.168.60.16

/ip firewall mangle
add action=change-mss chain=forward in-interface=ppp-yota new-mss=1460 \
    protocol=tcp tcp-flags=syn,!rst tcp-mss=1461-65535
add action=change-mss chain=postrouting new-mss=1460 out-interface=ppp-yota \
    protocol=tcp tcp-flags=syn,!rst tcp-mss=1461-65535

DEVICE_2:

/interface pppoe-client
add allow=chap,mschap1,mschap2 disabled=no interface=ether6-wan max-mru=1492 \
    max-mtu=1492 name=PPPoE-VTC password=****** profile=VTC use-peer-dns=yes \
    user=frozer
/interface ipip
add !keepalive local-address=2.2.2.2 mtu=1600 name=ipip-to-tax-yota \
    remote-address=1.1.1.1
    
/ip address
add address=192.168.60.2/30 interface=ipip-to-tax-yota network=192.168.60.0

/ip firewall mangle
add action=change-mss chain=forward in-interface=PPPoE-VTC new-mss=1452 \
    protocol=tcp tcp-flags=syn,!rst tcp-mss=1453-65535
add action=change-mss chain=postrouting new-mss=1452 out-interface=PPPoE-VTC \
    protocol=tcp tcp-flags=syn,!rst tcp-mss=1453-65535

DEVICE_3:

/interface ethernet
set [ find default-name=ether6 ] mac-address=00:1C:C4:47:79:F5 name=\
    ether6-wan
/interface ipip
add !keepalive local-address=3.3.3.3 mtu=1600 name=ipip-to-tax-yota2 \
    remote-address=1.1.1.1
    
/ip address
add address=192.168.60.17/30 interface=ipip-to-tax-yota2 network=\
    192.168.60.16
    
/ip firewall mangle
add action=change-mss chain=forward in-interface=ether6-wan new-mss=1460 \
    protocol=tcp tcp-flags=syn,!rst tcp-mss=1461-65535
add action=change-mss chain=postrouting new-mss=1460 out-interface=ether6-wan \
    protocol=tcp tcp-flags=syn,!rst tcp-mss=1461-65535
3

This issue is very important for me because I would like to finally get that tunnel though would be (formally) transmits packets larger than the MTU of physical channels through which the tunnel is organized. Affliction with changing TCP SYN values is not very suitable because it is only suitable for the TCP protocol as I understand. I have observed problems with HTTPS protocol also within tunnels with MTU lower then 1500.

4

Interesting problem for sure.

I would not think you would need to specify source IP as you are using /30 addressing so it should only go one direction anyway.

Given your wan connections have MTU limits of either 1500 or 1492 themselves I cannot see how you can push 1600 through any tunnel without fragmentation. WAN MTU less IPIP Protocol Overhead = 1500-

Do your IPIP tunnels have do-not fragment=inherit set or the default (no) which allows fragmentation ?

IPIP can auto-clamp mss so your mangle rules should not be required if you set this :slight_smile:

Here is a related topic http://forum.mikrotik.com/t/ipip-fragmentation-help/50227/1

Hope this helps !!

5

scampbell, It is not possible to transfer larger then MTU packets without fragmentation - I understand it. But I’d like fragmentation performed at ip-encap protocol level. It is possible judging by the fact that one tunnel formally pass packets of 1600 bytes. A similar mechanism exists in PPTP tunnels - you need only specify MRRU parameters for both side. ROS developers claim that this method is much less resource-intensive than the use of Mangle rules (change MSS). Probably so, but empirically it found out that such mechanism on PPTP tunnels mechanism works badly - reducing the rate of the channel at least 2 times, packet loss, etc…

As for IPIP tunnel which successfully transmits packets of 1600 bytes - it works perfectly. I can not understand why does not work the second tunnel in a similar way :confused:

I would like to hear from developers. Is there a such mechanism for IPIP tunnels? Or how to explain that one of the tunnels can formally transmit packets of 1600 bytes through physical interfaces, MTU one of them even lower than 1500.

6

At the moment I can only assume that this mechanism is still implemented, but does not work very well as planned. Perhaps at the time the negotiation of the tunnel both sides inform each other about phisical interfaces MTU but for some reason, one side reports the wrong data… Just my guess.

7

Have you checked your PPPoE tunnel can pass packets with MTU=1492 natively without fragmenting ?

Some ISP’s use VLAN’s and we have found the MTU=1480 is the best we can achieve in this case ?

8

I think what’s happening is that while the encasulated IP packet has DF bit set, the outer IPIP packet does not, therefore, the IPIP packet is being fragmented. (It’s like the magician’s saw the woman in half trick). Then the outer packets are reassembled, and the never-been-fragmented internal packet of 1600 comes out.

The reason it’s failing on the way to R2 is because R1 is fragmenting at the outer limit of 1500, not 1492, so even the fragmented IPIP outer packet is too large.

Try knocking 8 more bytes off of the IPIP interface between R2 and R1.