I’m very new (3 months) to Network administration and Mikrotik (1 week), but would like to learn more and help our business to get a more secure network. I found the following http://wiki.mikrotik.com/wiki/Maximum_Transmission_Unit_on_RouterBoards#MAC.2FLayer-2.2FL2_MTU, but still confuse on some problems I have with packet size. We are using Oracle for our warehouse app and since we build the IPSec Tunnel between our RB1100 (running version 4.10) Some offices work some other don’t and make the SQL Net application hang. By changing the Oracle TDU/SDU packet size, we kind of solve most of the SQLNet trouble, but are having other issue with other type of network issues. After two full day of work trying to diagnose the problem, I decided to post a message to get some feedback on how to resolve my problem.
here is more info on my ipsec tunnel
Win XP 1 RB1100-1 LAN / WAN RB1100-2 PPPoe / LAN WIN XP
192.168.0.10 > 192.168.0.1 / 1.1.1.1 > 2.2.2.2 / 192.168.1.1 > 192.168.1.10
MTU 1500 MTU1500 / MTU1500 MTU 1480 / MTU1500 MTU 1500
First I ran on both side of the tunnel some ping -l -f to find the maximum MTU that was going to repond without frag and without modifying my default MTU value. Also some reference are mentionning adding 8 to this value to get the optimal MTU, for the ICMP header, should I?. Those ping were made from the two windows XP machine at both ends.
Once I get those value I wasn’t sure where to put them. Especially on my RB1100-2, that has a ADSL MTU, and also a MTU for my WAN interface. After many hours of testing and not sure if I was doing te right thing but modifying the LAN interface MTU gave me the best result. I really tought first that I should modify RB1100-1 WAN interface and RB1100-2 PPPoE Interface MTU, not the LAN Interface. Modifying the WAN interface and PPPoE, just made those max size decrease and I constantly need to decrease the max size of the packet size in my ping command.
I also find it strange using the PPPoE MTU of 1480 instead of 1492 I saw often on other router, can someone explain to me the difference?
Some article I read also said that I should play with the mss. I think I do understand that mss if the MTU without the header. Should I change mss instead of MTU?
I know IPSec as variable size header, should I proviosionned extra space by lowering my MTU value?
I’m very confuse on how to log and debug all this using the mikrotik logging feature and will much appreciate people help on how you solve MTU issue.
Generally speaking assigning MTU on the WAN interface is best practice so that a reduced MTU isn’t applied to traffic flowing through LAN interfaces on the same router. Apply the lower MTU at the actual choke point interface - for you that appears to be the WAN.
IPSec doesn’t really have a variable header size (and thus variable MTU), it just has several protocol options that influence header size such as transport mode vs tunnel mode. Verify what options you are using and add them up. http://www.networkstuff.eu/index.php/IPsec_Bandwidth_Overhead_Using_AES is an OK reference.
ICMP does add an additional 8 bytes.
MSS is a TCP thing. It refers to the payload inside a TCP segment, which in turn with the TCP header is the payload in an IP packet. So MSS is essentially the TCP MTU. TCP headers are variable as TCP headers can have between 0 and 40 bytes for options. To ensure that there is no segment fragmentation assume the biggest possible TCP header. http://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_segment_structure has all the numbers to add up.
Almost everything (including XP) sets the IP ‘dont fragment’ flag these days.
ICMP PMTU (Path MTU) discovery is blocked in so many places that it is also basically redundant.
MTU refers to the IP layer of the connection. It is the biggest packet that IP will allow.
Inside the IP packet is the TCP packet, the UDP packet, the ICMP packet etc.
MSS is the Maximum Segment Size, and refers to the TCP packet.
strange using the PPPoE MTU of 1480 instead of 1492
PPP adds (maybe 7E) then FF 03 to the packet, then a 2-byte checksum and another 7E, so 5~6 bytes. I suspect the ‘oE’ part of PPPoE adds some more, so 1480 is a ‘safe’ value.
The only reliable way i have found is to try pinging with 1500 bytes, see if it fails.
If so, go to 1499 and keep trying --1 until it gets through (and back).
Then you have an MTU you can use. Try not to confuse that with L2MTU etc.
Be aware that some stupid servers (or possibly their routers) out there do not know how to do MTU, so if you drop it too low, some Big websites will not work properly, if at all.
Don’t be too confused.
The Internet is basically broken, so it doesn’t always behave like you expect.
Once I find out maximum ping packet size with DF bit set, should I really add 20 IP haeder bytes and 8 ICMP header bytes?
So for example, MD5 and 3DES, which I think is the default IPSec mode in Mikrotik, will give 1564 packet size. So if I have a 1472 maximum with DF I will then set my WAN MTU on the side I made my ping to 1408?
On my other side, I’m using PPPoE, should I lowered this value to 1388 because of the overhead cause by the PPPoE wrapper. Again, I will be tempted to modify the MTU value of my PPPoE interface and not the WAN, can someone confirm this is the case.
Is there a way in log to see when fragmentation occur ? And most important when packet are dropped because of DF bit set and packet size bigger than MTU.
The only reliable way i have found is to try pinging with 1500 bytes, see if it fails.
If so, go to 1499 and keep trying --1 until it gets through (and back).
Great will try this out, and I read that when traffic goes over ATM MTU should be a multiple of 53 or 48 don’t remember. But anyway optimal is less concerned when it works without problem.
Can someone please confirm
that using PPPoE, I still need to change the WAN Interface MTU and not he PPPoE interface MTU which I use to do.
Logging info about fragmentation in Mikrotik such as debug ip packet detail in cisco
