I have a CCR running OSPF, and run OSPF links at 1600 MTU. I noticed the Loopback MTU is 1500. Does an MTU have any meaning on the LOOPBACK interface?
19 R sfp28-11-123main-fiber ether 1600 1700 9570 48:A9:8A:18:83:C1
20 R sfp28-12-123cal-radio ether 1600 1700 9570 48:A9:8A:18:83:C2
21 R LOOPBACK bridge 1500 65535 BE:14:C4:01:66:3B
22 R bridge bridge 1600 1700 48:A9:8A:18:83:C3
23 R gre0 gre-tunnel 1476 65535
24 R vlan363 vlan 1600 1696 48:A9:8A:18:83:C3
25 R vlan596 vlan 1600 1696 48:A9:8A:18:83:C3
I ran a quick test and set the df-bit on pings to the router. At 1500 and 1600 I could still ping the LOOPBACK ip, (sanity check, at 1601 packet size, the packet did not make it there, as expected).
Yes, it matters. I mean, wireless paths, technically, can do 9k MTU for layer 2 if the vendor supports, like some units from Ubiquiti.
But the point is, layer 2 MTU should always be MAXED out on ALL Devices, even if it’s different between them. Layer 3 MTU needs to be designed in a way that it ensures:
PMTUD works correctly in ALL paths and directions (MPLS TE? LDP? OSPF? BGP? GRE? WireGuard? Bridge, VLAN, VPLS? Etc)
L3 MTU should be 9k in the backbone on the physical ports, but you cap it wherever required on the layer 3 sub interface, such as LACP bonding to your upstream devices or the L3 VLAN interface towards another device etc
MTU design and training, is often something I don’t see very often in network training programmes and certs. MTU itself is a piece of cake, however, when your network has all types of devices with varying MTUs, proper planning is required. Don’t justify to yourself with this “1600” cap, do 9k MTU on the PHYs everywhere when possible to ensure future overhead room for anything.
My loopbacks are all 9k L3 MTU, most of my paths are 9k L3 MTU back-to-back, but some paths are 1500, some paths like wireless are 1500 on layer 3, but 1600 on layer 2 on the radios, but 9k L2 and L3 MTU on the transport ports on my PE router and P for example. Nothing breaks, because layer 3 MTU matches correctly between each termination L3, so PMTUD does it’s job. No packet fragmentation or VPLS Control word bullshit nor frame drops nor MTU mismatch.
MTU is a complex topic, that unfortunately is spread among many books and documentations.
The simplified version of it is:
Larger MTU = larger frames = larger IP datagram = larger pay load = fewer frames/packets required to complete the payload from start to end = less PPS required to transfer data from A to B = overall less utilisation of the path
I’m not sure why MTU deep-dive is not part of CCNA, JNCIA, MTCNA, Network+ or even in school/college.
I think MTU deep dive is covered only on CCDE and CCIE course work.