DarkNate isn’t wrong, and gives decent advice – if you want to set L2MTU to max that’s fine and leave headroom for whatever in future. But still don’t think it being higher is going to have any effect from the defaults values, unless you also change change the L3MTU higher too.
And default L2MTU in recent v7 does allow for full frame 1500 L3 MTU…as L2MTU is 1568 or higher for most ethernet things. At L2MTU = 1568, that’s enough for VXLAN over VLAN-enabled ethernet without any changes from defaults. No arithmetic math required, which seems to be goal?
DarkNate suggest same as everyone else:
So if you’re sure that none of the “internal network” with a higher L3MTU will traverse the internet, then sure set a higher L3MTU on internal, non-internet hosts. As @pe1chl notes jumbo frames have a use, but typically:
I’m not so sure about that. When this setting was only cosmetical it probably would not be there.
I guess this setting is used to allocate receive (and maybe transmit) buffers for the device driver, and setting it needlessly high will just use more memory and maybe be slower.
And of course it will bring no immediate benefit. You can just as well increase it when required.
Higher number on L2 doesn’t increase memory usage.
But if you set varying L2 MTU profiles, then it will affect the number of possible profiles loaded into memory. Each ASIC has limited amount of capacity for storing MTU profiles.
Hence, max it all out on all ports to create a single (or two) MTU profile-only, on the ASIC.
only practical to use on a very limited local network like SAN or NAS network between fixed hosts and storage, usually in a data center scenario.
This is clearly written by someone who’s an expert.
Jumbo frames benefits ISPs, Telecom, IXPs and carriers wherever possible, whoever supported. Enable 9000 L3 MTU and of course maxed L2 MTU.
Just remember to ask for jumbo frame VLAN from your IXP provider and also confirm with your upstream transit provider if they do jumbo frames and ask for the value.
The way I see it there are two (distinct) use cases where frame sizes larger than standard 1500 bytes is desirable:
transit and backbone networks
These typically involve adding some kind of overhead necessary for efficient traffic separation, such as VLAN, MPLS, GTP, PPPoE, etc overhead. If MTU in those networks is kept low, then L3 MTU may have to be decreased, leading to necessity to fragment packets. And as we all know, fragmentation causes both higher L3 overhead due to additional packet headers as well as higher CPU load on routers performing fragmentation, any firewalls beyond that point and receiver.
In order to avoid need for fragmentation in transit/backbone, there’s the famous PMTUD with ICMP Fragmentation Needed, meaning that sender effectively performs fragmentation, which then avoids CPU overhead in transit/backbone nodes but still causes increased L3 overhead (less than fragmentation in transit/backbone which in worst case doubles the overhead while reduction of MTU increases overhead by some percent depending on actual PMTU).
datacenter LANs
It’s important to remember why jumbo frames came along in data centers. It was a feature of FDDI networks (only later jumbo frames became optional normality in ethernet) and it was there to make processing overhead lower. Because those were the days of 286-class PCs and older (and much slower by today’s standards) mainframes whichndid not really have processing power for real-time processing of packet headers.
In short: to reduce PPS.
This problem is much smaller (or even non-existant) these days and benefits of having jumbo frames are much less. Today it’s fairly easy to saturate most (but fastest) interdace rates regardless of MTU used using average hardware. Which means that drawbacks of using jumbo frames can overshadow benefits.
It’s clear that @DarkNate is talking mainly about case #1 above while @pe1chl is talking about case #2 above. It’s not entirely clear which case covers OP’s use case, but if I have to guess I’d choose #2. Based on discussion from his side I doubt OP knows what he wants to achieve and probably he’s not aware of problems he’d get into with increasing the MTU (specially if that would be made a vendor standard which is what he’s proposing if I correctly understand initial post of this thread). Again: there are a few very good reasons to stick to industry standard settings and it’s clearly advanced topic to decide when to deviate from that standard. Yes, technical act of changing the value is indeed “beginner basic”.
I work with both ISPs/Telcom networking and DC networking.
Everywhere I go, intra-AS it’s all 9K MTU on L3 and maxed on L2 on each network devices. If Device A<>Device B is less than 9k, I simply configure the max L3 MTU for that particular interconnect. PMTUD takes care of the rest.
While not all customers/hosts are able to utilise 9K MTU, this allows future-proofing of the network for use such as VXLAN/EVPN encap/decap, and also customers can create WireGuard//IPSec tunnels between different end-points via the same backbone no problem with 8000+ MTU inside the tunnels.
The biggest problem with MTU/Jumbo frames config and deployment is lack of fundamentals understanding, concept and experience of doing jumbo frames everywhere you go.
The second problem is poor PMTUD config in the network backbone, it’s less common, but still fairly present out there.
And then we come back to the fact that blindly changing MTU, without understanding consequences in each particular case very well, can cause big problems…
[moderator]
removed big part of post
[/moderator]
Otherwise, simply quit tech, and move to arts and humanities, you don’t need fundamental understanding of MTU or BGP over there, your “feelings” are enough to get by.
One might say you strongly remind me of once upon a time a talented but (in)infamous Northern European network specialist who acctively took part to build the first commercial IP networks in Europe. At first he refused to accept dial-up internet but was later ditched due to customer demand.
He later disappeared to the Big network company and hated all other providers. Since you’re hanging around here I conclude it can’t be the same guy but unfortunately you have the same condescending attitude. However, his was due to ASD (autism spectrum disorder) and once you got to know him he was a pretty decent bloke.
I’m not him and never heard of this. Either way, I certainly don’t “hate”, hate is a strong word, and it requires energy to hate. You could say I have a strong dislike for stupidity – You can’t blame me for faulting stupidity.
And I don’t see the connection between him being against dial-up (at that time) and me. I prefer modern day networking approaches end-to-end. I ain’t about to sit here and say MPLS is king, where VXLAN/EVPN can do many of the things already that MPLS was once required such as L2 VPN/VRFs etc.
Thank you all for answering.
DarkNate, at least 50% of total “Thanking” goes to you.
mkx, I’m talking about a mostly wireless network.
I’m dealing with PPPoE over EoIP planning to use OSPF to add L3-bonuses into WLan.
Maybe use MPLS\VPLS but not sure about that.
The way I see it now - deal with MTU and set PPPoE MTU to 1500 instead of 1480 (Is that a worthy effort? )
You can’t go to 1500 inside pppoe, unless your ISP allows you to.
Regardless of the settings you can set on the device, if the provider doesn’t allow you an MTU of 1500, you can’t have it.
Sure, you can handwrite 1500, but still you’ll have problems because you don’t actually have a MTU of that value on the ISP side.
It’s not clear at all what you’re talking about and how your network is made, if you want advice for the inside, you’d better be more specific.
If the ultimate goal of networking is NOT to get out on the Internet, anything can be done
(and you can have specific suggestions if you let us understand how the network is made).
However, if the ultimate goal of the network is to get devices out to the Internet, the MTU must necessarily be 1500 or less, again depending on the ISP.
If you make PPPoE connections, through EoIP tunnels, you have to calculate all the necessary overhead, otherwise there is no real MTU of 1500.
For example, physical interface MTU 1500, EoIP MTU (without fragmentation) must be 1500 - 42 = 1458
PPPoE MTU (without fragmentation) must be 1458 - 8 = 1450.
Therefore, to have a link that does not generate more traffic than it needs, the MTU must be 1450.
Instead, for have a real MTU of 1500 from device vs. internet,
MTU on physical interface must be at least 1550 for have EoIP without fragmentation of 1550- 42 = 1508
and real PPPoE MTU of 1508 - 8 = 1500.
Layer 1 (physical medium) doesn’t matter, it’s still two distinct use cases. If you’re building your own (wireless) backbone network, then it’s case #1 from my post … and yes, you can increase L2 MTU as far as you want/need … given that used hardware lets you.
But that doesn’t mean that changing default MTU size in ROS is feasible. You’d always have problems. Default means that L3 interface on CPE, facing your backbone, would have MTU of 2k. And your backbone creates some overhead (EOIP, PPPoE, … you name it), so you’d still have to set up your backbone nodes with larger MTU to avoid fragmentation on your nodes. Since you seem to know what you’re doing, it’s sensible to expect that you’ll set up your backbone the way you want/need and leave MT defaults at values safe to use by everybody, even those who don’t know squat about networking.
So back to post #2 above …
I still don’t understand the use case for setting MTU to 2000.
1512, that I can understand. Or maybe 1600 as a “set and forget” case for all common encapsulations.
But I would not know any encapsulation protocol that has 500 bytes of overhead and requires MTU 2000 to transport the de-facto standard 1500 byte MTU.
Then there is MTU 9000 (jumbo frames), with the application I already described.
As others have written, you cannot have >1500 byte MTU on the internet traffic. That will not work.
Will 1512 be enough to set PPPoE MTU to 1500, send it through EoIP and provided by another network vlan (they have max-L2MTU=9000) to PPPoE-Server?
If PPPoE MTU=1500, haw can a larger L3 MTU of backbone network do harm of any kind?
I’ve got the point that L2 MTU can be set to maximum
EoIP can pass 1508 PPPoE (for have user’s LAN MTU 1500) frame FRAGMENTING it on two frames.
Approx example: for send one fragmented packet of 1508, must transmit two packet, one that transmort 1458, and the other that transport the remaining 50,
But for each packet are needed 42 bytes to be sended over EoIP… In total for send one fragmented packet of 1508, send two packet:
one of 1500 (42 + 1458 data) and another of 92 (42 + 50 of data), without count other 20 bytes for guard interval between two packets, and another 4 bytes for FCS.
So, for transmit 1508, fragmenting the packet, are needed ~1616 and more than the double of the time,
one high percentage than instead increase L2 MTU and EoIP MTU (without fragmentation)…
