Hey Guys,

I have looked at the wiki and the forum and i cant seem to find conclusive information as to what the MTU should be in order to run MPLS with VPLS tunnels over it.

It seems that VPLS requires a 1540 MTU in order to move 1500 byte packets.

here are my questions:

Will setting the MPLS MTU under MPLS Interface override the MTU settings under the general tab (winbox) for the individual interfaces?

We use the RB1000s to run all the routing and they will be the only devices that run routing, but we use RB333 and RB433 to create L2 links between these devices (n-streame-2 links with a bridge to ether1 that in return connect to the RB1000 on each side. see attachment) i know the nstreame2 links have a default MTU of 1600. The hardware of the 433AH can exceed the standard MTU by 22 bytes (ether1 ONLY), is this also true for the RB333?

Do I add the 22 extra bytes to 1518 or 1500? in other words can i set the MTU for ether1 on a 433AH to 1540? and is that what i need to do in order for VPLS links to accept 1500 byte packets without fragmentation.

Regards,

Martin Madsen

Will setting the MPLS MTU under MPLS Interface override the MTU settings under the general tab (winbox) for the individual interfaces?

They do not override each other - these are different settings and each is used for different purposes. In order to control the size of MPLS frame you only have to change MPLS MTU.

Do I add the 22 extra bytes to 1518 or 1500? in other words can i set the MTU for ether1 on a 433AH to 1540? and is that what i need to do in order for VPLS links to accept 1500 byte packets without fragmentation.

You add 22 to 1500. So you can set MTU (but you do not have to) or MPLS MTU on 433 to 1522 (assuming you do not use VLAN). In order to forward VPLS encapsulated frames without fragmentation, you need MPLS MTU at least 1526.

Hey,

Here is the followup, to make sure im getting it all:

The MTU for each interface is in effect when non-mpls packets are received on an interface, and the MPLS MTU is used for MPLS packets, correct?

Since im not running MPLS on the equipment that makes the wireless links between the RB1000's (the rb433ah boards), how will they treat frames that are bigger than the MTU hardcoded under the tab for ether1 (1500) will they attemt to fragment, even though they do not understand the content of the packet?

Since they work as bridges i assume they invoke the MTU for the interface. Or will they simply drop any packet that exceeds the MTU of 1500 since the "payload" beyond the L2 header is MPLS ?

Since the boards i use for bridgeing the RB1000's together, can only achive a 1522 MTU and i really need 1526 in order to avoid any fragmentation, should i just stick with the RB1000's @ MPLS MTU=1508 and the up the MTU of the 433ah boards to 1522 on ether1? (to allow for future VPN use)

Is that the best solution?

Regards,

Martin Madsen

The MTU for each interface is in effect when non-mpls packets are received on an interface, and the MPLS MTU is used for MPLS packets, correct?

The MTU is actually about transmitting packets over interface. "/interface" MTU is for layer 3 (IP) packets - IP protocol will not send IP packets larger than this MTU for transmission over interface - it will either fragment or drop. "/mpls interface" MTU is for MPLS packets - MPLS will not send packets (including MPLS labels) larger than this. And that's all about MTUs - e.g. you do not have to set MTU for ethernet interface to 1508 in order to add "chained" VLAN interfaces to send QinQ frames over ethernet interface.

Since im not running MPLS on the equipment that makes the wireless links between the RB1000's (the rb433ah boards), how will they treat frames that are bigger than the MTU hardcoded under the tab for ether1 (1500) will they attemt to fragment, even though they do not understand the content of the packet?

Bridge will not fragment packets.

Since they work as bridges i assume they invoke the MTU for the interface. Or will they simply drop any packet that exceeds the MTU of 1500 since the "payload" beyond the L2 header is MPLS ?

Well, the process of bridging currently is somewhat inconsistent with overall "MTU policy" - when bridging (which is action happening at layer 2 and therefore should have nothing to do with layer 3 MTU), bridge actually checks that packet (minus link layer header) does not exceed "/interface" MTU, so you are right. As to bridged interfaces - currently you have to set their "/interface" MTU to value to accomodate any VLAN tags or MPLS labels. In your case this means that "/interface" MTU for bridged interfaces must be set to the same value as "/mpls interface" MTU on your MPLS routers.

Since the boards i use for bridgeing the RB1000's together, can only achive a 1522 MTU and i really need 1526 in order to avoid any fragmentation, should i just stick with the RB1000's @ MPLS MTU=1508 and the up the MTU of the 433ah boards to 1522 on ether1? (to allow for future VPN use)

Is that the best solution?

As long as you configure "/interface" MTU on bridged interfaces to be the same or bigger than MPLS MTU on MPLS routers, you should be OK.

Thanks, that was very informative. Appriciate all the time you spend on this.


Regards,

Martin Madsen
Bel Air Internet

Hey,

I have set the MTU of the bridged boards (rb433) to 1522 on both ether1 and Nsteme2 on both sides. No problems, i can ping and do speed test, everything works.

The minute i change the MTU on the RB1000's, i can ping (1500 byte packet) but no speedtest (the speedtest authenticates, but then stays at 0bps), ospf sees its neighboer over the link, but no routes are exchanged.

I bring the /interface MTU on the RB1000's down to 1500 again, everything starts to work again.

I have not enabled MPLS yet, i need to figure out why this is happening first.

Any suggestions?

Regards,

Martin Madsen
Bel Air Internet

Hey,

Tried to enable MPLS on a backup link, with all /interface MTU=1500 (have to do it with that MTU otherwise OSPF does not route over the link) and i can only get packets that are 1496 bytes or less through. Makes sense since simple MPLS has a 4 byte label. That means that winbox cannot retrieve preferances on units behind this link.

I tried to set all /interface MTU to 1522 (like in the last post) and this time i tried to change the /bridge MTU on the rb433's as well, i was not able to, it would ONLY accept 1500. Could that be the issue that the bridge will only operate with MTU 1500?

I then dropped the /MPLS MTU to 1500 from the default 1508. (so it is inline with the /interface MTU across the link) now i could login with winbox no problem, but traffic dropped significantly, and customers cant get to pages.

This is maybe all irrelevant, since it seems that the real problem is that the bridge interface will not change MTU. Is this a correct observation?
Seems to have some support in this post: http://forum.mikrotik.com/viewtopic.php ... bridge+mtu

I have to run a bridge, if i used all routers i would exceed 150+ OSPF/MPLS enabled routers, and im sure i would have problems with resources on the smaller boards. We are trying to only have RB1000's do routing.

Can I please get support for "bridge MTU" > 1500? If that is really the issue.

Regards,

Martin Madsen
Bel Air Internet

You can expect multiple MTU related improvements and fixes in one of future releases. Bridge interface MTU applies only to packets that are sent over bridge interface (routed over it). Normally you do not want to change that. All that currently affects max size of bridged packets is /interface MTU of bridge ports.

Hey,

All i can seem to figure out is this:

Setting all bridged /interface mtu=1522 and RB1000's /interface mtu=1500: ospf=ok, mpls=not ok

Setting all bridged /interface mtu=1522 and RB1000's /interface mtu=1522: ospf=not ok, mpls=not ok because of IGP state (assumed).

If you are telling me there should be no problems, then ill keep trying to figure out what im doing wrong, but at this point it looks like passing an MTU bigger than 1500 does not work through a routerboard where the interfaces are bridged.

EDIT: I setup a test where the bridge consisted of 2 ethernet ports (no nsteame2 in the bridge), that works and can move frames up to the MTU limit of the interfaces in the bridge.
It seems that when a Nsteme2 and an fastethernet interface is bridged, frame limit is 1500 period. Can you confirm this please?

Regards,

Martin Madsen
Bel Air Internet

Yes, I can confirm this - this is a problem in nstreme2 interface MTU handling. Going to be fixed in one of the next releases.

After applying v3.25 everything now works. Everything bridges across as it should.

Thanks for the fix,

Martin Madsen
Bel Air Internet

=/ I applied 3.25 to all my bridges and routers and am still having the same issues as this guy. Is 3.25 the future release you were referring to, or do I need to wait? Thank you!

I'm getting a bit confused here. Please help me sort it out:

I will be running my MPLS network with jumbo frames (no wireless), so
1: /interface set ether1 l2mtu=9216
This should correspond to the Cisco on the other end: 2: /mpls interface set all mtu=9000
This gives me an MPLS mtu of 9000.
3: There is a remaining MTU on /interface that remains 1500. What do I set that to? Is that the IP MTU, meaning MPLS mtu - MPLS header or what?

I'm still a bit confused. The cisco mtu might not be the raw layer2 mtu, and if it's not, I need to bump up the mikrotik l2mtu. Can anyone clarify?

First of all - you must make sure that your hardware does support changing l2mtu and that it supports the value you intend to set.

'/interface mtu' is IP MTU. It affects IP routed packets. If you want to route jumbo frames at MPLS ingress router, then you should increase it (because packet initially gets IP routed and only then MPLS encapsulated). If you plan to bridge (e.g. do VPLS) then you do not have to change IP MTU (because when bridge is forwarding frames, it looks at l2mtu).

I have the same customer need and the network diagramm the same as the one at the start of the thread.
The only difference are some additional switches between RB1000 and RB433AH.

After working a while I think I will not be able to transfer the customer VLAN with tag over the VPLS-tunnel because the L2MTU of RB433AH (in MPLS router mode) is to low. Is that right?

Is there another way to transport a VLAN trunk with IP-traffic inside between the edge routers (if it is beccessary with fragementation)?
What does the "User Service Tag" in VLAN configuration to?
IS Q-in-Q a solution?

After working a while I think I will not be able to transfer the customer VLAN with tag over the VPLS-tunnel because the L2MTU of RB433AH (in MPLS router mode) is to low. Is that right?

Yes. L2MTU of 433 is not big enough to be able to forward VPLS frames w/o fragmentation.

Is there another way to transport a VLAN trunk with IP-traffic inside between the edge routers (if it is beccessary with fragementation)?

RouterOS VPLS supports frame fragmentation, so you can still use VPLS.

What does the "User Service Tag" in VLAN configuration to?

It changes what ethernet protocol value is used for VLAN encapsulation, nothing more.

IS Q-in-Q a solution?

Probably, but then your network core must be bridged, instead of routed.

Thank you for your reply.

When I configure the VPLS and I make a bridge with the vpls-tunnel and the ethernet-interface no customer vlan will travel through. Untaged packets are comming through.

I want to push it all to the tunnel. What I have to regard for?

Please post relevant parts of edge router configuration - bridge, vpls, vlan. Make sure that l2mtu of VPLS interface supports sending packet with VLAN tag.