VPLS L2MTU / PW-MTU calculation seems wrong

RouterOS Forwarding Protocols
1

Hi all,

Background: I’ve setup a lab to test transmission of non-fragmented 1500 byte IP packets over PPPoE over VPLS. I have this working well. I then progressed to test the maximum packet size I could transmit through VPLS given the L2MTU of the PE’s ethernet interface is 1600.

I’m now successfully passing 1570 byte IP packets in a VPLS tunnel which results in a 1614 byte ethernet frame on the wire (verified with Wireshark) - perfect. (1570 IP + 14 ETH + 4 VPLS + 8 MPLS + 14 ETH + 4 VLAN = 1614).

I wanted to verify my VPLS L2MTU values as I wasn’t too sure of their calculation. Through experimentation I found this setup continues to work (non-fragmented) with VPLS L2MTU values as low as 1556. Below that I get packet loss.

So now I’m confused. How is the VPLS L2MTU calculated. By all accounts I can bridge non-fragmented 1570 byte IP packets (1584 ethernet frames) over a VPLS interface with L2MTU of 1556.

FWIW, my setup is:

Ether L2MTU = 1600
Ether.VLAN L2MTU = 1596
MPLS MTU = 1596
VPLS L2MTU = Anything from 1556 upwards!

This is with ROS v6.22.

Any thoughts on how this works?

Regards,
Richard

2

The packet is transparently fragmented and re-constructed by the VPLS interface driver in RouterOS.

If you set a do-not-fragment flag in ICMP, that only applies to routing (L3) logic.
MPLS and VPLS are L2.5, they do NOT care about a ICMP do-not-fragment flag, and will happily fragment the frame anyway.

Also, if using PPPoE, be careful of the MRRU value, which can transmit bigger packets then MTU/MRU allow as well.
Disable MRRU completly when testing MTU with PPPoE.

3

Hi tomaskir.

That’s what I thought but the wireahark trace says otherwise. I see 1614 bytes on the wire and no fragmentation. Even with the transparent fragmentation I can’t see how the calculation for 1556 l2mtu works out for my setup.

Now if I decrease the mpls MTU below 1596 then yes i do see the vpls “transparent” fragmentation.

BTW thanks for the fantastic presentation on the same topic. Was very helpful.

Rich

4

Where and how are you capturing the packets?

Can you draw a diagram of your setup, including where the packet capture device is?

Also please include /exports :slight_smile:

5

I’ll post the diagram and exports tomorrow as not by a PC. But the capture taken from a rb acting as an Ethernet bridge between the two pe’s (no mpls just bridging two Ethernet ports).

6

Here’s a diagram of the setup.

MPLS nodes are POP-AP, CORE-AP and CORE-PE.

POP BACKHAUL is just an ethernet bridge and used to capture packets

Orange line is the VPLS. Blue line is PPPoE between CPE RADIO and CORE-PPPoE
pppoeovpls.png
The config from POP-AP is below. In this config I’ve left VPLS pw-mtu as 1556.

/interface bridge
add name=br-lo
add name=br-ppp
add name=br-wds
/interface wireless
set [ find default-name=wlan1 ] band=5ghz-onlyn channel-width=20/40mhz-Ce \
    country="new zealand" disabled=no frequency=5200 l2mtu=1600 mode=\
    ap-bridge ssid=Pop tx-power=0 tx-power-mode=all-rates-fixed \
    wds-default-bridge=br-wds wds-mode=dynamic wireless-protocol=nv2
/interface vlan
add interface=ether1 l2mtu=1596 name=ether1.101 vlan-id=101
add interface=br-wds l2mtu=1596 name=wlan1.11 vlan-id=11
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/routing bgp instance
set default router-id=10.255.1.1
/interface bridge port
add bridge=br-ppp interface=wlan1.11
/interface vpls bgp-vpls
add bridge=br-ppp bridge-cost=0 export-route-targets=1:1 \
    import-route-targets=1:1 name=CORE-PE pw-mtu=1556 route-distinguisher=1:1 \
    site-id=2
/ip address
add address=10.1.1.1/29 interface=ether1.101 network=10.1.1.0
add address=10.255.1.1/32 interface=br-lo network=10.255.1.1
/mpls interface
set [ find default=yes ] mpls-mtu=1596
/mpls ldp
set enabled=yes loop-detect=yes lsr-id=10.255.1.1 transport-address=\
    10.255.1.1 use-explicit-null=yes
/mpls ldp interface
add interface=ether1.101
/routing bgp peer
add address-families=l2vpn name=peer1 remote-address=10.255.1.3 remote-as=\
    65530 ttl=default
/routing ospf interface
add network-type=broadcast passive=yes
add interface=ether1.101 network-type=broadcast
/routing ospf network
add area=backbone

To test, I added IP addresses 5.5.5.1 and 5.5.5.2 to CPE-RADIO and CORE-PPPoE devices and ran a 1570 byte ping. Packets were captured on wlan1 interface of the POP-BACKHAUL device and output pcap is attached. You can see the 1614 byte ethernet frames being transmitted as a whole.

Regards,
Richard
2.pcap.zip (35 KB)

7

Check it out, there presentation about MPLS/VPLS.
It helps me to find out how the fragmentation happen.

https://mum.mikrotik.com/presentations/EU18/presentation_5162_1523282068.pdf