Community discussions

MikroTik App
 
mducharme
Trainer
Trainer
Topic Author
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

MPLS/VPLS and HTB / EXP bits

Wed Aug 17, 2016 6:47 pm

We have a chain of Mikrotiks in a test lab, with VPLS and MPLS-TE tunnels.

MKT1 <--> MKT2 <--> MKT3 <--> MKT4

We have a TE tunnel and VPLS tunnel connecting MKT3 to MKT1, and the same connecting MKT4 to MKT1. On MKT4, we are using a bridge filter to set the priority for all packets entering the tunnel to MKT1 as priority 7. We have no filter on MKT3 since they should be experimental bits at that point, and the tunnel from MKT3 to MKT1 should not be prioritized.

To simulate congestion, on MKT3's interface that connects to MKT2, we implemented a queue tree with a 20Mbps limit, matching no-mark. It matches all packets and shapes to 20Mbps. However, it seems to ignore the MPLS EXP bits set by MKT4 and gives the traffic the same priority as the MKT3 traffic.

NOTE: To test this, we have a UDP packet generator on MKT4 sending to an IP of a computer plugged into the VPLS tunnel endpoint at MKT1. We have a similar UDP packet generator on MKT3 sending to the same computer. The rate of the UDP packet generator on MKT4 is the bare minimum packets per second required to create a 5Mbps rate (the guaranteed rate for the customer), but have UDP packets going through MKT3 with a much higher rate (30 or 40 Mbps). We can see the rate that packets are arriving at the tunnel endpoint at MKT1. When we start the traffic generator on MKT3, the traffic from MKT4 drops down from 5Mbps to 4.2. Disabling the traffic generator causes the MKT4 tunnel traffic to return to the customer guaranteed 5Mbps. If we try disabling and enabling the bridge filter on MKT4 that applies the priority to the packets, it has no impact on the 4.2 value, which suggests that the MPLS experimental bits are either not being set or not being honored.

I'm not sure whether this is caused by the HTB not recognizing that the EXP bits should have priority, or whether the EXP bits are not getting applied properly from the beginning.

Any ideas?
 
leviton
just joined
Posts: 2
Joined: Wed Aug 17, 2016 8:58 pm

Re: MPLS/VPLS and HTB / EXP bits

Wed Aug 17, 2016 9:00 pm

This is good topic! I like it!
 
mducharme
Trainer
Trainer
Topic Author
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: MPLS/VPLS and HTB / EXP bits

Fri Aug 19, 2016 9:28 am

I believe I have found a successful method of marking mpls frames which works in the queue tree.

On MKT3's interface that receives data from MKT4 I created a bridge. I made that bridge the ldp and mpls TE interface. I applied a bridge filter on that router that takes packets with ingress priority 7 and applies a mark. That mark then works on the queue tree on the port going to mkt2 and seems to work. I will do more testing tomorrow.
 
mducharme
Trainer
Trainer
Topic Author
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: MPLS/VPLS and HTB / EXP bits

Sun Aug 21, 2016 9:51 pm

Yes, I have confirmed that it works. This provides a method for QoS with MPLS/VPLS, and allows marking of MPLS frames for use in queue trees. The last post was slightly incorrect. It is actually quite simple:

- Both VPLS tunnel endpoints should be connected to bridges
- A bridge filter should be set up on the 'forward' chain, to 'set priority' to a value 0-7 that you want the MPLS experimental bits to be set to.

On other routers where you have congested links (ex wireless), set up a queue tree on either side of the link that shapes to the link's maximum. Then, do the following to properly mark the MPLS packets for use in your queue tree:
- At the egress interface that has your queue tree set up, create a bridge that includes only that interface
- Add a bridge filter for chain 'output' that matches 'ingress priority' equal to the value that you had assigned on the endpoints (i.e. the experimental bit values)
- Set the action to 'mark packet'

The queue tree will now properly classify the MPLS packet in the queue and properly do QoS.
 
User avatar
shaoranrch
Member Candidate
Member Candidate
Posts: 184
Joined: Thu Feb 13, 2014 8:03 pm

Re: RE: Re: MPLS/VPLS and HTB / EXP bits

Sun Aug 21, 2016 11:02 pm

Yes, I have confirmed that it works. This provides a method for QoS with MPLS/VPLS, and allows marking of MPLS frames for use in queue trees. The last post was slightly incorrect. It is actually quite simple:

- Both VPLS tunnel endpoints should be connected to bridges
- A bridge filter should be set up on the 'forward' chain, to 'set priority' to a value 0-7 that you want the MPLS experimental bits to be set to.

On other routers where you have congested links (ex wireless), set up a queue tree on either side of the link that shapes to the link's maximum. Then, do the following to properly mark the MPLS packets for use in your queue tree:
- At the egress interface that has your queue tree set up, create a bridge that includes only that interface
- Add a bridge filter for chain 'output' that matches 'ingress priority' equal to the value that you had assigned on the endpoints (i.e. the experimental bit values)
- Set the action to 'mark packet'

The queue tree will now properly classify the MPLS packet in the queue and properly do QoS.
Hi, really interesting way of doing things. However I think this uses a bridge and adds processing overhead, how's the performance you have noticed using this method?

Hopefully someday MikroTik will allows us to process "natively" MPLS trafic with QoS

Enviado desde mi MotoE2(4G-LTE) mediante Tapatalk
 
mducharme
Trainer
Trainer
Topic Author
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: RE: Re: MPLS/VPLS and HTB / EXP bits

Mon Aug 22, 2016 12:37 am

Hi, really interesting way of doing things. However I think this uses a bridge and adds processing overhead, how's the performance you have noticed using this method?

Hopefully someday MikroTik will allows us to process "natively" MPLS trafic with QoS
With 3 tunnels totaling 20Mbps and extensive queueing we are only seeing ~1% CPU usage on Routerboard 1100AHX2's in our lab. I'm sure the impact will be bigger on higher bandwidth links, but well within reason.
 
oscarif
just joined
Posts: 1
Joined: Wed Sep 12, 2018 6:42 pm

Re: MPLS/VPLS and HTB / EXP bits

Wed Sep 12, 2018 7:01 pm

Good morning Partners,

I have a test lab with 3 RB MIKROTIK with OSPF / MPLS - VPLS structure to connect a transparent LAN network but I can not apply to limit the MPLS-VPLS bandwidth. Could you help me.
 
mducharme
Trainer
Trainer
Topic Author
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: MPLS/VPLS and HTB / EXP bits

Wed Sep 19, 2018 8:42 pm

Good morning Partners,

I have a test lab with 3 RB MIKROTIK with OSPF / MPLS - VPLS structure to connect a transparent LAN network but I can not apply to limit the MPLS-VPLS bandwidth. Could you help me.
To limit the VPLS bandwidth, all you need to do is create a queue tree on both ends of the VPLS tunnel, set to match no-mark, with the limit you want, and the parent set to the VPLS interface. The upload limit will be on one side and the download limit on the other.
 
mikeeg02
Member Candidate
Member Candidate
Posts: 162
Joined: Fri Mar 30, 2018 2:28 am
Location: Pennsylvania

Re: MPLS/VPLS and HTB / EXP bits

Sat Oct 17, 2020 7:16 am

Bringing back an old thread.

I have tested and accomplished what you have shown above, in the middle routers.

Lets say you wanted to accomplish this on the link between mtk1 and mtk2..... mtk1 who is a vpls tunnel end point. How do you queue the mpls out frames from there and include the traffic from the source router (mtk1) (Aside from using the vpls tunnel interface itself). In the scenario I need to implement, I would have an additional link from mtk1 to mtk4 forming a ring, and the vpls tunnel could go out either interface to get to its tunnel end point.
 
mikeeg02
Member Candidate
Member Candidate
Posts: 162
Joined: Fri Mar 30, 2018 2:28 am
Location: Pennsylvania

Re: MPLS/VPLS and HTB / EXP bits

Mon Nov 09, 2020 11:17 pm

Bringing back an old thread.

I have tested and accomplished what you have shown above, in the middle routers.

Lets say you wanted to accomplish this on the link between mtk1 and mtk2..... mtk1 who is a vpls tunnel end point. How do you queue the mpls out frames from there and include the traffic from the source router (mtk1) (Aside from using the vpls tunnel interface itself). In the scenario I need to implement, I would have an additional link from mtk1 to mtk4 forming a ring, and the vpls tunnel could go out either interface to get to its tunnel end point.
Is this a silly question? I cant seem to get the packets into a HTB queue on the mpls-out interface that are from the start of the tunnel. Though I can get the passthrough mpls traffic on the out interface, like described above. Help?
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: MPLS/VPLS and HTB / EXP bits

Tue Nov 10, 2020 4:12 pm

Can you post details of the config?
 
mikeeg02
Member Candidate
Member Candidate
Posts: 162
Joined: Fri Mar 30, 2018 2:28 am
Location: Pennsylvania

Re: MPLS/VPLS and HTB / EXP bits

Wed Nov 11, 2020 4:04 am

Can you post details of the config?
Heres a dev router. Effectively everything that comes in ether5 is being packet marked and priority set before it goes into the vpls tunnel that starts here. If this router was in the middle, and using the bridge method described above, I can queue the mpls packets based on priority of mpls packets going through the router. But I cant seem to queue the packets that originate from the router. In the code below, I've tried both using ip firewall and bridge filters to mark. Neither shows up in the ether1 queue, which is the ldp interface to the next router. How can I get the packets that get put into the vpls tunnel at this router, into my output queue? If that makes sense.

/interface ethernet
set [ find default-name=ether1 ] advertise=100M-full,1000M-full l2mtu=1584 \
    name="ether1 to top3011"
set [ find default-name=ether2 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether3 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether4 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether5 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether6 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether7 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether8 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether9 ] l2mtu=1514 speed=100Mbps
set [ find default-name=ether10 ] l2mtu=1514 name="ether10 - admin"
set [ find default-name=sfp1 ] l2mtu=1514

/interface bridge
add name="bridge ether1"
add fast-forward=no name="bridge vpls" protocol-mode=none
add fast-forward=no name=bridge_Loopb protocol-mode=none

/interface vpls
add disabled=no l2mtu=1500 mac-address=02:F9:0F:DD:10:BA name="vpls to 2" \
    remote-peer=10.0.0.2 vpls-id=1:1

/queue tree
add max-limit=10M name=ether1_queue parent="ether1 to 750"
add name=ether1_prio2 packet-mark=prio2_mpls_out_ether1 parent=ether1_queue
add name=ether1_prio1 packet-mark=prio1_mpls_out_ether1 parent=ether1_queue
add name=ether1_prio3 packet-mark=prio3_mpls_out_ether1 parent=ether1_queue

/routing ospf instance
set [ find default=yes ] router-id=10.0.0.1

/interface bridge filter
add action=mark-packet chain=forward in-interface=ether5 new-packet-mark=\
    prio2_mpls_out_ether1
add action=set-priority chain=forward in-interface=ether5 new-priority=2 \
    passthrough=no
add action=mark-packet chain=output ingress-priority=1 new-packet-mark=\
    prio1_mpls_out_ether1 out-bridge="bridge ether1"
add action=mark-packet chain=output ingress-priority=2 new-packet-mark=\
    prio2_mpls_out_ether1 out-bridge="bridge ether1"
add action=mark-packet chain=output ingress-priority=3 new-packet-mark=\
    prio3_mpls_out_ether1 out-bridge="bridge ether1"

/interface bridge port
add bridge="bridge ether1" hw=no interface="ether1 to 750"
add bridge="bridge vpls" hw=no interface=ether5
add bridge="bridge vpls" interface="vpls to 2"

/interface bridge settings
set allow-fast-path=no use-ip-firewall=yes

/ip address
add address=10.0.0.1 interface=bridge_Loopb network=10.0.0.1
add address=172.11.0.9/29 interface="ether1 to 750" network=172.11.0.8
add address=10.6.0.2/24 interface="ether10 - admin" network=10.6.0.0

/ip firewall mangle
add action=mark-packet chain=postrouting new-packet-mark=\
    prio2_mpls_out_ether1 out-bridge-port="vpls to 2" passthrough=no

/mpls interface
set [ find default=yes ] mpls-mtu=1540

/mpls ldp
set enabled=yes hop-limit=100 lsr-id=10.0.0.1 path-vector-limit=100 transport-address=10.0.0.1 use-explicit-null=yes

/mpls ldp interface
add interface="ether1 to 750" transport-address=10.0.0.1

/routing bfd interface
set [ find default=yes ] interval=0.1s min-rx=0.1s multiplier=2

/routing ospf interface
add interface=bridge_Loopb network-type=broadcast passive=yes
add dead-interval=10s hello-interval=5s interface="bridge ether1" \
    network-type=point-to-point use-bfd=yes
add dead-interval=10s hello-interval=5s interface="ether10 - admin" \
    network-type=broadcast passive=yes

/routing ospf network
add area=backbone network=172.11.0.0/24
add area=backbone network=10.0.0.1/32
add area=backbone network=10.6.0.0/24

/system identity
set name="Bottom 3011"
 
mducharme
Trainer
Trainer
Topic Author
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: MPLS/VPLS and HTB / EXP bits

Mon Nov 23, 2020 9:14 pm

But I cant seem to queue the packets that originate from the router.
Yes, this is not possible, unfortunately. The issue is that ingress-priority is only set automatically when the packet first arrives at the router, and you can only match ingress-priority in bridge filter rules (not priority). Any packets that are created by the router (in this case, the VPLS packet is created by the router) is ingress-priority 0 and this cannot be changed. So the originating router will treat the packet as best effort on egress, and it will be queued as best effort no-mark traffic (ingress-priority=0, the default) instead of the desired priority.

The way we have worked around this is by always splitting our P and PE router roles, which sometimes means we have to use two routers where we otherwise would just use one. The PE device and P device would have a high speed dedicated link between them (wired or wireless) so that the lack of QoS on that first PE-P hop isn't a big issue.

The only place we have routers doing double duty as both P and PE devices is for retail customer traffic, which is all best effort and so the lack of QoS is not problematic there.
 
mikeeg02
Member Candidate
Member Candidate
Posts: 162
Joined: Fri Mar 30, 2018 2:28 am
Location: Pennsylvania

Re: MPLS/VPLS and HTB / EXP bits

Mon Nov 23, 2020 9:27 pm

But I cant seem to queue the packets that originate from the router.
Yes, this is not possible, unfortunately. The issue is that ingress-priority is only set automatically when the packet first arrives at the router, and you can only match ingress-priority in bridge filter rules (not priority). Any packets that are created by the router (in this case, the VPLS packet is created by the router) is ingress-priority 0 and this cannot be changed. So the originating router will treat the packet as best effort on egress, and it will be queued as best effort no-mark traffic (ingress-priority=0, the default) instead of the desired priority.

The way we have worked around this is by always splitting our P and PE router roles, which sometimes means we have to use two routers where we otherwise would just use one. The PE device and P device would have a high speed dedicated link between them (wired or wireless) so that the lack of QoS on that first PE-P hop isn't a big issue.

The only place we have routers doing double duty as both P and PE devices is for retail customer traffic, which is all best effort and so the lack of QoS is not problematic there.
That makes sense, and now that I know the secret to making the non chr routers properly utilize EXP/COS (as of your comments last night) I was thinking adding the second router as a P router would do as you said above. I havent tested the loss of QOS with php using implicit-null, but in the manual it says you "may" lose it. In your experience, is there any effect on EXP/COS that using implicit? Or it just saves the double lookup on the PE router? Wireshark captures look like the EXP is always properly copied.

Thanks again.
 
mducharme
Trainer
Trainer
Topic Author
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: MPLS/VPLS and HTB / EXP bits

Mon Nov 23, 2020 9:49 pm

That makes sense, and now that I know the secret to making the non chr routers properly utilize EXP/COS (as of your comments last night) I was thinking adding the second router as a P router would do as you said above. I havent tested the loss of QOS with php using implicit-null, but in the manual it says you "may" lose it. In your experience, is there any effect on EXP/COS that using implicit? Or it just saves the double lookup on the PE router? Wireshark captures look like the EXP is always properly copied.
In most cases I don't believe there should be any effect on QoS with implicit nulls (but I'm not 100% sure). We use implicit nulls everywhere, which is the MikroTik default. The final P router should read ingress priority before the penultimate hop popping takes place, which should get carried properly to CoS on egress to the PE device via the mechanism that I described in the other thread. This is different than on Cisco routers, where you read the MPLS EXP bits directly in the queue on egress and so if the label is gone you can no longer match the EXP bits because they are gone as well. Because MikroTik uses ingress-priority and priority for everything (which are copied from the EXP bits), and those survive the popping of the final MPLS label, things should still work fine on egress to the PE device.

The main use case that I would see for Explicit nulls with MikroTik is if you had a wireless radio that specifically supported reading MPLS EXP bits for QoS between the P and PE device - some manufacturers support this. However, most such radios could be configured to read CoS instead, so that isn't necessarily a big issue.
 
mikeeg02
Member Candidate
Member Candidate
Posts: 162
Joined: Fri Mar 30, 2018 2:28 am
Location: Pennsylvania

Re: MPLS/VPLS and HTB / EXP bits

Mon Nov 23, 2020 9:52 pm

That makes sense, and now that I know the secret to making the non chr routers properly utilize EXP/COS (as of your comments last night) I was thinking adding the second router as a P router would do as you said above. I havent tested the loss of QOS with php using implicit-null, but in the manual it says you "may" lose it. In your experience, is there any effect on EXP/COS that using implicit? Or it just saves the double lookup on the PE router? Wireshark captures look like the EXP is always properly copied.
In most cases I don't believe there should be any effect on QoS with implicit nulls (but I'm not 100% sure). We use implicit nulls everywhere, which is the MikroTik default. The final P router should read ingress priority before the penultimate hop popping takes place, which should get carried properly to CoS on egress to the PE device via the mechanism that I described in the other thread. This is different than on Cisco routers, where you read the MPLS EXP bits directly in the queue on egress and so if the label is gone you can no longer match the EXP bits because they are gone as well. Because MikroTik uses ingress-priority and priority for everything (which are copied from the EXP bits), and those survive the popping of the final MPLS label, things should still work fine on egress to the PE device.

The main use case that I would see for Explicit nulls with MikroTik is if you had a wireless radio that specifically supported reading MPLS EXP bits for QoS between the P and PE device - some manufacturers support this. However, most such radios could be configured to read CoS instead, so that isn't necessarily a big issue.
Perfect, that makes sense. I didnt think about what could happen interfacing with cisco or another manufacturer. Thanks again!

Who is online

Users browsing this forum: No registered users and 17 guests