We have a chain of Mikrotiks in a test lab, with VPLS and MPLS-TE tunnels.
MKT1 ↔ MKT2 ↔ MKT3 ↔ MKT4
We have a TE tunnel and VPLS tunnel connecting MKT3 to MKT1, and the same connecting MKT4 to MKT1. On MKT4, we are using a bridge filter to set the priority for all packets entering the tunnel to MKT1 as priority 7. We have no filter on MKT3 since they should be experimental bits at that point, and the tunnel from MKT3 to MKT1 should not be prioritized.
To simulate congestion, on MKT3’s interface that connects to MKT2, we implemented a queue tree with a 20Mbps limit, matching no-mark. It matches all packets and shapes to 20Mbps. However, it seems to ignore the MPLS EXP bits set by MKT4 and gives the traffic the same priority as the MKT3 traffic.
NOTE: To test this, we have a UDP packet generator on MKT4 sending to an IP of a computer plugged into the VPLS tunnel endpoint at MKT1. We have a similar UDP packet generator on MKT3 sending to the same computer. The rate of the UDP packet generator on MKT4 is the bare minimum packets per second required to create a 5Mbps rate (the guaranteed rate for the customer), but have UDP packets going through MKT3 with a much higher rate (30 or 40 Mbps). We can see the rate that packets are arriving at the tunnel endpoint at MKT1. When we start the traffic generator on MKT3, the traffic from MKT4 drops down from 5Mbps to 4.2. Disabling the traffic generator causes the MKT4 tunnel traffic to return to the customer guaranteed 5Mbps. If we try disabling and enabling the bridge filter on MKT4 that applies the priority to the packets, it has no impact on the 4.2 value, which suggests that the MPLS experimental bits are either not being set or not being honored.
I’m not sure whether this is caused by the HTB not recognizing that the EXP bits should have priority, or whether the EXP bits are not getting applied properly from the beginning.
I believe I have found a successful method of marking mpls frames which works in the queue tree.
On MKT3’s interface that receives data from MKT4 I created a bridge. I made that bridge the ldp and mpls TE interface. I applied a bridge filter on that router that takes packets with ingress priority 7 and applies a mark. That mark then works on the queue tree on the port going to mkt2 and seems to work. I will do more testing tomorrow.
Yes, I have confirmed that it works. This provides a method for QoS with MPLS/VPLS, and allows marking of MPLS frames for use in queue trees. The last post was slightly incorrect. It is actually quite simple:
Both VPLS tunnel endpoints should be connected to bridges
A bridge filter should be set up on the ‘forward’ chain, to ‘set priority’ to a value 0-7 that you want the MPLS experimental bits to be set to.
On other routers where you have congested links (ex wireless), set up a queue tree on either side of the link that shapes to the link’s maximum. Then, do the following to properly mark the MPLS packets for use in your queue tree:
At the egress interface that has your queue tree set up, create a bridge that includes only that interface
Add a bridge filter for chain ‘output’ that matches ‘ingress priority’ equal to the value that you had assigned on the endpoints (i.e. the experimental bit values)
Set the action to ‘mark packet’
The queue tree will now properly classify the MPLS packet in the queue and properly do QoS.
Hi, really interesting way of doing things. However I think this uses a bridge and adds processing overhead, how’s the performance you have noticed using this method?
Hopefully someday MikroTik will allows us to process “natively” MPLS trafic with QoS
With 3 tunnels totaling 20Mbps and extensive queueing we are only seeing ~1% CPU usage on Routerboard 1100AHX2’s in our lab. I’m sure the impact will be bigger on higher bandwidth links, but well within reason.
I have a test lab with 3 RB MIKROTIK with OSPF / MPLS - VPLS structure to connect a transparent LAN network but I can not apply to limit the MPLS-VPLS bandwidth. Could you help me.
To limit the VPLS bandwidth, all you need to do is create a queue tree on both ends of the VPLS tunnel, set to match no-mark, with the limit you want, and the parent set to the VPLS interface. The upload limit will be on one side and the download limit on the other.
I have tested and accomplished what you have shown above, in the middle routers.
Lets say you wanted to accomplish this on the link between mtk1 and mtk2… mtk1 who is a vpls tunnel end point. How do you queue the mpls out frames from there and include the traffic from the source router (mtk1) (Aside from using the vpls tunnel interface itself). In the scenario I need to implement, I would have an additional link from mtk1 to mtk4 forming a ring, and the vpls tunnel could go out either interface to get to its tunnel end point.
Is this a silly question? I cant seem to get the packets into a HTB queue on the mpls-out interface that are from the start of the tunnel. Though I can get the passthrough mpls traffic on the out interface, like described above. Help?
Heres a dev router. Effectively everything that comes in ether5 is being packet marked and priority set before it goes into the vpls tunnel that starts here. If this router was in the middle, and using the bridge method described above, I can queue the mpls packets based on priority of mpls packets going through the router. But I cant seem to queue the packets that originate from the router. In the code below, I’ve tried both using ip firewall and bridge filters to mark. Neither shows up in the ether1 queue, which is the ldp interface to the next router. How can I get the packets that get put into the vpls tunnel at this router, into my output queue? If that makes sense.
Yes, this is not possible, unfortunately. The issue is that ingress-priority is only set automatically when the packet first arrives at the router, and you can only match ingress-priority in bridge filter rules (not priority). Any packets that are created by the router (in this case, the VPLS packet is created by the router) is ingress-priority 0 and this cannot be changed. So the originating router will treat the packet as best effort on egress, and it will be queued as best effort no-mark traffic (ingress-priority=0, the default) instead of the desired priority.
The way we have worked around this is by always splitting our P and PE router roles, which sometimes means we have to use two routers where we otherwise would just use one. The PE device and P device would have a high speed dedicated link between them (wired or wireless) so that the lack of QoS on that first PE-P hop isn’t a big issue.
The only place we have routers doing double duty as both P and PE devices is for retail customer traffic, which is all best effort and so the lack of QoS is not problematic there.
That makes sense, and now that I know the secret to making the non chr routers properly utilize EXP/COS (as of your comments last night) I was thinking adding the second router as a P router would do as you said above. I havent tested the loss of QOS with php using implicit-null, but in the manual it says you “may” lose it. In your experience, is there any effect on EXP/COS that using implicit? Or it just saves the double lookup on the PE router? Wireshark captures look like the EXP is always properly copied.
In most cases I don’t believe there should be any effect on QoS with implicit nulls (but I’m not 100% sure). We use implicit nulls everywhere, which is the MikroTik default. The final P router should read ingress priority before the penultimate hop popping takes place, which should get carried properly to CoS on egress to the PE device via the mechanism that I described in the other thread. This is different than on Cisco routers, where you read the MPLS EXP bits directly in the queue on egress and so if the label is gone you can no longer match the EXP bits because they are gone as well. Because MikroTik uses ingress-priority and priority for everything (which are copied from the EXP bits), and those survive the popping of the final MPLS label, things should still work fine on egress to the PE device.
The main use case that I would see for Explicit nulls with MikroTik is if you had a wireless radio that specifically supported reading MPLS EXP bits for QoS between the P and PE device - some manufacturers support this. However, most such radios could be configured to read CoS instead, so that isn’t necessarily a big issue.