I currently have a network that is routed using OSPF and intend on enabling MPLS and using BGP to send public traffic over MPLS-TE’s. I have many sites where I have 2 radio links as backhaul to aggregate bandwidth. Currently, I have LACP enabled to balance traffic based on layer 3+4. When I enable MPLS on the network, LACP will no longer be able to balance the traffic. In the future there will be VPLS tunnels to each site as well.
Has anyone found a good way to balance traffic across 2 links with MPLS enabled? I don’t believe there’s a way to create 2 VPLS tunnels to the same location so I couldn’t just separate the the links into 2 OSPF paths to split the traffic that way.
It sure can, but please remember this will only work reliably when it is Mikrotik <—> Mikrotik or Mikrotik <—> Linux with both ends configured for balance-rr
Yes - I was going to mention that both sides would have to support balance-rr, which is mostly just MikroTik and Linux. I have heard that HPE supports balance-rr, but I’ve never tried it on HPE devices.
This is a good question.
balance-rr is not a good solution and should be avoided.
To make flow-aware load-balancing router needs to use some more information to calculate hash value.
With “pure” IP traffic it uses 5-tuple, but with MPLS it’s more challenging.
Some ways to do that:
Flow-label in tunnel. Additional label that have to be calculated at the endpoints and added to label stack. Must be supported by both sides, and transit router should use is for hash calculation.
Inspect VPLS/PWs packets deeper to the IP headers and take src/dst values. Works only if there is no additional encapsulation in the tunnel (ex. pppoe).
I think MT should add some of this mechanisms to ROS7, especially if they want to enable MPLS ECMP.
Load-balancing by flow-label or IP header in MPLS on the switches LAGs must be supported by switch-chip. I don’t know if Mikrotik switches can support this.
It looks like the MPLS ECMP coming in v7 is really what I’m looking for. I just don’t know how long it will be until Mikrotik adds it. I’m assuming there isn’t any information about that? I don’t believe there’s a roadmap.
I have about 1/3 Mikrotik switches and the rest vendors which don’t support balance-rr. I suppose I could change out the other switches but I was a little worried about packets in the same flows coming in out of order.
Sorry to resurrect an old post, but I am facing the same issue. I use a CCR1036 on 6.47.10 as a traffic aggregation point at our core for our WISP network. Said router is uplinked to our core switch stack via 2x 10Gbps interfaces in a 802.3ad bond using l3+l4 hashing. This router establishes two VPLS tunnels per site router to carry VLAN tagged public IP and VLAN tagged private IP (NAT) traffic. Absolutely none of this traffic is load balanced and one of the two sfp+ interfaces in the bond takes the brunt of the work. The issue is becoming more prevalent as we grow with this one interface coming within 90% of saturation during peak while the other interface is at about only 20% capacity. balance-rr is not an option here.
Further in the network at the sites, I see the same behavior where we use 802.3ad bonded interfaces between 2116’s and 317’s. Only one of the two member interfaces will take the traffic load. What’s interesting is that these sites exhibiting this behavior are all on new releases of RouterOS 7. I might try the balance-rr method where I have Mikrotik ↔ Mikrotik cases.
As stated earlier in this thread by another user, the Mikrotik is probably not inspecting the MPLS/PW headers and therefore unable to calculate a hash value.
I opened SUP-170589 yesterday about this behavior. At this point I am trying to figure out if this is a limitation of RouterOS, a misconfiguration on my part, or time to just throw in a 2216 in place of the 1036 and let the problem migrate over to a device with higher capacity interfaces!