Community discussions

MikroTik App
 
RcRaCk2k
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

v7.18.2 ECMP and Layer-4 Hash - is it working?

Tue Apr 01, 2025 10:21 pm

Hi,

i have 4 static routes with BFD enabled. But only one interface is used to route traffic.

The Test was a speedtest via OOKLA Speedtest. This uses many connections with random tcp src ports. So each connection (as far the policy hash gets different 4 times) should be randomly assigned to a default-route. But it is not.

> /ip/route/print where dst-address=0.0.0.0/0
Flags: D - DYNAMIC; A - ACTIVE; s - STATIC, b - BGP; + - ECMP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#      DST-ADDRESS  GATEWAY                                              DISTANCE
0  As+ 0.0.0.0/0    172.17.0.30%ipip-at-sbg-technoz-vpn3                       10
  D b  0.0.0.0/0    172.17.0.30                                                20
1  As+ 0.0.0.0/0    172.16.16.66%ipip-telekom2-de-fre-breslauerstr-vpn7        10
2  As+ 0.0.0.0/0    172.17.0.128%ipip-de-fre-breslauerstr-vpn7                 10
  D b  0.0.0.0/0    172.17.0.128                                               20
3  As+ 0.0.0.0/0    172.16.16.64%ipip-telekom2-sbg-technoz-vpn3                10

> /ip/settings/print
                                         ;;; ipv6 neighbor configuration has changed, please restart the device in order to apply the new settings
                                 ip-forward: yes                                                                                                  
                             send-redirects: yes                                                                                                  
                        accept-source-route: no                                                                                                   
                           accept-redirects: no                                                                                                   
                           secure-redirects: yes                                                                                                  
                                  rp-filter: no                                                                                                   
                 ipv4-multipath-hash-policy: l4                                                                                                   
                             tcp-syncookies: no                                                                                                   
                             tcp-timestamps: enabled                                                                                              
                       max-neighbor-entries: 8192                                                                                                 
                                arp-timeout: 30s                                                                                                  
                            icmp-rate-limit: 10                                                                                                   
                             icmp-rate-mask: 0x1818                                                                                               
  icmp-errors-use-inbound-interface-address: no                                                                                                   
                            allow-fast-path: yes                                                                                                  
                      ipv4-fast-path-active: yes                                                                                                  
                     ipv4-fast-path-packets: 3685145400                                                                                           
                       ipv4-fast-path-bytes: 3345628160850                                                                                        
                      ipv4-fasttrack-active: no                                                                                                   
                     ipv4-fasttrack-packets: 0                                                                                                    
                       ipv4-fasttrack-bytes: 0
What is wrong with my configuration?
 
RcRaCk2k
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: v7.18.2 ECMP and Layer-4 Hash - is it working?

Wed Apr 02, 2025 11:36 pm

Solved the issue, but is not clear to me, why that solves the issue.

As you can see on my earlier posted route-export, i have distance 10 for my default-static-routes and distance 20 for default-route learned via BGP.

In my opinion, i guess, that a distance of 10 kicks before 20.

But in this case, that is not true.

The default-route was only used from BGP, not my static routes.

After setting static routes to distance 20, ECMP starts working.

That makes no sense to me.

Screenshot 2025-04-02 223109.jpg
Screenshot 2025-04-02 223615.jpg
You do not have the required permissions to view the files attached to this post.
 
RcRaCk2k
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: v7.18.2 ECMP and Layer-4 Hash - is it working?

Wed Apr 02, 2025 11:45 pm

OH NO ITS NOT SOLVED - ISSUE DETECTED

As long as i have Torch running on the customers interface, ECMP will work. All 4 Upstream-Interfaces are in equal use.

While having speedtest running at client side (Mikrotik Bandwith-Test) and closing TORCH on the Router, upload went back to only one upstream interface.

So there seems to be an issue with FastPath?

Disable FastPath under /ip/settings instead enables ECMP. Is this a documented behaivour?
 
blacksnow
Frequent Visitor
Frequent Visitor
Posts: 63
Joined: Wed Feb 15, 2023 4:46 pm

Re: v7.18.2 ECMP and Layer-4 Hash - is it working?

Thu Apr 03, 2025 12:51 am

It's not fast path or any of the other things you mentioned. The issue is the hash policy, ECMP is broken when you use the L4 hash, or L3-inner (somtimes works but mostly broken) only when you have it on just "l3" does it work properly. Not sure what changed recently but this is a bug, and I think I posted it somewhere else but I guess I should make a ticket.
 
RcRaCk2k
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: v7.18.2 ECMP and Layer-4 Hash - is it working?

Thu Apr 03, 2025 3:06 pm

ECMP works correct for me when disabling FastPath.

This is reproducible.

Video of issue:
https://youtu.be/xSx3AG7173s
 
jlaham
just joined
Posts: 3
Joined: Wed Apr 02, 2025 6:04 pm

Re: v7.18.2 ECMP and Layer-4 Hash - is it working?

Tue Apr 29, 2025 7:25 pm

I wonder if this is related to what I'm experiencing (was going to create a new post, but this seems to be concluding in the same direction). I have a CRS310 connected to three servers, each of which are reachable by the same VIP that is assigned to their respective loopbacks (as well as some other services, etc. but that is out of scope here).

I simplified things as much as I could, down to three static routes for this VIP, with next-hop for each of the servers, I get three routes with ECMP in my route table. The issue is that once I enable l3-hw-offloading things break; I turn it off and everything works great again. Interestingly enough, things seem to be working fine once I drop down to just two routes with l3-hw-offloading enabled, doing ECMP, or at least it appears to be more stable, but I haven't done much testing to validate this.

The reason why this seems like a similar case is that from what I understand when l3-hw-offloading is enabled L3+L4 hashing is turned on automatically, and if it's broken I think that aligns with what's being discussed here, if I'm not mistaken?!