Hello everyone ..
i have configured two microtic routers with multipath (ECMP)
everythink wors fine BUT
scenario1
currently i have 3 routes to the same network. when i make the test
i make iperf from client1 → server 1 on port 5901 → router take routes path no1
i make iperf from client1 → server 1 on port 5902 → router take route path no2
i mak iperf from client1 → server1 on port 5903 → router take route path no3
scenario2
if i change setination IP
i make iperf from client1 → server1 on port 5901 → router take route path no1
i make iperf from cleitn1 ->server1:1(alias interface- other IP) on port 5902 → router take route path no2
i make iperf from cleitn1 ->server1:2(alias interface - other IP ) on port 5903 → router take route path no3
all tests was prepared at the same time.
conclusion is that ECMP works (in my case) per-src-dst-address combination load balancing.
now question. is there any way to change some settings that Mikrotic can take different route in traffic on the same IPS (src & dst) but other ports ? (in my case scenario no 1)
Yes, but not using ECMP. ECMP does not look at port numbers, period.
So you need routing marks and mangle rules assigning them if you want to distribute the traffic with same src and dst addresses across multiple routes.
Can You tell me how to do that ?
from example page there is only this
"/ ip firewall mangle
add chain=input in-interface=wlan1 action=mark-connection new-connection-mark=wlan1_conn
add chain=input in-interface=wlan2 action=mark-connection new-connection-mark=wlan2_conn
add chain=output connection-mark=wlan1_conn action=mark-routing new-routing-mark=to_wlan1
add chain=output connection-mark=wlan2_conn action=mark-routing new-routing-mark=to_wlan2
"
but this is reposible for symetric answear (packet incoming from wlan1 must return using the same interface)
currently i have config as follow (there is better but still the traffic are not evenly divided. (as attachment sent 3 screens - 1) iperf to 5901 port 3 threads, 2)iperf to 5902 port 3 threads, 3) ipers to 5903 port 3 threads) - look at mark connection (not symetric)
If you want an exact distribution then the use of PCC is not the best. Also think about when you spread traffic, don’t define the last line and just mark what is left over. You create then also a catch-all in case one of you earlier lines are not working.
I have spent an posting on this and have a look at NTH and that will spread evenly every connection.
Use 3:1 then 2:1 and 1:1 and 1:1. 1:1 Is then the one you can omit and it collects all what is left and don’t forget to set passthrough to NO because then the same connection will be evaluated again and get an other mark.
You are marking connections so you only have to do that once when it is a new connection status. If you need connections and routing then first mark the connections and then the routing based on the connection marking. As soon as you process a connection in any way the new connection status is removed and you loose the trigger to only mark new connections.
One way to evenly distribute the traffic among all interfaces with using pure ECMP (no routing marks, etc), is by disabling route cache in IP > Settings.
This will make each packet to go through a different interface in round-robin fashion (I think), instead of per src/dst ip.
So even a single connection can utilize all interfaces.
Of course disabling route cache may have other side-effects, even break features altogether, depending on the rest of your configuration on both sides.
@powiadamiacz, a statistics from just a few samples is no statistics at all. You’d have to create thousands of connections with different source an destination addresses and ports to see the actual evenness of their distribution across the rules/routes. The key here is how the 32-bit hash is generated from the total of 96 bits of the two IP addresses and ports.
What @msatter suggests is a more deterministic approach which distributes the connections
evenly among the paths - 1/3 of the total to the first, 1/2 of the remaining 2/3, which is another 1/3, to the second, and the rest, which is yet another 1/3, to the third. But that still says nothing about traffic volume distribution, as the total amount of data transported by any given connection is unpredictable.
@Cha0s, what you suggest would likely work, but sending individual packets of the same TCP session via different physical paths is what 9 of 10 dentists advice against as that may cause packets to arrive to destination in shuffled order. So a good exercise but a possible headache if deployed in production.
Regarding the settings below - the trafic not working when i remove the last line - ( i thought that this line is not nessesery ) - can you explain me ?
I am not a routing specialist but with distance and specific WAN you create a problem.
The connection with the shortest distance gets priority to transport traffic but then only for WAN3 traffic leaving WAN2+3 to be catched by the last routing rule.
Why not only use the routing-mark and set the distance to one for all three.
Does it stop working for iperf running outside the router or you’ve only tried pinging from the router itself? Packets sent by the router itself do not pass through the prerouting path but through the output one.
So if there is no other route to 172.31.19.0/24 (a default one would be sufficient) in routing table main (the one chosen if no routing-mark is assigned), it would explain this behaviour for locally originated packets only.
But even if you add connection-marking and routing-marking to chain output of mangle, it still won’t work without a working route in table main because the handling of locally originated packets is a bit complex. A locally originated packet is first routed and only then mangled, and if mangling eventually results in assignment of a routing-mark, it gets routed one more time. So if the first routing step using table main finds no route, the further steps do not happen.
@msatter, this is a misleading statement. The distance value is only used to choose among routes with identical routing-mark and identical dst-address.
So if you have dst-address=0.0.0.0/0 distance=1 and dst-address=1.2.3.0/24 distance 10, the latter one will be used to deliver a packet to 1.2.3.4. Both routes match, but the most narrowly matching routes are always chosen.
@sindy Only the routing-mark differs and the destination address is in all four lines the same. Which is not present if run directly from the router as you wrote.
Differing routing-mark means different routing table, so it always overrides distance as I wrote above.
If no route with the required routing-mark matches, routing table main is used instead (unless you explicitly prohibit such a fallback using /ip route rule action=lookup-only-in-table table=therouting-mark-value), but that’s another can of worms.
This method is not for every scenario. Obviously.
And obviously you don’t use this on anything other than fixed/wired network with predictable performance (ie no congestion that can cause out of order packets etc).
And it can work. I’ve used this method for years in production where I needed to distribute traffic among multiple interfaces but NOT touch (in any way shape or form) the dreaded Connection Tracking Table.
