Split UDP Traffic - Route Realtime Application to Reliable WAN

RouterOS General
1

Hello,

I have a unique scenario where I have 2 WAN.

WAN1 has very high speed and unlimited data, but not very good at doing real-time traffic such as VoIP, Game, live streaming, Video conferencing, etc. Sometimes, the quality gets very choppy. I mainly use this for for heavy traffic and browsing where real-time application does not really matter

WAN2 has very reliable connection, very good for gaming, VoIP, live streaming, etc., but not as fast as WAN1, and it doesn't offer unlimited data.

Here is I want to accomplish:
All UDP below 2 Mbps will be go to WAN2. Anything else will be going to WAN1

I have tried to use the connection-rate per documentation, but it is for packet marking for queue, I need to find somehow to get it to route. However, I can't get it to work.

I don't want to go down the road where I need to identify every single UDP port and mark it down one by one application. All I know that if UDP traffic is below 2 Mbps, it will be treated as real-time application.

Please help how to configure

Thank you so much!

2

I think it can be done through following example rules:

/ip firewall mangle
add action=mark-connection chain=prerouting in-interface="WAN1_int" connection-mark=no-mark connection-state=new new-connection-mark=WAN1
add action=mark-routing chain=output connection-mark=WAN1 new-routing-mark="WAN1_routing_table_name"
add action=mark-connection chain=prerouting in-interface="WAN2_int" connection-mark=no-mark connection-state=new new-connection-mark=WAN2
add action=mark-routing chain=output connection-mark=WAN2 new-routing-mark="WAN2_routing_table_name"
add action=mark-connection chain=prerouting in-interface="LAN_int" new-connection-mark=all_conn
add action=mark-connection chain=prerouting connection-bytes=500000-0 connection-mark=all_conn connection-rate=0-2M new-connection-mark=real_time_conn protocol=udp
add action=mark-routing chain=prerouting connection-mark=real_time_conn new-routing-mark="WAN2_routing_table_name"
add action=mark-routing chain=prerouting connection-mark=all_conn new-routing-mark="WAN1_routing_table_name"

First four rules make sure that traffic that traffic that came through one WAN will leave through the same, second two rules catch UDP traffic below 2Mbps, last rules route connection to appropriate WAN.

Hopefully someone like @anav could review my rules to see whether I have written nonsense or not

3

Who me,,,,,,,,,,, I have no clue these days, losing brain cells faster than I grow em. :wink:
On the surface looks good. just ensure to add passthrough=no on the last two mark-routing rules.

I would not 'confuse the issue' or 'introduce unknown requirements' with the mangles for services aka VPN traffic to the router.......................... the OP has not been forthcoming on requirements!!

Traffic Monitor can based on traffic ( transmit or receive ) trigger ( above, below, always ) aka a stated threshold in Mbps, conduct a basic script action.

However, there is no way to isolate UDP........ until I saw what theCat12 just did,,,,,,,,,

But thecat didnt state why he added connection-bytes=500000-0 in the mix ????????????????
I mean connection-rate=0-2M makes sense and he did mention it.....

/routing table
add fib name=ISP2_route

/ip firewall address-list
add address=ISP1subnet list=Connected
add address=ISP2subnet list=Connected
add address=192.168.22.0/24 list=Connected comment="Local LAN subnet"

/ip firewall mangle
add chain=prerouting action=accept src-address-list=Connected
dst-address-list=Connected 
+++++++++++++++++++++++++++++++
add chain=prerouting action=mark-connection connection-mark=no-mark \
in-interface="LAN_int"  connection-bytes=500000-0 connection-rate=0-2M \
new-connection-mark=trafficUDP protocol=udp  dst-address-type=!local \
dst-address-list=!Connected  passthrough=yes
add action=mark-routing chain=prerouting connection-mark=trafficUDP \
new-routing-mark=ISP2_route passthrough=no

I didnt see the point of being concerned about identifying all lan to wan traffic and then udp traffic (although it was logical), so I just identified the UDP traffic on the first rule. Also, only that limited UDP traffic needs to go to WAN2 and thus we only need one route-marking as all other traffic will go automatically out WAN1 due to distance.

/ip route
add dst-address=0.0.0.0/0  gateway=ISP1 table=main
add distance=2 dst-address=0.0.0.0/0 gateway=ISP2 table=main
4

I don't think this will work if both WAN require the usual srcnat (masquerade) to change the source IP address of outgoing packets to the public address of the WAN interface, because such masquerading is only performed once per connection.

Let's look at the sequence. When a client from inside the LAN starts sending packets to an UDP port on the internet, the first packet will create a conntrack entry, which will be marked with all_conn and the packet will get thet routing mark WAN1_routing_table_name, then it will reach SRCNAT and masquerade will change the source IP address to the one of WAN1 and will remember this in the conntrack entry. Furture packets of the connection will be NATed/un-NATed automatically based on this information, and the SRCNAT rule will no longer be hit (no counter increment).

After 500kB, and because the rate is low, the mangle rule change the connection mark to real_time_conn, this cause subsequent packets to have the routing mark WAN2_routing_table_name (here you've missed the condition so that the routing mark only applies to outgoing packets), so they will use WAN2 to go to the internet. However the masquerade rule is not reapplied and the conntrack entry still has the old NAT data. The packet will go out of WAN2 with the IP address of WAN1 as source. Most ISP will reject this and will not forward the packet to the destination.

Even if you have a sloppy ISP that happily forward packets with foreign source, and the packets reach the remote host, because the source address is from ISP1, when the remote host sends response, the response packets will come through WAN1!

Even if you theoretically manage to force change the source address to the one of ISP2, the remote host will then not accept the packet as belonging to the existing connection and treat it as a new incoming connection. Depending on the application details, this will with very high probability not work correctly.

In conclusion, you will not be able to switch an ongoing connection to another WAN on the fly like that, it will not work with most application protocol. Only applications that has special support for roaming (like WireGuard) can handle that.

A working idea might be to let the existing connection continues to use WAN1, but also remember the destination address of the slow connection (with the add destination to address list action) for a few days, and have mangle rules that put future UDP connections with destination in that list on WAN2. These rules should have precedence over the one that marks all_conn. Unfortunately the address list only store the IP addresses, the is no way to remember the destination port, so this method is not granular enough.

EDIT: And we probably want to exclude destination port UDP 443 from the connection rate check because that's used by QUIC and HTTP/3.

5

Perhaps one could include the routing mark as a matcher in the masquerade rules and that could do the trick

6

The problem is that the NAT rules only match the first packet of a connection, as documented by MikroTik here:

NAT - RouterOS - MikroTik Documentation

Nat matches only the first packet of the connection, connection tracking remembers the action and performs on all other packets belonging to the same connection.

So once the connection has the all_conn mark and the first packet has the WAN1_routing_table_name routing mark, srcnat masquerade of WAN1 is applied to the connection and the NAT table are not consulted anymore for future packets of that connection. Which means if you later swap the routing mark of future packets, those packets still have the address translation belonging to WAN1.

7

@ CGGXANNX

/routing table
add fib name=ISP2_route

/ip firewall address-list
add address=ISP1subnet list=Connected
add address=ISP2subnet list=Connected
add address=192.168.22.0/24 list=Connected comment="Local LAN subnet"

/ip firewall mangle
add chain=prerouting action=accept src-address-list=Connected
dst-address-list=Connected 
+++++++++++++++++++++++++++++++
add chain=prerouting action=mark-connection connection-mark=no-mark \
in-interface="LAN_int"  connection-bytes=500000-0 connection-rate=0-2M \
new-connection-mark=trafficUDP protocol=udp  dst-address-type=!local \
dst-address-list=!Connected  passthrough=yes
add action=mark-routing chain=prerouting connection-mark=trafficUDP \
new-routing-mark=ISP2_route passthrough=yes

Does adding the additional mangle below to the mix change your mind??

add chain=prerouting action=mark-connection connection-mark=trafficUDP \
routing-mark=ISP2_route new-connection-mark=StickyWAN2 passthrough=yes
add chain=prerouting action=mark-routing connection-mark=StickyWAN2 \
new-routing-mark=ISP2_route passthrough=no
8

Without making unnecessary conjectures, tests, or hypotheses,
simply:

It can't be done.

Quick explanation:
If a service expects incoming UDP packets from the IPx of WANx and then UDP packets arrive from WANy, which certainly has a different IPy,
obviously, unless it's the latest s–t server, the packets will be ignored because they no longer arrive from the same IPx, causing the service to be interrupted.

9

Hi @anav. The problem is that once a connection is already underway, it not a good idea to change the outgoing interface when the interfaces go to different ISPs, with different interface IP addresses. Even more so if NAT are involved. I've written above about NAT, why with SRCNAT involved the source address of the outgoing packet will not change anymore. And in case you somehow are able to manipulate the address so that it changes and matches the outgoing ISP, then as both @rextended as I have explained above, the remote side most probably won't accept the new packets as part of an existing connection anymore.

So IMHO the routing change can only affect new connections from the very first packet of those connections. And we can direct them to the routing table of the 2nd WAN based on historical data of previous connections.

That's why my idea above was that we'll remember that these and these destination IP addresses have had "slow rate" UDP connections in the past, by using mangle rules with action=add-dst-to-address-list when we see that the current UDP connection is "slow". Then for future connections to those same IP addresses within 48h for example, we'll use mark-connection with dst-address-list=xxx and together with mark-routing to have them use the alternative routing table.

10

Okay understood, we cannot change the current connection, only affect new ones, and thus always chasing the tail so to speak.............

Well then I go to my fallback solution
WIFI1 SSID - all uses for low UDP throughput
WIFI2 SSID - all other uses

OR
wired connections= low UDP throughput
wifi connections=everything else

OR
one vlan=low UDP throughput
all other vlans= everything else

you get the idea

11

Hello,

Thank you so much for the contribution. I have seen the concern is about the switch over from WAN2 to WAN1. I believe, the UDP nature that they will automatically re-establish connectivity immediately when the switchover occurs.

Of course, if the UDP traffic speed is faster than 2 Mbps, I can generally conclude that the traffic is most likely to be non-real-time application, such as VPN, torrent, and etc. At this point, they will experience split seconds outage due to re-establish UDP connectivity on WAN switch over, and I don't see any issue with that.

At this moment, I only separate based on the port number such as excluding port UDP 443, and bunch of IPsec, L2TP and WARP (Wireguard based) VPN ports. Any application can use UDP traffic to have large file transfer which I am trying to avoid to go down to WAN2.

Thank you

12

Instead match the important traffic by DSCP value. Modern applications (e.g. VoIP) will set this value to indicate the relative priority of their traffic, value 46 indicates packets you will want to send via WAN2.
Advantage is that the classification of your traffic will not change over time.