I am using two different providers, WAN-WE (primary) and WAN-A1 (secondary). Services are reachable from the outside via both IP’s. WAN-WE is used as the default-route.
In order to avoid asymmetric routing I tried to introduce these mangle rules:
However, as soon as I activate the mark-connection mangle rule, the secondary IP (WAN-A1) becomes inaccessible. What am I doing wrong? Using ROS 7.14.1.
Thing is, we did not care about asymmetric routing so far, all these years we used those two providers WE and A1
But now we are going to switch to a new provider. So currently we have 3 providers attached to our router. The third one being MAGenta.
Yesterday for a test we switched the default-route from WE to MAG. As a result all traffic coming in via A1 did not work anymore.
Our suspicion is, MAGenta is very strict with checking for asymmetric traffic, and drops it. Basically without mangle-rules, packets coming in on A1, will go out on MAG → that fails. With the WE default-route traffic coming in on A1 went out on WE → worked fine (without any mangle rules)
Also, while using the new MAG default-rule one VM (that is accessed via one of our A1 IP’s) continued to be reachable, and that was one that had a specific mangle rule set:
I’m frequently amazed about how few providers implement network ingress filtering per BCP38, if more did it it would help cut down spoofed address DDOS traffic.
The route marking should be applied to everything except the ingress interface, otherwise packets will immediately be forwarded back out as the routing table only contains one route:
/ip firewall mangle
add action=mark-routing chain=prerouting connection-mark=CONNECT-A1 in-interface=!eth11-WAN-A1 new-routing-mark=WAN-A1 passthrough=no
I’m not keen on the use of connection-mark=no-mark either, as it often has unintended side-effects, and normally use connection-state=new along with other conditions to apply the initial connection mark.
@OP: Still waiting for requirements because I have no clue as to what you mean. As for no-mark, not sure what you mean TDW but that is a separate discussion…
@tdw yes apparently Magenta has BCP38 “Ingress Filtering” enabled within their network.
I will definitely try your suggestions regarding both mangle rules. I can only do that at night, so not to interfere with productive traffic. I am really curious if in-interface=!eth11-WAN-A1 will do the trick. Basically I want to make sure that traffic that is meant for the internet (ie not local) will receive the routing-mark, just before heading out.
I got the symmetric routing working with these specific mangle rules:
Of course this only works because I check for a specific external IP. I need mangle-rules that work generally with any external IP. But basically this shows me, that the mangle rules are the right way to go.
Unfortunately in-interface=!eth11-WAN-A1 in the mark-routing rule does not work. Also removing connection-mark=no-mark, or adding connection-state=new did not make a difference.
What I do not understand is, I have this working pair of mangle rules. If I then change the specific dst-address=213.142.96.9 in the mark-routing rule to generally any non-local IP, like this: dst-address=!10.42.0.0/16 - this no longer works! How can that be? 213.142.96.9 literally is !10.42.0.0/16
I do have other mange rules, and also /routing rules (that I use to make sure VPN traffic is routed correctly on the two WANs). These all work fine on their own. But since they are disabled during my tests, I doubt they would interfere.
Are all of your internal addresses within 10.42.0.0/16, otherwise they are different.
For packets from the router itself you need a mark-routing rule in the output chain as this never hits prerouting.
I’ve found mixing marking in mangle and routing rules can be confusing in complex setups. Certainly in cases where the Mikrotik has multiple WANs, PPPoE and hotspot server(s) I’ve found it better to only use mangle, and to follow the exact packet path using https://help.mikrotik.com/docs/spaces/ROS/pages/328227/Packet+Flow+in+RouterOS to figure out the conditions for marking both connection and route.
Btw, I had a similar question back in 2023, when I had working mangle rules in ROS 6, but when I moved to ROS 7 they stopped working. The following rules worked up until 7.2.1, and stopped working with 7.2.2 → they still dont work with 7.16.2, so something fundamental must have changed back then.
In 7.2.2(ish) the routing was changed, so that if you route mark a packet, and a matching route with that route mark exists in the route table
it will use that route entry. It then processes the routing rules table, and finally the route table (again).
Previously routing rules occurred first, and then the route table.
This was done at least partially for VRF’s
If you want to go via the rule table, best don’t mark the packets with route marks for existing routing table entries.
I would commonly use a route mark like ruleWAN-2
that from the rule table could use a lookup in the WAN-2 table (after your rule that looks up in the main table).
Note also the min-prefix setting in routing rules is often a simplifying shortcut for your routing rule where multiple local address ranges need to be
allowed for.
I was not able to get symmetric routing to work with mangle rules. Instead I am using a different solution, that works nicely.
I assign two different IP’s to the same VM in the network 10.42.1.21 and 10.42.1.22
Traffic coming in on WAN-1 will be NATed to 10.42.1.21, and traffic from WAN-2 will be NATed to 10.42.1.22
When traffic is about to head out of the router, coming from IP 10.42.1.22, a set of routing rules makes sure that traffic meant for the local network (in my case 10.42.0.0/16) will be routed via the main-table.
And another rule right after that, with no specific dst-address, ie anything !10.42.0.0/16 will use routing table “WAN-2” (which of course needs to be created first)