Routing mark validity in filter rules and others.

kubiko · February 18, 2025, 2:32pm

Hello
I have an interesting situation.
I am using Router OS 6.49.13 (long term)
In my network temporarily I need have two separate WAN connections. Firewall Filter rules are set using interface lists, IP lists…
Until now I had only one existing internet connection WAN1 with his public IP address and this is still default gateway (route do destination 0.0.0.0)
I want to switch several public services to secondary WAN2 connection with different public IP address. This services uses DST-NAT and UPD/TCP ports.
DST-NAT rules are also set using interface lists so it is valid and should be functioning for both WAN connections.
Each WAN connection has it’s own interface.

When I trying to connect to these services using WAN2’s IP address, it does not work. I have tracked packets by packet sniffer and found that outgoing packets have the correct source address for WAN2 but are sent from the interface for WAN1 which is out of scope. How is this possible? Even if source address is for WAN2?
I have used routing mark to communicate several LAN systems using WAN2, it works correctly. Also I have set mark connections on output chain which has WAN2 source IP and destination to the WAN and it seems that this marking is not applied to DST-NAT response packets and therefore packets is still sent to WAN1 interface even though this packets has WAN2 source IP.
This is confusing for me. I understand that as default gateway in main routing table is WAN1 and therefore all packets to the world should go through WAN1 interface but using this route it should also use WAN1 IP address as source address in packets, not this confusing combination. Could someone explain this behavior? I think it is because DST-NAT input is from WAN2, mikrotik tries to response using WAN2 using correct IP address for WAN2 but by default route it is forced to WAN1 and that’s why it’s getting crazy.

Then I think the solution should be using packet marking using prerouting chain, but I am not sure how the filter rules is applied. In filter rules is not set routing mark and my question is if the filter rules will be valid also for marked packets or it is needed to set standalone set of filter rules for this routing mark? I tested it and it seems that filter rules without routing mark parameter set is applied on each packet but I need to be sure. Because this behavior is not valid for routes and I was unable to find some description to this.

Hello
I have an interesting situation.
I am using Router OS 6.49.13 (latest long term)
In my network temporarily I need have two separate WAN connections. Firewall Filter rules are set using interface lists, IP lists…
Until now I had only one existing internet connection WAN1 with his public IP address and this is still default gateway (route do destination 0.0.0.0)
I want to switch several public services to secondary WAN2 connection with different public IP address. This services uses DST-NAT and UPD/TCP ports.
DST-NAT rules are also set using interface lists so it is valid and should be functioning for both WAN connections.
Each WAN connection has it’s own interface.

When I trying to connect to these services using WAN2’s IP address, it does not work. I have tracked packets by packet sniffer and found that outgoing packets have the correct source address for WAN2 but are sent from the interface for WAN1 which is out of scope. How is this possible? Even if source address is for WAN2?
I have used routing mark to communicate several LAN systems using WAN2, it works correctly. Also I have set mark connections on output chain which has WAN2 source IP and destination to the WAN and it seems that this marking is not applied to DST-NAT response packets and therefore packets is still sent to WAN1 interface even though this packets has WAN2 source IP.
This is confusing for me. I understand that as default gateway in main routing table is WAN1 and therefore all packets to the world should go through WAN1 interface but using this route it should also use WAN1 IP address as source address in packets, not this confusing combination. Could someone explain this behavior? I think it is because DST-NAT input is from WAN2, mikrotik tries to response using WAN2 using correct IP address for WAN2 but by default route it is forced to WAN1 and that’s why it’s getting crazy.

Therefore I think the solution should be using packet marking using prerouting chain and then use separate default route, but I am not sure how the filter rules is applied. In filter rules is not set routing mark and my question is if the filter rules will be valid also for marked packets or it is needed to set standalone set of filter rules for this routing mark? I tested it and it seems that filter rules without routing mark parameter set is applied on each packet but I need to be sure. Because this behavior is not valid for routes. I was unable to find some description to this.

So I’m trying to point out that the routing mark parameter works differently for firewall rules and differently for routes. For routes, an unset routing mark parameter behaves the same as if it were set to main, so it always only applies to main, but this (probably) does not apply to filter rules.

panisk0 · February 18, 2025, 3:31pm

https://help.mikrotik.com/docs/spaces/ROS/pages/59965508/Policy+Routing

anav · February 18, 2025, 7:12pm

Many words, little clarity.
Suggest showing examples of what you mean, provide some context, otherwise, Im with panisko.

kubiko · February 19, 2025, 12:05am

Thank You guys
Ok, I can write an example.
WAN1 Public IP 20.30.40.50 on eth1 default GW 20.30.40.49
WAN2 Public IP 60.70.80.90 on eth2 default GW 60.70.80.89

LAN1 Addr 192.168.1.0/24 eth3
LAN2 Addr 192.168.2.0/24 eth4

VPN server on port 1111 on LAN1 with addr 192.168.1.2 accessible via dst-nat on UDP port 1111 on IP 20.30.40.50
I want VPN to be accessible also through IP 60.70.80.90
firewall config: dst-nat on chain dst-nat on iface list WAN port UDP 1111 (iface list WAN includes eth1 and eth2)
default GW for router: 20.30.40.49 for dst 0.0.0.0/0

When I am connecting to VPN from outside using 20.30.40.50 everything works
When I try to connecting VPN using 60.70.80.90 packets arrive to VPN server, VPN server responds by packets with src IP 60.70.80.90, but packets are routed to eth1 not to eth2 which is incorrect . My first question is why this occurs.

I am planning to use routing mark for incomming packets on prerouting chain from eth2 and for this separate mark I have set default GW as 60.70.80.89 for dst 0.0.0.0/0, so it should work.
But I have defined several filter rules for security purposes which is based on IP lists and interface lists.
For ex when I want to drop TCP port 80 packets from WAN, I have it defined in rule conditions for WAN iface list so everything incoming from WAN ifaces eth1 or eth2 and tries to connect using TCP port 80 is dropped.
My question is if I can rely on the fact that my original filter rules stay be applied for entire communication including marked packets. I tested several rules and it Works also for marked communication but I need to be sure.
I have read the mentioned docs before creating this topic but what I’m asking here is not addressed there.
My second question is: I need to know how routing marks and filter rules are related.

anav · February 19, 2025, 2:23am

You need to mangle traffic such that traffic aimed at a WANIP subsequently leaves the same WANIP.
As per normal no questions can be answered specific to your scenario because you have not provided the config. Heck we have no idea how your wans are setup.

/export file=anynameyouwish (minus router serial number, any public WANIP information, keys )

kubiko · February 19, 2025, 8:44am

Hello
There is no special config for the WANs, only basic config, one default route belonging to/valid for unmarked connections routed to WAN1 and second default route for marked connections routed to WAN2, assigned IP addresses, one SRC-NAT for WAN1 and one for WAN2. Nothing special, only necessary for basic functioning.

I have more LANs but they are not related…
I don’t want to provide config because it is quite large because of several LANs and huge filter rules related to WAN iface list and LANs, also because of for sensitive data such as blocked/enabled another public IP addresses in filter rules…

Important for my are firewall filter rules and I want to know and be sure if the filter rules will be applied when I will mangle the trafic incomming to the WAN2.

And I would also like to know why, if I don’t use traffic marking, the outgoing packets for DST-NAT use the WAN2 IP address but the packets are routed to WAN1. I have a feeling that this is some kind of stupidity/madness or undefined behavior on the part of Mikrotik if the config is not correct. Since this doesn’t make sense, I would like to understand more deeply why this is happening.

I would like to know and learn general mikrotik’s behavior, also for the future, I don’t like to obtain only exact solution for this case. Therefore it is better for me if You will provide general answer instead of config command.

panisk0 · February 19, 2025, 9:33am

@ kubiko look here:
http://forum.mikrotik.com/t/dual-pppoe-wan-failover-with-specific-port/181541/8

table main

/routing table
add fib name=to_WAN_main
/ip route
add dst-address=0.0.0.0/0 gateway="20.30.40.49" routing-table=to_WAN_main
/routing rule
add routing-mark=to_WAN_main table=to_WAN_main
add src-address=20.30.40.48/30 table=to_WAN_main

table bkp

/routing table
add fib name=to_WAN_bkp
/ip route
add dst-address=0.0.0.0/0 gateway="60.70.80.89" routing-table=to_WAN_bkp
/routing rule
add routing-mark=to_WAN_bkp table=to_WAN_bkp
add src-address=60.70.80.88/30 table=to_WAN_bkp

default gw

/ip route
add dst-address=0.0.0.0/0 gateway="20.30.40.49" distance=1
add dst-address=0.0.0.0/0 gateway="60.70.80.89" distance=2

kubiko · February 19, 2025, 7:02pm

Thank You for hint but it is similar as my tested config, but I tried to use routing mark instead of routing rules.

Temporarily I test to mark exactly only connection to VPN on desired port but even in this case, the response packets from the VPN server is still routed to WAN1 instead of WAN2 with source address set for WAN2.

Everything works as desired, ping from outside to WAN2 or access to the internet for desired machines through WAN2 but for services where I use DST-NAT it does not work…
For to_WAN_bkp routing table I have set only routes related to WAN2 IP address, I don’t understand where the mikrotik gets that it should send packets to iface WAN1 when I have no record of how to get to WAN1 for the routing table to_WAN_bkp.

anav · February 19, 2025, 7:13pm

Sorry the way you express requirements doesnt compute.
It would appear you have two WANs.

Are you saying external users are coming in over both WANs to servers on your LAN side?
Are you saying that some internal users should use WAN1 and some should use WAN2?
Are you saying you have VPNs connecting to the router and if so through which WAN?? ( type of vpn )
without a complete understanding of the needed traffic flows, the questions cannot be answered.

Marking connnections and Routing those markings are used in mangling.
Not sure where filter rules comes into this??
Filter rules simply allow traffic flow if attempted, they have no routing powers…
They can be used to deny access to an interface is probably the closest thing

kubiko · February 19, 2025, 8:03pm

Thanks for patience.

It would appear you have two WANs. → YES, I have two WANs, WAN2 was recently installed and several services should be migrated to use WAN2

Are you saying external users are coming in over both WANs to servers on your LAN side? → YES
Are you saying that some internal users should use WAN1 and some should use WAN2? → YES - working as expected
Are you saying you have VPNs connecting to the router and if so through which WAN?? ( type of vpn ) → OPENVPN, through WAN1 but I want to switch to be accessible through WAN2 but there is the mentioned problem

packet flow is following: packet is RX-ed to/by eth2-WAN2 then is DST-NATed, then packet is TX-ed via ethx-LANx to OVPN server.
OVPN server responds and response packet is RX-ed by ethx-LANx, then reverse DST-NAT is executed in the mikrotik, (OVPN’s local address is back-replaced by eth2-WAN2 IP address) and then response packet is TX-ed, but not over eth2-WAN2 but over eth1-WAN1. eth1-WAN1 has absolutely different address set - so I am confused.

Marking connnections and Routing those markings are used in mangling.
Not sure where filter rules comes into this??
Filter rules simply allow traffic flow if attempted, they have no routing powers… BUT filter rules also have parameter routing mark… so you can command to which packets the filter rule will be applied and one of my question is if it is true that when this parameter is not set in filter rule, then filter rule will be applied to packets regardless to routing mark. It sounds logical but I want to be sure.
They can be used to deny access to an interface is probably the closest thing

anav · February 19, 2025, 9:30pm

You are dabbling in uncharted waters, in that I have never used any markings in firewall rules, so I will bow out gracefully for someone with deeper experience.

In terms of VPNs, my recommendation is not to have any priority WAN.
As you have done two tables and four routes ( two routes main, one route to each table )
mangle input and output for both WANS → for services to the router
mangle forward and prerouting both wans bound for LAN → for servers behind the router
mangle accept traffic from both servers → so traffic goes to where it needs to go…
That should do it.

If traffic out specific wan is by subnet use routing rules to do so.

panisk0 · February 19, 2025, 9:31pm

Looks like we got a ‘man of the pen’ here

kubiko · February 20, 2025, 12:23am

Now I resolved the issue!!
Problem was that there is not any automatic routing for response to DST-NATed packets and this packets needs to be routed separately.
Because VPN’s WAN iface is on separated LAN DMZ (as mentioned ETHx-LANx), therefore there was a need to route also response packets comming from VPN to mikrotik.
So for this response packets was used default route to WAN1 and because mikrotik’s NAT recognized this packet as backward / return packet, therefore source IP address was correctly replaced by WAN2 IP…(my opinion) so it looked confusing - IP address of WAN2 but packet was transmited from WAN1.

When I force route outcoming VPN’s connection to WAN2, there is no option to connect to VPN through WAN1 - now is situation exactly opposite. So interesting task is to distinguish and split the outcoming communication from the VPN server and route it to appropriate WAN according to from which WAN iface communication was initiated to the VPN server. Probably I can use autofilling dynamic IP address lists based on WAN iface and according to the address membership in list I can route the traffic to correct WAN. Or Maybe I can use two mikrotik ifaces for connection between mikrotik and VPN server but I have to do this with multiple services and I would need many ports, or I can use VLANs… but this makes everything more complicated for me.

Yes, man of the pen… how funny. All I wanted to do was describe my situation in general and so that even a person who doesn’t have Mikrotik education but understands networks can understand the situation. Really, the configuration for my Mikrotik is quite large and I only needed to focus only on the important.

I really prefer to read the text than sometimes researching lengthy Mikrotik configurations in other threads and only then find out if it applies to me at all and is useful to me.