Policy routing confusion

Hi,

for a while now I’ve been trying to find a good setup in ROS to cover our needs (redundant VPN with policy routing/failover via Multi-WAN, details are not too important). While I have some generally working solution, I would prefer another one, but I’m stumbling about one certain problem again and again.

I will give an example for what I would like to do, and I do on purpose not just post a config, as I intend to show a more abstract picture, not just focus on 1 special config.

Connection from LAN should be routed through one of 2 GRE tunnels, and as different type of traffic should go through different tunnel, use Routing-Marks.

So I do the following: Packet “P” from PC to Router → mangle Prerouting, mark connection “CON1”-> mangle prerouting, mark routing “RM1” if connection mark = “CON1”
Add route “R1” with routing-mark “RM1”, ping check, distance 1 → gateway1
Add route “R2” with routing-mark “RM1”, ping check, distance 2 → gateway2
Add route “R3” with no routing-mark, ping check, distance 3 → gateway2
Add route “R4” with no routing-mark, ping check, distance 4 → gateway1
(Dst-address is always same, gateway is same for R1/R4, and for R2/R3 … so traffic type with RM1 would be preferably routed via gateway1, all other traffic would preferrably be routed via gateway2)


Now when I look at this:

http://wiki.mikrotik.com/wiki/File:IP_final.png

I would expect the after pre-routing, when the routing decision is done, it would match the route with best match, which should be R1 … at it matches the dst-address AND the routing mark AND shortest distance.
However, when I then log the packet “P” in forward chain, it will show that the out-interface is gateway2. So it selected “R3”. Even worse , if I disable R3 and R4, it will not work at all, which shows to me that the “Routing decision” after “PreRouting” does care about routing marks. And after that point i don’t see any chance to force “P” into the correct tunnel. :frowning:
Am I doing anything wrong, am I misunderstanding something, or this even a bug? Are there better ways to achieve this kind of policy routing?

Thank you in advance for your answers.

I may have to add something …

after doing config reset on my test MetaRouters, I applied my test scenario fresh from start, and this time it seemed to work.

But then I applied IPSec tunnel policy, and here is where it starts to act up
(I swear i had the problem before even without IPSec, and not only in one configuration … but maybe something got screwed up throughout my tests , who knows)

So what I see now is … if IPSec policy is active, the log entry in the forward mangle chain will show the wrong out-interface, even though IPSec processing should follow only later in the packet flow. And I feel here is is where my trouble starts. Because then after IPSec is done, it won’t change the out interface anymore , even though the routing mark seems to survive the IPSec encapsulation (log in mangle chain Output, filter routing mark, the IPSec packet is logged … but the out interface does not get adjusted afterwards)

So maybe my question should be rephrased to: how can I do selective policy routing and combine that with IPSec?

My main idea for this looks like:

Traffic A GRE-TUNNEL1 → ISP1
\ /
IPSec
/
Traffic B GRE-TUNNEL2 → ISP2

(Failover must be possible for both A & B )

The GRE tunnels are because I have dynamic IPs in many locations, and dyndns is less compicated to do for tunnels then for IPSec.

Looking forward to any input you can give :slight_smile: