Since routeros version 7.2.2. mark routing has a higher priority than routing/rules. version 7.2.3 and 7.3 also like this.
Is this a bug or a feature change?

Since routeros version 7.2.2. mark routing has a higher priority than routing/rules. version 7.2.3 and 7.3 also like this.
Is this a bug or a feature change?

its not clear yet, i believe bug no much information about that in changelog also, i stay in 7.2.1 until this bug fix

Mangle has now strictly the highest priority.

Mangle has now strictly the highest priority.

Why isn't there any information about this in the changelog.
This is a big change. Many people think this is a bug.

Mangle on Forward happen after Routing decision.
Is perfectly normal the behaviour that what is done on Routing can be replaced on Mangle.
Then the actual behavior on v7 is what is expected.
On previous v6, is like a bug, following the packet flow.

@rextended routing can be marked only in prerouting and output

For explanation how it works see this:
viewtopic.php?p=938483#p938483

The change was made to fix the problems marking VRF traffic with the mangle.
It was always not recommended to override mangle with routing rules, if you had this setup previously, then yes unfortunately you have a broken setup now.

I never really played with it there, but AFAIK Linux has this configurable (see "ip rule") and default is what RouterOS had previously. It's not necessarily the only right choice, it can be what RouterOS has now. The bad thing is changing something like this without any warning whatsoever. I'm pretty sure that many people with load balancing previously marked routing without excluding router as destination, something like:
And why not, it always worked. But now if they upgrade, they suddenly won't be able to access router. Most will probably have enabled access using MAC address, but it will still cause a lot of confusion. And again, the main problem, there was no warning.

Your mentioned setup will not break, there is no routing rules

Well, then maybe it's not exactly this but something related and unintended, but it does break. If destination is local address, then packet with routing marked like this no longer goes in chain=input, but in chain=forward. Something that previously was possible only using action=route.

And the intended change in routing rules, it's also slightly annoying. If I understand it correctly, it means that my handy foolproof shortcut (= use main routing table for selected destination no matter what routing mark it has):
is now dead. With a config like I posted, it nicely handled both other LAN subnets and hairpin NAT. It can be done differently by adding dst-address-list=!LANs dst-address-type=!local to mangle rules, so I guess it won't be a problem. But it will requires some config fixes. Oh well..

@Sob thanks for mention this problem, most of us choosing MikroTik because load balance, now the change break something important like this i will considered not upgrading to newer version, mrz say that change for fix VRF marking but honestly most of us dont even use VRF just a simple load balance like in version 6, the reason i upgrade to ROS v7 is wireguard and new queue type (Codel, Cake etc) well maybe is pain the ass but i will bear it

It was curious what was up with this and it indeed a nasty one. My config was also broken. Looking in the mangle pre-routing routing marking the counters did not increase. Only the one catching traffic that should be mangled but has not been not mangled and setting the TTL to zero, to avoid traffic going out a wrong exit.

In the new-mark-routing lines, I had the condition that traffic should be routing table main. Now traffic is not automatically put in table main first, and then you can change that to an other routing table.

My routing table looks like this:
I have had to remove all checks on routing table main and I have no replacement for main on the moment.

Is this the way to get the previous working back to have all traffic be default "main"?

@dioeyandika: Well, it definitely needs clarification what's intended and what isn't. But it's not hopeless, you should be able to do what you need, you'll just have to be more careful with what you mark routing for.

It was curious what was up with this and it indeed a nasty one. My config was also broken. Looking in the mangle pre-routing routing marking the counters did not increase. Only the one catching traffic that should be mangled but has not been not mangled and setting the TTL to zero, to avoid traffic going out a wrong exit.

In the new-mark-routing lines, I had the condition that traffic should be routing table main. Now traffic is not automatically put in table main first, and then you can change that to an other routing table.

My routing table looks like this:

Code: Select all

   Name FIB
   aaa   yes
   bbb   yes
D  main  yes
   wg-1  yes
   wg-2  yes

I have had to remove all checks on routing table main and I have no replacement for main on the moment.

Is this the way to get the previous working back to have all traffic be default "main"?

Code: Select all

/routing rule
add action=lookup disabled=no table=main

hem i thought remove all checks in FIB it wont show in mangle routing mark, and how about 0.0.0.0/0 destination are we need to delete this too in ip route?

If traffic is not route marked main by default anymore how do I see what the assigned routing mark is. Mikrotik writes the one with the strictly order/priority.

Then having a "no-mark" routing-mark in the mangle like as with connection-mark could then detect traffic not yet routing marked. Main is an mark, and now the same status as other routing marks except it dynamically generated. If I look in /ip/route loads of lines have main as default routing table.

@msatter: Can you give some more understandable simple example? I'm trying different things and so far I didn't find any problem with main routing table. Even if I do action=mark-routing new-routing-mark=main, it seems to work as if I didn't do any marking.

Filtering on active routing-mark:

Looking at routing-mark=main to see if there is already a routing mark present. When not then route-mark traffic

I don't see any change there. If I do this:
Then I get: 1-notmain, 2-notmain, 3-main in 7.4beta2 as well as in 6.49.6.

before: 1-main 2-not-main 3-main
now: 1-not-main 2-not-main 3-main

My problem is 1-not-main now because before all traffic was main. That is why we don't have no no-mark option in routing...as it is always marked according to ROS. That has been changed now since 7.2rc6 and before.

It looks like there was change between v6 and v7 and later in v7 there was another. I now tested two (edit: four) more and I see:

1-notmain, 2-notmain, 3-main in 6.48.6
1-main, 2-notmain, 3-main in 7.1.5
1-main, 2-notmain, 3-main in 7.2.1
1-notmain, 2-notmain, 3-main in 7.2.2

So yes, it's also between 7.2.1 and 7.2.2. And nothing in changelog that would warn you.

I would like to see a feature in /routing/tables, where there would be a checkmark for a table "add connected routes".
When set, it would add the interface connected routes (that are in table main) also to this table.

I think that would solve a lot of the issues that people now encounter with the changed route mark behavior. A packet with a route mark would still be able to be locally routed, because local connections appear in that table (when you wish).
Do others agree that this would be a welcome addition?

I would like to start with clarification about current state, all the things that were changed and how, what's intended behaviour (because at least something may not be), etc.

Adding connected routes to other routing tables sounds possibly useful, but I'm not sure how it would interact e.g. with VRF where there are also connected routes, but in other than main routing table. So if it should be only for connected routes in main routing table, or also for others. It's complex stuff, I can't say that I understand all details.

Thanks for confirming this. I am back to 7.2rc6 due to a problem with IPTV streams breaking and now that gives problems. :(

Update:
Found the cause why the IPTV steams where breaking up. It was quite a search and even WireGuard did not give away why. Using torch I noticed that traffic was returning that had as destination the VPN gateway. That traffic intermingled with the IPTV traffic and changed the destination address.

The traffic that caused it where pings to test if the connection was still open and those where generated on the router itself. As soon as I deactivated ping the IPTV stream was rock solid again.

I am using a lookup only routing for IPTV traffic and the ping uses the same table. Strange that parallel traffic has this effect on traffic that has a different destination address.

Has anyone a clue how this can be avoided without disabling the pings I use?

addition: the destination address (traffic returning) is in the same /24 range as the VPN gateway. This way I don't have to use NAT for this traffic. It could be that the VPN provider prefers to return traffic to the gateway when things are not clear to them.

Table can be vrf and adding the same connected routes to every table is very bad idea especially if the table is vrf.

When you use VRF this option would of course not be chosen.
Please make a documentation page that describes exactly how the route marking, routing rules and VRF operate together, including all priorities and possibilities to mix them (or things you cannot mix).
In our network, the use of VRF often results in transmissions from router with wrong source address (traffic to tunnels with outside address as source), and I want to know exactly what can be done and what not.

Today I have received promising reply to my SUP-81294:

Hi,
The issue is still unresolved that could cause this, we are working on possible fixes. I cannot give you any specific ETD for this, unfortunately.
Best regards,
Oskars K.

So it seems that this undocumented routing priority changes they understand as an "issue" I stay on 7.2.1 or 6.49.5 and waiting...

"issue" in a sense that many expect old behaviour. Probably this will be solved by making order selectable.

"issue" in a sense that many expect old behaviour. Probably this will be solved by making order selectable.

So now it sounds like an intention from your side. Then I do not undestand why this major change was not documented in fw 7.2.2/7.2.3 change log (?) and why you cannot provide step-by-step help what to change in related config of i.e. dualwan scenario to be working again?

With current version, if you don't want something use another routing table, then don't mark routing for it. So e.g. if problem is access to router like in this example, then add dst-address-type=!local to first two rules.

With current version, if you don't want something use another routing table, then don't mark routing for it. So e.g. if problem is access to router like in this example, then add dst-address-type=!local to first two rules.

Thanks but I cannot see the posiblity to add this option to rule via Winbox?

It is on the "Extra" tab. Once you open the Dst.Address Type it will show the options.

Could you show me some screenshot? I have no Dst Address Type option there under ip/route. Maybe is not available on 7.2.1? (I am trying to add this option first before the upgrade now)

Eh..but it is firewall rule. First two rules in the refered example are for /ip route , not /ip firewall

Was referring to the mangle rules in the reply from sob (in the linked post):

Sob wrote: With current version, if you don't want something use another routing table, then don't mark routing for it. So e.g. if problem is access to router like in this example, then add dst-address-type=!local to first two rules.

Was referring to the mangle rules in the reply from sob (in the linked post):

Sob wrote: With current version, if you don't want something use another routing table, then don't mark routing for it. So e.g. if problem is access to router like in this example, then add dst-address-type=!local to first two rules.

OK, did a try, it solves unreachability of the router itself but still issue with nonworking routing rules under Routing/Rules ?

OK, did a try, it solves unreachability of the router itself but still issue with nonworking routing rules under Routing/Rules ?

Because in this release, the routing-mark assigned by a mangle rule cannot be superseded using a routing rule. So the rules are not "nonworking" per se, it's just that their purpose was to supersede the verdict of the mangle rules and they can't do that any more.

We'd have to see the configuration export to say how exactly the mangle rules need to be modified to gain the same effect you currently obtain by overriding them using routing rules.

Because in this release, the routing-mark assigned by a mangle rule cannot be superseded using a routing rule. So the rules are not "nonworking" per se, it's just that their purpose was to supersede the verdict of the mangle rules and they can't do that any more.

We'd have to see the configuration export to say how exactly the mangle rules need to be modified to gain the same effect you currently obtain by overriding them using routing rules.

I have a static routing and related routing rules to reach directly (not via internet) some public IPs assigned to local devices or other routers in our network but also some sort of local subnets and device IPs to reach localy...all of them does not work. As I mentioned above, if developers do silently this major change, I would expect BIG warning and step by step manual what need to be check and change in the configuration to meet this new router behaviour. Now it seems that they will provide some option to switch router to the previous standard behaviour.

As mentioned before, I hope the actual behavior will be well-documented. It does not matter that much what it exactly is, as long as we know what it is without blindly trying.
(this is about routing rules, routing marks, and also VRF)

As I see it (corrections welcome), previously it was:

- if destination is local, send it to router (input chain; routing rules and routing marks are ignored) (#1)
- check routing rules (can override routing marks) (#2)
- if there's routing mark, use given routing table
- otherwise use main routing table

Currently it's:

- if there's routing mark, use given routing table (is there's default route in this table, it's the end)
- check routing rules
- otherwise use main routing table

So there are two problems:

#1 - this is gone, local destination is no longer special
#2 - you can no longer override routing marks using routing rules

I don't know about VRF, that thing is kind of weird as whole (but I admit that I'm probably wrong).

Yes, what I want is a complete picture that includes VRF, because like you I have never been able to design a system using VRF that does exactly what I want.
I think it would be possible when I exactly knew the interaction between VRF, routing rules, and routing marks.
(i.e. how you can use routing rules and/or routing marks to partially override the operation of VRF)
The existing documentation suggests that it would be possible, but doesn't tell how. And it is too much work for me finding out by poking in the black box.

At the moment I run a VRF-like configuration using routing rules with source interface as a matcher, but it requires additional rules based on source address because "local router originated" traffic cannot be matched that way. And unfortunately it also requires manual copying of connected routes to the second table.
With VRF it could be better, but unfortunately it also makes wrong decisions w.r.t. source address selection, at least as far as I observed.
With a combination, it could be OK.