Routing Bug or 'by design' ??

RouterOS General
1

Hi

I’m not sure if this is ‘by design’, but if it is, its fairly cumbersome if you have more than 1 routing mark or more than 1-2 routes. Here is the situation:

RB450G. RouterOS 3.3. 5 interfaces. All on different subnets. This is a simplified analysis of what we figured out in another thread (see bottom for original data).

[admin@hh.ii.jj.kk] /ip address> p
Flags: X - disabled, I - invalid, D - dynamic
#   ADDRESS            NETWORK         BROADCAST       INTERFACE
0   192.168.2.1/24     192.168.2.0     192.168.2.255   e2
1   192.168.0.1/24     192.168.0.0     192.168.0.255   e4-168-0-1
2   192.168.1.1/24     192.168.1.0     192.168.1.255   e3
3 D aa.bb.cc.dd/24    aa.bb.cc.0       aa.bb.cc.255    e1-gateway
4 D ww.xx.yy.zz/24    ww.xx.yy.0      ww.xx.yy.255   e5-gateway2

e1-gateway and e5-gateway are DHCP supplied WAN addresses, for different kinds of traffic

so your basic routing table looks like this (with aa.bb.cc.254 being the ‘active’ default route, just because it got activated first)

0 ADS  dst-address=0.0.0.0/0 gateway=aa.bb.cc.254 interface=e1-gateway gateway-state=reachable distance=0 scope=30 target-scope=10

1  DS  dst-address=0.0.0.0/0 gateway=ww.xx.yy.254 interface=e5-gateway2 gateway-state=reachable distance=0 scope=30 target-scope=10

2 ADC  dst-address=aa.bb.cc.0/24 pref-src=aa.bb.cc.dd interface=e1-gateway distance=0 scope=10

3 ADC  dst-address=192.168.0.0/24 pref-src=192.168.0.1 interface=e4-168-0-1 distance=0 scope=10

4 ADC  dst-address=192.168.1.0/24 pref-src=192.168.1.1 interface=e3 distance=0 scope=10

5 ADC  dst-address=192.168.2.0/24 pref-src=192.168.2.1 interface=e2 distance=0 scope=10

6 ADC  dst-address=ww.xx.yy.0/24 pref-src=ww.xx.yy.zz interface=e5-gateway2 distance=0 scope=10

now if you routing-mark some packets, like so (using any criteria for marking, it doesn’t matter:

[admin@hh.ii.jj.kk] /ip firewall mangle> p
Flags: X - disabled, I - invalid, D - dynamic
0   chain=prerouting action=mark-routing new-routing-mark=type_A passthrough=yes src-address-list=192.168.2.0/24

1   chain=prerouting action=mark-routing new-routing-mark=type_B passthrough=yes src-address-list=192.168.0.0/24

and now if you setup default routes for the marked packets, like so (with some failover), so that all packets marked ‘type_A’ go through e1-gateway, and all packets marked ‘type_B’ go through e5-gateway (with dual failover for each). Note these were ‘inserted’:

2 A S  dst-address=0.0.0.0/0 gateway=aa.bb.cc.254 interface=e1-gateway check-gateway=ping gateway-state=reachable distance=1 scope=30 target-scope=10 routing-mark=type_A

3   S  dst-address=0.0.0.0/0 gateway=ww.xx.yy.254 interface=e5-gateway2 gateway-state=reachable distance=2 scope=30 target-scope=10 routing-mark=type_A

4 A S  dst-address=0.0.0.0/0 gateway=ww.xx.yy.254 interface=e5-gateway2 check-gateway=ping gateway-state=reachable distance=1 scope=30 target-scope=10 routing-mark=type_B

5    S  dst-address=0.0.0.0/0 gateway=aa.bb.cc.254 interface=e1-gateway gateway-state=reachable distance=2 scope=30 target-scope=10 routing-mark=type_B

the end result, is that NO PACKETS WITH ANY ROUTING MARKS will ever go to the 192.168.0.0/24, 192.168.1.0/24, and 192.168.2.0/24 default routes. They should, shouldn’t they ? It appears the deafault route for marked packets trumps the IP address for any unmarked packets. I think this is not correct.

The only way a marked packet will ever go to any of the pre-established routes is by adding something like, FOR EACH AND EVERY MARK, FOR EACH AND EVERY ROUTE:

add dst-address=192.168.0.0/24 gateway=192.168.0.1 routing-mark=type_A

The end result is you have to manually add routes, for each instance of any routing mark, to each instance of any route you have. This shouldn’t be this way, or if it is, its a huge PITA if you use any routing marks with more than a very simple setup. Maybe there is a switch or setting for this ?

p.s.
here is the original analysis thread: http://forum.mikrotik.com/t/routing-between-seperate-protected-lans/32005/1

2

Build an address-list that contains all the local networks:

/ip firewall address-list
add list=Local_networks address=192.168.0.0/24
add list=Local_networks address=192.168.1.0/24
add list=Local_networks address=192.168.2.0/24

Change your mangle rules that set routing marks to only set marks for traffic that isn’t destined to local networks:

0   chain=prerouting action=mark-routing new-routing-mark=type_A passthrough=yes src-address=192.168.2.0/24 dst-address-list=!Local_networks
1   chain=prerouting action=mark-routing new-routing-mark=type_B passthrough=yes src-address=192.168.0.0/24 dst-address-list=!Local_networks

That way local traffic doesn’t have any routing marks and the most specific route (by IP) wins.

3

yes, there are workarounds, but isn’t the fact that the routes for unmarked traffic aren’t being followed (at all) a bug ? “all marked traffic needs explicit routes, and will not follow non-default routes for un-marked traffic”. or is there a such thing as routing-mark=*

PLUS, your workaround only sorts out the routing issue. We mark not only for routing but QoS. In other words, if typeA=higher prio than typeB, then we actually do need the marks. so normally we would do “if routing-mark=typeA then packet-mark=hiprio”. Now if that changes, then we need two set of different marks and mangle rules, one set of routing marks, and another set of packet-marks, plus their definitions. again, it can be worked around, but surely this cant be intentional ? how does this now work with the new ‘no-mark’… ??

4

I do think it’s intentional - routing marks are considered a more specific match than just by IP address. Makes sense for policy routing where you usually want to definitively override whatever decision would be made just based on the IP routing table alone.

If the above is true, I don’t think not marking traffic you don’t want to policy route is a ‘workaround’ - mark all traffic you want to policy route, leave all other traffic alone.

It’s my understanding ‘no-mark’ isn’t an actual mark attached to the connection/packet, it’s just a new way to identify connections/packets without marks set so you can simplify rule logic.

5

hmmm… src-address-list?.. replace with src-address - maybe it will start working %)

p.s. just a joke, I think…

6

no its just a typo when I was stripping the numbers. it DOES NOT work. the marks are not the problem, the problem is the routing logic where any packet with any mark must have all the possible routes defined for it.

having something like

add dst-address=ll.mm.nn.oo gateway=ll.mm.nn.pp routing-mark=*

makes for about 20 less routing instructions everytime you add a routing mark

7

http://forum.mikrotik.com/t/routing-between-seperate-protected-lans/32005/12

you should just understand how routing works