Routing rule not working

ishanjain · September 19, 2021, 3:59pm

I am on Routeros 7.1rc2 on RB450GX4.

I have a vlan-160 with network, 10.0.160.0/24.

I want ALL traffic from this VLAN to go over the lte-vlan. 1 way to do this is with mangle rules but since I want this for this entire network, I tried to do it with policy routing rules but it doesn’t seem to be working. All the packets are still going through the primary wan in the main routing table.

Routing rule

admin@Ishan's Mikrotik] > /routing/rule/print
Flags: X - disabled, I - inactive 
 0   src-address=10.0.160.0/24 action=lookup-only-in-table table=lte-failover

Short routing table

[admin@Ishan's Mikrotik] > /routing/route/print 
Flags: X - disabled, F - filtered, U - unreachable, A - active; c - connect, s - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, m - modem, a - ldp-address, l - ldp-mapping, y - copy; H - hw-offloaded; + - ecmp, B - blackhole 
     DST-ADDRESS        GATEWAY            AFI         DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW                                                                                                                                                                                                                                                       
As   ;;; Main WAN Route
     0.0.0.0/0          pppoe-bsnl         ip4                1    30           10 pppoe-bsnl                                                                                                                                                                                                                                                         
 s   0.0.0.0/0          lte-vlan           ip4                2    30           10 lte-vlan                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
Ac   10.0.150.0/24      vlan-150           ip4                0    10              vlan-150                                                                                                                                                                                                                                                           
Ac   10.0.160.0/24      vlan-160           ip4                0    10              vlan-160                                                                                                                                                                                                                                                           
Ac   10.11.11.0/24      personal-vpn       ip4                0    10              personal-vpn                                                                                                                                                                                                                                                       
Ac   100.0.0.0/8        lte-vlan           ip4                0    10              lte-vlan                                                                                                                                                                                                                                                           
Ac   <bsnl-wan-gateway>/32    pppoe-bsnl         ip4                0    10              pppoe-bsnl                                                                                                                                                                                                                                                         
Ac   192.168.1.0/24     ether5             ip4                0    10              ether5                                                                                                                                                                                                                                                             
As   192.168.8.1/32     lte-vlan           ip4                1    30           10 lte-vlan                                                                                                                                                                                                                                                           
As   0.0.0.0/0          10.11.11.1%pers... ip4                1    30           10 10.11.11.1%personal-vpn                                                                                                                                                                                                                                            
As   0.0.0.0/0          100.0.0.1%lte-vlan ip4                1    30           10 100.0.0.1%lte-vlan                                                                                                                                                                                                                                                 
As   0.0.0.0/0          pppoe-bsnl         ip4                1    30           10 pppoe-bsnl                                                                                                                                                                                                                                                         
A H  ether1                                link               0
A H  ether2                                link               0
A H  ether4                                link               0
A H  ether5                                link               0
A H  personal-vpn                          link               0
A H  pppoe-bsnl                            link               0
A H  bridge                                link               0
A H  lte-vlan                              link               0
A H  vlan-150                              link               0
A H  vlan-160                              link               0

Detail routing table

[admin@Ishan's Mikrotik] > /routing/route/print detail 
Flags: X - disabled, F - filtered, U - unreachable, A - active; c - connect, s - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, m - modem, a - ldp-address, l - ldp-mapping, y - copy; H - hw-offloaded; + - ecmp, B - blackhole 
 As   ;;; Main WAN Route
    afi=ip4 
       contribution=active dst-address=0.0.0.0/0 routing-table=main pref-src="" gateway=pppoe-bsnl immediate-gw=pppoe-bsnl distance=1 scope=30 target-scope=10 belongs-to="Static route" 
  s   afi=ip4 contribution=candidate dst-address=0.0.0.0/0 routing-table=main pref-src="" gateway=lte-vlan immediate-gw=lte-vlan distance=2 scope=30 target-scope=10 belongs-to="Static route" 
 Ac   afi=ip4 contribution=active dst-address=10.0.150.0/24 routing-table=main gateway=vlan-150 immediate-gw=vlan-150 distance=0 scope=10 belongs-to="Connected route" local-address=10.0.150.1%vlan-150
 Ac   afi=ip4 contribution=active dst-address=10.0.160.0/24 routing-table=main gateway=vlan-160 immediate-gw=vlan-160 distance=0 scope=10 belongs-to="Connected route" local-address=10.0.160.1%vlan-160
 Ac   afi=ip4 contribution=active dst-address=10.11.11.0/24 routing-table=main gateway=personal-vpn immediate-gw=personal-vpn distance=0 scope=10 belongs-to="Connected route" local-address=10.11.11.2%personal-vpn
 Ac   afi=ip4 contribution=active dst-address=100.0.0.0/8 routing-table=main gateway=lte-vlan immediate-gw=lte-vlan distance=0 scope=10 belongs-to="Connected route" local-address=<lte-vlan-ip>%lte-vlan 
 Ac   afi=ip4 contribution=active dst-address=<bsnl-wan-gateway>/32 routing-table=main gateway=pppoe-bsnl immediate-gw=pppoe-bsnl distance=0 scope=10 belongs-to="Connected route" local-address=<bsnl-wan-ip>%pppoe-bsnl
 Ac   afi=ip4 contribution=active dst-address=192.168.1.0/24 routing-table=main gateway=ether5 immediate-gw=ether5 distance=0 scope=10 belongs-to="Connected route" local-address=192.168.1.2%ether5 
 As   afi=ip4 contribution=active dst-address=192.168.8.1/32 routing-table=main pref-src="" gateway=lte-vlan immediate-gw=lte-vlan distance=1 scope=30 target-scope=10 belongs-to="Static route" 
 As   afi=ip4 contribution=active dst-address=0.0.0.0/0 routing-table=via-personal-vpn pref-src="" gateway=10.11.11.1%personal-vpn immediate-gw=10.11.11.1%personal-vpn distance=1 scope=30 target-scope=10 belongs-to="Static route" 
 As   afi=ip4 contribution=active dst-address=0.0.0.0/0 routing-table=lte-failover pref-src="" gateway=100.0.0.1%lte-vlan immediate-gw=100.0.0.1%lte-vlan distance=1 scope=30 target-scope=10 belongs-to="Static route" 
 As   afi=ip4 contribution=active dst-address=0.0.0.0/0 routing-table=primary-wan pref-src="" gateway=pppoe-bsnl immediate-gw=pppoe-bsnl distance=1 scope=30 target-scope=10 belongs-to="Static route"
 A H  afi=link contribution=active dst-address=ether1 routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=ether2 routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=ether4 routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=ether5 routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=personal-vpn routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=pppoe-bsnl routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=bridge routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=lte-vlan routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=vlan-150 routing-table=main distance=0 belongs-to="Interface" 
 A H  afi=link contribution=active dst-address=vlan-160 routing-table=main distance=0 belongs-to="Interface"

anav · September 19, 2021, 4:14pm

Not sure why you post pictures,
one should post their config
/export hide-sensitive file=anynameyouwish

ishanjain · September 19, 2021, 4:21pm

I have not posted pictures. My post includes the routing rule, short and detailed version of the routing table.

I will have to remove a lot of stuff if share the exported config. Is anything else other than this needed? I will try to generate a version of the config I can share here.

sindy · September 19, 2021, 4:26pm

?
It’s not pictures, it’s proper text-mode prints of the actual routes. Export only shows you the static configuration, which is sometimes insufficient, especially in cases like this one where everything seems right configuration-wise. With RouterOS 7.x, you cannot refer to a routing table (routing-mark) unless you’ve explicitly created it before, so the fact that the value lte-failover can be seen in both the rule and the route says that a missing routing table configuration is not the issue.

The only thing I’d be interested in is export of the routing table configuration, some parameter may be wrong there.

ishanjain · September 19, 2021, 4:31pm

I have attached the complete configuration in this post. I am still working on the firewall filter rules so that’s probably a bit ugly section of the config…
routing.rule.help.rsc (22 KB)

anav · September 19, 2021, 4:40pm

Thanks Sindy, I did look at the pictures and I saw a horror show of ip routes., Glad they looked okay to you though for the Ops sake.
Yeah way over my head, pass!!.

New and interesting… using the INCLUDE rule in the interface members list!!

routing table fib??

What happens to vla30 on the bridge and yet associated with LTE but not defined for dhcp??
add interface=lte-vlan list=ExternalFailoverLAN
add interface=lte-vlan list=WAN

ishanjain · September 19, 2021, 4:50pm

While I agree, This is a mess in general but these are all the mangle rules.

There is only 1 mangle rule in the forward chain and I have that there to ease up the transition when the primary WAN is back online.

Without this rule The traffic currently passing through lte-vlan is not marked, So when the primary wan comes back online, All the connections active via lte-vlan interface are stuck because of the difference in distance in the main routing table.

With this mangle rule I mark all traffic so, When the primary wan does comes back up, Everything will keep working normally and the new connections will be setup over the primary wan. This was a decent QoL improvement so I added it.

ishanjain · September 19, 2021, 4:52pm

@anav

I have the LTE modem connected to an access port on the switch upstairs. This switch adds the tag 30 and the LTE traffic is brought in this VLAN 30 to the router. I don’t need a DHCP Client for this since there are only 2 devices in that VLAN.

New and interesting… using the INCLUDE rule in the interface members list!!

Yeah. I am not yet sure if I’ll keep that. I am redoing all the firewall rules. I wanted some way of applying some firewall rules to the LAN interface list and the LAB interface list so, creating a A(ll)LAN interface list was 1 way to do it.

After completing this rewrite, Maybe I won’t need that.. Don’t know that yet.

routing table fib???

I am not sure what are you referring to here.

sindy · September 19, 2021, 4:55pm

The following piece of configuration,
/routing table
add fib name=via-personal-vpn
add fib name=lte-failover
add fib name=primary-wan,
also seems fine to me. So if it works if you use mangle rules to assign the routing-mark, I’m afraid there must be something wrong with /routing/rule in 7.1rc2.

ishanjain · September 19, 2021, 5:04pm

Okay, Thank you so much for reviewing my config.

I have had bad experience with rc3 before with some random stuff just not working properly so I stayed back on rc2 but I’ll try rc3 again. I hope I don’t run into this issue in rc3, If I do I’ll probably roll back to rc2 and just use mangle rules to mark it all.

sindy · September 19, 2021, 5:13pm

Since multiple people have reported complete loss of configuration with 7.1rc3, I’d say don’t bother trying, use mangle, and try /routing/rule again in 7.1rc4 once it appears.

ishanjain · September 19, 2021, 5:16pm

Alright, Thank you!

These two vlans, 150 and 160 are for the lab. In this, I’ll only be running two instances of ripe-atlas so there is not much traffic. (Irregardless of that, There is some packet processing overhead because I am adding those mangle rules but should be fine)

ishanjain · September 20, 2021, 3:57pm

Hey @sindy,

I updated to rc4 and routing rules work again!

It was sort of glitch-y at first. 1 routing rule, src=10.0.150.0/24 action=drop was not working but src=10.0.160.0/24 action=lookup-only-in-table table=lte-failover was working.

So, I removed and re added both of them and now both of them work perfectly.

Thanks again for reviewing my config back then.

ThoKoch · December 22, 2022, 4:52pm

I have the same issue with RouterOS v7.6 (stable), it seems like the rules stop working once you change any of the attached entries (routing table, rules, routes), not sure if there is any relation to the complexity of rules in combination with the same routing table. Deleting and re-adding seems to be the only chance to get it working again.

alisalehiman · January 26, 2023, 10:32pm

On stable 7.7, the same problem still exists,please submit bug report.

warenbe · April 15, 2023, 5:14pm

Hi
i havve the same issue with 7.8
see here: http://forum.mikrotik.com/t/send-all-traffic-from-host-to-specific-gw-performance-issue/165995/4