MikroTik

Hi all

I've come across what seems to me to be a bug/issue, whereby if I have no default route (0.0.0.0/0) on main routing table that is marked as "active", then default routes on all other routing tables do not work (internet is inaccessible)

The problem is remedied by creating an "active" default route on the main routing table which does not even need to be pointing to a working route/internet connection

For example, I am fortunately able to work around this behavior by creating a dummy gateway with high metric default route pointing nowhere, as follows:

/interface virtual-ethernet add name=vif1
/ip route add dst-address=0.0.0.0/0 gateway=vif1 routing-mark=main distance=100

This workaround is fine for me, but seems like it shouldn't be necessary as it was unexpected that I would run into this issue, which took some time to diagnose

Any ideas why this happens? Is this known/intentional operation?

I've tried searching the forums but can't seem to find any posts relating to this issue

Thanks !

You are providing far to little information to say anything meaningful about it.
Likely there are other things involved like IPsec.
And it also depends on what your route rules and firewall mangling rules (w.r.t. routing) look like.

Please provide a representative export showing what you are doing. (and make sure that the issue can still be reproduced when you cut that down to some example)

Hi
Thanks for response
However I don't think it's too little information, on the contrary I think I've kept it concise for easy understanding. I can replicate this on a newly reset system without default configuration.

Steps to reproduce:

1. Start from a clean slate to keep it simple for you.
/system reset-configuration no-defaults=yes

2. Assume internet router at 192.168.0.1 and our mikrotik is 192.168.0.2, and configure basic connectivity accordingly
/ip address add address=192.168.0.2/24 interface=ether1

3. Create new routing table and tell it which interface is the gateway
/ip route add dst-address=192.168.0.0/24 gateway=ether1 routing-mark=foo

4. Assign gateway to the new routing table
/ip route add dst-address=0.0.0.0/0 gateway=192.168.0.1 routing-mark=foo

5. Try to ping via gateway through that routing table
/ping address=8.8.8.8 routing-table=foo

Request times out, unless there is an active (black, not blue or red color) "0.0.0.0/0" route on the main table, even if that main default route doesn't provide internet
Ok it works. you are right that something else is causing my problem...

You ve set Routing Marks, so there is no Main Routing Table...
It is actually your mistake...
Tell the Router to use the Table named "foo" and everything will work just fine...

Ok wait.. actually I stand corrected. I was logically convinced that the behavior was as described based on what I was experiencing, however I just tested my own setup as described in my last post, and it is actually working now as expected. I will need to do further tests to find the real cause of the problem. I apologize for any confusion and will share findings. I suspect something in the firewall rules caused this

You ve set Routing Marks, so there is no Main Routing Table...
It is actually your mistake...
Tell the Router to use the Table named "foo" and everything will work just fine...

Yes I think this is it. Ok I will try to find a workaround as the solution is not immediately obvious to me (given that I am using routing marks as you describe). What I'd like to know is, why does creating a bogus default route on the main routing table otherwise solve my issue?

why does creating a bogus default route on the main routing table otherwise solve my issue?

No it does not solve the issue...

I have just reproduced the error again. i will post details in a moment please bear with me. i am not going insane after all

I have just reproduced the error again. i will post details in a moment please bear with me. i am not going insane after all

I reproduced it as well and it gave me the expected result...

Try to disable the default route on the non-main routing table, and re-enable it. The pings will not flow until you add a default gateway on main (which doesnt even need to be "routing" anywhere, it just needs to be a "black"(active) 0.0.0.0/0 route on main table). There is the problem. You can trigger it by disabling the default route on "foo" and re-enabling it. Something is not right!

I did disable/enable and again i do get the expected result...
What is your ROS versions?

v6.46.5

In my case the route is via PPPoE-client. I don't know why I'm having intermittent troubles reproducing this issue but I definitely experienced it. I will try to produce a consistent result and advise. Thank you for your assistance so far I do appreciate it.

Okay I think the problem is caused by "/ip settings set rp-filter=loose". I am still investigating. Regards

Yes. Try to enable loose rp filter, and then disable and re-enable the route. It is guaranteed to produce the behavior I describe, even without PPPoE client. Maybe my understanding of rp-filter needs some more learning, but I still cannot understand why adding a bogus default gateway to main should resolve this

Edit: Same thing with strict RP Filter.

So in summary, I don't understand why enabling rp-filter should cause a custom routing table to stop operating if main is missing a default route, or why adding an "arbitrary" default route to main should cause that custom routing table to operate as normal again. So yes, pe1chl was correct that I was not providing enough information. The information in this case being that rp-filter is on a non-default setting. However, I still suspect that this is a bug after all. Please share feedback, as I'd like to know if you are now seeing this as well ?

Ok, it's pretty clear what's going on now. Your routing works as expected. It is not your outgoing ICMP echo-request packets (pings) that are being mis-routed and/or discarded, but rather incoming ICMP echo-reply packets get rejected by your rp-filter.

The rp-filter=strict works by checking if the source address from incoming packet can be reached via the interface where that packet came from. It takes into account your routing rules and routing tables (not sure about mangle rules, but I think those are unlikely to be considered). It won't do lookup in your custom routing table unless instructed by routing rules. Whatever you specified in your ping command is never considered by rp-filter, and that's as expected.

Ok, it's pretty clear what's going on now. Your routing works as expected. It is not your outgoing ICMP echo-request packets (pings) that are being mis-routed and/or discarded, but rather incoming ICMP echo-reply packets get rejected by your rp-filter.

The rp-filter=loose works by checking if the source address from incoming packet can be reached via the interface where that packet came from. It takes into account your routing rules and routing tables (not sure about mangle rules, but I think those are unlikely to be considered). It won't do lookup in your custom routing table unless instructed by routing rules. Whatever you specified in your ping command is never considered by rp-filter, and that's as expected.

Thanks for this breakdown. I do understand much better now. At least the mysticism is gone, we have pinpointed the cause and have something to work with now

However I suppose that my question still stands though, about why adding a bogus default gateway to main routing table, corrects the timeouts? And then I guess I would also like to know, what would be the preferred way to address this besides for adding such a route?

It's obviously not ideal to add a virtual interface and set up a default gateway via that interface...surely? On the one hand, yes I am glad/fortunate that it works, as I know it's a dodgy workaround that for all intents and purposes, probably shouldn't be working... but on the other hand I'm just wondering what the best approach for dealing with such a scenario would be, as it seems I'm torn between two less than ideal alternatives - namely disabling rp-filter completely (bad for security of course), versus needing to add a seemingly pointless/"irrelevant" route that would never actually be used for anything, for no other reason than to ensure that everything else works as expected? :/

I'm sure there's an urge to suggest "Why would you not want to have any default gateway on the main routing table anyway?", but I should stress that I think that is besides the point. I find this behavior somewhat counter-intuitive and baffling, in spite of the explanations thus far, so hoping we can add further conclusive clarity and put this to rest.

I hope this makes sense... Any ideas welcome:)

Oh and I do also have to make a point of saying that the level of support on these forums in terms of number of volunteers and their skillset, and TTL for responses on these forums has left me in awe. You guys are amazing. Cheers for that

/ip settings set rp-filter is generally unusable. it is too rigid and it cannot be monitored.
there is a way to mark packets failing rpfilter in the mangle table in generic Linux but RouterOS lacks that possibility.
with that marking it is at least possible to log and view packets that get dropped and make exceptions depending on such things as multiple route tables.

I recommend not using rp-filter.

/ip settings set rp-filter is generally unusable. it is too rigid and it cannot be monitored.
there is a way to mark packets failing rpfilter in the mangle table in generic Linux but RouterOS lacks that possibility.
with that marking it is at least possible to log and view packets that get dropped and make exceptions depending on such things as multiple route tables.

I recommend not using rp-filter.

I decided to enable rp-filter after watching Andis Āriņš's lecture, "TOP 10 RouterOS configuration mistakes". See https://www.youtube.com/watch?v=BkZHRD6svQU&t=38m32s

So while I am not fully satisfied with the suggestion to just disable it again, as I don't think that is the ideal solution we are all looking for; At the same time, I do feel that there's seemingly no use keeping something enabled that is only creating more problems than it solves, or that works in ways that are not fully understood (at least by me) under the hood. Like driving a car, configuring a router needs to be handled by someone who knows what they're doing or things are bound to go wrong. So I'm going to take your suggestion about disabling rp-filter and doing just that for now. Hopefully this thread can be revisited in more depth by an "rp-filter" wizard in time to come, who can persuade us why we should re-enable this function after all, but in order to do that they will first need to solve this problem as detailed, in a more elegant way than I had in creating arbitrary default routes... But until then, for sanity's sake, I will be keeping it disabled instead.

However I suppose that my question still stands though, about why adding a bogus default gateway to main routing table, corrects the timeouts?

Sorry, what I wrote above describes rp-filter=strict, not loose. I have just edited my message to correct this. For loose to pass packet it is only necessary that it is routeable via any interface (no matter which exactly) and adding "bogus" default route solves that routeability problem for you.

it seems I'm torn between two less than ideal alternatives - namely disabling rp-filter completely (bad for security of course), versus needing to add a seemingly pointless/"irrelevant" route that would never actually be used for anything, for no other reason than to ensure that everything else works as expected? :/

I fully agree with pe1chl that that rp-filter thing is dumb and is not flexible enough to be useful. If you are worrying about IP spoofing, you can in most cases build an explicit equivalent of rp-filter using /ip firewall filter rules.

Actually, rp-filter=loose does absolutely nothing in any router config where a default route is present and not explicit "unreachable" routes.
I usually put such routes in the table: In this case, rp-filter=loose would filter traffic with such addresses as source, but you could also make an address list for such sources and filter that with a single firewall rule.

rp-filter=strict could have some usefulness as it would prevent incoming packets from internet with a source on the local network, but it can be similarly filtered in the firewall as well. The main thing remains that you cannot see what rp-filter=strict does. There are no counters, there is no logging option. Not good.

I do not think the presenter in that video is correct in that it is so much faster to use this method. There will be a small difference but in the total scheme of things it does not matter.
(of course you should always use an address-list when you want to filter a group of source addresses, not separate rules for each of them)

Interesting DM, that the comments about the video do call into question some of the ideas proposed.

The moral of the story is trust the folks here that have much practical experience with many different setups.
Trusting a single Youtube video as the defacto source of true information is a fools game.

In addition the best ways to get accurate and efficient help is
a. describe your requirements without referring to solutions or configurations but in use cases.
b. /export hide-sensitive file=anynameyouwish
c. draw a diagram of your network

Recommend starting here:
https://help.mikrotik.com/docs/display/ROS/RouterOS

and of course many of mikrotik Wikis are still out there......
https://wiki.mikrotik.com/wiki/Main_Page

/ping address=8.8.8.8 routing-table=foo

This works when you specify routing table. But it doesn't when you don't. It's known problem and it doesn't depend on RP filter. If you check routing diagram, you'll see that packets from local process (K) first go to "routing decision" and it works with main routing table. You can use mangle rules to set different routing mark for packets (and have them use different routing table), but it won't help with router's own output, because that's in "output" box, which is after already failed routing decision, so it won't get there. You won't have this problem with forwarded packets, because they start at (I), then there's prerouting with mangle rules and routing decision comes after, so packets already have correct routing mark.