VRFs prevent me from using good firewall rules :-( What shall I do?

[divB]

It's crazy how every new thing created a boatload of new stuff that doesn't work: By just using VRFs, RouterOS effectively forces me to migrate everything to VRF (*). But now VRFs destroy properly designed firewall rules:

Started from version 7.14 when interfaces are added in VRF - virtual VRF interface is created automatically. If it is needed to match traffic which belongs to VRF interface, VRF virtual interface should be used in firewall filters, [...] If there are several interfaces in one VRF but it is needed to match only one of these interfaces - marks should be used.

WHY? Why this terrible change?? I could cry.

This leaves me effectively with just two very sub-optimal options:

1. Avoid filtering by interface completely
2. Using connection or packet marks

Good practice is not to use addresses but interfaces in filter rules. This is clean, avoids spoofing and such. Which rules out #1.

There can only be one conn/packet mark per packet which means if marking is used for any other purpose it interferes with firewall rules (which really should be filter rules, not mangle rules) and rules out #2

#2 has an additional issue: Good practice is to write firewall rules as "block everything and selectively allow" to avoid rules unintentionally passing traffic. But with marks in mangle it's extremely hard to follow the logic since these can be overwritten unintentionally. It's very easy to unintentionally rip holes into the firewall because a mark was overwritten later. It actually happened to me already.

And one more reason why this is terrible: When using interface lists, it can generate overlapping rules.

Hence I really don't understand how on earth anyone at Mikrotik could think this was a good idea.

I'm really stuck with my migration and going back and forth what I should do.

My current firewall is designed to simulate zones based on interface lists and jumping to various sub chains. Overall, it starts with rules that allow traffic from one "zone" to the other. Then more specific "traffic_rules" for the rest. I started to just create marks for each interface and interface list but quickly ran into conflicts based on what I wrote above.

Any words of wisdom?

PS: I use marking already for certain hacks and bug fixes (e.g. setting BFD source addresses, policy routing) incl queues/traffic shaping ... which also require marks for everything but super basic setups.

(*) because proper dynamic route leaking does not work with main VRF.

[rextended]

good firewall rules

This is just your statement, with NOTHING to support it.
Perfect.

[Larsa]

@divB: please provide an export of your config (minus serial number and similar sensitive stuff) if you want more precise support from the forum community.

[BartoszP]

@divB

There can only be one conn/packet mark per packet which means if marking is used for any other purpose it interferes with firewall rules (which really should be filter rules, not mangle rules) and rules out #2

You can remark already marked packets to have conditions "summed" and make rules depend on these final combined marks.

[Amm0]

I guess my question is why are you using VRFs in the first place? They have there place, but typically it's when you don't want to use the RouterOS firewall and want to use L3HW, or have VRF-based topology elsewhere. Otherwise VRFs are not "first class" citizens in RouterOS, so using them comes with some complexity...so if you don't need them, why use them?

[anav]

Concur with above, I think the OP was looking for therapy, not actual advice or assistance.

[divB]

@BartoszP

You can remark already marked packets to have conditions "summed" and make rules depend on these final combined marks.

Can you give an example what you mean by this?

@Amm0

I guess my question is why are you using VRFs in the first place? They have there place, but typically it's when you don't want to use the RouterOS firewall and want to use L3HW, or have VRF-based topology elsewhere. Otherwise VRFs are not "first class" citizens in RouterOS, so using them comes with some complexity...so if you don't need them, why use them?

I could in principle do it without VRFs but this comes with many other headaches, hacks and wild combos of mangle/nat/route rule/scripts etc. I have different internet uplinks, tunnels that need to be established over specific ones, network with public IP space that needs to go over a specific interface. I have worked with these with a non-VRF config over the years and find overall VRFs are way more elegant. It works very smoothly that each internet uplink lives in their own VRF, no issues with DHCP leases, wrongly assigned source addresses, wrong routing decisions etc. And my config shrunk 3x.

Sadly I have to agree, VRFs seem to be second class citizens with not many people using them and not en par with Cisco, Juniper etc.

Regardless, this question is more about the lesser evil of #1 vs #2, not VRF vs not VRF.

[Amm0]

@divB

Regardless, this question is more about the lesser evil of #1 vs #2, not VRF vs not VRF.

Fair enough. Personally, I'd say route tables and /routing/rule is "easier"/cleaner as a general point, since I try get "mangle" needs as minimal as possible. If RouterOS has more direct support for more "pure" VRF topologies... I might have a different opinion.

But you've come this far down the VRF route, so get your point.... So given the "lesser evil" question...you come to this:

@divB

Good practice is not to use addresses but interfaces in filter rules.

I think this advice is better/"correct" when you're more directly using the default firewall without VRFs. While the "interface-list" is helpful since it decouples both the IP subnet and interfaces, and also may "save rules" since you can apply same rule to a set of networks.

But at same time, /ip/firewall is a Layer3 construct, so while it appears to use interfaces... in reality it's just looking up the subnet associated with the elements in the interface-list. This could lead to casual user to think the firewall is blocking more than IPv4 stuff (which it isn't), so address-list or specific queries are sometime "more exact", which has some benefits for clarity.

So I would not get too hung up on the "using interface lists is always the right design"... Since that for sure will cause you trouble with VRFs, as you've seen.

[CGGXANNX]

@divB

Avoid filtering by interface completely

Since 7.23 we have this:

@EdPa

• firewall - added "action=drop" to mangle;

Which means you can do some filtering in the mangle table, where the interface matcher behaves in the way that you want.

Of course, when dropping in mangle you should also use the connection-state=new condition, because usually there is no accept established, related rule at the top of the chains like in the filter table.

[divB]

@Amm0

I think this advice is better/"correct" when you're more directly using the default firewall without VRFs. While the "interface-list" is helpful since it decouples both the IP subnet and interfaces, and also may "save rules" since you can apply same rule to a set of networks.

But at same time, /ip/firewall is a Layer3 construct, so while it appears to use interfaces... in reality it's just looking up the subnet associated with the elements in the interface-list.

Uhm, is this true?

Not my understanding: Interface is really the interface over which the packet comes in, despite being a Layer3 interface, right?

For example, if a completely broken packet arrives on vlan33 with source address 0.0.0.0 and destination address 0.0.0.0 it would still match in-interface=vlan33

@Amm0

This could lead to casual user to think the firewall is blocking more than IPv4 stuff (which it isn't), so address-list or specific queries are sometime "more exact", which has some benefits for clarity.

So I would not get too hung up on the "using interface lists is always the right design"... Since that for sure will cause you trouble with VRFs, as you've seen.

So you don't think there is a security benefit wrt to IP spoofing?

Suppose for example, I have a vlan3 with 192.168.3.0/24, a vlan4 with 192.168.4.0/24 and vlan5 with 192.168.5.0/24. vlan3 can access vlan4 freely so I have the a rule like "in-interface=vlan3 out-interface=vlan4 connection-state=new action=accept". But then I have "in-interface=vlan5 out-interface=vlan4 connection-state=new action=drop".

If I were to change these into "source-address=192.168.3.0/24 destination-address=192.168.4.0/24 connection-state=new action=accept" and "source-address=192.168.5.0/24 destination-address=192.168.4.0/24 connection-state=new action=drop", then an attacker on the "untrusted vlan5" could just send a packet with source 192.168.3.10 from vlan5 which would not be dropped but accepted.

Understand that the return packet wouldn't reach vlan5 (because no route for 192.168.3.10 via vlan5) but there are cases where a single packet may be an issue already (e.g. UDP).

Do I misunderstand?

@CGGXANNX

Which means you can do some filtering in the mangle table, where the interface matcher behaves in the way that you want.

Of course, when dropping in mangle you should also use the connection-state=new condition, because usually there is no accept established, related rule at the top of the chains like in the filter table.

This is very interesting, thanks for sharing!

Sadly it's not perfect because it breaks the "disallow everything except" rule ... but I think I could some coarse "zone" (=interface) based filtering there.

[CGGXANNX]

Ok, I totally forgot that when the change was introduced, I briefly looked at it and already found out back then that this is a non-issue.

If we look at the warning: VRF | RouterOS Manual

Before RouterOS v7.14, firewall filter rules with the property in/out-interface would apply to interfaces within a VRF instance. Starting from RouterOS v7.14, these rules no longer target individual interfaces within a VRF, but rather the VRF interface as a whole.

Yep, it says right there, in-interface and out-interface. But also only that! What still behave normally are the in-interface-list and out-interface-list matchers! Let's see some tests:

In this test configuration, ether3 is in VRF vmhost with a static IP address 172.20.80.2. Both DNS and WinBox services are set to listen on the vmhost VRF. As expected, the input drop rule with in-interface=ether3 doesn't work, and DNS queries are processed normally, with the FW rule counter not incrementing. WinBox using 172.20.80.2 works too, but is not part of the screenshot.

But now I just have to create an interface list and put ether3 in this list, and use in-interface-list=VMHOST in the rule:

The rule now properly drops incoming requests on ether3. DNS queries (and WinBox using IP address) no longer work.

This applies to other chains too, of course:

Here the drop rule on input with ether3 doesn't work, but the one on output with the interface list is able to do its work.

Remember that if interface list was also broken with VRF and filter, then all those firewall configurations with "drop incoming from WAN" using the WAN interface list would have introduced vulnerabilities!

Conclusion: Just use interface lists as usual.

EDIT: To confirm that the interface list does not pull in all interfaces of the same VRF, the following test adds ether2 to the VRF too, and switch the interface list membership to ether2. The drop rule does not affect ether3, and the router can answer DNS queries and accepts WinBox IP connections:

Which means out-interface-list=VMHOST only acts on ether2, and not on the other member of the VRF (ether3).

[Amm0]

@divB

Uhm, is this true?

Not my understanding: Interface is really the interface over which the packet comes in, despite being a Layer3 interface, right?

I was a bit sloppy in comment. But if you use an interface as a /ip/firewall matcher, it's still only matching IPv4 packages, not all traffic on the interface was my point. I sorta implied there is "IP lookup" but AFAIK it's is matching on the ingres/egress interface... I meant the list of interfaces it looks up.

My singular point was more when you use interfaces as firewall matcher it can "look like" it's blocking all traffic. Since often folks ignore the IPv6 side and/or L2 protocols when trying "block VLANs".

@divB

So you don't think there is a security benefit wrt to IP spoofing?

Well, spoofing is different topic. But using address-list or specific subnet matchers is better way to express the allowed IPs for an interface, and drop all else... Your VRF situation is what make interface matching less of an option in addition to IP subnet/address-lists.

Now, I might have said rp_filter (in /ip/settings rp-filter) default of "loose" would catch some spoofing. But the default is now "none", and that's a pretty old school (but simple) way to handle spoofing.

@divB

Do I misunderstand?

I could be clearer. And I'm not the VRF expert ...

Just for multiwan uplinks I, personally, find the combo of /routing/rule and /routing/table sometime with ECMP... a pretty clean solution. And the intervlan blocking expressed as /routing/rules avoid needing to muck with the firewall for very much. So I believe you:

@divB

It works very smoothly that each internet uplink lives in their own VRF, no issues with DHCP leases, wrongly assigned source addresses, wrong routing decisions etc. And my config shrunk 3x.

Your key phrase: "tunnels that need to be established over specific ones" ... and tunnels is where the routing rules/tables do get tricky.

@divB

overall VRFs are way more elegant

Although you just move the complexity. Routing rules/tables are "elegant" too...until there not

[divB]

@CGGXANNX

Yep, it says right there, in-interface and out-interface. But also only that! What still behave normally are the in-interface-list and out-interface-list matchers! Let's see some tests:

OMG, you're saving my day week month a second time now! Thank you!

This really seems to work, also in my quick tests. I didn't even give this a thought because I was thinking there is a deeper underlying issue why filter on interfaces doesn't work.

Now this way it's really a non-issue, especially since I'm using only interface lists anyway.

Mikrotik should update their documentation to suggest this as a solution rather than the mangle hack ...

[Larsa]

@divB

Mikrotik should update their documentation to suggest this as a solution rather than the mangle hack ...

Yes, totaly agree. MikroTik documentation is generally too ambiguous in too many places, and this is another example.

[anav]

Glad to hear Larsa has volunteered to find the relevant section, in the new document approach MT is using and, provide an update on this functionality through the bug report system. :-))

[Amm0]

I'm not sure it's just a doc problem.

There is a logic problem in that it really should not matter if you specify interface indirectly via -interface-list or directly via -interface that is the core problem. Both should work the same. e.g. a interface-list should behave identically as if you expressed the same list as multiple firewall filters rules using an interface matcher per item in the list.

Now certainly if it going to be confusing, that does mean it for sure be documented. But it be better if it was not confusing in the first place.