I’ve been getting a hang of firewall mangle and filter flow. After years of just copying Mkrotik wikis, I’m forced to learn to write my own rules. After reading manuals and wikis as good as I could, I still have a few unanswered questions:
How many packet/connection/routing marks can you add to one packet. Eg I use connection/route marks for load balance but would also like to shape P2P and QoS for VoIP. Would second connection mark override first one, or be added? Eg. if I want to mark according to load balance connection/route marks, but then also mark them to be shaped for QoS and/or other firewall rules like brute force attacks, etc?
2a) Would router automatically mark all packets related to a previously conn-marked connection with the same conn-mark during conn-track phase? If so, will it then also add previous routing mark automatically? If so, why isn’t there rules before these checks to passthrough=no for already marked connections? Say on the first packet, the connection is marked on second rule(wlan2_conn). The next packet that arrives, hits the first rule and it is the first in a ‘batch’ of two, will the first rule not then add/override with wlan1_conn mark? How will it stay marked as wlan2_conn? (Hope this makes sense)
2b) In http://wiki.mikrotik.com/wiki/NTH_in_RouterOS_3.x and in mentioned PCc wikis, there is no connection-state=new like in original Nth wiki. Why?
3a) When creating your own chains in mangle, if you passthrough=no in your custom chain, does it just exit that chain, or the default chain that jumped to it as well?
3b) How can you prevent or allow an accept/drop or passthrough=yes/no action to also continue/stop processing in the parent chain that called it? eg. in my own chain, sometimes I just want to exit somewhere in chain and continue in parent chain. Sometimes I want to stop all further processing on that packet.
Some very good questions. I will be in a place to answer some of these this afternoon. I am posting this message so that I have a “placemarker” to come back and find your post.
I would only guess that you can have only 1 type of each marking (1x packet, 1x connection & 1x route) per packet and that if you mark it again with the same type, that it would override the previous marking.
one mark of each type: connection, packet and routing. by the way, in last versions, default mark is ‘no-mark’ (except connection mark, see 2a). each marking overrides current mark
2a) yes, if you marked a connection then each packet of that connection (both client-to-server and server-to-client directions!) have that connection-mark in the beginning of Packet Flow. all other marks (routing and packet) should be applied manually for each packet)
2b) me too thinks that the example is not ideal. there _should_be ‘connection-state=new’ in connection marking rules
3a) ‘passthrough=no’ will completely exit the current Packet Flow step(either prerouting, or forward, or input, etc.), not jump out of current subchain
3b) if you need to stop processing of current step - use passthrough=no; if you need to return back to parent chain - use ‘action=return’
Yes, thanks from me to.
Two more questions on firewall:
eg. in postrouting, mangle, if I passthrough=no on a packet, will it still continue to src-nat (next step in postrouting), or jump completely out of whole postrouting process. i.e. jump only out of postrouting mangle, or whole postrouting process? In other words, how can I prevent src-nat except for my method below in 2?
What cause a connection to be connection-state=invalid ? I have a mailserver with public IP on one gateway’s subnet, and prevent clients masquerade only to that server so it can see client IPs (chain=src-nat dst-address=!123.45.67.89 out-interface=bye_bye action=masquerade) If I drop invalid connections, then dude shows http service for mailserver is offline but ping is ok.
First let me say that I GREATLY appreciate the way this question was asked. It is clear (based on your questions) that you have, indeed, studied the manual. This is the proper way to ask a question.
How many packet/connection/routing marks can you add to one packet. Eg I use connection/route marks for load balance but would also like to shape P2P and QoS for VoIP. Would second connection mark override first one, or be added? Eg. if I want to mark according to load balance connection/route marks, but then also mark them to be shaped for QoS and/or other firewall rules like brute force attacks, etc?
When you create a connection mark, that mark will be applied automatically to each packet in the connection. This connection mark is available to you in other parts of firewall. For example, if you mark a connection in prerouting, you will be able to match that packet using the filter in both forward and input or even in later mangles (such as forward/input or postrouting). Each connection can have only 1 connection mark.
It should be noted that you can have a packet that has a connection mark, a routing mark and a packet mark. It is possible to have one of each, but each “type” is limited to ONE. In other words, 1 [packet|connection|route] mark/packet. Another thing to keep in mind is that connection mark, once added, will be automatically added (as stated above) to future packets in this connection, however, packet marks AND route marks are only added to individual packets. I hope that does not add confusion.
2a) Would router automatically mark all packets related to a previously conn-marked connection with the same conn-mark during conn-track phase? If so, will it then also add previous routing mark automatically?
Answered above, but the answers are “yes” and “no” (in the order you asked).
If so, why isn’t there rules before these checks to passthrough=no for already marked connections?
The wiki does not always present the most efficient example. It’s purpose is to show examples of how things work. The specific article you posted is one that does a good job of showing HOW to do the task, but is not necessarily the most efficient method.
Say on the first packet, the connection is marked on second rule(wlan2_conn). The next packet that arrives, hits the first rule and it is the first in a ‘batch’ of two, will the first rule not then add/override with wlan1_conn mark? How will it stay marked as wlan2_conn? (Hope this makes sense)
This should not be the case, since it is already marked by the second. I have not tested this, but I believe that is the case. From my reading about PCC (both here and other places), I think the example you are asking about will work correctly, but is simply not the most efficient use of the PCC matcher. I may be able to spend some time testing this and see what I can come up with.
I don’t think it is necessary as I stated above. I DO think that the example is just that. Not a complete application, but an example showing how to design the rules using the PCC matcher.
3a) When creating your own chains in mangle, if you passthrough=no in your custom chain, does it just exit that chain, or the default chain that jumped to it as well?
This is a little more difficult to explain without writing a LONG answer. Here’s how this works:
Firewall rules (including mangle) are processed in order from the top down, according to which BUILT-IN chain they are in. In other words, if we are in “forward”, then rule 0, then 1, then 2, etc. in the forward chain will be processed in order.
when we encounter a matching rule that has action “jump”, the firewall will begin processing THAT chain from the top down (again, it order). IF we encounter a matching rule in that chain that has the action “accept” or “no passthrough”, then ALL processing of that packet in forward mangle will be stopped.
The packet processing will continue where it left the forward chain under 2 circumstances:
a. We encounter a matching rule that has action “return”
b. We have processed the last rule in the user chain WITHOUT seeing a matching rule with action “accept” or
“no passthrough”
I hope this is clear.
3b) How can you prevent or allow an accept/drop or passthrough=yes/no action to also continue/stop processing in the parent chain that called it? eg. in my own chain, sometimes I just want to exit somewhere in chain and continue in parent chain. Sometimes I want to stop all further processing on that packet.
Passthrough=no will ONLY stop processing in postrouting mangle. It does not prevent src-nat. An “invalid” connection state is only possible IF the router sees a tcp packet with the “ack” flag when there has been no previously seen tcp packet with the “syn” flag only. I can provide a more complete answer later if someone doesn’t beat me to it. Time for lunch!
First let ME say that I greatly appreciate your comprehensive answers.
Another thing to keep in mind is that connection mark, once added, will be automatically added (as stated above) to future packets in this connection, however, packet marks AND route marks are only added to individual packets. I hope that does not add confusion.
Understood.
The wiki does not always present the most efficient example. It’s purpose is to show examples of how things work. The specific article you posted is one that does a good job of showing HOW to do the task, but is not necessarily the most efficient method.
Which is why just blindly copying the wiki can cause some unexpected results. Don’t ask…
This should not be the case, since it is already marked by the second. I have not tested this, but I believe that is the case. From my reading about PCC (both here and other places), I think the example you are asking about will work correctly, but is simply not the most efficient use of the PCC matcher. I may be able to spend some time testing this and see what I can come up with.
I just added a route-mark and no passthrough on already marked connections before it reach that, which seems to work fine.
This is a little more difficult to explain without writing a LONG answer. Here’s how this works:
Firewall rules (including mangle) are processed in order from the top down, according to which BUILT-IN chain they are in. In other words, if we are in “forward”, then rule 0, then 1, then 2, etc. in the forward chain will be processed in order.
when we encounter a matching rule that has action “jump”, the firewall will begin processing THAT chain from the top down (again, it order). IF we encounter a matching rule in that chain that has the action “accept” or “no passthrough”, then ALL processing of that packet in forward mangle will be stopped.
The packet processing will continue where it left the forward chain under 2 circumstances:
a. We encounter a matching rule that has action “return”
b. We have processed the last rule in the user chain WITHOUT seeing a matching rule with action “accept” or
“no passthrough”
And you don’t call that a long answer… Thanks, I think (and hope) I get it now.
If you only use the packet-mark (in queue tree) in any case and each new packet get checked anyway (so you don’t need to preserve connection-mark across packets), why ‘waste’ a good connection mark if the following should then also work? (If my understanding is correct):
I’d say ‘either forward or input’ - connection cannot pass through both forward and input %)
simply add a rule at the top of NAT, that will accept packets to the mailserver (action=accept). that will prevent other NAT rules from processing these packets
well, I’m not fully understand how p2p matcher in firewall works internally, so I simply describe one of the reasons why it’s better to use connection-marks
for example, you need to simply match all packets against address-list, containing thousands of addresses.
but searching huge lists can be cpu-expensive, so we remember that we have connection tracking enabled and all packets are going through it =) one of conntrack functions is to preserve connection marks among packets, so we can simply lookup address-list for each new connection, then mark connection with needed mark - and use that mark to identify necessary packets, something like
With P2P especially, but with other protocols as well (ftp is a good example), the ability to detect the protocol involves more than one packet. The connection mark is used in this case. It is not necessary to use connection mark for these protocols even unless you are using NAT (source nat) on the router that is detecting the protocol anyway. The reason for needing connection mark with NAT is because the process translating the NAT when it “returns” happens AFTER mangle. In other words, consider the following communication sequence:
Packet1:
10.10.10.10->12.12.12.12 (initial connection OUT): SRC is natted to 12.12.11.1 (router’s IP), so that packet that LEAVES the router is: 12.12.11.1->12.12.12.12
Packet2:
12.12.12.12->12.12.11.1 (response from the initial connection). The NAT process will change the destination of this packet to be 10.10.10.10 so that the packet that leaves the router is: 12.12.12.12->10.10.10.10
HOWEVER, when you use mangle, mangle will see these packets as follows:
packet1: 10.10.10.10->12.12.12.12
packet2: 12.12.12.12->12.12.11.1
Using connection marking allows you to “associate” those 2 packets with one another. Again, it is only needed IF you are using NAT on the router that is doing the mangle.
Although it makes perfect sense, it adds a spanner to the works. A small detail with major consequences… In your example:
Packet1:
10.10.10.10->12.12.12.12 (initial connection OUT): SRC is natted to 12.12.11.1 (router’s IP), so that packet that LEAVES the router is: 12.12.11.1->12.12.12.12
Packet2:
12.12.12.12->12.12.11.1 (response from the initial connection). The NAT process will change the destination of this packet to be 10.10.10.10 so that the packet that leaves the router is: 12.12.12.12->10.10.10.10
HOWEVER, when you use mangle, mangle will see these packets as follows:
packet1: 10.10.10.10->12.12.12.12
packet2: 12.12.12.12->12.12.11.1
As the routing descision is only made between prerouting and forward, therefore in prerouting mangle, I cannot use my natted LAN IPs as dest-addr or dst-addr-list, nor my LAN interface as out-interface, only in forward-mangle and postrouting-mangle. (Correct me if I’m wrong) (only a problem for natted interfaces/IPs) Here I see the help that conn-tracking (& conn_marking) gives in prerouting where conventional rules can’t work.
This is a little gem, although logical, I haven’t though of it in that way.
Bruce, you should apply for a job to update Mikrotik manuals to give a bit more practical ‘meaty’ info than the technical-spec style they use at the moment. It is good, but does not give the nicer details in such a wholistic fashion like you and Chupaka do.
Thanks a million Chupaka & Bruce, I understand the firewall proceses much better now.
Butch, p2p matcher is actually L7 matcher, and I think it have nothing to do with natted addresses… anyway, you cannot detect p2p from the first packet - that’s why you cannot redirect p2p to another internet uplink, for example =)
huh… maybe Cisco calls it something like ‘best practices’? ))
It is a layer 7 matcher. The natted address issue exists BECAUSE it is more than one packet. The question was about how to follow p2p in mangle. IF you are natting AND you want to follow p2p in mangle AND you want to use the private address space as part of your match, THEN you MUST use connection mark. That is the point I was making.
Option “in-interface=Local” solve the One of the problems I was getting when tried this - packet from 1.1.1.1 to 2.2.2.2 have different PCC value than packet from 2.2.2.2 to 1.1.1.1, so we need to specify direction, but in this case we will miss all “connection-state=new” packets with “in-interface=Public”.
But there is one way to optimize it - to use new “Connection-mark=no-mark” default value in these rules.
Basically create a rule that states “If connection does not have connection-mark, then mark it.”
But there is one way to optimize it - to use new “Connection-mark=no-mark” default value in these rules.
Basically create a rule that states “If connection does not have connection-mark, then mark it.”
Although a bit more long-winded, I did something with similar net result in accepting all packets already marked, so as to prevent them from passing through the marking process again.
Something like
chain=prerouting connection-mark=mark1 action=accept
chain=prerouting connection-mark=mark2 actio=accept
chain=prerouting connection-state=new action=mark-connection… (Although I suppose connection-state=new is rather irrelevant in this case)
Now my understanding of the chains marking & pass-through is way better…