I try to configure load balance and failover at 2011UiAS-2HnD.
When i was at MTCNA, MTCRE courses it was easy to configure almoust all tasks. The problem is when sit at course you doing some task then going to blank configuration mikrotik and do another task. There is wasnt legacy between past and current task+ trainer is near so can ask advice when I cant do something. After courses in my head left questions that i did not have time to ask trainer.on courses and questions that comes to head after courses.
So. I have 2 ISP, both give me static IP and i have from them guaranteed speed. ISP1 give me 15Mbit\sec, ISP2 give me 10Mbit\sec.
I have resourses that remote users use when connect to mikrotik, i mean “dst-nat” to RDP.
Have filials that connect to this Mikrotik by VPN l2tp+ipsec
Ether1 - ISP1(1.1.1.1), main ISP
Ether2 - ISP2(2.2.2.2)
Ether4 - local network 192.168.0.0/24
Ether4 - VPN network 172.0.151.0/24, this network is for remote users that use more than dst-nat, they use VPN by this network so get access to all resourses that need them to work
CLI:
/interface bridge port
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=wlan1
Here question how use rule to get for LAN users access to internet. In default configuration there is in NAT default rule that use masquerade, something like this:
If i understand all right the first 4 raw is for correct working “dst-nat”.
Second, i cant understand in PCC what means numbers, for example when i use “both-addresses-and-ports”. What does means this numbers? How correct set up them? As i understand it is need when configure in my case 15Mbit\sec and 10Mbit\sec ISP’s, to do about 50/50 load. Or another, speed of ISP’s not equal each other, so need to do 2 parts of traffic to ISP1, 1 part of traffic to ISP2, as i understand?
Here i cant understand why we left “main table” rules, raw 5,6 . I try ask trainer about it but i dont get it.
When i try to connect to VPN with this settings and next step try to authorize to RDP it is work as expected, system ask me login/password to access to server, but i cant pass to RDP it is just freeze on step initialization. I think that i configure mikrotik not correct.
You remember correctly that a masquerade rule causes higher CPU load than a src-nat rule when the related interface goes down.
Here i cant understand why we left “main table” rules. I try ask trainer about it but i dont get it.
I did not get what you are asking. Did you mean “почему мы не ввели никаких правил для таблицы main”?
In fact, all routes without the ****
routing-mark
parameter belong to table “main”.
And in fact, you should not need any routes in table “main” because the way the routes and connection-marking is intended in your configuration, the uplink (actually, the routing table) should be stritcly assigned to each connection depending on the (address:port) combinations of source and destination thanks to the ****
matcher in the classification rules, so you do not need any rules in table main - no packet would use them. But several mistakes in your configuration contradict this concept, see below.
Here question how use rule to get for LAN users access to internet.
That’s a misunderstanding. ****
src-nat
or
masquerade
rules are not there specifically for LAN users; they are there for any traffic being sent out and matching the conditional part of the rule. So for your case in particular, this means traffic originating in LAN or VPN and leaving towards the internet via
ether1
or
ether2
, and you need to choose the appropriate public address to use as source, which is the job of these two rules:
would mean that anything sent to the LAN (ether4 or wlan) would be src-nat’ed to 1.1.1.1 (because both rules have identical match part so the packet would always matched by the first one), which is definitely not what you want.
Now to make your RDP access to the machine in LAN to work, what you need to achieve is that if a RDP session (or, in generic case, any other connection) comes in from the internet through ****
ether1
, the response packets of that connection would be sent out via
ether1
as well (and analogically for
ether2
). This is, as you have noticed, what the following rules are responsible for:
However, there is an issue - these rules are in chain ****
input
of table
mangle
so they only apply to packets with one of Mikrotik’s own addresses as destination. And this doesn’t apply to the RDP packets, because the rules in
dstnat
chain of table
nat
are processed before rules in chain ****
input
of table
mangle
, and once the packet gets dst-nat’ed, it becomes a packet for other than Mikrotik’s own address and thus needs to be handled by rules in chain
forward
of table
mangle
.
So this is the first reason why your RDP sessions have a problem - the connections are not marked up to their source interface when they come in. To fix that, you need to add another pair of same rules to chain ****
forward
of table
mangle
. It is possible that you could instead move the existing rules to chain
prerouting
of table
mangle
, but I’m not 100% sure here how the connection marking would work.
handle any packet which comes in from LAN (and only LAN, not VPN, because VPN has nothing to do with bridge1) and assign a new connection mark to it, even if the connection to which the packet belongs has already been marked before.
So even if the first pair of rules above would mark the incoming packet of connections properly, the rules we talk about now change the connection marks for their responses, making it a lottery whether the right WAN interface is chosen.
To get rid of overwriting the connection marks, add condition ****
connection-mark=no-mark
to all rules which assign connection marks. And to save CPU, add to them also another condition,
connection-state=new
. We could do some more optimization to save even more CPU but let’s first make it work.
i cant understand in PCC what means numbers, for example when i use “both-addresses-and-ports”. What does means this numbers? How correct set up them? As i understand it is need when configure in my case 15Mbit\sec and 10Mbit\sec ISP’s, to do about 50/50 load. Or another, speed of ISP’s not equal each other, so need to do 2 parts of traffic to ISP1, 1 part of traffic to ISP2, as i understand?
The processing is the following:
the addresses and ports indicated by the first parameter (in your case, ****
both-addresses-and-ports
) are hashed to form a 32-bit integer number
that integer is divided by the first number (in your case, 3)
the remainder of that division (which for divider 3 may be 0,1, or 2) is compared to the third number, and if it matches, the whole condition matches.
So the way you’ve set it up, 1/3 of the connections get one connection mark, another 1/3 gets another connection mark, and the last 1/3 either keeps a connection mark assigned before or gets no connection mark at all.
You have 15 Mbit/s from one provider and 10 Mbit/s from another, so instead of 2:1 distribution, 3:2 would be more appropriate. The figures only make sense if you expect similar traffic structure from all sources, but you’ll see the practical outcome. Also bear in mind that the distribution rules will affect only traffic originating from your LAN and VPN, not the incoming traffic.
To implement 3:2 for ISP1(ether1):ISP2(ether2), you could use five rules, three of them (say, 5/0, 5/1, 5/2) assigning ****
connection-mark=ISP1_routing
, and two of them (5/3 and 5/4) assigning
connection-mark=ISP2_routing
. But that would be a waste of CPU. Instead, use two rules with 5/something to assign
connection-mark=ISP2_routing
, and a rule following them without the
per-connection-classifier
at all, assigning
connection-mark=ISP1_routing
. As I’ve said before that all rules setting a connection mark must only match on packets of not-yet-marked connections, this rule will do the same job as the three other rules in the 5-rule variabt above.
“translate” the connection marks into routing marks. The point is that connection marks cannot be used directly for routing and are assigned once (when the connection gets established by its first packet), while routing mark must be assigned to every single packet, otherwise it would be routed using routing table “main”. So these rules must not contain the condition ****
connection-state=new
.
Rules in chain ****
prerouting
of table
mangle
do not handle packets originated from the Mikrotik itself, therefore you need to create the same rules in chain
output
, otherwise traffic like NTP or DNS would not be routed anywhere. This is relevant for both
action=mark-connection
and
action=mark-routing
rules.
Another point is that policy routing (=routing according to routing marks) is incompatible with fasttrack handling. So if there eventually is a rule in chain ****
forward
of table
filter
with
action=fasttrack-connection
, you have to disable it. Do not remove it, it will make sense to re-enable it later to do some optimisation, but let’s first make your whole setup run in the systematic way, and only then start optimizing.
So fix all the above and come back with a complete export of the modified configuration.
Sorry but if I ask it by parts it will be not understand fo me, problem that I need to combine it all together.
You remember correctly that a masquerade rule causes higher CPU load than a src-nat rule when the related interface goes down.
Problem that i cant understand how to correct use src-nat but not masquerade, lower post.
Here i cant understand why we left “main table” rules. I try ask trainer about it but i dont get it.
I did not get what you are asking. Did you mean “почему мы не ввели никаких правил для таблицы main”?
In fact, all routes without the > ****
routing-mark
>
> parameter belong to table "main".
> And in fact, you > **should not** > need any routes in table "main" because the way the routes and connection-marking is > **intended** > in your configuration, the uplink (actually, the routing table) > **should be** > stritcly assigned to each connection depending on the (address:port) combinations of source and destination thanks to the > ****
>
> ```text
per-connection-classifier=both-addresses-and-ports:3/2
matcher in the classification rules, so you do not need any rules in table main - no packet would use them. But several mistakes in your configuration contradict this concept, see below.
No, the problem is why it cant works without rules in main table. If we marking traffic so there is no more place for main table rules, it is useless i think, but in fact need to use it to work, maybe i do something wrong.( На курсах когда пробовали на лабораторной работе потребовались правила в том числе в таблице main, не могу понять почему, хотя по логике вещей если мы маркируем траффик и по марке используем маршрутизацию, то ничего оставаться не должно что шло бы по main)
Here question how use rule to get for LAN users access to internet.
That’s a misunderstanding. > ****
src-nat
>
> or
>
> ```text
masquerade
rules are not there specifically for LAN users; they are there for any traffic being sent out and matching the conditional part of the rule. So for your case in particular, this means traffic originating in LAN or VPN and leaving towards the internet via
ether1
>
> or
>
> ```text
ether2
, and you need to choose the appropriate public address to use as source, which is the job of these two rules:
>
> Any packet routed to go out via one of (> ****
>
> ```text
ether1
,
ether2
>
> ) will be src-nat'ed to the address indicated in the rule.
>
> The rules you suggest,
>
> ```text
add action=src-nat chain=srcnat comment="ISP1 src-nat LAN" out-interface=bridge1 to-addresses=1.1.1.1
add action=src-nat chain=srcnat comment="ISP2 src-nat LAN" out-interface=bridge1 to-addresses=2.2.2.2
would mean that anything sent to the LAN (ether4 or wlan) would be src-nat’ed to 1.1.1.1 (because both rules have identical match part so the packet would always matched by the first one), which is definitely not what you want.
The reason why i try to use src-nat but not masquerade, because it is less load for CPU. When we use masquerade it is easy to understand, even in default configuration exist this rule, but when you try use src-nat need to understand flow diagram, not so easy. The rule that you suggest, i cant understand u use src-nat ether1 to address 1.1.1.1. I understand this like loop, i mean we src-nat WAN to WAN, dont get it. (Не могу понять как должно выглядеть правило заменяющее masquerade. В моем случае можно и нужно использовать более узкий вариант,src-nat, который меньше ест CPU. Если соотвественно ни masquerade ни src-nat у нас нет, то нет и доступа в ресурсы глобальной сети, ведь мы за NAT. Соответственно по логике вещей трафик уходящий, out interface=ether4 должен адресовываться на wan IP ether1, 1.1.1.1. Ваше правило не могу понять мы адресуем трафик с порта WAN ether1 на IP адрес всё того же порта, вообще не понял, словно кольцо. При попытке указать ether4 в качестве out я получал ошибку, поэтому я использовал bridge1, но я думал эти правила разные, не думал что если два разных правила для разных WAN они будут обрабатываться как одно и выбрано будет только одно из-за bridge1)
Now to make your RDP access to the machine in LAN to work, what you need to achieve is that if a RDP session (or, in generic case, any other connection) comes in from the internet through > ****
ether1
>
> , the response packets of that connection would be sent out via
>
> ```text
ether1
as well (and analogically for
ether2
>
> ). This is, as you have noticed, what the following rules are responsible for:
>
> ```text
/ip firewall mangle
add action=mark-connection chain=input in-interface=ether1 new-connection-mark=ISP1_con passthrough=yes
add action=mark-connection chain=input in-interface=ether2 new-connection-mark=ISP2_con passthrough=yes
However, there is an issue - these rules are in chain > ****
input
>
> of table
>
> ```text
mangle
so they only apply to packets with one of Mikrotik’s own addresses as destination. And this doesn’t apply to the RDP packets, because the rules in
dstnat
>
> chain of table
>
> ```text
nat
are processed before rules in chain > ****
input
>
> of table
>
> ```text
mangle
, and once the packet gets dst-nat’ed, it becomes a packet for other than Mikrotik’s own address and thus needs to be handled by rules in chain
forward
>
> of table
>
> ```text
mangle
.
So this is the first reason why your RDP sessions have a problem - the connections are not marked up to their source interface when they come in. To fix that, you need to add another pair of same rules to chain > ****
forward
>
> of table
>
> ```text
mangle
. It is possible that you could instead move the existing rules to chain
prerouting
>
> of table
>
> ```text
mangle
, but I’m not 100% sure here how the connection marking would work.
>
> only apply to packets sent by Mikrotik itself because they are in chain > ****
>
> ```text
output
, and, quite important, only the first one of them ever matches because it matches all the packets the second one could match.
As i understand i just need to delete old rules by changing output and input rules to forward? I use that rules to correct work dst-nat. (Насколько я помню при работе балансировки надо учитывать работу пробросов ресурсов, для этого я и делал эти правила, чтобы установленное соединение по провайдеру шло на того же провайдера, поскольку есть проблема что пакет по установленному соединению попадет на другой WAN IP и произойдет обрыв соединения. Насколько понял output, input правила убрать и заменить только на два forward для 2 провадйеров соотвествено)
>
> handle any packet which comes in from LAN (and only LAN, not VPN, because VPN has nothing to do with bridge1) and assign a new connection mark to it, even if the connection to which the packet belongs has already been marked before.
>
> So even if the first pair of rules above would mark the incoming packet of connections properly, the rules we talk about now change the connection marks for their responses, making it a lottery whether the right WAN interface is chosen.
>
> To get rid of overwriting the connection marks, add condition > ****
>
> ```text
connection-mark=no-mark
to all rules which assign connection marks. And to save CPU, add to them also another condition,
connection-state=new
>
> . We could do some more optimization to save even more CPU but let's first make it work.
>
>
> > i cant understand in PCC what means numbers, for example when i use "both-addresses-and-ports". What does means this numbers? How correct set up them? As i understand it is need when configure in my case 15Mbit\sec and 10Mbit\sec ISP's, to do about 50/50 load. Or another, speed of ISP's not equal each other, so need to do 2 parts of traffic to ISP1, 1 part of traffic to ISP2, as i understand?
>
> The processing is the following:
>
> * the addresses and ports indicated by the first parameter (in your case, > ****
>
> ```text
both-addresses-and-ports
) are hashed to form a 32-bit integer number
that integer is divided by the first number (in your case, 3)
the remainder of that division (which for divider 3 may be 0,1, or 2) is compared to the third number, and if it matches, the whole condition matches.
So the way you’ve set it up, 1/3 of the connections get one connection mark, another 1/3 gets another connection mark, and the last 1/3 either keeps a connection mark assigned before or gets no connection mark at all.
You have 15 Mbit/s from one provider and 10 Mbit/s from another, so instead of 2:1 distribution, 3:2 would be more appropriate. The figures only make sense if you expect similar traffic structure from all sources, but you’ll see the practical outcome. Also bear in mind that the distribution rules will affect only traffic originating from your LAN and VPN, not the incoming traffic.
To implement 3:2 for ISP1(ether1):ISP2(ether2), you could use five rules, three of them (say, 5/0, 5/1, 5/2) assigning > ****
connection-mark=ISP1_routing
>
> , and two of them (5/3 and 5/4) assigning
>
> ```text
connection-mark=ISP2_routing
. But that would be a waste of CPU. Instead, use two rules with 5/something to assign
connection-mark=ISP2_routing
>
> , and a rule following them without the
>
> ```text
per-connection-classifier
at all, assigning
connection-mark=ISP1_routing
>
> . As I've said before that all rules setting a connection mark must only match on packets of not-yet-marked connections, this rule will do the same job as the three other rules in the 5-rule variabt above.
>
> The last two mangle rules,
>
> ```text
add action=mark-routing chain=prerouting connection-mark=ISP1_con dst-address-type=!local new-routing-mark=ISP1_routing passthrough=yes
add action=mark-routing chain=prerouting connection-mark=ISP2_con dst-address-type=!local new-routing-mark=ISP2_routing passthrough=yes
“translate” the connection marks into routing marks. The point is that connection marks cannot be used directly for routing and are assigned once (when the connection gets established by its first packet), while routing mark must be assigned to every single packet, otherwise it would be routed using routing table “main”. So these rules > must not > contain the condition > ****
>
> do not handle packets originated from the Mikrotik itself, therefore you need to create the same rules in chain
>
> ```text
output
, otherwise traffic like NTP or DNS would not be routed anywhere. This is relevant for both
action=mark-connection
>
> and
>
> ```text
action=mark-routing
rules.
Another point is that policy routing (=routing according to routing marks) is incompatible with fasttrack handling. So if there eventually is a rule in chain > ****
forward
>
> of table
>
> ```text
filter
with
action=fasttrack-connection
>
> , you have to disable it. Do not remove it, it will make sense to re-enable it later to do some optimisation, but let's first make your whole setup run in the systematic way, and only then start optimizing.
Still can get understand mathematic of numbers in PCC, why you advice 5 parts in PCC? Why you advice to delete connection mark? You tell that it is more load for CPU, but maybe i remember something wrong, but we doing connection mark because of it make less load CPU. As i understand connection mark it is some kind of "pipe" that gives opportunity to mark connection once and packets just will identify by this mark connection. We use states for marking, like youu write state "new", but i cant understand this at all before and now in task load and balancing.
(Немного не понял вы советуете убрать connection mark, но мы вроде его как раз задействовали для того чтобы не каждый пакет маркировался каждый раз а пакеты уже по connection mark направлялись в "трубу" по connection mark. То есть по раз установленному соединению остальные пакеты уже попадали по этому установленному соединению на нужного провайдера. Единственное но я так и не понял как именно в балансировке и отказоустойчивости использовать состояние new. То есть в общем я понимаю что каждое из состояний значит, но как его применить именно в отказе/балансировке не понял)
So fix all the above and come back with a complete export of the modified configuration.
The table “main” handles anything which is not marked with a routing mark. I’ve tried to explain that in your initial configuration you were not marking any outgoing traffic from the LAN, only traffic originating in Mikrotik itself and coming from the internet. If there is really no traffic without routing marks, you should not need the routing table main, except if you e.g. use recursive routes and you tell the route to test accessibility of a reference address in the internet, e.g. for a failover configuration without PCC - in this case, the machine pings a remote address to find out whether the path via that WAN interface is functional, and if you would route-mark also these test pings, you would have to provide the route via the correct WAN for them in each “non-main” routing table, otherwise the purpose of the pings would not be met - it would not be a test of the path via a particular WAN as intended but merely a test that the remote IP is alive.
Here question how use rule to get for LAN users access to internet.
That’s a misunderstanding. > ****
src-nat
>
> or
>
> ```text
masquerade
rules are not there specifically for LAN users; they are there for any traffic being sent out and matching the conditional part of the rule. So for your case in particular, this means traffic originating in LAN or VPN and leaving towards the internet via
ether1
>
> or
>
> ```text
ether2
, and you need to choose the appropriate public address to use as source, which is the job of these two rules:
>
> Any packet routed to go out via one of (> ****
>
> ```text
ether1
,
ether2
>
> ) will be src-nat'ed to the address indicated in the rule.
>
> The rules you suggest,
>
> ```text
add action=src-nat chain=srcnat comment="ISP1 src-nat LAN" out-interface=bridge1 to-addresses=1.1.1.1
add action=src-nat chain=srcnat comment="ISP2 src-nat LAN" out-interface=bridge1 to-addresses=2.2.2.2
would mean that anything sent to the LAN (ether4 or wlan) would be src-nat’ed to 1.1.1.1 (because both rules have identical match part so the packet would always matched by the first one), which is definitely not what you want.
The reason why i try to use src-nat but not masquerade, because it is less load for CPU. When we use masquerade it is easy to understand, even in default configuration exist this rule, but when you try use src-nat need to understand flow diagram, not so easy. The rule that you suggest, i cant understand u use src-nat ether1 to address 1.1.1.1. I understand this like loop, i mean we src-nat WAN to WAN, dont get it. (Не могу понять как должно выглядеть правило заменяющее masquerade. В моем случае можно и нужно использовать более узкий вариант,src-nat, который меньше ест CPU. Если соотвественно ни masquerade ни src-nat у нас нет, то нет и доступа в ресурсы глобальной сети, ведь мы за NAT. Соответственно по логике вещей трафик уходящий, out interface=ether4 должен адресовываться на wan IP ether1, 1.1.1.1. Ваше правило не могу понять мы адресуем трафик с порта WAN ether1 на IP адрес всё того же порта, вообще не понял, словно кольцо. При попытке указать ether4 в качестве out я получал ошибку, поэтому я использовал bridge1, но я думал эти правила разные, не думал что если два разных правила для разных WAN они будут обрабатываться как одно и выбрано будет только одно из-за bridge1)
There must be something totally wrong in either how I understand what you want to use the NAT for or in how you understand the packet flow. Let’s see what are the stages of processing of a packet for a device in LAN coming from the internet:
internet → WAN (in-interface) → firewall chains prerouting in tables 1. raw, 2. mangle, 3. (dst-)nat → routing → firewall chains forward in tables 1. mangle, 2. filter → firewall chains postrouting in tables 1. mangle, 2. (src-)nat → bridge1 (out-interface) → physical interface (ether4) → device on LAN
So for a packet received from the internet, bridge1 is the out-interface, so if you apply masquerade (or src-nat) on such packets, your LAN devices would see any packet actually coming from the internet as coming from the Mikrotik’s address attached to that interface (bridge1). Is this really what you want? It is useful in particular scenarios, but normally you only use src-nat at WAN interfaces so that anything what gets out from LAN devices to the internet via your Mikrotik gets the address of Mikrotik’s internet-facing (WAN) interface.
You cannot bind a src-nat rule to input interface (because the postrouting chain of nat table is common for the forward and output paths), and you have to choose the proper to-addresses for the proper output-interface (the WAN). With masquerade, this choice is automatic; with src-nat, you have to have one rule per each WAN (or out-interface in general).
With dst-nat, the situation is reverse - dst-nat changes the destination address of packet incoming via the WAN interface from that interface’s address to the address of a device in LAN where you want these packets to be forwarded to.
Also you have to realize that when a connection is initiated from inside and a src-nat rule changes the source address of the packet sent out via WAN, the connection tracker (see below) remembers this, so when a response packet matching that connection comes back to the public address of the machine, it is automatically dst-nat’ed in order to be delivered to the device on LAN which has initiated the connection.
Now to make your RDP access to the machine in LAN to work, what you need to achieve is that if a RDP session (or, in generic case, any other connection) comes in from the internet through > ****
ether1
>
> , the response packets of that connection would be sent out via
>
> ```text
ether1
as well (and analogically for
ether2
>
> ). This is, as you have noticed, what the following rules are responsible for:
>
> ```text
/ip firewall mangle
add action=mark-connection chain=input in-interface=ether1 new-connection-mark=ISP1_con passthrough=yes
add action=mark-connection chain=input in-interface=ether2 new-connection-mark=ISP2_con passthrough=yes
However, there is an issue - these rules are in chain > ****
input
>
> of table
>
> ```text
mangle
so they only apply to packets with one of Mikrotik’s own addresses as destination. And this doesn’t apply to the RDP packets, because the rules in
dstnat
>
> chain of table
>
> ```text
nat
are processed before rules in chain > ****
input
>
> of table
>
> ```text
mangle
, and once the packet gets dst-nat’ed, it becomes a packet for other than Mikrotik’s own address and thus needs to be handled by rules in chain
forward
>
> of table
>
> ```text
mangle
.
So this is the first reason why your RDP sessions have a problem - the connections are not marked up to their source interface when they come in. To fix that, you need to add another pair of same rules to chain > ****
forward
>
> of table
>
> ```text
mangle
. It is possible that you could instead move the existing rules to chain
prerouting
>
> of table
>
> ```text
mangle
, but I’m not 100% sure here how the connection marking would work.
>
> only apply to packets sent by Mikrotik itself because they are in chain > ****
>
> ```text
output
, and, quite important, only the first one of them ever matches because it matches all the packets the second one could match.
As i understand i just need to delete old rules by changing output and input rules to forward? I use that rules to correct work dst-nat. (Насколько я помню при работе балансировки надо учитывать работу пробросов ресурсов, для этого я и делал эти правила, чтобы установленное соединение по провайдеру шло на того же провайдера, поскольку есть проблема что пакет по установленному соединению попадет на другой WAN IP и произойдет обрыв соединения. Насколько понял output, input правила убрать и заменить только на два forward для 2 провадйеров соотвествено)
It depends on what you want. Input and output chains of all tables are relevant to communication of the Mikrotik itself. So if you want to be able to access Mikrotik’s management interfaces (ssh, https, maybe Winbox, hopefully not telnet and http) via both its public addresses, you have to use connection marking in input and routing marking based on connection mark in output as well. If you want that the Mikrotik can query DNS and NTP via any of the WANs, you need to assign connection marks also in output.
If you only want the traffic between the internet and your LAN devices to be load balanced, and local traffic to go via just one of the WANs, you do not need to connection-mark it and route-mark it, but in that case, you need the routing table main to contain at least the default route via that WAN’s gateway.
>
> handle any packet which comes in from LAN (and only LAN, not VPN, because VPN has nothing to do with bridge1) and assign a new connection mark to it, even if the connection to which the packet belongs has already been marked before.
>
> So even if the first pair of rules above would mark the incoming packet of connections properly, the rules we talk about now change the connection marks for their responses, making it a lottery whether the right WAN interface is chosen.
>
> To get rid of overwriting the connection marks, add condition > ****
>
> ```text
connection-mark=no-mark
to all rules which assign connection marks. And to save CPU, add to them also another condition,
connection-state=new
>
> . We could do some more optimization to save even more CPU but let's first make it work.
>
>
> > i cant understand in PCC what means numbers, for example when i use "both-addresses-and-ports". What does means this numbers? How correct set up them? As i understand it is need when configure in my case 15Mbit\sec and 10Mbit\sec ISP's, to do about 50/50 load. Or another, speed of ISP's not equal each other, so need to do 2 parts of traffic to ISP1, 1 part of traffic to ISP2, as i understand?
>
> The processing is the following:
>
> * the addresses and ports indicated by the first parameter (in your case, > ****
>
> ```text
both-addresses-and-ports
) are hashed to form a 32-bit integer number
that integer is divided by the first number (in your case, 3)
the remainder of that division (which for divider 3 may be 0,1, or 2) is compared to the third number, and if it matches, the whole condition matches.
So the way you’ve set it up, 1/3 of the connections get one connection mark, another 1/3 gets another connection mark, and the last 1/3 either keeps a connection mark assigned before or gets no connection mark at all.
You have 15 Mbit/s from one provider and 10 Mbit/s from another, so instead of 2:1 distribution, 3:2 would be more appropriate. The figures only make sense if you expect similar traffic structure from all sources, but you’ll see the practical outcome. Also bear in mind that the distribution rules will affect only traffic originating from your LAN and VPN, not the incoming traffic.
To implement 3:2 for ISP1(ether1):ISP2(ether2), you could use five rules, three of them (say, 5/0, 5/1, 5/2) assigning > ****
connection-mark=ISP1_routing
>
> , and two of them (5/3 and 5/4) assigning
>
> ```text
connection-mark=ISP2_routing
. But that would be a waste of CPU. Instead, use two rules with 5/something to assign
connection-mark=ISP2_routing
>
> , and a rule following them without the
>
> ```text
per-connection-classifier
at all, assigning
connection-mark=ISP1_routing
>
> . As I've said before that all rules setting a connection mark must only match on packets of not-yet-marked connections, this rule will do the same job as the three other rules in the 5-rule variabt above.
>
> The last two mangle rules,
>
> ```text
add action=mark-routing chain=prerouting connection-mark=ISP1_con dst-address-type=!local new-routing-mark=ISP1_routing passthrough=yes
add action=mark-routing chain=prerouting connection-mark=ISP2_con dst-address-type=!local new-routing-mark=ISP2_routing passthrough=yes
“translate” the connection marks into routing marks. The point is that connection marks cannot be used directly for routing and are assigned once (when the connection gets established by its first packet), while routing mark must be assigned to every single packet, otherwise it would be routed using routing table “main”. So these rules > must not > contain the condition > ****
>
> do not handle packets originated from the Mikrotik itself, therefore you need to create the same rules in chain
>
> ```text
output
, otherwise traffic like NTP or DNS would not be routed anywhere. This is relevant for both
action=mark-connection
>
> and
>
> ```text
action=mark-routing
rules.
Another point is that policy routing (=routing according to routing marks) is incompatible with fasttrack handling. So if there eventually is a rule in chain > ****
forward
>
> of table
>
> ```text
filter
with
action=fasttrack-connection
>
> , you have to disable it. Do not remove it, it will make sense to re-enable it later to do some optimisation, but let's first make your whole setup run in the systematic way, and only then start optimizing.
>
> Still can get understand mathematic of numbers in PCC, why you advice 5 parts in PCC? Why you advice to delete connection mark? You tell that it is more load for CPU, but maybe i remember something wrong, but we doing connection mark because of it make less load CPU. As i understand connection mark it is some kind of "pipe" that gives opportunity to mark connection once and packets just will identify by this mark connection. We use states for marking, like youu write state "new", but i cant understand this at all before and now in task load and balancing.
> (Немного не понял вы советуете убрать connection mark, но мы вроде его как раз задействовали для того чтобы не каждый пакет маркировался каждый раз а пакеты уже по connection mark направлялись в "трубу" по connection mark. То есть по раз установленному соединению остальные пакеты уже попадали по этому установленному соединению на нужного провайдера. Единственное но я так и не понял как именно в балансировке и отказоустойчивости использовать состояние new. То есть в общем я понимаю что каждое из состояний значит, но как его применить именно в отказе/балансировке не понял)
>
> So fix all the above and come back with a complete export of the modified configuration.
Where have I said you should remove connection marking? I've only said that you should disable the fasttrack-connection rule instead of removing it completely, and Ive said that because the final optimisation step will be to use WAN 1 (the faster one) as a default and use connection-marks etc. as an exception from that, routed via WAN2, but we haven't got there yet.
Other than that - there is a firewall module called connection tracker. Each incoming or outgoing packet which may potentially be a member of an already known connection (which in particular means any udp, tcp or icmp packet) is compared against every item in the connection tracker list. If it is not found there, a new connection is added to the list, and such packet bears a label "connection-state=new". If the packet is found to match a connection in the list, it gets a label "connection-state=established". It is possible to mark these connections, and the mark can be set by any packet. All packets belonging to a marked connection also get a label "connection-mark=the-mark-previously-assigned-to-that-connection". But only one mark can be assigned to a connection at a time, so an eventual newer mark overwrites the older one.
To assign a connection mark, not only per-connection-classifier match (condition) can be used. So although the per-connection-classifier likely (I'm not sure here but it makes sense) only matches on the first packet of each connection, this is not the case for other conditions which could match on subsequent packets and change an already assigned connection mark. That's why I've written to add connection-state=new as an additional condition for any rule adding the connection mark. But it is true that if the connection is established from the internet side and thus the connection mark is assigned based on in-interface WAN, the per-connection-classifier won't likely match on the response packets; if the connection is established from inside (LAN or the Mikrotik itself), the PCC rule assigns a connection mark and the connection is established via one of the WANs, it doesn't mind that the same connection mark will be assigned once again by the rule handling the response coming from the internet via the same WAN.
\
\
One more point, I'm just a bit confused about the RDP - is it really open for direct access via the public (WAN) addresses, as the presence of dst-nat rules and the fact that you've XXXXed the dst-port suggest, or only for VPN users? Because the fact that it freezes is an issue independent on the PCC and the explanation may depend on the answer.
