Hello,

I would like to request that someone with mangle experience takes a quick look at my mangle config and let me know what tha problem is.
I am working on a lab that will allow me to finish the setup of a client but it seems i am doing something wrong.
In the following example i just want my computer (192.168.100.150) to pass all traffic through ISP1. I have placed my connection marks but for some reason
i cannot comprehend it keeps passing through ISP2.

I would really appreciate the feedback as to why this is happening.

/interface ethernet
set [ find default-name=ether1 ] name=01-WAN1
set [ find default-name=ether2 ] name=02-LAN
set [ find default-name=ether3 ] master-port=02-LAN name=03-LAN
set [ find default-name=ether4 ] master-port=02-LAN name=04-LAN
set [ find default-name=ether5 ] loop-protect=off name=05-WAN2

/ip neighbor discovery
set "01-WAN1" discover=no
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-128-cbc
/ip pool
add name=default-dhcp ranges=192.168.100.10-192.168.100.150
/ip dhcp-server
add address-pool=default-dhcp always-broadcast=yes authoritative=yes disabled=\
no interface=02-LAN name=master_dhcp
/ip address
add address=192.168.100.254/24 comment="default configuration" interface=02-LAN \
network=192.168.100.0
add address=10.111.0.1/24 comment="OTE HOME GATEWAY" interface=01-WAN1 network=\
10.111.0.0
add address=10.112.0.1/24 comment="FORTHNET NEIBOUR" interface=05-WAN2 network=\
10.112.0.0
/ip dhcp-client
add comment="default configuration" dhcp-options=hostname,clientid interface=\
01-WAN1
/ip dhcp-server lease
add address=192.168.100.5 client-id=Kodi mac-address=10:00:00:34:EB:57 server=\
master_dhcp
/ip dhcp-server network
add address=192.168.100.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.100.254 \
netmask=24
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,10.112.0.2,10.111.0.2,8.8.4.4
/ip dns static
add address=10.112.0.1 name=router

/ip firewall mangle
add action=mark-connection chain=input connection-state=new in-interface=\
01-WAN1 new-connection-mark=from_ISP1 passthrough=no
add action=mark-connection chain=input connection-state=new in-interface=\
05-WAN2 new-connection-mark=from_ISP2 passthrough=no
add action=mark-routing chain=output connection-mark=from_ISP1 \
new-routing-mark=to_ISP1 passthrough=no
add action=mark-routing chain=output connection-mark=from_ISP2 \
new-routing-mark=to_ISP2 passthrough=no
add action=accept chain=prerouting dst-address=10.111.0.0/24 in-interface=\
02-LAN
add action=accept chain=prerouting dst-address=10.112.0.0/24 in-interface=\
02-LAN
add action=mark-connection chain=prerouting connection-mark=no-mark \
connection-state=established,new in-interface=01-WAN1 new-connection-mark=\
from_ISP1 passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark \
connection-state=established,new in-interface=05-WAN2 new-connection-mark=\
from_ISP2 passthrough=yes
add action=mark-connection chain=prerouting comment="PRE-PCC 1" \
connection-mark=no-mark connection-state="" in-interface=02-LAN \
new-connection-mark=to_ISP1 passthrough=yes src-address=192.168.100.150
add action=mark-connection chain=prerouting comment="PCC 1" connection-mark=\
no-mark connection-state=new dst-address-type=!local in-interface=02-LAN \
new-connection-mark=to_ISP1 passthrough=yes per-connection-classifier=\
src-address:2/0
add action=mark-connection chain=prerouting comment="PCC 2" connection-state=\
new dst-address-type=!local in-interface=02-LAN new-connection-mark=to_ISP2 \
packet-mark=no-mark passthrough=yes per-connection-classifier=\
src-address:2/1
add action=mark-routing chain=prerouting connection-mark=to_ISP1 in-interface=\
02-LAN new-routing-mark=to_ISP1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=to_ISP2 in-interface=\
02-LAN new-routing-mark=to_ISP2 passthrough=no
/ip firewall nat
add action=masquerade chain=srcnat out-interface=01-WAN1
add action=masquerade chain=srcnat out-interface=05-WAN2
add action=dst-nat chain=dstnat dst-port=1194 in-interface=01-WAN1 protocol=udp \
to-addresses=192.168.100.253 to-ports=1194
/ip route
add check-gateway=ping distance=10 gateway=10.111.0.2 routing-mark=to_ISP1
add disabled=yes distance=10 dst-address=8.8.8.8/32 gateway=10.111.0.2 \
routing-mark=to_ISP1
add check-gateway=ping distance=10 gateway=10.112.0.2 routing-mark=to_ISP2
add disabled=yes distance=10 dst-address=8.8.4.4/32 gateway=10.112.0.2 \
routing-mark=to_ISP2

It looks like in your "PCC 2" rule, you added packet-mark=no-mark by mistake, instead of correct connection-mark=no-mark. As it is now, this rule will happily overwrite previously set connection mark from "PRE-PCC 1" rule.

Sent from my iPhone using Tapatalk

Thank you for all your time and effort to go through my config and provide me with the troubleshooting.
I appreciate it.
Devil hides in the details.

I am still trying to graps the passthrough concept in mangle though.
Lets suppose that PRE-PCC1 is a match and that its a no-passthrough.
Is it normal for all traffic the internet to halt?

Passthrough is very simple, it just tells the router to continue processing rules. And you need it, because you need to do more actions for same packet. You want to mark connection and immediately mark routing for first packet of this connection. Without passthrough, you only mark connection, but then poor little packet doesn't know where to go, because you don't have default route in main routing table.

Thank you Sob,

I am getting the idea now although a few questions emerge...
Lets say that with my config above as a starting point i want to achieve having to gateways with 2 static external ip addresses
(lets suppose nat is in place and working) and have all traffic originating from either WAN interface be answered back via the same interface.
My goal is to be able to access from outside a service on a specific server inside via 2 gateways at the same time.
That mangle rule must be placed before PCC rules for the LAN PC clients (needed for general load balancing)
How can i achieve that? I know that once the packet exits the router the mark is lost.
Do i have to use the "established" option for the server to identify the origin of the packets?

Sorry for late reply, I wasn't here for few days.

If you want to access one internal server from both WANs at the same time, you must mark incoming connections and then route them back properly. You already have part of that, you just need to mark routing: Or you can simplify it and just use one connection mark for each WAN, e.g. "ISP1" instead "from_ISP1" and "to_ISP1".

Thank you Sob!
I have already managed to make it work in the meantime.
At the moment the device is configures to load balance 2 WAN interfaces while port forwarding successfully both connection destined to the main server.
I also configured the Failover that automatically increases the route value if one connection is for some reason down, via netwatch.
Now all that remains is to transfer the SSTP config successfully from my Mikrotik (working for 1 wan) to my client's in order to be able to establish a VPN (dual wan).
But i suppose that it will not give me a hard time.
Mikrotik are awesome
Will let you know of the final outcome since i am very close to the final config.

Ok i think this is my nemessis....

While everything is working properly, SSTP to mikrotik is not.
I have tried everything i can think of.
The logs show that the connection matches the input 443 tcp rule but then nothing happens.
The connection tracking shows the incoming connection to WAN 1 443 but it stops at TCP SYN.
I have to assume that for some reason i really cannot understand the request does not get answered.
Can u please check my config and explain what am i doing so wrong???
Routing Works
Nat works
Connections from both WANS at the same time to my server 192.168.1.1 work
SSTP port from local network to my mikrotik interface work
The only thing that does not work is SSTP from WAN to Router.

/interface ethernet
set [ find default-name=ether3 ] name=Local
set [ find default-name=ether4 ] master-port=Local name=Local2
set [ find default-name=ether5 ] master-port=Local name=Local3
set [ find default-name=ether1 ] name=WAN1
set [ find default-name=ether2 ] name=WAN2
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=sstp-pool ranges=192.168.10.5-192.168.10.15
/ppp profile
add dns-server=8.8.8.8 local-address=192.168.10.1 name=SSTP remote-address=\
sstp-pool use-compression=yes
/interface sstp-server server
set authentication=mschap2 certificate=CA1 default-profile=SSTP enabled=yes
/ip address
add address=192.168.1.254/24 interface=Local network=192.168.1.0
add address=10.111.0.1/24 interface=WAN1 network=10.111.0.0
add address=10.112.0.1/24 interface=WAN2 network=10.112.0.0
/ip dns
set allow-remote-requests=yes cache-size=5000KiB max-udp-packet-size=512 \
servers=8.8.4.4,8.8.8.8

/ip firewall filter
add action=accept chain=input dst-port=443 log=yes protocol=tcp
add action=accept chain=input dst-port=8291 protocol=tcp
add action=drop chain=output dst-address=208.67.220.220 out-interface=WAN2
add action=drop chain=output dst-address=208.67.222.222 out-interface=WAN1
add action=accept chain=output
add action=accept chain=forward

/ip firewall mangle
add action=mark-connection chain=prerouting comment="MARK WAN CONNECTIONS" \
in-interface=WAN1 log=yes new-connection-mark=WAN1_mark passthrough=yes
add action=mark-connection chain=prerouting in-interface=WAN2 \
new-connection-mark=WAN2_mark passthrough=yes
add action=accept chain=input dst-port=8291 protocol=tcp
add action=mark-connection chain=input dst-address=10.111.0.1 dst-port=443 \
in-interface=WAN1 log=yes new-connection-mark=WAN1_mark passthrough=yes \
protocol=tcp
add action=mark-routing chain=output connection-mark=WAN1_mark log=yes \
new-routing-mark=to_ISP1 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN2_mark \
new-routing-mark=to_ISP2 passthrough=yes
add action=accept chain=prerouting dst-address=10.111.0.0/24 in-interface=\
Local
add action=accept chain=prerouting dst-address=10.112.0.0/24 in-interface=\
Local
add action=mark-connection chain=prerouting comment=\
"Send all server traffic back where it came from" connection-mark=\
WAN1_mark in-interface=Local new-connection-mark=WAN1_mark passthrough=\
yes src-address=192.168.1.1
add action=mark-connection chain=prerouting connection-mark=WAN2_mark \
in-interface=Local new-connection-mark=WAN2_mark passthrough=yes \
src-address=192.168.1.1
add action=mark-connection chain=prerouting connection-mark=no-mark \
in-interface=Local new-connection-mark=WAN1_mark passthrough=yes \
src-address=192.168.1.1
add action=mark-connection chain=prerouting comment=PCC1 connection-mark=\
no-mark connection-state=new dst-address-type=!local in-interface=Local \
new-connection-mark=WAN1_mark passthrough=yes per-connection-classifier=\
both-addresses-and-ports:2/0
add action=mark-connection chain=prerouting comment=PCC2 connection-mark=\
no-mark connection-state=new dst-address-type=!local in-interface=Local \
new-connection-mark=WAN2_mark passthrough=yes per-connection-classifier=\
both-addresses-and-ports:2/1
add action=mark-routing chain=prerouting connection-mark=WAN1_mark \
in-interface=Local new-routing-mark=to_ISP1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=WAN2_mark \
in-interface=Local new-routing-mark=to_ISP2 passthrough=no
/ip firewall nat
add action=masquerade chain=srcnat out-interface=WAN1
add action=masquerade chain=srcnat out-interface=WAN2
add action=dst-nat chain=dstnat dst-port=3389 in-interface=WAN1 log=yes \
protocol=tcp to-addresses=192.168.1.1 to-ports=3389
add action=dst-nat chain=dstnat dst-port=3389 in-interface=WAN2 log=yes \
protocol=tcp to-addresses=192.168.1.1 to-ports=3389
/ip route
add check-gateway=ping comment=PROVIDER1 distance=10 gateway=10.111.0.2 \
routing-mark=to_ISP1
add check-gateway=ping comment=PROVIDER2 distance=10 gateway=10.112.0.2 \
routing-mark=to_ISP2
add distance=5 dst-address=208.67.220.220/32 gateway=10.111.0.2 scope=10
add distance=5 dst-address=208.67.222.222/32 gateway=10.112.0.2 scope=10
/ip service
set winbox address=0.0.0.0/0
/ppp secret
add local-address=192.168.10.1 name=client1 password=secret1 profile=SSTP \
service=sstp
/system routerboard settings
set memory-frequency=1200DDR protected-routerboot=disabled
/tool netwatch
add down-script="ip route set [find comment=\"PROVIDER1\"] distance=20 \r\
\n" host=208.67.220.220 interval=10s up-script=\
"ip route set [find comment=\"PROVIDER1\"] distance=10 \r\
\n"
add down-script="ip route set [find comment=\"PROVIDER2\"] distance=20 \r\
\n" host=208.67.222.222 interval=10s up-script=\
"ip route set [find comment=\"PROVIDER2\"] distance=10 \r\
\n"

I'm affraid I don't see what's wrong.

You don't need to mark connections in input chain: They already get marked in prerouting. Also these do not seem to do anything at all, they just re-set the same mark again: But none of this explains your problem.

Try to add some logging to see where your reply packets go:

I found the problem again. The issue was located in the routes.
I had to add 2 default routes, one for each interface.
The routes were added with a higher than the marked routes used by the ECC, 15 and 16 distance.
Now i can connect with my windows 10 SSTP client, and i get successfully verified with my credentials and
certificate.
I can also ping the remote router. Now the only issue that remains is that i can only ping the
router at 192.168.10.1 ip but not the 192.168.1.1, which is the ip of the server at the far end.
Now i have to find out if its routing related or firewall related issue.
Your feedback is more than welcome as always

UPDATE!

It works. I had to remove the 3rd mangle rule that was directing all traffic from 192.168.1.1 only through WAN1.

Good catch with routes.

If you want to force WAN1 for outgoing connections from 192.168.1.1 (using the rule you removed), you can either mark connections from VPN with another mark and your "from LAN" marking rules won't touch them, because they have connection-mark=no-mark condition, or you can use routing rule:

Thanks Sob.
Mikrotik configuration is fun but puzzling some times
At least now everything works.
I ll keep you as a friend for the hard days thanks again for all the help.

You're right, it's fun. Keep playing and enjoy.

Here is a question for you
In PCC i have checked the new connection check box (connection-state=new).
My purpose was when a new connection from a Lan workstation was initiated the PCC would kick in
and ignore all the established and related connections.
Is that correct ?

I suppose it is. I mean, rule with connection-state=new definitely won't process packets for established or related connections. But checking for connection state may be redundant when rule also has connection-mark=no-mark, because you mark all connections anyway. I'm not sure about possible performance differences, but I think it should be nothing major.

Happy new year,

Seems i have a question regarding my PCC functionality
Lets suppose that one of the DSL connections drops or hangs.
My script via netwatch is configured to demote the failed route's distance and to restore it to its original state when the line is available again.
While my wan2 dropped today and my route got demoted some of the workstation did not have access to the Internet during that time.

May i ask how exactly PCC works in regard to the routes?

PCC doesn't care. And I also don't think that your netwatch scripts actually do anything useful, because no matter how they change the distance, each to_ISPx routing table will always have only one active route, which will be selected.

You can try this, it should be fully automatic without need for netwatch. A word of warning, this is a little new to me, so no guarantees (not that I give any otherwise ): It's called recursive nexthop and there's some description here. This is how it looks when both connections work: And now with failed WAN2 (unreachable 208.67.222.222): Edit: Original code was wrong.

First of all thanks again for your time and effort.
I would like to grasp the mechanics of the routes before applying a solution.
Recursive routes are reported to cause issues in failover. People claim that they have
An eradic behavior.
You mentioned that Pcc doesnt care about route distance?
even if i set a default route with higher priority will the pcc continue to ignore it because of the marking?
do u think recursive routes is the only choise for mikrotikik failover that is setup to use pcc"?

Well, I won't lie, this part of RouterOS is something where I'm sure that I still have a lot to learn.

But AFAIK PCC is just a matcher for connections, I don't see any indication anywhere that it should care about anything else.

Regarding recursive routes, I already admitted to them being new to me, so there might be some problems I'm not aware of. But in this specific case, I don't see any reason why it wouldn't work. I actually did test it. Not exactly with real two ISPs, only in simplified minilab, but if nothing else, it at least isn't completely broken.

Recursive routes is not the only way. If you keep netwatch, then with four routes like this: raising distance of first one in each table above 10 would make the other one (backup) active.

References to any materials proving that I may be wrong are welcome. Not that it would make me exactly happy, but better than keep being wrong in future.

You are an extremely valuable helper.
I will experiment on both your approaches and let you know when i have some decent input to bring back on the table.
Many thanks again!