Community discussions

MikroTik App
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Multi WAN Connection Tracking

Fri Apr 29, 2022 5:21 pm

Is there a way to have an incoming packet on the WAN 1 to always be returned out WAN 1 and Incoming traffic on WAN 2 to always be returned on WAN 2 regardless of source IP? With default router setup everything is returned to WAN 1 by default routing rules. I can add rules to route specific IP traffic to WAN 2, but would like it to be more of a connection tracked route, not IP range specific.

Here's my setup:
Image

This is going to be replicated 200 times across multiple sites and I'd rather not hard code IP routes in case one of the server addresses changes at some point. In that case I would have to log into all 200 sites and reconfigure the routes which I would prefer to avoid.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7042
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 5:29 pm

Mark new incoming connections in mangle and then reroute traffic from local hosts based on that connection mark (by using mangle routing mark or route rules)
 
tdw
Forum Guru
Forum Guru
Posts: 1843
Joined: Sat May 05, 2018 11:55 am

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 5:37 pm

Mark connections which are new or have no mark on ingress via the respective WAN interfaces, apply routing marks based on the connection marks to lookup outbound routes. It is mentioned in the Wiki PCC example https://wiki.mikrotik.com/wiki/Manual:P ... cy_routing and is still relevant even if you do not implement PCC for internally initiated outbound traffic.
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 5:58 pm

Thanks for the quick replies! After some searching I came up with this: https://aacable.wordpress.com/2018/11/1 ... wan-links/

This walks through things differently than the Wiki. Is there a right or best way to do this? I'm not familiar with text commands and haven't used the terminal for configuration (I'm assuming that's where I'd type these instructions in). Thanks for the assistance. As you can tell I'm just getting my mind wrapped around all this and my routing depth of knowledge is pretty shallow.
 
tdw
Forum Guru
Forum Guru
Posts: 1843
Joined: Sat May 05, 2018 11:55 am

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 6:26 pm

As with many blogs/videos it is incomplete - it only handles connections to the Mikrotik itself, not connections passing through the Mikrotik (both directly routed and destination NAT), and re-marks connections for every packet received, not just the first one which would have a connection state of new and also no existing connection mark, so using unecessary CPU resources.

Command-line examples precisely describe the settings made, using the Web UI / Winbox there may be settings on multiple tabs of a items properties which would require several screenshots and the necessary fields highlighted to convey the same information. You can always look through the graphical UI settings of an object to work out which fields correspond to the command-line example.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2990
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 6:42 pm


This is going to be replicated 200 times across multiple sites and I'd rather not hard code IP routes in case one of the server addresses changes at some point. In that case I would have to log into all 200 sites and reconfigure the routes which I would prefer to avoid.

you have to reconfigure 200 sites and you expect that to be simple without central management or automation system established and configured?

how you think to achieve that?
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 6:59 pm

Mark new incoming connections in mangle and then reroute traffic from local hosts based on that connection mark (by using mangle routing mark or route rules)
So I easily setup the inbound marks and can monitor that connections are getting marked via IP/Firewall/Connections, inbound are WAN1_conn and WAN2_conn. However, I'm having a hard time with routing out via those marks. I created two new tables in Routing/Tables: out_WAN1 and out_WAN2, but I don't know where to build the rules for routing and what settings. Here is what I've got that's not working at the moment:
Image

Any direction? Thanks!
 
tdw
Forum Guru
Forum Guru
Posts: 1843
Joined: Sat May 05, 2018 11:55 am

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 7:32 pm

Use /ip firewall mangle to also apply routing marks to marked connections, as you have only a screenshot of firewall connections what you have there is not displayed. For traffic from the Mikrotik itself it should be:
/ip firewall mangle
add chain=output connection-mark=WAN1_conn action=mark-routing new-routing-mark=out_ISP1
add chain=output connection-mark=WAN2_conn action=mark-routing new-routing-mark=out_ISP2


It is a multi-step process:
Applying the connection mark at the beginning of a new conversation identifies the stream of packets in that particular conversation.
Applying the routing mark identifies which routing table to use for each packet of the conversation.
The routing table identifies the destination to send the packet to.
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 7:49 pm

Command-line examples precisely describe the settings made, using the Web UI / Winbox there may be settings on multiple tabs of a items properties which would require several screenshots and the necessary fields highlighted to convey the same information. You can always look through the graphical UI settings of an object to work out which fields correspond to the command-line example.
This was the answer! Thank you so much tdw and mrz for you assistance! Once I added the rest of the mark rules the routing I had already setup worked. I've been working on this for WAY too long and you all got me where I needed to be. Thanks!
 
User avatar
pcunite
Forum Guru
Forum Guru
Posts: 1345
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 9:38 pm

This was the answer! Thank you so much tdw and mrz for you assistance! Once I added the rest of the mark rules the routing I had already setup worked. I've been working on this for WAY too long and you all got me where I needed to be. Thanks!

May I see your configuration? I'm in this same boat and getting to see a complete example really helps.
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 9:42 pm

IDK how to do that...?
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 9:45 pm

you have to reconfigure 200 sites and you expect that to be simple without central management or automation system established and configured?

how you think to achieve that?
I'm loading a templated config into 200 new routers on the bench. Changes in the future should be one off, not en mass. Is there a good solution to do centeral management with these?
 
User avatar
pcunite
Forum Guru
Forum Guru
Posts: 1345
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 9:57 pm

IDK how to do that...?

From within the Winbox GUI tool, open up the menu item "New Terminal". Then type the following command: export file="Export.rsc". Then navigate to the "Files" menu option and you'll note the newly exported configuration. Right click on it and download it to your PC. Open it up in an editor (like Notepad) and change personal information.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 9:58 pm

Remove also serial number from export
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 10:10 pm

Here's what I've got. I know there is probably extra stuff in there. I modified a default config, not knowing enought to set it up from scratch.
# jan/02/1970 01:35:57 by RouterOS 7.2.1
# software id = V57V-SU19
#
# model = RB750r2

/interface bridge
add admin-mac=  auto-mac=no comment=defconf name=bridge
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=192.168.0.150-192.168.0.159
/ip dhcp-server
add address-pool=dhcp interface=bridge name=defconf
/routing table
add disabled=no fib name=out_WAN2
add disabled=no fib name=out_WAN1
/interface bridge port
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
/ip firewall connection tracking
set enabled=yes
/ip neighbor discovery-settings
set discover-interface-list=WAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=ether2 list=WAN
/ip address
add address=192.168.0.1/24 comment=defconf interface=bridge network=\
    192.168.0.0
add address=192.168.13.101/24 interface=ether1 network=192.168.13.0
add address=192.168.200.2/24 interface=ether2 network=192.168.200.0
add address=192.168.200.101/24 interface=ether2 network=192.168.200.0
add address=192.168.13.1/24 interface=ether1 network=192.168.13.0
add address=192.168.13.102/24 interface=ether1 network=192.168.13.0
add address=192.168.13.103/24 interface=ether1 network=192.168.13.0
add address=192.168.200.102/24 interface=ether2 network=192.168.200.0
add address=192.168.200.103/24 interface=ether2 network=192.168.200.0
/ip dhcp-client
add comment=defconf disabled=yes interface=ether1
/ip dhcp-server network
add address=192.168.0.0/24 comment=defconf dns-server=192.168.88.1 gateway=\
    192.168.0.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.0.1 comment=defconf name=router.lan
/ip firewall filter
add action=accept chain=input comment=WinBox dst-port=8291 protocol=tcp
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \
    in-interface=ether1 new-connection-mark=WAN1_Conn passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark \
    in-interface=ether2 new-connection-mark=WAN2_Conn passthrough=yes
add action=mark-routing chain=output connection-mark=WAN1_Conn \
    new-routing-mark=out_WAN1 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN2_Conn \
    new-routing-mark=out_WAN2 passthrough=yes
add action=accept chain=prerouting dst-address=172.17.32.0/21 \
    in-interface-list=LAN
add action=mark-connection chain=prerouting connection-mark=no-mark \
    dst-address-type=!local in-interface-list=LAN new-connection-mark=\
    WAN1_Conn passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark \
    dst-address-type=!local in-interface-list=LAN new-connection-mark=\
    WAN2_Conn passthrough=yes per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting connection-mark=WAN1_Conn \
    in-interface-list=LAN new-routing-mark=out_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_Conn \
    in-interface-list=LAN new-routing-mark=out_WAN2 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN1_Conn disabled=yes \
    new-routing-mark=out_WAN1 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN2_Conn disabled=yes \
    new-routing-mark=out_WAN2 passthrough=yes
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
add action=dst-nat chain=dstnat dst-address=192.168.13.101 dst-port=4000 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.101 to-ports=\
    4000
add action=dst-nat chain=dstnat dst-address=192.168.13.103 dst-port=4000 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.103 to-ports=\
    4000
add action=dst-nat chain=dstnat dst-address=192.168.13.102 dst-port=4000 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.102 to-ports=\
    4000
add action=dst-nat chain=dstnat dst-address=192.168.200.101 dst-port=4000 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.101
add action=dst-nat chain=dstnat dst-address=192.168.200.103 dst-port=4000 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.103
add action=dst-nat chain=dstnat dst-address=192.168.200.102 dst-port=4000 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.102
add action=dst-nat chain=dstnat dst-address=192.168.13.1 dst-port=8291 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.1 to-ports=8291
add action=dst-nat chain=dstnat dst-address=192.168.200.2 dst-port=8291 \
    in-interface-list=WAN protocol=tcp to-addresses=192.168.0.1 to-ports=8291
/ip route
add check-gateway=ping disabled=no distance=2 dst-address=0.0.0.0/0 gateway=\
    192.168.200.1 pref-src=0.0.0.0 routing-table=out_WAN2 scope=30 \
    suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=2 dst-address=0.0.0.0/0 gateway=\
    192.168.13.31 pref-src=0.0.0.0 routing-table=out_WAN1 scope=30 \
    suppress-hw-offload=no target-scope=10
add disabled=yes distance=1 dst-address=0.0.0.0/0 gateway=192.168.200.1 \
    pref-src=0.0.0.0 routing-table=main scope=30 suppress-hw-offload=no \
    target-scope=10
/ip service
set www address=0.0.0.0/0
set winbox address=0.0.0.0/0
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
    33434-33534 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
    udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
    "defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
    hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
    500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=forward comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
/routing rule
add action=lookup disabled=yes dst-address=192.168.200.0/24 interface=ether2 \
    src-address=192.168.0.0/24 table=main
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool sniffer
set file-name=Test filter-interface=all filter-ip-protocol=tcp
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 10:11 pm

Edit: Disregard, I had to exclude traffic directed at the WAN IP address port from getting marked.

I had been able to access the router through WinBox on the server throug the WAN but, now that is broken. Any one have any ideas?
Last edited by Ufdah on Fri Apr 29, 2022 11:14 pm, edited 1 time in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19103
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 10:40 pm

If you want to ensure traffic coming in on WAN1 goes out WAN1 etc..
This sweet config example from sob is all you need! (Assumes v7 firmware)
/routing table
add name=ISPx_route fib
/ip route
add dst-address=0.0.0.0/0 gateway=<ISPx gateway> table=main  { comment:  For local originated traffic }
add  dst-address=0.0.0.0/0 gateway=<ISPx gateway>  table=ISPx_route  { comment: For external originated traffic to servers } 
/ip firewall mangle
add chain=prerouting in-interface=WANx connection-state=new action=mark-connection new-connection-mark=WANx_conn
add chain=prerouting in-interface-list=LAN connection-mark=WANx_conn action=mark-routing new-routing-mark=ISPx_route
Documented at PARA K here - viewtopic.php?t=182373

Which includes failover from primary to secondary etc...... and recursive.
Last edited by anav on Fri Apr 29, 2022 11:20 pm, edited 1 time in total.
 
Ufdah
just joined
Topic Author
Posts: 19
Joined: Thu Apr 21, 2022 7:23 pm

Re: Multi WAN Connection Tracking

Fri Apr 29, 2022 11:17 pm

Thanks, I believe this is virtually the same as what I have setup now. It took a bit of brain damage to wrap my mind around the concepts, but it makes sense now.

Who is online

Users browsing this forum: No registered users and 19 guests