Mikrotik 4011 2x ISP LAN routing

Hi,

I kindly ask for your help, as I have been struggling with an issue for several days now.

I am using Mikrotik 4011 and have two ISP connections:

  1. Main PPPoE GPON with a dynamic external IP address (83.x.x.x),
  2. ETTH connection with a static external IP address via DHCP (178.x.x.x).

My LAN network uses the 10.10.0.0/24 range. I have multiple VPN servers – WireGuard, IPsec IKE, SSTP, L2TP – operating through connection 2. In the LAN, I also have a NAS server (10.10.0.3). Is it possible to configure the router so that all traffic to the VPN servers and the NAS server is routed through connection 2, while the rest of the LAN clients (10.10.0.4–10.10.0.253) use connection 1?

I tried to solve the issue using appropriate NAT configuration and address lists, but without success.
Is it possible to use just one LAN subnet?

YES

any hint :slight_smile:?

It will be a complicated setup but assuming you have port forwarding to servers on WAN2 and VPNS coming into WAN2,
Simply make two routes,
WAN1 PRIMARY
WAN2 Secondary

Mangle traffic to the router, ( vpns )
Mangle traffic through the router ( port forwarding )

Send all return traffic back out WAN2 that came in on WAN2.

Special case for wireguard as the protocol is not quite behaving properly in mangling for WAN2 situations.
++++++++++++++++++++++++++++++++++++++++++++++++

To be sure, it would be much easier if you put the NAS on its own subnet which is very easy to do.

Thank you, Anav, for the help, but I need more.

I have now created 2 LAN networks:

10.10.0.0/24 <---- Need to use with ISP1
10.11.0.0/24 <---- Need to use with ISP2 (for NAS)

I have 2 LAN bridges:

LAN ISP 1 with ports (1,2,3,4,5)
LAN ISP 2 with ports (6,7)

WAN ISP1 is on ETH8
WAN ISP2 is on ETH9

route:

add comment=ISP1 disabled=no distance=1 dst-address=0.0.0.0/0 gateway=
PPPoE-OUT routing-table=main scope=30 suppress-hw-offload=no
target-scope=10
add comment=ISP2 disabled=no distance=2 dst-address=0.0.0.0/0 gateway=
178.x.x.x routing-table=main scope=30 suppress-hw-offload=no
target-scope=10

NAT:

/ip firewall nat add chain=srcnat src-address=10.10.0.0/24 action=masquerade out-interface=PPPoE-OUT
/ip firewall nat add chain=srcnat src-address=10.11.0.0/24 action=masquerade out-interface=WAN

How to mark this traffic to setup corectly mangle table?Or maybe i dont need to use mangle at all?

And setup like this?

/ip/route/add dst-address=0.0.0.0/0 gateway=PPPoE-OUT routing-table=main distance=1
/ip/route/add dst-address=0.0.0.0/0 gateway=178.x.x.x routing-table=isp2 distance=1

/routing/rule/add src-address=10.10.0.0/24 action=lookup table=main
/routing/rule/add src-address=10.11.0.0/24 action=lookup table=isp2

/ip/firewall/nat/add chain=srcnat src-address=10.10.0.0/24 action=masquerade out-interface=PPPoE-OUT
/ip/firewall/nat/add chain=srcnat src-address=10.11.0.0/24 action=masquerade out-interface=WAN

With VPNs to the router involved mangling is required.
Since you didnt state otherwise, it would appear you have no port forwardings involved.

post your config on what you have so far…
/export file=anynameyouwish ( minus router serial number, any publicWANIP information, keys etc. )

In this setup connection is working
10.10.0.0/24-> ISP1
10.11.0.0/24-> ISP2

Main problem at this moment is no communication between subnets(PCs, server)
10.10.0.253 can ping 10.11.0.1(gateway) but not server 10.11.0.3 ( no icmp no access SMB/www/etc)

/interface bridge
add name=LAN_ISP_ETTH
add arp=proxy-arp fast-forward=no name=LAN_ISP_PPPoE port-cost-mode=short
/interface ethernet
set [ find default-name=ether1 ] rx-flow-control=auto tx-flow-control=auto
set [ find default-name=ether10 ] poe-out=forced-on rx-flow-control=auto tx-flow-control=auto
/interface wireguard
DELETED
/interface vlan
add interface=ether8 name=VlanPPPoE vlan-id=35
/interface pppoe-client
add allow=pap disabled=no interface=VlanPPPoE name="PPPoE Gateway" user=DELETED
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec mode-config
set [ find default=yes ] connection-mark=no-mark src-address-list=local-wifi
/ip ipsec peer
add address=DELETED/32 exchange-mode=ike2 local-address=DELETED name=we port=500
/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 dpd-interval=2m dpd-maximum-failures=5 enc-algorithm=aes-128 name=q
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-128-cbc lifetime=1d
/ip pool
add name=LAN_ ranges=10.10.0.2-10.10.0.254
add name=DELETED ranges=10.240.0.2-10.240.0.10
add name=DELETED_vpn ranges=172.16.1.10-172.16.1.20
add name="BAD LAN VPN" ranges=10.50.0.2-10.50.0.100
add name=test_vpn ranges=172.16.2.2-172.16.2.10
add name=LAN_ETTH ranges=10.11.0.2-10.11.0.254
/ip dhcp-server
add address-pool=LAN interface=LAN_ISP_PPPoE lease-time=10m name=LAN_PPPoE
add address-pool=LAN_ETTH interface=LAN_ISP_ETTH lease-time=10m name=LAN_ETTH
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
set 1 name=serial1
/interface l2tp-client
add allow=mschap2 connect-to=DELETED disabled=no name=l2tp-out1 profile=default user=DELETED
/interface sstp-client
add authentication=mschap2 certificate=DELETED connect-to=DELETED disabled=no http-proxy=0.0.0.0 name=sstp-out1 profile=default-encryption user=DELETED
/routing bgp template
set default disabled=no output.network=bgp-networks
/routing ospf instance
add disabled=no name=default-v2
/routing ospf area
add disabled=yes instance=default-v2 name=backbone-v2
/routing table
add disabled=no fib name=ISP_VIA_ETTH
/snmp community
add addresses=::/0 name=DELETED
/system logging action
set 3 remote=10.10.0.3 src-address=10.10.0.1
/interface bridge port
add bridge=LAN_ISP_PPPoE ingress-filtering=no interface=ether1 internal-path-cost=10 path-cost=10 trusted=yes
add bridge=LAN_ISP_PPPoE ingress-filtering=no interface=ether4 internal-path-cost=10 path-cost=10 trusted=yes
add bridge=LAN_ISP_PPPoE ingress-filtering=no interface=ether3 internal-path-cost=10 path-cost=10 trusted=yes
add bridge=LAN_ISP_PPPoE ingress-filtering=no interface=ether2 internal-path-cost=10 path-cost=10 trusted=yes
add bridge=LAN_ISP_PPPoE ingress-filtering=no interface=ether10 internal-path-cost=10 path-cost=10 trusted=yes
add bridge=LAN_ISP_PPPoE interface=ether5 internal-path-cost=10 path-cost=10
add bridge=LAN_ISP_ETTH interface=ether7
add bridge=LAN_ISP_ETTH interface=ether6
/ip firewall connection tracking
set enabled=yes udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=all
/ip settings
set max-neighbor-entries=8192 rp-filter=strict
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface l2tp-server server
set authentication=mschap2 default-profile=*1 use-ipsec=yes
/interface ovpn-server server
set auth=sha1,md5
/interface sstp-server server
set authentication=mschap2 certificate=server_xxx enabled=yes pfs=yes port=20197 tls-version=only-1.2
/interface wireguard peers
DELETED
/ip address
add address=10.210.0.1/24 interface=ether9 network=10.210.0.0
add address=10.10.0.1/24 interface=LAN_ISP_PPPoE network=10.10.0.0
add address=10.100.0.1/30 interface=wireguard1 network=10.100.0.0
add address=172.16.2.2/24 interface=xx network=172.16.2.0
add address=10.200.0.1/24 interface=xx network=10.200.0.0
add address=10.5.0.1/24 interface=PHONE network=10.5.0.0
add address=10.11.0.1/24 interface=LAN_ISP_ETTH network=10.11.0.0
/ip cloud
set update-time=no
/ip dhcp-client
add add-default-route=no interface=ether9 use-peer-dns=no
/ip dhcp-server
add address-pool=*4 disabled=yes interface=*12 lease-time=10m name=server1
/ip dhcp-server lease
add address=10.11.0.4 client-id=1:0:11:32:ba:a4:82 mac-address=00:11:32:BA:A4:82 server=LAN_ETTH
add address=10.11.0.3 client-id=1:0:11:32:ba:a4:81 mac-address=00:11:32:BA:A4:81 server=LAN_ETTH
/ip dhcp-server network
add address=10.10.0.0/24 dns-server=DELETED gateway=10.10.0.1 netmask=24
add address=10.11.0.0/24 dns-server=DELETED gateway=10.11.0.1 netmask=24
add address=10.50.0.0/24 dns-server=10.10.0.3 gateway=10.50.0.1 netmask=24
/ip dns
set servers=8.8.8.8,8.8.4.4
/ip firewall filter
add action=fasttrack-connection chain=forward comment="Fast Track Established / Related Forward" connection-state=established,related hw-offload=yes
add action=log chain=input log=yes protocol=tcp src-port=2049
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="Port scanners to list " protocol=tcp psd=21,3s,3,1
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP FIN Stealth scan" protocol=tcp tcp-flags=fin,!syn,!rst,!psh,!ack,!urg
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="SYN/FIN scan" protocol=tcp tcp-flags=fin,syn
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="SYN/RST scan" protocol=tcp tcp-flags=syn,rst
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="FIN/PSH/URG scan" protocol=tcp tcp-flags=fin,psh,urg,!syn,!rst,!ack
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="ALL/ALL scan" protocol=tcp tcp-flags=fin,syn,rst,psh,ack,urg
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP NULL scan" protocol=tcp tcp-flags=!fin,!syn,!rst,!psh,!ack,!urg
add action=accept chain=input comment="Accept Established / Related Input" connection-state=established,related
add action=accept chain=input src-address=10.10.0.0/24
add action=accept chain=input src-address=10.11.0.0/24
add action=accept chain=input src-address=172.16.1.0/24
add action=accept chain=input src-address=10.100.0.0/24
add action=accept chain=forward comment="Accept Established / Related Forward" connection-state=established,related
add action=accept chain=forward src-address=10.210.0.0/24
add action=accept chain=forward dst-address=10.11.0.0/24 src-address=10.10.0.0/24
add action=accept chain=forward dst-address=10.10.0.0/24 src-address=10.11.0.0/24
add action=accept chain=forward dst-address=10.0.0.0/24 src-address=10.10.0.0/24
add action=drop chain=forward dst-address=10.10.0.0/24 src-address=192.168.250.0/24
add action=drop chain=input comment="Drop Input" log-prefix="drop input"
/ip firewall nat
add action=masquerade chain=srcnat out-interface="PPPoE Gateway" src-address=10.10.0.0/24
add action=masquerade chain=srcnat out-interface=ether9 src-address=10.11.0.0/24
add action=accept chain=srcnat dst-address=192.168.60.0/24 src-address=10.10.0.0/24
add action=accept chain=srcnat dst-address=10.11.0.0/24 src-address=10.10.0.0/24
add action=accept chain=srcnat dst-address=10.10.0.0/24 src-address=10.11.0.0/24
add action=masquerade chain=srcnat dst-address=192.168.255.0/24 out-interface=xx
add action=masquerade chain=srcnat dst-address=192.168.254.0/24 out-interface=sstp-out1
add action=masquerade chain=srcnat out-interface=l2tp-out1 src-address=10.10.0.0/24
add action=masquerade chain=srcnat disabled=yes
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set h323 disabled=yes
set sip disabled=yes
set pptp disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip ipsec identity
add generate-policy=port-strict peer=we
/ip ipsec policy
set 0 disabled=yes
add dst-address=192.168.60.0/24 peer=we src-address=10.10.0.0/24 tunnel=yes
/ip route
add disabled=no distance=1 dst-address=192.168.254.0/24 gateway=sstp-out1 pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=192.168.20.0/24 gateway=*C routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=10.0.0.0/24 gateway=172.16.1.4 pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=172.16.0.0/16 gateway=l2tp-out1 routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=192.168.250.0/24 gateway=10.100.0.2 routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=192.168.255.0/24 gateway=172.16.2.1 pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=178.x.x.x pref-src="" routing-table=ISP_VIA_ETTH scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway="PPPoE Gateway" pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox address=10.10.0.0/24,10.210.0.0/24
set api-ssl disabled=yes
/ip smb shares
set [ find default=yes ] directory=/pub
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ip upnp
set show-dummy-rule=no
/ppp secret
DELTED
/routing bfd configuration
add disabled=no
/routing rule
add action=lookup-only-in-table disabled=no interface=LAN_ISP_ETTH table=ISP_VIA_ETTH
add action=lookup-only-in-table disabled=no src-address=10.10.0.0/24 table=main
/snmp
set enabled=yes trap-community=xxx trap-version=2
/system clock
DELETED
/system identity
set name=wtf
/system leds
set 0 type=off
/system logging
set 0 action=remote
set 1 action=remote
set 2 action=remote
set 3 action=remote
add action=disk topics=critical
add action=disk topics=error
add action=disk topics=info
add action=disk topics=warning
add disabled=yes topics=ipsec,!packet
add disabled=yes prefix=ipsec topics=ipsec
/system note
set note=Access to this device is monitored
/system ntp client
set enabled=yes
/system ntp server
set enabled=yes manycast=yes
/system resource irq rps
set sfp-sfpplus1 disabled=no
/tool bandwidth-server
set enabled=no
/tool mac-server
set allowed-interface-list=none
/tool mac-server mac-winbox
set allowed-interface-list=none
/tool mac-server ping
set enabled=no

The first thing to do in mangling is to ensure traffic heading for WAN2 is sent back out via WAN2.
This should direct all VPNs coming on WAN2 to go back out WAN2.
Of course one must have standard sourcenat rule in place as well

Your wireguard rule seems to be deleted not sure why… in anycase
will call the interrface wg-vpn and the port being used 15667

/ip mangle
add chain=input action=mark-connections connection-mark=no-mark in-interface=ether9
new-connection-mark=incoming-wan2 passthrough=yes
add chain=output action=mark-routing connection-mark=incoming-wan2
new-routing-mark=ISP_VIA_ETTH passthrough=no

Currently one needs to add some trickery for wireguard as it doesnt play nice and will respond to initial handshake incorrectly via WAN1despite the mangling.
This can be overcome by adding a NAT rule.
add action=dst-nat chain=dst-nat dst-address-type=local in-interface=ether9 dst-port=15667 protocol=udp to-addresses=fixedWAN2-IPaddress

This basically tricks the router into undestinatting any wireguard traffic back through WAN2.


NEXT we mangle for all LAN TRAFFIC GOING OUT WAN2…
/ip mangle
add action=mark-connections chain=forward in-interface-list=LAN src-address=10.11.0.0/24
dst-address-type=!local connection-mark=no-mark new-connection-mark=from-LAN2 passthrough=yes

{ route the traffic out the special table for WAN2 }
add action=mark-routing chain=prerouting connection-mark=from-LAN2
new-routing-mark=ISP_VIA_ETTH passthrough=no

Note: To ensure normal traffic going out WAN is not affected in terms of fastrack rule in the forward chain ensure you add to the fastrack rule: connection-mark=no-mark

+++++++++++++++++++++++++++++++++++++++++++++++

Then we can deal with routes…
I am going to assume that there is no real backup mode, aka WAN1 users should not go to WAN2 and WAN2 user should not go to WAN1 if the respective internet is not working, or more clearly you didnt state requirements for such, and thus not entertained. The only real difference here is that I have used distance to ensure that The main gateway is the default gateway that all users will be pointed towards, and that will be used to ensure your normal LAN uses WAN1. We use mangling to force the NAS lan to use WAN2.
We use check-gateway=ping to ensure that when the internet comes back up the router keeps checking to see when it does and enables it for use.

/ip route
add check-gateway=ping dst-address=0.0.0.0/0 gateway=“PPPoE Gateway” routing-table=main
add distance=2 dst-address=0.0.0.0/0 blackhole=yes routing-table=main
add check-gateway=ping distance=4 dst-address=0.0.0.0/0 gateway=ISP2-gatewayIP routing-table=main

Now by default all users will use WAN1 for internet and if its unavailable, will be sent to black hole aka nowhere
Next we ensure that NAS subnet users that are captured by our mangling get routed out WAN21.

add dst-address=0.0.0.0/0 gateway=ISP2-gatewayIP routing-table=ISP_VIA_ETTH

Next lets look at other rules…

  1. CHANGE THIS TO
    /ip settings
    set max-neighbor-entries=8192 rp-filter=loose

  2. DECIDE!!! Either use IP DHCP client for ether9 OR assign an IP address but not both !!!
    Since its a static IP, setting the IP address makes sense and I ssuspect you just entered a generic representative entry ( but get rid of client )
    (also confusing as you use an address that is markedly different from its gateway in your routes… not a big deal…)

  3. Firewall rules need work, simplify and add…

/ip firewall filter
{ default rules to keep }
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=icmp

{ admin rules}
add action=accept chain=input comment=“wireguard handshake” dst-port=15667 protocol=udp
{ ADD ALL SIMILAR NEEDED RULES FOR OTHER VPNS}
add action=accept chain=input comment=“admin access only” src-address-list=TRUSTED
add action=accept chain=input comment=“users to services” in-interface-list=LAN dst-port=53,123 protocol=udp
add action=accept chain=input comment=“users to services” in-interface-list=LAN dst-port=53protocol=tcp
add action=drop chain=input comment="drop all else
" { add this rule last or you could lock yourself out }
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
{default rules to keep}
add action=fasttrack-connection chain=forward connection-state=established,related connection-mark=no-mark
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid

{admin rules}
add action=accept chain=forward comment=“internet” in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment=“user LAN to NAS” src-address=10.10.0/24 dst-address=NAS-IP

{ ADD ANY ALLOW SIMILAR RULES FOR TRAFFIC NEEDED}
add action=drop chain=input comment=“Drop all else”
/ip firewall nat
add action=masquerade chain=srcnat out-interface=“PPPoE Gateway”
add action=masquerade chain=srcnat out-interface=ether9

Since I dont understand the need for any other sourcenat rules I didnt add them back in.
Make sure you actually need them…

Missing items so the above works…
/interface list
add name=WAN
add name=LAN
/interface list members
add interface=ether9 list=WAN
add interface=“PPPoE-Gateway” list=WAN
add interface=LAN_ISP_PPPoE list=LAN
add interface=LAN_ISP_ETTH list=LAN
add interface=wg-vpn list=LAN

/ip firewall address-list
add address=10.10.0.X list=TRUSTED comment=“admin device on normal LAN subnet”
add address=10.11.0.Y list=TRUSTED comment=“admin device on NAS subnet”
add address=10.100.0.2 list=TRUSTED comment=“remote admin laptop”

{ ADD ANY OTHER ADDRESSES for access to config the router }

Thank you.
I did apply mangle rules as you told. Connections between subnets are fine(working)
But there is no Internet Access from 10.11.0.0/24
Any ideas? // Sry didnt see your second post i will check IT tommorow.