VPN & Port forward through 1 Interface

djferdinad · Fri May 24, 2024 10:04 pm

Hello MK Community.

Currently I have 2 ISPs, (ISP1 - higher bandwidth & private IP, ISP2 - Lower bandwidth and Dynamic public IP).

With that being said, I have already set up Failover configuration using ISP1 as primary and ISP2 as secondary.

Expected behaviour

Having the failover configuration, would like to set up a Wireguard VPN and Port forwarding ONLY using ISP2.

Actual behaviour

When ISP1 is up, there is no packet responses. As soon as there is not ISP1 set as priority everything works, I did the PF and works, Wireguard VPN works.

/interface bridge
add admin-mac= auto-mac=no comment=defconf name=bridge
/interface ethernet
set [ find default-name=ether3 ] loop-protect=off
/interface wireguard
add listen-port=13231 mtu=1420 name=wireguard1
/disk
set usb1 type=hardware
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
/ip dhcp-server
add address-pool=default-dhcp interface=bridge lease-time=23h59m59s name=\
    defconf
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=sfp1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface detect-internet
set detect-interface-list=WAN internet-interface-list=WAN lan-interface-list=\
    LAN wan-interface-list=WAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=wireguard1 list=LAN
add interface=*9 list=WAN
add interface=*A list=WAN
/interface wireguard peers
add allowed-address=10.20.1.2/32 interface=wireguard1 public-key=\
    "VnTNWEPEIGe4ehffWqtG8GdIb+HKxcpSvACRekuVa1I="
add allowed-address=10.20.1.3/32 interface=wireguard1 public-key=\
    "D2bLdRCWi8QS/xznIUHNzufVZOpwX2pVdnf+0WcNr1k="
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=\
    192.168.88.0
add address=10.20.1.1/24 interface=wireguard1 network=10.20.1.0
/ip cloud
set ddns-enabled=yes ddns-update-interval=5m
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server lease
add address=192.168.88.252 client-id=1:d8:32:14:e4:1c:88 mac-address=\
    D8:32:14:E4:1C:88 server=defconf
add address=192.168.88.247 mac-address=08:00:27:A1:50:F7 server=defconf
add address=192.168.88.253 client-id=1:90:61:ae:16:b0:1b mac-address=\
    90:61:AE:16:B0:1B server=defconf
add address=192.168.88.251 client-id=1:3c:19:cb:6e:7:2f mac-address=\
    3C:19:CB:6E:07:2F server=defconf
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=\
    192.168.88.1
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
/ip firewall address-list
add address=192.168.0.0/16 list=Access
add address=190.0.0.0/8 list=Access
add address=186.0.0.0/8 list=Access
add address=200.0.0.0/8 list=Access

/ip firewall filter
add action=reject chain=forward dst-address-list=Ads protocol=tcp reject-with=icmp-network-unreachable src-address-list=Access
add action=accept chain=input comment="allow WireGuard" dst-port=13231 \
    protocol=udp
add action=accept chain=input comment="allow WireGuard traffic" src-address=\
    10.20.1.0/24
add action=accept chain=input in-interface=ether2 src-address-list=Access
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid disabled=yes
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid disabled=yes
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new disabled=yes in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
add action=dst-nat chain=dstnat dst-port=23000 in-interface=ether1 protocol=\
    tcp to-addresses=192.168.88.252 to-ports=80
add action=dst-nat chain=dstnat dst-port=24000 in-interface=ether1 protocol=\
    tcp to-addresses=192.168.88.247 to-ports=443
add action=dst-nat chain=dstnat dst-port=25000 in-interface=ether1 protocol=\
    udp to-addresses=192.168.88.247 to-ports=5060
add action=dst-nat chain=dstnat disabled=yes dst-port=10000-20000 \
    in-interface=ether1 protocol=udp to-addresses=192.168.88.247 to-ports=\
    10000-20000

Thank you

anav · Fri May 24, 2024 10:57 pm

Hi there, yes there is a bug in Wireguard firmware, which prevents success using Wireguard on WAN2, when WAN1 is primary.
We can fix that with some trickery.

A quick perusal of your config also shows that you have other issues that need to be addressed first.
a. you are port forwarding on ether1, HOW? If you cannot use ether1 because its a private IP for wireguard, equally you cannot use ether1 for port forwarding.
b. You have other issues with WANs ( I dont see a working WAN2 ) and see errors..
add interface=*9 list=WAN
add interface=*A list=WAN

Until we get to the truth about requirements not worth it to look at the config.

I see you have two wireguard remote connections, that seems fine, wireguard address is fine.
The access list you created is problematic, especially using it as a WAN input source for ether2??
Not sure what you are trying to do here...........more mysteries that need to be unravelled prior to any config review.

djferdinad · Sat May 25, 2024 5:31 am

Thank you for your review.

Indeed, PF on ether1 was an old configuration that is not in use and I just deleted. When needed I can post the most recent changes.

Access Lists (not sure if that is a correct configuration) does intend to restrict the login access to the router, and allow only certain IPs.

anav · Sat May 25, 2024 7:50 pm

Yes the latest config,
In terms of restricting access TO the router input chain..
The only connection, TO the ROUTEr, should be VPN connections and thus no restrictions required.
Connection to the LAN, aka to servers, can be limited by sourc address list on the dstnat rules.

djferdinad · Tue May 28, 2024 3:38 pm

Sure, that totally makes sense. It is more secure.

Here is the latest config

# 2024-05-27 13:33:36 by RouterOS 7.15rc3

 /interface bridge
add admin-mac=74:4D:28:84:6C:D2 auto-mac=no comment=defconf name=bridge \
    port-cost-mode=short

 /interface wireguard
add listen-port=13231 mtu=1420 name=wireguard1

 /ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254

 /ip dhcp-server
add address-pool=default-dhcp interface=bridge lease-time=10m name=defconf

 /routing table
add fib name=useWAN2
add fib name=useWAN1

 /interface bridge port
add bridge=bridge comment=defconf interface=ether3 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether4 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether5 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=sfp1 internal-path-cost=10 \
    path-cost=10

 /interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=ether2 list=WAN

 /interface wireguard peers
add allowed-address=10.20.1.2
 /32 interface=wireguard1 name=peer1 public-key=\
    "Q9DT32+x1OrsRAJkdUbxsHG5Bj8BGmWIqeOBZ9ilOiw="

 /ip address
add address=192.168.88.1
 /24 comment=defconf interface=bridge network=\
    192.168.88.0
add address=10.20.1.1
 /24 interface=wireguard1 network=10.20.1.0

 /ip cloud
set ddns-enabled=yes ddns-update-interval=1m

 /ip dhcp-client
add add-default-route=no comment=defconf interface=ether1
add default-route-distance=5 interface=ether2 script=":if (\$bound=1) do={\r\
    \n:local gw \$\"gateway-address\"\r\
    \n
 /ip route set [ find comment=\"Secondary WAN\" gateway!=\$gw ] gateway=\
    \$gw\r\
    \n
 /ip route set [ find comment=\"Wan2 Routing table\" gateway!=\$gw ] gate\
    way=\$gw\r\
    \n}" use-peer-dns=no use-peer-ntp=no


 /ip firewall address-list
add address=cloud.mikrotik.com list=MyCloud
add address=cloud2.mikrotik.com list=MyCloud
add address=192.168.0.0
 /16 list=Access
add address=190.120.0.0
 /16 list=Access
add address=186.0.0.0
 /8 list=Access
add address=200.0.0.0
 /8 list=Access

 /ip firewall filter
add action=accept chain=input comment="allow WireGuard" dst-port=13231 \
    protocol=udp
add action=accept chain=input comment="allow WireGuard traffic" src-address=\
    10.20.1.0
 /24
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related disabled=yes hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN

 /ip firewall mangle
add action=mark-routing chain=output dst-address-list=MyCloud dst-port=15252 \
    new-routing-mark=useWAN2 passthrough=no protocol=udp
add action=mark-routing chain=prerouting dst-port=13231 new-routing-mark=\
    useWAN2 passthrough=no protocol=udp
add action=mark-connection chain=input connection-mark=no-mark in-interface=\
    ether2 new-connection-mark=incomingWAN2 passthrough=yes
add action=mark-routing chain=output connection-mark=incomingWAN2 \
    new-routing-mark=useWAN2 passthrough=no
add action=mark-connection chain=input comment=Wan1 connection-mark=no-mark \
    in-interface=ether1 new-connection-mark=wan1_conn passthrough=yes
add action=mark-routing chain=output connection-mark=wan1_conn \
    new-routing-mark=useWAN1 passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark disabled=yes \
    dst-address-type=!local new-connection-mark=wan1_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:2
 /0 src-address=\
    192.168.88.0
 /24
add action=mark-connection chain=forward connection-mark=no-mark disabled=yes \
    dst-address-type=!local new-connection-mark=wan2_conn passthrough=yes \
    per-connection-classifier=both-addresses-and-ports:2
 /1 src-address=\
    192.168.88.0
 /24
add action=mark-routing chain=prerouting connection-mark=wan1_conn disabled=\
    yes new-routing-mark=useWAN1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=wan2_conn disabled=\
    yes new-routing-mark=useWAN2 passthrough=no
add action=mark-routing chain=prerouting connection-mark=wan1_conn disabled=\
    yes new-routing-mark=useWAN1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=wan2_conn disabled=\
    yes new-routing-mark=useWAN2 passthrough=no

 /ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
add action=dst-nat chain=dstnat comment="serv " dst-port=24000 \
    in-interface-list=WAN protocol=tcp src-address-list=Access to-addresses=\
    192.168.88.10 to-ports=22

 /ip route
add check-gateway=ping comment=Netw disabled=no distance=1 dst-address=\
    0.0.0.0
 /0 gateway=1.1.1.1 pref-src="" routing-table=main scope=30 \
    suppress-hw-offload=no target-scope=32
add comment="Secondary WAN" disabled=no distance=3 dst-address=9.9.9.9
 /32 \
    gateway=190.199.224.1 pref-src="" routing-table=main scope=10 \
    suppress-hw-offload=no target-scope=11
add disabled=no distance=1 dst-address=1.1.1.1
 /32 gateway=172.16.0.1 \
    pref-src="" routing-table=main scope=30 suppress-hw-offload=no \
    target-scope=31
add disabled=yes distance=3 dst-address=8.8.8.8
 /32 gateway=172.16.0.1 \
    pref-src="" routing-table=main scope=30 suppress-hw-offload=no \
    target-scope=31
add check-gateway=ping comment="Wan2 Routing table" disabled=no distance=1 \
    dst-address=0.0.0.0
 /0 gateway=190.199.224.1 pref-src="" routing-table=\
    useWAN2 scope=30 suppress-hw-offload=no target-scope=10

anav · Tue May 28, 2024 5:34 pm

(1) Okay so if your port forwarding was old news and not required, then why do you still have dstnat port forwarding rule........
add action=dst-nat chain=dstnat comment="serv " dst-port=24000 \
in-interface-list=WAN protocol=tcp src-address-list=Access to-addresses=\
192.168.88.10 to-ports=22 ???????

(2) I don't recall seeing a name entry in allowed IPs?
/interface wireguard peers
add allowed-address=10.20.1.2/32 interface=wireguard1 name=peer1??? public-key=\
"Q9DT32+x1OrsRAJkdUbxsHG5Bj8BGmWIqeOBZ9ilOiw="

(3) Add to list.
/interface list member
add comment=defconf interface=bridge list=LAN
add interface=wireguard1 list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=ether2 list=WAN

(4) Filter rules modified.
/ip firewall address-list
add address=192.168.88.XY list=Authorized comment="admin local PC"
add address=10.20.1.2 list=Authorized comment="admin remote laptop"

/ip firewall filter
{ default rules to keep }
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=icmp
add action=accept chain=input dst-address=127.0.0.1
{ admin rules }
add action=accept chain=input comment="WG handshake" dst-port=13231 protocol=udp
add action=accept chain=input comment="admin access" in-interface-list=LAN src-address-list=Authorized
add action=accept chain=input comment="users services" in-interface-list=LAN dst-port=53 protocol=udp
add action=accept chain=input comment="users services" in-interface-list=LAN dst-port=53 protocol=tcp
add action=drop chain=input comment="drop all else" { ensure you put this rule in LAST }
+++++++++++++++++++++++++++++++++++++++++++++++++++
{ default rules to keep }
add action=fasttrack-connection chain=forward connection-state=established,related disabled=no hw-offload=yes connection-mark=no-mark
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward comment="internet traffic" in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment="Wg to LAN" in-interface=wireguard1 dst-address=192.168.88.0/24
add action=accept chain=forward comment="port forwarding" connection-nat-state=dstnat disabled=yes { enable or remove as required }
add action=drop chain=forward comment="drop all else"

(5) MANGLING, we know that mangling doesnt quite solve the wireguard requirements due to bug in the software, however we will approach the problem as though it works and make some necessary adjustments.

Typically mangling has three main thrusts.
a. traffic to the router ( services on the router, such as wireguard )
b. traffic to the LAN ( servers etc....)
c. Traffic from the LAN ( PCC or forcing specific users out a specific WAN )

In this scenario I only see A, which means we need to attempt to ensure any traffic coming on WAN2 departs on WAN2

/ip firewall mangle
add chain=input action=mark-connection in-interface=WAN2 connection-mark=no-mark \
new-connection-mark=fromWAN2 passthrough=yes
add chain=prerouting action=mark-route connection-mark=fromWAN2 \
new-routing-mark=useWAN2 passthrough=no

Done!

6. Routing is the next hurdle. In this case we need the two main table routes and then one special table route for WAN2 and wireguard handshake and tunnel traffic.
Appears to be recursive based, fixed:

add check-gateway=ping distance=1 dst-address=0.0.0.0 /0 gateway=1.1.1.1 routing-table=main scope=30 target-scope=32
add check-gateway=ping distance=3 dst-address=0.0.0.0/0 gateway=9.9.9.9 routing-table=main scope=30 target-scope=32
+++++++++++++
add distance=1 dst-address=1.1.1.1/32 gateway=172.16.0.1 routing-table=main scope=30 target-scope=31
add distance=3 dst-address=9.9.9.9/32 gateway=190.199.224.1 routing-table=main scope=30 target-scope=31 comment="WAN2 Routing"

Now for the special table.
add dst-address=0.0.0.0/0 gateway=190.199.224.1 routing-table=useWAN2 comment="WAN2 Routing"

7. Now, we will revisit IP Routes, additions in purple.
The issue in this para is your IP DHCP CLIENT script and settings for the dynamic WAN2.
To ensure the IP routes are changed when WAN2 is updated there is a couple of more steps to consider.
Its mostly there actually.... but simplified!!

From:
add default-route-distance=5 interface=ether2 script=":if (\$bound=1) do={\r\
\n:local gw \$\"gateway-address\"\r\
\n
/ip route set [ find comment=\"Secondary WAN\" gateway!=\$gw ] gateway=\
\$gw\r\
\n
/ip route set [ find comment=\"Wan2 Routing table\" gateway!=\$gw ] gate\
way=\$gw\r\
\n}" use-peer-dns=no use-peer-ntp=no

TO:

add default-route-distance=255 interface=ether2 script=":if (\$bound=1) do={\r\
\n:local gw \$\"gateway-address\"\r\
\n/ip route set [ find comment=\"WAN2 Routing\" gateway!=\$gw ] gateway=\$gw\r\
\n}\r\
\n" use-peer-dns=no use-peer-ntp=no

Note: The entry of a default route distance means default route=yes has been selected.

8. Now the secret sauce for WAN2 wireguard to actually work, a NAT RULE.

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
ipsec-policy=out,none out-interface-list=WAN
add action=dstnat chain=dst-nat in-interface=ether2 dst-address-type=local dst-port=13231 protocol=udp to-address=172.16.0.1

Basically, we trick the router into ensuring any wireguard traffic that incorrectly bleeds from WAN1 is un-destinatted to WAN2!

VPN & Port forward through 1 Interface

VPN & Port forward through 1 Interface

Re: VPN & Port forward through 1 Interface

Re: VPN & Port forward through 1 Interface

Re: VPN & Port forward through 1 Interface

Re: VPN & Port forward through 1 Interface

Re: VPN & Port forward through 1 Interface

Who is online