WG over secondary WAN

goebeler · September 27, 2023, 2:44pm

Hi everyone,
I just got myself my first Mkrotik Router and wanted to move my wireguard VPN from a dedicated server to the router. But I cant get it to work.

My problem is very similar to the one described here: http://forum.mikrotik.com/t/wireguard-on-one-of-multiple-wan-interfaces/169843/1 but I have not been able to fix it, and after two days of trying different approaches the configuration might be a total mess now.

FYI the second WAN (ether2) should only be used for WG. The use case for WG is accessing service in the Lan from remote laptops and phones (mail, rdp, internal websites etc.). Using it as a secure tunnel for browsing the internet would be bonus.

My setup:

/interface/ print
Flags: R - RUNNING; S - SLAVE
Columns: NAME, TYPE, ACTUAL-MTU, L2MTU, MAX-L2MTU, MAC-ADDRESS
 #    NAME        TYPE    ACTUAL-MTU  L2MTU  MAX-L2MTU  MAC-ADDRESS      
 0 R  ether1      ether         1500   1598       8156  78:9A:18:21:F9:58 <= Connection to ADSL-Router
 1 R  ether2      ether         1500   1598       8156  78:9A:18:21:F9:59 <= Direct ISP Connection with public static IP
 2 RS ether3      ether         1500   1598       8156  78:9A:18:21:F9:5A <= LAN Switch
 3  S ether4      ether         1500   1598       8156  78:9A:18:21:F9:5B
 4  S ether5      ether         1500   1598       8156  78:9A:18:21:F9:5C
 5  S ether6      ether         1500   1598       8156  78:9A:18:21:F9:5E
 6  S ether7      ether         1500   1598       8156  78:9A:18:21:F9:5F
 7  S ether8      ether         1500   1598       8156  78:9A:18:21:F9:60
 8  S ether9      ether         1500   1598       8156  78:9A:18:21:F9:61
 9  S ether10     ether         1500   1598       8156  78:9A:18:21:F9:62
10  S sfp1        ether         1500   1600       8158  78:9A:18:21:F9:5D
;;; defconf
11 R  bridge      bridge        1500   1598             78:9A:18:21:F9:59
12 R  wireguard1  wg            1420

Adresses:

/ip/address/ print
Flags: D - DYNAMIC
Columns: ADDRESS, NETWORK, INTERFACE
#   ADDRESS           NETWORK        INTERFACE 
;;; defconf
0   192.168.1.254/24  192.168.1.0    bridge
1   [Our-public-IP]/30  xx.yy.zz.128   ether2    
2   10.23.5.1/24      10.23.5.0      wireguard1
3 D 192.168.2.4/24    192.168.2.0    ether1  <= assigned via dhcp from adsl router

wireguard:

/interface wireguard
add listen-port=34081 mtu=1420 name=wireguard1
/interface wireguard peers
add allowed-address=10.23.5.107/32,fc00:23:5::107/128 comment=CG-Phone interface=wireguard1 persistent-keepalive=25s public-key="mHptTViD+kONfL5y97Z2d3+xxxxxxxxxxxxxxxxxxxxx"

firewall rules:

/ip firewall filter
add action=accept chain=input comment="Accept WireGuard on Port 34081" dst-port=34081 log=yes log-prefix=wg_in protocol=udp
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN log=yes log-prefix=DROPPED
/ip firewall mangle
add action=mark-packet chain=prerouting connection-mark=no-mark dst-port=34081 in-interface=ether2 log-prefix=MARK-PACK new-packet-mark=wg passthrough=yes protocol=udp
add action=mark-connection chain=prerouting log-prefix=MARK-CONN new-connection-mark=from-vpn packet-mark=wg passthrough=yes
add action=mark-routing chain=prerouting connection-mark=from-vpn log-prefix=MARK-ROUTE new-routing-mark=wgtab passthrough=no
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

Routing:

/routing table
add disabled=no fib name=wgtab
/routing rule
add action=lookup-only-in-table disabled=no dst-address=0.0.0.0/0 routing-mark=wgtab src-address=0.0.0.0/0 table=wgtab

Tell me if anything is missing.

goebeler · September 27, 2023, 2:50pm

Oh and The problem seems to be that all wireguard answers are being sent through ether1, following the default route. It’s as if the mangling didn’t really do anything. also, wen looking at the wireguard interface using the packet sniffer, there seems to be no traffic at all.

On the other hand, whenn I sniff all interfaces I see that after every handshake request coming in through ether2 there is a packet being sent back through ether1.

Also: Torch is not showing any traffice on the wireguard interface (besides some discovery/neighborhood packets)

anav · September 27, 2023, 3:36pm

When you provide a complete config, I can answer, wont attempt on what you perceive are the issue areas…

There are two ways to ensure traffic coming in on WAN2 goes out WAN2
Either way will need table, and the following route in addition to the standard routing for both WANs…
add dst-address=0.0.0.0 gateway=ISP2-gateway-IP routing-table=TO_WAN2

Then you can use routing rule or mangle to ensure traffic hittting WAN2 (handshake) goes back out WAN2
/routing rule
add action=lookup-only-in-table src-address=WAN2-IP-address table=TO_WAN2

or mangle…

/ip firewall mangle
add chain=prerouting action=mark-connection connection-mark=no-mark
in-interface=WAN_2 new-connection-mark=incomingWAN2 passthrough=yes
add chain=output action=mark-routing connection-mark=incomingWAN2
new-routing mark=TO_WAN2 passthrough=no

Note1: add action=fasttrack-connection chain=forward comment=“defconf: fasttrack”
connection-state=established,related hw-offload=yes connection-mark=no-mark

Note2: USES the same table and IP route we created above.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

goebeler · September 27, 2023, 3:49pm

# 2023-09-27 17:45:49 by RouterOS 7.11.2
# software id = 2R1F-QITK
#
# model = RB3011UiAS
# serial number = 
/interface bridge
add admin-mac=78:9A:18:21:F9:59 auto-mac=no comment=defconf name=bridge
/interface wireguard
add listen-port=34081 mtu=1420 name=wireguard1
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/routing table
add disabled=no fib name=wgtab
/interface bridge port
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=ether6
add bridge=bridge comment=defconf interface=ether7
add bridge=bridge comment=defconf interface=ether8
add bridge=bridge comment=defconf interface=ether9
add bridge=bridge comment=defconf interface=ether10
add bridge=bridge comment=defconf interface=sfp1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add comment=Boerde interface=ether2 list=WAN
add interface=wireguard1 list=LAN
/interface wireguard peers
add allowed-address=10.23.5.107/32,fc00:23:5::107/128 comment=CG-Phone interface=wireguard1 persistent-keepalive=25s public-key="XXX"
/ip address
add address=192.168.1.254/24 comment=defconf interface=bridge network=192.168.1.0
add address=[PUBLIC_IP]/30 interface=ether2 network=XX.YY.ZZ.128
add address=10.23.5.1/24 interface=wireguard1 network=10.23.5.0
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server network
add address=192.168.1.0/24 dns-server=192.168.1.249 gateway=192.168.1.254
/ip dns
set allow-remote-requests=yes
/ip firewall filter
add action=accept chain=input comment="Accept WireGuard on Port 34081" dst-port=34081 log=yes log-prefix=wg_in protocol=udp
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN log=yes log-prefix=DROPPED
/ip firewall mangle
add action=mark-packet chain=prerouting connection-mark=no-mark disabled=yes dst-port=34081 in-interface=ether2 log-prefix=MARK-PACK new-packet-mark=wg passthrough=yes protocol=udp
add action=mark-connection chain=prerouting disabled=yes log-prefix=MARK-CONN new-connection-mark=from-vpn packet-mark=wg passthrough=yes
add action=mark-routing chain=prerouting connection-mark=from-vpn disabled=yes log-prefix=MARK-ROUTE new-routing-mark=wgtab passthrough=no
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=[ISP-Gateway-IP] pref-src="" routing-table=wgtab scope=30 suppress-hw-offload=no target-scope=10
/ipv6 route
add disabled=no dst-address=::/0 gateway=ether2 routing-table=wgtab
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=33434-33534 protocol=udp
add action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
/lcd
set time-interval=hour
/lcd pin
set pin-number=XXXX
/routing rule
add action=lookup-only-in-table disabled=no dst-address=0.0.0.0/0 routing-mark=wgtab src-address=0.0.0.0/0 table=wgtab
/system clock
set time-zone-name=Europe/Berlin
/system note
set show-at-login=no
/system routerboard settings
set auto-upgrade=yes
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool sniffer
set filter-interface=all filter-operator-between-entries=and filter-port=34081

goebeler · September 28, 2023, 8:36am

Well, I do have such a routing rule and second routing table.

I also changed the mangle rules as per your example. But I noticed that the output chain rule never gets triggered.