I have an office network with a hEX that behaves quite nicely. This has a static IP connection to the internet. LAN is 192.168.0.0/24 and it serves Wireguard as 10.23.1.0/24.
I have a cloud server running Ubuntu Linux - also has a static IP. This serves Wireguard as 10.23.2.0/24. This also connects to the office as 10.23.1.10.
Previously my hEX (actually, another MT device behind the hEX but ignore that for now) was the Wireguard server for my remote access. I have moved that role to my cloud server - it has a much faster internet connection. I retain a WG connection between my cloud server and my office hEX.
For...reasons...I need to be able to reach the internet (well...really only 1 external IP but the principle is the same) from my cloud server based clients via my office IP. So the path is <wg peer> --> <cloud server> --> <office hEX> --> internet
Straight wg connections work just fine. From within my office network I can reach the internet in general and the target IP in particular perfectly. From my cloud server I can reach the internet directly and my office network perfectly, also cloud wireguard peers work fine accessing the office - yet every attempt I've made to perform the specific routing fails. It looks from a traceroute I hit the hEX on the wg IP and it just disappears. I tried doing some packet sniffing and I *think* the issue is srcnat is not being applied - I just don't understand why.
Looking at other posts it seems I *may* need to start learning policy routing for this - I've experimented a bit and so far no luck.
My excerpted export below. I deleted most of what I thought was irrelevant. I left some of my filtering rules in place just in case they're either relevant or worthy of comment.
On the subject of understanding why - looking at my srcnat rules you'll see three rules commented "Wireguard to others". The multiple rules are my just throwing things at this to try to get it to work but...I see traffic on all three rules. The first rule references the interface directly - how do the other IP based rules ever get processed?
Code: Select all
# nov/24/2022 10:29:13 by RouterOS 7.5rc2
# software id = 6MPE-115F
#
# model = RB760iGS
/interface wireguard
add listen-port=<mywgport> mtu=1420 name=wg0
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/routing table
add disabled=yes fib name=wireguard
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1-Internet list=WAN
/interface wireguard peers
add allowed-address=10.23.1.10/32,10.23.2.0/24 \
endpoint-address=<my.linux.wg> endpoint-port=<mywgport> interface=wg0 \
persistent-keepalive=25s public-key="myverysuperspecialsecret"
/ip address
add address=<my.pub.ip>/30 comment="Public IP" interface=ether1-Internet \
network=<my.pub.net>
add address=192.168.0.1/24 comment="Primary LAN" interface=bridge network=\
192.168.0.0
add address=10.23.1.1/24 interface=wg0 network=10.23.1.0
/ip dns
set servers=192.168.0.2,1.1.1.1,8.8.8.8
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
/ip firewall filter
add action=drop chain=input comment="drop temporary IPSEC blacklist" \
src-address-list=ipsec-temp-blacklist
add action=accept chain=input in-interface=wg0
add action=accept chain=input comment=\
"accept established,related...and untracked" connection-state=\
established,related,untracked
add action=accept chain=input comment=\
"Allow IPSEC traffic (established via policy)" in-interface=\
ether1-Internet ipsec-policy=in,ipsec
add action=accept chain=input comment="Accept Wireguard" dst-port=<mywgport> \
in-interface=ether1-Internet protocol=udp
add action=accept chain=input comment="Accept new,untracked from Bridge" \
connection-state=new,untracked in-interface=bridge
add action=accept chain=input comment="Allow self-connections - for The Dude" \
dst-address-type=local src-address-type=local
add action=accept chain=input comment="Allow IPSEC connections" dst-port=\
500,4500 in-interface=ether1-Internet protocol=udp
add action=accept chain=input comment="Allow IPSEC connections" in-interface=\
ether1-Internet protocol=ipsec-esp
add action=accept chain=input comment=\
"Allow IPSEC connections - seems unnecessary." in-interface=\
ether1-Internet protocol=ipsec-esp
add action=accept chain=input comment="Accept SSH,Webcfg,Winbox from Bridge" \
dst-port=22,80,8291 in-interface=bridge protocol=tcp
add action=accept chain=input comment="Accept SSH,Webcfg,Winbox from IPSec" \
dst-port=22,80,8291 ipsec-policy=in,ipsec protocol=tcp
add action=accept chain=output comment="Section Break" disabled=yes
add action=drop chain=input comment="drop ssh brute forcers" \
src-address-list=ssh_blacklist
add action=drop chain=input comment="dropping port scanners" \
src-address-list="port scanners"
add action=drop chain=input comment="drop all from Internet via Fail2Ban" \
in-interface=ether1-Internet src-address-list=fail2ban
add action=drop chain=input comment=\
"dropping blacklist - Joshaven compiled Spamhaus, malc0de, dshield" \
src-address-list=blacklist
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=jump chain=input comment="Jump to RFC SSH Chain" connection-state=\
new dst-port=22 in-interface=ether1-Internet jump-target="RFC SSH Chain" \
protocol=tcp
add action=jump chain=input comment="Jump to RFC Port Scanners UDP Chain" \
in-interface=ether1-Internet jump-target="RFC Port Scanners UDP Chain" \
protocol=udp src-address-list=!non-public
add action=jump chain=input comment="Jump to RFC Port Scanners TCP Chain" \
in-interface=ether1-Internet jump-target="RFC Port Scanners TCP Chain" \
protocol=tcp src-address-list=!non-public
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
"defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="Block DHCP to/from Internet" \
in-interface=ether1-Internet port=67-68 protocol=udp
add action=accept chain=input comment=\
"Accept all Bridge input (safety to prevent being locked out)" \
in-interface=bridge
add action=drop chain=input comment="drop all from Internet" in-interface=\
ether1-Internet
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
disabled=yes in-interface-list=!LAN
add action=drop chain=input comment=\
"Final Rule - shouldn't be needed but JIC CYA..."
add action=accept chain=forward comment=\
"accept in ipsec policy - must be before fasttrack" ipsec-policy=in,ipsec
add action=accept chain=forward comment=\
"accept out ipsec policy - must be before fasttrack" ipsec-policy=\
out,ipsec
add action=accept chain=forward comment="Forward ProtectedExternal" \
src-address-list=ProtectedExternal
add action=accept chain=forward comment=\
"avoid fasttrack for OpenDrive (to use Simple Queue for rate limiting)" \
connection-state="" packet-mark=OpenDrive
add action=accept chain=forward comment="Forward from Wireguard" \
in-interface=wg0
add action=accept chain=forward comment="Forward to Wireguard" out-interface=\
wg0
add action=drop chain=forward comment="defconf: drop invalid" \
connection-state=invalid
add action=drop chain=forward comment="dropping port scanners" \
src-address-list="port scanners"
add action=drop chain=forward comment="drop all from Internet via Fail2Ban" \
in-interface=ether1-Internet src-address-list=fail2ban
add action=drop chain=forward comment=\
"dropping blacklist - Joshaven compiled Spamhaus, malc0de, dshield" \
src-address-list=blacklist
add action=fasttrack-connection chain=forward comment="fasttrack non-ipsec" \
connection-mark=!ipsec connection-state=established,related disabled=yes \
hw-offload=yes
add action=accept chain=forward comment=\
"Accept established,related - and untracked" connection-state=\
established,related,untracked
add action=jump chain=forward comment="Jump to RFC Port Scanners TCP Chain" \
jump-target="RFC Port Scanners TCP Chain" protocol=tcp src-address-list=\
!non-public
add action=jump chain=forward comment="Jump to Vulnerable TCP Chain" \
jump-target=vulnerabilities-tcp protocol=tcp src-address-list=!non-public
add action=jump chain=forward comment="Jump to RFC Port Scanners UDP Chain" \
jump-target="RFC Port Scanners TCP Chain" protocol=udp src-address-list=\
!non-public
add action=jump chain=forward comment="Jump to Vulnerable UDP Chain" \
jump-target=vulnerabilities-udp protocol=udp src-address-list=!non-public
add action=drop chain=forward comment=\
"defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
connection-state=new in-interface-list=WAN
add action=drop chain=forward comment="drop all from fake LAN on Internet" \
connection-nat-state=!srcnat connection-state="" in-interface=\
ether1-Internet src-address-list=non-public
add action=accept chain=forward comment="Forward LAN-to-LAN" \
dst-address-list=non-public in-interface=bridge out-interface=bridge
add action=accept chain=forward comment="Forward LAN-to-WAN" \
dst-address-list=!non-public in-interface=bridge out-interface=\
ether1-Internet
add action=accept chain=forward comment="Forward from WAN if DSTNATed" \
connection-nat-state=dstnat connection-state=new in-interface=\
ether1-Internet
add action=accept chain=forward comment="Forward ICMP from LAN" in-interface=\
bridge protocol=icmp
add action=accept chain=output comment="Section Break" disabled=yes
/ip firewall nat
add action=src-nat chain=srcnat comment="Wireguard to others" out-interface=\
wg0 to-addresses=10.23.1.1
add action=src-nat chain=srcnat comment="Wireguard to others" dst-address=\
192.168.0.0/24 src-address=10.23.1.0/24 to-addresses=192.168.0.1
add action=src-nat chain=srcnat comment="Wireguard to others" dst-address=\
!10.23.1.0/24 src-address=10.23.1.0/24 to-addresses=<my.pub.ip>
add action=src-nat chain=srcnat comment="Wireguard to others - never see packets here" dst-address=\
10.23.1.0/24 src-address=<my.pub.ip> to-addresses=10.23.1.1
add action=src-nat chain=srcnat comment="LAN to Wireguard" dst-address=\
10.23.1.0/24 src-address=192.168.0.0/24 to-addresses=192.168.0.1
add action=src-nat chain=srcnat comment=LAN out-interface=bridge \
to-addresses=192.168.0.1
add action=src-nat chain=srcnat comment=\
"Should be last srcnat rule. non-IPSec to Internet NAT to public IP" \
ipsec-policy=out,none out-interface=ether1-Internet to-addresses=\
<my.pub.ip>
add action=src-nat chain=srcnat comment="In theory - either this rule or the a\
bove rule aren't needed. Because we have an explicit \"srcnat-accept\" rul\
e above for the IPSec networks no \"IPSec policy\" matching should be requ\
ired. Simple srcnat to public IP for internet access." out-interface=\
ether1-Internet to-addresses=<my.pub.ip>
add action=masquerade chain=srcnat comment=\
"Should be no forking reason for this. But."
/ip route
add comment="Default internet" disabled=no dst-address=0.0.0.0/0 gateway=\
<my.pub.gw>
add comment="Cloud Wireguard network" disabled=no distance=1 \
dst-address=10.23.2.0/24 gateway=10.23.1.10 pref-src="" \
routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/routing rule
add action=lookup-only-in-table disabled=yes dst-address=<the.special.external.address> \
interface=wg0 table=wireguard