Hairpin NAT over 2 WireGuard connected routers

ilgmars · April 14, 2024, 9:10am

Hello all,

I am kind of a fresh Mikrotik user but with some grasp on networking things.
Google came up with “a lot” of solutions that seem not to work at all, so I am finally, after the 2-day fight, before moving my local client machine (192.168.88.50) to a separate subnet, turning to you guys.

Not sure what am I missing, but the network looks as follows:
Local router(7.14.2, 192.168.88.0/24) connected with WireGuard to a CHR (7.14.2), that is being used as an internet gateway for some of the machines that sit on 192.168.88.0/24.
Everything seems to be working as expected apart from hairpin-nat, when trying to reach locally hosted resources over public IP (Tried only via the CHR).

The way the connection is set up for those machines that chould have CHR address is via the routing tables and routing rules on local router:
 /routing> rule print
Flags: X - disabled, I - inactive 
 0 X src-address=192.168.88.123/32 action=lookup-only-in-table table=chr1 

 1   src-address=192.168.88.200/32 action=lookup-only-in-table table=chr1 

 2   src-address=192.168.88.202/32 action=lookup-only-in-table table=chr1 

 3   src-address=192.168.88.105/32 action=lookup-only-in-table table=chr1 
/routing> table/ print
Flags: D - dynamic; X - disabled, I - invalid; U - used 
 0 D   name="main" fib 

 1     name="chr1" fib 


Local Network NAT:
 > ip/firewall/nat print
Flags: X - disabled, I - invalid; D - dynamic 
 0    chain=srcnat action=masquerade src-address=192.168.88.0/24 dst-address=192.168.88.0/24 log=no log-prefix="" 

 1    ;;; defconf: masquerade
      chain=srcnat action=masquerade src-address=192.168.88.0/24 out-interface=ether1 log=no log-prefix="" 
      ipsec-policy=out,none 

 2 X  chain=srcnat action=masquerade src-address=192.168.88.0/24 dst-address=192.168.88.0/24 out-interface=bridge 
      log=no log-prefix="" 

 3 X  ;;; defconf: masquerade
      chain=srcnat action=masquerade out-interface=WG-majas log=no log-prefix="" ipsec-policy=out,none 

 4 X  chain=srcnat action=passthrough connection-mark=use-chr-1 routing-mark=chr1 out-interface=WG-majas log=no 
      log-prefix="" 

 5    chain=dstnat action=dst-nat to-addresses=192.168.88.103 protocol=tcp src-address-list=somelist
      in-interface=ether1 dst-port=111 log=no log-prefix="" 

 6    chain=dstnat action=dst-nat to-addresses=192.168.88.103 protocol=tcp src-address-list=somelist
      in-interface=ether1 dst-port=222 log=no log-prefix="" 

 7    chain=dstnat action=dst-nat to-addresses=192.168.88.105 protocol=udp in-interface=ether1 dst-port=1234 log=no 

CHR NAT:
/ip/firewall/nat> print
Flags: X - disabled, I - invalid; D - dynamic 
 0    chain=srcnat action=masquerade src-address=192.168.88.0/24 dst-address=192.168.88.0/24 log=no log-prefix="" 

 1    chain=srcnat action=masquerade out-interface=ether1 log=no log-prefix="" 

 2    chain=dstnat action=dst-nat to-addresses=192.168.88.105 protocol=udp dst-port=1234 log=yes log-prefix="" 

 3    chain=dstnat action=dst-nat to-addresses=192.168.88.202 protocol=udp dst-port=12345 log=no log-prefix=""

The masquerade rules for the local network, that are appearing in a lot of Google tutorials, seem not to be catching any traffic.
What could I be missing?

anav · April 14, 2024, 10:47am

First will need both the CHR config and your home config
/export file=anynameyouwish ( minus router serial number, any public WANIP information, keys etc.).

Second are you getting your external users (that need access to the server on your home LAN) to use the public IP of the CHR.

dyndnsURL:port# where the dyndns URL points to the CHR public IP?

Best if you probably provided a network diagram as well.

ilgmars · April 14, 2024, 1:39pm

Hi,
Thank you for your reply. Adding the details.

I think that this is the first schematic I have ever made
Hope that it is not too basic.

I am selectively getting them to use the CHR Public address by adding routing rules.
chrexport2.rsc (2.88 KB)
localrouterexport.rsc (10.8 KB)

anav · April 14, 2024, 2:16pm

Getting there…
So far I dont see any remote users requiring access, all the users are connected to the local router??
If that is the case then..

Assuming you have some servers on the LOCAL router?

a. why not access them by direct LANIP.
b. if needing to access them by DYDNS URL (representing the WANIP of the local router ) then will need hairpin nat associated rules but would be much easier to create a separate subnet for servers… or users…

There is no circumstance here requiring port forwarding on CHR that I can see.
In fact, the only need you have stated wrt wireguard is that some users, need to go out internet of CHR, vice local WANIP.
This is fine, but as stated would be easier, if that was a separate subnet of users but still could be handled if you have an address list of those needing CHR wan access etc… or those going out local only, whichever list is shorter LOL.

Will look at configs after clarification provided on requirements.

ilgmars · April 14, 2024, 4:35pm

Hi, Thank you for helping out.

So the big idea is to have the ability to switch them over when needed. Previously it was achieved by implementing part of this config and then just adding/removing addresses to/from the list:
https://www.youtube.com/watch?v=gXBXTWy62X8

So far I dont see any remote users requiring access, all the users are connected to the local router??

There are no remote users via the routers, just local.

Assuming you have some servers on the LOCAL router?

Everything is hosted locally and then some machines are using CHR for traffic

a. why not access them by direct LANIP.

Some things that I self-host break when accessed directly. It is possible by playing with DNS, but I would rather get the network sorted than host a DNS server.

b. if needing to access them by DYDNS URL (representing the WANIP of the local router ) then will need hairpin nat associated rules but would be much easier to create a separate subnet for servers… or users…

Yes, that is what I was thinking of as a last resort. Currently, I am trying to get the pinhole thing going, not sure why the standard pinhole NAT setup would not work at all.

There is no circumstance here requiring port forwarding on CHR that I can see.

CHR is receiving incoming traffic and then dst-natting them to machines that are hosted behind the home network. Essentially, from the outside, it looks like the service has the CHR address.

In short, yes, If all fails it is either another DNS server or just splitting the network, but that would make the network less flexible, as currently, I can do the switchover to a local connection by disabling a routing rule.
Before that, let’s see if there is some weird overlooked thing that would fix the pinhole NAT setup.

anav · April 14, 2024, 8:04pm

Before doing anything fancy with wireguard. Lets get port forward working properly.

The easiest method which should work is the direct LANIP address that the user uses to reach the server as in their browser or particular application.
This occurs at layer2 and is oblivious to firewall rules etc… just works.

Server-LANIP:Port#

Next is the hairpin nat problem, you want users to access the Server by DYNDNS URL that identifies the public IP of the router. It could be a paid or free one available on the internet, it could be using the IP Cloud on your router 'mynetname***.net"

In this case we want to ensure your firewall forward rule is good, and you have a hairpin source nat rule in place and you have the correct dst-nat rules in place.
The second half of that equation, depends upon the type of public IP you have. Is it a fixed static public IP, is it a dynamic IP?

Right now by the way I see a mess and NO port forwarding rules aka DST NAT rules???

anav · April 14, 2024, 8:56pm

Only changes noted:

Added wireguard to LAN interface list.
Note to remove static DNS default setting.
Remove limitation on ICMP, not required and can get in the way of testing etc.
Removed blocking dst nat rules on input chain ( also were not in the right sequence anyway ) useless since you are already dropping everything not from LAN!
Better to allow everything from LAN, then drop everything else. Cleaner and better.
Firewall rules fixed up.
Removed mangling not required anytime soon.
Source nat rules simplified.
7 DID NOT ADD DST NAT RULES AS YOU HAVE TO DO THAT PART and cannot be completed until we know more about your WAN side.
DID NOT ADJUST ROUTES YET as its not clear what type of ISP connection you have and what are 10.100.100 addresses ???
Routing rules can be left for now, but may not be required.
Why is winbox address not even an existing local subnet ???
Disable IPV6 if not required and remove all iPV6 firewall settings etc…

/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=Wireguard-B-sanitized list=LAN
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan  { REMOVE this static DNS entry }
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input comment="allow only LAN traffic" in-interface-list=LAN
add action=drop chain=input comment="Drop all else"
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related disabled=yes hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=accept chain=forward comment="internet traffic"  in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment="port forwarding"  connection-nat-state=dstnat 
add action=drop chain=forward comment="Drop all else"
/ip firewall nat
add action=masquerade chain=srcnat comment=hairpin  dst-address=192.168.88.0/24 src-address=192.168.88.0/24
add action=masquerade chain=srcnat comment="defconf: masquerade"  out-interface-list=WAN
/ip service
set winbox address=\   ???????????????????????????
    192.168.87.0/24

ilgmars · April 15, 2024, 4:47pm

All good points, thank you. Some of the configs got mangled when sanitizing

So I got this fixed by monkeying around. Not sure about the exact mechanics for it, but the local network masquerade started to work after I added these 2 rules, address list entry and a route:

 > ip firewall/mangle/ print
Flags: X - disabled, I - invalid; D - dynamic 
 0  D ;;; special dummy rule to show fasttrack counters
      chain=prerouting action=passthrough 

 1  D ;;; special dummy rule to show fasttrack counters
      chain=forward action=passthrough 

 2  D ;;; special dummy rule to show fasttrack counters
      chain=postrouting action=passthrough 

 3    chain=prerouting action=mark-connection new-connection-mark=use-chr-1 passthrough=yes src-address-list=CHR-WG log=no log-prefix="" 

 4    chain=prerouting action=mark-routing new-routing-mark=chr1 passthrough=yes src-address-list=CHR-WG connection-mark=use-chr-1 log=no log-prefix="" 

----------------------
17 CHR-WG      192.168.88.200
----------------------
  DAc 10.20.20.1/32    WG-majas with routing table chr1
----------------------
> ip firewall/nat print
Flags: X - disabled, I - invalid; D - dynamic 
 0    chain=srcnat action=masquerade src-address=192.168.88.0/24 dst-address=192.168.88.0/24 log=yes log-prefix="" 

 1    ;;; defconf: masquerade
      chain=srcnat action=masquerade src-address=192.168.88.0/24 out-interface=ether1 log=no log-prefix="" ipsec-policy=out,none

It is as messy as it gets, but if someone stumbles upon this, witht he same issue, maybe it will help.

anav · April 15, 2024, 5:07pm

Post your complete config minus the usual, please, the snippets you show are not useful without context of the rest of the config.
I also have my doubts as to complete success…