Brain fade: routing to internally hosted servers doesn't work

Complete brain fade here. What’s the term/phrase used when routing to internal servers doesn’t work? I’ve got an instance of SimpleHelp hosted on my own server. It’s accessed via help.maltsystems.co.uk. By default, this resolves to the public IP address. Works flawlessly externally but any computers on the LAN can’t route correctly because the router gets confused over routing out and the back. This “thing” has a name. Just can’t recall what it’s called!

I’ve implemented the workaround on the Windows DNS servers on my domain by adding a new zone for maltsystems.co.uk and manually adding the internal IP addresses so that help.maltsystems.co.uk resolves to the internal IP address.

Hairpin nat.

It’s not the router that’s confused in these instances, but the client btw.

Hairpin NAT - that’s it! I recall there is an in-depth article on this subject. What perplexed me is that before I installed the Mikrotik router, I used the Virgin Media Superhub and I didn’t have the same issue with that. I need to do some more reading.

A lot of consumer routers include a blanket masquerade rule of the form

/ip firewall nat add chain=srcnat action=masquerade

This is of course bad: e.g. even external dst-nat connections appear to come from the router instead of the proper external IP. It makes things a bit easier to configure, and they don’t care.

A more proper rule looks like this:

/ip firewall nat 
add chain=srcnat action=masquerade in-interface=srv-bridge dst-address=se.rv.er.ip protocol=tcp dst-port=80,443

Dst-addr/proto/port should match your dst-nat to-address/to-port. In ye olden times the in-interface matcher was unavailable in the srcnat chain, on these versions src-address=su.bn.et.0/24 should be used instead.

2 Likes

I usually setup my local network to use the router ip for DNS. Then I can manage things like this using static entries or forwarders so the internal machines use the internal address instead of the public.

These days the only time I use hairpin is for special cases where someone internally might be using a VPN and needs to go to another DNS, but still needs internal access to an internal server, usually email or web.

Those use cases have to be considered individually to determine if it’s a security concern or not to do that.

You bring up some interesting points that merit consideration.

Redirecting DNS is an okay solution. The first problem with it is that it’s increasingly difficult to do: due to constant ill-intentioned(1) meddling with DNS, many systems and programs default to not using the resolver specified by DHCP or even use DNS-over-HTTPS by default. In many scenarios this can be mitigated, but in many it cannot. The other problem is that the TTL of the public DNS entry must be sufficiently short to not interfere with this scheme via caching.

As to security concerns. Basically I only suggest hairpin for scenarios where the entire world was already given access to a given resource. This resource either contains information that is sufficiently public as to enable this, or has some sort of adequate authentication as to enable making the gateway open. The only associated problem is that the requester’s IP remains cloaked - but only in the sense that we don1t know which host on the subnet made the request, which would also be unavailable for anyone behind NAT, so everyone else as well.

Anyway, I am of the opinion that hosted services should generally reside in a non-user (non-client, whatever) subnet, where this becomes moot.

(1) By ill-intentioned meddling I mean things like returning the addresses of ad-laden landing pages instead of NXDOMAIN (which breaks a lot of things), or hijacking the DNS of ad networks, again to insert the provider’s ads into other people’s pages. Both were done primarily in the US by cable-turned-ISP monopolies. The default usage of external DNS was mostly a reaction to such breakage.

I like that option as well when available.

I’ll have a look at that option as well. Shouldn’t be too difficult to implement. The additional zone on the DNS servers works fine most of the time but I’m currently building a PC image for a client and it works 90% of the time but sometimes doesn’t get the internal IP but the external. That’s probably a problem with my DNS set-up, it’s mainly a lab environment.

I will read up on hairpin as I do generally like to understand that’s going on.

Bull pucky Lurker, , your sourcenat rule is OVERCOOKED.
It should be kept a simple as possible, its NOT meant to take over responsibilities normally handled by other tools such as IP route or Firewall Rules, there are exceptions of course.

In addition, clearly servers being accessed from external are best handled in a separate vlan from local users, for serveral reasons.
Removes hairpin nat issues as this only occurs if user are in the same subnet
Is generally a more secure approach.

You’re too kind. :slight_smile:

I know of many more convoluted ways, but none that are simpler. Maybe you can give an example?

Keep in mind that a host in a given subnet may attempt to send a packet to another client in the same subnet via the gateway. In this case the gateway router should do two things:

  • forward that packet with unchanged addressing
  • issue an ICMP redirect, notifying the sender of a more direct path

As to the best practice being to put servers in another subnet: I fully agree (as I stated).

The subject is hairpin nat, not how uneccessarily convoluted one can make the sourcenat settings.

The normal sourcenat ( masquerade or not ) is NOT the issue.
To address hairpin shenanigans there are several viable methods.
One is a simple hairpin sourcenat rule
add chain=srcnat action=masquerade src-address=serversubnet dst-address=serversubnet

DONE!

The DNS solution as indicated is probably the better choice, if one cannot move the server to its own subnet…

Concur, port forwarding should be avoided if possible, use wireguard instead to reach a device internally.

Just saying “done” doesn’t make it so. Let me illustrate the problem. With your rule installed:

Assume the following: 192.168.80.1 is the router, 192.168.80.3 and 4 are both hosts on this /24 subnet. Watch what happens is 192.168.80.3 pings 192.168.80.4 via 192.168.80.1:

Transmitted packets on .3:

#  NUM  TIME   SRC-MAC            DST-MAC            SRC-ADDRESS   DST-ADDRESS   IP-PROTOCOL
0    1  2.873  4A:8F:5A:84:1F:98  DE:2C:6E:48:95:17  192.168.80.3  192.168.80.4  icmp       
1    2  3.876  4A:8F:5A:84:1F:98  DE:2C:6E:48:95:17  192.168.80.3  192.168.80.4  icmp       
2    3  4.87   4A:8F:5A:84:1F:98  DE:2C:6E:48:95:17  192.168.80.3  192.168.80.4  icmp

Received packets on .4:

#  NUM  TIME    SRC-MAC            DST-MAC            SRC-ADDRESS   DST-ADDRESS   IP-PROTOCOL
0    1  10.771  DE:2C:6E:48:95:17  4A:8F:5A:84:20:64  192.168.80.1  192.168.80.4  icmp       
1    2  11.774  DE:2C:6E:48:95:17  4A:8F:5A:84:20:64  192.168.80.1  192.168.80.4  icmp       
2    3  12.768  DE:2C:6E:48:95:17  4A:8F:5A:84:20:64  192.168.80.1  192.168.80.4  icmp

As you can see, the exact opposite of what I said should happen is observed:

  • addresses are changed
  • an ICMP redirect is not emitted

With my overly convoluted rule, this behavior does not occur.