Help to get Hairpin NAT working please?

a752412341 · Sat Feb 14, 2015 8:15 pm

Hi all,

My LAN is on 192.168.27.0/24 on Ether1 There is a server at 192.168.27.4 that I want to access from LAN, and outside on a random port 21392.

I have 2 ADSL lines which are bonded togeter using PPoE on Ether2 and Ether3

I have put int the src-nat entry as per the wiki but can't get it to work. Any pointers greatly appreciated?

Thanks!

Here are the NAT Rules

[admin@MikroTik] /ip firewall nat> print
Flags: X - disabled, I - invalid, D - dynamic
 0    ;;; CAM incoming
      chain=dstnat action=dst-nat to-addresses=192.168.27.4 to-ports=21392
      protocol=tcp in-interface=all-ppp dst-port=21392 log=no log-prefix=""

 1    ;;; HS3 incoming
      chain=dstnat action=dst-nat to-addresses=192.168.27.8 to-ports=44443
      protocol=tcp in-interface=all-ppp dst-port=44443 log=yes log-prefix=""

 2    ;;; HS3 HSTOUCH incoming
      chain=dstnat action=dst-nat to-addresses=192.168.27.8 to-ports=44444
      protocol=tcp in-interface=all-ppp dst-port=44444 log=yes log-prefix=""

 3    ;;; Hairpin NAT - CAM
      chain=srcnat action=masquerade src-address=192.168.27.0/24
      dst-address=192.168.27.4 out-interface=ether1-LAN log=no log-prefix=""

 4    ;;; Outbound Internet Access
      chain=srcnat action=masquerade out-interface=all-ppp log=no
      log-prefix=""
[admin@MikroTik] /ip firewall nat>

Here are Firewall filter rules

[admin@MikroTik] /ip firewall filter> print
Flags: X - disabled, I - invalid, D - dynamic
 0    ;;; LAN traffic can go anywhere
      chain=forward action=accept in-interface=ether1-LAN log=no
      log-prefix=""

 1    ;;; Established traffic
      chain=forward action=accept connection-state=established log=no
      log-prefix=""

 2    ;;; Related traffic
      chain=forward action=accept connection-state=related log=no
      log-prefix=""

 3    ;;; ICMP
      chain=forward action=accept protocol=icmp log=no log-prefix=""

 4    ;;; cam allow incoming traffic
      chain=forward action=accept protocol=tcp dst-address=192.168.27.4
      dst-port=21392 log=yes log-prefix=""

 5    ;;; HS3 allow incoming traffic
      chain=forward action=accept protocol=tcp dst-address=192.168.27.8
      dst-port=44443 log=yes log-prefix=""

 6    ;;; HS3 HSTOUCH allow incoming traffic
      chain=forward action=accept protocol=tcp dst-address=192.168.27.8
      dst-port=44444 log=yes log-prefix=""

 7    ;;; Drop the rest
      chain=forward action=drop log=no log-prefix=""

 8    chain=output action=accept log=no log-prefix=""

 9    ;;; LAN traffic can go anywhere
      chain=input action=accept in-interface=ether1-LAN log=no log-prefix=""

10    ;;; Established traffic
      chain=input action=accept connection-state=established log=no
      log-prefix=""

11    ;;; Related traffic
      chain=input action=accept connection-state=related log=no log-prefix=""

12    ;;; ICMP
      chain=input action=accept protocol=icmp limit=5,5 log=no log-prefix=""

13    ;;; Drop the rest
      chain=input action=drop log=no log-prefix=""

[admin@MikroTik] /ip firewall filter>

a752412341 · Tue Feb 17, 2015 2:22 am

Anyone got any pointers? Cheers

DLNoah · Tue Feb 17, 2015 10:42 pm

The src-nat rule isn't matching anything because the original dst-nat rules aren't matching packets that came from your LAN (only matching in from the ppp interfaces).

The normal setup for using Hairpin NAT would be similar to the following:

- Public IP of 1.2.3.4, clients requesting on port 21392
- Private IP of 192.168.27.4, server listening on port 21392

/ip firewall nat
add chain=dstnat action=dst-nat to-addresses=192.168.27.4 protocol=tcp dst-address=1.2.3.4 dst-port=21392 log=no
add chain=srcnat action=masquerade src-address=192.168.27.0/24 dst-address=192.168.27.4 log=no

For your dst-nat rule, you don't want to specify the in-interface -- you want to dst-nat all traffic that requests the NAT'd public IP, whether it came from inside your network or from outside. You don't need to specify the to-ports, because you're not changing the original port (if no to-port is specified, the port number will remain unchanged).

On the hairpin rule, you don't really need to specify the in-interface (the src-address is sufficient to create a unique match for traffic from the LAN to the after-NAT internal IP of the server)

a752412341 · Wed Feb 18, 2015 12:24 am

Hi DLNoah, thank you for the reply.

I have made the changes you suggest, and now I sometimes get a "connected" from the browser but it then hangs, and doesn't return the web page. What am I doing wrong?

[admin@MikroTik] /ip firewall nat> print
Flags: X - disabled, I - invalid, D - dynamic
 0    ;;; CAM incoming
      chain=dstnat action=dst-nat to-addresses=192.168.27.4 protocol=tcp
      dst-address=[PUBLIC IP] dst-port=21392 log=no log-prefix=""

 1    ;;; CAM hairpin NAT
      chain=srcnat action=masquerade src-address=192.168.27.0/24
      dst-address=192.168.27.4 log=no log-prefix=""

 2    ;;; HS3 incoming
      chain=dstnat action=dst-nat to-addresses=192.168.27.8 protocol=tcp
      dst-address=[PUBLIC IP] dst-port=44443 log=no log-prefix=""

 3    ;;; HS3 hairpin NAT
      chain=srcnat action=masquerade src-address=192.168.27.0/24
      dst-address=192.168.27.8 log=no log-prefix=""

 4    ;;; HS3 HSTOUCH incoming
      chain=dstnat action=dst-nat to-addresses=192.168.27.8 protocol=tcp
      dst-address=[PUBLIC IP] dst-port=44444 log=yes log-prefix=""

 5    ;;; Outbound Internet Access
      chain=srcnat action=masquerade out-interface=all-ppp log=no
      log-prefix=""

[admin@MikroTik] /ip firewall nat>

here are the filter rules

[admin@MikroTik] /ip firewall filter> print
Flags: X - disabled, I - invalid, D - dynamic
 0    ;;; LAN traffic can go anywhere
      chain=forward action=accept in-interface=ether1-LAN log=no log-prefix=""

 1    ;;; Established traffic
      chain=forward action=accept connection-state=established log=no log-prefix=""

 2    ;;; Related traffic
      chain=forward action=accept connection-state=related log=no log-prefix=""

 3    ;;; ICMP
      chain=forward action=accept protocol=icmp log=no log-prefix=""

 4    ;;; cam allow incoming traffic
      chain=forward action=accept protocol=tcp dst-address=192.168.27.4 dst-port=21392 log=yes log-prefix=""

 5    ;;; HS3 allow incoming traffic
      chain=forward action=accept protocol=tcp dst-address=192.168.27.8 dst-port=44443 log=yes log-prefix=""

 6    ;;; HS3 HSTOUCH allow incoming traffic
      chain=forward action=accept protocol=tcp dst-address=192.168.27.8 dst-port=44444 log=yes log-prefix=""

 7    ;;; Drop the rest
      chain=forward action=drop log=no log-prefix=""

 8    chain=output action=accept log=no log-prefix=""

 9    ;;; LAN traffic can go anywhere
      chain=input action=accept in-interface=ether1-LAN log=no log-prefix=""

10    ;;; Established traffic
      chain=input action=accept connection-state=established log=no log-prefix=""

11    ;;; Related traffic
      chain=input action=accept connection-state=related log=no log-prefix=""

12    ;;; ICMP
      chain=input action=accept protocol=icmp limit=5,5 log=no log-prefix=""

13    ;;; Drop the rest
      chain=input action=drop log=no log-prefix=""
[admin@MikroTik] /ip firewall filter>

DLNoah · Wed Feb 18, 2015 1:46 am

Uhm, the most obvious thing I can think of would be that there's more than one stream as part of the connection. But if that were the case, I would expect external users to have problems to, and I'm reading that you're not having that problem?

What does Torch show when you're trying to make the connection from an internal user?

a752412341 · Wed Feb 18, 2015 2:02 am

Hi, it works fine externally.

Here is the torch screenshot after putting http://domain name.com:21392 into a browser which resolves to my external ip. I'm not sure what this is telling me.

ScreenClip.png

DLNoah · Wed Feb 18, 2015 3:33 am

It looks like the NAT connection isn't getting created correctly in the firewall. Can you try rebooting the firewall or going to IP > Firewall > Connections and deleting all entries? If you had connections open in conntrack from before you changed the rules, they don't get automatically flushed on rule change.

a752412341 · Wed Feb 18, 2015 12:06 pm

I tried rebooting, but still the same issue.

For completeness here is the routes screenshot.

ScreenClip.png

Wed Feb 18, 2015 12:15 pm

Seems to me that situation must be fixed if you change this rule:
1 ;;; CAM hairpin NAT
chain=srcnat action=masquerade src-address=192.168.27.0/24
dst-address=192.168.27.4 log=no log-prefix=""

Try to run:
"/ip firewall nat set 1 out-interface=ether1-LAN"

a752412341 · Wed Feb 18, 2015 12:24 pm

Hi Strods, thank you for the comment.

I just tried that, but no change.

There are some packets getting through somehow, because firefox says connected, and eventually says transferring data...

Wed Feb 18, 2015 1:24 pm

Try this - it works for my local Lotus server which could be accessed from LAN via external public IP.

add action=masquerade chain=srcnat comment="HarpIn Lotus" dst-address=192.168.1.0/24 src-address=192.168.1.0/24
add action=dst-nat chain=dstnat comment="Lotus SATDAT step 1" dst-address-list=ETH_WAN_GROUP dst-port=1352 protocol=tcp to-addresses=LOTUS_LOCAL_IP
add action=src-nat chain=srcnat comment="Lotus SATDAT step 2" dst-address=LOTUS_LOCAL_IP dst-port=1352 protocol=tcp to-addresses=THIS_ROUTER_IP

When the packet from your computer goes to the server via it's external address then router changes dst address to local server's ip and sends the packet to server's local ip.
Then server, which sees the packet coming from local address responds to this packet with own local ip and sends directly to your computer as it knows that src-address is from LAN so why it should be send back to router -it is shortening the path and it is IMHO the problem you are facing.
Your computer sees the packet coming from server with local ip but the computer does not know nothing about this incoming connection from server as it expects to match packet coming from external ip. Packet is dropped.

The crucial part of these three lines above is line #1. It is making masquarade for packets coming from LAN to LAN when the packet is and substitutes src address of this packet (your computer) with router LAN IP. With this masquarade your local server responds to router as it sees the masquaraded original packet with router src-address substituted, router unmasquarades this packet and sends it back to your computer. There is no "short path" which fools your computer.

I could be wrong but it works for me

DLNoah · Wed Feb 18, 2015 3:39 pm

I see from your route table you're using routing marks. Can you post your mangle rules that are applying the routing marks? I think that may be causing the problem; if the initial connection is hitting that routing mark, it isn't going to run the directly connected route to 192.168.27.0/24 in the main routing table, and that would then cause the "two connections" that are showing in torch.

a752412341 · Wed Feb 18, 2015 4:26 pm

Hi DLNoah, here you go.

thanks, Al

[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
 0    chain=prerouting action=mark-packet new-packet-mark=one passthrough=yes
      dst-address=0.0.0.0/0 in-interface=ether1-LAN nth=2,1 log=no
      log-prefix=""

 1    chain=prerouting action=mark-packet new-packet-mark=two passthrough=yes
      dst-address=0.0.0.0/0 in-interface=ether1-LAN nth=2,2 log=no
      log-prefix=""

 2    chain=prerouting action=mark-routing new-routing-mark=one passthrough=no
      packet-mark=one log=no log-prefix=""

 3    chain=prerouting action=mark-routing new-routing-mark=two passthrough=no
      packet-mark=two log=no log-prefix=""
[admin@MikroTik] /ip firewall mangle>

DLNoah · Wed Feb 18, 2015 4:37 pm

Ok, those are definitely effecting this traffic.

Can you try replacing dst-address=0.0.0.0/0 with dst-address=![PUBLIC IP]

Again, after you change the firewall rule, you will need to delete all connections (or restart the router) before the changes will take full effect.

a752412341 · Wed Feb 18, 2015 6:02 pm

I tried that but didn't make any difference.

[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
 0    chain=prerouting action=mark-packet new-packet-mark=one passthrough=yes
      dst-address=![public ip] in-interface=ether1-LAN nth=2,1 log=no
      log-prefix=""

 1    chain=prerouting action=mark-packet new-packet-mark=two passthrough=yes
      dst-address=![public ip] in-interface=ether1-LAN nth=2,2 log=no
      log-prefix=""

 2    chain=prerouting action=mark-routing new-routing-mark=one passthrough=no
      packet-mark=one log=no log-prefix=""

 3    chain=prerouting action=mark-routing new-routing-mark=two passthrough=no
      packet-mark=two log=no log-prefix=""
[admin@MikroTik] /ip firewall mangle>

DLNoah · Wed Feb 18, 2015 7:09 pm

Can you post another Torch, and a filtered "IP > Firewall > Connections" list, please?

a752412341 · Wed Feb 18, 2015 8:07 pm

Here's another screenshot of Torch and Connections while opening the url in the browser. There's also an external connection happening which I was testing.

ScreenClip3.png

ScreenClip2.png

DLNoah · Wed Feb 18, 2015 8:22 pm

Hm, I'm not sure what else to look at. The rules you have in place work for me on RouterOS v6.19, for similar functionality I have up and going.

When you double-click one of the IP > Firewall > Connections from your internal connection attempt, what does it show for "Reply Src Address" and "Reply Dst Address"?

a752412341 · Wed Feb 18, 2015 10:16 pm

I'm on Firmware version 3.17 - it is a Routerboard 951G-2HnD

The odd thing is that it is almost working - I get half a web page, very slowly.

The packet marking rules are to split the upload between the 2 links. My ISP splits the download packets between the 2 links on their end.

It seems like perhaps half the packets are getting through one way or another?

Here is the screenshot of one of the connections.

ScreenClip4.png

DLNoah · Wed Feb 18, 2015 10:41 pm

I assume by "firmware version" you mean the output of System > RouterBOARD. What version shows in the title bar when you log in: "user@ip (identity) - Winbox vX.XX ..."?

The connection you're showing is what I would expect to see with the NAT rules in place.

My expectation when you posted the mangle rules was that half the time (at random) it would work, since half of your packets were getting marked with routing mark "two", which had a special routing rule active, and the other half were getting marked with routing mark "one", which didn't have any active special routing rules.

I can think of two possible troubleshooting steps from here:

1) Use Tools > Packet Sniffer in order to record the traffic while you try to make a connection from an internal computer. After you record the data with Packet Sniffer, you can download the resulting file to your computer and open it with Wireshark/tcpdump or a similar program. This will let you verify the actual responses coming back from the server -- maybe you're getting an ICMP reject because of a firewall or other setting on the server?

2) Make one more set of changes to the mangle rules, to make it so IP > Firewall > Connections should potentially have useful information to show.
Change these:

 0    chain=prerouting action=mark-packet new-packet-mark=one passthrough=yes
      dst-address=![public ip] in-interface=ether1-LAN nth=2,1 log=no
      log-prefix=""

 1    chain=prerouting action=mark-packet new-packet-mark=two passthrough=yes
      dst-address=![public ip] in-interface=ether1-LAN nth=2,2 log=no
      log-prefix=""

Change the action from "mark-packet" to "mark-connection", remove "new-packet-mark" and insert "new-connection-mark=one" (or two) -- no other changes, just switch from a packet mark to a connection mark. In general, for PCC load balancing and bonding, it's recommended to mark connections instead of packets, so that you can apply special handling to both directions of the traffic. In your case, the connection-mark isn't strictly necessary, because you only care about the round-robin handling on the outbound leg -- responses don't have to travel out the exact same interface they came in.

Along with that change, you would also need to change:

 2    chain=prerouting action=mark-routing new-routing-mark=one passthrough=no
      packet-mark=one log=no log-prefix=""

 3    chain=prerouting action=mark-routing new-routing-mark=two passthrough=no
      packet-mark=two log=no log-prefix=""

Remove the packet-mark match, and add a connection-mark match with the same match (so match connection-mark=one or two, depending on which rule).

The advantage we get in this particular case is that IP > Firewall > Connections will show us which connection mark is applied (if any) to a connection, but cannot show us packet or routing marks.

----

Personally, I would probably try the Packet Sniffer step first, because it should be a lot shorter and less disruptive to do.

a752412341 · Fri Feb 20, 2015 3:44 pm

DLNoah, thank you for sticking with this!

I've enabled the correct routes dealing with the marked packets, they were incorrect in a previous screenshot. Here are also the screenshots of the sniffer test.

ScreenClip3.png

ScreenClip5.png

ScreenClip.png

DLNoah · Fri Feb 20, 2015 4:41 pm

Given that Sniffer is showing the packets coming in the PPPoE interfaces, I believe that something must be causing the "to be hairpinned" traffic to get packet-marked and forced out the bonded-DSL routing rules.

Could you get by with disabling all four mangle rules and the two "routing mark" routes for a few minutes, and then run another test to see if the hairpin NAT rule is behaving as expected without the bonded-DSL routing rules?

a752412341 · Fri Feb 20, 2015 5:23 pm

Yes, with the mangle rules disabled, and the 2 routes dealing with the marked packets, the hairpin works, and I can access the webserver using the external ip from the LAN.

The (disabled) mangle rules currently have ![public ip] as the Dst. Address.

DLNoah · Fri Feb 20, 2015 5:29 pm

Ok, so let's try changing the mangle rules as follows:

Add two new mangle rules at the first priority (priority #0):
/ip firewall mangle add chain=prerouting action=accept passthrough=no dst-address=192.168.27.0/24 in-interface=ether1-LAN log=no log-prefix="" place-before=0
/ip firewall mangle add chain=prerouting action=accept passthrough=no dst-address=[PUBLIC IP] in-interface=ether1-LAN log=no log-prefix="" place-before=0

Change the two mark-packet rules back to their previous dst-address=0.0.0.0/0, and re-enable your mangle rules & routes.

What this should accomplish is to catch the traffic headed for the Public IP of the web service and post-NAT traffic heading back into your network (though I would expect that only the Public IP traffic should match, since mangle > prerouting runs before nat > dst-nat), and "accept" it out of the mangle chains -- basically, match the packet and prevent it from running any of the remaining rules. You should only see the counters for this rule increment when you make a request for the web service -- regular outbound traffic should not be caught by this rule. I'd urge you to pay close attention after your initial setup (and/or even add the new accept rule with log=yes initially, to make sure it's only matching what you want it to match).

a752412341 · Fri Feb 20, 2015 5:57 pm

Ok, Hairpin is now working.

I added a log onto the 1st rule you suggested as the counters were continously increasing even when not accessing the external IP.

Here is a screenshot. Is there anything to worry about here?

ScreenClip.png

DLNoah · Fri Feb 20, 2015 7:15 pm

Looks like all of that traffic is destined for your local router (DNS traffic on UDP 53, Winbox traffic on TCP 8291, regular Windows/NETBIOS broadcasts on 137-138), but should be fine.

You might also try disabling the "accept" for destination 192.168.27.0/24 and seeing if the hairpin still works without it, but again, I don't see anything that should break normal network operation within that log screenshot.

a752412341 · Fri Feb 20, 2015 10:38 pm

Great, thanks again DLNoah.

It stopped working if I disabled the accept rule.

a752412341 · Sun Feb 22, 2015 7:16 pm

All working, thanks for the help

Help to get Hairpin NAT working please?

Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

Re: Help to get Hairpin NAT working please?

[SOLVED] Re: Help to get Hairpin NAT working please with 2 bonded ADSL connections

Who is online