[SOLVED] General NAT access between local networks using multiple public WAN address?

Larsa · Fri Jan 27, 2017 4:57 pm

NAT between local networks.png

Problem:
At the moment we can’t use the Guest Network (1) to reach the services at the Office Network (3) through NAT. The reason is that the local networks are totaly isolated (internal routing disabled) to prevent possible leaks thus hairpin-nat is not possible.

Workaround:
We’ve created a temporary solution for one of the services by allowing internal routing for one specific port plus added a local dns entry for the guest network. But this is not feasible for a long term solution as there are currently more than 30 service ports and more will be added soon.

The only solution I've found so far is to direct all guest traffic to a separat interface connected to a WAN-switch thus forcing src-nat before entering the Office Network. Could it possibly be solved in some way by using for example an internal bridge or a VRRP-interface that will process all steps between from pre-routing to post-routing for the guest subnet before handing over the package for futher proccessing to the office subnet?

Question:
Should it be possible to accomplish a general NAT access from the Guest Network (1) to all the services at the Office Network (3) through a logical “loop-back” using the public ip-address assigned to the WAN (2) interface without any special measures or does it require special configurations using mangle and pre/post routing rules?

The objective and reason I want a general access to all services from both Internet (2) and the Guest Network (1) in order to maintain just a single set of dst-nat rules on the Office Network (3) thereby avoiding local routing and duplicate hair-pin roles for each service. There are currently more than 30 service ports and more will be added soon.

I’ve tried to follow the flow diagrams (http://wiki.mikrotik.com/wiki/Manual:Packet_Flow) but I don't seem to spot where the problem acctually might occur. Not sure if the in-interface-list in the dst-nat rule that might be the culprit? If so how should it be replaced as best practice (dst-address-type=local)?

Current configuration (also see "export" down below)
RouterOS 6.37.3

Public ip-address 1.1.1.1 and 1.1.1.2 assigned to Eth1 (WAN)
Local networks are connected to Eth7 with the Guest Network isolated using VLAN 172

Guest Network 172.16.0.0 is src-nat’ed to public ip-address 1.1.1.2 on Eth1 (WAN)
Office Network 192.168.0.0 is src-nat’ed to public ip-address 1.1.1.1 on Eth1 (WAN)
All services are dst-nat’ed from 1.1.1.1 to Office Network 192.168.0.0

# SRC-NAT
/ip firewall nat
add action=src-nat chain=srcnat comment="src-nat Office Network" \
    src-address=192.168.0.0/24 to-addresses=1.1.1.1
add action=src-nat chain=srcnat comment="src-nat Guest Network" \
    src-address=172.16.0.0/24 to-addresses=1.1.1.2
	
# DST-NAT. There are currently more than 30 service ports and more are coming... 
add action=dst-nat chain=dstnat comment="SMTP" dst-address=\
    1.1.1.1 dst-port=25 in-interface-list=WAN log=yes log-prefix=SMTP \
    protocol=tcp to-addresses=192.168.0.25 to-ports=25
    
#
# VLAN
#
/interface vlan
add interface=ether7 name=#7-VLAN-172 vlan-id=172

#
# Interfaces lists
#
/interface list
add name=LAN
add name=WAN
add name=GUEST

add interface=ether6 list=LAN
add interface=#7-VLAN-172 list=GUEST
add interface=ether1 list=WAN
add interface=ether2 list=WAN

#
# IP address
#
/ip address
add address=192.168.0.254/24 comment="Office Network" interface=ether7 
add address=172.16.0.254/24 comment="Guest network" interface=#7-VLAN-172
add address=1.1.1.1 comment="WAN Eth1 (Office Network)" interface=ether1
add address=1.1.1.2 comment="WAN Eth1 (Guest Network)" interface=ether1

#
# Address lists
#
/ip firewall address-list
add address=0.0.0.0/8 comment=RFC6890 list=PrivateNetworks
add address=10.0.0.0/8 comment=RFC6890 list=PrivateNetworks
add address=100.64.0.0/10 comment=RFC6890 list=PrivateNetworks
add address=127.0.0.0/8 comment=RFC6890 list=PrivateNetworks
add address=169.254.0.0/16 comment=RFC6890 list=PrivateNetworks
add address=172.16.0.0/12 comment=RFC6890 list=PrivateNetworks
add address=192.0.0.0/24 comment=RFC6890 list=PrivateNetworks
add address=192.0.2.0/24 comment=RFC6890 list=PrivateNetworks
add address=192.168.0.0/16 comment=RFC6890 list=PrivateNetworks
add address=192.88.99.0/24 comment=RFC3068 list=PrivateNetworks
add address=198.18.0.0/15 comment=RFC6890 list=PrivateNetworks
add address=198.51.100.0/24 comment=RFC6890 list=PrivateNetworks
add address=203.0.113.0/24 comment=RFC6890 list=PrivateNetworks
add address=224.0.0.0/4 comment=RFC4601 list=PrivateNetworks
add address=240.0.0.0/4 comment=RFC6890 list=PrivateNetworks

add address=192.168.0.0/24 comment="Office Network" list=IsolatedLANs
add address=172.16.0.0/24 comment="Guest Network" list=IsolatedLANs

#
# Firewall filters
#
/ip firewall filter
add chain=forward comment="Accept established and related packets" \
    connection-state=established,related
add chain=input comment="Allow Established connections" connection-state=\
    established
add action=accept chain=forward comment=\
    "Forward packets belonging to natted connections" connection-nat-state=\
    dstnat connection-state=established,related in-interface-list=WAN
add action=accept chain=input comment="Allow ICMP" in-interface-list=all \
    protocol=icmp
add action=accept chain=input comment="Accept all input from Office Network" \
    in-interface-list=LAN src-address=192.168.0.0/24
add action=accept chain=input comment="Accept all input from GUEST Network" \
    in-interface-list=GUEST src-address=172.16.0.0/24
add action=accept chain=input comment="Internal DNS server and forwarder" \
    dst-port=53 in-interface-list=!WAN protocol=udp
add action=drop chain=input comment="Drop Invalid connections" \
    connection-state=invalid
add action=drop chain=input comment="Drop all packets which are not destined t\
    o routes IP address (addr type not local)" dst-address-type=!local
add action=drop chain=input comment=\
    "Drop all packets which does not have unicast source IP address" \
    src-address-type=!unicast
add action=drop chain=input comment="Drop all packets from public internet whi\
    ch should not exist in public network" in-interface-list=WAN \
    src-address-list=PrivateNetworks
add action=drop chain=forward comment="No routing between LAN" \
    dst-address-list=IsolatedLANs src-address-list=IsolatedLANs
add action=drop chain=forward comment="Drop invalid packets" \
    connection-state=invalid
add action=drop chain=forward comment="Drop not DestNATed from WAN" \
    connection-nat-state=!dstnat connection-state=new in-interface-list=WAN \
    log-prefix="Not DestNATed - "
add action=drop chain=forward comment="Drop Private Networks from WAN" \
    in-interface-list=WAN src-address-list=PrivateNetworks
add action=drop chain=forward comment="Drop all packets from local network to \
    internet which should not exist in public network" dst-address-list=\
    PrivateNetworks in-interface-list=LAN
add action=drop chain=forward comment="Bad Office Network address" \
    in-interface-list=LAN src-address=!192.168.0.0/24
add action=drop chain=forward comment="Bad Guest Network address" \
    in-interface-list=GUEST src-address=!172.16.0.0/24
add action=drop chain=input comment="Drop everything else"

Thanks in advance anyone!
Lars.

Larsa · Sat Jan 28, 2017 12:10 am

Can anyone please verify if the following argument is correct:

If I understand the packet flow diagram correctly, Source-NAT will never be reached since the packet path takes a shortcut directly to Dest-NAT in the prerouting step and will set the Guest Network 172.16.0.0 as the source address. This means you still need to use hairpin NAT to make it work, for example:

add action=masquerade chain=srcnat src-address=172.16.10.0/24 dst-address=192.168.6.0/24 out-interface=LAN

Btw, Is it possible to force a Source-NAT before prerouting or is there any other way to accomplish this behavior?

Larsa · Sun Jan 29, 2017 5:56 pm

[Placeholder]
Example of a working solution based on the picture above is coming soon...

Sob · Mon Jan 30, 2017 9:16 pm

Remove "in-interface-list=WAN" from here:

/ip firewall nat
add action=dst-nat chain=dstnat comment="SMTP" dst-address=\
    1.1.1.1 dst-port=25 in-interface-list=WAN log=yes log-prefix=SMTP \
    protocol=tcp to-addresses=192.168.0.25 to-ports=25

Remove these rules completely:

/ip firewall filter
add action=accept chain=forward comment=\
    "Forward packets belonging to natted connections" connection-nat-state=\
    dstnat connection-state=established,related in-interface-list=WAN
add action=drop chain=forward comment="Drop not DestNATed from WAN" \
    connection-nat-state=!dstnat connection-state=new in-interface-list=WAN \
    log-prefix="Not DestNATed - "

And instead add this as last rule:

/ip firewall filter
add action=drop chain=forward comment="Drop not DestNATed from WAN" \
    connection-nat-state=!dstnat in-interface-list=WAN \
    log-prefix="Not DestNATed - "

And you're done.

You don't even need any extra hairpin rules, your current srcnat rules will take of that. Accesses from guest network will look to internal servers like they come from 1.1.1.2 and accesses from office network will use 1.1.1.1.

Sob · Mon Jan 30, 2017 9:22 pm

Correction, you need one extra rule to specifically allow all forwarded ports and it needs to be before the one blocking direct communication between subnets (I'm just not used to this reversed logic of blacklist-style firewall where everything is allowed by default):

/ip firewall filter
add action=accept chain=forward connection-nat-state=dstnat

Larsa · Mon Jan 30, 2017 11:49 pm

Correction, you need one extra rule to specifically allow all forwarded ports and it needs to be before the one blocking direct communication between subnets (I'm just not used to this reversed logic of blacklist-style firewall where everything is allowed by default):
Code: Select all
/ip firewall filter
add action=accept chain=forward connection-nat-state=dstnat

Sob, thanks a million for the suggestions! Btw, is it the line above that makes the trick and creates a shortcut direct to post-routing thus bypassing the internal routing or where does the magic happen?

Sob · Tue Jan 31, 2017 1:38 am

There's no magic, just normal routing and everything. The only trick is good srcnat rules. If traffic comes from 172.16.0.0/24, its source address will be changed to 1.1.1.2, no matter where it goes to. If it goes somewhere to internet or if it's forwarded to any LAN, it makes no difference, the source will always be changed to 1.1.1.2. And same way for other LAN. Think about it as implicit hairpin NAT.

All you need to take care of is allowing traffic from one LAN to another in forward chain if it's forwarded by dstnat rule. Your original rules are little strange in this regard. If you have public service accessible for whole world, it makes no sense to not have it accessible from your guest LAN. So the only important change is to allow forwarded ports from anywhere, in both dstnat and filter rules. You almost had it already, but limited to established & related connections and only for connections coming from WAN. Remove these limitations and that's it.

For the record, there need to be two rules with connection-nat-state, because of how your firewall is made. You have everything allowed by default and you're blocking selected stuff. So you first need to allow dstnatted connections, because by next rule you block direct communication between LANs. But because everything is allowed by default, you need another rule to drop connections from WAN, unless they are dstnatted.

The other approach (which I'm a fan of) is to have everything blocked by default (by unconditional drop/reject rule at the end of forward chain) and only allow selected stuff:

/ip firewall filter
add action=accept chain=forward connection-state=established,related
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward in-interface-list=LAN src-address=192.168.0.0/24 out-interface=WAN
add action=accept chain=forward in-interface-list=GUEST src-address=172.16.0.0/24 out-interface=WAN
add action=accept chain=forward connection-nat-state=dstnat
add action=drop chain=forward

Private subnets leaking to internet can be either specifically blocked by filter rule or marked as unreachable, so the router won't ever try to route them to its default gateway:

/ip route
add dst-address=10.0.0.0/8 type=unreachable
add dst-address=172.16.0.0/12 type=unreachable
add dst-address=192.168.0.0/16 type=unreachable

Larsa · Tue Jan 31, 2017 11:00 pm

It's really quite obvious when you think about it in peace and quiet, right?

I was totally focused on the wrong areas looking for a solution. Partly because I was informed that a “Double-NAT“ (:-) or “logical loop-back” was very difficult and required a substantial work effort to implement and secondly, probably because I misinterpreted “http://forum.mikrotik.com/viewtopic.php?t=40113#p198823” and took for granted that a hit in the dstnat chain might trigger an exit to the final routing thus bypassing the srcnat chain.

Anyhow, all has been tested and verified in a lab environment. I'll update a complete example of the solution somewhat later on. Interesting enough it was pretty easy to find out if the internal NAT-on-NAT worked out correctly using the log, especially in the second line where you can spot the whole packet path:

firewall,info DST-NAT OFFICE (HTTP) dstnat: in:#VLAN-172 out:(none), src-mac 00:15:5d:ca:45:1d, proto TCP (SYN), 172.16.10.199:49685->1.1.1.1:80, len 52 
firewall,info SRC-NAT GUEST srcnat: in:(none) out:ether4, src-mac 00:15:5d:ca:45:1d, proto TCP (SYN), 172.16.10.199:49685->192.168.0.1:80, NAT 172.16.10.199:49685->(1.1.1.1:80->192.168.0.1:80), len 52

Btw, are you aware if it exist some kind of architectural design description that illustrates the routing engine and the complete packet process in relation to all chains and as well to the Mikrotik flow diagrams found in the wiki? Also, do you happen to know if the routing engine is based purely on Linux iptable (xtables/netfilter) and follow the same design principles? If that’s the case there might be a ton of detailed info on the net regarding the packet flow through all chains...

In any case, thank you very much for all the elaborated answers and for your time and effort to help out!

ps...
"/ip route add dst-address=x.x.x type=unreachable" was the finishing touch and is much cleaner than just a drop-filter!

Sob · Wed Feb 01, 2017 3:01 am

RouterOS is fun, powerful and admin-friendly at the same time. It basically is Linux with iptables in very nice package. There might be some changes to internal working, but it's hard to tell what exactly. Some features are omitted, it can't do everything that iptables can, but important parts are there and principles remain. And again, such nice package, in the end it's worth it.

Packet flow is documented in manual (Packet Flow v6). A little scary at first, but it's all there.

AzAel76 · Tue Sep 11, 2018 10:18 am

sob i just wanted to thank you for your excellent explanation on this.

Cheers

[SOLVED] General NAT access between local networks using multiple public WAN address?

[SOLVED] General NAT access between local networks using multiple public WAN address?

Re: General NAT access between local networks using multiple public WAN address?

Re: General NAT access between local networks using multiple public WAN address?

Re: General NAT access between local networks using multiple public WAN address?

Re: General NAT access between local networks using multiple public WAN address?

Re: General NAT access between local networks using multiple public WAN address?

Re: General NAT access between local networks using multiple public WAN address?

Re: General NAT access between local networks using multiple public WAN address?

Re: [SOLVED] General NAT access between local networks using multiple public WAN address?

Re: [SOLVED] General NAT access between local networks using multiple public WAN address?

Who is online