RouterOS - public subnet - NAT 1:1 - good practices

tyby · October 10, 2020, 2:38pm

Hi guys!

One of my customers has a /29 public subnet routed by the ISP through a link public IP (static assigned) for quite a few years.

All customer services (like MTA, DVR, monitoring etc). running on various machines on the LAN, are bound to different public IP from the /29 subnet (with VIPs configured on a Fortigate 40C).

As the Forti is pretty obsolete, I’m trying to find the best option to replicate the configuration with a RouterOS (most probable will be a entry level CCR).

The main scenario is following (I will put it with below test rig information, in order to be easier to follow, please don’t mind the non-public link IP):

ISP link IP: 192.168.0.253/24
gateway for link IP: 192.168.0.254
/29 subnet: 2.2.2.0/29 routed through 1.1.1.1 above (link IP).
/24 local subnet for clients: 10.0.0.0/24

From the /29 above I need to assign some public IPs for specific access to / from LAN side: ex: one for MTA, one for httpd, one for NAT access for rest of the clients.

I managed to configure the above setup (or similar) on different routers (Juniper, AT, Fortigate), using different approaches (eNat, VIPs, etc) Mikrotik is giving me some issues. There is also an older topic on the subject, which is now closed, and now I found some spare time (due to overall global status) and I am back on the issue.

Starting from http://forum.mikrotik.com/t/nat-plus-1-1-nat/50487/1 i managed to get some results.

This is what I managed to do (full script below, test rig is a hAP AC2, clients and test - different real and virtual machines in my office, with proper routes added):

1:1 NAT for 2.2.2.2 to 10.0.0.199 - the “httpd machine” - which is fine, and works as intended
1:1 NAT for 2.2.2.3 to 10.0.0.198 - the “MTA machine” - which is fine, and works as intended
masquerade for rest of the clients: 2.2.2.4 to 10.0.0.0/24
hairpin NAT from 10.0.0.0/24 to above services (httpd - TCP 80 to 10.0.0.199, SMTP - TCP 25 to 10.0.0.198, from rest of the LAN clients) (now seem to work after removing interface binding)

What I am missing:

confirmation that this is the right approach for the job. Should I do it otherwise? Is there any risk doing it like this?
firewall access rules for 1:1 NAT public IPs (will do, they are not important at this time, as the test rig is behind a local subnet) - if you have any advice on this one, would be welcome!

I filtered the config below (like setting interfaces names, wifi setup, etc):

/interface bridge
add name=bridge-local

/interface bridge port
add bridge=bridge-local interface=ether2-master-local
add bridge=bridge-local interface=ether3-slave-local
add bridge=bridge-local interface=ether4-slave-local
add bridge=bridge-local interface=ether5-slave-local
add bridge=bridge-local interface=wlan1
add bridge=bridge-local interface=wlan2

/ip address
add address=10.0.0.254/24 comment="LAN interface" interface=bridge-local network=10.0.0.0
add address=192.168.0.253/24 comment="WAN interface" interface=ether1-gateway network=192.168.0.0
add address=2.2.2.1/29 comment="public /29" interface=ether1-gateway network=2.2.2.0
add address=2.2.2.2/29 interface=ether1-gateway network=2.2.2.0
add address=2.2.2.3/29 interface=ether1-gateway network=2.2.2.0
add address=2.2.2.4/29 interface=ether1-gateway network=2.2.2.0

/ip route
add distance=1 gateway=192.168.0.254

/ip dhcp-server network
add address=10.0.0.0/24 comment="default configuration" dns-server=192.168.0.246 gateway=10.0.0.254 netmask=24

/ip dns
set allow-remote-requests=yes servers=192.168.0.246

/ip firewall filter
add chain=input comment="default configuration" protocol=icmp
add chain=input comment="default configuration" connection-state=established
add chain=input comment="default configuration" connection-state=related
add chain=input comment="Winbox accept from 192.168.0.0/24" dst-port=8291 protocol=tcp src-address=192.168.0.0/24
add chain=forward comment="default configuration" connection-state=established
add chain=forward comment="default configuration" connection-state=related

/ip firewall nat
add action=masquerade chain=srcnat comment="2.2.2.2 traffic - HTTP server" dst-address=10.0.0.199 src-address=10.0.0.0/24
add action=dst-nat chain=dstnat dst-address=2.2.2.2 to-addresses=10.0.0.199
add action=src-nat chain=srcnat src-address=10.0.0.199 to-addresses=2.2.2.2

add action=masquerade chain=srcnat comment="2.2.2.3 traffic - MTA" dst-address=10.0.0.198 src-address=10.0.0.0/24
add action=dst-nat chain=dstnat dst-address=2.2.2.3 to-addresses=10.0.0.198
add action=src-nat chain=srcnat src-address=10.0.0.198 to-addresses=2.2.2.3

add action=src-nat chain=srcnat comment="2.2.2.4 - rest of the clients" out-interface=ether1-gateway src-address=10.0.0.0/24 to-addresses=2.2.2.4

add action=masquerade chain=srcnat comment="general public maquerade" out-interface=ether1-gateway

Thank you!

T

PS: added MTA rules, corrected 1:1 masquerade, correct topic title (initially left by mistake as “RouterOS”).

Sob · October 10, 2020, 4:01pm

For start, if whole /29 is routed to you, I wouldn’t assign those addresses to router like you do. If you want to go with NAT (you don’t have to, it’s also possible to give those addresses directly to target machines), add them with /32 mask and you can use all eight of them, including the first and last one. But it’s just a bonus.

Hairpin NAT can be simplified, one common rule is enough in most cases, and it covers all servers and ports:

/ip firewall nat
add chain=srcnat src-address=10.0.0.0/24 dst-address=10.0.0.0/24 action=masquerade

And in fact, it can be done even without that. If you add your other srcnat rules like this, they will also work as hairpin rules:

/ip firewall nat
add chain=srcnat src-address=10.0.0.199 action=src-nat to-addresses=2.2.2.2
add chain=srcnat src-address=10.0.0.198 action=src-nat to-addresses=2.2.2.3
add chain=srcnat src-address=10.0.0.0/24 action=src-nat to-addresses=2.2.2.4

Only problem would be if you’d have some other local subnets, traffic to them would be also affected by this, but those could be excluded using other rules.

tyby · October 10, 2020, 7:19pm

Hi Sob, thanks for quick insight!

I am not a fan of assigning that subnet directly to servers, I prefer servers having only their local C class addresses, and work with vip-like setup at router side, this seems easier to manage (for me, at least), both router and the machines.

This is just a test rig. In fact, the IPs are assigned individually with a /32, but the interface addressing just feels somehow more “crowded”.

What about the bigger /28 or /27 subnets, is adding each ip as a /32 an option also? Does this have any impact on the performance (I imagine having a 32 IPs interface is not so easy to process)?

Regarding hairpin, I intend to go with a needed-services/ports only dst-nat, and use the same ports also for hairpin, thus a separate hairpin rule seems a better choice.

On the other hand, a single hairpin rule, while having a somehow straightforward LAN addressing, so not very demanding, might be a good idea! :coolspeak:

I will play with your suggestions, and revert!

Thanks again!

PS: sorry for might-be typos, wrote above on my mobile.

tyby · October 10, 2020, 7:41pm

PS: would it be a better idea to assign the public subnet IPs to bridge-local side?

Sob · October 10, 2020, 8:10pm

If in the end you add all addresses from given subnet to router, I don’t think it matters what mask you use, it’s still the same amount of addresses. Or maybe you meant something else?

I don’t see good reason for multiple separate hairpin rules. The universal one will cover everything and since normally there’s no traffic going from LAN back to same LAN, unless it’s dstnatted, it doesn’t interfere with anything it shouldn’t. One exception is when you have VPN where clients get addresses from LAN subnet and there are overlapping subnets, but even separate rules for individual services have similar problem with that.

The best (for clarity, because I don’t think that router really cares in this case) would be to assign addresses to some loopback interface. RouterOS doesn’t have that, but it’s possible to use empty bridge.

tyby · October 12, 2020, 10:28am

Thanks, Sob, for patience!

Regarding the /32 addressing for the public subnet to one interface, I was referring at the amount of lines required (i thing the network setting is correct below, 192.168.0.254 being the nexthop / my office router LAN address):

/ip address
[...]
add address=2.2.2.1 comment="public /29" interface=ether1-gateway network=192.168.0.254
add address=2.2.2.2 interface=ether1-gateway network=192.168.0.254
...
add address=2.2.2.7 interface=ether1-gateway network=192.168.0.254

For a /26 network, if you need 50 public IP addresses of the /26 space, you will need to add each and every one like above.

Am I getting this all wrong?!

Understood the loopback idea, and I agree it’s cleaner that way. Did it, and works just fine! :coolspeak:

Basically I’ve reduced the public subnet NAT configuration to just 2 main sections for now (while Hairpin NAT, and also general masquerading do not seem to be necessary)

DISABLED Hairpin NAT - I’ve tested on Saturday also without, but - somehow - did not get the expected result, will play some more during today - already test with below hairpin disabled, seems to work just fine, at least in my small test environment, will check further when configuration will get somehow more complicated)

/ip firewall nat
add action=masquerade chain=srcnat comment="general hairpin NAT" disabled=yes dst-address=10.0.0.0/24 src-address=10.0.0.0/24

2.2.2.x local services trafic to / from 10.0.0.x (I am thinking that a good practice is that only required ports will be dstnat-ed, each batch depending of further acl, like for 2.2.2.2 below)

 /ip firewall nat
# 2.2.2.2 traffic
add action=dst-nat chain=dstnat comment="2.2.2.2 traffic - restricted RDP at T450-TM" dst-address=2.2.2.2 dst-port=3389 protocol=tcp src-address-list=myHosts to-addresses=10.0.0.198
add action=dst-nat chain=dstnat comment="2.2.2.2 traffic - general SMTP(s), HTTP(s) at T450-TM" dst-address=2.2.2.2 dst-port=25,465,80,443 protocol=tcp to-addresses=10.0.0.198
add action=src-nat chain=srcnat src-address=10.0.0.198 to-addresses=2.2.2.2

# 2.2.2.3 traffic
add action=dst-nat chain=dstnat comment="2.2.2.3 traffic - MTA on vBox @T450" dst-address=2.2.2.3 to-addresses=10.0.0.197
add action=src-nat chain=srcnat src-address=10.0.0.197 to-addresses=2.2.2.3

2.2.2.4 traffic for the rest of the LAN clients

/ip firewall nat
add action=src-nat chain=srcnat comment="2.2.2.4 - rest of the clients" out-interface=ether1-gateway src-address=10.0.0.0/24 to-addresses=2.2.2.4

DISABLED the general public maquerade (is this really needed?!, as it normally should not be needed)

/ip firewall nat
add action=masquerade chain=srcnat comment="general public maquerade" disabled=yes out-interface=ether1-gateway# 2.2.2.2 traffic

I will revert with more testing results, after getting the config right for the customer.

Thanks again for your help!

T

Sob · October 12, 2020, 1:32pm

If you want to have all addresses assigned to router (instead of routing them somewhere else), you need to add them all. Unless you’d choose to not add them at all.
You do need extra hairpin NAT rule with your other current rules, but with a small change you can get rid of it. See below.
Advantage of forwarding only selected ports is that you can be sure that nothing else will pass. The same can be achieved with firewall on target device, but sometimes you may not have enough trust in that, or in device’s admin, so doing it on router is sure way.
Your srcnat rule for rest of clients has out-interface=ether1-gateway. If you remove this condition, it will also start working as hairpin NAT rule. Depending on what other config you’ll have, it may or may not cause some problems. For example, if you’d have other networks that should be accessible from 10.0.0.x without NAT (another LAN, VPN, …) it would affect traffic to them. But that could be solved too, for example with accept rule for this traffic earlier in srcnat chain.
If your other srcnat rules cover everything you need, you don’t need default masquerade.

tyby · October 12, 2020, 4:00pm

Noted, thanks!

I’d prefer to have the IPs / subnet “visible”, for that whomever will see it to not have to dig or make educated guesses …

As with below config I could not reach outside world (could not reach 192.168.0.254, which is my local network primary gw) - I think that it has to do with the way I’ve added the 2.2.2.0/29 network (I think am I doing it wrong?!)

add address=2.2.2.1 comment="public /29" interface=bridge-29-loopback network=192.168.0.254
add address=2.2.2.2 interface=bridge-29-loopback network=192.168.0.254
add address=2.2.2.3 interface=bridge-29-loopback network=192.168.0.254
add address=2.2.2.4 interface=bridge-29-loopback network=192.168.0.254

route print is as below (and thus I think it explains why clients do not have valid route to external):

Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          192.168.0.254             1
 1 ADC  10.0.0.0/24        10.0.0.254      bridge-local              0
 2 ADC  192.168.0.0/24     192.168.0.253   ether1-gateway            0
 3 ADC  192.168.0.254/32   2.2.2.1         bridge-29-loopback        0

So I will use the 2.2.2.0/29 added to the loopback bridge:

/ip address
add address=10.0.0.254/24 comment="LAN interface" interface=bridge-local network=10.0.0.0
add address=192.168.0.253/24 comment="WAN interface" interface=ether1-gateway network=192.168.0.0
add address=2.2.2.0/29 comment="WAN interface" interface=ether1-gateway network=2.2.2.0

(Or just all /29 needed IP’s, added one by one, to keep a better track of them - I will play a little more with this later … )

Regarding dstnat - I prefer to have a better control at what is passing, and I think it’s easier to have main filtering layer at router side.

As I’ve added also a ppp VPN test config, with it’s own IP pool in the same 10.0.0.0/24, and it’s using it’s own srcnat rule (to 2.2.2.5), just above the general clients.

General masquerading indeed not needed, so it’s out of use.

Will revert with news!

Thank you again!

T

Sob · October 12, 2020, 4:51pm

IP addresses should not have network=192.168.0.254, omit that parameter completely when adding them. With it you created point to point config and it breaks things.

Otherwise you can use /29 too, /32 was to allow you to use all addresses, even those that are normally wasted on network address and broadcast. Actually, router allows to add them even with non-/32 mask, but I’m not sure if it doesn’t cause some problem. When they have /32, it’s safe.

If you removed out-interface from srcnat rule for clients like I wrote, then if you now connect from 10.0.0.x to something in VPN subnet, it will get the source changed to 2.2.2.4. I described that too, including fix. Of course you can also keep out-interface and then have separate srcnat rule for hairpin, instead of this. Choose what you like better.

tyby · October 13, 2020, 10:12am

Hi Sob, noted with thanks.

I imagined it was due to network settings - it did not feel right, but I followed man pages

Test NAT rules are like below at the moment, and seem to work just fine. Will test more later, and revert.

/ip firewall nat

# 2.2.2.2 traffic
add action=dst-nat chain=dstnat comment="2.2.2.2 traffic - restricted RDP at T450-TM" dst-address=2.2.2.2 dst-port=3389 protocol=tcp src-address-list=myHosts to-addresses=10.0.0.198
add action=dst-nat chain=dstnat comment="2.2.2.2 traffic - general SMTP(s), HTTP(s) at T450-TM" dst-address=2.2.2.2 dst-port=25,465,80,443 protocol=tcp to-addresses=10.0.0.198
add action=src-nat chain=srcnat src-address=10.0.0.198 to-addresses=2.2.2.2

# 2.2.2.3 traffic
add action=dst-nat chain=dstnat comment="2.2.2.3 traffic - MTA on vBox @T450" dst-address=2.2.2.3 dst-port=25,465,993,995 protocol=tcp to-addresses=10.0.0.197
add action=src-nat chain=srcnat src-address=10.0.0.197 to-addresses=2.2.2.3

# 2.2.2.5 - VPN traffic
add action=src-nat chain=srcnat comment="2.2.2.5 - VPN clients" src-address=10.0.0.64/28 to-addresses=2.2.2.5

# 2.2.2.4 - general clients
add action=src-nat chain=srcnat comment="2.2.2.4 - rest of the clients" src-address=10.0.0.0/24 to-addresses=2.2.2.4

Will keep you posted. Hopefully this will help also others at some point.

Thanks!