1:1 NAT / DNAT configuration help

juwalter · Mon Mar 24, 2025 5:48 pm

Hello,

we are using "CCR2004-16G-2S+” and need to configure 1:1 NAT (bi-directional) from 8x internal device to 8x external IP address.

           +-------------------------------+
           |   Linux Edge Device (Client)  |
           |   IP: 172.29.10.10/24         |
           +---------------+---------------+
                           |
                           |
     +---------------------+--------------------------+
     |           |     |     |     |      |      |    |
172.29.10.101  .102  .103  .104  .105   .106   .107 .108    
                           |  (eth all "virtual" IPs are e.g. 172.29.10.x)
     +---------------------+----------------------+
     |                  MikroTik                  |
     |             IP: 172.29.10.1/24             |
     +---------------------+----------------------+
     |       |       |       |       |       |     |
   Eth1    Eth2    Eth3    Eth4    Eth5    Eth6  ... Eth8
     |       |       |       |       |       |       |
     |       |       |       |       |       |       |
+----+--+ +--+----+ +--+----+ +--+----+ +--+----+ +---+---+
| PLC1 | | PLC2  | | PLC3  | | PLC4  | | PLC5  | | PLC8  |
| IP:  | | IP:   | | IP:   | | IP:   | | IP:   | | IP:   |
|192.168.0.2     (same IP for all PLCs)          |
+------------+ +-------+ +-------+ +-------+ +--------+

Our setup is like this:

we have 8 PLC; each PLC has same IP address 192.168.0.2

we cannot change IP address of PLC or modify network settings of PLC

each PLC will be connected to a dedicated ethernet port Eth1, Eth2, etc. on Mikrotik CCR2004-16G-2S+

we also have one Linux PLC edge device; this edge device needs to access each PLC individually

plan is to use IP address range 172.29.10.0/24 for “outside” network: edge device will have IP address 172.29.10.10; Mikrotik CCR2004-16G-2S+ will e.g. have 172.29.10.1

Mikrotik CCR2004-16G-2S+ will route access to 172.29.10.101 (=outside / NAT IP address for PLC1) to PLC1 connected to Eth1 (PLC1 has IP address 192.168.0.2)

Mikrotik CCR2004-16G-2S+ will route access to 172.29.10.102 (=outside / NAT IP address for PLC2) to PLC2 connected to Eth2 (PLC2 has also IP address 192.168.0.2) for all 8x PLC

only IP addresses for PLC are fixed to 192.168.0.2 - for everything else we are free to do as we see fit

only ports 102 and 80 and 443 need to be mapped/accessible

What would be the best way to configure this on Mikrotik CCR2004-16G-2S+? Should we do 8x 172.29.10.101-108 on WAN interface and then use netmap which seems exactly to have been created for such a use case. (I have seen this here: https://help.mikrotik.com/docs/spaces/R ... 211299/NAT - table very and of page)

Below is some cobbled together code that kind of illustrates the idea. Will this work/is this a practical solution for RouterOS?

We are very much looking forward to your suggestions set this up. Many thanks in advance! Jürgen

# WAN interface (Edge device connection)
/ip address add address=172.29.10.1/24 interface=ether9 comment="Outside Edge Network"

# Create separate bridge for each PLC to isolate networks
/interface bridge
add name=bridge-plc1 comment="PLC1 Network"
add name=bridge-plc2 comment="PLC2 Network"
add name=bridge-plc3 comment="PLC3 Network"
add name=bridge-plc4 comment="PLC4 Network"
add name=bridge-plc5 comment="PLC5 Network"
add name=bridge-plc6 comment="PLC6 Network"
add name=bridge-plc7 comment="PLC7 Network"
add name=bridge-plc8 comment="PLC8 Network"

# Assign physical interfaces to bridges (one port per bridge)
/interface bridge port
add bridge=bridge-plc1 interface=ether1
add bridge=bridge-plc2 interface=ether2
add bridge=bridge-plc3 interface=ether3
add bridge=bridge-plc4 interface=ether4
add bridge=bridge-plc5 interface=ether5
add bridge=bridge-plc6 interface=ether6
add bridge=bridge-plc7 interface=ether7
add bridge=bridge-plc8 interface=ether8

# Assign same IP to each bridge (router will have 192.168.0.180 in each isolated network)
/ip address
add address=192.168.0.180/24 interface=bridge-plc1 comment="Gateway for PLC1"
add address=192.168.0.180/24 interface=bridge-plc2 comment="Gateway for PLC2"
add address=192.168.0.180/24 interface=bridge-plc3 comment="Gateway for PLC3"
add address=192.168.0.180/24 interface=bridge-plc4 comment="Gateway for PLC4"
add address=192.168.0.180/24 interface=bridge-plc5 comment="Gateway for PLC5"
add address=192.168.0.180/24 interface=bridge-plc6 comment="Gateway for PLC6"
add address=192.168.0.180/24 interface=bridge-plc7 comment="Gateway for PLC7"
add address=192.168.0.180/24 interface=bridge-plc8 comment="Gateway for PLC8"

# 1:1 NAT rules using netmap for bidirectional NAT
/ip firewall nat
# Destination NAT (incoming traffic from Edge device to PLCs)
add chain=dstnat dst-address=172.29.10.101 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc1
add chain=dstnat dst-address=172.29.10.102 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc2
add chain=dstnat dst-address=172.29.10.103 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc3
add chain=dstnat dst-address=172.29.10.104 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc4
add chain=dstnat dst-address=172.29.10.105 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc5
add chain=dstnat dst-address=172.29.10.106 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc6
add chain=dstnat dst-address=172.29.10.107 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc7
add chain=dstnat dst-address=172.29.10.108 action=dst-nat to-addresses=192.168.0.2 to-ports=0-65535 in-interface=ether9 out-interface=bridge-plc8

# Source NAT (outgoing traffic from PLCs to Edge device)
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.101 to-ports=0-65535 in-interface=bridge-plc1 out-interface=ether9
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.102 to-ports=0-65535 in-interface=bridge-plc2 out-interface=ether9
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.103 to-ports=0-65535 in-interface=bridge-plc3 out-interface=ether9
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.104 to-ports=0-65535 in-interface=bridge-plc4 out-interface=ether9
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.105 to-ports=0-65535 in-interface=bridge-plc5 out-interface=ether9
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.106 to-ports=0-65535 in-interface=bridge-plc6 out-interface=ether9
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.107 to-ports=0-65535 in-interface=bridge-plc7 out-interface=ether9
add chain=srcnat src-address=192.168.0.2 action=src-nat to-addresses=172.29.10.108 to-ports=0-65535 in-interface=bridge-plc8 out-interface=ether9

lurker888 · Tue Mar 25, 2025 12:41 pm

Hi,

what you want to do comes up quite frequently in industrial automation type scenarios.

The good news is that it absolutely can be done, however it requires something called "policy routing", which is not the easiest for a first try at configuring Mikrotik devices.

Some ideas:
viewtopic.php?t=187178

Just a side note. You bought a quite expensive router for what I assume to be a very low amount of traffic. I hope that for any sort of industrial setting you bought the PC (passively cooled) version, if not, then exchange it if possible. Passively cooled stuff have much better life expectancies and require less maintenance (cleaning). Also - if you need the number of ports - any of the CRS series may be suitable for your use case.

juwalter · Tue Mar 25, 2025 2:24 pm

Hello lurker888,

many thanks for your reply! And pointing out the somewhat interesting choice of router model (sadly, I was not involved in the purchasing decision) - we might revisit this decision!

I had a look at the posts you linked - good call re: "action=mark-connection" and "action=mark-routing" (firewall mangling). I did some more studying as well and came up with this solution. Any thoughts? Is choice of "ether1" as "WAN" sound or am I missing something?

Many thanks in advance!! Jürgen

(simplified for 2x PLC only)

# Identity for better management
/system identity set name="PLC-Gateway"

#######################################
# Interface Organization
#######################################

# Create interface lists for better management
/interface list add name=PLC-PORTS
/interface list add name=WAN

# Add interfaces to lists
/interface list member
add interface=ether1 list=WAN                # Using default WAN port
add interface=ether2 list=PLC-PORTS
add interface=ether3 list=PLC-PORTS

#######################################
# Bridge Configuration
#######################################

# Create isolated bridges for each PLC
/interface bridge
add name=plc1-bridge protocol-mode=none
add name=plc2-bridge protocol-mode=none

# Assign physical interfaces to bridges
/interface bridge port
add bridge=plc1-bridge interface=ether2
add bridge=plc2-bridge interface=ether3

#######################################
# IP Configuration
#######################################

# WAN interface (Edge device connection)
/ip address 
add address=172.29.10.1/24 interface=ether1 comment="Edge Network Primary IP"

# Add virtual IPs for each PLC on the WAN interface
add address=172.29.10.101 interface=ether1 comment="PLC1 External IP"
add address=172.29.10.102 interface=ether1 comment="PLC2 External IP"

# Configure IP addresses for each bridge using 192.168.0.180
add address=192.168.0.180/24 interface=plc1-bridge comment="PLC1 Network"
add address=192.168.0.180/24 interface=plc2-bridge comment="PLC2 Network"

#######################################
# Connection Marking for Policy Routing
#######################################

# Mark connections based on the external IP being accessed
/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"

# Convert connection marks to routing marks for policy routing
add chain=prerouting connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Route to PLC1"
add chain=prerouting connection-mark=plc2-conn action=mark-routing new-routing-mark=to-plc2 comment="Route to PLC2"

#######################################
# Policy-Based Routing Rules
#######################################

# Policy routes to direct traffic to the correct PLC
/ip route
add dst-address=192.168.0.2/32 gateway=plc1-bridge routing-mark=to-plc1 comment="Route to PLC1"
add dst-address=192.168.0.2/32 gateway=plc2-bridge routing-mark=to-plc2 comment="Route to PLC2"

#######################################
# NAT Configuration - Inbound Only
#######################################

# Destination NAT only (no source NAT needed for inbound-only)
/ip firewall nat
add chain=dstnat in-interface=ether1 dst-address=172.29.10.101 action=dst-nat to-addresses=192.168.0.2 comment="PLC1 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.102 action=dst-nat to-addresses=192.168.0.2 comment="PLC2 Inbound"

#######################################
# Connection Tracking Optimization
#######################################

# Optimize connection tracking for industrial environment
/ip firewall connection tracking
set enabled=yes tcp-established-timeout=1h udp-timeout=3m

#######################################
# Simplified Firewall for Inbound Connections
#######################################

# Basic firewall setup with connection tracking
/ip firewall filter
add chain=input action=accept connection-state=established,related comment="Allow Established Connections"
add chain=input action=accept in-interface-list=PLC-PORTS comment="Allow PLC Access"

# Allow specific protocols needed for your environment
add chain=input action=accept in-interface=ether1 dst-port=102 protocol=tcp comment="Allow S7 Communication from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=80 protocol=tcp comment="Allow HTTP from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=443 protocol=tcp comment="Allow HTTPS from Edge Network"

add chain=input action=drop comment="Drop everything else"

add chain=forward action=accept connection-state=established,related comment="Allow Established Connections"
add chain=forward action=accept connection-state=new in-interface=ether1 out-interface-list=PLC-PORTS comment="Allow Edge to PLCs"
add chain=forward action=drop comment="Drop everything else"

lurker888 · Tue Mar 25, 2025 3:24 pm

Hello lurker888,

many thanks for your reply! And pointing out the somewhat interesting choice of router model (sadly, I was not involved in the purchasing decision) - we might revisit this decision!

Don't worry then. If you're looking for long-term reliability then I would seriously consider something passive. One of the nice things about working with Mikrotik devices is that they are all configured in the same way, therefore if you have a correctly working configuration, it's quite easy to adapt it to another device.

I had a look at the posts you linked - good call re: "action=mark-connection" and "action=mark-routing" (firewall mangling). I did some more studying as well and came up with this solution. Any thoughts? Is choice of "ether1" as "WAN" sound or am I missing something?

Many thanks in advance!! Jürgen

(simplified for 2x PLC only)

You're off to a very good start. However it won't work just yet. I'll comment on the errors inline.

# Identity for better management
/system identity set name="PLC-Gateway"

#######################################
# Interface Organization
#######################################

# Create interface lists for better management
/interface list add name=PLC-PORTS
/interface list add name=WAN

# Add interfaces to lists
/interface list member
add interface=ether1 list=WAN                # Using default WAN port
add interface=ether2 list=PLC-PORTS
add interface=ether3 list=PLC-PORTS

#######################################
# Bridge Configuration
#######################################

# Create isolated bridges for each PLC
/interface bridge
add name=plc1-bridge protocol-mode=none
add name=plc2-bridge protocol-mode=none

# Assign physical interfaces to bridges
/interface bridge port
add bridge=plc1-bridge interface=ether2
add bridge=plc2-bridge interface=ether3

Although I understand what you want to do, the router won't

You have decided to use bridges for the ports to the PLCs. While this is certainly possible, I see no possible reason to do so. Anyhow, once a port becomes part of a bridge, it disappears as an IP interface (and cannot be assigned an address, routed to, become part of an interface list - this is accepted by the system, but it1s meaningless, etc.) So either use bridges, then use the bridge as the interface *everywhere* OR what I would suggest - just forego the use of bridges altogether.

#######################################
# IP Configuration
#######################################

# WAN interface (Edge device connection)
/ip address 
add address=172.29.10.1/24 interface=ether1 comment="Edge Network Primary IP"

# Add virtual IPs for each PLC on the WAN interface
add address=172.29.10.101 interface=ether1 comment="PLC1 External IP"
add address=172.29.10.102 interface=ether1 comment="PLC2 External IP"

# Configure IP addresses for each bridge using 192.168.0.180
add address=192.168.0.180/24 interface=plc1-bridge comment="PLC1 Network"
add address=192.168.0.180/24 interface=plc2-bridge comment="PLC2 Network"

#######################################
# Connection Marking for Policy Routing
#######################################

# Mark connections based on the external IP being accessed
/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"

# Convert connection marks to routing marks for policy routing
add chain=prerouting connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Route to PLC1"
add chain=prerouting connection-mark=plc2-conn action=mark-routing new-routing-mark=to-plc2 comment="Route to PLC2"

#######################################
# Policy-Based Routing Rules
#######################################

# Policy routes to direct traffic to the correct PLC
/ip route
add dst-address=192.168.0.2/32 gateway=plc1-bridge routing-mark=to-plc1 comment="Route to PLC1"
add dst-address=192.168.0.2/32 gateway=plc2-bridge routing-mark=to-plc2 comment="Route to PLC2"

#######################################
# NAT Configuration - Inbound Only
#######################################

# Destination NAT only (no source NAT needed for inbound-only)
/ip firewall nat
add chain=dstnat in-interface=ether1 dst-address=172.29.10.101 action=dst-nat to-addresses=192.168.0.2 comment="PLC1 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.102 action=dst-nat to-addresses=192.168.0.2 comment="PLC2 Inbound"

#######################################
# Connection Tracking Optimization
#######################################

# Optimize connection tracking for industrial environment
/ip firewall connection tracking
set enabled=yes tcp-established-timeout=1h udp-timeout=3m

#######################################
# Simplified Firewall for Inbound Connections
#######################################

# Basic firewall setup with connection tracking
/ip firewall filter
add chain=input action=accept connection-state=established,related comment="Allow Established Connections"
add chain=input action=accept in-interface-list=PLC-PORTS comment="Allow PLC Access"

# Allow specific protocols needed for your environment
add chain=input action=accept in-interface=ether1 dst-port=102 protocol=tcp comment="Allow S7 Communication from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=80 protocol=tcp comment="Allow HTTP from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=443 protocol=tcp comment="Allow HTTPS from Edge Network"

add chain=input action=drop comment="Drop everything else"

add chain=forward action=accept connection-state=established,related comment="Allow Established Connections"
add chain=forward action=accept connection-state=new in-interface=ether1 out-interface-list=PLC-PORTS comment="Allow Edge to PLCs"
add chain=forward action=drop comment="Drop everything else"

Again, either use bridges *everywhere* and forget about the individual ports, or don't use them at all.

The conntrack timeouts don't have to be optimized in any way for your use case, however if you go ahead with setting them manually, setting udp timeout to 3m is not really recommended (I usually use 35s), however setting udp stream timeout to at least 6m is.

Your firewall filter rules are approximately ok. I wouldn't allow input (this is the chain that handles input to the router itself) from the PLC side. Also the connection-state=new is redundant.

My recommendation would be to disable all your firewall filter rules while you configure and verify everything related to the policy routing stuff. After that is done, it is best practice to reinstate and refine you filtering rules, but I think it's best to do things one thing at a time.

The other thing that you are missing is that the PLC devices actually have to be able to give replies to the packets that initiate packets towards them. How this is best done actually depends on the IP configuration on the PLCs. Your PLCs currently will see the incoming packets as coming from their real source address, and when they reply, they will attempt to send it there. If they have a default gateway configured then there, if they have none, it will get dropped.

A sure way to fix this is to rewrite the packets towards them as always coming from 192.168.0.180. This is done using srcnat rules, such as:
add chain=srcnat out-interface=ether2 action=src-nat to-addresses=192.168.0.180 comment="PLC1 outbound"
(repeated for all PLC ports)

This can be done for all ports in one rule:
add chain=srcnat out-interface-list=PLC-PORTS action=src-nat to-addresses=192.168.0.180 comment="PLC1 outbound"

You may see elsewhere action=masquerade with no to-addresses specified - this does the same thing.

Although the routing part of what you propose will work correctly as-is, I would suggest the following modifications:
add dst-address=192.168.0.2/32 gateway=plc1-bridge routing-mark=to-plc1 comment="Route to PLC1"
to
add dst-address=192.168.0.0/24 gateway=plc1-bridge routing-mark=to-plc1 comment="Route to PLC1"

Also, please pay attention to the fact that as your mangle rules are currently written, the "to-plc1" routing mark will be applies to packets coming *from* PLC1 (because the connection mark marks the *connection*, that is both direct and reply packets). For reply packets policy routing is not needed, therefore these mangle rules should be rewritten as:
add chain=prerouting connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Route to PLC1"
to
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Route to PLC1"

EDIT: Formatting.

juwalter · Wed Mar 26, 2025 12:48 am

Hello lurker888,

many, many thanks for giving such thorough and tremendously helpful feedback! Heh, the bridges are way over-engineered (and even wouldn't work) - good thing you explained this; I did away with them, and use the interfaces directly; the only thing I cannot wrap my head around is that the same "address=192.168.0.180/24" can actually be applied to multiple different interfaces without causing RouterOS to get confused.
Also removed custom settings for connection tracking, gone now - we are only doing TCP/IP. Nice suggestion about "out-interface-list=PLC-PORTS" (now I get the idea why defining those "lists" at the top makes sense). Big part is about lack of masquerading - somehow I thought "dstnat" will take care of this as well. I have also adjusted the policy-based routing.

Regarding:

Although the routing part of what you propose will work correctly as-is, I would suggest the following modifications:
add dst-address=192.168.0.2/32 gateway=plc1-bridge routing-mark=to-plc1 comment="Route to PLC1"
to
add dst-address=192.168.0.0/24 gateway=plc1-bridge routing-mark=to-plc1 comment="Route to PLC1"

I am going to use the broader "dst-address=192.168.0.0/24" vs "dst-address=192.168.0.2/32" as suggested, but I thought it made the overall configuration clearer to have the (only) destination address "192.168.0.2" here as well; on the other hand, it is less flexible and more maintenance burden to keep it in sync in case the destination IP every needs to be adjusted.

Below the "final" solution based on your comments. If you have a chance to give it another look, I would be grateful!

Thank you!! Jürgen

# Create interface lists for better management
/interface list
add name=PLC-PORTS
add name=WAN

# Add interfaces to lists
/interface list member
add interface=ether1 list=WAN
add interface=ether2 list=PLC-PORTS
add interface=ether3 list=PLC-PORTS

IP Configuration

# WAN interface (Edge device connection)
/ip address
add address=172.29.10.1/24 interface=ether1 comment="Edge Network Primary IP"
# Add virtual IPs for each PLC on the WAN interface
add address=172.29.10.101/32 interface=ether1 comment="PLC1 External IP"
add address=172.29.10.102/32 interface=ether1 comment="PLC2 External IP"
# Configure IP addresses for each PLC interface - directly using Ethernet ports
add address=192.168.0.180/24 interface=ether2 comment="PLC1 Network"
add address=192.168.0.180/24 interface=ether3 comment="PLC2 Network"

Routing

# Mark connections based on the external IP being accessed
/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"

# Convert connection marks to routing marks for policy routing - ONLY for packets coming from WAN
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Route to PLC1"
add chain=prerouting in-interface=ether1 connection-mark=plc2-conn action=mark-routing new-routing-mark=to-plc2 comment="Route to PLC2"

# Policy routes for the entire PLC subnet (/24), not just the PLC (/32)
/ip route
add dst-address=192.168.0.0/24 gateway=ether2 routing-mark=to-plc1 comment="Route to PLC1"
add dst-address=192.168.0.0/24 gateway=ether3 routing-mark=to-plc2 comment="Route to PLC2"

NAT

# Destination NAT
/ip firewall nat
add chain=dstnat in-interface=ether1 dst-address=172.29.10.101 action=dst-nat to-addresses=192.168.0.2 comment="PLC1 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.102 action=dst-nat to-addresses=192.168.0.2 comment="PLC2 Inbound"

# Source NAT for replies (makes router appear as 192.168.0.180 to PLCs)
add chain=srcnat out-interface-list=PLC-PORTS action=src-nat to-addresses=192.168.0.180 comment="Masquerade to PLCs"

Firewall for later, after all tested and working

# Minimal firewall setup with connection tracking
/ip firewall filter
add chain=input action=accept connection-state=established,related comment="Allow Established Connections"
# Allow specific protocols needed for S7 PLC communication and HTTP/HTTPS
add chain=input action=accept in-interface=ether1 dst-port=102 protocol=tcp comment="Allow S7 Communication from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=80 protocol=tcp comment="Allow HTTP from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=443 protocol=tcp comment="Allow HTTPS from Edge Network"
add chain=input action=drop comment="Drop everything else"

add chain=forward action=accept connection-state=established,related comment="Allow Established Connections"
add chain=forward action=accept in-interface=ether1 out-interface-list=PLC-PORTS comment="Allow Edge to PLCs"
add chain=forward action=drop comment="Drop everything else"

lurker888 · Wed Mar 26, 2025 1:28 am

[...] the only thing I cannot wrap my head around is that the same "address=192.168.0.180/24" can actually be applied to multiple different interfaces without causing RouterOS to get confused.

Having the same address and subnet on multiple interfaces only becomes problematic when routing to those addresses. This is because an address on a broadcast interface (such as ethernet) creates a "direct conected" (sometimes called "on-link") route to that subnet on the given interface implicitly. You can see this in the IP->Route display. This route is created in the "main" routing table; however whenever we are routing to these addresses, we mark the packet to be resolved not in the main but in a specific routing table, where only one matching route exists. This is exactly the reason why policy routing is needed.

Big part is about lack of masquerading - somehow I thought "dstnat" will take care of this as well.

Nope. Those are two separate things and both are needed in this case. Some home/consumer routers do both automatically, which in my view is incorrect and actually leads to other problems down the line.

I am going to use the broader "dst-address=192.168.0.0/24" vs "dst-address=192.168.0.2/32" as suggested, but I thought it made the overall configuration clearer to have the (only) destination address "192.168.0.2" here as well; on the other hand, it is less flexible and more maintenance burden to keep it in sync in case the destination IP every needs to be adjusted.

That is one point of view (and as I said, it would work correctly.) Another POV is that it is best to have all devices attached to a network have the same understanding of its nature/parameters, hence my suggestion. Also, if a /32 route is specified, I would have suggested a /32 address as well... and those have some peculiarities (again, not insurmountable.)

# Create interface lists for better management
/interface list
add name=PLC-PORTS
add name=WAN

# Add interfaces to lists
/interface list member
add interface=ether1 list=WAN
add interface=ether2 list=PLC-PORTS
add interface=ether3 list=PLC-PORTS

IP Configuration

# WAN interface (Edge device connection)
/ip address
add address=172.29.10.1/24 interface=ether1 comment="Edge Network Primary IP"
# Add virtual IPs for each PLC on the WAN interface
add address=172.29.10.101/32 interface=ether1 comment="PLC1 External IP"
add address=172.29.10.102/32 interface=ether1 comment="PLC2 External IP"
# Configure IP addresses for each PLC interface - directly using Ethernet ports
add address=192.168.0.180/24 interface=ether2 comment="PLC1 Network"
add address=192.168.0.180/24 interface=ether3 comment="PLC2 Network"

What you refer to as "virtual addresses" should also be /24. (Maybe I didn't spot this earlier?)

Routing

# Mark connections based on the external IP being accessed
/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"

# Convert connection marks to routing marks for policy routing - ONLY for packets coming from WAN
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Route to PLC1"
add chain=prerouting in-interface=ether1 connection-mark=plc2-conn action=mark-routing new-routing-mark=to-plc2 comment="Route to PLC2"

# Policy routes for the entire PLC subnet (/24), not just the PLC (/32)
/ip route
add dst-address=192.168.0.0/24 gateway=ether2 routing-mark=to-plc1 comment="Route to PLC1"
add dst-address=192.168.0.0/24 gateway=ether3 routing-mark=to-plc2 comment="Route to PLC2"

The parameter is actually called "routing-table".

NAT

# Destination NAT
/ip firewall nat
add chain=dstnat in-interface=ether1 dst-address=172.29.10.101 action=dst-nat to-addresses=192.168.0.2 comment="PLC1 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.102 action=dst-nat to-addresses=192.168.0.2 comment="PLC2 Inbound"

# Source NAT for replies (makes router appear as 192.168.0.180 to PLCs)
add chain=srcnat out-interface-list=PLC-PORTS action=src-nat to-addresses=192.168.0.180 comment="Masquerade to PLCs"

I wouldn't comment a rule "masquerade" if the actual action is not masquerade. It only invites confusion.

Firewall for later, after all tested and working

# Minimal firewall setup with connection tracking
/ip firewall filter
add chain=input action=accept connection-state=established,related comment="Allow Established Connections"
# Allow specific protocols needed for S7 PLC communication and HTTP/HTTPS
add chain=input action=accept in-interface=ether1 dst-port=102 protocol=tcp comment="Allow S7 Communication from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=80 protocol=tcp comment="Allow HTTP from Edge Network"
add chain=input action=accept in-interface=ether1 dst-port=443 protocol=tcp comment="Allow HTTPS from Edge Network"
add chain=input action=drop comment="Drop everything else"

add chain=forward action=accept connection-state=established,related comment="Allow Established Connections"
add chain=forward action=accept in-interface=ether1 out-interface-list=PLC-PORTS comment="Allow Edge to PLCs"
add chain=forward action=drop comment="Drop everything else"

I forgot to mention previously that it's usually best to allow ICMP communication both in the input and forward chains. Both for ping and other reasons.

You do not have to allow port 102 in the input chain (actually you shouldn't.) The dst-nat chain precedes the filter stage of the firewall, so by the time your packets get there they are processed by the forward chain, where your second rule correctly allows them to proceed.

I'm sure we've missed something, but it's time to test it and see why it doesn't work correctly.

EDIT:
Also, I don't know if you have considered this or not, but you have to add a default route for your router. (It doesn't have to actually work, but policy routing in Linux and therefore in RouterOS needs this to function properly. It's lengthy to explain exactly why. But it's better if it actually works, because that way you can do software updates, clock updates, etc. normally.)

lurker888 · Wed Mar 26, 2025 1:36 am

Also please update the software on your device to something recent; I would suggest the latest stable version 7.18.2. Bugs are fixed all the time and devices usually ship with quite ancient versions. Please don't forget that the bootloader/firmware (called RouterBoot) has to be updated as a separate step.

juwalter · Wed Mar 26, 2025 2:44 am

Many thanks again!! will change the "virutal IPs" from /32 to /24 and fixed routing-mark=" -> "routing-table" in section "/ip route"

Also please update the software on your device to something recent; I would suggest the latest stable version 7.18.2. Bugs are fixed all the time and devices usually ship with quite ancient versions. Please don't forget that the bootloader/firmware (called RouterBoot) has to be updated as a separate step.

will do!

Also, I don't know if you have considered this or not, but you have to add a default route for your router. (It doesn't have to actually work, but policy routing in Linux and therefore in RouterOS needs this to function properly. It's lengthy to explain exactly why. But it's better if it actually works, because that way you can do software updates, clock updates, etc. normally.)

will do as well! (good point re NTP/clock - this will help a lot in the logs ...)

I'm sure we've missed something, but it's time to test it and see why it doesn't work correctly.

heh - will be able to do this on Thursday (fingers crossed)

thanks again!!

juwalter · Fri Mar 28, 2025 3:12 am

today I managed to implement the setup! looking good so far, only got this error:

input does not match any value of new-routing-mark

right after this command

add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"

previous commands so far:

# Identity for better management
/system identity set name="PLC-Gateway"

#######################################
# Interface Organization
#######################################

# Create interface lists for better management
/interface list
add name=PLC-PORTS
add name=WAN

# Add interfaces to lists
/interface list member
add interface=ether1 list=WAN
add interface=ether2 list=PLC-PORTS
add interface=ether3 list=PLC-PORTS
add interface=ether4 list=PLC-PORTS

#######################################
# IP Configuration
#######################################

# WAN interface (Edge device connection)
/ip address
add address=172.29.10.1/24 interface=ether1 comment="Edge Network Primary IP"
# Add virtual IPs for each PLC on the WAN interface
add address=172.29.10.101/24 interface=ether1 comment="PLC1 External IP"
add address=172.29.10.102/24 interface=ether1 comment="PLC2 External IP"
add address=172.29.10.103/24 interface=ether1 comment="PLC3 External IP"
# Configure IP addresses for each PLC interface - directly using Ethernet ports
add address=192.168.0.180/24 interface=ether2 comment="PLC1 Network"
add address=192.168.0.180/24 interface=ether3 comment="PLC2 Network"
add address=192.168.0.180/24 interface=ether4 comment="PLC3 Network"

# Mark connections based on the external IP being accessed
/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc3-conn comment="Mark PLC3 Connections"

I did add those:

/routing table
add disabled=no fib name=to-plc1
add disabled=no fib name=to-plc2
add disabled=no fib name=to-plc3

/routing rule
add action=lookup-only-in-table routing-mark=to-plc1 table=to-plc1
add action=lookup-only-in-table routing-mark=to-plc2 table=to-plc2
add action=lookup-only-in-table routing-mark=to-plc3 table=to-plc3

after that, I could continue successfully with those:

# Convert connection marks to routing marks for policy routing - ONLY for packets coming from WAN
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"
add chain=prerouting in-interface=ether1 connection-mark=plc2-conn action=mark-routing new-routing-mark=to-plc2 comment="Mark route to PLC2"
add chain=prerouting in-interface=ether1 connection-mark=plc3-conn action=mark-routing new-routing-mark=to-plc3 comment="Mark route to PLC3"

and the rest:

# Policy routes for the entire PLC subnet (/24), not just the PLC (/32)
/ip route
add dst-address=192.168.0.0/24 gateway=ether2 routing-table=to-plc1 comment="Route to PLC1"
add dst-address=192.168.0.0/24 gateway=ether3 routing-table=to-plc2 comment="Route to PLC2"
add dst-address=192.168.0.0/24 gateway=ether4 routing-table=to-plc3 comment="Route to PLC3"

# Destination NAT
/ip firewall nat
add chain=dstnat in-interface=ether1 dst-address=172.29.10.101 action=dst-nat to-addresses=192.168.0.2 comment="PLC1 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.102 action=dst-nat to-addresses=192.168.0.2 comment="PLC2 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.103 action=dst-nat to-addresses=192.168.0.2 comment="PLC3 Inbound"

# Source NAT for replies (makes router appear as 192.168.0.180 to PLCs)
add chain=srcnat out-interface-list=PLC-PORTS action=src-nat to-addresses=192.168.0.180 comment="Source NAT to PLCs"

However, when I try the connection to the any of the PLC (web traffic) I am getting "bounced around" seemingly random to any of the PLC, instead of deterministically to the PLC matching the external/virtual IP address. So, if I do "curl http://172.29.10.101" and "curl http://172.29.10.102" and so on, I can see the randomly (after restarting router and PLC) web pages from sometimes wrong PLC.

Any idea what could cause this?

Many thanks!! Jürgen

juwalter · Fri Mar 28, 2025 3:24 am

duh, good old copy/paste! wrong (last line):

/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc3-conn comment="Mark PLC3 Connections"

fixed:

/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.103 action=mark-connection new-connection-mark=plc3-conn comment="Mark PLC3 Connections"

now it seems to work fine!!

btw- is it worthwhile to set proxy-arp?

/interface ethernet set ether2 arp=proxy-arp
/interface ethernet set ether3 arp=proxy-arp
/interface ethernet set ether4 arp=proxy-arp

and is "connection tracking" enabled by default, or should I set it?

/ip firewall connection tracking set enabled=yes

lurker888 · Fri Mar 28, 2025 3:56 am

Good to hear.

So... in order:

You have to add the tables manually, and the routing marks are created by the table. I missed that part.

The routing rules that you added are not strictly necessary. Without the rules, the behavior would be very similar, only with the action "lookup" instead of "lookup-only-in-table". For this use case I always specify the latter, as you did - I just think it's better. (It would work correctly with the default action as well.)

I'm glad that only this and the copy-paste stuff was all there was to it.

As to proxy-arp. In this case enabling it would have no effect. It's basically a workaround - even if a common one, and so if not needed it shouldn't be enabled.

The use case for proxy-arp is the following scenario.

Hosts, when they send packets over ethernet must address it using the MAC address of the device they are transmitting to. This means that if the address is in a directly connected (on-link) subnet then they do an ARP query and send the packet directly to its destination. If the destination is not directly connected, then they look up the appropriate gateway, get its MAC address via ARP and transmit the packet to it. As long as you follow the basic principles of network architecture this all works out correctly.

Proxy arp comes into play when you lie about where the destination actually is. You present it as being in the same ethernet segment, but in reality you want the route (not bridge/switch) packets to it. In this case an attempt at ARP will fail, and the packet will be dropped with some variation of "destination unreachable". In these instances the router must "capture" the packet, and to do so it is instructed to respond to the ARP with its own MAC address. This is proxy arp.

In your case the packet is already addressed (by IP) to your router (both on the 172.* subnet and the PLC subnets, because of the src-nat to 192.168.0.180), so ARP proceeds normally. You could also say that the router actually possesses these addresses on the given subnet (meaning the ones you called "virtual".) The packet's addressing once received is then modified (and replies translated back, etc.) but this has no impact on ARP behavior, which is performed as usual.

You're basically done. Go ahead with adding the firewall rules according to your liking.

EDIT:
It's also fine to leave connection tracking on "auto". It will be on anyway. This setting is basically a remnant of the time when the firewall raw table was not exposed in the interface (and that was a really long time ago.)

lurker888 · Fri Mar 28, 2025 4:02 am

This forum also has a section named "Useful user articles". This thing comes up on the forum from time to time - maybe consider doing a write-up...

juwalter · Fri Mar 28, 2025 7:33 pm

Hello lurker888! Many thanks for the detailed write-up and background on "proxy-arp" - very good! Now its clear to me that there is no need to include it.

re: write-up - in fact I was planning on doing this on the weekend (had a summary as "last" post in this thread in mind - but a dedicated section is much better yet!)

btw- the final configuration works well, tested extensively over 12h or so. Only one thing came up (not critical, but my OCD kind of pushes me to somehow get it solved).

When ping a "ping 192.168.0.180" from any of the PLC back to the router it only works for one PLC; when any of the others try to ping there is no reply. I suspect it is because the router always responds to the same PLC. I did some more digging and back-and-forth with various LLM and the only suggestions that came up where to not use a single IP 192.168.0.180 on interfaces ether2, ether3, ether4, etc. but each one a dedicated one.

# Configure same IP address on each PLC interface - directly using Ethernet ports
add address=192.168.0.180/24 interface=ether2 comment="PLC1 Network"
add address=192.168.0.180/24 interface=ether3 comment="PLC2 Network"
add address=192.168.0.180/24 interface=ether4 comment="PLC3 Network"

I wonder if it is possible to somehow mark the "incoming" traffic (on the interface-level) so that the router knows where to send back the ping from each PLC. Do you have any ideas do to that in an idiomatic way?

Many thanks again!! Jürgen

current config for 3x PLC

# Identity for better management
/system identity set name="PLC-Gateway"

#######################################
# Interface Organization
#######################################

# Create interface lists for better management
/interface list
add name=PLC-PORTS
add name=WAN

# Add interfaces to lists
/interface list member
add interface=ether1 list=WAN
add interface=ether2 list=PLC-PORTS
add interface=ether3 list=PLC-PORTS
add interface=ether4 list=PLC-PORTS

#######################################
# IP Configuration
#######################################

# WAN interface (Edge device connection)
/ip address
add address=172.29.10.1/24 interface=ether1 comment="Edge Network Primary IP"
# Add virtual IPs for each PLC on the WAN interface
add address=172.29.10.101/24 interface=ether1 comment="PLC1 External IP"
add address=172.29.10.102/24 interface=ether1 comment="PLC2 External IP"
add address=172.29.10.103/24 interface=ether1 comment="PLC3 External IP"
# Configure same IP address on each PLC interface - directly using Ethernet ports
add address=192.168.0.180/24 interface=ether2 comment="PLC1 Network"
add address=192.168.0.180/24 interface=ether3 comment="PLC2 Network"
add address=192.168.0.180/24 interface=ether4 comment="PLC3 Network"

#######################################
# Create Routing Tables
#######################################

/routing table
add disabled=no fib name=to-plc1
add disabled=no fib name=to-plc2
add disabled=no fib name=to-plc3

#######################################
# Create Routing Rules
#######################################

/routing rule
add action=lookup-only-in-table routing-mark=to-plc1 table=to-plc1
add action=lookup-only-in-table routing-mark=to-plc2 table=to-plc2
add action=lookup-only-in-table routing-mark=to-plc3 table=to-plc3

#######################################
# Connection Marking for Policy Routing
#######################################

# Mark connections based on the external IP being accessed
/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.103 action=mark-connection new-connection-mark=plc3-conn comment="Mark PLC3 Connections"

# Convert connection marks to routing marks for policy routing - ONLY for packets coming from WAN
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"
add chain=prerouting in-interface=ether1 connection-mark=plc2-conn action=mark-routing new-routing-mark=to-plc2 comment="Mark route to PLC2"
add chain=prerouting in-interface=ether1 connection-mark=plc3-conn action=mark-routing new-routing-mark=to-plc3 comment="Mark route to PLC3"

#######################################
# Policy-Based Routing Rules
#######################################

# Policy routes for the entire PLC subnet (/24), not just the PLC (/32)
/ip route
add dst-address=192.168.0.0/24 gateway=ether2 routing-table=to-plc1 comment="Route to PLC1"
add dst-address=192.168.0.0/24 gateway=ether3 routing-table=to-plc2 comment="Route to PLC2"
add dst-address=192.168.0.0/24 gateway=ether4 routing-table=to-plc3 comment="Route to PLC3"

#######################################
# NAT Configuration - Inbound & Reply Traffic
#######################################

# Destination NAT
/ip firewall nat
add chain=dstnat in-interface=ether1 dst-address=172.29.10.101 action=dst-nat to-addresses=192.168.0.2 comment="PLC1 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.102 action=dst-nat to-addresses=192.168.0.2 comment="PLC2 Inbound"
add chain=dstnat in-interface=ether1 dst-address=172.29.10.103 action=dst-nat to-addresses=192.168.0.2 comment="PLC3 Inbound"

# Source NAT for replies (makes router appear as 192.168.0.180 to PLCs)
add chain=srcnat out-interface-list=PLC-PORTS action=src-nat to-addresses=192.168.0.180 comment="Source NAT to PLCs"

# might be default but just in case
/ip firewall connection tracking set enabled=yes

lurker888 · Fri Mar 28, 2025 10:10 pm

With your current setup the ping is supposed to work (or not work) as you describe. This does not indicate that you didn't accomplish what you set out to do, or that somehow it's wrong; this is just not implemented. Most people are happy to get this far, and accomplish the bare minimum, so really just want to sit back and be done with it.

Of course what you want can be done, and it's not that hard. In order to make this function some observations have to be made first.

First of all, you currently attach connection marks to the connections initiated on the non-PLC side of the network. Connection marks differ from routing marks in that the routing marks are discarded when the packet is processed, however the connection marks persist for the life of the connection and all packets (both direct and reply) of the connection are automatically labeled by the conntrack mechanism. (Packet marks - that you don't currently use - behave the same way as routing marks.)

Your mangle rules for a connection coming from ether1 look like this:

add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"

Here the connection mark is not needed, because all the information used to select the correct routing table is available on a per-packet basis. These rules can easily (and more succinctly) be written as:

add chain=prerouting dst-address=172.29.10.101 action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"

We'll see that this is a bit better. (Not that the original rule didn't accomplish the correct thing, rather in the sense that this formulation will be more useful to us going forward.)

This of course solves nothing about the pings. I think we can agree that the pings reach the router successfully, so there's nothing to be done/fixed there. It's the return packets that only reach one of your PLCs. This is not in the least unexpected, because no routing marks are attached to these packets, and therefore the "main" routing table is used for route lookup, which obviously has conflicting entries for the PLC addresses. That one is chosen (preferred is you will) is as good an implementation in the router software as any - it really has no idea where these are intended to go to. There are two things to be aware of:
1. The response packets have no information in the packets themselves that would allow us to make a decision with regard to their intended destination. (They are emitted by the router, which is of no use, and addressed to 192.168.0.2, which similarly bears no useful information.) In these cases connection marks are actually our friends. (Or more generally, connection marks are useful in policy routing setups to maintain information regarding who *initiated* the connection originally. Who they're addressed to is right there in the packet header as dst-address.)
2. Also, packets coming from the router don't traverse the prerouting mangle chain, but instead use the prerouting output. For these sorts of setups it's almost mandatory to study the packet flow diagram. A version of this can found at (just disregard the boxes labelled "security" - those are an SELinux thing)
https://stuffphilwrites.com/fw-ids-ipta ... 024-05-22/
Mikrotik has its own version at (this is more detailed, but I generally find it a bit less useful)
https://help.mikrotik.com/docs/spaces/R ... n+RouterOS

So in order for the pings to work, we have to add connection marks and appropriate routing marks. We also mark the connection initiated from the non-plc side as from-mgmt. Together with only marking as yet unmarked connections, this is useful to maintain the "who initiated" sense of connection marks.

add chain=prerouting connection-mark=no-mark in-interface=ether1 action=mark-connection new-connection-mark=from-mgmt comment="Mark connection from MGMT side"
add chain=prerouting connection-mark=no-mark in-interface=ether2 action=mark-connection new-connection-mark=from-plc1 comment="Mark connection from PLC1"

Now we can select the appropriate routing marks for the ping response packets:

add chain=output connection-mark=from-plc1 action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"

Now pinging the router from the PLCs will work.

juwalter · Fri Apr 04, 2025 3:38 am

Hello again lurker888,

finally managed to get to the writeup - I don't want to use more of your time, but feel free to have a look viewtopic.php?t=215953 - if you have any feedback for me, I am happy to edit!!

Many thanks again for helping us on this one!! Jürgen

lurker888 · Fri Apr 04, 2025 9:43 am

Glad it's working, and a great writeup! Now I will know where to refer people asking for this. It comes up several times a month.

The following part should be however corrected.

second one has connection and routing marks on traffic coming from the PLCs to the router.

/ip firewall mangle
# Mark connections based on the external IP being accessed
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.103 action=mark-connection new-connection-mark=plc2-conn comment="Mark PLC2 Connections"
add chain=prerouting in-interface=ether1 dst-address=172.29.10.104 action=mark-connection new-connection-mark=plc3-conn comment="Mark PLC3 Connections"

# Convert connection marks to routing marks for policy routing - ONLY for packets coming from WAN
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"
add chain=prerouting in-interface=ether1 connection-mark=plc2-conn action=mark-routing new-routing-mark=to-plc2 comment="Mark route to PLC2"
add chain=prerouting in-interface=ether1 connection-mark=plc3-conn action=mark-routing new-routing-mark=to-plc3 comment="Mark route to PLC3"

# Mark connections for return traffic from the PLCs to the WAN/Router
add chain=prerouting in-interface=ether1 connection-mark=no-mark action=mark-connection new-connection-mark=from-mgmt comment="Mark connection from MGMT side"
add chain=prerouting in-interface=ether2 connection-mark=no-mark action=mark-connection new-connection-mark=from-plc1 comment="Mark connection from PLC1"
add chain=prerouting in-interface=ether3 connection-mark=no-mark action=mark-connection new-connection-mark=from-plc2 comment="Mark connection from PLC2"
add chain=prerouting in-interface=ether4 connection-mark=no-mark action=mark-connection new-connection-mark=from-plc3 comment="Mark connection from PLC3"

add chain=output connection-mark=from-plc1 action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"
add chain=output connection-mark=from-plc2 action=mark-routing new-routing-mark=to-plc2 comment="Mark route to PLC2"
add chain=output connection-mark=from-plc3 action=mark-routing new-routing-mark=to-plc3 comment="Mark route to PLC3"

A connection can have only one connection mark, and in these scenarios we use it to mark the *initiator* of the connection (hence the connection-mark=no-mark in the marking rules). The connections initiated on the WAN side get marked as from-mgmt. Marking the connections as to-plcX defeats this. It's unnecessary and should not be done.

The rules

add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"

should be condensed to one rule that directly applies the routing mark

add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"

The solution as presented will still work in your use case, so it's technically correct. The problem comes when you try to extend this to the case where the PLCs want to reach out to a host on the WAN side. (It's not an uncommon next request that when PLC1 initiates a connection to 192.168.2.180, it should connect to 172.29.10.xxx, and the connection should be seen as coming from 172.29.10.102.)

If I drink too much coffee, I may even contribute to your article with a VRF-based approach to the same problem. I like it more, but it always gets brushed aside when I suggest it on this forum

juwalter · Mon Apr 07, 2025 2:32 am

Hi lurker888,

thanks again(!) for the review! So, if I understand correctly, two concerns:

1. redundancy / over-complication by using first a connection-mark and then converting to routing-mark
2. not only redundancy but potential problem due to conflict: same traffic (e.g. "chain=prerouting in-interface=ether1") will/might receive 2x connection mark: new-connection-mark=plc1-conn and new-connection-mark=from-mgmt

To fix this, simply collapsing the redundant rules into one would fix both issues?

# before
/ip firewall mangle
add chain=prerouting in-interface=ether1 dst-address=172.29.10.101 action=mark-connection new-connection-mark=plc1-conn comment="Mark PLC1 Connections"
add chain=prerouting in-interface=ether1 connection-mark=plc1-conn action=mark-routing new-routing-mark=to-plc1 comment="Mark route to PLC1"

# after
add chain=prerouting in-interface=ether1 dst-address=172.29.10.102 action=mark-routing new-routing-mark=to-plc1 comment="Route to PLC1"

(already changed in the guide post).

Thanks you!! Jürgen

P.S. if you really "overdose" on that coffee, I am happy to test this VRF configuration

but really no pressure!! I just tested the final configuration from the write-up and it seems fine!

lurker888 · Mon Apr 07, 2025 2:48 am

Exactly. The redundancy I don't mind so much, but a connection can only carry one mark. (In this case it doesn't cause any trouble though.)

As to the VRF version: I have a few of these projects on my list. Sooner or later I'll get around to it. It would of course be very welcome if you'd test it out.

oliverfriedmann · Sun Apr 13, 2025 8:03 am

FWIW - I wrapped this into a templated script "config.template":

<%
    systemName = "PLC-Gateway"
    interfaceDevices = "PLC-PORTS"
    interfaceWAN = "WAN"
    interfaceForWAN = "ether1"
    interfaceForDevices = ["ether2", "ether3", "ether4"]
    primaryIP = "172.29.10.1"
    externalIPs = ["172.29.10.102", "172.29.10.103", "172.29.10.104"]
    internalIP = "192.168.2.180"
    toIP = "192.168.2.20"
    dstIP = "192.168.2.0"
%>

## /export
## /export verbose
## /system reboot
## /system reset-configuration

# Identity for better management
/system identity set name="<%= systemName %>"

#######################################
# Interface Organization
#######################################

# Create interface lists for better management
/interface list
add name=<%= interfaceDevices %>
add name=<%= interfaceWAN %>

# Add interfaces to lists
/interface list member
add interface=<%= interfaceForWAN %> list=<%= interfaceWAN %>
<% interfaceForDevices.forEach(function (device) { %>add interface=<%= device %> list=<%= interfaceDevices %>
<% }) %>

#######################################
# IP Configuration
#######################################

# WAN interface (Edge device connection)
/ip address
add address=<%= primaryIP %>/24 interface=<%= interfaceForWAN %> comment="Edge Network Primary IP"
# Add virtual IPs for each PLC on the WAN interface
<% externalIPs.forEach(function (ip, i) { %>add address=<%= ip %>/24 interface=<%= interfaceForWAN %> comment="Idx <%= i %> External IP"
<% }) %>
# Configure IP addresses for each device interface - directly using Ethernet ports
<% interfaceForDevices.forEach(function (device, i) { %>add address=<%= internalIP %>/24 interface=<%= device %> comment="Idx <%= i %> Network"
<% }) %>

#######################################
# Create Routing Tables
#######################################

/routing table
<% externalIPs.forEach(function (_, i) { %>add disabled=no fib name=to-idx<%= i %>
<% }) %>

#######################################
# Create Routing Rules
#######################################

/routing rule
<% externalIPs.forEach(function (_, i) { %>add action=lookup-only-in-table routing-mark=to-idx<%= i %> table=to-idx<%= i %>
<% }) %>

#######################################
# Connection Marking for Policy Routing
#######################################

/ip firewall mangle
# Mark inbound traffic from WAN to devices with routing marks directly
<% externalIPs.forEach(function (ip, i) { %>add chain=prerouting in-interface=<%= interfaceForWAN %> dst-address=<%= ip %> action=mark-routing new-routing-mark=to-idx<%= i %> comment="Route to Idx <%= i %>"
<% }) %>

# Mark connections based on initiator
add chain=prerouting connection-mark=no-mark in-interface=<%= interfaceForWAN %> action=mark-connection new-connection-mark=from-mgmt comment="Mark connection from MGMT side"
<% interfaceForDevices.forEach(function (device, i) { %>add chain=prerouting connection-mark=no-mark in-interface=<%= device %> action=mark-connection new-connection-mark=from-idx<%= i %> comment="Mark connection from device <%= i %>"
<% }) %>

# Route return traffic from router to devices (where connection-mark=from-idx)
<% interfaceForDevices.forEach(function (_, i) { %>add chain=output connection-mark=from-idx<%= i %> action=mark-routing new-routing-mark=to-idx<%= i %> comment="Route to idx <%= i %>"
<% }) %>

#######################################
# Policy-Based Routing Rules
#######################################

# Policy routes for the entire device subnet (/24), not just the device (/32)
/ip route
<% interfaceForDevices.forEach(function (device, i) { %>add dst-address=<%= dstIP %>/24 gateway=<%= device %> routing-table=to-idx<%= i %> comment="Route to idx <%= i %>"
<% }) %>

#######################################
# NAT Configuration - Inbound & Reply Traffic
#######################################

# Destination NAT
/ip firewall nat
<% externalIPs.forEach(function (ip, i) { %>add chain=dstnat in-interface=<%= interfaceForWAN %> dst-address=<%= ip %> action=dst-nat to-addresses=<%= toIP %> comment="Idx <%= i %> Inbound"
<% }) %>

# Source NAT for replies (makes router appear as <%= internalIP %> to devices)
add chain=srcnat out-interface-list=<%= interfaceDevices %> action=src-nat to-addresses=<%= internalIP %> comment="Source NAT to devices"

# might be default but just in case
/ip firewall connection tracking set enabled=yes

To execute (meaning: replace template code with actual values):

npm install -g ufpp
cat config.template | ufpp -t

juwalter · Tue Apr 15, 2025 1:21 pm

thanks oliverfriedmann - sounds like a great idea to template this. when researching ufpp it looked like the version on npm is pretty outdated and also points to a defunct repo. also noticed you have this repo on GH https://github.com/Jsonize/universal-file-preprocessor - just wondering if it would make sense to synchronise your GH repo with the npm project? Right now, it could raise some concerns re: software supply chain attacks due to some rogue npm package. wdyt?

1:1 NAT / DNAT configuration help

1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help

Re: 1:1 NAT / DNAT configuration help