I have a 'multi-tenant' network with multiple private networks, each with their own public IP addresses. Each network can't access anything in the others, with some exceptions in the form of 'hole punching'. We have this all automated via some Terraform configuration, where we define a list of networks and hole-punches, and it creates all of the firewall rules, VLANs, NAT rules, DHCP server configurations, Wireguard configurations, etc.
We want to add support for hairpin NAT to this configuration, such that if a user on network A connects to the public IP of network B, the connection gets NATed as if it was originally from the wider internet, just with a source IP of network A's public IP. For example, if I have the following configuration (note that our actual config is much more complex and has, if I remember correctly, eight networks at the moment):
Network A
Internal range of 172.20.1.0/24, router at .1
External IP of aaa.bbb.ccc.1
Internal hosts
node-a1 on 172.20.1.101
node-a2 on 172.20.1.102
Network B
Internal range of 172.20.2.0/24, router at .1
External IP of aaa.bbb.ccc.2, port 80/443 routed to 172.20.2.101
Internal hosts:
node-b1 on 172.20.2.101
node-b2 on 172.20.2.102
All hosts from network A can connect to node-b2 on network B
What we want is this:
node-a1 cannot directly connect to node-b1 (even on publicly exposed ports)
node-a1can connect to aaa.bbb.ccc.2 on ports 80/443
node-b1 gets the connection, sees the source IP as aaa.bbb.ccc.1
node-a1 and node-a2 can connect to 172.20.2.102
node-b1 gets the connection, sees the source IP as 172.20.1.10x
What would the best way to accomplish this configuration be? If it helps, I have the Terraform configuration over at https://gitlab.com/kbity/terraform, minus the 'private' parts of the config (such as the exact networks used, although there is a mostly-up-to-date sample there that shows a slightly simplified version of the configuration).
Here you probably meant node-b2 instead of node-b1? If that's the case, this might be a possible firewall configuration:
# (0) Let's use address lists to have fewer properties to hard-code
/ip firewall address-list
add list=NETWORK-A address=172.20.1.0/24
add list=NODE-A1 address=172.20.1.101
add list=NODE-A2 address=172.20.1.102
add list=NETWORK-B address=172.20.2.0/24
add list=NODE-B1 address=172.20.2.101
add list=NODE-B2 address=172.20.2.102
# (1) External IP of aaa.bbb.ccc.2, port 80/443 routed to 172.20.2.101
# Note: 172.20.2.101 is hardcoded here, address list cannot be used :(
/ip firewall nat
add chain=dstnat action=dst-nat dst-address=aaa.bbb.ccc.2 \
protocol=tcp dst-port=80,443 to-addresses=172.20.2.101
# (2) All hosts from network A can connect to node-b2 on network B
/ip firewall filter
add chain=forward action=accept \
src-address-list=NETWORK-A dst-address-list=NODE-B2
# (3) node-a1 can connect to node-b1 if DSTNAT rule (1) was applied
/ip firewall filter
add chain=forward action=accept connection-nat-state=dstnat \
src-address-list=NODE-A1 dst-address-list=NODE-B1
# (3b) Optional: repeat for node-a2 if needed, if not, skip this rule
/ip firewall filter
add chain=forward action=accept connection-nat-state=dstnat \
src-address-list=NODE-A2 dst-address-list=NODE-B1
# (4) Network A otherwise has no access to node-b1, except for what
# was allowed by (3) and optionally (3b). Rules' order is important!
# Note: Can also consider dst-address-list=NETWORK-B to be stricter.
/ip firewall filter
add chain=forward action=drop \
src-address-list=NETWORK-A dst-address-list=NODE-B1
# (5) node-b1 gets the connection, sees the source IP as aaa.bbb.ccc.1
# Note: due to (1) (3) (4) this rule will only sees connections from
# node-a1 (optionally node-a2 too, if rule (3b) was added) to external
# address aaa.bbb.ccc.2 on TCP port 80 or 443, and no other connections.
# Thus, we don't need to repeat too many conditions in this rule!
/ip firewall nat
add chain=srcnat action=src-nat src-address-list=NETWORK-A \
dst-address-list=NODE-B1 to-addresses=aaa.bbb.ccc.1