Make 2 devices with same IP addressable through a MikroTik Router

Hello,

I am experiencing some difficulties with a router config, and I am asking for help.
I am quite new to MikroTik and RouterOS. I bought my router 2 weeks ago.

TL;DR
I cannot manage to srcNAT and dstNAT to 2 devices with same IP, that are on 2 VLANs on the same port of the router.

Edit : solution
Be careful with fasttrack firewall rules (from default ruleset) and with dynamic routes, when playing with complicated source NAT and destination NAT.

The Context
I am working with industrial devices that have a network port for debug usage. Each device has an identical IP configuration (addr 10.0.0.1/24, gateway 10.0.0.2).
I do not want to change it, since this configuration is the target when the device will be deployed. Anyway, this is a debug port, there are not supposed to pe publicly accessible.
My company is writing the software inside the device.

The Use Case
For testing purposes, I would like to make many devices accessible from a LAN at the same time. There will be ~ 80 devices.

The Plan
I plan to expose a public external address for each device, and use a source NAT and a destination NAT in relation to the port where the device is plugged.
I have bought a RB760iGS (MikroTik Hex S) and a smart managed switch (Netgear GS110TPP).

A small trap for some fun : :wink:
I have plenty of devices : they will not fit a 10-port MikroTik router. I have created VLAN-interfaces and configured my switch accordingly.

Here is my plan :

                    ____________________                                    _____________
                   |                    |                                  |             |
                   |      RB760iGS      |                                  |     VLAN 31 | --- 10.0.0.1 - device 1
                   |                    |                                  |             |
     ---- LAN ---- | ether1             |                                  |     VLAN 31 | --- 10.0.0.1 - device 2
     192.168.1.250 |                    |                                  |             |
+ .251 for device1 |     VLAN_31/ether5 | --- 10.0.0.2 --- |               |     VLAN X  | --- 10.0.0.1 - device X
+ .252 for device2 |     VLAN_32/ether5 | --- 10.0.0.2 --- |               |             |
                   |     ....           |                  | --- trunk --- | trunk       |
                   |     VLAN_XX/ether5 | --- 10.0.0.2 --- |               |_____________|
                   |                    | 
                   |____________________|

Similar solutions from other users
I have read solutions from these topics :
http://forum.mikrotik.com/t/port-to-interfacing/97096/1 and https://gist.github.com/0x4C4A/3ba83e2e26cddd75c35e2bf6ee6e60d4
http://forum.mikrotik.com/t/how-to-allow-two-devices-with-same-ip-access-internet/115990/1
http://forum.mikrotik.com/t/access-multiple-devices-with-same-ip-address/106788/1
I cannot manage to make them work.
I have tried some changes (mark-packet instead of mark-connection -see below-, disabling any masquerade…)

The Router Configuration
Let’s start with 2 devices only. Here is my configuration :

/ip address
add address=192.168.1.251/24 interface=ether1 network=192.168.1.0
add address=192.168.1.252/24 interface=ether1 network=192.168.1.0
add address=10.0.0.2/24 interface=VLAN_K1 network=10.0.0.0
add address=10.0.0.2/24 interface=VLAN_K2 network=10.0.0.0
# + a dynamic address 192.168.1.250 on ether1 given by DHCP. This is my management address.

/interface vlan
add interface=ether5 name=VLAN_K1 vlan-id=31
add interface=ether5 name=VLAN_K2 vlan-id=32
               
/ip firewall nat
add action=dst-nat chain=dstnat dst-address=192.168.1.251 to-addresses=10.0.0.1
add action=dst-nat chain=dstnat dst-address=192.168.1.252 to-addresses=10.0.0.1
add action=src-nat chain=srcnat out-interface=ether1 packet-mark=VLAN_K1 src-address=10.0.0.1 to-addresses=192.168.1.251
add action=src-nat chain=srcnat out-interface=ether1 packet-mark=VLAN_K2 src-address=10.0.0.1 to-addresses=192.168.1.252

/ip firewall mangle
add action=mark-packet chain=prerouting dst-address=192.168.1.251 in-interface=ether1 new-packet-mark=VLAN_K1 passthrough=yes
add action=mark-packet chain=prerouting dst-address=192.168.1.252 in-interface=ether1 new-packet-mark=VLAN_K2 passthrough=yes
add action=mark-packet chain=prerouting in-interface=VLAN_K1 new-packet-mark=VLAN_K1 passthrough=yes
add action=mark-packet chain=prerouting in-interface=VLAN_K2 new-packet-mark=VLAN_K2 passthrough=yes
add action=mark-routing chain=prerouting dst-address=192.168.1.251 new-routing-mark=VLAN_K1 passthrough=no
add action=mark-routing chain=prerouting new-routing-mark=VLAN_K1 packet-mark=VLAN_K1 passthrough=no
add action=mark-routing chain=prerouting dst-address=192.168.1.252 new-routing-mark=VLAN_K2 passthrough=no
add action=mark-routing chain=prerouting new-routing-mark=VLAN_K2 packet-mark=VLAN_K2 passthrough=no

/ip route
add distance=1 dst-address=10.0.0.0/24 gateway=VLAN_K1 routing-mark=VLAN_K1
add distance=1 dst-address=10.0.0.0/24 gateway=VLAN_K2 routing-mark=VLAN_K2

I am running the latest stable RouterOS v6.48.2

The Problem
The point is that, it seems to work… when I connect to the first device. But when I connects to a second device, it leads to random connection failures.
It looks like the router was trying to kill multiple/concurrent/unused connections, as if NAT-masquerade cleanup routine was performed.

The Strange Workaround
I tried to debug with the Packet sniffer tool. When I enable it, the connections are magically valid and everything works great !
It even works when I changed the rules of the sniffer, to make it capture nothing (For example, by changing the sniffed port to an unused port).
But this is not a true workaround if I do not understant why this does work… :-/

The Clues
The VLANs are working correctly, and the router have detected each device on its port :

[admin@MikroTik] > /ip arp print
Flags: X - disabled, I - invalid, H - DHCP, D - dynamic, P - published, C - complete
 #    ADDRESS         MAC-ADDRESS       INTERFACE
 0 DC 192.168.1.1     xx:xx:xx:xx:xx:xx ether1      <- my gateway on my LAN
 1 DC 192.168.1.18    yy:yy:yy:yy:yy:yy ether1      <- my PC
...
 7 DC 10.0.0.1        B8:27:EB:AB:xx:xx VLAN_K1     <- device 1, mocked with a RaspberryPi
 8 DC 10.0.0.1        B8:27:EB:10:xx:xx VLAN_K2     <- device 2

The Questions
Does anybody have an idea of what is missing in my router configuration ?
Why does the Packet Sniffer tool make it work?
Will there be any overhead if I let this (workaround) tool enabled, even if I make it capture nothing (no matching rule)?

Thank you very much for reading this loooong question. :slight_smile:
Kind regards,
K.B.

Hi,

I would remove addresses 192.168.1.251 and 192.168.1.252 from ehter1. I don’t think they are needed.

I would specify packet and routing marks more clearly. Like this:

add action=mark-packet chain=prerouting dst-address=192.168.1.251 in-interface=ether1 new-packet-mark=VLAN_K1-pm-out passthrough=yes
add action=mark-packet chain=prerouting in-interface=VLAN_K1 new-packet-mark=VLAN_K1-pm-in passthrough=yes
add action=mark-routing chain=prerouting dst-address=192.168.1.251 new-routing-mark=VLAN_K1-rm-out passthrough=no packet-mark=VLAN_K1-pm-out

Please remove the last mangle rule for VLAN_K1 because its a doublicate of the previous mangle rule.
Change NAT rule accordingly.

Can you please post configuration in “/ip route”?

Hi DeJoe,

Thank you for your reply.

I do need the adresses 192.168.1.251 and 192.168.1.252. I do not want to forward some ports from 192.168.1.250 (router’s public IP) to VLAN_K1 (device1) or VLAN_K2 (device 2).
I want to expose device1 as 192.168.1.251 on public network. And so does device2 as 192.168.1.252. The dst-address determines the target VLAN.

You are right, I forgot to mention the IP routes. I have just added them to the initial post.

I have changed my configuration according to your answer. I have also added some comments :

/interface vlan
add interface=ether5 name=VLAN_K1 vlan-id=31
add interface=ether5 name=VLAN_K2 vlan-id=32

/ip address
# IP adresses on public network for devices
add address=192.168.1.251/24 interface=ether1 network=192.168.1.0
add address=192.168.1.252/24 interface=ether1 network=192.168.1.0
# Gateway addresses on VLANs
add address=10.0.0.2/24 interface=VLAN_K1 network=10.0.0.0
add address=10.0.0.2/24 interface=VLAN_K2 network=10.0.0.0
# + a dynamic address 192.168.1.250 on ether1 given by DHCP. This is my management address.


/ip firewall mangle
# Device1, public->VLAN #1, mark packet  
add action=mark-packet chain=prerouting dst-address=192.168.1.251 in-interface=ether1 new-packet-mark=VLAN_K1-pm-incoming passthrough=yes
# Device1, public->VLAN #1, copy packet mark to routing mark (for later routing)
add action=mark-routing chain=prerouting packet-mark=VLAN_K1-pm-incoming new-routing-mark=VLAN_K1-rm-incoming passthrough=no
# Device1, VLAN #1->public, mark packet 
add action=mark-packet chain=prerouting in-interface=VLAN_K1 new-packet-mark=VLAN_K1-pm-outgoing passthrough=yes

# Device2, public->VLAN #2, mark packet  
add action=mark-packet chain=prerouting dst-address=192.168.1.252 in-interface=ether1 new-packet-mark=VLAN_K2-pm-incoming passthrough=yes
# Device2, public->VLAN #2, copy packet mark to routing mark (for later routing)
add action=mark-routing chain=prerouting packet-mark=VLAN_K2-pm-incoming new-routing-mark=VLAN_K2-rm-incoming passthrough=no
# Device2, VLAN #2->public, mark packet 
add action=mark-packet chain=prerouting in-interface=VLAN_K2 new-packet-mark=VLAN_K2-pm-outgoing passthrough=yes

               
/ip firewall nat
# dst-nat, public->private
add action=dst-nat chain=dstnat dst-address=192.168.1.251 to-addresses=10.0.0.1
add action=dst-nat chain=dstnat dst-address=192.168.1.252 to-addresses=10.0.0.1
# src-nat, outgoing packet mark -> correct public IP
add action=src-nat chain=srcnat out-interface=ether1 packet-mark=VLAN_K1-pm-outgoing src-address=10.0.0.1 to-addresses=192.168.1.251
add action=src-nat chain=srcnat out-interface=ether1 packet-mark=VLAN_K2-pm-outgoing src-address=10.0.0.1 to-addresses=192.168.1.252


/ip route
# Route packet marked for VLAN_K1/K2 to gateway VLAN_K1/K2
add distance=1 dst-address=10.0.0.0/24 gateway=VLAN_K1 routing-mark=VLAN_K1-rm-incoming
add distance=1 dst-address=10.0.0.0/24 gateway=VLAN_K2 routing-mark=VLAN_K2-rm-incoming

However the beaviour is the same :

  • ssh’ing to device 1 is fine
  • ssh’ing to device 2 fails after password challenge, and drops connection 1

Enabling Packet Sniffer tool make it works.

I answer to your question about the ip route configuration.

I have just realised that the router added a Dynamic route to the devices’ subnet (rule #3). It is a runtime information, that is not visible in the configuration (ip route print, not ip route export).

[admin@MikroTik] /ip route> print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  10.0.0.0/24                        VLAN_K1                   1
 1 A S  10.0.0.0/24                        VLAN_K2                   1
 2 ADS  0.0.0.0/0                          192.168.1.1               1
 3 ADC  10.0.0.0/24        10.0.0.2        VLAN_K1                   0
                                           VLAN_K2
 4 ADC  192.168.1.0/24     192.168.1.251   ether1                    0

Is there any chance that this dynamic 0-distance rule does some kind of load-balancing between my two VLANs?

I did a test. In order to remove this dynamic rule to 10.0.0.0/24, I changed the net mask for the 10.0.0.0 addresses :

[admin@MikroTik] > /ip address print
Flags: X - disabled, I - invalid, D - dynamic
 #   ADDRESS            NETWORK         INTERFACE
 0   192.168.1.251/24   192.168.1.0     ether1
 1   192.168.1.252/24   192.168.1.0     ether1
 2   10.0.0.2/32        10.0.0.0        VLAN_K1
 3   10.0.0.2/32        10.0.0.0        VLAN_K2
 4 D 192.168.1.250/24   192.168.1.0     ether1

Now the dynamic rule will never match, when routing to 10.0.0.1 :

[admin@MikroTik] > /ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  10.0.0.0/24                        VLAN_K1                   1
 1 A S  10.0.0.0/24                        VLAN_K2                   1
 2 ADS  0.0.0.0/0                          192.168.1.1               1
 3 ADC  10.0.0.0/32        10.0.0.2        VLAN_K1                   0
                                           VLAN_K2
 4 ADC  192.168.1.0/24     192.168.1.251   ether1                    0

This works (connections are no longer dropped), but it is extremely slow.
When I start the packet sniffer tool, the speed is back to normal.

Does anybody have any idea ?

Further investigation gave me the guilty : fasttrack rule in default firewall ruleset.

 /ip firewall filter add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related

Starting Monitoring / Sniffing tools is typically something that disable some hardware/software optimization (hardware offloading, FastPath…).

Now it works fine.

Sorry to revive an old thread, but could you please post an export of your full config (without sensitive info of course)? I have the same use-case and have been trying to replicate your config for days without success, so I must be missing something and can’t for the life of me figure it out (wiki and google are not helping). Thanks in advance.