NAT + Tag/Untag multiple identical devices

RouterOS Beginner Basics
1

I have in hands the setup shown in the picture. On the left side is PC connection, which is connected via managed switch to ether2 interface of CCR1016-12G router. On the right side are 10 pieces of equipments, which are connected to ether3-ether12 interfaces of the router. The equipments are all identical with unchangeable overlapping MAC and IP addresses. I need means to connect from the ether2 interface to all of the ether3-ether12 interfaces. However, from each ether3-ether12 interface it should be possible to connect only to ether2 interface, but not others.

For the PC side there is need to strip the VLAN200 tag (and vice versa), since the equipments only understand VLAN201, VLAN202 and untagged. For the equipment side also IP and MAC NAT processing should be done in addition to tagging/untagging.

I believe that the necessary measures could be taken with the CCR1016-12G router, but still rookie with routers… Any recommendations which steps in RouterOS are needed and in which order to carry out the needed processing?

2

I think the first step is to decide which method for vlans will be used by the OP.
a. switch chip method (older but applicable to switches and routers that could take advantage of such setup)
b. vlan bridge filtering method (newer but may not be applicable).

Hopefully ones familiar with your device will chime it with some advice on the method to be used.
I would prefer they state A. because then I dont have to help being a complete noob at switch chip configurations. :slight_smile:

3

The clue is VRF.

4

Variable Refrigerant Flow ?? :stuck_out_tongue_winking_eye:

Good thing because I get a headache looking at that diagram but why does VRF pop into your head and not lets say firewall rules?

5

The CCR1016-12G is based on TILE-Gx16 CPU. All the ether ports are directly connected to the 16 core CPU - no switch chips.

6

My bad,however xvo, smart as a tack, sharp as a whip LOL, probably has the right path for you to investigate.

7

Which parts of the processing would you do with the virtual routing and forwarding (VRF) technology? The VLAN tag/untag and NATing has to be done in lower layers than L3 anyway, right?

8

VLAN tagging/untagging don’t really complicate anything, you’ll just end with bunch of VLAN-interfaces on each of the ethernet ports.
You can treat it like a separate pre-task.

The tricky part is to route between same addresses on different interfaces, that’s where VRF + mangle will help.
NAT will do the rest: mapping address - VRF table pairs to new address and back.

I’ve done this once for only two interfaces, and config is rather huge.
In your case it will be massive, I guess.

Here’s is a link to an article in Russian: https://habr.com/ru/post/262091/
Try to google-translate it. However the code blocks are kind of self-explanatory.
Almost exactly your case: but because of the VLANs you will end up with the same number of VRFs (ok, not the same - you have 10 pieces of equipment, not 4), and each will contain 3 interfaces (for each of the VLANs).

9

Thanks xvo! I’ll try the same approach

10

xvo, I tried your example in https://habr.com/ru/post/262091/, but it doesn’t work as such. I used Raspberry PIs in ether3, ether4 and ether5 with identical addresses as yours. No connection available from 192.168.2.2 to 192.168.2.13 or 192.168.2.14. Is there something missing?

/ip route vrf
add interfaces=ether3 routing-mark=DEV3
add interfaces=ether4 routing-mark=DEV4

/ip address
add address=192.168.2.1/24 interface=ether5 network=192.168.2.0
add address=192.168.1.2/24 interface=ether3 network=192.168.1.0
add address=192.168.1.2/24 interface=ether4 network=192.168.1.0

/ip address
add address=192.168.2.13/24 interface=ether5 network=192.168.2.0
add address=192.168.2.14/24 interface=ether5 network=192.168.2.0

/ip firewall mangle
add action=mark-routing chain=prerouting dst-address=192.168.2.13 new-routing-mark=DEV3
add action=mark-routing chain=prerouting dst-address=192.168.2.14 new-routing-mark=DEV4

/ip firewall mangle
add action=mark-routing chain=prerouting dst-address=192.168.2.2 new-routing-mark=main

/ip firewall nat
add action=dst-nat chain=dstnat dst-address=192.168.2.13 in-interface=ether5 to-addresses=192.168.1.1
add action=src-nat chain=srcnat out-interface=ether3 to-addresses=192.168.1.2
add action=dst-nat chain=dstnat dst-address=192.168.2.14 in-interface=ether5 to-addresses=192.168.1.1
add action=src-nat chain=srcnat out-interface=ether4 to-addresses=192.168.1.2
11

You should have your management PC (or in your case management RPi) on ether5 with an address 192.168.2.2
And two managed devices (RPi’s) on ether3 and ether4, both with 192.168.1.1
And with this config it has to be possible to reach managed devices by 192.168.2.13 and 192.168.2.14 from management PC.
That is what you do, right?
And that don’t work?

12

Yes the setup is like in this picture that feel probably familiar:

There is basic ether1 address setup for management purposes. Your config is set for ether3, ether4 and ether5. No other setting in the box. The dark red arrows and partly hidden addresses in the picture are connections for management/debug purposes.


No ping or ssh connection work from management RPi to DEVs. But…

[root@localhost ~]# arping -c 1 -s 192.168.2.2 -I eth0 192.168.2.13
ARPING 192.168.2.13 from 192.168.2.2 eth0
Unicast reply from 192.168.2.13 [??:??:??:??:??:??]  0.991ms

…where the response is with MikroTik’s MAC address

13

I have one idea.
For returning packets you do this:

/ip firewall mangle
add action=mark-routing chain=prerouting dst-address=192.168.2.2 new-routing-mark=main

Is this rule being hit at all?
The idea is, dst-nat is performed after the prerouting chain, so probably the action reversing the src-nat too (although it’s not placed anywhere on the packet flow diagram).
What if you change 192.168.2.2 to 192.168.1.2 in thе rule above?

14

Now ping and ssh connection are working from management RPi to DEVs! Thanks xvo!

Here is the working version of the settings:

/ip route vrf
add interfaces=ether3 routing-mark=DEV3
add interfaces=ether4 routing-mark=DEV4

/ip address
add address=192.168.2.1/24 interface=ether5 network=192.168.2.0
add address=192.168.1.2/24 interface=ether3 network=192.168.1.0
add address=192.168.1.2/24 interface=ether4 network=192.168.1.0

/ip address
add address=192.168.2.13/24 interface=ether5 network=192.168.2.0
add address=192.168.2.14/24 interface=ether5 network=192.168.2.0

/ip firewall mangle
add action=mark-routing chain=prerouting dst-address=192.168.2.13 new-routing-mark=DEV3
add action=mark-routing chain=prerouting dst-address=192.168.2.14 new-routing-mark=DEV4

/ip firewall mangle
add action=mark-routing chain=prerouting dst-address=192.168.1.2 new-routing-mark=main

/ip firewall nat
add action=dst-nat chain=dstnat dst-address=192.168.2.13 in-interface=ether5 to-addresses=192.168.1.1
add action=src-nat chain=srcnat out-interface=ether3 to-addresses=192.168.1.2
add action=dst-nat chain=dstnat dst-address=192.168.2.14 in-interface=ether5 to-addresses=192.168.1.1
add action=src-nat chain=srcnat out-interface=ether4 to-addresses=192.168.1.2
15

Niiice!
However that makes me wonder if the guy who wrote the article ever tried it himself in the exact way he wrote.
I once built a test setup somehow using this article as a guidance, but the setup itself had some major differences.

16

So, this works for connections to DEVs initiated by RPi. How would you enable connections to RPi initiated by DEVs?

Two wan with in the same subnet
17

I got also connections initiated by RPi working, but only with connection marks in NAT, which was generated in mangle preprocessing phase. Otherwise I was not able to get srcnat working.

How should the VLANs work with VRF? Should there be one bridge that include all the interfaces or multiple bridges?

18

I don’t think you need bridges at all: just vlan-interfaces on top of each of ethernet ports.
It’s not that you will be switching between to ports with vlan tag in mind, but rather untagging - routing - then tagging again.