Setting up a dedicated port to be used for management only

marafado88 · Thu Jun 02, 2022 5:15 pm

Hello everyone,

I've a noob question on how to setup a remote gateway for a Console/boot interface on a Mikrotik CRS209+1G+8S+IN, that will be connected to another subnet that already exists (a subnet dedicated to management were I already have other devices such as smart switchs and other firewalls).

I've added a new route with the gateway of that subnet (172.31.100.254), and I can check on RouterOS that ether1 port (Console/boot interface) reachs that gateway (from what I see on Mikrotik webgui), but still, its not being used by that interface (If I connect that interface to that management subnet, I can only communicate within that network, not from other networks, and no there is no ports being blocked):

Anyone here knows how to achieve this?

Thanks in advance!

Sob · Thu Jun 02, 2022 9:31 pm

Like any other multi WAN. Add new default route in separate routing table. Then if you need access only to router itself, you can either use static IP address 172.31.100.x/24 and add routing rule telling router that when source is this address, it should look up destination in that new routing table. Or if you need to keep DHCP, then you'll have to mark new incoming connections from that interface and mark routing for responses, to again use the new routing table.

marafado88 · Fri Jun 03, 2022 12:16 pm

I know, that is in fact what I am trying to do here, but still dont know why that ether1 can only communicate with other devices (both inbound and outbound) in the same subnet, and not with other devices for both inbound and outbound traffic (there is no firewall rules on other firewall blocking or rejecting traffic for that 172:31.100.0/24 or even for that NIC of mikrotik with 172.31.100.102), with current setup.

I just want to setup a static IP of 172.31.100.102 on ether1, and setup a gateway for it to 172.31.100.254, for every connection received on that port.

For example that sfp-sfpplus1_vlan20 is in fact the primary gateway (a remote one)(default one), that its used to get internet access.

I've tried to set a Pref. Source for that interface ether1 static IP, for that gateway rule, but same result: https://prnt.sc/oGB_NhyVBgiZ

Also on firewall rules from mikrotik side, there is no rules.

Note that its my first time setting up a mikrotik routeos.

pe1chl · Fri Jun 03, 2022 4:45 pm

It is probably easiest to setup VRF for that.
The problem is that it is not possible to set VRF for each and every communication purpose. Where you cannot set VRF, it will use the default VRF.
So you need to decide whether you want the default VRF for obscure things like DNS lookups by the router, logging to syslog, etc to be the management network or the production network.
Then add a VRF either for management (including only your ether1 port) or for production (including all ports except the ether1 port).

Sob · Fri Jun 03, 2022 6:23 pm

If you can set static 172.31.100.102, then just add route in new routing table:

/ip route
add dst-address=0.0.0.0/0 gateway=172.31.100.254 routing-mark=management

And a rule telling router that traffic from 172.31.100.102 should use that routing table:

/ip route rule
add src-address=172.31.100.102/32 table=management

I'm not a big fan or VRF, I'm sure that it's useful, but so far for simple things I tried it seemed to only complicate them (but it could be just me).

pe1chl · Fri Jun 03, 2022 6:40 pm

Same for me, but it seems like the natural solution for people who want a completely separate management network that is secure without using a firewall...

marafado88 · Fri Jun 03, 2022 8:03 pm

Thank you pe1chl and Sob for your help.

I've made has you have said Sob but still no luck:

https://prnt.sc/DDe2BYU-cbiy
https://prnt.sc/oCfrgJxTLFch
https://prnt.sc/x7csBs2zoypA

If I traceroute from this mikrotik, it works:

https://prnt.sc/CkJnPPrfn281

If I ping, no luck:

https://prnt.sc/wUVPafoc-Ct6

From the other firewall, where that subnet gateway is, also no luck (only works if I use the gateway IP as source address):

https://prnt.sc/08XZ1Nd9Du-j

Sob · Fri Jun 03, 2022 10:01 pm

Don't set any interface in routing rule. And when you test it using traceroute or ping, don't select interface either, only set source address.

marafado88 · Mon Jun 06, 2022 12:37 pm

First I tried to traceroute with current config, without assign an interface, and also worked.

Second, I tried to change that rule interface to nothing, and I lost immediatly the ability to reach the 8080 for webconfig. From my pc, mikrotik responds only to pings if 172.31.100.254 (gateway for that management subnet) is present in subnet, if I try to connect directly (from that pc to mikrotik), mikrotik dont respond. If I try to disable that NIC with 172.31.100.150, and use only 10.0.13.100, there is also no ping response from 172.31.100.102. From Firewall A1 perspective, I can ping from both 172.31.100.254 and 10.0.13.1. To regain access again to this mikrotik, I had to connect my pc directly to mikrotik and set his NIC from 172.31.100.150 to 172.31.100.254.

Right now, I have the following lab setup: https://prnt.sc/TZwS_hxxDZ0y

marafado88 · Mon Jun 06, 2022 12:51 pm

I've tried to set the following rules (https://prnt.sc/4UKl-pskaRiV), and I was able to ping from 10.0.13.0/24 network to 172.31.100.102, and from that pc (172.31.100.150) directly to mikrotik, but still the webconfig page only responds to 172.31.100.254.

pe1chl · Mon Jun 06, 2022 2:27 pm

To get meaningful help, post your /export here instead of screenprints.

marafado88 · Mon Jun 06, 2022 2:47 pm

Webconfig was only accepting connections from 172.31.100.0/24, but I've discovered how to add more so I had 10.0.13.0/24 (https://prnt.sc/RhCsmAxNnV05).

As a test, I've tried to use only that rule without interface (like you have mentioned before), but from 10.0.13.100 machine there is no response (webconfig and ping), now not even ping responses to 10.0.13.1 from Firewall A1. Connected with 172.31.100.150, and with 172.31.100.254, in the same subnet, my laptop receive ping responses, but webconfig doesnt open. Connecting laptop and mikrotik directly, there is no weconfig and ping responses.

marafado88 · Mon Jun 06, 2022 2:56 pm

To get meaningful help, post your /export here instead of screenprints.

How do I export?

rextended · Mon Jun 06, 2022 2:58 pm

on terminal:

/export hide-sensitive

and before post the results on forum, search and remove remaining serial number, email, username, password, private data inside scripts or schedulers,
and replace public IP (if any) with something like 127.1.2.3
but do not delete anything you think that is not relevant.

marafado88 · Mon Jun 06, 2022 4:25 pm

Thank you @rextended!

With @Sob suggestion, were I got lockdown (and the only way to gain access, is using 172.31.100.254 on my pc directly connected with this mikrotik), I have the following export:

# jun/06/2022 14:18:19 by RouterOS 6.47.9
# software id = 7XZS-YPG3
#
# model = CRS309-1G-8S+
# serial number = x
/interface bridge
add admin-mac=2C:C8:1B:8D:F4:01 auto-mac=no comment="SFP LAN" name=bridge
/interface ethernet
set [ find default-name=ether1 ] comment="POE BOOT Port" mac-address=2C:C8:1B:8D:F4:01
set [ find default-name=sfp-sfpplus1 ] mac-address=2C:C8:1B:8D:F3:F9
set [ find default-name=sfp-sfpplus2 ] mac-address=2C:C8:1B:8D:F3:FA
set [ find default-name=sfp-sfpplus3 ] comment="Firewall A1" mac-address=2C:C8:1B:8D:F3:FB
set [ find default-name=sfp-sfpplus4 ] mac-address=2C:C8:1B:8D:F3:FC
set [ find default-name=sfp-sfpplus5 ] comment="Firewall A2" mac-address=2C:C8:1B:8D:F3:FD
set [ find default-name=sfp-sfpplus6 ] mac-address=2C:C8:1B:8D:F3:FE
set [ find default-name=sfp-sfpplus7 ] mac-address=2C:C8:1B:8D:F3:FF
set [ find default-name=sfp-sfpplus8 ] mac-address=2C:C8:1B:8D:F4:00
/interface vlan
add comment="SFP WAN VLAN20" interface=sfp-sfpplus1 name=sfp-sfpplus1_vlan20 vlan-id=20
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/interface bridge port
add bridge=bridge comment=defconf interface=sfp-sfpplus2
add bridge=bridge comment=defconf interface=sfp-sfpplus3
add bridge=bridge comment=defconf interface=sfp-sfpplus4
add bridge=bridge comment=defconf interface=sfp-sfpplus5
add bridge=bridge comment=defconf interface=sfp-sfpplus6
add bridge=bridge comment=defconf interface=sfp-sfpplus7
add bridge=bridge comment=defconf interface=sfp-sfpplus8
/interface list member
add interface=sfp-sfpplus2 list=LAN
add interface=sfp-sfpplus3 list=LAN
add interface=sfp-sfpplus4 list=LAN
add interface=sfp-sfpplus5 list=LAN
add interface=sfp-sfpplus6 list=LAN
add interface=sfp-sfpplus7 list=LAN
add interface=sfp-sfpplus8 list=LAN
add interface=sfp-sfpplus1_vlan20 list=WAN
add interface=ether1 list=LAN
/ip address
add address=172.31.100.102/24 comment="POE/BOOT RJ45 port" interface=ether1 network=172.31.100.0
add address=172.16.2.1/29 comment="Bridge SFP plus from 2 to 8 ports" interface=bridge network=172.16.2.0
add address=x.x.x.226/30 comment="ONT 2 Uplink port" interface=sfp-sfpplus1_vlan20 network=x.x.x.224
/ip dns
set servers=1.1.1.1
/ip firewall nat
add action=masquerade chain=srcnat out-interface=sfp-sfpplus1_vlan20
add action=dst-nat chain=dstnat comment="TCP DMZ" in-interface=sfp-sfpplus1_vlan20 protocol=tcp to-addresses=172.16.2.6 to-ports=0-65535
add action=dst-nat chain=dstnat comment="UDP DMZ" in-interface=sfp-sfpplus1_vlan20 protocol=udp to-addresses=172.16.2.6 to-ports=0-65535
/ip route
add distance=1 gateway=172.31.100.254 routing-mark=to_MANAGEMENT
add distance=1 gateway=x.x.x.225
add disabled=yes distance=1 gateway=172.31.100.254 pref-src=172.31.100.102
/ip route rule
add src-address=172.31.100.102/32 table=to_MANAGEMENT
add disabled=yes dst-address=10.0.13.0/24 src-address=172.31.100.102/32 table=to_MANAGEMENT
/ip service
set telnet address=172.31.100.0/24 disabled=yes
set ftp address=172.31.100.0/24 disabled=yes
set www address=172.31.100.0/24,10.0.13.0/24 port=8080
set ssh address=172.31.100.0/24
set www-ssl address=172.31.100.0/24
set api address=172.31.100.0/24
set winbox address=172.31.100.0/24 disabled=yes
set api-ssl address=172.31.100.0/24
/system clock
set time-zone-name=Atlantic/Madeira
/system identity
set name="Router A2 WAN2"
/system routerboard settings
set boot-os=router-os
/system swos
set address-acquisition-mode=static allow-from-ports=p1,p2,p3,p4,p5,p6,p7,p8,p9 identity=switchawan2 static-ip-address=172.31.100.102
/tool bandwidth-server
set enabled=n

marafado88 · Mon Jun 06, 2022 5:01 pm

Well, this was the workaround that I've found:

/ip route
add distance=1 gateway=172.31.100.254 routing-mark=to_MANAGEMENT
add distance=1 gateway=x.x.x.225
add disabled=yes distance=1 gateway=172.31.100.254 pref-src=172.31.100.102
/ip route rule
add dst-address=172.31.100.0/24 routing-mark=to_MANAGEMENT src-address=172.31.100.102/32 table=to_MANAGEMENT
add dst-address=10.0.13.0/24 src-address=172.31.100.102/32 table=to_MANAGEMEN

This way I will have to add manually if needed, in the future ... if any of you had a better config, in a way that it will not be needed to add each subnet, please share here.

Sob · Mon Jun 06, 2022 10:38 pm

That's weird. If you posted complete config, then this:

/ip route rule
add dst-address=172.31.100.0/24 routing-mark=to_MANAGEMENT src-address=172.31.100.102/32 table=to_MANAGEMENT

shouldn't do anything, because there are no routing marks. And then the only diffence between yours and mine is that your other routing rule has dst-address=10.0.13.0/24 as additional condition. So for connections from 10.0.13.x they work the same.

There's probably some misundestanding. For example this:

From my pc, mikrotik responds only to pings if 172.31.100.254 (gateway for that management subnet) is present in subnet, if I try to connect directly (from that pc to mikrotik), mikrotik dont respond. If I try to disable that NIC with 172.31.100.150, and use only 10.0.13.100, there is also no ping response from 172.31.100.102.

Yes, there must be 172.31.100.254 as gateway, you can't connect PC with 10.0.13.100 directly to ether1 and expect it to work, because router has no idea that this subnet is directly reachable there. And if PC has 172.31.100.150/24 (which is in router's subnet) and also has 10.0.13.100 on same or another interface, and you use 10.0.13.100 as source when connecting to router's 172.31.100.102, it can fail because there will be asymmetric routing, and if the Firewall device has statetul firewall, it will block it, because it will see only half of packets.

marafado88 · Tue Jun 07, 2022 12:24 pm

During those tests, I was enabling and disabling those NICs.

It was a trial and error in that config, after you pointing me to Rules tab.

I had to add that network 10.0.13.0/24 (where there is the IT staff computers) because I didnt found a way to allow any inbound and oubound traffic on mikrotik ether1, letting Firewall A1 to decide that. But after thinking a while, since on mikrotik management services I've also to specify what subnets should reach those, that should not be a problem.

The main objective here was to add that mikrotik and another one (same model) on a seperated subnet dedicated for management services for security reasons, and dont be lockdown in case of a major network problem, being able to directly connect a pc on ether1, and be able through ether1 to access that unit instead of using serial ports. A full scope of this setup still without ether1's connected: https://forum.netgate.com/assets/upload ... 0643-1.jpg

Setting up a dedicated port to be used for management only

Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Re: Setting up a dedicated port to be used for management only

Who is online