Can not access the CPU via incomming vlan !! :(

Hello,

I am actually managing my CRS317 RouterOS 7.18.2 via the console. However I try to manage the switch via a VLAN arriving from my firewall.
Since I did not manage that using the main bridge. I decided to do some testing using a second bridge.

The setup I use is as follows:

  • defined a second bridge ‘the management bridge’
  • assigned two interfaces to that bridge
  • define the management vlan in the ‘interface menu’ using the managment bridge as interface
  • then defined the managment vlan in the bridge menu as ‘management bridge vlan’
  • added the two interfaces and the bridge itself to that vlan
  • defined two addresses in IP addresses menu related to the vlan subnet one for the bridge and one for the vlan
  • one of the interfaces is the trunk towards the firewall (and vlan gateway)
  • one interface connected to a small switch which which can be managed via the lan (no other connections to that switch
  • defined the default gateway as being the vlan 0000/0 => 192.168.10.1 (the vlan gateway on the firewall) action lookup
  • pvid on interface vlan and bridge vlan definition the same (10)

No doing the following tests, using my PC at some other vlan

  • ping to the test switch connected to test interface-2 => OK
  • ping to the vlan gateway on the CRS => OK
  • ping from the CRS vlan gateway to the GW on the router => OK
  • ping to the managment bridge … no way :open_mouth:

I tried this with with multiple settings

  • pivd in bridge vlan filtering on 10 (same as vlan) (all all / allow tagged only) => NOK
  • changed vlan filtering pivd to 1 which seems absolutely nok whatever => NOK
  • changed the pivd on the two interfaces to 10 (not ok but ala) => NOK

Before I created this setup I partly did the same test, with the same result. Bridge does not react.

So big question, is what I am doing wrong! What is wrong with this setup !! ???

Is this the same device that mkx was trying to help you with??

Yes of course. I did create a setup which do allow me to do very specific tests. Using a second bridge allows me to test things without losing connectivity to the CRS.

And I did create even more than before a test setup which do allows me to very specific test the problem. And as you can read everything works apart from the most important thing access to the switch its CPU.

And really do not know why it does not work and what to test more :frowning:

The vlan itself works. I can communicate with the test point (the extra switch I connect for that purpose) at the other site of the CRS.
I can ping the the CRS vlan gw address I did extra add as extra test possibility. But I never managed to access the CPU itself
Even trying IMHO obscure settings

I am really really lost …

Well, its pretty straightforward…
Only one vlan is identified on the switch, the management vlan and in IP address is where switch gets its IP address from.
Only the managment vlan is tagged with the bridge, the rest are tagged on the incoming trunk port and as required on outgoing ports ( untagged or tagged ).

The only interface list needed is TRUSTED
which the management interface is part of.
Identified in ip neighbours discovery and tool mac-server mac-winbox

The only other recommendation is to take one port off the bridge and make it an off bridge access.

Many Thanks !!! …

I changed only one thing, I added trusted to the trunk towards pfsense leaving the rest as it was before.
So fixed address assigned to the bridge on the CRS.

And it works !!

Still in the situation with two switches which I did need to do testing, without losing the gui all the time !!
In fact that one setting did cost me probably two days.
My setup was not far off from the very beginning !

Thanks again, I am going to try now if it works with one bridge as well. It needs to be one switch I assume, because

  • interfaces can only belong to one bridge
  • and two bridges do probably imply two level-2 domains

If that is not working I will probably assign the second bridge to ether1, which is a domain by itself

It also works with one bridge. So the main problem was all the time the trusted setting !!

However I tried with two trusted interfaces …
The original not connected … and that does not work …
It is weird …

Back to the one trusted interface …
and that does not work any more …
perhaps it will work after some time expired (could be one the rest of the network as well)

No idea it did work … but not any longer … do not know why
arps are not answered … which have been a problem for days …
lets hope the arp will return … I think there is an arp setting somewhere …

After I set the incoming connection to “trusted” for a moment the setup seems to work. However not for long (Strange!!) .
Which could be related to reboot. I might have a routing issue.

The actual situation is as follows:

I defined the vlan in the interface menu. I did that for two reasons:

  1. When defining the default routing rule, I have to add the interface and that should be the vlan and not physical interface
  2. some tools more or less require an interface as selection criteria

Since the cpu only seems to listen to trusted interfaces, I marked the vlan as ^trusted^

I assigned an IP belonging to the vlan subnet to the bridge.

In the routing menu I added the vlan as default gateway.
source 0.0.0.0/0;
destination the remote vlan gateway address;
routing mask main;
interface mngt-lan;
action lookup (also tried only this table);
table main

Bridge

  • added the vlan to the bridge (not sure if that is necessary)
  • vlan filtering on (should IMHO always be on)
  • admit all
  • pivd-1; (which is NOT the pivd of the vlan(10))

bridge vlan

  • bridge = bridge
  • vlanId = 10 (the pivd of the vlan)
  • tagged; two interfaces (required) and the bridge; note that I tried also including the vlan, but that is not necessary
  • note that the interfaces are not trusted, the vlan is.

The vlan seems works as vlan, however the bridge is not accessible.
Not OK is that in the interface menu I just see Tx and not Rx data

I could try the dual bridge option again, however I did that retry all ready … and even that did not work (strange!).
I think that using two bridges is not possible since I assume that a bridge is related to physical interfaces.

What may(!!?) happen is asymmetric routing, traffic arriving via vlan-x should of course be answered via vlan-x as well
(which is not so obvious, given historic use of only one routing table (very very wrong IMHO)

What ever if some one knows what is wrong and how to solve, I would appriciate!

One would have to provide the fact.
/export file=anynameyouwish (minus router serial number, any public WANIP informaiton, keys)

In both cases, device as a switch or router:
The fact of the matter is the bridge does NOT get an address. The vlan gets an address.

The only route required on a switch, and its optional but not a bad idea is to the gateway of the vlan
add dst-address=0.0.0.0/0 gateway=gatewayIP of vlan routing-table=main

Same with DNS
Same with NTP ( to sync to upstream router )

Also on a switch setup, Only vlan-ID tagged with bridge is vlan management, rest should be tagged to ether1 (or whatever is incoming trunk port) and rest of ports untagged or tagged to vlans as applicable.

I will try.

For info:
I have defined three bridges now and use the third bridge to login. That allows me to change the settings of two other bridges without losing connectivity. To do this I

  • defined the third bridge and assigned one port to that bridge.
  • assigned an ip to that bridge using an ip-range from a new vlan
  • assigned the pivd to the port
  • I am not using a vlan for this purpose (unatagged)
  • connect to that port directly connected to my pc
    Note that I did NOT make the port trusted !! and it works never the less!

The Idea was/is to change the PIVD from the main switch to the PIVD of my managment lan and change the PIVD of the second bridge to one in favor of ether1. And move ether1 to bridge2

That is the brand new actual situation. Where I must state that at the moment I can access the switch via bridge3 but not via bridge2 (ether1) or the main bridge (the management lan).

Do not yet understand why

Sorry cannot help further. The advice from the beginning has been one bridge…lead a horse to water…

Thanks for the given assistance!

Two points which are perhaps interesting:

  • I can access the switch via the two extra bridges
  • perhaps more interesting using the build in tools and the graphs … I get the impression that a lot of traffic passing the switch is not seen by the OS … perhaps passing the switch via the switch chip … could be a reason that I do not manage to reach the CPU …

Note this is just a weird feeling …

I fixed it … it was very simple … (At least for my actual setup)

I did of course know that I had to define a route towards pfSense to make it possible for the CRS to send a replay.
And I did create that route in the … routing menu …
But that is not where it should be …

The return route should be under IP routes :open_mouth:
So after adding a default rule towards the router there (0.0.0.0/0 => router GW) … solved the problem !!

Solved :smiley:

Have a look at https://www.youtube.com/watch?v=YLtGQAQ8iS0&t=1126s and watch very carefully which menu item he chooses when he defines the backup route.

PS
He also uses for the bridge pivd1 that is probably only related to untagged ports and not to vlans. Which is very confusing IMHO

And people wonder why we ask for config exports from their routers… :wink: