I am (quite) a bit confused about VLANs with Mikrotik Router OS.
So far I just basically setup a management VLAN on some Openwrt powered Switches and these work without issue.
Now to get to do more advanced stuff with Router OS on a CRS326 Switch, but it seems I have to do things twice in the web interface:
a. Define VLAN Table in Bridge → VLAN → Add New → e.g. VLAN 10 → Select Tagged Ports & Untagged Ports
b. On top of already having selected the Untagged Ports in step a, I still need to do it under Bridge → Ports and configure
PVID
Frame Types
Ingress Filtering
Tag Stacking
c. Assign an IP address to the management interface via IP → Addresses
d. Test that it works
e. Enable VLAN Filtering
Probably I have some misunderstanding of some aspects of VLAN, but if I configure a Port as “Untagged” for e.g. VLAN 10, why do I still need to configure it for the same thing (access port) in Bridge → Ports ? Is it about “Direction” of the traffic (incoming vs outgoing) that this “double” configuration is required ?
And what is PVID and how is it different from the VLAN ID used for Untagged port purposes in step a ?
Unfortunately, since I have to have an IP address before I can enable VLAN filtering, I cannot use a DHCP server to dynamically assign the management interface an IP address, Well, I guess many of you would tell me that a DHCP-provided IP address for a managed switch is a really bad idea (e.g. if DHCP server goes down or simply if I unplug the switch and want to test it on its own, after a while I cannot manage the switch anymore since it won’t have a valid IP address [anymore]).
Yes, there are things to be done in different places and, when one gets very intimate with ROS VLAN, it all makes sense. Because ROS allows you to do things others don’t (whether that makes sense or not is a completely different thing).
As a rule of thumb: bridge/port section is about ingress properties while bridge/vlan is about egress properties (things can get tied with ingress-filtering=yes).
So it is, for example, possible to set certain port as untagged member of some VLAN but set pvid to different value. Meaning that port can be untagged member of several VLANs for egress (for ingress it can only be access port of a single VLAN).
And this might be useful if some VLAN carries e.g. broadcast streams and there’s a client beyond a bridge port that is supposed to receive those broadcasts … but duplex communication is not needed (or wanted).
I’m struggling to understand the example you provided but, I guess, for my use case, it’s not really needed.
Actually for now I’m configuring the CRS317 but should be similar …
Right now I don’t even have a “stable” definition of ports, so by default:
All Ports Untagged to a “Default” VLANs
All Ports Tagged to all Required VLANs (all are basically trunks)
Then I’ll do the required VLAN setup either in a Hypervisor (Proxmox VE) or a GNU/Linux Computer or SBC.
I added a new IP Address for the Router OS Switch in IP → Addresses: 192.168.150.71/22 with Network 192.168.148.0 for Interface “bridge”. However, I cannot ping it from my local PC (tried both on eno1.100 which was VLAN Tagged 192.168.148.6 and eno1 which was untagged 192.168.148.12). I also weirdly cannot ping between Router OS Switches with their new .71 / .72 / .73 address.
I plan to always leave a port to NOT require any management VLAN on ether1 (unused by default) in case I need to rescue the installation.
But why are the new IP address not accessible ? Do I need to reboot the Switch for changes to take effect ?
EDIT: it actually works if I do a ifconfig eno1.100 down, since VLAN filtering is not yet enabled
Without VLAN-filtering each port gets VLAN 1 untagged and all the other VLANs tagged. I know, this is weird, but I learned the hard way. Do not forget to add the bridge as a tagged member of all the VLANs other than 1.
The test you did with eno1/eno1.100 (Is this the correct VLAN ID?) - were the IP enabled at the same time? Can you try again with only one interface getting an IP address? That is the either eno1 or eno1.100 but not both having an IP in the 192.168.148.0/22 network at the same time.If you have the possibiliy, tcpdump (with the -evvv argument) or wireshark/tshark is your friend. That will tell you what traffic comes tagged and which isn’t.
Keep trying! At some points, you’ll have the “of course!” moment It took me a while, being used to other vendors with different “configuration philosophies” and blocked the Mikrotik a couple of times.
[bHint:[/b] configure an interface as pure L3 while you test, that way you want shot yourself in the foot.
Shouldn’t I just add the “bridge” (AKA “Management Interface” or “CPU”) ONLY to the Management VLAN as “Tagged” rather ? And ALL other Interfaces (sfp-sfpplus1…16 etc) TAGGING ALL other VLANs as well ?
That’s also what I realized … That’s why an ifconfig eno1.100 down resolved the issue. Right now it’s NOT tagged because on Mikrotik VLAN filtering is NOT enabled yet. I think it’s a bit of a chicken and egg problem … How can I test that VLAN works and I can access the management VLAN, without risking getting locked out in the process ?
OpenWRT did some smart things in that regards, whenever you do a config change, if there is no further communication (i.e. you got locked out of the Switch/Router), then the configuration changes are automatically reverted within 90 seconds. I don’t see why Mikrotik is not doing something similar …
What do you mean ? Right now I just left it as a “No VLAN” interface (ether1) so I can just plug in a PATCH cable and resolv issues, should I get locked out. Do you have another idea ? Is that what you meant by Layer 3 (= NO VLAN) ?
I also need to configure the Zyxel (X)GS1910-24 I have in my homelab. Yeah it’s an old switch, but I don’t really want to spend like 200 EUR for a new CRS326 or similar when all I need is mostly 1gbit for management (if at all). I don’t see how it’s possible to select the management VLAN though …
I do not know the exact details yet - I am now using my test equipment for my home wifi and trying not to break it too many times. I have an order for more test equipment ready to go in a few weeks. As far as I understand, the VLANs for which there is a corresponding VLAN interface should be added. Untagged for the PVID of the bridge - often 1 - and tagged for the others. Definitely something I will test.
Yup, my previous wifi home solution used OpenWRT routers and switches. I guess Mikrotik is more like the big names with which you can screw yourself out of access, and have to use the serial port. Sad thing is while most Mikrotik devices do not have a serial port, they all have the same possibility to lock oneself out.
Exactly that: an interface that is not part of any bridge and has an IP address directly assigned to it. Don’t forget to adapt the firewall rules. Just realized my “won’t” became “want” in the post.
Without vlan-filtering enabled ROS device doesn’t touch 802.1q (a.k.a. VLAN) headers … and that includes those with VLAN ID 1. It’d hard to figure VLAN 1 as most vendors treat VLAN 1 as special (MT included to certain extent), but in vast majority of cases frames “belonging to VLAN 1” are passed tagless over wire.
Enabling vlan-filtering doesn’t actually enable pasing frames with 802.1q headers, it rather enables orocessing those headers. In that includes adding/removing header and VLAN-related security features.
However: /interface/vlan interfaces do their job without vlan-filtering enabled on bridge. So if you configure management access on ROS device over VLAN and use VLAN interface on linux (same VID), you should be able to access management on ROS device over this VLAN (using correct IP address). When this part succeeds, you can enable safe mode and then enable vlan-filtering. If there are things still missing, your management session will drop and safe mode will roll vack any changes done while it was enabled. If management session survives, disable safe mode and configuration changes will become permanent.
But should you do that in /interface/vlan ? I guess in this case it doesn’t really matter, as management VLAN will go through the CPU anyway. But I heard that in general it’s better to configure VLANs under “Bridge” to take advantage of the hardware accelerated VLAN switching that can be done directly in the Switching chip.
So how should I configure the management VLAN here, if I want it accessible from ALL interfaces ? From the WebFig I mean …
If ROS device needs to communicate with certain VLAN (msanagement, routing), then it’s the only way.
Since you’re mentioning linux: these two are identical:
ROS:
/interface/vlan
add name=e1v100 interface=ether1 vlan-id=100
/ip/address
add interface=e1v100 address=10.0.100.1/24
Linux:
ip address add 10.0.100.2/24 dev eth1.100
It’s just that in linux case, the vlan interface is created implicitly.
However, the magic above doesn’t have much with bridge. Bridge is specific because it has multiple personalities (depending on how one counts it may have up to 4 personalities) without clear distinction in ROS config, so it’s hard to wrap one’s head around it. Reading through this article should help.
Having VLANs enabled doesn’t prevent devices from different VLANs from communicating with each other if there’s an IP route between them … so it’s combination of VLAN conffig, routing and firewall which eventually controls connectivity.
Meaning: if a device from random VLAN wants to connect management IP of some other device, it’ll try to utilize its own gateway. And gateway may know how to route towards that device. And it’s then up to configuration of “some other” device if it can actually send replies (i.e. it’s got an aporopriate gateway configured as well). You get the picture, right?
But since I have multiple ports where MANAMEGEMENT VLAN needs to be accessed from, why cannot just use interface=bridge ?
Or do I need to EXPLICITELY create multiple times (16-24 times !) the MANAGEMENT VLAN on each port separately ?
Did you read about bridge personalities? I don’t think you understand what bridge in ROS is and that causes lots of confusion in your head
The example you quoted shows difference using single interface because it’s illustrating exactly same use case. And as I mentioned, there’s much more in bridge so I didn’t want to add more confusion in that particular example which was explaining role of vlan interface in ROS.
When you set some device’s interface as a bridge port, you should not use it for any other config. It’s enslaved and can have only one master - the bridge.
Bridge (the switch-like entity) will pass frames between member ports - according to port and vlan setup.
One of bridge personalities is “CPU-facing” port, created implicitly, is an equal member port and one has to / can configure all the same properties as for other ports members of the bridge. If you set bridge port as member (tagged or untagged) member of certain VLAN, thus allows router CPU to communicate with that VLAN.
Then you need to use “switch-facing” interface (aptly named bridge) to set untagged (affected by PVID setting) IP access … or create vlan interfaces, bound to bridge port (in which case bridge port has to be tagged member of appropriate VLAN(s)) and use that vlan interface to add necessary IP config.
Note use of “port” vs. “interface” throughout this post, they are different entities based on same (physical or software) device.
Right now ALL interfaces are in the bridge (default configuration). I don’t know why this should ever be changed (unless you want to “cut” a switch in half, e.g. take a 48 port switch and make 2x24 port switch ?).
I’m trying to make sense of it. What you explained kinda matched my previous understanding (“bridge” => CPU interface => management VLAN).
But just before you said, when I asked “But should you do that in /interface/vlan ?”
If ROS device needs to communicate with certain VLAN (msanagement, routing), then it’s the only way.
Whereas now you seem to imply that I can also do it in Bridge Settings (as I previously thought and planned to do).
To guarantee the independence of the rescue interface: you limit the risk of something causing an issue, and regardless of what you do to the switch or the ports associated with the switch, that interface will have an IP address and can be used to recover the access if needed.
Exactly: without vlan-filtering, all VLANs defined on the bridge are passed to all attached ports as these VLANs exist on the switch: pvid for the bridge in untagged, everything else is tagged. A test I have for later is to change the pvid of the bridge to see what happens.
Also note that, unless vlan-filtering is enabled, you will not see the VID in the list of hosts.
Again, exactly: with VLAN-filtering, one selects what goes on what port and in what form (tagged/untagged). My biggest surprise was to discover that unless vlan-filtering is enabled, one gets only VLAN 1 (or the bridge pvd) untagged. And the VID appears in the list of hosts. This latter was a gigantic surprise.
It seems that I don’t understand exactly what seems to bother you. I’m trying to explain to you that vlan-related settings under /interface/bridge and settings under /interface/vlan don’t overlap, they only touch each other. It’s hard for me to explain you things if you constantly consider them as a whole and you then try to make some perspective out of explanations where there isn’t one.
Setup of vlan-aware switch really is a two-step process:
setup layer 2: add ports to bridge, setup VLANs to pass between bridge ports
setup layer 3, needed to allow management access
And to make things (at least conceptually) easier, you can actually configure out-of-band management, where single port is not member of bridge. Instead you configure that port as interface, setting up IP address on it (e.g. ether16).
Again: I’m trying to help you understand things better, but I need you to give a good feedback (like: in description of bridge personalities, I don’t understand this and that). Your feedback so far doesn’t help me understand where your mental picture lacks details.
.
?
.