Weekly VLAN confusion post

RouterOS General
1

I wouldn’t blame the powers that be if my guru status was revoked, but I’m still struggling and (dependably at great cost to my sense of self) I need to ask for help (again) with VLANS.

I have found that the way I check of confirm my understanding of a topic is being able to explain it clearly, so that the completely unitiated can understand. (hint hint to all those generous souls who are inclined to reply.)

This is as far I can get:


“/interface vlan” defines vlans by id (number) and associates those vlan-ids with an interface (for example: bridge), as well with a name and/or comment.

/“interface bridge port” defines whether frames are allowed to ingress (enter) the router on the specified interface, and, if the frames are allowed to ingress, the frames are tagged with the specified pvid (or, if pvid is not specified then frames are tagged with pvid=1). The criteria for being allowed to ingress any specific interface is defined by the frame-type (admit-all, admit untagged-and-priority, or admit tagged-only).

“/interface bridge vlan” defines egress (leaving) behavior. I can’t put into words in a form similar to above the details of how this works – specifically, what tagged= and untagged= means.

Now, please, in the name of the last bit of my sanity, respond with clear and simple and non-technical/jargon answers.

2

OK, let’s do this in correct order and one step at a time. Correct order:

  1. /interface bridge add
  2. /interface bridge port add
  3. /interface bridge vlan add
  4. /interface vlan add

Do we agree on this?

3

If I understand correctly, what you’re saying is that a key to understanding this is to learn it in this specific order:

  1. Add a bridge
  2. Add bridge ports
  3. Add bridge vlans
  4. Add interface vlans

Correct?

4

With /interface/vlan command you are creating VLAN interface on your device and there you are defining what VLAN ID will that interface have, to what bridge will belong etc…

/interface/bridge/port here you are defining which physical ports will be bridge members, you can also select VLAN tagging and also you can specify which VLAN they belong to if you want.

/interface/bridge/vlan here you can untag specific ports for VLANs (I personally do it in bridge/port), you create trunk ports etc.

5

I would say this is better


  1. /interface bridge add
  2. /interface vlan add + configure IP address + DHCP server + interface list
  3. /interface bridge port add
  4. /interface bridge vlan add

Step 2 can be done without requiring the port/tag/untag assignment to be complete. And the whole interface with working IP addresses and DHCP + Firewall will be ready at the moment you add the /interface bridge vlan entry (you don’t need trunk port then the moment you make the change to the ports under /interface bridge port). Which means in most case you can make change to the VLAN configuration (add new VLANs for instance) without the need to temporarily turn of VLAN filtering on the main bridge.

6

  1. Add a bridge
    → in Layer 2: bridge is created to add ports to that bridge (bridge is just the logical concept to portrait the switch)

  2. Add bridge ports
    → in Layer 2: define which ports are contained in the bridge/switch

  3. Add bridge vlans
    → in Layer 2: define VLANs which should be available and active on said bridge

  4. Add interface vlans
    → in Layer 3(!): here are the layer3 (IP capable) interfaces are created and defined. at this point IP addresses and ip-forwarding becomes available. also without that there is no logical link to enable vlan routing, l3hw-offload or any kind which goes beyond switching in itself

up until point 3 routeros is just switching and at point 4 it a layer3-switch / vlan-router capability is available.

hope this helps in some kind of way

7

Trying with another words:


“/interface bridge vlan” defines egress (leaving) behavior … what tagged= and untagged= means

Ethernet frames, transmitted by (e.g.) RJ45 port over UTP cable, have certain structure. First come some low-level bits, then comes header, then comes payload and frame ends with frame check. The difference between tagged and untagged is in the header.
So ethernet header consists of:

  1. destination MAC address (6 bytes)
  2. source MAC address (6 bytes)
  3. ether type (2 bytes) … this contains indication about payload type (e.g. 0x0800 means IPv4, 0x86DD means IPv6, 0x8100 VLAN-tagged frame (“normal” VLAN), 0x88A8 VLAN-tagged frame (“S-Tag”), etc.)

If ether type is IPv4 or IPv6, then this is untagged frame. And payload is a normal IP (v4 or v6) packet, starting with IP header.

If ether type is one of VLAN types (usually 0x8100), then payload of ethernet frame consists of 802.1Q (or 802.1ad in case of S-Tag or Q-in-Q) header - a.k.a. “VLAN tag” … and original payload.
Or in simple words: VLAN tag is inserted inside ethernet header, between source MAC address and ether type (as it would be without VLAN tags).

VLAN tag consists of:

  1. “ether type” (2 bytes), which is 0x8100 (or 0x88A8 for Q-in-Q / S-Tag headers)
  2. PCP (3 bits) … priority bits
    this is where the frame-types=allow-only-untagged-and-priority-tagged … 802.1Q header could only set priority and not VLAN ID
  3. DEI (1 bit) … “allow drop” bit, in conjunction with priority bits mark frames which can be dropped by switch if congestion occurs
  4. VID (12 bits) … the actual VLAN ID. Value 0 means untagged (if only priority bits are set) and max value of 4095 (0xFFF) is reserved

To recap: when port is set as untagged in /interface/bridge/vlan, then ethernet frame, transmitted via physical port, will not have VLAN tag inserted behind source MAC address field in ethernet frame. When port is set as tagged in /interface/bridge/vlan, then ethernet frame, transmitted via physical port, will have corresponding VLAN tag inserted behind source MAC address field.
Since the rest of ethernet frame is identical in both cases, this means that tagged frame will be longer by 4 bytes (the size of 802.1Q / 802.1ad header).


“/interface vlan”

These interfaces are necessary for RouterOS (the software part) to communicate inside tagged network. IP services (e.g. router, winbox server, webfig server, ssh server, etc.) natively don’t know anything about ethernet frame structure nor about VLAN tags. “interface” entities will strip off ethernet headers (and trailers, checksum comes after actual payload in ethernet frame) and pass payload to higher layer according to ether type (i.e. IPv4 stack if ether type is 0x0800 or IPv6 stack if ether type is 0x86DD). But it doesn’t know directly how to deal with other ether types.

This is where vlan interface comes into play. It “anchors” to underlying interface (or rather a port) and provides untagged interface. The “anchor” port can be either bridge or physical ethernet port.
On “anchor” side it takes frames with ether type 0x8100 (with usual configuration) and VID set to value equal to own setting. It strips the 802.1Q header, reconstructs ethernet frame and passes it out on it’s “untagged” side. In the other direction, it takes ethernet frame, inserts 802.1Q header (with appropriate ether type and VID vlaue set equal to own setting) and passes tagged frame to the “anchor” side. It is customary then to attach router’s IP stack to the untagged end of VLAN interface where frames are tag-less and can IP stack thus work as usually (actually unaware of VLAN technicalities).

8

Thank you all very much for your efforts.

Trying to consolidate and reconsile what everyone has written so far. I’m sure it doesn’t look like it to some, but the descriptions and explanations are not easy to make consistent with each other and complete.

The use of some terms (port, etherport, interface, anchor, physical port) is very confusing. I get a sense that some might have multiple meanings, and some have meaning for those of us “in the know” but the inconsistent use makes it very hard to learn this stuff.

Trying to bridge (oh no!!!) between the theoretical and practical (and not between the physical and the logical and the abstract, as the word “bridge” is used hereinafter):

\

  1. Create a bridge (this is a layer 2 and operates on ethernet, also known as MAC address level; this bridge can be conceptualized as a switch within a device such as a router) – let’s name it bridge=TESTBRIDGE

    \
  2. Add ports to TESTBRIDGE

/interface/bridge/port defines the ingress behavior of frames to the specified port, specifically (1) which physical ports (that is, etherports) will be a member of TESTBRIDGE, (2) the “default” VLAN-ID of each port in the form of pvid, (3) type VLAN tagging for that will be admitted by that port.

For example:

/interface bridge port

add bridge=bridge frame-types=
admit-only-vlan-tagged interface=ether2 internal-path-cost=10 path-cost=10

** This config line instructs the router to allow frames to ingress on ether2 only if they have vlan tags. This makes ether2 a trunk port.


add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether4 internal-path-cost=10 path-cost=10 pvid=22

** This config line instructs the router to allow frames to ingress on ether4 only if those frames are untagged (or have priority tagging) and to then insert a vlad tag with vlan-id=22. This makes ether4 an access port.



3) Add vlans to TESTBRIDGE

/interface/bridge/vlan defines the egress behavior of frames (that is, for frames moving from inside the MT device, out the physical jack (RJ45) to the twisted pair and destined for the next network device; OR frames internal to the MT device and moving to a virtual device such as a bridge) according to the following rules: When port is set as untagged the frames leaving that port will not have the VLAN tag. When port is set as tagged, the frame will have the VLAN tag inserted.

That means that the following config line:

add bridge=TESTBRIDGE tagged=ether2,ether3 vlan-ids=2,12,32

allows frames with vlan-ids of 2, 12, and 32 to egress on ether2 and ether3



4) Add /interface/vlan.

This connects layer 2 (the first 3 steps above) to layer 3 (IP).

/interface/vlan creates VLAN interfaces and defines (1) the VLAN-ID of that interface, (2) the bridge to which it belongs.


This is still an extremely confusing, and I am certain that I do not understand it, despite massive amounts of effort and help. I am really growing to dislike this endeavor.

9

Nothing works without (1) the bridge. I put interface vlan last because while they can be added immediately after the bridge, they won’t work until bridge port and bridge vlan are correct.

10

Useful MikroTik Help and Forum links:

Rule 6, Lock down bridge MAC address: admin-mac=xx:xx:xx:xx:xx:xx auto-mac=no
Define bridge priority for all interlinked switches: priority=0x8000
Don’t lock yourself out while setting up: vlan-filtering=no
Make sure ARP is not disabled: fast-forward=no

/interface bridge add name=TESTBRIDGE admin-mac=xx:xx:xx:xx:xx:xx auto-mac=no fast-forward=no priority=0x8000 vlan-filtering=no

MSTP highly recommended for VLAN networks.
Agreed? Questions?

11

disables ARP? really? haven’t seen that ever after
and i have setup a lot of bridges… old and new fashioned way
arp always working so far with fast-forward set to “yes”

12

Bridging and Switching § Fast Forward says,

Fast Forward disables MAC learning, this is by design to achieve faster packet forwarding. MAC learning prevents traffic from flooding multiple interfaces, but MAC learning is not needed when a packet can only be sent out through just one interface.

Fast Forward is disabled when hardware offloading is enabled. Hardware offloading can achieve full write-speed performance when it is active since it will use the built-in switch chip (if such exists on your device), fast forward uses the CPU to forward packets. When comparing throughput results, you would get such results: Hardware offloading > Fast Forward > Fast Path > Slow Path.

If packets are processed by Fast Path, then Fast Forward is not active. Packet count can be used as an indicator of whether Fast Forward is active or not.

Disabling or enabling fast-forward will temporarily disable all bridge ports for settings to take effect. This must be taken into account whenever changing this property on production environments since it can cause all packets to be temporarily dropped.

It's all there in plain sight in colored boxes with alert icons.

Fast Forward has many conditions before taking effect but when it does ARP learning stops; I learned the hard way.

13

For this topic, where VLAN Filtering is the goal, Fast Forward’s status is irrelevant, it would be inactive anyway.

14

When omitted from

/interface bridge

the

fast-forward

value defaults to

yes

so adding

fast-forward=no

is at least prudent and sometimes necessary.

15

As written in manual, for fast-forward to actually kick in there are a number of prerequisites which have to be fulfilled. One is “Bridge VLAN Filtering is disabled”, the other is “Bridge has only 2 running ports” and another one is “MAC address for the bridge matches with a MAC address from one of the bridge slave ports” (this one is not fulfilled if one follows rule #6 of The twelve Rules of Mikrotik Club).

So it’s very unlikely (but not impossible) to actually see fast-forward in action. Not in context of this thread.

16

Yes, it’s quite unlikely indeed but we build VLAN Filtering one command at a time and setting

vlan-filtering=yes

should occur after defining

bridge port

and

bridge vlan

steps otherwise filtering with incomplete port & vlan setup risks getting locked out. Setting

fast-forward=no

during VLAN Filtering setup is prudent as ARP Learning disabled happened accidentally here and took substantial time to diagnose. Telling new VLAN Filtering users to set

fast-forward=no

saves everybody time and grief.

17

Is anyone else’s mind blown with the surrealism and irony going on with this thread?

I started it to (1) vent and (2) beg for help due to the massive cognitive-pain being caused by VLANs. Now, this thread has evolved to a discussion/argument about minutae I did not know existed.

Part of me loves it (super funny, if you’ve got that kind of odd sense of humor); the other part is so ready to give up (again) on VLANs with the (same as always) conclusion that VLANs are way too difficult, complicated, confusing and just not worth it.

18

You’re asking questions … and there are two ways of answering, each appropriate to two distinct goals of asking those questions:

  1. simple “yes” or “no” or perhaps a simple “cookbook recipe” … good if person asking wants to get things done as quickly as possible … or if person only wants a confirmation
  2. extensive explanation … required if person asking is eager to learn and understand things, so the answer not only consists of answer/description, but also explanation of why certain things are done in some way and not in another.

I’ve got impression that your questions fall into category described in bullet #2. However, when the question is from #2 category, it takes extra effort (principally on side of person asking) to properly understand and digest answer. New knowledge is almost never “brought on a platter” … And it takes effort on side of person answering to adjust the level of details to knowledge of person asking. The problem doing it is either that person answering isn’t able to explain things plainly or (as in this particular case) one is answering not only to person asking, but to many other readers (who can then steer discussion away from original goals of original poster) and due to those many readers things might have to be explained into more details (which might be too much for OP at the moment of reading the answer but they might become understandable after continued Q&A session).

19

VLAN Filtering has many options (hence choices to make) where some matter little but a few have major impact leading to failure if chosen poorly.

My responses to your posts are stripped down to what I found are the critical options showing the values for each command and step in the build sequence to avoid the major pitfalls.

My responses to other posts on specific options emphasize the reasons supporting choices made from hard experience for VLAN CRS305, CRS309, CRS326, CRS326, hAP ax3 fleet.

I suggest trying to integrate everyone’s opinions into a single perspective is counter productive at this time as:

  1. my exposition to you is sparse by design to avoid overwhelming and far from complete now
  2. giving all opinions equal weight is unrealistic as in real life some opinions are just better
  3. weighing opinions without substantial evidence at hand is impossible

I suggest reconsidering your goals here and provides us with guidance on what are constructive paths that work for you.