RouterOS bridge mysteries explained

This topic is a reaction to the countless cases of confusion regarding why the bridge “itself” must be listed among tagged or untagged ports of a VLAN in some cases, why the VLAN interfaces must be attached to the “bridge” rather than to its member ports etc. The fact that both “interface” and “port” have multiple meanings in the networking vernacular contributes to that confusion.

This topic does not deal with the details of the “new” way of handling multiple VLANs on a single bridge; for that, a great topic by @pcunite exists.

So, let’s start from a picture of a hardware setup:
router_and_switch_hw.png
Here, everything is clear – there’s a switch to take care about the L2 forwarding and VLAN filtering, and to one of the physical Ethernet ports of that switch, a router is connected using one of its own physical Ethernet ports.

We can tell the switch which VLANs are permitted on each port or, in another words, which ports are members of each VLAN. Some flavor of the Spanning Tree Protocol may run on the switch to prevent L2 loops caused by wrong wiring. And on advanced models, some filters may control forwarding of frames from the ingress port to the egress one(s).

On the router, we can assign an IP address directly to the physical interface; if we want the router to have accesss to multiple VLANs, we could use as many physical interfaces as needed and connect each of them to an access port to that VLAN on the switch, but a better approach is to connect the physical interface of the router to a trunk port of the switch and attach a virtual VLAN interface to the physical one for each VLAN. The virtual VLAN interface picks frames tagged with the right VLAN ID from those that ingress through the physical interface, untags them, and forwards them to the routing engine. In the egress direction, the frames the routing engine sends via this interface get tagged with the right VLAN ID and egress via the physical interface.

We can use a slightly more abstract diagram to represent the same arrangement:
router_and_switch_schematic.png
This diagram may still represent the topology shown on the first picture, only the perspective is slightly different. The abstraction allows us to forget about the hardware nature of the functional components and concentrate on their functionality alone. Even if we implement most of it in pure software, the roles of the functional components do not change, i.e. there is still the router whose one interface is connected to one port of the switch, the switch itself, and the other ports of the switch.

Now the software module called “bridge” in RouterOS implements three of the elements above: the switch-facing interface of the router, the router-facing port of the switch, and the switch itself.
The drawing below illustrates that:
router_and_switch_schematic_bridge.png
So far pretty clear, isn’t it?

And now the confusing points.

Since, when you add a “bridge”,

  • all the three elements (interface of a router, port of a switch, and the switch itself) are created,
  • the relationship among the three is a fixed one,
  • none of the elements has got any configuration parameter that would overlap with a configuration parameter of the remaining two,

the configuration parameters for all the three elements have been grouped on a single row of the /interface bridge table.

In particular:

  • parameters bearing the same names as those on the /interface bridge port rows, such as pvid or ingress-filtering, are parameters of the router-facing port of the virtual switch
  • parameters bearing the same names as those on the /interface ethernet rows, such as mtu or arp-timeout, are parameters of the switch-facing interface of the router
  • parameters that don’t fit to any of the two groups above are mostly parameters of the virtual switch; an exception are the admin-mac and auto-mac parameters, which are also parameters of the switch-facing interface of the router.

The name item is common for all the three elements. So when specifying the membership of the router-facing port of the switch in VLANs, the same name is used to identify the virtual switch and to identify the router-facing port:
/interface bridge vlan
add bridge=bridge-row-name vlan-ids=10,… tagged=bridge-row-name,…

In the bridge filter, the relationship of the frame to the router-facing port of the switch determines the chain to be used: input handles frames that egress from the virtual switch through that port, output handles frames that ingress to the virtual switch through that port, and forward handles frames that bypass that port.

I hope you add information about creating VLANs by new way… and how live with General vlan id:1 and others as Trunk.
For now I use always this howto: https://mbum.pl/archive/vlan-po-nowemu.pdf (Presentation from Ihor Hreskiv at Mikrotik Beer User Meating at Poland 2019, mbum.pl)
Pages: 34 - 50

I can take picture from that PDF and create a Photo-story here. I hope you can do that way edition of your post - this will be greate have it in English.


BTW, It’s always good to setup with every bridge a Admin. Mac Address.


Now, the first bridge have a HARDWARE ACCELERATE feature, means we should use one bridge and on it use many vlans

What is missing in this topic about the subject of “VLANs the new way” that I should repeat it here?

This thread should be as first post of your “Using RouterOS to VLAN your network”.
In my opinion people not use that grate vlan article because they must go into .rsc file and LEARN / ANALYZE that all stuff.
Here the Photo Story with description for easy examples to reproduce will be great.

History point…
Creating vlans at every ether port and assigning all of them to one dedicate bridge-vlan100 and repeat for every vlan is like ros5 doing.
Next people try learn Switch menu for HW acceleration.
Next are this new way when we use one Bridge with VLAN tab.
Who know, we will have next generation of doing vlans in ros7 or maybe few of them, who know.

Ahh you mean at the beginning of pcunites thread!!!

@sindy,

To my understanding, what you describe as the switch-facing interface of the router, and the router-facing port of the switch is actually what enables the switch to communicate with the CPU of the Device (Management Access, Layer 3 Services and more ), the communication line of the switch with the Routing Block ( Switch - > CPU Port ).

So the Bridge consists of the Switching Block and the Switch-CPU Port.
Am i missing something ?

It’s unfortunately not that simple. The internal Ethernet link between the physical switch chip and the CPU is not a 100 % functional equivalent of the link between the router-facing port of the virtual switch and the virtual-switch-facing port of the router. VLAN frames from wireless and L2-tunnel interfaces can flow to and from the virtual switch even if the router-facing port of the virtual switch is not a member one of the respective VLAN. But the /interface vlan attached to the virtual-switch-facing interface of the router can only send/receive traffic to/from the virtual switch if the respective VLAN is allowed on the router-facing port of the virtual switch.

Yes, but how can you not include the CPU Port in the Bridge Entity ?

I think that gist of @sindy’s description while it’s convenient to think of the “bridge” as being the L2 access to “CPU port”, like it for “/ethernet switch”. And, this simplifying assumption likely works in most cases – e.g. you probably need to add the bridge as a VLAN port in Bridge VLANs if you’re doing L3 stuff too. But that’s not exactly true in all cases. And, it’s also a counter-intuitive configuration: treating the bridge as a port in the VLAN tab, while a lot of times necessary, isn’t natural esp. if you think of “classic” access port/truck terms.

So the “router side” and “bridging side” way of describing “under the covers” of the “new VLAN method” is actually a helpful way if you’re trying to understand the “why you need to add the bridge as port”. @pcunite VLAN manifesto describes the various options, some good practices, and mechanics, BUT not the why.

And @SiB is also right, some pictures next to packet traces/“L2 frames breakouts” is awful helpful way of presenting it. While the Vinn diagram shows the multi-faceted roles, it doesn’t actually help understanding the relationships.

Easily. The bridge is a software thing, and so is the router, so both run “inside” the CPU, hence the traffic between the “software switch” facing “software port” of the “software router” and the “software router” facing “software port” of the “software switch” never passes across the CPU port. Think of CCR or CHR where no switch chips exist and bridging is nevertheless possible.

Hardware-wise, the CPU port ↔ switch chip port connection is inside the bridge, not between the bridge and the router. A frame coming from the wire first hits the Ethernet port of the switch chip, then it gets sent via the CPU-facing port of the switch chip to the physical-switch-facing port of the CPU, and only then it gets to the “software router” facing “software port” of the “software switch” (or doesn’t, if it instead gets forwarded to the “software switch” port to which a wireless interface or an L2 tunnel interface are connected).

Internally, the “hardware accelerated bridging” in fact means that one of the ports of the full-software bridge is the switch-chip-facing port of the CPU, so frames that can be forwarded directly between Ethernet ports of the switch chip don’t ever get to the CPU. But the aim of the concept is not to bother the administrator with these under the hood details and let them see it as a single bridge with both physical and virtual ports.

Ok @sindy i agree…

the CPU port ↔ switch chip port connection is inside the bridge

So it is part of the Bridge Entity…

I don’t think it matters how things are implemented to the last detail. Surely implementation is different on devices with switch chip from implementation on devices without switch chip. Only MT developers can tell how exactly are things implemented under the hood.

What matters is conceptual design and that one is same for all devices. Understanding concept is key to using bridge correctly and effectively. And first post of this thread explains concept quite nicely. Even if I personally agree with explanation by @sindy this doesn’t mean it’s entirely correct. Too bad nobody from Mikrotik chimed in to add any details (or even only to tell that @sindy’s explanation is spot on).

AFAIU the CPU-Port is called the same as the bridge. This is very confusing, MT should have named this “CPU-Port” or something.

Short explanation:

  • If the CPU-Port is set untagged, this is the only way to communicate with the Bridge Interface (itself) and services “behind”, like DHCP, NTP… In this state “bridge1” is CPU-Port and Interface.

  • If the CPU-Port is set tagged, you will need a dedicated VLAN-Interface - bound to the bridge. But you wont be able to connect to the Bridge Interface itself! Services behind are reached through this dedicated VLAN-interface via the CPU-Port (called “bridge1”, but this refers to the CPU-Port only - not to the Bridge as dedicated Interface).

Long explanation:

Default settings without any VLAN-Interfaces, the Bridge is CPU-Port and Interface at the same time:
Screenshot 2022-01-02 204430.jpg
Screenshot 2022-01-02 204731.jpg
DHCP will work:
Screenshot 2022-01-02 205738.jpg

Same settings, changing to “tagged only” results in a total connectivity lost:
Screenshot 2022-01-02 204916.jpg
Now, the ingress-setting filters all tagless/untagged traffic, but the Frames get tagless/untagged forwarded to the Bridge-Interface+CPU-Port (as defined in the first picture).


Now we change the VLAN-setting, placing the CPU-Port into tagged state:
Screenshot 2022-01-02 205959.jpg
At this point the connectivity is lost as well!

For my understanding the bridge itself (as Interface AND CPU-Port) can work only untagged.
Why was the connectivity lost? Via the PVID the ingressing frame gets its (internal) tag. Via the VLAN-tab the bridge/CPU-Port is egressed tagged as well… I see no other conclusion, the bridge-interface+CPU-Port works only untagged.

Now, we enable/create a VLAN-Interface, bound to the bridge:
Screenshot 2022-01-02 210612.jpg
And suddenly, in a mysterious way, everything works again:
Screenshot 2022-01-02 210525.jpg
To me, it seems the way you place the bridge/CPU-Port into tagged or untagged state triggers something:

CPU-Port/bridge = untagged → in this state, the only working settings (in the bridge/VLAN) menu are “admit all” and “admit only untagged…”. This refers to the Interface and the CPU-Port.

CPU-Port/bridge = tagged → in this state, the bridge-interface itself wont work anymore (because the frame to it are tagged). The only working settings are “admit all” and “admit only tagged…”. But they refer to the CPU-Port only (as the Bridge-Interface isnt reachable anymore) → VLAN-Interfaces bound to the bridge.

Screenshot 2022-01-02 211252.jpg
Screenshot 2022-01-02 211332.jpg
And this is very confusing to me and to all I have spoken! I dont understand why MT has created such a “Frankenstein-like” monster as their VLAN-Implentation. I have never seen such a complicated way (and undocumented too) by any other vendor.

Id recommend simply not to deal with the Bridge as dedicated Interface. Simply create VLAN-interfaces, bound them to the bridge and see the “bridge itself” as CPU-Port (under Bridge → VLAN).

I think that before v7.20 we will know next generation of “new vlan setup”, we should create some versioning for that.
PS. Nice post.

Well yes,
If you want to use the Bridge Interface itself and the Bridge CPU port for Management access then you must use untagged Traffic.
If you want to allow only Tagged Traffic to the Bridge Interface then you must create a VLAN Interface and access the CPU Port through that ( and ofcorse the appropriate VLAN configuration is needed ) …

@Guscht, the “CPU port” is an oversimplification, based on an assumption that CPU is equivalent to the router process and that the bridge process runs somewhere else than on the CPU. If this simplification helps you understand the concept, stick with it, but actually the “port of the virtual switch” and “interface of the virtual router” form up the demarcation line between the bridge process and the router process, which both run at the CPU.

This confusing blending of multiple logical objects into a single one is not an invention of the Mikrotik team - this is how bridge is implemented in Linux kernel and represented in its configuration layer.

What I likely haven’t stressed enough in my OP is that the /interface vlan configuration is related to the router part, and the /interface bridge vlan configuration is related to the virtual switch part. So there is nothing surprising on the fact that if a tagless frame carrying an IP packet arrives to an Ethernet port of a virtual switch with pvid=N, it will only reach the router process if either the pvid of the “router-facing port of the virtual switch” is also N, or if an /interface vlan with vlan-id=N is attached to the “bridge-facing port of the router”. In the former case, the in-interface as seen by the router will be the “bridge-facing port of the router” itself; in the latter one, it will be that /interface vlan.

What I find annoying is that you cannot set pvid to none to get all VLANs tagged at the respective port, i.e. there must always be some “native vlan” associated to each port (including the “router-facing virtual port of the virtual switch”).

Yup that’s a problem & very cleaver/clean solution. Wish MT did that.

If you set ingress-filtering=yes frame-types=admit-only-vlan-tagged on a port it removes the untagged membership, i.e. the pvid= setting is ignored, making the port tagged-only.

Wow, thank you, it seems I haven’t checked the behaviour for a long time… actually, it seems it’s even not necessary to activate the ingress-filtering to make the port accept frames tagged with the VID which is set as the pvid of that port, as adding the interface explicitly to the tagged list on /interface bridge vlan row for that VID prevents/supersedes the automatic addition of the port to the untagged list.

[me@myTik] > interface bridge port print detail where interface=ether5
Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload
 0     interface=ether5 bridge=bridge priority=0x80 path-cost=10 internal-path-cost=10 edge=auto point-to-point=auto learn=auto horizon=none hw=no auto-isolate=no restricted-role=no restricted-tcn=no
       pvid=3333 frame-types=admit-all ingress-filtering=no unknown-unicast-flood=yes unknown-multicast-flood=yes broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no
[me@myTik] > interface bridge vlan print where vlan-ids=3333
Flags: X - disabled, D - dynamic
 #   BRIDGE                                                     VLAN-IDS  CURRENT-TAGGED                                                   CURRENT-UNTAGGED
 0   bridge                                                     3333      bridge
                                                                          ether5

Of course, it does make sense to use ingress-filtering to prevent tagless frames from being accepted on that port (and probably tagged with the VID stated in its pvid).

So it should even be possible to handle the “mysterious” VLAN 1 on the bridge using an /interface vlan, by adding “bridge” (the router-facing port of the virtual switch) to the tagged list on the /interface bridge vlan row with vlan-ids=1.

@sindy,
That is documented in the manual.. When frame-type=admit-only-vlan-tagged is used on a port, then the port is not dynamically added as an untagged port for the PVID.

Source:
https://help.mikrotik.com/docs/display/ROS/Bridge+VLAN+Table