I’ve got a couple of CRS317’s on my test bench and I’m having trouble working out why they behave as they do. One side of the connection is configured to tag VLAN1, the other side is configured for untagged VLAN1. They are passing VLAN1 traffic in both directions without issue. I would expect them to be dropping packets in at least one direction. But that is not the case.
I run /system/reset-configuration and lay in the following on the clean router:
SFP16 on CRS1 is connected to SFP16 on CRS2 via pig tail. I am pinging between routers and between laptops connected at ether1 on each end. I am expecting CRS1-SFP16 to egress frames tagged 1 and only ingress frames tagged as such. Vice versa with CRS2. While the sniffer output shows frames traveling tagged in one direction and untagged in the other. So, obviously, my expectations are wrong.
I’m hoping someone might be able to point me at what I’m missing. I’ve got about a dozen probably dumb questions. Here’s a few to start: Is tagging a VLAN on the bridge the ‘correct’ way to give a device presence on a VLAN? Am I missing some deeper config related to vlan-filtering? Is the Mikrotik way of isolating VLANs using chain rules; and does that change the hardware offload/have performance implications?
That’s one of the problems with ROS giving you so much control of how things are configured. It will allow you to configure things in ways that are ambiguous while it still applies default values that may not be obvious.
Post the output of
/interface/bridge/print detail
/interface/bridge/vlan/print detail
/interface/bridge/port/print detail
/interface/vlan print detail
Normally the bridge device itself has pvid 1 and the “internal hybrid port between the CPU and the switch ASIC” also normally passes vlan 1 traffic untagged between the CPU and the switch ASIC and the “interface” associated with vlan 1 on the switch ASIC is the base bridge device, not the vlan interface you have named vlan1, and bridge-ports also have pvid 1 if not specified.
So this is my conjecture:
Case when pinging from CRS1 to CRS2:
ping proceess sends icmp ping request to 10.10.10.2
route on CRS1 shows this needs to leave via the vlan1 interface
vlan1 adds tag for vlan1 and sends out parent interface “bridge”
Switch ASIC receives tagged frame for vlan1 and classifies it into vlan 1 on vlan-aware switch ASIC
Switch ASIC forwards frame to sfp-sfpplus16
Switch ASIC has been programmed via /interface/bridge/vlan/add bridge=bridge **tagged=**bridge,sfp-sfpplus16 untagged=ether1 vlan-ids=1 to send vlan 1 as tagged frames, so it sends the frame tagged out sfp-sfpplus16.
Frame arrives on CRS2-SFP16 with vlan 1 tag.
I think this may be one place that you may find confusing, because the “standard” is permissive, and you have not limited the frame types that will be accepted.
The standard allows (at least) three different frames to be received into a specific vlan (vlan 1 in this example)
Tagged frame with vlan 1 (this is the case in this example)
Tagged frame with vlan 0 received on a port with pvid 1 (this is “priority only” tag)
Untagged frame received on a port with pvid 1
Once received and classified (both vlan and priority), any “memory” of how the frame was received is forgotten (at least I am not aware of any switches that remember this, there may be, the IEEE 802.1Q spec does not require the switch to remember whether a frame was received tagged or untagged, at least if it does, I do not know where).
So the vlan 1 tagged frame received on CRS2-SFP16 is classified into vlan 1 by the CRS2 switch ASIC.
The dst mac address is for the CRS2 vlan1 (vlan interface). This is via the bridge port “bridge” and bridge vlan for “bridge” is specified to be sent tagged, so it is sent tagged to the “cpu”.
The cpu receives the vlan 1 tagged frame and sees there is an interface for vlan 1 tagged frames (vlan1).
The vlan1 interface removes the tag and passes to kernel routine to process icmp requests.
Play like you are the CRS2 responding to the ping request and follow the above to figure out how the ping response gets back. (remember to look at the pvids from the output of /interface/bridge/port print detail.
OK I see why CRS2 is able to successfully send untagged frames to CRS1. Default config specifies PVID=1 on CRS1-SFP16. Those frames ingress without tag and are unintentionally, by me at least, being put in VLAN1. From there, the switch just passes the frame to the correct MAC. Which means my config needs to change. The how of it appears connected with the traffic flowing in the other direction.
I suspect you’re hinting at interface/bridge/port/frame-types as a permissive standard. Pair spotty documentation with some testing and my scenario may be debugged. vlan-filtering and ingress-filtering being enabled already, I set frame-types=admit-only-vlan-tagged on both SFP16 to halt the unintended traffic. And then I can set CRS2-SFP16 to permit tagged V1 ingress and the link works again.
This just leaves me with a few more questions. Namely, did I just succeed wrongly? Did I miss a cleaner path to deploying separate data and management networks? Were these devices intentionally designed to handle VLANs in this manner?
I do appreciate the help.
Edit: nearly forgot this output, just in case it’s useful:
The permissive standard I was specifically referring to was the IEEE 802.1Q standard. You can sign up to get access to the latest standard, there is also an old 2003 version of the IEEE 802.1Q spec in pdf format here; in the referenced 802-1Q-2003.pdf document, see E.2.2 Consistent view of the "untagged VLAN(s)" on a given LAN segment on page 293 and 8.9 VLAN classification on pages 51-52. Note that VLAN classification will always use the vlan specified by the tag (as long as the port is a member of the vlan and the frame is not filtered out), otherwise if there is no explicit vlan specified, then it will either be classified by the PVID or by protocol/mac classification. There is nothing that prevents a frame tagged for vlan 1 being classified into vlan 1 if the pvid is 1. However, the single format rule specified in Note 4 of 7.5 Ingress, Forwarding, and Egress Rules on pages 29-30 states that for a specific vlan, a specific bridge port is required to always transmit frames for that vlan using the same format. That seems to me to say that the bridge is not allowed to remember how the frame was received, and therefore not able to transmit as it was received (i.e. tagged or untagged) in the general case. The following is what The single format rule says:
Frames transmitted on a given LAN segment by a VLAN-aware Bridge for a given VLAN shall be either
d) All untagged; or
e) All VLAN tagged with the same VID.
NOTE 4—In other words, a Bridge can transmit untagged frames for some VLANs and VLAN-tagged frames for other
VLANs on a given link, but cannot transmit using both formats for the same VLAN. The single format rule only applies
to the frame transmission behavior of individual VLAN-aware stations; i .e., it does not express a requirement for a
Bridge to police the behavior of the other stations attached to a segment and enforce a single format.
So the standard "allows" one side to send a specific vlan tagged and the other side to send the same vlan untagged (in which case the PVID would be the same as the tag specifies). But that can lead to confusing situations as you have discovered, and configuring it that way is discouraged.
The normal method of avoiding the confusion is to not use vlan 1 for data, either user data or management data. If you want untagged traffic use a pvid other than 1. The other option is to filter based on frame type.
Another problem is that there is not a consistent meaning for terminology by different vendors. Take for example "hybrid". Here is what HP's definition from this document Introduction to port-based VLAN is: (and it is different than MikroTiks definition, and what MikroTik refers to as hybrid, most other vendors refer to as trunk (possibly with one untagged vlan allowed). HP's definition allow multiple vlans to be untagged. How those multiple vlans are classified on receiption is not specified, but it could be protocol or mac based, or perhaps an asymmetric vlan.
Hybrid port—A hybrid port allows traffic of some VLANs to pass through untagged and traffic of some other VLANs to pass through tagged. Usually, hybrid ports are configured to connect devices whose support for VLAN-tagged packets you are uncertain about. As shown in Figure 5, Device C connects to a small-sized LAN in which some PCs belong to VLAN 2 and other PCs belong to VLAN 3, and Device B is uncertain about whether Device C supports VLAN-tagged packets. On Device B, configure the port connecting to Device C as a hybrid port to allow packets from VLAN 2 and VLAN 3 to pass through untagged.