SXTsq 5 ax PTP bridge with multiple VLANs, is my configuration correct?

Hi!

In the past I have been using two Ubiquiti NBE-5AC devices as a transparent bridge between 2 buildings. I need to pass multiple VLANs, one of them being the management VLAN, so for performance reasons, as per the manufacturer's recommendation I set them up to work in bridge mode and using the web interface i gave them IP addresses from the pool of my management VLAN, with the actual "management VLAN" option turned off. On each side on the switch port I'm stripping away the VLAN tags of the management traffic and leave the rest alone. This is working well.

I decided to replace these devices with two SXTsq 5 axs running ROS 7.22.3. I intend to keep the old devices as cold backup, so i would like to set up my new devices to be a drop-in replacement. This is the (relevant part) of the configs i came up with:

AP:

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=wifi1
/interface bridge vlan
add bridge=bridge1 tagged=ether1,wifi1 vlan-ids=2-4094
/ip address
add address=192.168.50.27/24 interface=bridge1 network=192.168.50.

The client is basically the same with only the IP address being different.

At first glance this seems to work as expected, I can access both devices and both tagged and untagged traffic is going through. As I'm not familiar with the MT world I'd like to ask whether this is the correct way to set this up.

Thank you!

It is correct way.

Some people on this forum (myself included) would set up management on tagged VLAN (but some other manufacturers require management on untagged) and mostly the same people would set up list of actually needed VLANs instead of "2-4094". But these details are probably up to anyone's personal preferences.

@mkx
Sorry, but I have to ask, won't adding VLAN's 2-4094 be similar to "all".

And what happens in this case to VLAN 1?

I am struggling to understand the difference between this setup and one without tagged VLAN's (and no VLAN filtering).

Thank you for the information and your suggestions!

It's been a while since I set this up on the Ubiquiti devices but I believe I did it this way because their documentation stated that the management traffic being untagged reduces CPU load for all traffic passing through the bridge. Everywhere else in my network the management traffic is tagged.

As for the 2-4094, i'd like the bridge to resemble a wired connection as much as possible, I can filter what I want to pass through on the switches that the Sxtsqs connect to. Setting up the actual VLANs required would further complicate the configuration without any benefit i can see, other than being neat.

Something I dislike as a newcomer is that lots of things are implied in ROS.

In this case the ether1 and wifi1 interfaces have PVID 1, so - as far as i understand the docs - the traffic coming in on these interfaces becomes VLAN 1. That's fine for me because i don't use that anywhere else, as it is the default on many devices and might create unintended connectivity if I'm not paying attention.

As for your other question, initially i wanted to set things up that way but I'm not sure it wouldn't create unintended leaks, eg. the management interface being available from tagged VLANs. I tried both and didn't see a performance difference, so i went with the config i was more comfortable with.

As you mentioned, "all" woukd cover also VID 1, which is by default used as PVID for all bridge ports (including bridge cpu-facing port). Setting it as tagged (while other devices, connected to SXT, are configured with "native VLAN"), would break communication via this (otherwise untagged over wire) "VLAN" because it would make device to egress tagged frames with VID 1 (setting frame-types=admit-all and pvid=1 - these are default settings BTW - would still correctly allow untagged frames on ingress).

The difference between such setup and setup without any VLAN settings is similar to difference betwern managed switch with VLAN setup and dumb switch (without VLAN setup). VLAN-less setup would work but wouldn't offer any control over where certain VLAN's can go. Admitedly setup with pair of SXT used instead of UTP cable (so no actual L2 decissions, more or less just media converter) doesn't need much control ... apart from protecting management interface from evil LAN users.

It's not implied, it's default setup ... which might be slightly hidden.

E.g. when running /interface/bridge/export those pvid=1 settings are not shown ... however running /interface/bridge/export verbose it will be there.

I'd even say that ROS requires quite a bit more explicit configuration than some other NOSes. For example: ROS's IP layer can only work with untagged interfaces. In your particular setup that's bridge interface due to its (hidden) setting of pvid=1. If you'd want management IP on some other VLAN, you'd have to (explicitly) create vlan interface (with appropriate vlan-id setting) and then assign IP address to that vlan interface. And, BTW, that's why I wrote about all VLANs tagged ... it makes explicit which VLAN is used for IP communication.

I don't know about the other vendor, but ROS doesn't suffer from performance degradation of VLAN-tagged traffic depending on tagged/untagged management.

Maybe I am wrong, but I see a PtP (not a PtMP) link as Mikrotik "friendly name" "Wireless Wire".

I.e. a replacement of an otherwise extremely long and difficult to put on ground ethernet cable.

In this sense just like you have no vlan filtering on the physical copper cable (vlan tagging, filtering and what not is set on the devices at either ends of the cable not "on the cable") , I have difficulties in understanding why it is used in this case.

As I see it, the two devices in such a setup are not even "switches", but more like "connectors" and - in case - I would rather explicitly set a VLAN for management - though personally - but again I might well be wrong - I would see "safer" to have both devices with no IP and use Winbox MAC to access them. :woozy_face:

Wires don't have management interfaces that need to be available to me but not to others. I did explicitly set a management VLAN, de untagged one.

I don't see how MAC access is safer than IP acces in a separate management VLAN. It would also make remote access more complicated.

I don't know if it is, surely it won't be accessible by anything but something using Winbox-MAC protocol.

But right now your management is on the (default) VLAN 1.

If I get it right, your snippet (commented by me):

/interface bridge
#use vlan-filtering (default is vlan-filtering=no)
add name=bridge1 vlan-filtering=yes frame-types=admit-all

/interface bridge port
# pvid=1 is default, but explicitly stating it ensures untagged traffic works
add bridge=bridge1 interface=ether1 pvid=1
add bridge=bridge1 interface=wifi1 pvid=1

/interface bridge vlan
# 1. This allows your untagged management traffic to reach the router CPU (bridge1)
add bridge=bridge1 untagged=bridge1,ether1,wifi1 vlan-ids=1

# 2. This safely trunks all other tagged VLANs (2 through 4094) across the wireless wire
add bridge=bridge1 tagged=ether1,wifi1 vlan-ids=2-4094

/ip address
# Now this will work, because untagged traffic (VLAN 1) can talk to the bridge CPU
add address=192.168.50.27/24 interface=bridge1 network=192.168.50.0

Allows packets belonging to ALL tagged VLANs (2-4094) and the single not explicitly tagged (but tagged implicitly) to "pass through" the wireless wire" link.

But - and here is my doubt - as I see it there is no functional difference with a setup like:

/interface bridge
#Do not use vlan-filtering (default)
add name=bridge1 frame-types=admit-all

/interface bridge port
add bridge=bridge1  interface=ether1
add bridge=bridge1  interface=wifi1

/ip address
# Plain IP address assignment (on default VLAN 1)
add address=192.168.50.27/24 interface=bridge1 network=192.168.50.0

in the specific case of a "wireless wire" link.

When talking about tagged and untagged, normally we talk about tags present or not present on "outer" side of device (i.e. on UTP cable or radio). And in this case the "single not explicitly tagged" should read "and single explicitly not tagged" ... because it's not tagged (outside of device).

BTW, in your "commented" configuration snippet you ommited to mention another (IMO very important) setting: ingress-filtering (yes | no; Default: yes) ... it does affect how traffic is allowed to ingress ... admittedly doesn't affect operation of SXTs in this particular case due to allowing all VLANs to pass.

Other than that: as I already wrote, in this particular case there's no difference between proper "tagged" setup and the short "untagged" one.
My personal preference though is to configure all network infrastructure devices (and endpoints of PtP link are network infrastructure devices) properly ... if network includes VLANs then all network infrastructure devices should be properly configured for VLANs. And configuration shown in opening post by @nyebi IMO isn't entirely proper (e.g. leaving default setting of frame-types=admit-all on bridge port in this case is wrong as bridge port is supposed only to pass untagged frames on CPU side (because there are no vlan interfaces anchored off bridge port).

But - and here is my doubt - as I see it there is no functional difference with a setup like:

I did a quick test and it looks like that if I leave VLAN filtering off broadcast traffic (eg. ARP requests) coming in on the tagged VLANs can be sniffed on bridge1. If it's on then they don't seem to show up.

Stuff like this is what I meant when I said I'm unsure about the implications of leaving it off.

Yes, you are right about all this. Originally i had everything set up as you suggest with the Ubiquiti devices, but after a firmware upgrade a warning saying "For best PTP performance it is recommended to use PTP bridge mode without any VLAN interfaces, without a bridge firewall and with MTU less or equal to 1500 bytes." popped up on the WEB interface. That's when I changed my config to what it is right now, and that's what I'd like to keep for compatibility's sake.

You are right about the bridge port setting (although there is no way for tagged traffic to show up there in this case), details like this is why I posted this question in the first place.

It's always hard to get things right if one starts from "allow all" and later blocks traffic which is troublesome. It's often easier to start with restrictive setup (only allow what needs to be allowed, block everything else) as it becomes obvious what the missing traffic is. People mostly apply this principle to firewall setups, but it can apply also to VLAN setup.

In this particular scenario, setting up SXTs without any VLAN settings would count as "allow all" setup. And setting things up with VLAN settings (and particularly if allowed VLANs are only those actually expected to pass the SXT-SXT bridge) would count as "allow what's necessary, block everyting else".

Ok, so - all this said - which is/are the - if not "undoubtedly proper" at least "usually advised" configuration(s) in this specific case of the OP?

As I see it, the posted original configuration, and as well the simpler alternative I proposed suffer from the same "defect"[1] (independently from their different level of complexity), in both cases the "management" is on VLAN 1 (please read as "no VLAN")

[1] if it is a defect[2].

[2] Personally but I am admittedly a careless, unscrupolous and reckless (small) network manager, living in the conviction that evil maid attacks are the result of security experts fantasies, that if someone can physically connect to the hardware it is game over and that if "they" are after me, they already pwn me, I would mitigate risks by removing the IP and using MAC Winbox for management of the two devices.

As I already wrote in my initial reply: setup is more or less fine.

About the "less" parts:

  • as @nyebi stated: he wants the pair of SXTs do be drop-in replacement for his current gear. As his current gear insists on untagged/native for management, he can't really go for all-tagged setup, so hte PVID=1 stuff has to remain.
    What can be done is to "convert" bridge CPU-facing port to all-tagged with "VLAN ID 1" being tagged (I can hear @anav screaming now :wink:). Like this:
/interface bridge
add name=bridge1 vlan-filtering=yes frame-types=admit-only-vlan-tagged
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=wifi1
/interface bridge vlan
add bridge=bridge1 tagged=ether1,wifi1 vlan-ids=2-4094
add bridge=bridge1 tagged=bridge1 untagged=ether1,wifi1 vlan-ids=1
/interface/vlan
add interface=bridge1 name=vlan-1 vlan-id=1
/ip address
add address=192.168.50.27/24 interface=vlan-1

To avoid using VID 1 as tagged (and keep @anav happy), one could set PVID on ether1 and wifi1 interfaces to some VID which is curently not used in LAN (e.g. 99) and then configure bridge as tagged member of that VID ... and create vlan interface with vlan-id appropriately set. This would make VID 99 "native" (keep in mind that "native" VLAN is untagged outside each individual device, can be different on devices on both end of physical connection and can be different on each bridge port ... so using VID 99 as native on SXT wouldn't break the "drop-in replacement" requirement).
But it's largely cosmetics (serves mostly to move management IP address from bridge (with its multiple personalities) to VLAN interface). So I'd probably skip this part if the "drop-in replacement" requirement makes @nyebi 's life easier.

  • I'd definitely limit VLAN range, permitted to pass SXT bridge, to some sensible list. Depends on actual needs, but something like
/interface bridge vlan
add bridge=bridge1 tagged=ether1,wifi1 vlan-ids=2,3,6,99

As to the "defect" part: personally I'm not the most paranoid person on the block. But I do tend to make things as secure as possible while avoiding degradation of usability and/or performance. Both suggested changes don't degrade anything, they do add to security ... with first bullet being questionable as being on the border of unnecessary complication (it is however preparation for dedicated tagged management VLAN which, when introduced, would require minimal changes in PVID/VID/tagged/untagged config of bridge ports if VLAN ID would be he same as selected "in advance").

I know that I asked for it :woozy_face:, but this configuration is exactly the kind that should (IMHO) not be made public/suggested, as it is blatantly breaking Rules #1 and #2. :astonished_face:

Beginners (and also many long-time but non-really-expert members) may believe that with this they can tame VLAN1 behaviour, and use it in other configurations successfully, which is unlikely.

This seems to be a good idea and the VLAN id should probably be 50 as that is my management VLAN everywhere else in the network anyway.

As for the other part about limiting the VLANs that can pass, I still don't see much benefit there as it is already done on the switchports the two SXTsqs connect to. On the other hand, if I want to add another one it's two more devices that require configuration changes and all the documentation and backup requirements that come with it.

I also don't take the whole thing too seriously, it's just an - admittedly "slightly" overcomplicated - home setup.

Everywhere else in the network the management VLAN is tagged 50. On the two ports these two devices are attached to I strip away the tag of egress traffic and add it for ingress. I could only keep it tagged 50 by sacrificing performance, which is something I don't want to do.

VLAN 1 is not allowed on either switchports. On the two Sxtsqs the ethernet interfaces are not part of tagged VLAN 1, so even if I misconfigure the two ports on the switches frames tagged with VLAN id 1 would be dropped. Only the untagged traffic would be allowed, which can only come from VLAN50 and leave to VLAN 50, as that is set as "native" on both switchports.

Yes, this isn't pretty or neat but as far as I understand still secure.

The problem with default use of VLAN ID is that it will break things in such scenario.

Port with PVID=1 and with frame-types=admit-all will allow tagged frames with VID=1 on ingress, they won't be dropped. But frames with VID=1 will be stripped of tags on egress. So net effect would be that device would accept either tagged or untagged frames but would only transmit untagged frames. Setting ingress-filtering=yes (which is default setting) doesn't help as port is member of VLAN 1 this way or another (regardless the presence or absence of VLAN tag on a particular ethernet frame).
And this is the reason for avoiding VID=1 if possible ...

And actually the same is true for any native VID ... but VIDs other than 1 will be explicitly shown in configuration export while VID=1 is not.

So my suggestion: since you're using tagged VLAN 50 as management VLAN on other LAN gear, I'd set up SXT for tagged management VLANs as well. But you'd have to set both switch ports on both sides as tagged for VLAN 50 (you wrote that currently those ports are access for VID=50), otherwise it would not work as explained in paragraph above.
But beware that this change would diverge from your current setup which you said you'd like to have drop-in replacement.

BTW, AFAIK tagging/untagging costs slight work from switch port so having tagged management VLAN would actually increase switch port performance. As to SXT it's similar on the wired port side of bridge (adding/removing VLAN tags), while on CPU-side of bridge it's the same (with vlan-filtering enabled all frames passing bridge - the switch like entity - will be tagged (with PVID value if none other) and for L3 those tags will have to be added/removed ... either by bridge port is set with PVID or by vlan interface). Since these tag manipulations will be done only for management traffic, I sincerely don't think it would cause any kind of observable difference.