I'm pretty sure I've encountered a bug with WDS. I'm just trying to run a WDS bridge across 2 devices with both devices having 3 SSIDs. I've tried 4 different configs but it seems every time I pass VLANs over WDS it becomes unstable. The fact it can work without vlans seems to indicate I haven't done anything wrong and I've encountered a bug. Basic config attached. I've only included one device as the other is largely the same.



The errors I get are:

disconnected, no beacons received
disconnected, unicast key exchange timeout, signal strength -60

I've read a bunch of posts and they mostly talk about interference. I've removed everything else off channel 6 and I don't have wifi from my neighbors. I've set adaptive noise immunity to "ap and client mode". Signal is pretty good with devices about 10m apart. I do have a reasonable number of wifi clients at home, about 50 but I can get this to work with very basic config so I don't think it's interference. Also when this is failing a reboot of both devices will fix it for a while. After a period of time it will come up with a couple of disconnects, then work for a while but slowly get worse until it's unusable. The issue appears to be related to the time it has been running, not what's going on in the house. When it is working it works well, I used it for teams meetings during the day, no lag no issues etc and can get 60-90Mbit transfer.

Configs I have tried:
1) WDS with 3 SSIDs but all done without vlans, used 3 bridges, 3 WLAN interfaces and 3 WDS interfaces
2) WDS with 3 SSIDs using vlan filtering, each SSID using a different VLAN
3) very basic WDS config, 1 SSID, vlan only on the device connected to the internet
4) WDS with 3 SSIDs using switch vlan, 2 SSIDs on VLANs and master wireless interface not using vlan

Results were
1) Pretty sure this was stable, was the first config I tried and wasn't checking for errors
2) Quite unstable, would last an hour maybe
3) Completely stable
4) Lasts about 6 hours, unusable after 18 hours

The config attached is the config I would like to use (no 4 above). It only has 2 SSIDs simply because that's what I was trying at the moment, 2 or 3 isn't that relevant.

Devices are both 951ui-2HnD running ROS v7.3.1

Any particular reason not to configure devices into ap-bridge and station-bridge modes? The wireless station modes document in part with description of station-bridge states:

This mode is safe to use for L2 bridging and is the preferred mode unless there are specific reasons to use station-wds mode.

Any particular reason not to configure devices into ap-bridge and station-bridge modes? The wireless station modes document in part with description of station-bridge states:

I thought station mode had the 3 mac address issue? Besides that I don't have any specific reason to use WDS

Edit: I do want both devices to provide the same SSIDs, is that an issue?

There is station mode and the there's station-bridge mode, the later is using 4-address mode. Virtual interfaces should not interfere, there is a bridge handling VLAN tags in the packets path which should deal with interface separation (if this part is configured correctly).

Looking at config, I see you're configuring VLANs in both legacy and contemporary way. The former includes configuring vlans in /interface ethernet switch and /interface wireless subtrees and the later includes configuring vlans in /interface bridge subtree. I suggest you to go with contemporary way (you don't loose any performance for non-ethernet interfaces while you gain some simplicity and uniformity of vlan configuration).

Thanks, it looks like it is working. I'll reboot and see if it's still running in the morning. When you say I'm using both vlan modes are you saying that because I have this line below? If so that was just a hangover from when I switched from vlan filtering to switch vlans. I thought "no" was the default but I had since switched that back the yes. From my understanding that config has no effect if vlan-filtering is set to no

add bridge=BRIDGE_MAIN ingress-filtering=no interface=WLAN_BLOCKIOT

As for WDS would there be a reason it's not working? I deleted all the vlan config off the first mikrotik (left side of diagram), rebooted both and it failed within 3 hours. That was a config I had previously thought was successful. So now I'm just thinking WDS has issues. It can take quite a while to determine if a config is good or bad, and I have tried a LOT of different stuff over 2 weeks, so unfortunately I don't have perfect test results.

You're right, the /interface bridge vlan setup is inactive because you don't have vlan-filtering=yes ... so the remnants of that config are only aesthetical problem.

If I was in your place, I'd get rid of WDS ... the text in Mikrotik's own documentation talks about it plenty.

If I was in your place, I'd get rid of WDS ... the text in Mikrotik's own documentation talks about it plenty.

Yeah, more than happy to do that. It looks like station bridge does work. I'm starting to think I should run a dedicated device for the bridge. Having multiple APs on the same frequency with the bridge on the same frequency could just be a flawed idea.

After testing for a couple of days station bridge mode is working very well. It does come up with some errors in the logs like "key exchange timeout" but I'm not sure it's an issue. One of my tests was to leave winbox running and drag it to the left side of the screen and drag a second copy to the right side for the other mikrotik. If winbox disconnects then it changes the window size when it reconnects. After running overnight they had kept their size.

I'm still curious about what the purpose is of WDS. Is it just something that Mikrotik don't test anymore but is left in for some reason? Or is it something that should work? Has anyone had success with WDS?

In theory WDS should be compatible between vendors. In practice it has its share of problems.

WDS has another purpose than station-bridge. It is intended for operation where there are multiple AP's linked together.
It is not intended for transparent VLAN bridiging.
Things are made confusing because the other manufacturer uses WDS as a config name for a transparent link.
Unfortunately all manufacturers have their own specific workarounds and tweaks to the actual specs, and you cannot use knowledge about 1 manufacturer for another...
(and their systems do not correctly inter-operate either)
Hopefully 802.11ax will change all that. We'll see.

Re WDS, Mikrotik ROS refuses to link with any non-Mikrotik device in WDS mode. For more details see: viewtopic.php?t=177451
I have reported this to Mikrotik and they looked into it (sent them Wireshark captures and more info from lab testing), but actual fix was never implemented.
It's a simple software only check, basically ROS device will only establish WDS link with other ROS devices that send Mikrotik specific beacon elements (or join elements).
If these are not present, WDS is not possible.

While WDS was not meant to be used for wireless bridges, it kind of offers standardized way of doing so, in a clean way and without any vendor specific locks.
But sadly many vendors (not just Mikrotik) have to keep adding custom extensions and more to the protocol so in the end even basic static WDS only works in their own AP/CPE ecosystem.

And so instead of having fully standard and compliant way of doing transparent links, we have to deal with ugly NATs and Masquerading just to have wifi repeaters working across devices from multiple vendors... ah... the (still) sad state of wireless networking in 2022...

WDS has another purpose than station-bridge. It is intended for operation where there are multiple AP's linked together.
It is not intended for transparent VLAN bridiging.

I have moved on to use station bridge but I'm still curious about this. My limited testing shows WDS doesn't like VLANs and if you enable VLANs it works perfectly for several hours and then deteriorates until the logs are full of the key exchange timeout error and you get zero traffic. I appreciate that Mikrotik doesn't stop you doing stupid things but curious why WDS doesn't like vlans, surely it is a bug.

I'm also curious for what cases WDS is required. It seems station bridge does the same thing, the only difference would seem to be dynamic mesh

Yes, I think that is what it is for. Making a mesh of several APs that can be used by roaming clients. It is not the preferred way of course, having a separate link infrastructure to interconnect the APs (either wired or a separate PtP link) will perform much better (due to the hidden terminal problem).

VLAN over WiFi is often problematic. Sometimes it works, often it doesn't. There are tricks like ARP snooping that learn the AP how to bridge and they don't properly work when the ARP traffic is VLAN encapsulated.
E.g. with the competitor equipment in WDS mode it is possible to make a VLAN transparent PtP link that works fine, but at one time I made a link that has 2 tagged VLANs (among others) that at a MikroTik router terminated in a single bridge. So the same broadcast traffic was seen on 2 different VLANs.
That TOTALLY CONFUSED the APs. Connections made over one VLAN suddenly broke and the traffic was sent to the other VLAN, even though the client wasn't at all present on that. Fortunately I was able to consolidate the two VLANs into one (it was actually part of a migration) and the problem was resolved. But I did not expect to see this, the VLAN pair was sent over many ethernet trunks between switches without any issue but it was only that single link across the street that had the issue.

But I did not expect to see this, the VLAN pair was sent over many ethernet trunks between switches without any issue but it was only that single link across the street that had the issue.

It's interesting because I feel like I had a similar problem just yesterday where mikrotik was mixing up vlans. I had vlans 3, 4 and 5 setup and I was getting IP from vlan 3 on ports assigned to vlan 4. I spent ages trying to work out what was wrong and then a reboot fixed it. I didn't consider that a big issue as I often find I get problems when making a large amount of config changes while experimenting and a reboot often fixes them.

To be clear, in that case it was with the equipment of the competitor (Powerbeam 5AC ISO GEN2) and to trigger it I have to bridge 2 VLANs together on the router side.
So it is a MikroTik router with an ethernet port configured with several VLAN interfaces, some of them bridged to the LAN, to a VoIP VLAN, etc and two of the VLANs both bridged to the same subnet with NAT to internet. The WiFi link bridges this whole trunk (VLAN bundle) to a second location across the street. Inside the building it worked fine but on the second location users reported abysmal internet performance. Some tracing revealed the problem.
As this was only a step in a migration (to a different VLAN tag) it could be resolved by accelerating the migration and again having separate router networks on each VLAN.

However, it still surprises me that there is a problem and that it can be triggered this way. Why is there no mode where the WiFi link just functions as a plain ethernet bridge/switch where it forward traffic only by MAC address (and a bridge table that tells it on which side each MAC address is), and does not peek into VLAN or network layer stuff. No idea.
I guess when you really want to solve all issues you have to run a EoIP tunnel over the WiFi link. When the WiFi allows a slightly larger MTU (like 1550) that could be done without fragmentation overhead. Of course it would still be less efficient.

VLAN over WiFi is often problematic. Sometimes it works, often it doesn't.

I've done some further investigation and I find the issue with WDS is not related to vlans. I setup a box from blank config with just a single SSID and WDS and it's very unstable. The link drops about 60 times an hour. There's nothing wrong with my environment because station bridge mode is 100% stable if I don't run on SSID on the station side. I think the issues are related to using the same radio for bridge and for an SSID. I'm getting problems with both WDS and station bridge. If I turn off the virtual AP on the station side and run clients wired to the station then it's 100% stable and will run 24hrs without a single link down. If I add a virtual AP with no clients then it's also stable, but if clients connect to the AP then I get "key exchange timeout errors"

Any particular reason not to configure devices into ap-bridge and station-bridge modes?

It seems that I get the same problems with station bridge mode. The root cause appears to be having the same radio being used for uplink and being used for client connections. I've tried all sorts of different configs and the only thing that works is using wired clients on the station bridge side. If I do that the connection is 100% solid. Any wireless clients and the connection is flaky. The station bridge mode was the most stable but it still had dropouts every few hours

A wired uplink is of course always better, but when you do a wireless link I would at least configure a separate virtual wireless (with its own SSID) for the link...
You can then select the proper mode for each of the applications.

A wired uplink is of course always better, but when you do a wireless link I would at least configure a separate virtual wireless (with its own SSID) for the link...
You can then select the proper mode for each of the applications.

I think you misunderstood. I'm comparing running wired vs wireless clients, in both cases I'm using wireless bridge. See diagram below. It seems no matter what I do, using a device with a single radio is a flawed idea. It's fine for the ap bridge side but for the station bridge side if I enable a virtual ap then things fall apart once clients connect. It works ok for a while but will get link downs quite regularly. If I don't enable an SSID on the station bridge and have wired clients then it's absolutely rock solid and will run 24hrs without a single link down.

If I was in your place, I'd get rid of WDS ... the text in Mikrotik's own documentation talks about it plenty.

I've been experimenting with this quite a bit. You are right about WDS, I found it to be very unstable. Station bridge was a LOT better but still had dropouts with the "unicast key exchange timeout" error. I found another solution that works but it is a bit hacky. The solution I found was to connect to the remote AP using station mode and then run EoIP over the top of that. Doing that the connection has been up for 24hrs without a single dropout. I think running a separate radio for the uplink would be better but I'm running off batteries so don't want to double my power usage. Below is cutdown config. I still think this is a bug with WDS and station bridge mode. If it works in station mode and not in station bridge then it can't be interference, which is Mikrotik always blame it on.

station-bridge mode, the later is using 4-address mode

So @mkx even in station bridge mode, 4-address format is used ?
If for example, two clients ( PCs ) need to communicate through a wireless link, e.g. PC -> AP Bridge .... Station Bridge -> PC2, the Frame will include all RA, TA, SA and DA ?
And how does that exactly work ? The client PC will forward first to the AP bridge ?

station-bridge mode, the later is using 4-address mode

So @mkx even in station bridge mode, 4-address format is used ?
If for example, two clients ( PCs ) need to communicate through a wireless link, e.g. PC -> AP Bridge .... Station Bridge -> PC2, the Frame will include all RA, TA, SA and DA ?
And how does that exactly work ? The client PC will forward first to the AP bridge ?

When AP communicates with SB, both use 4-address format. When AP communicates with "normal" clients, both use 3-address format. Use of 4-address format is the only difference between modes station and station-bridge from radio link point of view.

So when PC sends a frame towards PC2, PC-AP will have 3-address frame (source address, destination address and AP's address as radio receiver address).
Then AP will determine that it needs to send frame to SB. It'll use 4-address frame, this time it'll include source address from original frame, own address as radio sender address, destination address from original frame and SB's address as radio receiver address.
SB will see it needs to pass frame via wireless to "normal" client using 3-address frame and will include PC's address as source address, own address as radio sender address and destination address from original frame as destination/radio receiver's address.

Without using 4-address frame between AP and SB the destination address gets lost because according to 802.11 the 3-address format only includes one destination address when sending frame from AP to client (because radio receiver is assumed to be destination).

If PC and PC2 are clients of same AP (either AP bridge or Station Bridge), they both communicate ot same AP and 4-address format is not needed (or, one can hypotheticize that it's used internally in AP's software).

If transferring data wifi device to wifi device and each device doesnt have two radios, then you might as well tie a string to two empty campbell soup cans and talk through that.
Get proper APs for this purpose, or wire or Buy a pair of 60hz devices and put each at the corner of your house pointing at each other LOL.
OR
continue to put a wet noodle up a straw,,,,,,,

If transferring data wifi device to wifi device and each device doesnt have two radios, then you might as well tie a string to two empty campbell soup cans and talk through that.

It'd work if payload is VoIP. I don't think soup cans bare well with video streams though

Actually things are no different if AP-STA connection uses same radio as AP-SB or different one, the AP-SB still needs to carry 4 addresses, difference is how that's done. One can use nv2 over 2.4GHz just fine (as slave interface on AP even) because nv2 has 4-address format supported natively (so does station-bridge, but we can't call that native implementation because it's extension of 802.11 standard). Air-time utilization is a completely different matter though, it's because of that issue we should legaly ban use of WiFi repeaters.

If transferring data wifi device to wifi device and each device doesnt have two radios, then you might as well tie a string to two empty campbell soup cans and talk through that.

I get where you're coming from and I realise it has some major downfalls but end of the day if it works, it works. Other vendors manage to do it without dropouts. I haven't tried it with ubiquiti but I'm sure it would work fine. Google sell their mesh with only 1x5GHz radio. For mikrotik it does work if I use EoIP so it is possible. The fact it works there shows it's just bugs in the software. For the key exchange timeout they could just retry. It seems something about sending 4 MAC addresses causes the issue. If I use EoIP with 3 addresses it works.

As for buying 2 new radios, if this was for home I would, I have 4 Ubiquiti APs at home + have spent plenty of money elsewhere. But this is for a remote property that we don't go to that often. It needs to run off batteries so I don't want to double the power usage. 4W doesn't sound like much but that's 8Ah per day. So you really need a 24Ah battery to cover for cloudy days and 50Ah for 2 devices.

@mkx thanks for the explanation.