Using WifiWave2 to bridge two Audience wirelessly, thoughts? == 4-address mode

Amm0 · Sun Nov 14, 2021 8:42 pm

Trying the wifiwave2 package on the Audience using v7.1rc6. My goal was to bridge two Audience devices using the 3rd 5Ghz "mesh" interface of the devices, while using newer wireless package on the "client" Wi-Fi interfaces. Basically want to test just bridging one LAN between the two units wireless; with the non-mesh Wi-Fi also just bridged to same LAN. Ideally being able to use VRRP on the bridge, so either LTE interface could provide internet if separated. This is obviously possible using standard Wi-Fi package, but was trying to see if possible someway using wifiwave2.

At the start, I actually though this be straightforward: set "station-pseudobridge" mode on "mesh" wi-fi interface on one of the Audiences, and I'd be done. Assuming all interfaces are just bridged on both units. But there is only "station" mode in wifiwave2, so not so simple. Certainly using ap on one, station on another does just connect. But the "mesh" wifi interface are ports of the bridge interface, the ONLY traffic that passes are the "Neighbors"/CDP broadcast but ARP goes unanswered so anything IP fails. Changing the ARP mode of the "mesh" wifi interface had no effects – tried them all (proxy-arp, local-proxy-arp, enabled, & disabled even) , in all combos on both sides, just in case. I would have though at least the far end of the wifi link would work, but if the connected router does not respond to requires for it's own IP. Basically it seems once a wifiwave2 "station" goes into a bridge, ARP doesn't work, at all.

Removing the "mesh" wifi interface from the bridge, assigning IPs and routing between the two Audience does work - so the "station" to "ap" does seem to work at 802.11-level. But my goal is bridge ONE L2 LAN between the two units, not just have them do L3 routing between them.

Not detoured, and curious to see if there was at least someway to make this work. My next idea was to use a EoIP tunnel (sorta like DTLS tunnels used CAPsMAN) over the non-bridge "mesh" interface, then add the EoIP tunnel to the bridge. This does in fact work, but obviously MTU is squashed. I ran into another issue with wifiwave2 in that it seemingly doesn't allow the MTU to be changed (or perhaps the UI/CLI doesn't correctly reflect the change) – for my testing, I just changed the EoIP tunnel to 1500 MTU.

Obviously the EoIP tunnel isn't ideal and I suspect I've run into bugs/limits of the wifiwave2 package. So perhaps this is just a feature request for "station-pseudobridge" in wifiwave2 package.

Be also curious if I'm missing something/someway to bridge between two Audience using wifiwave2, to eventually eat some CAKE

.

I'll post the configuration, but basically I think there are still some bugs/missing things in wifiwave2...

The "Hub" Audience (e.g. "wifi" mesh interface is in AP mode):

# nov/14/2021 10:31:11 by RouterOS 7.1rc6
# software id = AC0K-0Z8R
# model = RBD25GR-5HPacQD2HPnD
/interface bridge
add name=bridge1
/interface lte
# SIM not present
set [ find ] allow-roaming=no name=lte1
/interface eoip
add comment="tunnel using non-bridged \"mesh\" wifi interface to another Audie\
    nce in station mode, so bridge1 is connect between an in-range peer" \
    local-address=192.168.71.1 mac-address=02:23:06:EB:8C:53 mtu=1500 name=\
    eoip-wifimesh remote-address=192.168.71.72 tunnel-id=71
/interface vrrp
add comment="use VRRP on bridge1 so only one default route for \"mesh bridge\"\
    \_(if peer cannot connect or is off, each unit still operates independentl\
    y)" interface=bridge1 name=vrrp-bridge1 priority=99
/interface list
add name=WAN
add name=LAN
/interface wifiwave2 security
add authentication-types=wpa2-psk,wpa3-psk disable-pmkid=yes name=mobileskyfi
add authentication-types=wpa2-psk,wpa3-psk disable-pmkid=yes name=noproblem
add authentication-types=wpa2-psk,wpa3-psk disable-pmkid=yes name=ztsecure
/interface wifiwave2 configuration
add channel.skip-dfs-channels=all .width=20mhz country="United States3" name=\
    M171 security=mobileskyfi ssid=M171
add channel.skip-dfs-channels=all country="United States3" name=SkyTower \
    security=noproblem ssid=SkyTower
add channel.skip-dfs-channels=all country="United States3" hide-ssid=yes \
    name=SkyBridge security=ztsecure ssid=SkyBridge+SkyTower
/interface wifiwave2
set [ find default-name=wifi1 ] configuration=M171 configuration.mode=ap \
    disabled=no name=wifi-2.4Ghz
set [ find default-name=wifi2 ] configuration=SkyTower configuration.mode=ap \
    disabled=no name=wifi-5Ghz
set [ find default-name=wifi3 ] arp-timeout=30s configuration=SkyBridge \
    configuration.mode=ap disabled=no l2mtu=1600 mtu=1550 name=wifi-MeshAP
/ip pool
add name=dhcp ranges=192.168.0.201-192.168.0.249
add name=dhcp_pool1 ranges=192.168.71.201-192.168.71.249
/ip dhcp-server
add address-pool=dhcp interface=vrrp-bridge1 lease-time=1h name=dhcp1
add address-pool=dhcp_pool1 interface=wifi-MeshAP name=dhcp2
/port
set 0 name=usb1
/interface ppp-client
add apn=internet name=ppp-out1 port=usb1
/zerotier
set zt1 comment="ZeroTier Central controller - https://my.zerotier.com/" \
    identity="xxx" name=zt1 \
    port=9993
/zerotier interface
add instance=zt1 mac-address=2E:D1:09:E9:A3:35 name=zerotier1 network=\
   xxx
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2
add bridge=bridge1 interface=wifi-2.4Ghz
add bridge=bridge1 interface=wifi-5Ghz
add bridge=bridge1 interface=eoip-wifimesh
/ipv6 settings
set disable-ipv6=yes
/interface list member
add interface=lte1 list=WAN
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=wifi-2.4Ghz list=LAN
add interface=wifi-MeshAP list=LAN
add interface=wifi-5Ghz list=LAN
add interface=bridge1 list=LAN
add interface=vrrp-bridge1 list=LAN
add interface=eoip-wifimesh list=LAN
add interface=ppp-out1 list=WAN
add interface=zerotier1 list=LAN
/ip address
add address=192.168.0.71/24 interface=bridge1 network=192.168.0.0
add address=192.168.0.1 interface=vrrp-bridge1 network=192.168.0.1
add address=192.168.71.1/24 interface=wifi-MeshAP network=192.168.71.0
/ip cloud
set ddns-enabled=yes
/ip dhcp-server lease
add address=192.168.71.72 client-id=1:c4:ad:34:86:57:ae comment=\
    "for M172-Rizi station (routing & EoIP need fixed IP address)" \
    mac-address=C4:AD:34:86:57:AE server=dhcp2
/ip dhcp-server network
add address=192.168.0.0/24 comment="bridge1 LAN (via VRRP)" dns-server=\
    208.67.222.222,8.8.4.4 gateway=192.168.0.1 netmask=24
add address=192.168.71.0/24 comment=\
    "PtMP Hub network for using \"mesh\" wifi interface without bridging" \
    dns-server=208.67.222.222,8.8.4.4 gateway=192.168.71.1
/ip dns
set servers=208.67.222.222,8.8.4.4
/ip firewall filter
add action=accept chain=forward comment="allow zerotier" in-interface=\
    zerotier1
add action=accept chain=input comment="allow zerotier" in-interface=zerotier1
add action=accept chain=input protocol=icmp
add action=accept chain=input connection-state=established
add action=accept chain=input connection-state=related
add action=accept chain=input comment="always allow winbox" connection-state=\
    related dst-port=8291 protocol=tcp
add action=drop chain=input in-interface-list=!LAN
add action=drop chain=forward comment=\
    "invalid connections should not go out to LTE" connection-state=invalid \
    log=yes log-prefix=lteinvalid out-interface=lte1
add action=drop chain=output comment=\
    "invalid connections should not go out to LTE" connection-state=invalid \
    log=yes log-prefix=lteinvalid out-interface=lte1
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip packing
add disabled=yes interface=eoip-wifimesh packing=compress-all unpacking=\
    compress-all
/ip route
add check-gateway=ping comment=\
    "static route to M172-Rizi \"station\" in \"mesh\" wifi" disabled=no \
    distance=12 dst-address=0.0.0.0/0 gateway=192.168.71.72 pref-src="" \
    routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/ip service
set telnet disabled=yes
set ftp disabled=yes
set ssh disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ip upnp
set enabled=yes
/ip upnp interfaces
add interface=bridge1 type=internal
add interface=lte1 type=external
/system clock
set time-zone-name=America/Los_Angeles
/system identity
set name=M171-Yizi
/system logging
add topics=debug
/system routerboard settings
set auto-upgrade=yes cpu-frequency=auto
/tool graphing interface
add
/tool netwatch
add down-script=\
    ":log warn \"netwatch ping over local mesh NOT responding within 100ms\"" \
    host=192.168.71.72 interval=1s timeout=100ms up-script=\
    ":log info \"netwatch ping over local mesh responding within 100ms\""
add down-script=\
    ":log info \"netwatch ping to Google NOT responding within 250ms\"" host=\
    8.8.8.8 interval=1s timeout=250ms up-script=\
    ":log debug \"netwatch ping to Google responding within 250ms\""
add down-script=\
    ":log error \"netwatch ping to Google NOT responding within 500ms\"" \
    host=8.8.8.8 interval=1s timeout=500ms up-script=\
    ":log info \"netwatch ping to Google responding within 500ms\""
/tool sms
set port=lte1 receive-enabled=yes
/tool sniffer
set file-limit=10000KiB file-name=wifi-m171.pcap filter-interface=wifi-MeshAP

The "Station" Audience (e.g. "wifi" mesh interface is in Station mode):

# nov/14/2021 10:30:48 by RouterOS 7.1rc6
# software id = 8QEA-KKXK
# model = RBD25GR-5HPacQD2HPnD

/interface bridge
add name=bridge1
/interface lte
set [ find ] allow-roaming=no band="" mtu=1480 name=lte1
/interface eoip
add comment="tunnel using non-bridged \"mesh\" wifi interface to another Audie\
    nce in AP mode, so bridge1 is connect between an in-range peer" \
    local-address=192.168.71.72 mac-address=02:B4:CF:BA:59:9F mtu=1500 name=\
    eoip-wifimesh remote-address=192.168.71.1 tunnel-id=71
/interface vrrp
add comment="use VRRP on bridge1 so only one default route for \"mesh bridge\"\
    \_(if peer cannot connect or is off, each unit still operates independentl\
    y)" interface=bridge1 name=vrrp-bridge1
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] apn=broadband ip-type=ipv4 use-peer-dns=no
/interface wifiwave2 security
add authentication-types=wpa2-psk,wpa3-psk disable-pmkid=yes name=mobileskyfi
add authentication-types=wpa2-psk,wpa3-psk disable-pmkid=yes name=noproblem
add authentication-types=wpa2-psk,wpa3-psk disable-pmkid=yes name=ztsecure
/interface wifiwave2 configuration
add channel.skip-dfs-channels=all country="United States3" name=SkyTower \
    security=noproblem ssid=SkyTower
add channel.skip-dfs-channels=all country="United States3" hide-ssid=yes \
    name=SkyBridge security=ztsecure ssid=SkyBridge+SkyTower
add channel.skip-dfs-channels=all .width=20mhz country="United States3" name=\
    M172 security=mobileskyfi ssid=M172
/interface wifiwave2
set [ find default-name=wifi1 ] configuration=M172 configuration.mode=ap \
    disabled=no name=wifi-2.4Ghz
set [ find default-name=wifi2 ] configuration=SkyTower configuration.mode=ap \
    disabled=no name=wifi-5Ghz
set [ find default-name=wifi3 ] arp-timeout=30s configuration=SkyBridge \
    configuration.mode=station disabled=no l2mtu=1600 mtu=1550 name=\
    wifi-MeshStation
/ip pool
add name=dhcp ranges=192.168.0.201-192.168.0.249
/ip dhcp-server
add address-pool=dhcp interface=vrrp-bridge1 lease-time=1h name=dhcp1
/routing table
add fib name=""
/user group
set full policy="local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,pas\
    sword,web,sniff,sensitive,api,romon,dude,tikapp,rest-api"
/zerotier
set zt1 comment="ZeroTier Central controller - https://my.zerotier.com/" \
    identity="xxx" name=zt1 \
    port=9993
/zerotier interface
add instance=zt1 mac-address=2E:28:C3:96:80:D3 name=zerotier1 network=\
    xxx
/interface bridge port
add bridge=bridge1 comment=defconf ingress-filtering=no interface=ether2
add bridge=bridge1 comment=defconf ingress-filtering=no interface=wifi-2.4Ghz
add bridge=bridge1 comment=defconf ingress-filtering=no interface=wifi-5Ghz
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=eoip-wifimesh
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ipv6 settings
set disable-ipv6=yes
/interface list member
add interface=bridge1 list=LAN
add interface=lte1 list=WAN
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=wifi-2.4Ghz list=LAN
add interface=wifi-MeshStation list=LAN
add interface=wifi-5Ghz list=LAN
add interface=vrrp-bridge1 list=LAN
add interface=eoip-wifimesh list=LAN
add interface=zerotier1 list=LAN
/ip address
add address=192.168.0.72/24 interface=bridge1 network=192.168.0.0
add address=192.168.0.1 interface=vrrp-bridge1 network=192.168.0.1
/ip cloud
set ddns-enabled=yes
/ip dhcp-client
add comment=\
    "PtMP Station for \"mesh\" wifi interface (obtain IP from \"hub\" AP)" \
    default-route-distance=21 interface=wifi-MeshStation use-peer-dns=no \
    use-peer-ntp=no
/ip dhcp-server network
add address=192.168.0.0/24 comment="bridge1 LAN (via VRRP)" dns-server=\
    208.67.222.222,8.8.4.4 gateway=192.168.0.1 netmask=24
/ip dns
set servers=208.67.222.222,8.8.4.4
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
/ip firewall filter
add action=accept chain=forward comment="allow zerotier" in-interface=\
    zerotier1
add action=accept chain=input comment="allow zerotier" in-interface=zerotier1
add action=accept chain=input protocol=icmp
add action=accept chain=input connection-state=established
add action=accept chain=input connection-state=related
add action=accept chain=input comment="always allow winbox" dst-port=8291 \
    protocol=tcp
add action=drop chain=input in-interface-list=!LAN
add action=drop chain=forward comment=\
    "invalid connections should not go out to LTE" connection-state=invalid \
    log=yes log-prefix=lteinvalid out-interface=lte1
add action=drop chain=output comment=\
    "invalid connections should not go out to LTE" connection-state=invalid \
    log=yes log-prefix=lteinvalid out-interface=lte1
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip packing
add disabled=yes interface=eoip-wifimesh packing=compress-all unpacking=\
    compress-all
/ip route
add check-gateway=ping comment="static route to \"hub\" for internet routing, \
    used if local LTE interface is down" disabled=no distance=15 dst-address=\
    0.0.0.0/0 gateway=192.168.0.71 pref-src="" routing-table=main scope=30 \
    suppress-hw-offload=no target-scope=10
/ip service
set telnet disabled=yes
set ftp disabled=yes
set ssh disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ip upnp
set enabled=yes
/ip upnp interfaces
add interface=bridge1 type=internal
add interface=lte1 type=external
/system clock
set time-zone-name=America/Los_Angeles
/system identity
set name=M172-Rizi
/system logging
add topics=debug
/system package update
set channel=development
/system routerboard settings
set auto-upgrade=yes cpu-frequency=auto
/tool graphing interface
add
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool netwatch
add down-script=\
    ":log warn \"netwatch ping over local mesh NOT responding within 100ms\"" \
    host=192.168.71.1 interval=1s timeout=100ms up-script=\
    ":log info \"netwatch ping over local mesh responding within 100ms\""
add down-script=\
    ":log info \"netwatch ping to Google NOT responding within 250ms\"" host=\
    8.8.8.8 interval=1s timeout=250ms up-script=\
    ":log debug \"netwatch ping to Google responding within 250ms\""
add down-script=\
    ":log error \"netwatch ping to Google NOT responding within 500ms\"" \
    host=8.8.8.8 interval=1s timeout=500ms up-script=\
    ":log info \"netwatch ping to Google responding within 500ms\""
/tool sms
set port=lte1 receive-enabled=yes
/tool sniffer
set file-limit=10000KiB file-name=wifi-m172.pcap filter-interface=\
    wifi-MeshStation

Couple side notes:
1. The MTU docs don't specifically mention the Audience: https://help.mikrotik.com/docs/display/ ... n+RouterOS
2. Out of curiosity, I'm not sure ip/packing does anything in v7.1rc6 - though it be radically different way to clamp L2 MTU and/or reduce packet sizes, curious at CPU/latency for the compression on something more powerful like the Audience. But didn't seem to work, even though the interfaces I tried were all "neighbors".

gdanov · Tue Nov 16, 2021 3:29 pm

I don't have wifiwave2 device so I can't check if there's really no station-bridge mode. You should avoid station-pseudobridge. For 2 nodes using mesh is overkill.

Best improvization I can think of is EoIP tunnel over the wifi connection using the IPs you have assigned to the interfaces. Just be mindful that EoIP + parallel IP path may cause weird effects, maybe you want to filter all IP traffic between the station and the AP and leave only the EoIP one.

Amm0 · Thu Nov 18, 2021 3:57 am

I don't have wifiwave2 device so I can't check if there's really no station-bridge mode. You should avoid station-pseudobridge. For 2 nodes using mesh is overkill.

Best improvization I can think of is EoIP tunnel over the wifi connection using the IPs you have assigned to the interfaces.

Well, I was kinda hoping I just might be able to drop the wi-fi interface on both sides into the bridge, and have ROS "do the right thing" (e.g. infer some wifi bridging mode) based on the "station" being a bridge port member – no such luck. But @gdanov station-bridge is closer to what I was expecting exist in wifiwave2.

Just be mindful that EoIP + parallel IP path may cause weird effects, maybe you want to filter all IP traffic between the station and the AP and leave only the EoIP one.

That's a very fair concern - hadn't dissected the packet flow diagram. But my bigger concern is I'm not sure the wifiwave2 MTU can go above 1500. This is a problem since any tunnel has overhead, with EoIP +42. Since so for the "wireless bridge" between the two Audience units to avoid fragmenting/cramping-MSS, the wifiwave2 interfaces needs >1542 MTU.

The photo above shows my concern, which appears to a bug in v7.1rc6 – winbox shows the "Actual MTU" as "1500". But I can't find "actual-mtu" in the CLI, so hard to know if just a "winbox things"...

But since I also tried using a VxLAN approach, instead of EoIP, it also thinks the MTU of the wavewifi2 interface is 1500, so it limits the VxLAN MTU to 1450. The following CLI snippet shows my MTU concerns, since EoIP would need a higher L2MTU (>1542) over the "wifi bridge" to avoid reducing MTU on either side:

[skyfi@M172-Rizi] /interface/vxlan> export
# nov/17/2021 17:43:05 by RouterOS 7.1rc6
# model = RBD25GR-5HPacQD2HPnD
/interface vxlan
add group=239.11.37.100 interface=wifi-MeshStation mtu=1450 name=\
    vxlan-skybridge port=8472 vni=100
[skyfi@M172-Rizi] /interface/vxlan> print detail 
Flags: X - disabled, R - running 
 0 R name="vxlan-skybridge" mtu=1450 mac-address=EA:D5:A5:1A:C5:73 arp=enabled 
     arp-timeout=auto loop-protect=default loop-protect-status=off 
     loop-protect-send-interval=5s loop-protect-disable-time=5m vni=100 
     group=239.11.37.100 interface=wifi-MeshStation port=8472 
[skyfi@M172-Rizi] /interface/vxlan> set 0 mtu=1500
failure: could not set mtu
[skyfi@M172-Rizi] /interface/vxlan> set 0 mtu=1451
failure: could not set mtu
[skyfi@M172-Rizi] /interface/vxlan> set 0 mtu=1449
[skyfi@M172-Rizi] /interface/vxlan> set 0 mtu=1450

Now so far the EoIP tunnel over WifiWave2 does seem to work & seems stable from logs. In fact, since only one has an active LTE connection to internet, the 2nd audience needs to wireless link to get to the internet. I've also had ZeroTier enabled on both Audience in my testing here, that seems to be work the few times I tried from my Mac over the internet (with ZeroTier Mac client connected to same ZT network).

But the MTU issue is my first concern – but what's going with the [un]packing of the EoIP tunnel in the bridge is worth more look too. Basically, I think there are still some bugs in WifiWave2...

mducharme · Thu Nov 18, 2021 4:34 am

They need to add "four address frame support" into wifiwave2 before it will support bridging.

Amm0 · Thu Nov 18, 2021 9:30 am

They need to add "four address frame support" into wifiwave2 before it will support bridging.

No doubt, seemingly why only "station" is supported - more testing to find these things out

.

But the MTU should be settable >1500 since 802.11 MTU is >2000. In my use case, being able to trunk VLANs over EoIP (or using a set of VXLANs to carry individual VLAN using Bridge "VLAN Filtering") to tunnel inside a ap-to-station wi-fi link isn't necessarily the worst approach – it's just I rather not have the MTU compressed/fragmented by the tunnel, especially since 802.11 has a higher possible MTU than being allowed by the wavewifi2 package.

gdanov · Fri Nov 19, 2021 10:24 am

They need to add "four address frame support" into wifiwave2 before it will support bridging.

oh, obvious reason.

tapir · Mon Jan 10, 2022 12:44 pm

This can't come soon enough.. I invested in 3 audiences and regretted it since.
Cannot get more than around 160Mbps between them even if they are in the same room!

bpwl · Mon Jan 10, 2022 2:45 pm

Is it feasible to use MPLS/VPLS for better performance than EoIP? It is often done in PtP to have higher speed.
MPLS/VPLS needs a larger MTU, or the inner MTU should be reduced accordingly. (A smaller MTU, leading to smaller MSS, does not lead to fragmentation and is not so bad for performance).

viewtopic.php?t=35330

Dutch video: https://www.wirelessinfo.be/mikrotik-po ... p-vs-mpls/

Just thoughts !

Amm0 · Mon Jan 10, 2022 9:08 pm

Is it feasible to use MPLS/VPLS for better performance than EoIP? It is often done in PtP to have higher speed.
MPLS/VPLS needs a larger MTU, or the inner MTU should be reduced accordingly. (A smaller MTU, leading to smaller MSS, does not lead to fragmentation and is not so bad for performance).

viewtopic.php?t=35330

Dutch video: https://www.wirelessinfo.be/mikrotik-po ... p-vs-mpls/

Just thoughts !

MPLS+VPLS, now that's cleaver for a PtP link. Didn't try, but had a wacky idea to use IP>Packing to further reduce IP headers with EoIP but wasn't that crazy.

I have a couple of Audience that we don't use since the cell antennas aren't that great. I've been using them to testing WW2 etc in V7. Basically bridging them without WDS/four-addresses I took as a Mikrotik puzzle. Since wireless bridging audience isn't a critical use case, was kinda looking more looking for thoughts on the problem since it was kinda curious what to could be done without WDS.

But the Dutch article/video was interesting (Google's English version). I'd have refresh my memory about MPLS etc. too – but will try at some point.

But yeah the MTU problem makes it more challenging, as the Dutch MPLS+VPLS PtP article points out:

MTU must be adjusted at MPLS / VPLS configuration to avoid fragmentation.
When the Mikrotik SXT encapsulates the Ethernet frame to send over VPLS pseudowire, it checks if the packet size + VPLS CW + MPLS tags exceed the MPLS MTU of the outgoing interface. If so, VPLS will fragment the frame. In this example, 1514 byte layer2 packets are forwarded over VPLS, router adds CW (4 bytes) and one MPLS tag (4 bytes). To avoid fragmentation, the MPLS MTU should be increased to 1522

It's the current WW2 limit of 1500 MTU throws a wrench in either EoIP or MPLS (or even VXLAN): the network-wide MTU still have to lower than 1500 (e.g. 1478 it seems with MPLS+VPLS).

I did report the MTU being limited to 1500 as a bug. Wi-Fi MTU is certainly complex, but I know it's always >1500 & so even with the saving from MPLS, it's still more than 1500 limit in WW2 today. The case is still open, so I've taken it that it's in queue to be fixed. As I said, this isn't critical for me, and imagine other bugs lurking WW2. Now WDS, that seem like a missing feature, so hard to know when that will come.

But MPLS was an idea I'd missed here, specifically VPLS I hadn't thought about.

bpwl · Sun Jan 16, 2022 1:28 am

Maybe some more protocols to explore, besides EoIP and VPLS

BCP, will split the packet over multiple PPTP channels to create a bigger MRRU to allow for the full MTU ? : https://wiki.mikrotik.com/wiki/Manual:B ... _bridging)

When bridging packets PPP tunnel need to pass packets with Layer-2 (MAC) header included , so default interface MTU (in case of pptp it is 1460) is not sufficient for this task. To ensure proper operation itbis suggested to override the value by specifying MRRU option in server settings to a higher value.

MRRU allows to enable multi-link support over single link, it divides the packet to multiple channels therefore increasing possible MTU and MRU (up to 65535 bytes)

.
VXLAN: yet another protocol ? https://help.mikrotik.com/docs/display/ROS/VXLAN (ROS 7.1)

mducharme · Sun Jan 16, 2022 5:49 am

I use VPLS and EoIP and BCP and VXLAN all the time. All are overcomplicated for this scenario and not suitable workarounds for that reason. We need four address mode support.

Amm0 · Sun Jan 16, 2022 7:52 am

We need four address mode support.

Don't disagree, but right now the bigger bug is MTU being limited to 1500 on the wifiwave2.

@mducharme, when you say "four address mode", to me that's just the 802.11 data structure used to carry the add'l MACs – how it should be used is a better question in V7. V6 used it a few different ways however... Are you advocating for "station pseudo-bridge", or standard WDS, or maybe HWMP+?

Personally I'd prefer Wi-Fi 6 support, which isn't possible on the Audience. Which is kind why I'd rather see Mikrotik spend some time to think about new/better/cleaner ways of wireless bridging until they come out with 802.11AX hardware...

mducharme · Sun Jan 16, 2022 8:08 pm

WDS at the very least, if not station-bridge.

station-pseudobridge I don't believe uses four address mode at all. It only supports bridging one MAC and my understanding is that it is just simulated bridging.

Amm0 · Sun Jan 16, 2022 10:36 pm

WDS at the very least, if not station-bridge.

station-pseudobridge I don't believe uses four address mode at all.

Yup. I’m hopeful, something, will come soon. Not an expert on the driver/kernel stuff but imagine WDS is covered by Linux stuff - they’d may just need to add the UI. I’m guessing what ever bug is causing the MTU may be standing in the way.

i ask since personally , I’m not sure all the V6 options aren’t need - the mesh vs WDS, pseudo vs regular, etc seems like too many modes. I always have to go back to wiki/help every time to recall which mode does what

bpwl · Mon Jan 17, 2022 2:46 am

wiki/help every time to recall which mode does what

Don't think you have a problem with these modes ....

(not wifiwave2)

"AP -bridge" (bridge function (4-address) can be disabled, WDS function is included), "bridge" is the level 3 single connection version of AP-bridge
"Station - bridge" the only real station setting. "Station" can be used to not request the bridge (4-address) function from the AP.

Normally that's all one needs. Bridge (4-address) function is great for PtP and PtMP wifi links between MT devices.

WDS is not needed for 4-address connections between MT.
WDS with "station-WDS" as WDS client, or "WDS slave" as WDS client+AP function.

Pseudobridge is a MAT (MAC address translation) L2.5 bridge. Has a mapping table for multiple IPv4 addresses for one receiving MAC (wifi is 3-address, MAT expands to multiple IPv4 clients)
(MT DHCP usually has problems with this). Is one-way for setting up connectios, because the table has to be built Not recommended. Is a typical function in non-MT "repeater" cases.

Waiting for 4-address bridge function in wifiwave2, or it is not suitable for PtP/PtMP.

mducharme · Mon Jan 17, 2022 3:21 am

WDS works with other vendors with 4 address mode though, if you make it an open AP without security. In some cases this can be a reasonable compromise even for PTP across vendors (ex. where you are encrypting everything over the link through some other means like IPsec anyway). It is the only way to get true bridging between MikroTik and non-MikroTik devices. I've used this where I have had to do a phased migration from third party radios to MikroTik. Obviously when WPA2 is involved WDS bridging only works between devices of the same vendor. station-bridge is preferable from MikroTik to MikroTik though, probably slightly more efficient than WDS.

BitHaulers · Tue Nov 01, 2022 1:53 am

I haven't seen it mentioned in the change logs, but I'm assuming we still don't have 4 address/bridging available in the Wave2 drivers?

mkx · Tue Nov 01, 2022 7:45 am

You are right, 4-address mode is still not available with wifiwave2 drivers.

BitHaulers · Thu Jan 12, 2023 6:06 pm

4-address mode must be a very low priority.

Three months later, and not a peep about it.

Amm0 · Thu Jan 12, 2023 6:57 pm

4-address mode must be a very low priority.

Long enough that I don't even have the test setup anymore.

My business partner took one as a travel router, and a customer has been borrowing one for months. The Audience is a nice device and the WifiWave2 has worked flawless as a single AP, which why I don't have them back

. Have a few wAPac's but worthless for testing WW2. Been waiting for newer"ax" devices for a while...

But anyone know if the MTU issue been fixed? It was limited to 1500, when I last tested it.

anav · Thu Jan 12, 2023 7:30 pm

Hi Ammo, you need a liferaft not an ax3 ;-P
Mine came in yesterday still have not had a chance to play........
I know nothing about how to setup its wifi, do I need this wifiwave 2 crapola...........

Amm0 · Thu Jan 12, 2023 7:38 pm

I know nothing about how to setup its wifi, do I need this wifiwave 2 crapola...........

If it's the only AP, it works great. It was night and day on the Audience compared with "stock" drivers. Suspect be same on the ax3's (which I'm waiting for)

Now I'm 100% sure you're not going to like the "CAPsMAN for everyone" way you configure WifiWave2 drivers....

mkx · Thu Jan 12, 2023 10:03 pm

Normis explained a while ago that ax devices can only run wifiwave2 "crapola" ... not supported by legacy driver at all.

So let's brace ourselves while waiting for @anav to axe that ax3

bpwl · Thu Jan 12, 2023 10:12 pm

Is the ax hardware getting some MT legacy driver options, sooner than the ac hardware?

Release notes 7.7rc5:
*) wifiwave2 - added option to set per-client vlan-id in access list (only supported on 802.11ax interfaces) (CLI only);

anav · Thu Jan 12, 2023 11:23 pm

hi bpwl there are many CLI only wifiwave doo dahs noted on the 7.7 release.
Do you have clear way ahead for a standard setup of one hap ax3 using these things.....
I was hoping MT was going to make things clearer but it seems worse now LOL.

bpwl · Fri Jan 13, 2023 12:05 am

@anav, No idea how to do the "wifiwave2 cli only", have no MT AX device yet. Just the release notes hint it's different. Forum members said they didn't find it.

Amm0 · Fri Jan 13, 2023 1:34 am

hi bpwl there are many CLI only wifiwave doo dahs noted on the 7.7 release.

It actually be good to know if QuickSet does something reasonable on the ax3. There just aren't as many options in the WW2 drivers without the missing "4 address mode" (e.g. WDS, bridging modes, etc)

.

BitHaulers · Tue Feb 07, 2023 11:57 pm

Bumping thread cuz I want this to be visible to the staff.

marekm · Tue Feb 21, 2023 4:59 am

+1
Please implement station-bridge/ap-bridge and WDS in wifiwave2, making sure it is inter-operable over the air with the older driver too (at least in plain 802.11 if nstreme/nv2 are too difficult with the new code base).
The fully transparent bridge mode (without ugly "MAC address NAT" hacks) is an important feature with many good uses, from extending home WiFi networks to WISP PtMP setups. While not a true WiFi standard, it is standard enough that I have sometimes used "static mesh WDS" for connecting UBNT M5 series stations to MT APs (802.11 only as the proprietary TDMA protocols are not compatible, but WPA2-PSK AES encryption works fine, and connected stations appear as bridge ports so all features of the new bridge implementation can be used). Actually, it would be nice if ordinary WiFi stations could appear as bridge ports as well - then you don't need a separate bridge implementation hidden inside the wireless driver in AP mode, just re-use the same existing fully-featured one that can do VLAN filtering, R/M/STP, bridge firewalling etc.

gd_19 · Mon Mar 13, 2023 11:54 am

+1
I've been waiting for this feature for a long time too. Perhaps at some point mikrotik staff could hint at whether this is on the roadmap at all? I understand it's difficult to commit to an implementation date, but if there is no intention of introducing this functionality to wifiwave2 it would be great to share this information with users.

emunt6 · Sun Mar 19, 2023 6:49 pm

https://help.mikrotik.com/docs/display/ROS/WifiWave2

Lost features

The following notable features of the bundled wireless package do not yet have equivalents in the wifiwave2 package

Station-bridging or other 4-address modes
Nstreme and Nv2 wireless protocols

Your only option left, change firmware to OpenWRT.

Amm0 · Sun Mar 19, 2023 7:21 pm

I do hope that isn't the end of the story here. But "meshing" is the unique feature in the Audience... And yes it's poor choice today: Wave2, no mesh, OR use the "old" wireless package and have a mesh...

VXLAN: yet another protocol ? https://help.mikrotik.com/docs/display/ROS/VXLAN (ROS 7.1)

I do have a couple hAPax3 I have been testing, so tried the VXLAN approach. The hard limit of 1500 MTU is still present which is still my personal biggest grip here. So this any alternative approaches will result in fragmentation. I'd actually be fine with some tunnel over wi-fi — but I don't want MTU to get squashed by the 1500 MTU limit in wifiwave2.

Since I was also curious how well AX does in 2.4GHz, as these hAPax3 were my first AX devices. In theory it should perform a lot better with congestion than ac...so I created a PtMP link using the hAPax3's "wifi2" interface that carried a VXLAN tunnel over the 2.4Ghz AX interface.

Summary is a VXLAN does work. And, yes, it does get unnecessarily fragmented but "works" nonetheless. Basically you (from a winbox POV):
- take wifi2 out of bridge on both
- one becomes get set to "station", other remains "ap"
- set SSID "AX 2488 Mesh" on both (or whatever)
- assign 172.22.88.1/24 as IP for "wifi2" on ax3 "ap" (or whatever unique subnet you want)
- assigned 172.22.88.2/24 as IP for wifi2" on the "station" ax3
- create VXLAN interface using defaults on both
- hit "VTEP" button to link the VXLAN to the IP address of the far end Wi-Fi interface's IP (e.g. on AP, 192.22.88.2... on one/more stations, the VTEP address is 192.22.88.1)
- add VXLAN to bridge as a port (and set PVID if vlan-filtering=yes) on both

FWIW, ZeroTier also works to bridge a network over Wi-Fi PtP/PtMP here too. The key is the Wi-Fi interfaces have to have routable L3 addresses, just like for EoIP or VXLAN, for the tunnel endpoints.

ormandj · Sun Apr 16, 2023 7:18 pm

Add me to the list of people waiting on this functionality. I was just trying to set this up today with an ax2 I purchased to extend a network for radio equipment and ran into this limitation; I did not want to deal with the complexity of another network, but here we are.

sirbryan · Fri Apr 28, 2023 9:20 pm

Summary is a VXLAN does work. And, yes, it does get unnecessarily fragmented but "works" nonetheless. Basically you (from a winbox POV):
- take wifi2 out of bridge on both
- one becomes get set to "station", other remains "ap"
- set SSID "AX 2488 Mesh" on both (or whatever)
- assign 172.22.88.1/24 as IP for "wifi2" on ax3 "ap" (or whatever unique subnet you want)
- assigned 172.22.88.2/24 as IP for wifi2" on the "station" ax3
- create VXLAN interface using defaults on both
- hit "VTEP" button to link the VXLAN to the IP address of the far end Wi-Fi interface's IP (e.g. on AP, 192.22.88.2... on one/more stations, the VTEP address is 192.22.88.1)
- add VXLAN to bridge as a port (and set PVID if vlan-filtering=yes) on both

Thanks for the writeup. I got things working between two Audiences for a customer, who wanted to take advantage of the greater bandwidth offered by the newer drivers.

hafte · Wed Jun 07, 2023 4:50 pm

It works. Thanks.
I have sucessfully briged two hAP AX3 routers to extend home network.

One remark is to check on brige ports and include VXLAN in LAN. This can save you a lot of time wondering why it doesn't work.

I had issues with FW rule "Drop All Not Coming From LAN" as connections were dropped.
- DHCP was not working on station wifi clients due to this.
- manual IP set was also not working.

crowfeatherjoe · Thu Jun 08, 2023 1:55 am

Add me to the list as well of those who hope that 4-address comes back. I have a WISP with hundreds of Audiences in production that I cant take advantage of wifiwave2 on due to this. I will try the VXLAN workaround, but I will not accept this as a solution. Due to the limited MTU and the complexity added by using VXLAN, it will probably never see the field until this is resolved. This is making it very difficult to maintain Mikrotik as a primary vendor for Client-side WIFI.

Amm0 · Thu Jun 08, 2023 2:36 am

One remark is to check on brige ports and include VXLAN in LAN. This can save you a lot of time wondering why it doesn't work.

I had issues with FW rule "Drop All Not Coming From LAN" as connections were dropped.
- DHCP was not working on station wifi clients due to this.
- manual IP set was also not working.

It's actually the "wifi2" that need to go into the LAN interface-list (or tunnel subnet "172.22.88.0/24" needs to be allowed in the firewall) – that part may not have been listed

.

But the VXLAN itself is bridged, so to the firewall the "inner" traffic comes from bridge/VLAN as the interface, not the VXLAN interface. But the "outer" VXLAN packets (with 172.22.88.0/24 address) come via wifi2 which should NOT be in the bridge (but does need to be allowed by firewall).

dad2312 · Mon Jun 12, 2023 12:54 pm

Thank you Amm0 for your work.

Question : is it possible to make this config with 3 audiences in serial ?

AP -> STATION -> STATION

mkx · Mon Jun 12, 2023 2:50 pm

No, station can only communicate with single AP (and not directly with other stations, even if they all connect to same AP). However, this might work: station <-> AP <-> station ... Keep in mind, that from network infrastructure perspective AP doesn't have to be in some certain spot. E.g. in your case, AP can be device without wired connection. In your case it only matters where the L2 tunnel (whatever variety you're using) terminates, it has to terminate on devices which are connected to wired parts of network, in the topology mentioned at the beginning of this post this means the left and right station devices while device in the middle (AP) only serves as transport node (since 4-address mode is not available, you'll actually have two IP segments, AP In the middle being part of both and it will route between both stations).

Amm0 · Mon Jun 12, 2023 5:46 pm

However, this might work: station <-> AP <-> station ... Keep in mind, that from network infrastructure perspective AP doesn't have to be in some certain spot. E.g. in your case, AP can be device without wired connection.

That's what I was going to say. Both station use the "middle" as their VTEP address. VXLAN allows a one-to-many topology.

dad2312 · Tue Jun 13, 2023 8:52 am

Thank you Mkx

it seems to work !

I try with 1 rb4011 (main router with dhcp server) and 3 hapac2 configured with vxlan and all ports bridge except wifi2 and dhcp clients on the bridge (no firewall, nothing else)

station1 (wired) vxlan 172.22.88.1 => receive ip from rb4011 (192.168.....) =>
AP (non wired) vxlan 172.22.88.2 => receive ip via station1 (192.168.....) =>
station2 (non wired) vxlan 172.22.88.3 => receive ip via AP ? (192.168....)

VTEP remote ip station1 = 172.22.88.2 and 172.22.88.3
VTEP remote ip station2 = 172.22.88.1
VTEP remote ip AP = 172.22.88.1

does that seem right to you?

dad2312 · Thu Jul 20, 2023 1:37 pm

Hello everyone

the configuration below works fine but I am experiencing issues with the DFS channel
with the public, the third radio only uses DFS channels. To mesh these 3 audiences, I use the third radio with the same frequency (of course), but I have an environment with radar and at least one audience changes frequency and loses connection with the others

I am trying to set up multiple channels in the third radio with the expectation that if there is a DFS radar all audiences try to connect in another frequency configuration respecting the order, but no, the audience chooses their own frequency and all audiences disconnect.

for now I change the third radio to a 2nd (5ghz) with only channel 5180 (no DFS) and it's ok, but I only have 2 channels (4 channels with third radio) so I lose half my speed.

is it possible to configure with vxlan the third radio with a 2nd backup radio when DFS cuts the first link (like a failover)?

Amm0 · Thu Jul 20, 2023 5:26 pm

is it possible to configure with vxlan the third radio with a 2nd backup radio when DFS cuts the first link (like a failover)?

Sure, but not exactly "easy". VXLAN using Layer3 IP to make the tunnel – so the solutions are roughly the same as having a "dual WAN", except it's a "dual WLAN". So, essentially, more IP address/subnets, and importantly a check-gateway=ping on the "primary" wireless IP route is what's needed (or, for the adventurous, check-gateway=bfd perhaps). Let me think about this one. Maybe others have suggestion?

dad2312 · Thu Jul 20, 2023 7:08 pm

maybe bonding the 2 wlan ?

pekr · Sun Aug 20, 2023 11:56 am

As I did not want to give up on radio, did it a bit differently, seems to work. The difference is in adding a virtual station / AP on both ends

router 1, connected to the internet:
- AP 2.4 GHz
- AP 5 GHz,
- virtual AP 5 - repeater, master AP5, not in bridge

router 2, remote site:
- AP 2.4 GHz
- Station 5 GHz - repeater
- Virtaul 5 GHz AP, master Station 5 GHz, connected to remote virtual AP 5, not in bridge

The rest of config is the same, adding vxlan interface and putting it into the bridge. That way I have both 2.4 and 5 GHz APs on both sides.

Amm0 · Sun Aug 20, 2023 5:04 pm

The difference is in adding a virtual station / AP on both ends

Yeah the approach depends on one be "ap" and one/more being "station" and none being in a bridge directly – but it doesn't matter if any are virtual wireless interface.
Mixing "ap" and "station" seems to be a feature of the wifiwave2 drivers – at least on ax2/ax3. Likely Audience, but didn't test mixing station and AP on same interface on that one.

S8T8 · Thu Aug 24, 2023 9:35 pm

@Amm0

What's new in 7.12beta3 (2023-Aug-24 12:15):
*) wifiwave2 - enable changing interface MTU and L2MTU;

Amm0 · Wed Sep 13, 2023 5:14 pm

Well this one may be of some interest on this topic:

What's new in 7.12beta7 (2023-Sep-13 09:58):

*) wifiwave2 - added station-bridge interface mode (CLI only);

viewtopic.php?t=198723

Kevdevon · Thu Sep 14, 2023 1:58 pm

I'm in the process of setting this up...

One question I have, should the neighbour-groups be shared between both Audiences?
What do I need to do to make this happen, CAPSMAN, neighbour discovery?

I'm finding that devices are becoming confused currently, causing noticeable disconnects when roaming.
This didn't happen with one Audience, the roaming worked seamlessly, as you would expect.

As a side note, seems station-bridge is compatible with Fritzboxes, I've been able to effortlessly setup a bridge between the Fritzbox (wireless) and Draytek (wired) router.

Amm0 · Thu Sep 14, 2023 4:50 pm

One question I have, should the neighbour-groups be shared between both Audiences?

IDK, but that does not sound like a good idea. The station bridge is still a station, so it's not a "neighbour" (it's a "child") & BOTH bridging and neighbor-groups is relatively new features – combo'ing them seems like a bad idea...

BitHaulers · Mon Sep 18, 2023 2:15 am

I've got it working with two Audiences in bridge mode. The command line wasn't obvious. It's interface/wifiwave2>edit wifi3 > configuration.mode and then change the text and ctrl-o to save and close.

If you make changes to the wifi3 tab, it resets it to AP or Station, and you have to edit it again. Which is a bit maddening.

Mon Sep 18, 2023 2:45 pm

If you make changes to the wifi3 tab, it resets it to AP or Station, and you have to edit it again. Which is a bit maddening.

Consequence from this feature only available in CLI for now. Winbox doesn't know about it yet.
It will come.

BitHaulers · Sun Oct 29, 2023 6:24 am

It's live in 7.12rc2

It's in the GUI and works as expected, and I have seen not only a jump in performance, but range, while using WiFi 2 drivers.

The WiFi is also more stable at the edge of its range, unlike the old drivers.

ilium007 · Wed Dec 06, 2023 12:04 pm

Where is meshing at with the Audience running the wifiwave2 package?

pekr · Tue Jan 09, 2024 9:48 am

Sorry to bump-up the topic again, but on the other hand, it is in my opinion related. I successfuly ran vxlan based solution to bridge two ax2s, with 2.4 and 5GHz APs on both sides.

Recently, as 7.13 came out, providing station bridge mode (my understanding is, it is a so called 4-addess support), I have dissmantled the vxlan solution, trying to simplify it, using the station bridge mode on wlan1 5GHz radio for the repeater purposes.

I have tried the following scenarios:

Setting up the wlan1 as an AP and additionaly creating new instance for the station bridge repeater purpose as a slave. Because there is no client at the time connected to the master AP on wlan1, it goes to "sleep". So first - can I prevent this? Second - it does not allow me to add the slave station bridge instance to the bridge.
Reversed the scenario and used master as a station bridge for the repeater purposes, additionaly created a slave 5GHz instance for the AP purposes. But - I can't set frequencies (the tab is missing, probably taking over the master frequency), second - this 5GHz slave AP can't be added into the bridge either.

So - so far, the repeater mode works, but on the router2, there is only 2.4GHz AP (wlan2) and wlan1 serves only the purpose of repeater bridge station. Can it be somehow resolved, before reverting back to the vxlan solution, to allow me to have 2.4GHz + 5GHz APs on both sides?

Thanks,
/Petr

bpwl · Tue Jan 09, 2024 10:49 am

But - I can't set frequencies (the tab is missing, probably taking over the master frequency),

... Yes as with the legacy wlan driver, was also like that.

Not able to add this interface to the bridge ? Unexpected.

pekr · Tue Jan 09, 2024 4:58 pm

But - I can't set frequencies (the tab is missing, probably taking over the master frequency),
... Yes as with the legacy wlan driver, was also like that.

Not able to add this interface to the bridge ? Unexpected.

Using version 7.13.1, when I have added virtual 5GHz AP, it could not be added to the bridge. It also reset master station bridge to just station and disconnected from the main router.

After upgrade to latest beta 7.14beta4, it all seems to work. Will further watch it and if problems reappear, will revert back to vxlan based repeater solution.

Who is online