Page 1 of 1

RB951 - All wireless clients disconnect simultaneously

Posted: Tue Dec 11, 2012 9:49 am
by richinuk
Hi all,

I'm having a problem with a RB951 where, randomly, all clients are being disconnected. This is happening maybe 2-4 times per day. Once disconnected all clients remain disconnected until one of the following happens:
  • AP is rebooted
  • Wireless interface is shutdown/enabled
  • All registrations are removed
In fact, it seems the last point is the key - once all registrations are removed, clients can reconnect again.

The symptoms are as follows:
  • [1] Randomly, the clients disconnect from the network (in Windows, get a yellow ! on the wireless icon). No other message. Note that it's not just one client - ALL clients are disconnected a the same time
  • [2] If you try to reconnect (on any client) often the SSID is not shown, and when it is shown, windows will fail to connect
  • [3] The contents of the debug logs varies... sometimes there is nothing (not even any references to failed reconnects, disconnects etc). Other times, we get a few lines about key exchange timeout (unsure if it is related).
  • [4] In the registration table, the "last activity" counter for all clients gets higher and higher. TX rate drops and drops to eventually 1.0mbps.
  • [5] Clients are eventually kicked automatically by the router with the message "disconnected, extensive data loss". Or, if manually kicked, the expected message "disconnected, request to disconnect from registration table".
  • [6] Once all clients are removed from the registration table, everything returns to normal and clients reconnect.
It doesn't appear to be signal related. Office is small, AP is centrally located with no obvious obstruction (max 1 plasterboard wall) and signals are between -50 and -65 for all clients. TX/RX around 65mbps to 72mbps.

Here's the debug log for one of the events (I have full debugging on). At the time, there were only 3 devices associated.

A client associates...
09:25:48 wireless,debug wlan1: 8C:70:5A:XX:XX:XX attempts to associate 
09:25:48 wireless,debug wlan1: 8C:70:5A:XX:XX:XX not in local ACL, by default accept 
09:25:48 wireless,info 8C:70:5A:XX:XX:XX@wlan1: connected 
Some NTP stuff...
09:36:17 ntp,debug,packet sending to 91.216.168.42 NTP packet (48 bytes) 
09:36:17 ntp,debug,packet     VN=4 
... snip ...
09:36:17 ntp,debug gradually adjust by 1520c0d 

09:51:17 ntp,debug,packet sending to 91.216.168.42 NTP packet (48 bytes) 
09:51:17 ntp,debug,packet     VN=4 
... snip ...
09:51:17 ntp,debug gradually adjust by 14c1b96 
Then, all clients were disconnected (ether2 link up and route event is me connecting via wired to debug):
09:51:59 wireless,debug wlan1: 74:E5:0B:XX:XX:XX attempts to associate 
09:51:59 wireless,info 74:E5:0B:XX:XX:XX@wlan1: reassociating 
09:51:59 wireless,info 74:E5:0B:XX:XX:XX@wlan1: disconnected, ok 
09:51:59 wireless,debug wlan1: 74:E5:0B:XX:XX:XX not in local ACL, by default accept 
09:51:59 wireless,info 74:E5:0B:XX:XX:XX@wlan1: connected 
09:52:04 wireless,info 74:E5:0B:XX:XX:XX@wlan1: disconnected, unicast key exchange timeout 

09:52:33 interface,info ether2-master-local link up (speed 100M, full duplex) 
09:52:33 route,debug,event Interface change 
09:52:33 route,debug,event     interface=ether2-master-local 
09:52:33 route,debug,event     status=UP,RUNNING 
... snip ...
09:52:33 route,debug,calc End calculation 
I connect from Winbox. One client timesout, I also manually remove another client (the last remaining one)...
09:52:58 system,info,account user admin logged in via winbox 
09:53:01 wireless,info 00:26:C7:XX:XX:XX@wlan1: disconnected, extensive data loss 
09:58:07 wireless,info 8C:70:5A:XX:XX:XX@wlan1: disconnected, request to disconnect from registration table 
Then, everyone can connect again...
09:58:10 route,debug,event Interface change 
09:58:11 route,debug,event     interface=wlan1 
09:58:11 route,debug,event     status=UP 
... snip ...
09:58:11 route,debug,calc End calculation 

09:58:15 wireless,debug wlan1: 74:E5:0B:XX:XX:XX attempts to associate 
09:58:15 wireless,debug wlan1: 74:E5:0B:XX:XX:XX not in local ACL, by default accept 
09:58:15 wireless,info 74:E5:0B:XX:XX:XX@wlan1: connected 

09:58:15 route,debug,event Interface change 
09:58:15 route,debug,event     interface=wlan1 
09:58:15 route,debug,event     status=UP,RUNNING 
... snip ...
09:58:16 route,debug,calc End calculation 

09:58:35 wireless,debug wlan1: 00:26:C7:XX:XX:XX attempts to associate 
09:58:35 wireless,debug wlan1: 00:26:C7:XX:XX:XX not in local ACL, by default accept 
09:58:35 wireless,info 00:26:C7:XX:XX:XX@wlan1: connected 
09:59:00 wireless,debug wlan1: 8C:70:5A:XX:XX:XX attempts to associate 
09:59:00 wireless,debug wlan1: 8C:70:5A:XX:XX:XX not in local ACL, by default accept 
09:59:00 wireless,info 8C:70:5A:XX:XX:XX@wlan1: connected 
We are using WPA2. I saw one post about group key update intervals, so I increased this to the maximum (no real effect). Our clients are a mix of laptops, tablets and phone's. The RB951 config is pretty much out of the box using the quickset.

Here's a typical registration table under normal circumstances:
[admin@MikroTik] /interface wireless registration-table> pr d s
 0 interface=wlan1 mac-address=00:26:C7:XX:XX:XX ap=no wds=no bridge=no rx-rate="72.2Mbps" tx-rate="65.0Mbps" packets=17209,16014 bytes=10017338,3166582 
   frames=17209,16016 frame-bytes=10051752,3070736 hw-frames=17828,16518 hw-frame-bytes=10762793,3726501 tx-frames-timed-out=0 uptime=53m9s last-activity=0ms 
   signal-strength=-56dBm@HT20-7 signal-to-noise=56dB signal-strength-ch0=-56dBm signal-strength-ch2=-74dBm 
   strength-at-rates=-54dBm@1Mbps 7s780ms,-51dBm@5.5Mbps 1m12s220ms,-56dBm@6Mbps 7s300ms,-55dBm@36Mbps 53m8s800ms,-56dBm@48Mbps 53m8s740ms,-56dBm@54Mbps 
                  53m8s840ms,-57dBm@HT20-5 42m59s660ms,-57dBm@HT20-6 28m16s60ms,-56dBm@HT20-7 10ms 
   tx-ccq=99% p-throughput=56751 last-ip=10.1.0.108 802.1x-port-enabled=yes authentication-type=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm 
   management-protection=no wmm-enabled=yes 

 1 interface=wlan1 mac-address=8C:70:5A:XX:XX:XX ap=no wds=no bridge=no rx-rate="72.2Mbps" tx-rate="65.0Mbps" packets=3906,3835 bytes=1939148,1007014 
   frames=3906,3837 frame-bytes=1946986,984242 hw-frames=4421,6511 hw-frame-bytes=2215850,1202485 tx-frames-timed-out=0 uptime=53m9s last-activity=6s680ms 
   signal-strength=-60dBm@1Mbps signal-to-noise=52dB signal-strength-ch0=-60dBm signal-strength-ch2=-74dBm 
   strength-at-rates=-60dBm@1Mbps 49s730ms,-54dBm@5.5Mbps 33m58s690ms,-52dBm@6Mbps 6s680ms,-55dBm@24Mbps 53m8s730ms,-56dBm@36Mbps 53m8s190ms,-56dBm@48Mbps 
                  52m51s170ms,-56dBm@54Mbps 52m54s350ms,-52dBm@HT20-6 12m39s390ms,-54dBm@HT20-7 16s920ms 
   tx-ccq=89% p-throughput=19016 last-ip=10.1.0.129 802.1x-port-enabled=yes authentication-type=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm 
   management-protection=no wmm-enabled=yes 

 2 interface=wlan1 mac-address=74:E5:0B:XX:XX:XX ap=no wds=no bridge=no rx-rate="72.2Mbps" tx-rate="72.2Mbps" packets=16789,13867 bytes=15483225,3684992 
   frames=16789,13869 frame-bytes=15516901,3602028 hw-frames=17208,14404 hw-frame-bytes=16429933,4174341 tx-frames-timed-out=0 uptime=53m9s last-activity=810ms 
   signal-strength=-56dBm@HT20-7 signal-to-noise=56dB signal-strength-ch0=-56dBm signal-strength-ch2=-74dBm 
   strength-at-rates=-51dBm@1Mbps 9s130ms,-55dBm@5.5Mbps 18m32s600ms,-57dBm@6Mbps 4s770ms,-52dBm@24Mbps 53m8s780ms,-53dBm@36Mbps 53m8s700ms,-54dBm@48Mbps 
                  53m8s780ms,-53dBm@54Mbps 53m8s810ms,-52dBm@HT20-4 53m7s780ms,-52dBm@HT20-5 53m3s180ms,-55dBm@HT20-6 9m56s960ms,-56dBm@HT20-7 810ms 
   tx-ccq=99% p-throughput=61319 last-ip=10.1.0.105 802.1x-port-enabled=yes authentication-type=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm 
   management-protection=no wmm-enabled=yes 

 3 interface=wlan1 mac-address=10:40:F3:XX:XX:XX ap=no wds=no bridge=no rx-rate="65.0Mbps" tx-rate="65.0Mbps" packets=8924,9735 bytes=1768399,1538248 
   frames=8924,9737 frame-bytes=1786771,1480074 hw-frames=9327,10812 hw-frame-bytes=2095464,1895322 tx-frames-timed-out=0 uptime=45m54s last-activity=13s310ms 
   signal-strength=-57dBm@1Mbps signal-to-noise=55dB signal-strength-ch0=-57dBm signal-strength-ch2=-74dBm 
   strength-at-rates=-57dBm@1Mbps 45m33s230ms,-60dBm@HT20-4 14m10s190ms,-57dBm@HT20-7 13s310ms tx-ccq=100% p-throughput=56751 last-ip=10.1.0.100 
   802.1x-port-enabled=yes authentication-type=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm management-protection=no wmm-enabled=yes 
The attached file has a registration table snapshot'ed during one of the "disconnect" periods.

Anyone got any ideas?

Re: RB951 - All wireless clients disconnect simultaneously

Posted: Mon Dec 24, 2012 9:46 pm
by fgoldstein
I'm having a similar problem!

I have two 951s, one of which is in WDS mode and doubling as a station-side bridge.

When a link from the AP to a directly-connected (WDS) client gets weak, it sometimes logs an error message, and refuses to re-associate. Once a MAC has timed out, it's blocked until a reboot.

When a user is closer to the WDS (second) unit, the user associated through it. But every few days, the link to the WDS fails, and both units need rebooting.

Today one user associated with the WDS lost the connection at the same time while another's was still up, and it only came back when I hard-rebooted the WDS, though not the AP.

So something in the logic here doesn't try to reassociate or reauthenticate users. It just demands reboots whenever things get weak. This is not acceptable behavior and I really, really would like to see a fix.

Re: RB951 - All wireless clients disconnect simultaneously

Posted: Thu Dec 27, 2012 12:59 pm
by uldis
please make sure that you are using the latest RouterOS version and the Routerboard firmware.
also what WDS mode you are using?
We would need a support output file from the AP which is made during the time when the it doesn't communicate with the clients.

Re: RB951 - All wireless clients disconnect simultaneously

Posted: Thu Dec 27, 2012 7:44 pm
by richinuk
I'll work on getting the support output file.

In the meantime, I have found that enabling WPA (as well as WPA2) has fixed the problem (at least 1 week with no issues since doing this).

Rich

Re: RB951 - All wireless clients disconnect simultaneously

Posted: Mon Mar 04, 2013 4:25 pm
by Justaman
I have the same issue with two brand new 951G-2HnD and RouterOS 5.24. Some clients work well but others suffer continuous disconnects. There are a lot of errors in log:
18:22:40 wireless,info 44:6D:57:1F:E3:81@wlan1: connected
18:22:41 wireless,info 44:6D:57:23:8F:A2@wlan1: connected
18:22:46 wireless,info 44:6D:57:23:8F:A2@wlan1: disconnected, group key exchange timeout

I tried almost every possible change in settings. No luck.

Re: RB951 - All wireless clients disconnect simultaneously

Posted: Mon Jan 16, 2017 1:07 pm
by abouelrim
Hi Guys,

any update about this issue ?

Re: RB951 - All wireless clients disconnect simultaneously

Posted: Fri Feb 03, 2017 2:44 pm
by Paco
Same problem here on 951G-2HnD. Please provide us how to fix that.......