"joint timout" problem

Hardware MikroTik hardware general
1

Help with any pointers towards the trouble underlying the following wireless debug log would be appreciated.

The router is an RB532A running RouterOS 2.9.46. The WLAN card is an Atheros AR5212 In sation mode. It is a point-to-point backbone link with the AP identified in the connect list and this client identified in the AP’s access list with no default authentication.

The AP has the SSID ‘Clova’ on 2452 and is picked up by the STA scan three times yet the debug log reports “no network that satisfies connect-list” on the first two occasions.

During the connection loss beginning 20:50:27 I ran a scan on this interface and it picked up the AP with a signal strength of -66dB.

I’m puzzled by the “failed to join recently” report, which is not mentioned in the Wiki entry on wireless debug codes. Ditto “join timeout”.

When I check the status of this link when it’s up I invariably get a Tx/Rx of around -66, a noise floor of -96dB and a CCQ of 100%, which suggests that signal strength isn’t the problem.

The security profile at both ends is WPA2-PSK with dynamic keys. Both Unicast and Group Cyphers aes ccm are ticked. The “join timeout” reports would suggest a problem here, but this is outside my understanding.

Any lines of enquiry to be pursued would be appreciated.



20:27:54 wireless,debug wlan1: must select network
20:27:54 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
20:27:54 wireless,debug wlan1: no network that satisfies connect-list, by
default choose with strongest signal
20:27:54 wireless,debug wlan1: failed to select network
20:27:54 wireless,debug wlan1: delaying scanning
20:27:55 wireless,debug wlan1: STA starts scanning
20:27:57 wireless,debug wlan1: STA scan over, results:
20:27:57 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:27:57 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
20:27:57 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Manaroa caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:27:57 wireless,debug 00:80:48:3D:5A:60: on 2452 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:27:57 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
20:27:57 wireless,debug wlan1: must select network
20:27:57 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
20:27:57 wireless,debug wlan1: no network that satisfies connect-list, by
default choose with strongest signal
20:27:57 wireless,debug wlan1: failed to select network
20:27:57 wireless,debug wlan1: delaying scanning
20:27:58 wireless,debug wlan1: STA starts scanning
20:28:00 wireless,debug wlan1: STA scan over, results:
20:28:00 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:28:00 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
20:28:00 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Manaroa caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:28:00 wireless,debug 00:80:48:3D:5A:60: on 2452 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:28:00 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
20:28:00 wireless,debug wlan1: must select network
20:28:00 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
20:28:00 wireless,debug wlan1: no network that satisfies connect-list, by
default choose with strongest signal
20:28:00 wireless,debug wlan1: failed to select network
20:28:00 wireless,debug wlan1: delaying scanning
20:28:01 wireless,debug wlan1: STA starts scanning
20:28:03 wireless,debug wlan1: STA scan over, results:
20:28:03 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:28:03 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
20:28:03 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Manaroa caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:28:03 wireless,debug 00:80:48:3D:5A:60: on 2452 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
20:28:03 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
20:28:03 wireless,debug wlan1: must select network
20:28:03 wireless,debug wlan1: selected 00:80:48:3D:5A:60, SSID Clova on 2452
20:28:03 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2452,
SSID Clova

20:46:34 wireless,info 00:80:48:3D:5A:60@wlan1: lost connection, got deauth:
authentication not valid (2)
20:46:36 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2452,
SSID Clova

20:49:15 wireless,info 00:80:48:3D:5A:60@wlan1: lost connection, no beacons
20:49:17 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2452,
SSID Clova


20:50:27 wireless,info 00:80:48:3D:5A:60@wlan1: lost connection, no beacons
20:50:29 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:50:41 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:50:53 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:51:05 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:51:17 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:51:29 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:51:41 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:51:53 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:52:20 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:52:44 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:52:56 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:53:08 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:53:20 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:53:24 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:53:36 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
20:53:48 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2452,
SSID Clova

2

running RouterOS 2.9.46

Upgrade to at least 2.9.51

3

No good. Upgraded both ends of the link to 2.9.51 last night but saw this again this morning:

11:29:15 wireless,info 00:80:48:3D:5A:60@wlan1: lost connection, no beacons
11:29:17 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2452,
SSID Clova
11:30:26 wireless,info 00:80:48:3D:5A:60@wlan1: lost connection, no beacons
11:30:28 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
11:30:41 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
11:30:53 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
11:31:04 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2452,
SSID Clova

The “wireless debug log explained” says:

“join timeout” - happens on Station, failed to synchronize to AP (receive first beacon frame). Most likely weak signal, remote turned off, strong interference, some other RF related issue that makes communication impossible.

I don’t think weak signal is the problem as on other occasions during such disconnects I’ve run a scan and have picked up the AP at its usual strength of around -66 dB and in the example above the station was picking up the AP on its scans yet not connecting.

Any other ideas?

4

Have you tried changing frequencies?

5

I’ll give it a try and let you know. Thanks.

6

Frequency change didn’t help. Log of a 12s outage last night pasted below.

I’m puzzled by the comment “no network that satisfies connect-list, by default do not connect”. Yes, default authentication is turned off but the AP is correctly listed in the station’s connect-list. The AP appears on all the five scans carried out (SSID Clova) during the outage and on the first occasion is even selected for connection, which fails for the “joint timeout” reason.

On the following 3 scans the AP appears but the comment “failed to join recently” is attached to it. Then on the fifth scan it is selected and sucessfully connected to, and the connection has been fine for the ensuing nine hours.

The AP appears on the scans so it is clearly transmitting and is at around -66dB usually so the link is hardly borderline, and as the AP is on a very lonely 500m high hilltop it’s hard to think the problem is one of interference. Moreover another router also working on 2.4Ghz (2442 and 2412) on the same mast with the same power supply isn’t affected.


What would be the actual cause of “failed to join recently” and “join timeout”, reports?


00:50:28 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:28 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:28 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:28 wireless,debug 00:80:48:3D:5A:60: on 2472 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:28 wireless,debug wlan1: must select network
00:50:28 wireless,debug wlan1: selected 00:80:48:3D:5A:60, SSID Clova on 2472
00:50:28 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
00:50:28 wireless,debug wlan1: must select network
00:50:28 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
00:50:28 wireless,debug wlan1: no network that satisfies connect-list, by
default do not connect
00:50:28 wireless,debug wlan1: failed to select network
00:50:28 wireless,debug wlan1: delaying scanning
00:50:29 wireless,debug wlan1: STA starts scanning
00:50:31 wireless,debug wlan1: STA scan over, results:
00:50:31 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Manaroa caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:31 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:31 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:31 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:31 wireless,debug 00:80:48:3D:5A:60: on 2472 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:31 wireless,debug wlan1: must select network
00:50:31 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
00:50:31 wireless,debug wlan1: no network that satisfies connect-list, by
default do not connect
00:50:31 wireless,debug wlan1: failed to select network
00:50:31 wireless,debug wlan1: delaying scanning
00:50:32 wireless,debug wlan1: STA starts scanning
00:50:34 wireless,debug wlan1: STA scan over, results:
00:50:34 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Manaroa caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:34 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:34 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:34 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:34 wireless,debug 00:80:48:3D:5A:60: on 2472 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:34 wireless,debug wlan1: must select network
00:50:34 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
00:50:34 wireless,debug wlan1: no network that satisfies connect-list, by
default do not connect
00:50:34 wireless,debug wlan1: failed to select network
00:50:34 wireless,debug wlan1: delaying scanning
00:50:35 wireless,debug wlan1: STA starts scanning
00:50:37 wireless,debug wlan1: STA scan over, results:
00:50:37 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Manaroa caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:37 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:37 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:37 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:37 wireless,debug 00:80:48:3D:5A:60: on 2472 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:37 wireless,debug wlan1: must select network
00:50:37 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
00:50:37 wireless,debug wlan1: no network that satisfies connect-list, by
default do not connect
00:50:37 wireless,debug wlan1: failed to select network
00:50:37 wireless,debug wlan1: delaying scanning
00:50:38 wireless,debug wlan1: STA starts scanning
00:50:40 wireless,debug wlan1: STA scan over, results:
00:50:40 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Manaroa caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:40 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Totaranui caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:40 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jenny caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:40 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Office caps 0x421
rates 0xff0f basic 0x100 MT: yes
00:50:40 wireless,debug 00:80:48:3D:5A:60: on 2472 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
00:50:40 wireless,debug wlan1: must select network
00:50:40 wireless,debug wlan1: selected 00:80:48:3D:5A:60, SSID Clova on 2472
00:50:40 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2472,
SSID Clova

7

Still happening every now and then.

8

I’ll bet it’s related to the wireless module, and not software. We’ve had a number of R52H cards and NMP-8602 cards that have flaked out on us and have exhibited similar behaviour. The station trying (and failing) to join may not be transmitting a clean, strong signal.

9

It’s very intermittent, lasts only a couple of seconds and in between the signal is as good as you could want.

The attached file is a screen-shot pasted to Word with the log of an outage tonight and an interface monitor of only a minute or so later - RX/TX -63/-65, signal to noise 36dB, tx/rx ccq 100%/95%.

The screen shot is of the station end, which is picking up the AP (Clova) in the scans but saying there’s no station that satisfies the connect list. The connect list says accept anything down to -120dB, at which level there’d be nothing on the scan, surely.

I agree a card swap seems indicated, even if just as a diagnostic. They’ve both been going 18months now with very little down-time. Any idea which end of the link might be the problem one?
debug.doc (113 KB)

10

I have a site that is doing this every 7 minutes. I think that the time is entirely too consistant to be a hardware problem. It has to be a setting in the software. Anyway, we are changing the hardware today, RB532 x 2 with a single RB333 running ROS3.7. All new R52H cards and pigtails. Everything will be brand new except for the POE run.

Anyone have anything concrete that may explain this?

11

We’ve had 5" of rain in the last week which has made it impossible for me to get up the mountain to the AP and change the card, and it’s still happening.

During a 2-minute outage this morning while the station was “searching for network” I ran a scan from it and the AP was coming in as clear as a bell at -65dB. Yet even after stopping the scan it took about 40 seconds for the station to reconnect to the AP.

12

I’m having the same problem! Was the cause found?

13

“joint timout” happens when you break the “puff puff pass” rule.


Sorry, couldn’t resist.

14

I had similar problem recently.
Stupid thing helps me, :slight_smile:
Just try to change port that you attached antena to MiniPCI card.
I swiched from main to aux and, get much metter signal, error had never repeat again.

15

I’ll have to give that a try!

Did you change setting in AP from antenna A to B also?

16

Do I detect a hint of tongue-in-cheek here?

No, it’s still happening - actually changing the card in the particular location is such a major operation that at present the problem doesn’t justify it.

Moreover if is a card problem, at which end is it?

Even moreover, during these outages the both ends are still transmitting - ie here’s an extract from a log this am:

03:15:07 wireless,debug wlan1: must select network
03:15:07 wireless,debug wlan1: selected 00:80:48:3D:5A:60, SSID Clova on 2472
03:15:07 wireless,info 00:80:48:3D:5A:60@wlan1 established connection on 2472,
SSID Clova
03:15:22 wireless,info 00:80:48:3D:5A:60@wlan1: lost connection, no beacons
03:15:22 wireless,debug wlan1: STA starts scanning
03:15:24 wireless,debug wlan1: STA scan over, results:
03:15:24 wireless,debug 00:80:48:3D:5A:6A: on 2412 SSID Txxxxx caps 0x431
rates 0xff0f basic 0x100 MT: yes
03:15:24 wireless,debug 02:80:48:3D:5A:6A: on 2412 SSID Jxxxxxx caps 0x421
rates 0xff0f basic 0x100 MT: yes
03:15:24 wireless,debug 00:80:48:3D:5A:64: on 2412 SSID Mxxxxx caps 0x431
rates 0xff0f basic 0x100 MT: yes
03:15:24 wireless,debug 00:80:48:3D:5A:60: on 2472 SSID Clova caps 0x431
rates 0xff0f basic 0x100 MT: yes
03:15:24 wireless,debug 00:80:48:3D:5A:5A: on 2437 SSID Oxxx caps 0x421
rates 0xff0f basic 0x100 MT: yes
03:15:24 wireless,debug wlan1: must select network
03:15:24 wireless,debug wlan1: selected 00:80:48:3D:5A:60, SSID Clova on 2472
03:15:24 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join
timeout
03:15:24 wireless,debug wlan1: must select network
03:15:24 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
03:15:24 wireless,debug wlan1: no network that satisfies connect-list, by
default do not connect
03:15:24 wireless,debug wlan1: failed to select network
03:15:24 wireless,debug wlan1: delaying scanning
03:15:25 wireless,debug wlan1: STA starts scanning


At 03:15:07 a connection is re-established but 8 seconds later it goes down again. The station runs a scan, the AP is there (SSID Clova), the station correctly id’s it to connect to but fails to do so: ie

03:15:24 wireless,debug wlan1: must select network
03:15:24 wireless,debug wlan1: selected 00:80:48:3D:5A:60, SSID Clova on 2472
03:15:24 wireless,info 00:80:48:3D:5A:60@wlan1: failed to connect, join timeout
03:15:24 wireless,debug wlan1: must select network
03:15:24 wireless,debug wlan1: 00:80:48:3D:5A:60 failed to join recently
03:15:24 wireless,debug wlan1: no network that satisfies connect-list, by
default do not connect

WHAT DOES THIS MEAN?

SSID Clova is correctly identified in the connect list, so what does the "no network that satisfies the connect-list’ imply?

What does the “failed to join recently” mean? Failed to join what? What does “joint timeout” say about what is going on?

17

Change the LMR

T//

18

SweetSunday

I don’t believe that it is an LMR issue at this point. I have my wireless network setup in a star configuration with all my towers receiving their signal from a main tower in the center of my community. When I found the signal wasn’t good enough from three of my towers I rerouted them to go threw another tower that I have about six blocks away from my main tower. This tower is much taller and the towers that were having issues receive a much stronger signal going off my taller tower. Once I switched them I started to see the same thing you’re seeing on all three of those towers! My best guess is that the problem is with the card at that tower that my other three are attached to. This taller tower has an SR2 and the three other towers that get there signal from it are using a combination of r52H’s and a SR2 or XR2. The funny thing is that I have a client also receiving his signal from that taller tower that is using a 133c with an SR2 that isn’t having that problem. I have looked threw his logs and don’t see it happening to him. I will be looking into this much further and will update you on what I find; hopefully we both will have a resolution to this issue soon!

19

SweetSunday

"no network that satisfies the connect-list’ imply?
There seem to be a list that one can setup where you can say “connect to “A” and if you find another place “B” or “C” that has a stronger signal connected to the one that has the strongest signal. If one of them go’s down, scan again starting with “A”

“failed to join recently” mean?
My guess is that it tried but failed to connect to the AP that you have the SSID set to (Clova)

Failed to join what?
My guess is other places to connect to that would be in the list

What does “joint timeout” say about what is going on?
My guess is that there is a time limit for its search for other places to connect to.

Here’s the clincher if your not using that list “like me” than it shouldn’t start the scan. It should only connect to the SSID that you have specified. And if the tower that has the correct SSID go’s down than the tower that it gets its signal from should only search for a signal that matches the SSID you have specified.

It seems that there is a glitch some ware. I bet its in the software but I could be wrong! I will try reinstalling the ROS and see what happens.

20

I had said before that “The funny thing is that I have a client also receiving his signal from that taller tower that is using a 133c with an SR2 that isn’t having that problem. I have looked threw his logs and don’t see it happening to him.”

I checked and saw that I didn’t add wireless debug to his list to log. I have since added it and will watch to see if it happing to him as well. This should help to narrow down the issue!