Community discussions

MUM Europe 2020
 
User avatar
surfnet
Member Candidate
Member Candidate
Topic Author
Posts: 253
Joined: Wed Sep 01, 2004 6:38 pm

rouge CPE kill alls cleints

Thu Apr 26, 2007 4:03 pm

I am having a terrible problem on 1 of my 27 nodes. On Sunday I got complaints about packet loss and slow Speeds. I logged int he MK and did a scan and frequency test.. nothing showed out of the ordinary. I then noticed that when I stopped only about 4 out of 40 clients came back on. I looked at the noise and it is -102, and I just checked for interference and found none. So I checked the cable, antenna, and the db output of the SR2 and it was fine 24db.. I then changed the entire AP(mini-itx) with all new equip.. Same exact problem. Now here is where it gets interesting. I have a mixture of CPEs. I have CB3,dilberants, Tranzeos, and of course now I am only using RB133c. So I have about 5 133cs out there and the rest are mostly CB3s.
I found that If I pick the nosiest channel ( -88 ) and then disable and re-enable the wifi card a bunch of times.. every so often everybody comes on, and they works. The way I can tell is to be MAC pinging one of the Mikrotik 133c boards. I can tell when it *works* cause I get a 2ms ping time. If it doesn't work I get no clients and no pings. Sometime I see the Mikotik133c connect.. but pass no traffic and the ping fails.
I borrowed a full Spectrum Analyzer and did a sweep from 0-6000mhz and found no interference.
To further convince myself this is a broken CPE killing me. I logged into the 5 133c and changed the SSID on them, then I changed the SSID on the AP.. and those 5 MK CPES worked prefect every time. and no matter what channel I was on. When I switch back to the correct SSID to get all the rest of the clients on, I see the same problem. So it seems to me that some CPE has a bad association method, and if it does not associate correctly, it kills the AP.. if I disable/enable card I trick the CPE into trying over.. and if I do that enough.. the CPE finally connects, it stops killing the connection. The reason I have to be on the nosiest channel is because I believe the offending CPE has a weak signal, and by choosing a noisy channel , I lower it's overall affect and thus I am able to get some clients on..I did try blocking all CPEs on the access list and then only letting some on. but that does not work. The only thing I can make work every time is changing the SSID and having the few Mikrotiks only on the network, and that works perfect every time.

Maybe.. this is all a theory. I would love for someone to tell me I am wrong and what to do. It scares the hell out of me that a single CPE could take down the rest.
 
User avatar
GWISA-Kroonstad
Member Candidate
Member Candidate
Posts: 111
Joined: Fri Nov 10, 2006 3:34 pm

Thu Apr 26, 2007 4:38 pm

Having a similar situation on 4 towers. We think it is due to interference or noise. You've just changed our judgement with your spectrum analyser test. We've also thought of a possibly bad CPE or bad Client connection. Frustrating, specially that this usually happens only for above 20 clients.
 
User avatar
surfnet
Member Candidate
Member Candidate
Topic Author
Posts: 253
Joined: Wed Sep 01, 2004 6:38 pm

Thu Apr 26, 2007 4:42 pm

I don't believe it has anything to do with the number of clients on the AP. We have 25 APs with 50 or more clients with no problems. This node in question is not that loaded with only 40. So when you say you have this problem, are you able to get people on by enabling and disabling the interface?
BTW my CPU usage is a bout 20%.
Ken
 
jober
Long time Member
Long time Member
Posts: 692
Joined: Fri May 28, 2004 12:16 pm
Location: Louisiana,USA

Thu Apr 26, 2007 4:44 pm

That's interesting,
What mini-PCI radio card are you using on the AP?
Are you running in Band 2.4GHz-B/G or 2.4GHz-B?
If you are running in 2.4GHz-B/G have you tried changing to 2.4GHz-B?

I had an issue with an AP that had CB3's, Deliberant2300A, Deliberant1300 and RB532's. If I ran in B/G some of the clients would not connect and the ping times were in the thousands if not timing out. When I changed to B only every thing was fine.
At that point I started replacing all of the client units with RB532's.
 
User avatar
surfnet
Member Candidate
Member Candidate
Topic Author
Posts: 253
Joined: Wed Sep 01, 2004 6:38 pm

Thu Apr 26, 2007 5:02 pm

I am running SR2, but I have tested with 8602 and saw only worse noise. I have changed the SR2 twice to no avail.
I am only in the 2.4GHz-B

I went after the dilberant folks first.. I only have 8 of them. I was able to call 5 of them and have them turn off the CPE, and then I test. So far no luck

Currently I have an excel spreadsheet with all the clients on the AP on it. I am calling each one and having the turn off their CPE, then I log in to the MK and switch to a different channel.. in a perfect work all the CPEs would follow me.. with the current problem they don't come back. I then tell the client it is not them, and have them power up. I then sit at the AP and switch interface on and off till the clients make it back on and pass traffic. I hope to find the offending CPE that way.. but I feel like that is a shot in the dark, and I should be able to fix this problem on the AP side.
Ken
 
jober
Long time Member
Long time Member
Posts: 692
Joined: Fri May 28, 2004 12:16 pm
Location: Louisiana,USA

Thu Apr 26, 2007 5:58 pm

It sounds like the best way to find the problem CPE. Good Luck!

Where are you located?
 
jo2jo
Forum Veteran
Forum Veteran
Posts: 968
Joined: Fri May 26, 2006 1:25 am

Thu Apr 26, 2007 6:01 pm

surf net:

cant you just NOT allow CPE's to connect, one at a time, by mac address, and find out that way..

this way u dont have to annoy your customers..
:beep :beep :beep
 
User avatar
surfnet
Member Candidate
Member Candidate
Topic Author
Posts: 253
Joined: Wed Sep 01, 2004 6:38 pm

Fri Apr 27, 2007 12:46 am

ok I found the problem. the Atheros chipset.

I started thinking about the nodes I have and how many clients are on each antenna, rather than the AP as a whole. I realized that only 2 antennas have more than 40 people on them. the problem antenna I am working on.. and anther problem free antenna. I looked at the good AP and noticed I have an old Senao NL-2511MP 200mW prism 2.4 mini pci card in it. HMMMM, so I grabbed another one off the shelf and went out the AP and switched the cards.. and viola, the AP worked and 51 people logged in and started surfing. I can reset, scan, sniff.. whatever I want and all the clients come right back on and surf. I tired 2 different SR2sand a Senoa 8602.
So I wonder is the problem because of the 51 people on the AP, 40 of them are CB3s, and they have a hard time with atheros cards in heavy usage, or is it the card itself just card deal with all the connections at once.
Remember that with the Atheros in there I could get everybody to work, if I reset the card enough times. so it sounds kinda like the card is overloaded.

Help, Prism cards show no noise, and other advanced features that make me like the SR2 and other Atheros cards.
Ken
 
cmacneill
Member Candidate
Member Candidate
Posts: 294
Joined: Sun Apr 01, 2007 10:51 pm
Location: Christchurch, New Zealand

Fri Apr 27, 2007 11:23 am

I've had a lot of problems with Senao 860x cards, I can't get them to work at all, but MikroTik support just denies any knowledge of a problem saying the R52 cards work OK.

I had a bit of a rant in another post last night so maybe MikroTik support will actually start looking into this problem.

The supported hardware list says Atheros 860x cards are supported and it doesn't qualify this to say only R52 are supported, so either MikroTik haven't bothered to test other manufacturers cards or they should list EXACTLY which cards they HAVE tested.

I've switced to Wistron CM9 cards and they're better, but I still get some weirdness from RouterOS with the odd crash and lockups. The Senao 2511 802.11b cards work perfectly, but they don't work with WDS or virtual APs.


Regards

Chris Macneill
 
ldvaden
Member Candidate
Member Candidate
Posts: 201
Joined: Sun Oct 30, 2005 8:27 pm
Location: North Texas

Wed May 09, 2007 7:34 am

I don't believe it has anything to do with the number of clients on the AP.
Nor do I; I watched the same thing happen at a new pop with 8 initial subscribers we had acquired (read: no control over CPE choices); while manually searching for the best channel to use on 2.9.42, the average time to first association was <= 2 minutes. Longest time to associate was near the DHCP lease time, so IMHO it is CPE dependent.

It seems that certain CPE are not necessarily looking for an opportunity to associate as early as possible.

rgds/ldv
 
ekkas
Long time Member
Long time Member
Posts: 562
Joined: Mon Sep 26, 2005 1:01 pm
Location: South Africa

Thu May 10, 2007 1:15 am

I have the exact same problem. Had a PC with Atheros card. No problem. Then I changed to RB532A with same card and the problem started. I swopped it out with a SR2 but still same problem. I noticed enable/disable interface doesn't work as good, I change the preamble, apply, change back, apply again and all connect... My first thoughts is a strange 'bug' or limitation in ROS as the PC (VIA chipset) didn't give the problem but since using a RB532A it started, no matter what card. I say bug not meaning something wrong, but under certain conditions, someting doesn't tick into place and you have to give it a nudge to get it going.
Our clients (+-30) use Senao, DLink & one or 2 others, but now I'm only using RB133C, what a great solution compared to the other 'toys'...

I started to notice as we use MAC filtering; some DLinks doesn't report the correct MAC address(not even in browser interface), so we had to enable default-authentication, set the unit up and send the connected MAC to the access list, change back to not default forwarding...problems!!! We do not install DLinks anymore but the problem still persists when we need a reboot for some reason or the other.
 
jober
Long time Member
Long time Member
Posts: 692
Joined: Fri May 28, 2004 12:16 pm
Location: Louisiana,USA

Thu May 10, 2007 8:12 am

To: ekkas
Were the OS versions the same on the PC and Router Board?
Was the Router Board a 266 or the newer 400mhz R5?
If it's the 266mhz board was it clocked to 330mhz?

I'm just fishing here.
 
ekkas
Long time Member
Long time Member
Posts: 562
Joined: Mon Sep 26, 2005 1:01 pm
Location: South Africa

Thu May 10, 2007 9:05 am

Hi,
Same OS
RB532A
 
rodneal
Member Candidate
Member Candidate
Posts: 223
Joined: Mon Mar 12, 2007 7:49 pm

I've had the same problem for 8-9 months

Fri May 11, 2007 12:17 am

My problem was primarily with the Zcomanx and Tranzeo models.
I would loose my AP after 10-12 CPE clients would connect.

I could use all the other clients I wanted but anymore then that number and the AP would stop making any connections at all.

The AP is a PC system with SR5 backhauls and SR2 AP.

I've had no problem with Deliberant or Senao clients but the minute I start adding Zcomax and Tranzeo and get to that magic number 10-12 clients + all the others then the AP is down. I can have 40+ of the others and 9 Zcomax and all is OK but add 1-2 Zcomax and all hell breaks loose.
BTW - I think Zcomax makes the Tranzeo hardware.
I had a lot of problems with the Tranzeo 90s and called the company on the carpet about and they didn't want to talk to me so I looked up the part numbers and found Zcomax. I thought at first it was the Tranzeo firmware but now I'm thinking it was the hardware all along.

I can scan all day and see lots of other APs. The noise ratios are good and if I call my customers and have them unplug the power to the Zcomaxs then all others can connect. But they will not go online until that happens.
We reported this last October and nobody had an answer.

I would love a fix for this.

The quick fix is to put repeaters in the neighborhoods with the most Zcomaxs and point them at it. That breaks down the number of units pointing at the ailing AP. That fixed all my problems until we reached a saturation on those AP and then the same problem is breaking out now. I figured we have a solution by now.
Rod
 
spire2z
Long time Member
Long time Member
Posts: 517
Joined: Mon Feb 14, 2005 2:48 am

Tue May 22, 2007 2:24 pm

I think I am suffering this issue authough I have no Zecomax clients. I only have:

senao cb3plus
smart bridges air bridge
and pheenet wap 554g

Are any of these products having the same issue as your client hardware?
and has anybody found a fix?

I think service provider hardware really should be able to handle client hardware problems! If they can't it's makes a mockery of even being able to handle a DDOS attack if a simple client unit fault can take the whole network down!! Thats not what I would call service provider hardware.
 
ekkas
Long time Member
Long time Member
Posts: 562
Joined: Mon Sep 26, 2005 1:01 pm
Location: South Africa

Tue May 22, 2007 5:51 pm

I have senao cb3, dlink dwl2100ap & MT with atheros but the same equipment is running on 7 other towers, no problem. Must be a faulty client or something but if we can reproduce the problem consistently, I'm sure MT can sort it out. For now I get everyone connected (by changing some settings) and then just leave it alone!!! :-)
 
User avatar
surfnet
Member Candidate
Member Candidate
Topic Author
Posts: 253
Joined: Wed Sep 01, 2004 6:38 pm

Tue May 22, 2007 6:13 pm

the problem occurs when you have too many prism chipsets trying to connect to an atheros chipset. Check the chipset of all you CPE gear.. How may of those do you have? I fixed my problem by adding a prism back to the AP, then moving clients from a Prism (CB3) to a RB133 wth RB52 card. Then I put them on the atheros AP.. once I got the prism AP down to less than 40 prism clients on it, I switched it from a prism AP.. back to the atheros, and everybody connected.. with no problems. If I add more that 40 prism clients on the atheros, it will have the association problem, where you just sit and disable and re-enable the card, until everybody gets on..
Ken
 
User avatar
sten
Forum Veteran
Forum Veteran
Posts: 920
Joined: Tue Jun 01, 2004 12:10 pm

Tue May 22, 2007 7:10 pm

Are all those PRISM stations running latest PRISM firmware?
Do you have any way of excluding the problem being the PRISM stations not wanting to connect to the AP?
Have you tried a faster basic rate? Could be that they timeout while trying to associate.
Move along. Nothing to see here.
 
ekkas
Long time Member
Long time Member
Posts: 562
Joined: Mon Sep 26, 2005 1:01 pm
Location: South Africa

Re: rouge CPE kill alls cleints

Fri May 25, 2007 3:39 pm

This morning, after replacing a MiniPCI with a Compex card, all same started on another tower.We use MAC filtering and if I switch on 'Default authenticate/Forward' (switch filtering off), then all connects. With it switched on (filtering) only a few clients connect until I do the tricks as explained above.

Hope this help the MT guys, hope they care. :-0
 
ekkas
Long time Member
Long time Member
Posts: 562
Joined: Mon Sep 26, 2005 1:01 pm
Location: South Africa

Re: rouge CPE kill alls cleints

Fri May 25, 2007 3:41 pm

My other tower with same problem have a SR2 so I do not think it is the Compex's fault...
 
User avatar
surfnet
Member Candidate
Member Candidate
Topic Author
Posts: 253
Joined: Wed Sep 01, 2004 6:38 pm

Re: rouge CPE kill alls cleints

Fri May 25, 2007 4:08 pm

Are all those PRISM stations running latest PRISM firmware?
How I am supposed to go into 400 CB3s and figure out which prism firmware they are running. This problem has only started a few months ago.
 
User avatar
nickb
Member
Member
Posts: 407
Joined: Thu Jan 26, 2006 6:24 pm
Location: Southeast Kansas
Contact:

Re: rouge CPE kill alls cleints

Thu May 31, 2007 12:35 am

We have simmilar issues as well.
 
User avatar
sten
Forum Veteran
Forum Veteran
Posts: 920
Joined: Tue Jun 01, 2004 12:10 pm

Re: rouge CPE kill alls cleints

Thu May 31, 2007 11:16 am

This paper might help. Perhaps you should do some wireless sniffing to see whether this happens to you?
http://www.huwico.hu/~kodmon/cikk/firmware_attack.pdf

Other than that, try searching google for "802.11 DoS"
 
User avatar
surfnet
Member Candidate
Member Candidate
Topic Author
Posts: 253
Joined: Wed Sep 01, 2004 6:38 pm

Re: rouge CPE kill alls cleints

Thu May 31, 2007 4:20 pm

I have done the wireless sniffing, and there are only requests coming from the client CPEs. I think is a pretty weird thought that all the sudden 1/2 my nodes start getting a DOS attack, and that alot of other people on the list have the same problem. I think it more likely that there is an issue with the cards or with the OS, I sure wish Mikrotik would speak up here and let us know they are listening,. I can let anybody who wants to see this log into my node and just try to make a small change to the radio and watch what happens..
 
User avatar
sten
Forum Veteran
Forum Veteran
Posts: 920
Joined: Tue Jun 01, 2004 12:10 pm

Re: rouge CPE kill alls cleints

Sat Jun 09, 2007 1:25 am

 
illiniwireless
Member Candidate
Member Candidate
Posts: 152
Joined: Mon Dec 26, 2005 12:36 am
Location: USA

Re: rouge CPE kill alls cleints

Sat Jun 09, 2007 8:04 am

Manually setting ack-timeout didn't help me.

Who is online

Users browsing this forum: No registered users and 28 guests