Wireless link througput guideline versus data rate

RouterOS Wireless Networking
1

I am looking for a guideline on what the max. real life user data throughput can be for wireless link.

For normal mode (20Mhz channel) balanced link (both directions same connection rate) and no loss of packages (CCQ 100%) 50% of connection rate can be used as estimating max. total user data throughput? Is that right?
Thus 54M/54M would be max out at 27Mb data throughput? Or is this too conservative?
[Since single radio link is half duplex total data transport is 27Mb. If 20Mb goes in one direction other direction only can have 7Mb]

Would this ‘rule-of-thumb’ also count for proper configured and working 802.11n links?
If I see 216,0Mbps-HT/216,0Mbps Rate this would mean at best a data throughput of 100Mb unidirectional?
Or am I now mistaken?

2

I have a quiet backhaul link, so just did some tests with bandwidth test.

SNR is 42dB
CCQ between 90-100% over most of time
Wireless Data rate fixed at 24MBps only, using normal 802.11g on 5GHz

UDP Rx = 11MBps
UDP Tx = 11MBps
UDP Both = varying between 4.5 and 5 MBps both directions
TCP Rx = 10MBps
TCP Tx = 10MBps
TCP Both = varying between 4.5 and 5 MBps both directions

YMMV

We don’t have any 802.11n links yet, awaiting delivery of R52Hn from MT.

3

In general answer is - no. But it depends on how “precise” you want your estimate to be. Throughput as seen by user also depends on packet size and of course propagation delay. Throughput is the amount of data delivered over link in some period of time. So the key here is to understand what happens during this time. Here is a simplified version of how frame transmission looks (IFS meaning “interframe space”):
[fixed size preamble][frame at specified data rate][IFS][ACK frame][IFS][fixed size preamble][frame at specified data rate][IFS][ACK frame][IFS]…

The only “useful” usage of medium are periods where actual frame data get sent. It is obvious, that the smaller the frame, the less “efficient” is usage of medium (more “wasted” time per data byte, because more preambles, IFS and ACKs necessary).

Throughput will also depend on how big the IFS is. To get full info on this, refer to DCF procedure in 802.11 standard.

Thus 54M/54M would be max out at 27Mb data throughput? Or is this too conservative?
[Since single radio link is half duplex total data transport is 27Mb. If 20Mb goes in one direction other direction only can have 7Mb]

Would this ‘rule-of-thumb’ also count for proper configured and working 802.11n links?
If I see 216,0Mbps-HT/216,0Mbps Rate this would mean at best a data throughput of 100Mb unidirectional?
Or am I now mistaken?

Again - no. 802.11n changes the above picture so that actually multiple frames can be transmitted with single ACK and IFS overhead (so called aggregate). So 802.11n definitely provides opportunity to increase the efficiency of medium usage. Of course, how efficient this usage is in real situation again depends on multiple factors, including software implementation of aggregation.

4

I am looking for a “rule of thumb” not precise calculations.
It is more that if I see an air rate of for example 24Mbps on both Rx and Tx and CCQ’s are near 100% I still can’t expect 20Mb to go through this link in a real life network.
The other way round, If I see in general only 5Mbps total aggregated throughput (tx and rx at the same time) for this link while I know more traffic should be passed through (for instance bandwidth test over the link) I know something somewhere is not good.

The estimation is good enough if we consider 10% accuracy. On a 2Mb rate link 1Mb more or less make too much of a difference where a 10Mb accuracy on a ´n` link that could do 100Mb is still a workable estimate.

I know that package size and preamble etc (encryption!) have some influences on the real throughput but is not important in establishing ´rule of thumb´

propagation delay is hardly measurable on wireless links. Or it must be my client is on the moon…
signal travels almost with speed of light. :confused:

5

Depends on the size of your thumb. :laughing:

Seriously, it depends on what you want to push down the link. If all the traffic is high priority (eg VOIP) then I would not rely on saturating the link to capacity and hoping that adding queues and mangle rules and QoS will make any difference to your traffic quality as there is no spare capacity left to stop packets being dropped. So, if it is high quality I use a “rule of thumb” of 1/4 of the link. So, 24MBps wireless is roughly 6MBps throughput.

On the other hand if you want to push low quality traffic, like only web surfing and is nearly 100% TCP that can withstand many re-transmissions, then you go to a higher bandwidth, but still only to 1/2 of the speed as it is a “single frequency simplex” system. But if you want any kind of quality at all, to stop retransmissions of TCP (which leads to lower throughput for the client) then assume 80% is really available to give you some room for applying QoS rules, then take 1/2 of that. So 24MBps is about 10MBps. But that is MY rule of thumb. YOUR rule of thumb will be different, based upon YOUR situation and so it is for everyone else.

There is no universal “one size fits all”. It is not just a factor of 50% of the wireless speed but it also depends on the traffic you want to pass over that link. Which is why I said in the beginning. “Depends on the size of your thumb”.

But also of course, sometimes the link will self adjust to a lower speed, because of poor quality signal levels or interference, and then your rule of thumb fails unless you also take that into account too. So it also depends on how many 9’s uptime you want for that throughput as well.

6

Hi nest,

thanks for your input.
One of the reasons I also started this topic is that other less experienced guys entering the world of wireless data links know what to expect from it.

I mean, vendors always make a lot of bla bla with their “up to 108Mb throughput” or now with 802.11n “Speeds up to 300Mb!” claims and I can imagine many newcomers in forum are disappointed in what they actually get when they finally start using their purchase.
One of the reasons is the perception given by the vendors, but then even in their status screens they seen values (connection rate 54/54Mb!) what is not actually they can expect to have as data throughput.s.
[Don’t we all have possible customers that claim your network is slow compared to what his providers gives him. He sees his computer tells him he has a 100Mb connection while you only offer something with single digit numbers!?!

Particular in the discussion around 802.11n in this forum I sense a lot of disappointments. Not only because MT is obviously not ready with 802.11n yet but also because many users expectations are way to high. 300Mb on 802.11n is still future for 99% of all vendors but they all use it in their sales… and many buyers believe in it!
MT is under pressure because their ROS and drivers are not ready to deliver normal expectations but discussion on the forum become also blurred by people expecting speeds over 200-250Mb and then fall over MT for not making that happen!

Most WISPs will run networks that have a combination of HQ small packages combined with LQ tcp streams so a general “rule of thumb” can be very helpful for quick establishment of performance of links.
It will tell the first time installer what to expect under normal conditions and what to take in account when setting up a wireless network.

Even I was short-sighted in one of my links. I thought I had set-up a short distance (400mtrs) 802.11n link almost half a year ago which was capable of 80Mb tcp bandwidth test running over it. This was impressive enough for me to put it in service.

But after the growth in my network I found this link not performing well at all. I see regular connection rates belonging to low ´a´ values in the status when link is under load. I found ping times becoming crap when link was under load.
Now I realise this link in real world condition with at least 20 to 30% small packages over it becomes very poor. Since I thought this was a high capacity ´pipe´ I used this backbone for at times 20 to 30Mb traffic but now I realize that when that happens the conn.-rates drops into these fields and even a full 802.11a line would almost max. out with this kind of traffic.
So I started to re-design my back hauls and work more in planning with this ´rule of thumb´.

If I just would have embedded a “rule of thumb” in my head all the time it would have saved me the time and effort.

7

WirelessRudy, lets do some math. Consider perfect point to point 11a link (using 54Mb rate all the time) of length zero (to factor out any issues caused by propagation delay, and yes - although it is near speed of light it still is a factor, more on this later).

1500 byte IP packet becomes 1536 byte packet over air. It takes 256us (microseconds) of “air” (including preamble). After data packet, comes SIFS (short interframe space) interval, which is 16us. Next, ACK frame is transmitted back, takes 44us. Next, sending party must wait DIFS (DCF interframe space which is SIFS + 2 * SlotTime, where SlotTime is 9us), which is 34us. Then it must perform random backoff for 0-15 SlotTimes, lets say on average 7 SlotTimes, producing average backoff 63us.

By adding this all up we get time one successful transmission takes: 256+16+34+44+63=413us. So we can push 10^6/413=2421.31 packets in one second. This is 2421.31*1500=3631961.26 bytes or approx. ~29Mbps.

For IP packets of size 500 (closer to average internet packet size), the only thing that changes is “air time” of packet itself, it is 104us. This makes transmission take 216us. So you can send 3831.42 packets per second or approx. ~15.3Mbps.

For IP packets of size 100 (closer to median of internet packet size distribution and also more close to VOIP packet size), “air time” of packet is 48us. This makes transmission take 205us. Packets per second: 4878.05, approx. throughput: 3.9Mbps.

See, the same perfect link, but throughput, depending on packet size can change close to 10times. How can you have “rule of thumb” without taking this into account? So nest fairly says that your “rule of thumb” will depend on what you are trying to deliver. Thats why you may experience astonishing results using bandwidth test (using big packets), but never see that in real life (here I am not saying that 11n issues you are experiencing are related to this, I am sure there are other problems in routeros that are/will be worked on).

It will become even worse with retransmissions. It will only become worse with whatever signals (being interference or other networks on the same frequency) hold medium busy. And CCQ only shows you how efficient (from retransmission point of view) medium usage is (so 0 retransmissions - 100% CCQ). It does not take into account the time “wasted” because device does backoff due to medium held busy by someone else (either it is interfering signal or well behaving/coexisting device).

As to propagation delay - lets say speed of light is 3*10^8 metres per second. This means it is “only” 300metres per microsecond. So propagation delay for 10km link is 33.33us. This MUST be somehow compensated for in those interframe spaces (e.g. in our perfect link, space between data frame and ack frame was 16us. In 10km link by no means it can be smaller than those 33.33 us).

I hope I gave you the idea of the magnitude of “problem” here. I doubt you will be able to establish your rule of thumb by ignoring how 802.11 actually works.

8

Mplsguy;

I think you gave a very nice in-depth explanation on what link throughput is all about.
But in real life, we all have to make a rough guess on what kind of average package size (data!) actually travels over the link.
I don’t think most of us know how much small, how much medium and how much big packages travels over the link.
I think it is even safe to say that no network deploy-er will ever know at forehand what kind of traffic in which amounts will travel over his future link.
So, how do we decide if a link is going to be ´big´ enough for its purpose if you don’t know yet what the future has in store…
I have some links deployed with a stable rate of 54Mb both directions, but only 3 or 4 clients hanging on it.
At the same time I have some poor links deployed just to reach one or two guys and suddenly find after a year over 15 users on it. The link only has stable and high CCQ when I set the rate down to 24Mb. You can imagine the link is a bottleneck at times if several of these clients start to download their assigned 3Mb.

I think we all want a pipe as big as possible.

But we can also turn it around. Imagine you already have a link running for a while. And little by little you take more users on board. But as a small starting provider you always have some problems here and there. So now also some of these new users start complaining… you look at your network and see no immediate problems. So, where is the problem?
Is your Qos Working fine? Is your routing/bridging giving problems? Are there any IP conflicts around? Does the client have good connection to the AP? Is your feed to your ISP working fine (big problem here!) Is anyone playing around with your network? Is the clients own PC not the problem? Probably last thing you think about is that your always stable looking 54/54Mb - 90-100% CCQ link might run in capacity limits because you have now at times 20 users entitled to in total 60Mb hanging on that link!
With some contention ratio you think "well, 50% on-line users can do 35Mb, so my 54Mb link has no problem.
For such situations it is handy to realise that 50% of 54 = 26Mb is already the max. and you actually have a big capacity problem on that link! I think it an easy bet to state many of forum readers are not aware of this.

If I see on my mangle filters for QoS what kind of traffic on average is flowing over my main gateway I would say some 50% port 80 and other tcp traffic and the rest either P2P or Skype. But maybe should I start to take a closer look on my real network traffic to see if things need some more tuning here and there based upon what I have been learning in this topic.

Basically, if you want to stay on the safe side, and avoid any ´bottlenecks´ it is almost impossible to set-up a reasonable network with only a/b or g (and for now still ´n´) technology.
But lucky enough the real life situation is not as bad as sketched by your calculations.
I have almost 200 users entitled to 3M/256k with 80% skype users on my network and 80% of these run over a back-haul that can only establish 36Mb/36Mb with above 80% CCQ. And I get more enthusiast remarks then complaints from users so its not that bad after all…

I hope lots of other readers, special starters, will find this one a very helpful one. It is definitely for me! Thanks.

9

Hi all !

I think the calculation maybe is correctly, but remember that on one link are many sessions with same rules. So the summary should be able to fill out the available bandwidth. I can also tell that we use only wlan links (802.11a, most nstreme) to our towers where our customers are connected. We also use chains of 7 tower-hops where each hop is linked with wlan to the next. So - on this example - 14 MT devices have to be passed via 7 links over the air. And the last customers in the chain can easily use 4 MBit accounts with VOIP at delay time of 4-9ms. In my network this only works correctly with the help of QOS and 802.11a. With Nstreme (!) for point to point and no-nstreme (!) to the customers. This has the best performance. And if a link gets overloaded - nagios checks report ping delay - i switch to turbo mode. Then i can push 50 MBit mixed traffic from hundreds of customers over the air without problems.

I think that MTs 802.11n implementation has actualy an issue of delay time. maybe the wlan-drivers … maybe V4.x itselve … it gets stable if you increase the retry times but it doesn’t perform well.. I have tested excessive, but cannot push more than 25MBit TCP even with 40MHz and 300MBit connect via a real (!) link. I have tested on 3km and an on 500m links. I have tried every possible setting reading wiki, forum and intuition. It is possible to tune the link and get it stable, but it doesn’t perform as required. 11a is much better on nstreme. Activating turbo mode, if more bandwith is needed, is still best till today..

I think the problem with 11n is simply the packet delay, which should be 1ms via the link even if traffic is higher. You will get 40ms+ if 25MBit is reached. So it is not possible to get high TCP traffic over the link. nstreme would perform better, but doesn’t work correctly on 802.11n and is not supported. so nstreme is very instable, but delay time would be much much better …
It also doesn’t help if you test with more sessions on the link. I cannot get more than 25MBit TCP in summary with real traffic over the link as many sessions i try. I hope that MT development team can “see” this problem in short future and accepts that this is a fact, because at the moment there is no information about fixing this problem but very much information about “everything is under control” and the performance problem can be fixed with “intelligent settings” :wink:

best
Dieter

10

Hi Dieterk,

Make sure your info is also posted on main 802.11n topic: http://forum.mikrotik.com/t/802-11n/28753/1

and send a mail to support.
I don’t think the guys fm MT are ready all topic and don’t have the feeling they read this one. But your remark could prove to be very valuable to MT development dept.

rgds

11

oh sorry, I see you already did. I was to fast this time…

12

Hi WirelessRudy !

Yes i have posted on 802.11n Topic as well. Thanks for your message ! I hope that someone within MT development team accepts that there is a real problem and will put menpower on solving this … Because the product is great, but the acceptance of existent problems is low. Next days i will do some checks to verify my suspicion about the delay problem. I hope i can report some details which can help to solve the problem …

rgds
Dieter

13

No thanks,
I want as much people as possible to report their problems and complaints about MT’s approach in solving the problem on this forum.
I made a business plan decision half a year ago based on the usage of ´n´. I trusted MT would solve any problems that would pop up.
I looked with an half eye to Ubiguity with their new Airmax platform and tempting promotions and wordings on their website.
The only reason I am not moved away from MT yet is that I don’t like to move from a otherwise (so everything apart fm ´n´) good product with all its possibilities.
I also don’t like a to much mixed network when it comes to branches. It makes things complicated when there are problems.
I lost my confidence in the low-end bullets and nano’s of ubiquity since the failure rate is very high compared to MT hardware.
I also not jumped into Airmax since I picked up some messages here and there that also here expectations are a bit overthrown. Off course, you read about success stories on their forum, like we do on this one. But digging deeper reveals that also on the UBNT platform only some guys are lucky enough to have good fast running ´n´ links and searching for more info about how to handle ´n´ settings and their impact also very little info comes from them…

So now I am forced to start building more back hauls on ordinary ´a´ platform and wait until the ´n´ platform from MT becomes mature. What a disappointment..

And yes, MT needs to give much more feedback and support on this matter too. It is very irritating how blunt or misty MT’s response is on posts at times, even done by the more experienced forum members…
They don’t earn bonus points for this behaviour…

14

Why hasnt anyone mentioned the usable bandwidth ‘rule of thumb’ metric for throughput using Nstreme? It blows away the standard 50% rate. I know if you serve laptops, unless they are running ROS, of course you are out of luck.

I have several strong links that yield significant throughput because they are able to always run at 100/100 CCQ any 802.11a bitrate possible. So I spent a little time confirming a real ‘thumbnail’ view of what I have always used. Link lengths are from .5-to-15 miles. I know the short link will yield somewhat more throughput, but its fun to find a long link that performs similarly…

Link Conditions: Stable, solid, low -60’s (~-62Dbm) signal, SNR upper 30’s, max Pthroughput, test RBs outside of link radios
Z-Rule-of-Thumb:
If your UDP and/or TCP efficiencies are in the ~70s I would stop messin with it.
UDP bandwidth tests always seem to work and really the max rate you could possibly get anyway. TCP tests are affected by the return path of course.
I get 29.9Mbps UDP on a 36/36 raw data rate connection.
Pthrough at times is more often than not a built in ‘Rule-of-RB’. I believe if its bad, you have problems for sure.
With MT and ROS, we get it all in realtime, and the proprietary enabled bump in throughput isnt unique in the industry. Canopy did it long ago with what I called ‘Fixed TDD’. You could get away with setting ridiculously high percentages, and it worked–with perfect conditions.
Lower speed links curve higher than high bitrate links and could easily be close to 80% for up to 18Mbps (16QAM).

Bottom line is that the most efficient equipment/software (regardless of price) combos yield ~70% or greater device throughput.

Thats my rule and I have stuck with it for years…

15

Hi Rudy !!

It seems that MT has solved our major problems at long last … :wink: You can use 802.11n NOW with the help and work arround of nstreme and WDS. Only THIS config (at the moment) in combination with 4.9 gives us the performance from which we have ever dreamed !

I can get 90 MBit TCP traffic with single chain (one antenna) at 40 MHz even with RB433AH. The frequence usage is the same as 11a Turbo Mode. Turbo Mode brings 50-60MBit. Only the chanel separation is more sensitive on 11n. You can see this easily on the max Throughput and the WLAN link speed. If the signal changes speed more often there is a problem. I had another antenna on neigbour channel which directed to another direction but maye physically too close to the 11n antennas … This brings the result, that the speed isn’t best. Also a too high (!!!) signal can kill the link. I had this problem on a short link (500m) with Dual Polarity 23dbi Antennas at 11n 5GHz at 40MHz … The link wasn’t stable at 270MBit, it changed always to lower speed, but it didn’t look bad … I had full throughput BUT sometimes i had disconnects. (disconnects due to polling timeouts and excessive data loss) - I had to reduce the WLAN power of the card to 1db and seperate the frequencies better. If you have two antennas in the same direction (redundant links) you have to seperate most carefully and not to sent with too high signals, they can effect each other very hard. BUT: I have on this 500m link with redundant links (2 x Dual Polarity Antennas on each side) with 11n on 5 GHz and 40MHz (upper control) a extrem stable connect at 300MBit on both sides. (Even on Both links) Here i use RB600 with R52n, the indicated max. throughput on WLAN is app 150MBit i can put easily 120MBit tcp traffic on the link and it works fine at 1-4 ms :slight_smile: yeah !!!
It seems that RB800 can handle the speed up to the absolutly maximum with this outdoor 11n technology with 2 chains to app. 200 MBit … here the WLAN cards stopps speed, not the ROS :wink:
The explained ROS delay problem … which stopps TCP traffic quickly … is still within the ROS. But nstreme and WDS works arround that the ROS is not the bottleneck anymore !!!

So the MT ROS - with the help of this packet processing workarround - is still the best i have seen on market. Sometimes they put menpower not into the right direction, sometimes they cannot see real problems, sometimes it takes so much time to fix problems, some problems will never be solved because they don’t see them even if you explain, sometimes you have to do exactly the opposite thing as explained in wiki, sometimes someone tells nonsense and it tastes like truth, etc, etc, etc … but we are still on right way with MT, they only should have better ears to the customers, WISPs which test the SW for them …

rgds
Dieter

16

Dieterk!

Yes, I found out as well. I have presently two 500-600mtr links running with´n´on rb433AH boards running in a furthermore very densily used 5Ghz air spectrum (all regulatory allowed channels are in use within a radius of 1000 meters and some even twice).
With the wds/nstream solution I have stable links which had test loads of 80Mb tcp traffic running for long times without any problems.
But only after I carefully choose and separated radio channels between several units. I’ll bet with some more fine-tuning and trying even better results must be possible but for the time being the capacity is more then enough for me. I have other priorities to solve as well and wanted to leave these ´live´ links at ease for a while.

So yes, MT moved a bit forward with the help of their users base.

On the other hand, I recently bought two Ubiguity new Nanostation Loco M5 (Airmax platform=´n´) units with the new firmware (since a month) and with some very basic start configurations dropped two on the floor in two separate rooms not even looking in each other’s direction. Made a link between them and they worked immediately! 80-90Mb download fm my laptop to a file server remote! 5 min setup! Pings below 8ms! And a nice GUI with graphs in their main menu! They also improved their build-in webserver (was very slow and unreliable in the old nano’s etc) so an almost out of the box working link!

It is like MS-windows, other platforms will definitely work more stable or better or faster or or etc. but since windows has a smooth GUI combined with a bigger promotional budget it is a winner for years already.
Ubiguity seems to have understood that ´out of the box´ working systems are better marketable then ´build and troubleshoot yourself´kits from MT.
Hopefully MT will come to their senses soon or they will miss the boat in the future which would be a shame.