25Km 125Mbps FDX Link

RouterOS Wireless Networking
1

I have an important project to implement that requires 125Mbps FDX minimum on the 25Km Link, i need to know if any one has achieved this with mikrotik solution. I am thinking doing 3 nstreme2 links (6 CM9 cards at each end with 36dBi antenna from Radios Waves Inc.). I am a die hard Mikrotik user, and all solutions i have implement succefully with Mikrotik , in this case i am told that Mikrotik is only experimental for this kind of high level mission critical deployment. I was directed to use Proxim Tsunami Gigabit Ethernet bridge radio

http://www.proxim.com/learn/library/datasheets/tsunami_gig_A4.pdf

There should be a way to get 125Mbps FDX, any help, suggestions so that i can place order.

2

here is some proof of concept - we divided the traffic to four routerboards:
http://wiki.mikrotik.com/wiki/Super_wireless_test

this is indoors of course, but it proves that something like that can be done

3

thanks , that will mean 4 nstreme2 (using 8 CM9 cards and 8 antennas at each end). Any configurations sample for this 360Mbps link?

4

we used nstreme, but packet based load balancing. you should be able to do this as you say

5

I would use MT as much as I could. If you could not get this to work with MT, I would then use an Orthogon Spectra.

6

Thanks Normis, finally , i need to know what was the signal strength is the 360Mbps link at Wiki. I am thinking i need to have the same or better signal strength, so i need to do a link budget for 25km to see if the CM9 and 36dBi from radio waves Inc will hit or beter that signal strength.



regards

7

It was never explained how this was really done. Even with 4 routerboards on each end and two radios in each, that’s still 45Mbps per radio. I’ve never seen that performance on a routerboard.

8

Hi Normis, withouts some details like the signal strength on each of the 4 routerboard links then this achievement is useless to me, and i cannot use it as a yard stick to implement mission critical links. I need records of the signal strength , CCQ etc because that is the minimum required to reachieve such a throughput all things being equal.

regards

9

Well… as Normis clearly wrote this was done indoors, and as you can easily see if you look at the pictures the units are just a few feet apart! I’m pretty sure that you will have a very hard time duplicating their signal strengths on a 25km link!

However… as we can see from the pictures they are running something like 140-180Mb/s in each direction, according to the (very limited) text I am assuming that

  • they had one P4-based master router at each side
  • these two P4-based master routers did the actual load balancing on 4 Ethernet Gigabit interfaces
  • each Ethernet interface was connected to one RB5xx box with dual cards running Nstreme2 (total of 8 boxes counting both sides)

If these assumptions are correct (which we should be able to get confirmation on.. Normis?) then each of these 4 links would have to carry 35 to 45Mb/s

So… if we look at taking 45Mb/s over a 25km link we would need (assuming SR cards AND a absolutely clear los AND no interferense) -77dBm at the rx-side (sensitivity for 48Mb on the SR5 card)

So a link budget for that number…
tx SR5 power @ 48Mb = 22dBm
tx pigtail = -0.3dB
tx short good cable (1/2") = -1dB
tx high quality connectors to antenna = -3dB (approx)
tx high gain antenna = 30dB
25km link = -135dB
rx high gain antenna = 30dB
rx high quality connectors to antenna = -3dB (approx)
rx pigtail = -0.3dB
rx short good cable (1/2") = -1dB
leads to 22-0.3-1-3+30-135+30-3-0.3 = -61dB!

That should give you a link margin of roughly 16dB… not sure about your climate, inversion zones, humidity etc but it might be enough

So your scenario should work… HOWEVER if I where to build it I would do it slightly different. Do you customer absolutely require 125Mb or might 120Mb be enough?

If we where to take the test setup and instead of running 4 links flat-out we’d run 5 links capped to 24Mb… results?

  • gain tx power, 26dBm instead of 22dBm
  • improve rx sensitivity, -86dBm instead of -77dBm

This would give us a power on the rx-side of -57dB, coupled with a rx sensitivity of -86dB that is 29dB margin!

Taking this idea further… and assuming you can get enough free spectrum (narrow band maybe?) you could go with even more links on lower gain cheaper antennas. Thus gaining even more rx sensitivity since you would run lower bitrates on each link.

Well.. enough of my ramblings, hope I haven’t muddled the waters for you :slight_smile:

/Jörgen

10

HI Normis,
How I can found configuration including load balancing?

Regards
Xandor

11

configuration of what?

12

HI Normis,
Of your super wireless test… I want test similar configuration but with two radio cards and load balancing.. On Wiki web site I haven’t found any configuration about that as example…

Regards
Xandor

13

ok, i will make sure eugene finally put’s it on the wiki. sorry for keeping you all waiting.

14

HI Normis,
Thanks for your gentility

15

Eugene posted his high speed wireless test config:
http://wiki.mikrotik.com/wiki/Super_wireless_test

16

We’ve tested this some time ago, and few of our clients using it for long time now…

and this are problems we have met:

  1. Floating signal (for ex. day -69 night -75) = floating speed, you can’t do anything to remove this efect, that’s the worst problem…
    depends on weather/humidty/temperature/electromagnetic fields

  2. Sometimes nonsymetric [very irregular] (for ex 30Mbps download / 11Mbps upload) propably mikrotik bug, but we’re not sure about, cos such long link is very bit advantage…

  3. Problems with placing dish on high towers, we used 28dBi GRID instead

  4. Problems with high power mPCI’s - we’ve used only SR-5 on transmit side, on recive side we used wmia-123AG - that was working the best

  5. the best throughput of dual @ 26km

2x SR-5 (only for transmit)
2x Intel Bayfield 865GBFL
2x Intel Celeron (northwood, selected → lower temperature) 2,4GHz
2x WMIA-123AG (only for rec)
4x Grid 28dBi (5320MHz + 5620MHz)
2x Chieftec/Delta power supply
2x Intel Gigabit NIC’s
2x DOM with MT 2.8.21 and 2.9.x (where x was 5,6,8,11,13,14,17)
4x mPCI->PCI adapters
2x Legrand outdoor boxes
4x Andrew cabling with Andrew connectors, ANRITSU SiteMaster tested
4x Pigtails ufl->Nfemale
2x Fiber - GigabitETH coverter
Fiber uplink on towers,
2x online UPS from APC conected directly to PC’s power supply
WITHOUT any kind of surge/lighting protection

first conclusion:
best performance was on pair 2.9.11 + 2.9.11 mt’s
on table bandwidth test show us abut 75Mbps full duplex real world UDP (good™ :wink: ) but not as good as on two wmia 123AG where was about 81/83Mbps

on link:

Signal strengh :

second conclusion, normal traffic (backbone 30Mbps, but shaped per user, max was seen 18/12)) upto 10000pps, ping below 1ms to 1,5ms normal, sometimes 2-3ms

third - maximum throughput in normal condition:
half duplex 33-35Mbps download and 12-24 (very unstable) upload
full duplex… should be 30/10, but when download goes up to 25Mbps upload drop to 1-4Mbps, “little” asymetrical

best performance, wery good weather:
MT’s forced to 36Mbps (-67dBm / -68dBm)
45Mbps downlink hdx
28Mbps uplink hdx (unstable)
full duplex 29/8 - still asymetrical

we swaped chanels, (previous downlink was 5620, now 5320) but situation won’t change, still downlink has better signal, and better performance.
Because it’s on 100m towers we can’t experiment as long as we want, especialy that is 1200 user main (and only :/) backbone


so 130Mbps on 26km, i think that’s little hard to do, but not impossible.
If you will have tuned antennas and very much time to test you probably gain 130Mbps fdx.


If u want i will try new 2.9.23 and 2.9.22 versions maybe smtg will work better or maybe someone want to login with read priviledges and see it ?


UPDATE:

I forgot that isn’t single link but two links connected cos provider can’t see our tower :

Provider<-> FastETH ↔ MT < = 2km ~70Mbps fdx real world = > MT - GiabitETH - MT < = 26km as i wrote before = > MT - GigabitETH - copper2fiber converter 150m fiber ------ converter - GigabitETH - Linux server

UPDATE2

i forgot :wink:
You can make the same but 2x 13km links, we have experience with them (about 65Mbps fdx very stable throughput on 14km) even you don’t need high power cards, you’ll need only good 28dBm grid’s antennas

17

Note, when the rate jumps around, it is not a problem – that is how it is supposed to work.

John

18

yes if all working fine fallback is very helpful but if rate is going down rapidly (48 → 6Mbps or 48->36 ) we noticed very big performance loss, if we set up solid rates all is working fine, there’s someting with it…
On “other” devices rates are working with probing + delay, if 36M is max, then set 24. Mikrotik uses highest availabe rate, but depending on wheather condition 1dB +/- floating signal is normal, so mikrotik see that 1dB drop is not enough for that speed, and lower througput to 24Mbps, ther is little packet loss on this action, but this is aceptable on startup or once for some peroid of time (for ex. 30min), next signal is going 1dB up, so mikrotik set 36M (packet loss…) then 24M (packet loss…) […]

But if you will watch autonegotiation = 36, then set 24Mbps, latency will be perfect, performance ok.
Normaly we just disable higher speeds, but from 2.9.8 few times per minute speed is going down (this is connected with extensive data loss error in LOG) so we have few % packet loss /day, that is UNACEPTABLE, especialy for voip aplications…

19

OK, I´ve tried everything in the menu, but I still can´t get more than 70 Mbps with BandWidth Test using TCP. With UDP I have reached more than 300Mbps, but when I change to TCP things slow down A LOT.
What is the “state of the art”, the last word in reaching high bandwidth throughput using wireless connections (you can use everything in the kitchen: NStreme 2, Bonding, whatever)

I REALLY need to get more than 120 Mbps in 15 Kmts. It HAS to be wireless.
And I guess bonding 16 routerboards in each side is kind of too many radios… How can I achieve this kind of perfomance with the minimun acount of radios.

Any ideas???

20

If i need that kind of thorughput i just buy or lease Ericsson Minilink HC
but if you need to do this using MT calculate:

Maximum Performance is at -55dBm signal point, to archieve -55dBm on 15km, you’ll need two 28dBi (real gain) antennas
so then 2x Nstreme dual is about 120Mbps real throughput, but you’ll need to use 2 PC’s (with 2mPCI) per site
Then you shold buy management switch with link agereation option, but working per packet (switch is better, because it won’t add much delay)

Let’s make some calculations (in Euro):
4x Complete PC units with outdoor enclosure = 4x 500 = 2000
4x Antennas (pac wireless grid 28dBi) = 4x 100 = 400
4x Cables = 4x 3m CNT-400 /w Andrew Nm/Andrew Nm = 4x 20 = 80
2x Switch = 2x 400 = 800

= 3280Euro

And yes it’s 1/3 price of ericsson, but remember that you’re working in unlicensed band

Maximum possible bandwidth for wireless PtP link