Community discussions

MikroTik App
 
Skaught
Member Candidate
Member Candidate
Topic Author
Posts: 146
Joined: Mon Jun 19, 2006 9:31 pm

Running out of bandwidth - Ideas?

Wed Sep 30, 2009 10:14 am

We have a link with several thousand subscribers on it. It is two nstreme turbo links bonded together. So we are using 80mhz of bandwidth right now.

Unfortunately we are exhausting even this monster link and we are running out of channels fast as we also have sectors and other point to point links in use on these sites. The link runs 60-100mbit in one direction and another 20-40mbit in the other during peak times. Latency is getting pretty bad when at full load. We have tried packet packer, nstreme exact fit 4000, and even queue trees for voip and vpn.

We have considered several options.

- 802.11n
this is in beta and too unstable for thousands of users. I also have yet to see that it can carry as much traffic on 40mhz as nstreme turbo can in 80mhz. If it can do in 40mhz what 802.11a does in 80mhz, I could consider maybe 2 dual pol dishes bonded together. But I would need a RB that has enough cpu and watts to pull it off and a few examples of this working flawlessly for others under such heavy load.

- Adding yet another dish and another 40mhz of bandwidth.
We have run out room as allowed on our site lease at our fibre location and already have over a dozen dishes at each site. We are also very short on channels and we co-locate with another ISP at one end. We also are running RB400AH and the power watts and cpu are getting maxed. We used to use PCs on our towers but the extreme weather here and instability of PC hardware made this unreliable.

- A licensed link
This is so much more expensive than MT it is incredible. We would pay up to 10x as much for only 2x the speed. Unless there is a known product out there that is cheap and can do 400mbit+ (We will need room to grow of course)

- Free Space optics, 40ghz unlicensed etc.
The link is >12km and too far for such technologies.

- Fibre
The far end is a $1M+ build for fibre.

In the longer term we are building our network into a ring which means that all links in the ring must be able to handle all traffic for redundancy. So we would be looking at 6~12 links like this.

Any other Ideas out there? I would love to hear Normis or other's thoughts on this challenge.
 
thadem
Member Candidate
Member Candidate
Posts: 115
Joined: Fri Apr 18, 2008 1:40 am

Re: Running out of bandwidth - Ideas?

Wed Sep 30, 2009 11:54 am

you maxed it out, there is no other chance than bonding in another link and wasting another 40mhz or getting a licensed link. if you are running several thousand subscribers over that link, a licensed link should be no big financial issue and you get them up to several 100s mbit/s.

if you do not have spectrum left how else would you get more speed than by getting more or another spectrum available? of course, small tweaks like 802.11n makes it better, but just slightly and if you are already planning to grow there is no other chance than to get more spectrum -> licensed link
 
Skaught
Member Candidate
Member Candidate
Topic Author
Posts: 146
Joined: Mon Jun 19, 2006 9:31 pm

Re: Running out of bandwidth - Ideas?

Wed Sep 30, 2009 9:58 pm

The last licensed link I installed was a 4mbit Western Multiplex link back in 1999.

Are there any recommendations out there based on other's experiences?

(Update - My server admin discovered our mysql sync was using 25mbit for about 16 hours a day due to a misconfig. Fixing this gains us a fair bit, but it will only be matter of time until we are back in this boat again.)
 
User avatar
jp
Long time Member
Long time Member
Posts: 600
Joined: Wed Mar 02, 2005 5:06 am
Location: Maine
Contact:

Re: Running out of bandwidth - Ideas?

Wed Sep 30, 2009 10:48 pm

We've got a solectek excel that can do 100+mbps in 20mhz of 5.8 with dual pol dishes.

Other ISPs suggest radwin 2000 for similar performance and better frequency options including 5.4.

These are likely N/mimo based but tested and polished. ubnt has a new product with similar capacity for a lot less money but it's too new for me to trust or recommend. If you're going to mess with brand new stuff, you're just as well off playing with N in routeros. Same sort of russian roulette if you've got thousands of users depending on it.

Talk with the licensed link people like trango. 600-700mbps will cost a lot, but will be worth it if you don't have to mess with it and it doesnt get outdated right away.

Alvarion has b300 gear which probably works great, but it's in the licensed link price range.
 
Skaught
Member Candidate
Member Candidate
Topic Author
Posts: 146
Joined: Mon Jun 19, 2006 9:31 pm

Re: Running out of bandwidth - Ideas?

Wed Sep 30, 2009 11:20 pm

We looked at the UBNT gear but it has a major bottleneck. The specs claim that the eth is only 100mbit. Rather odd as they advertise higher speeds than that. Could be a typo but the bleeding edge aspect is an issue. It also cannot support bonding easily.

For this one link, expensive is not a massive issue. The bigger issue is that we will need to upgrade our whole ring and add in another 6-12 links and it gets VERY expensive even on our budget. When links start costing as much as the annual salary of one of my techs I tend to try and find a better way :).

Thanks to finding our 25mbit/s "leak" I think we can stay put for 3-6 months. I am hoping by then that two 2x2 mimo nstreme 40mhz bonded links might do the trick. I am just always paranoid about the "up to" tendancy of mimo speeds. It seems that rf path conditions will make the max speed vary. But I have not deployed any Mimo in real use so who knows.

I have used many many models of gear over the years, we have just had so much success with MT and UBNT in the last few years that I am a bit out of the loop. I have never heard of Solectek so this thread has already proven very useful. Thanks!

Last time I used Alvarion was the Breezeaccess 3mbit stuff. (I think this is my 10 years in wireless this year come to think of it)

I also just remembered I used Ceragon 17ghz licensed 600mbit years ago and remember the hefty price tag. A single link was over $100,000!
 
Muqatil
Trainer
Trainer
Posts: 574
Joined: Mon Mar 03, 2008 1:03 pm
Location: London - UK
Contact:

Re: Running out of bandwidth - Ideas?

Wed Sep 30, 2009 11:33 pm

Give 802.11n a try.. Not like i'm recommending it to you, but sounds like you have no choice..
ATM it has some little problem, but with some tweaking you can get a reliable enough link.
So maybe just start playing with it in a non-critical link to get used with it, then get your own idea about 802.11n.
It's solving me some issues with bandwidth in a overcrowded frequency site.. So maybe it might help you too..
Renato Bernardi

skype: medtech5
 
InoX
Forum Guru
Forum Guru
Posts: 1968
Joined: Tue Jan 09, 2007 6:44 pm

Re: Running out of bandwidth - Ideas?

Thu Oct 01, 2009 12:52 pm

you might use extrachannels if the legal stuff don't bothers you, 4.9-6.1Ghz
 
User avatar
jp
Long time Member
Long time Member
Posts: 600
Joined: Wed Mar 02, 2005 5:06 am
Location: Maine
Contact:

Re: Running out of bandwidth - Ideas?

Thu Oct 01, 2009 5:36 pm

We looked at the UBNT gear but it has a major bottleneck. The specs claim that the eth is only 100mbit. Rather odd as they advertise higher speeds than that. Could be a typo but the bleeding edge aspect is an issue. It also cannot support bonding easily.

For this one link, expensive is not a massive issue. The bigger issue is that we will need to upgrade our whole ring and add in another 6-12 links and it gets VERY expensive even on our budget. When links start costing as much as the annual salary of one of my techs I tend to try and find a better way :).

Thanks to finding our 25mbit/s "leak" I think we can stay put for 3-6 months. I am hoping by then that two 2x2 mimo nstreme 40mhz bonded links might do the trick. I am just always paranoid about the "up to" tendancy of mimo speeds. It seems that rf path conditions will make the max speed vary. But I have not deployed any Mimo in real use so who knows.

I have used many many models of gear over the years, we have just had so much success with MT and UBNT in the last few years that I am a bit out of the loop. I have never heard of Solectek so this thread has already proven very useful. Thanks!

Last time I used Alvarion was the Breezeaccess 3mbit stuff. (I think this is my 10 years in wireless this year come to think of it)

I also just remembered I used Ceragon 17ghz licensed 600mbit years ago and remember the hefty price tag. A single link was over $100,000!
You will find licensed links start at about $12k from various vendors. Plus licensing.

I've been using Alvarion since 1997. Still got some of the breezenet gear, but only buy the VL stuff now.

Solectek is one of the oldest brands around.

If you use something that does true bridging (UBNT hasn't in the past; it sort of spoofs macs if you don't use WDS, like MT's station pseudobridge). you could use a procurve ethernet switch's trunking to bond your links. The 2600 series or newer can know when a link is down regardless of the port status, such as you'd use with a fiber or wireless bridge. All the managed procurve switches can trunk loadbalancing the traffic by mac address.
 
Skaught
Member Candidate
Member Candidate
Topic Author
Posts: 146
Joined: Mon Jun 19, 2006 9:31 pm

Re: Running out of bandwidth - Ideas?

Sat Nov 28, 2009 1:13 am

Going illegal is not an option aside from the obvious issues.

We are co-located with Industry Canada at one of our sites. They would figure it out pretty quick! Plus it exposes us to a massive risk as other ISPs could rat us out at any time. Not a way to run at the level we do.

We have looked into licensed and the link distances involved seriously limit our options. Also even with 2500 customers the $$ is tricky as we will need to upgrade up to 10 links to make the ring work right. It would be well over $20,000 a month to finance the gear.

The solutions we have settled on for now are to abandon the ring for now and we have split our network into 2 areas and feed each with a link of 2 turbo nstreme bonded together.

The UBNT mimo PtP is only 100BaseT, too slow.

And ya, we have played with 802.11n on MT and it is no where near usable yet. When it works someday, we might try using a dual pol dish, and 2 combiners to bundle 2 2x2 mimo links together on one dish and get several hundred Mbit from 80mhz. But if that works is a huge guess as the 802.11n is not even working yet. If each link is 80mbit and will do 300mbit+ we can likely get this all to work. And for now we have adjusted our topology until mimo works.
 
RK
Long time Member
Long time Member
Posts: 560
Joined: Tue Nov 21, 2006 11:22 am
Location: Winnipeg, Canada and Central America

Re: Running out of bandwidth - Ideas?

Sun Nov 29, 2009 10:01 am

We are co-located with Industry Canada at one of our sites.
Canadian, Eh? Where are you?

You could replace one of your nstreme links with a 802.11n 40 MHz link using a powerful routerboard or ubnt rocket.
It will give you more bandwidth than 40 MHz with 802.11a.

You bond this link, with fail-over enabled, with the remaining nstreme link to maintain reliability.
When you are ready, switch both to 802.11n.
 
Skaught
Member Candidate
Member Candidate
Topic Author
Posts: 146
Joined: Mon Jun 19, 2006 9:31 pm

Re: Running out of bandwidth - Ideas?

Sun Nov 29, 2009 11:08 pm

802.11n is still unstable under heavy load. It is a catch 22 as I need a heavy loaded link to properly test 802.11n but I will not do experiments that impact that many users. The best option is to wait until others try progressively larger deployments until I see something working out there that approaches our scale.

For now we have carved out enough spectrum and made some topology changes to get by until 802.11n is a tested and known technology. We found a LOS that we had been led to believe was not an option which enabled us to drop in another 2x40mhz link to a section of the network that needed it most in a way that only lost us an extra 80mhz at one tower that has lots of spectrum available.

The minute we made our changes we maxed out the dragonwave that one of our upstreams was using! They had to scramble to upgrade it to meet SLA.
 
whitenoise
just joined
Posts: 10
Joined: Mon May 29, 2006 6:15 am
Location: Bulgaria

Re: Running out of bandwidth - Ideas?

Tue Dec 01, 2009 6:12 pm

"- A licensed link"

Nera, DragonWave... No prob with 300 FD... but you must pay for the channel.

(in Bulgaria --> 34km, 40 MHz RF at 11ghz = 5k euro per year, 20y contract)

thousand subscribers via MT.... omg dark ages

Who is online

Users browsing this forum: No registered users and 76 guests