Hey!

Warning: Wall of text ahead!

We are starting a WISP setup in a fairly large city. The only competition is a state owned crappy DSL service. Our goal is to provide reasonably priced flat rate Internet at 1+ Mbps.

In the beginning, we are only going to offer a max speed of 1Mbps. We will increase this based on customer demands and number of customers(Since buying uplink in bulk will be cheaper). We will only have one high site(tower) for now. We will be covering a area of 3-4Km radius.

We have planned to first saturate the 2.4 GHz band. This will allow us to keep deployment costs down. We will upgrade to 5.8GHz later as required.

I am fairly new networking including wireless comm and I have these doubt about our current setup.

1. Assuming that I only use one RB with 1 miniPCI, and attach a 801.11g or 802.11n card with a single antenna port. Can I or should I spread this signal using 3 sectors? Using 3 sectors instead of a single omni would be better or not?

2. I know that only a max of 3 independent networks work in the 2.4GHz band because of the limited bandwidth and overlapping channels. Now, I would like to use all the bandwidth that is available so that I can reach maximum customers. How would I do this? I have the two scenarios in mind and I don't know which is better.

(i) Have 3 RBs, each with 3 miniPCI. Attach 802.11n (20MHz channel) cards to each miniPCI of each RB. Now, put each 802.11n on one of the RBs on channel 1 and attach 3 sector antenna to distribute. Put the next RB's 802.11n card on channel 6 and again 3 sectors to distribute. The third RB works on channel 11.

802.11n advertises 72.2Mbit. I am assuming a throughput of 25Mbit. Please correct me here and provide some details. I will have total of 75Mbit throughput and I can connect customers in 360° coverage.

(ii) Another topology can be that for each 120°, we can have 3 sectors(Or maybe a single one will do? I also need to know this.) working on different channels. The same would be true for other remaining parts of space. Since each 120° won't interfere with each other, we can have 3x3x25= 225mbit! By just using 802.11n. But this seems too good to be true and hence I would like your feedback here. Also I would have to distribute customers evenly for max throughput.

What would be the best topology?

3. Throughput. From what I could understand, 802.11n boosts 802.11g's 54Mbps to 72.2 Mbps. Then if you use 40MHz channel width, you can have 150Mbit and a max of 2 channels(That too congested since max bandwidth is 60MHz or 72MHz depending on your region).

So with 802.11g, 3 independent channels of 20 MHz x 20mbit = 60 Mbit throughput. With 802.11n, 3 independent channels of 20 MHz x 25 Mbit = 70-75 Mbit of throughput. With 40MHz channel width and 2 channels, 100 Mbit of throughput?

I know real world conditions will alter the throughput. So how much can throughput can I really expect from only 2.4Ghz? How much for a dual 2.4 + 5.8 setup?

4. MIMO. How can this be implemented? Would we have to use 2 sectors spaced apart a little in the same 120° air space? How much bandwidth do we gain if the CPE has multiple antenna. How much if it does not? What type of AP and CPEs would I need for this?

5. If we are installing sectors, what kind of antennas should the CPE have? The cheapest are panels antennas and we have dual polarized on the expensive side. What other types of antennas are available and how do they compare to each other?

PS: So I just read that the improvement from 802.11g to 802.11n is only 54 to 65. For the 72.2Mbit, one need so set the guard interval to 400ns. This will decrease the stability under interference or high load?

Also, since we can have 3 perfectly independent channels with 20MHz bandwidth and the total throughput can be spread among users, it is always better to run 3 independent channels at 20MHz for a WISP offering <10 Mbps per customer?

Please let me know if this post is not appropriate or is asking too much.

Thank you for reading!

One radio per antenna. Do not use a splitter to feed three 120 degree sectors.

Use enough bandwidth to support actual customers. 20Mhz channels mean you hear twice the interference as you would at 10Mhz. 10Mhz also plays better with multiple sectors on the tower. You want a minimum of 10Mhz between channels if at all possible. 20Mhz is better. Whatever channel width you use, there will be some spectrum used outside the designated channel width. You can interfere with yourself even if your channels don't overlap.

MIMO, you need to do more research. This is something you should already understand before you get serious about starting a WISP.

I would start with 10Mhz channels. Especially if you think you are only going to sell 1Mbps packages.

I would look at noise floor levels for 2.4 GHz and 5.x GHz before ordering APs. Where I am, 5 GHz is cleaner and easier to support WISP type activities. In a city, expect a high noise floor on either band.

Forget 2.4ghz band and go by 5ghz. Use more sectors with their own radios. Make some additional shielding between the radios/antennas. Connect the clients by as much as directional antennas. Do not mix manufacturers of ap/client devices.

Sent from Android by Tapatalk.

One radio per antenna. Do not use a splitter to feed three 120 degree sectors.

Use enough bandwidth to support actual customers. 20Mhz channels mean you hear twice the interference as you would at 10Mhz. 10Mhz also plays better with multiple sectors on the tower. You want a minimum of 10Mhz between channels if at all possible. 20Mhz is better. Whatever channel width you use, there will be some spectrum used outside the designated channel width. You can interfere with yourself even if your channels don't overlap.

MIMO, you need to do more research. This is something you should already understand before you get serious about starting a WISP.

I would start with 10Mhz channels. Especially if you think you are only going to sell 1Mbps packages.

I would look at noise floor levels for 2.4 GHz and 5.x GHz before ordering APs. Where I am, 5 GHz is cleaner and easier to support WISP type activities. In a city, expect a high noise floor on either band.

I have almost decided to go for a 802.11g(3x120° sectors) setup for first few customers. How many users will I be able to support If I limit the channel to 10MHz?

There are no other WISPs in the city, so there will not be any competition for spectrum for now. I will keep in mind what you said about noise.

@pteek
Go and get practical experience from another WISP before you spend money on this project, budget (not joking) your max figure then multiply that by 10, you will require several sites as just one will not have the coverage even at 3kms radius - Line of sight(LOS) issues, basically you need to learn the limitations of wireless networking and not what theory says?

@pteek
Go and get practical experience from another WISP before you spend money on this project, budget (not joking) your max figure then multiply that by 10, you will require several sites as just one will not have the coverage even at 3kms radius - Line of sight(LOS) issues, basically you need to learn the limitations of wireless networking and not what theory says?

I have actually worked on small setup with one miktrotik RB and a omni. About 20 clients. I have a decent sized budget, about 2 times of the total that I was quoted for everything.

I really want to do this and I am ready for a slow start or even a back roll. But demand is very high here and I think we will pull it off.

@pteek
Go and get practical experience from another WISP before you spend money on this project, budget (not joking) your max figure then multiply that by 10, you will require several sites as just one will not have the coverage even at 3kms radius - Line of sight(LOS) issues, basically you need to learn the limitations of wireless networking and not what theory says?

I have actually worked on small setup with one miktrotik RB and a omni. About 20 clients. I have a decent sized budget, about 2 times of the total that I was quoted for everything.

I really want to do this and I am ready for a slow start or even a back roll. But demand is very high here and I think we will pull it off.

Good to read you have worked on a wireless system, this should tell you how demanding wireless is and actual throughput can be totally different to theory, sectors - co-location interference - wireless protocol used- LOS - freznel zone - repeater site(s) for coverage, etc. etc.etc.

Maybe best to plan your project with a professional consultant who can give advice on equipment requirement and help with configurations - remember you can only sell when you have a good and reliable service operational.

I have almost decided to go for a 802.11g(3x120° sectors) setup for first few customers. How many users will I be able to support If I limit the channel to 10MHz?

There are no other WISPs in the city, so there will not be any competition for spectrum for now. I will keep in mind what you said about noise.

I would go 802.11N MIMO from the beginning. There is no point in beginning with 10 year old technology.

Do not count on more than 30 - 40 clients per AP, whatever the channel width. You might get more, but do not count on happy customers if you do.

How many people have 2.4 GHz APs in their homes / apartments in your coverage area? Even if other WISPs existed, they would likely be the least of your worries in a city environment.

@lambert

Not a lot. There are lots of areas with no APs at all and maximum would be about 20% homes in a area. Also apartment buildings are not typically seen.


Is MIMO always dual polarized? I have seen a lot of dual polarized equipment and can't understand this. Will MIMO work with a dual polarized antenna using each polarization separately? Would this type antenna have two RF connectors?

Or MIMO NEEDS two antennas separated by a distance. Also, if I have a MIMO sector, then what type of panel antenna will work in MIMO? Is there a special design or just two panels antenna separated by some distance?

About the 2.4Ghz and 5Ghz thing, only unlicensed bands for outdoor use in here are 2400 MHz - 2483.5 MHz and 5825 MHz- 5875 MHz. 5GHz equipment costs more and has lower total bandwidth, tho it will be pretty much interference free.

How would I cover 360° with only 50MHz bandwidth? How bad would a 40MHz 2x2 MIMO omni would be? Is my only option for 5GHz is 3x10MHz?

Use good antennas with good shielding for any chance of frequency re-use. We are tending to use antennas from http://www.kpperformance.ca/2-4-ghz-antennas You may be able to use vertical antenna separation to improve frequency reuse.

I have much more frequency available in 5 GHz band. You could try to use an A B A B configuration where the A channel antennas are back to back, but separated by 3 or 4 meters vertical distance. The same for the B channel radios.

Most 2 chain MIMO antennas have vertical and horizontal polarity radiators separated by a few centimeters. They do not take up much extra space. They do have two coax connectors. MIMO radios have two transmitters / receivers. http://en.wikipedia.org/wiki/MIMO

MIMO will get you nearly twice the throughput per channel. At your speeds, you get may more customers connected by using SISO on different polarities. Your situation is very different from mine.

Are you using N-Stream or NV2 now? If not, that is likely your best bet for getting more clients per radio. http://wiki.mikrotik.com/wiki/Manual:Nv2 Ubiquiti's AirMax protocol is a similar attempt to bring TDMA to the 802.11 chipsets. Ubiquiti AirMax is not compatible with MikroTik N-Stream or NV2. TDMA should allow more clients per channel-width.

With any TDMA scheme you will have to ensure that all of your client radios are running the same firmware as the AP is running. If you've been using random devices as CPE, you will have to replace them to go to a TDMA protocol.

Ubiquiti claims up to 100 clients on their AirMax. I don't believe it. We have high noise levels here and faster packages than you sell, 3 - 10 Mbps. We try to keep each sector to less than 40 clients. Client connection quality is very important for maximum number of clients. Your clients all need to achieve maximum modulation rate, reliably. Any client that does not achieve maximum modulation slows the entire AP by a disproportionate amount.

Omni antennas are evil. They hear all the noise and contribute to the noise floor heard at your other towers. Spend time reading up on Radio Frequency topics. The more you understand RF propagation at these frequencies, the better you can design your network to avoid being your own worst enemy.

Email me if you are interested in having a phone conversation regarding these questions. I can help you with a lot of areas including creating RF propagation simulations that will show you your approximate coverage area with the various configurations and hardware that you are considering. josh (at) thinkboxi.com

@lambert

That cleared up a lot of things for me. Thank you very much sir.

@joshaven

Very generous offer. I will keep that in mind.

I have a few more questions. I want to use WPA2 enterprise in the network because that is the only good way to secure the networks IFAIK. A PSK could be easily compromised and a open network is a no-no.

Is WPA2 enterprise this feasible in a PtMP? Another thing I can't find out about is weather the dynamically generated keys are same of every user or different?

The most used AAA in WISPs is captive portal. I would like clients to have a true plug and play experience. I don't want to use static IPs and also want to avoid PPPoE. What are my options here with mikrotik? I mean if I use Mikrotik CPEs?

I am not sure I fully understand your situation... however I think it is entirely reasonable to use standard WPA or WPA2 for a connection between customers and your infrastructure equipment.

You don't have an obligation to secure the wireless data. The only party that can truly be responsible for the security of the information is the end parties not the ISP in the middle. For example a bank website will use https which means the information that must be secured will be encrypted in a SSL tunnel between the end web browser and the banks server. The consumer is not really any more or less protected by your choice of wireless encryption protocol because the information is secured from end point to end point with or without your wireless encryption. Furthermore the consumers network should be behind a NAT firewall which means that the consumers internal network is not risked by the security of the transport data.

Also, if you secure the wireless connection you have to realize the traffic is only encrypted while traveling between the access point and station... the rest of the way through the public internet it is not encrypted.

All of that said... I would use WPA2 in a mixed network however I would strongly prefer an all MikroTik network and I would use NV2 exclusively. NV2 does't use the security profiles so the question is not applicable to NV2.

In regards to AAA I would only consider the use of EAP if I was providing additional settings on a per connection basis like QoS profiles. However I would be more likely to run PPPoE (which could include encryption all of the way through my network without any consideration for wireless encryption.)

I don't understand your network plan but I would strongly urge you to consider using an OSPF routed infrastructure with MPLS/VPLS switched tunnels between your end clients and your data center (or primary internet connection) along with PPPoE tunnels for all customer traffic (you can encrypt the PPPoE tunnel for added security if you like). Here is a presentation urging the use of this network topology for an ISP from the last US MUM: http://mum.mikrotik.com/presentations/US13/kirnak.pdf

Well, I am not able to convince my self to simply use WPA2 since the key will be easily know by a customer and than an attacker can listen to all unencrypted web traffic. Why would I even bother with encryption and take the performance overhead if it is not secure. Might as well leave the network open. But that would suck too. This might just be me being paranoid.

Looking at the example provider network in the presentation, I can defiantly say my network will be very small. Also the config seems a lot for me to handle right now. First a perfectly working OSPF and then a MPLS over that and then VPLS over that and THEN PPPoE over that.

I am going to talk to the initial customers and will do my best to go fully Mikrotik. This way I will have NV2 and MIMO. NV2 seems more and more a must have for PtMP. Then there is AIRsync which is fancy but I don't like the proprietary feel of ubnt and the premium antenna pricing.

EAP seems like the go to solution. How would I handle public IPs with EAP? SNAT private IPs to public at core? Or use a public/24 with DHCP directly on the wireless L2?

Or a simple encrypted PPPoE over a open wireless network makes everything a breeze including AAA.

TOPOLOGY:

A will get a port at the upstream provider's facility. I will carry that to my NOC(nothing like a real NOC,yet) on fiber and a wireless backup(only 400 meters).

There will a tower at the NOC with 3-4 sectors on 2.4 and another 3-4 on 5.8. There will be two more towers in future. They will be connected by wireless back haul.

Future:

I expect about 1000 customers on wireless. About 40-50 per sector AP. I will of course get a second upstream provider.

This will saturate the ISM band and I will look into deploying FTTH/FTTC without going bankrupt. And I will launch in other cites.

EAP will integrate greatly with a wired Ethernet network.

Hopefully I am not being too naive.

Well, I am not able to convince my self to simply use WPA2 since the key will be easily know by a customer and than an attacker can listen to all unencrypted web traffic. Why would I even bother with encryption and take the performance overhead if it is not secure. Might as well leave the network open. But that would suck too. This might just be me being paranoid.

I would not be afraid of running an open network. However you would be running the risk of having clients connected that you don't want connected but this can be managed.

Looking at the example provider network in the presentation, I can defiantly say my network will be very small. Also the config seems a lot for me to handle right now. First a perfectly working OSPF and then a MPLS over that and then VPLS over that and THEN PPPoE over that.

I am going to talk to the initial customers and will do my best to go fully Mikrotik. This way I will have NV2 and MIMO. NV2 seems more and more a must have for PtMP. Then there is AIRsync which is fancy but I don't like the proprietary feel of ubnt and the premium antenna pricing.

Ubiquity is good in that it is easy to work with however I can assure you from extensive personal experience with both you can do more with MikroTik's not only in regard to capabilities but also in regards to the robustness of NV2 over AirMax not to mention the better packets per second due in most part to higher performance hardware. In regards to AirSync... RUN! don't walk RUN form AirSync... If you want sync then go Cambium! Camium's GPS sync has great advantages when your going for density. For instance they can perform very well with back to back frequency usage like 6 sectors using 3 frequencies. AirSync will greatly impact the performance of your network. It's never worked right unless something big has changed in the last few months that I haven't heard of. The problem is that AirMax is based on 802.11 which doesn't get along well with the sync technology.

EAP seems like the go to solution. How would I handle public IPs with EAP? SNAT private IPs to public at core? Or use a public/24 with DHCP directly on the wireless L2?

EAP is not that great in my opinion... however it accomplish much of what your talking about. EAP doesn't really relate to IP's... EAP is a gate keeper for who is allowed to make a wireless connection to the AP. The only special super power of EAP is that it does a RADIUS request through which you may be able to so some fancy things like configuring dynamic queues or mangle rules (I haven't tried setting QoS through an EAP connection but I suspect is would be pretty easily done).

I would prefer to solve your issues with PPPoE rather then EAP which is basically a way of using a username and password to connect to the wireless where PPPoE is a username & password tunnel between end points. With PPPoE each customer would have their own interface on your PPPoE concentration router which gives you some nice tools like speed limiting the interface as well as graphing etc.

Or a simple encrypted PPPoE over a open wireless network makes everything a breeze including AAA.

TOPOLOGY:

A will get a port at the upstream provider's facility. I will carry that to my NOC(nothing like a real NOC,yet) on fiber and a wireless backup(only 400 meters).

There will a tower at the NOC with 3-4 sectors on 2.4 and another 3-4 on 5.8. There will be two more towers in future. They will be connected by wireless back haul.

Future:

I expect about 1000 customers on wireless. About 40-50 per sector AP. I will of course get a second upstream provider.

This will saturate the ISM band and I will look into deploying FTTH/FTTC without going bankrupt. And I will launch in other cites.

EAP will integrate greatly with a wired Ethernet network.

Hopefully I am not being too naive.

I still recommend MPLS/VPLS right from the start. Yes it is complicated, yes it is overkill for the first customer... However I have helped out many small WISPs that have made a big messy flat network because it was not complicated and wasn't overkill.

First off, when I say client I am talking about homes or businesses as a "client" not individual devices... each "client" that I am talking about would represent multiple devices...
I do not recommend panning for 40-50 clients per sector. Try keeping that as a worst case maximum even with Cambium radios (which I believe deal better with higher density per AP). I have seen Cambium AP's perform well a few overs 60 customers although there is noticeable performance drops during busy times... I have never seen a healthy 802.11 based AP with over 50 customers on it. I would say that the top end for any 802.11 based AP should be 40 clients and that is probably pushing it beyond what your subscribers will tolerate... If your talking about 20MHz channel widths then I would plan on being able to deliver 30Mbps per AP (assuming decent signal levels)... I have found that you can easily oversell by a factor of 12. This means that you could budget to sell 12*30Mbps or 360Mbps from that AP... with 10Mbps connections that would be (36 10Mbps customers) on an AP. I believe this number is decent for a budget. If you intend to go for 20Mbps per customer then 18 customers per AP is a good budget. You could exceed these budget numbers by a few but I doubt anyone would argue that these are not sane. If you do plan to meet these budget numbers then please DON'T install pore connections. Watch your modulation rates, noise floor & signal levels!

In regards to saturating the available RF bands... If you plan well then you will be fine... others have successfully built large WISPs with unlicensed frequency bands.

Mikrotik+NV2+MIMO+PPPoE is the deal.

I am also keeping other things you said in mind.

Do you think I can pull off a selling factor of 12? Many(approx 30%) of my clients will be heavy users. I can't really predict this but I am expecting upto 70% of the clients being online at peak time and 35% doing heavy transfers. I am thinking of keeping the WAN contention to about 5-6, unless I see a lot of unused uplink. Is this my paranoia? What kind of clients have you dealt with?

A selling factor of 12 could mean 1000s of customer.

New questions pop up:

1) Why can't I have a simple IP network here? With every mikrotik being a router(more like a aggregation switch with isolated clients) and each wi-fi client being a host. Then all the mikrotiks go through a BRAS server/router which goes out via a edge router to uplink. Is there any way to auth client expect for captive portal or MAC or static IP? Again I look at EAP.

Why is this network not systematic(?) or ass good as compared to a MPLS/VPLS?

2) If I ditch Mikrotik CPEs and use simple(read cheap) CPEs that have 802.11g and a panel antenna. I can have four sectors, each will have two APs, one horizontal and one vertical. This will allow me to keep about 20 station per AP but without NV2 TDMA. How much latency, throughput and packet loss would I see at peak times?

Do I just need to rotate the panel antenna by 90° to switch between the APs or I need to have different vertical and horizontal polarized antennas?

Those are the last of my doubts. This place is wonderful. I will post my final network plan for your comments

One major caution I have for you is that your talking about customer installs. Customers will not know enough about the interface to login and get signal levels nor will they know that a -50 is a better receive signal then -80. The problem is that you'll end up with lower signal levels then if you have a qualified technician installing the equipment. One or more bad connections will have a large negative effect on everyone on that AP. If you are planning to proceed with this then I highly recommend that you find a successful provider that is doing it and have a good long conversation with them to talk about their experience. I expect that you will find people receptive to talking about their success. If others are not having success doing this then take extreme caution. One good way to do this would be going to trade shows I've learned a lot from other WISP's at: (MUM's mum.mikrotik.com, WISPA events http://www.wispa.org/, Animal Farm http://www.afmug.com/)

Regarding good links, I would recommend this resource for you to ensure you know most of what is involved with establishing good connections: http://mum.mikrotik.com/presentations/PL12/ahmad.pdf

Yes I believe you can pull off a 12x multiplier on an AP. You may need to employ QoS to keep low latency on important services like VoIP and to keep things like videos with constant streams but I defiantly believe this number can work. Keep in mind that 12x is a target and a maximum for an AP. Try to build to that but not exceed... a 10x is a good place to be 14x not so much however it used to be as high as 20x but that was before the days of Nexflix.

The larger the pool of customers using a shared resource the higher your multiple can be. If I run a 17x on an AP I will get complaints... If I run it on my up line connection that is the aggregate of thousands of customers I don't exceed 80% utilization. Consider a really small pool with a flat multiplier to see how it scales... consider a pool of 1 customer. Could you sell to one person 12Mbps and only purchase 1Mbps... or sell to two customers 12Mbps each and purchase 2Mbps... of course not so the multiplier scales with the size of the pool.

The network that I am getting my 12x from has two feeds of 250Mbps with an average peek load of about 60% to 80% per pipe and about 3500 customers. The most popular bandwidth package is 1.5Mbps although they range up to 20Mbps so I would assume the average is something like 2.5Mbps... This would be roughly 8750Mbps sold... that number divided by the bandwidth purchased is about 17.5x. At a size of 3500 customers I am sure we can exceed 20x.

This is a good resources that shows an example of managing bandwidth with QoS to offer good consistent connectivity for things that can consume resources like streaming videos. Implementing things like this would help you reach your budgets without having pore performance: http://mum.mikrotik.com/presentations/US13/steve.pdf


Answer to Question 1: Simple network (I am reading "flat" layer 2 network): "Friends don't let Friends bridge networks".

I hate arguing because that's the way its done but one of the quickest and best answers I can think of is that none of the carriers use large flat Layer2 networks and when the offer Layer 2 connectivity its with MPLS. Your offering internet service and internet service is a web and is routed... Break the best practice model at your own risk (read 2am network emergencies and large support call volumes).

Layer 3 offers a lot of packet efficiency. For example you want to limit your broadcast traffic. If every time one client does a broadcast (like asking for an IP via DHCP) the broadcast would go from the station to the AP... then The AP would repeat it to everything on the bridge (All of the other Stations as well as the upline router). Then the upline router would repeat it to your other routers at the other tower sites and those routers would repeat it to the AP's and those AP's to the Stations... until finally everything in your network has heard that broadcast and ignored it except for your DHCP server which would answer through with unicast directly to the requester. Now this can be greatly reduced by using VLANS to separate each tower site into it's own Layer 2 network and you can also use client isolation on the AP and drop broadcast on the routers bridge. What your doing with this concept is basically creating tunnels between your AP and head end... at this point you should be thinking: why not just go all the way and create VPLS tunnels over MPLS between your end customer and core router (not a bad idea). Another big reason to have a routed network is that you cannot have Layer 2 loops (meaning redundancy) in your network without a headache of a Layer 2 routing protocol for loop avoidance or it will lock up your equipment. (You want routed loops with a nice link state routing protocol like OSPF because this avoids 2am network emergencies!)

The bottom line is that you don't want a large flat network. Route traffic to each site (OSPF), interconnect each site (create a web) with multiple paths for the traffic so that a power outage or failure of a single piece of equipment doesn't take half of your network down. Did I mention not having 2am emergencies... I was happy when those went away.

Answer to Question 2:
If you go with another vendor for equipment be sure to have some form of TDMA. The standard 802.11G is designed to operate where each client can "hear" the other clients. It waits to transmit until it hears an open time similar to a bunch of people in a room that can hear each other talking. If the clients begin talking over each other to the AP because they don't hear someone else talking then you will have collisions which is the AP distinguish between the clients... The solution is TDMA (Time Division Multiple Access) (read everyone having a scheduled time to talk). For successful outdoor wireless a scheduling mechanism is a must.

It is best to use both H & V on the same AP to reach the higher modulations. It would be possible to increase the number of AP's by creating a grid of H & V antennas... however offsetting an antenna by 90º only gives you 20db of separation. If your going to re-use a frequency on a tower you will need a lot of vertical separation as well or you will have to have the AP's in sync with one another (read Cambium with GPS sync). Without this the AP on the tower that is on the same frequency or on a near by frequency will be louder then any of the clients even if the antenna is pointed the opposite direction and turned 90º. ( TX: 20db - Polarization: 20db - FrontToBackRatio: 35db - very little Freesapce loss = could be as high as -40 depending on free space loss).

I recommend using MIMO 2x2 radios with NV2... set your channel width to 10MHz if your wanting to double the number of AP's on a tower... 3 non-overlapping channels becomes 6 non-overlaping channels... An array of 6 60º sector antennas is nice (get the best AP antennas you can). If you improve your RX & TX on one side by having a better antenna then your improving the entire link... With a better AP you improve every customer link with a better customer antenna you improve only that customer. Don't skimp on the AP. Also, increasing the passive gain of an antenna is much better then increasing the transmit power because increasing the antenna gain also increases the receive signal whereas the transmit power does not effect the Rx levels.


I think you still should give me a call. I highly recommend using a good consultant to help you build your network. It may cost a little extra money up front (there is a real chance it will save you money too) but I believe your return on investment will be very rapid. You don't just need someone with networking knowledge, you need someone that has worked with multiple networking models in the real world, that understands when something doesn't look right because they have fixed this kind of problem before. From the sounds of it you will learn all of this really quickly but there is a lot to learn and some of it is more art then science. Also you should consider systems integration early on. When working on support you need to get form installation notes to call history to usage graphs to billing etc fairly easily and quickly. The more information you have easily accessible the quicker and more accurately you'll be able to get a solution. Support costs are one of those hidden costs that can be real high or real low but that depends on how you build things. Furthermore you don't want to get to the point where you have to call in an expert and have them work on your network but they are lost on how everything is connected and where your issues are etc. Good design good planning and good record keeping will make your network need less maintenance and less time to repair issues when there is a need. One such example would be consistent and simple naming and addressing schemes. You don't want to talk to an consultant who is charging by the hour where you talk about the issue with Fred's tower and then mention Jan's tower and 10 minutes later you find out that the engineer thought those were two different towers and really they are just a married couple... Believe me bad habits die hard so be thoughtful from the beginning about scaling issues, keep good notes and records, keep things clean.

One major caution I have for you is that your talking about customer installs. Customers will not know enough about the interface to login and get signal levels nor will they know that a -50 is a better receive signal then -80. The problem is that you'll end up with lower signal levels then if you have a qualified technician installing the equipment. One or more bad connections will have a large negative effect on everyone on that AP. If you are planning to proceed with this then I highly recommend that you find a successful provider that is doing it and have a good long conversation with them to talk about their experience. I expect that you will find people receptive to talking about their success. If others are not having success doing this then take extreme caution. One good way to do this would be going to trade shows I've learned a lot from other WISP's at: (MUM's mum.mikrotik.com, WISPA events http://www.wispa.org/, Animal Farm http://www.afmug.com/)

Regarding good links, I would recommend this resource for you to ensure you know most of what is involved with establishing good connections: http://mum.mikrotik.com/presentations/PL12/ahmad.pdf

I think you misunderstood somewhere. I will be installing the CPEs and will personally train my staff to do this. I have already done it at the previous WISP I worked at. I will thoroughly go through the documents you gave.

Yes I believe you can pull off a 12x multiplier on an AP. You may need to employ QoS to keep low latency on important services like VoIP and to keep things like videos with constant streams but I defiantly believe this number can work. Keep in mind that 12x is a target and a maximum for an AP. Try to build to that but not exceed... a 10x is a good place to be 14x not so much however it used to be as high as 20x but that was before the days of Nexflix.

The larger the pool of customers using a shared resource the higher your multiple can be. If I run a 17x on an AP I will get complaints... If I run it on my up line connection that is the aggregate of thousands of customers I don't exceed 80% utilization. Consider a really small pool with a flat multiplier to see how it scales... consider a pool of 1 customer. Could you sell to one person 12Mbps and only purchase 1Mbps... or sell to two customers 12Mbps each and purchase 2Mbps... of course not so the multiplier scales with the size of the pool.

The network that I am getting my 12x from has two feeds of 250Mbps with an average peek load of about 60% to 80% per pipe and about 3500 customers. The most popular bandwidth package is 1.5Mbps although they range up to 20Mbps so I would assume the average is something like 2.5Mbps... This would be roughly 8750Mbps sold... that number divided by the bandwidth purchased is about 17.5x. At a size of 3500 customers I am sure we can exceed 20x.

This is a good resources that shows an example of managing bandwidth with QoS to offer good consistent connectivity for things that can consume resources like streaming videos. Implementing things like this would help you reach your budgets without having pore performance: http://mum.mikrotik.com/presentations/US13/steve.pdf

Great! You have again put it perfectly.

Answer to Question 1: Simple network (I am reading "flat" layer 2 network): "Friends don't let Friends bridge networks".

I hate arguing because that's the way its done but one of the quickest and best answers I can think of is that none of the carriers use large flat Layer2 networks and when the offer Layer 2 connectivity its with MPLS. Your offering internet service and internet service is a web and is routed... Break the best practice model at your own risk (read 2am network emergencies and large support call volumes).

Layer 3 offers a lot of packet efficiency. For example you want to limit your broadcast traffic. If every time one client does a broadcast (like asking for an IP via DHCP) the broadcast would go from the station to the AP... then The AP would repeat it to everything on the bridge (All of the other Stations as well as the upline router). Then the upline router would repeat it to your other routers at the other tower sites and those routers would repeat it to the AP's and those AP's to the Stations... until finally everything in your network has heard that broadcast and ignored it except for your DHCP server which would answer through with unicast directly to the requester. Now this can be greatly reduced by using VLANS to separate each tower site into it's own Layer 2 network and you can also use client isolation on the AP and drop broadcast on the routers bridge. What your doing with this concept is basically creating tunnels between your AP and head end... at this point you should be thinking: why not just go all the way and create VPLS tunnels over MPLS between your end customer and core router (not a bad idea). Another big reason to have a routed network is that you cannot have Layer 2 loops (meaning redundancy) in your network without a headache of a Layer 2 routing protocol for loop avoidance or it will lock up your equipment. (You want routed loops with a nice link state routing protocol like OSPF because this avoids 2am network emergencies!)

The bottom line is that you don't want a large flat network. Route traffic to each site (OSPF), interconnect each site (create a web) with multiple paths for the traffic so that a power outage or failure of a single piece of equipment doesn't take half of your network down. Did I mention not having 2am emergencies... I was happy when those went away.

I think I got this. I will have to learn a lot more than I thought. I will have to hire a professional networking engineer sooner than I thought.

Answer to Question 2:
If you go with another vendor for equipment be sure to have some form of TDMA. The standard 802.11G is designed to operate where each client can "hear" the other clients. It waits to transmit until it hears an open time similar to a bunch of people in a room that can hear each other talking. If the clients begin talking over each other to the AP because they don't hear someone else talking then you will have collisions which is the AP distinguish between the clients... The solution is TDMA (Time Division Multiple Access) (read everyone having a scheduled time to talk). For successful outdoor wireless a scheduling mechanism is a must.

It is best to use both H & V on the same AP to reach the higher modulations. It would be possible to increase the number of AP's by creating a grid of H & V antennas... however offsetting an antenna by 90º only gives you 20db of separation. If your going to re-use a frequency on a tower you will need a lot of vertical separation as well or you will have to have the AP's in sync with one another (read Cambium with GPS sync). Without this the AP on the tower that is on the same frequency or on a near by frequency will be louder then any of the clients even if the antenna is pointed the opposite direction and turned 90º. ( TX: 20db - Polarization: 20db - FrontToBackRatio: 35db - very little Freesapce loss = could be as high as -40 depending on free space loss).

I recommend using MIMO 2x2 radios with NV2... set your channel width to 10MHz if your wanting to double the number of AP's on a tower... 3 non-overlapping channels becomes 6 non-overlaping channels... An array of 6 60º sector antennas is nice (get the best AP antennas you can). If you improve your RX & TX on one side by having a better antenna then your improving the entire link... With a better AP you improve every customer link with a better customer antenna you improve only that customer. Don't skimp on the AP. Also, increasing the passive gain of an antenna is much better then increasing the transmit power because increasing the antenna gain also increases the receive signal whereas the transmit power does not effect the Rx levels.


I think you still should give me a call. I highly recommend using a good consultant to help you build your network. It may cost a little extra money up front (there is a real chance it will save you money too) but I believe your return on investment will be very rapid. You don't just need someone with networking knowledge, you need someone that has worked with multiple networking models in the real world, that understands when something doesn't look right because they have fixed this kind of problem before. From the sounds of it you will learn all of this really quickly but there is a lot to learn and some of it is more art then science. Also you should consider systems integration early on. When working on support you need to get form installation notes to call history to usage graphs to billing etc fairly easily and quickly. The more information you have easily accessible the quicker and more accurately you'll be able to get a solution. Support costs are one of those hidden costs that can be real high or real low but that depends on how you build things. Furthermore you don't want to get to the point where you have to call in an expert and have them work on your network but they are lost on how everything is connected and where your issues are etc. Good design good planning and good record keeping will make your network need less maintenance and less time to repair issues when there is a need. One such example would be consistent and simple naming and addressing schemes. You don't want to talk to an consultant who is charging by the hour where you talk about the issue with Fred's tower and then mention Jan's tower and 10 minutes later you find out that the engineer thought those were two different towers and really they are just a married couple... Believe me bad habits die hard so be thoughtful from the beginning about scaling issues, keep good notes and records, keep things clean.

This makes sense. I will PM you.

Thank you for being such a sport.

............................................
Yes I believe you can pull off a 12x multiplier on an AP. You may need to employ QoS to keep low latency on important services like VoIP and to keep things like videos with constant streams but I defiantly believe this number can work. Keep in mind that 12x is a target and a maximum for an AP. Try to build to that but not exceed... a 10x is a good place to be 14x not so much however it used to be as high as 20x but that was before the days of Nexflix.

The larger the pool of customers using a shared resource the higher your multiple can be. If I run a 17x on an AP I will get complaints... If I run it on my up line connection that is the aggregate of thousands of customers I don't exceed 80% utilization. Consider a really small pool with a flat multiplier to see how it scales... consider a pool of 1 customer. Could you sell to one person 12Mbps and only purchase 1Mbps... or sell to two customers 12Mbps each and purchase 2Mbps... of course not so the multiplier scales with the size of the pool.

The network that I am getting my 12x from has two feeds of 250Mbps with an average peek load of about 60% to 80% per pipe and about 3500 customers. The most popular bandwidth package is 1.5Mbps although they range up to 20Mbps so I would assume the average is something like 2.5Mbps... This would be roughly 8750Mbps sold... that number divided by the bandwidth purchased is about 17.5x. At a size of 3500 customers I am sure we can exceed 20x.

This is a good resources that shows an example of managing bandwidth with QoS to offer good consistent connectivity for things that can consume resources like streaming videos. Implementing things like this would help you reach your budgets without having pore performance: http://mum.mikrotik.com/presentations/US13/steve.pdf
.............................................................

Excellent advice and recommendations but I don't think that 2.5Mbps will be enough bandwidth for each CPE, as not long ago each domestic household had just one desktop and maybe a laptop and now no desktop but several laptops and several smart phones and even more mobile devices and guess what they are all attempting to stream in HD, this just divides the incoming bandwidth too much!

Maybe a minimum of 5-10Mbps for each household which totally changes the design of the network,
a large number of households now have X10 bandwidth consumption and this figure is growing, has any WISP been able to capitalize financially without losing customers with the current trend!

..................used to be as high as 20x but that was before the days of Nexflix....
Looks like we will be putting up more and more AP's with bigger and bigger backhaul to cater for bandwidth requirement but can we charge more!

Attached is a listing of a tower showing the bandwidth speeds provisioned and a three day usage graph covering these customers... Make your own judgements about bandwidth overselling. As you can see 131Mbps has been provisioned on this tower... if the 12x multiplier is correct then we would need to budget about 11Mbps to fulfill this. In fact the graphs show even less then 11Mbps of peek usage. Also to be clear, this tower is able to receive over 100Mbps of internet the customers are just not demanding more. I can show you these same trends just about anywhere in our 3500 customer base. The more customers you average speeds over the more you can budget... its like filling a cup with rocks and sand. Most internet traffic is busty (even streamed videos) so one person pulls a lot then the the other and it all works out to about 1/12 according to the statistics I have run.

Also note that the graphs are averaged usage at check-in intervals. I am sure that the interface has exceeded 20Mbps for short bursts but the sustained throughput is closer to a 16x multiplier.

The graphs say it all, 29 clients of which 10 clients are on 1500K, the question I would ask is how long before they have to be moved to higher bandwidth package for no extra turnover?

I have seen AP bandwidth increase just because customers demand a minimum speed at peak usage hours and then as this speed is available more bandwidth is used, catch21

The graphs say it all, 29 clients of which 10 clients are on 1500K, the question I would ask is how long before they have to be moved to higher bandwidth package for no extra turnover?

I have seen AP bandwidth increase just because customers demand a minimum speed at peak usage hours and then as this speed is available more bandwidth is used, catch21

I don't want to be wrong and I know I am very capable of being wrong (happens on a regular basis). I would like to see data that disagrees with what I am showing you. I have done many detailed studies of traffic like this over many years covering thousands of connections and I have found the 12x budget to be reasonable. I have run statistics comparing various vendors equipment various noise levels etc. With residential internet access I find that you don't hit your head on the ceiling until about 12x once your scaled up to production sizes. If your serving hotels, coffee shops, office building, etc. then throw the budget idea out the window (and charge accordingly).

A little more info regarding the data I pictured, this is a new tower (first customer was in either Nov or Dec of last year). This is in an area that had DSL that was horribly overloaded. These are all customers that wanted higher speeds although some were cheep enough to want the minimums that were similar to how their DSL was before it began slowing down. To answer your question, I would guess that the majority of these customers will keep their current package for least a couple of years - and the bandwidth is ready as soon as their pocket books make a few more dollars available per month. This tower is currently able to serve over 100Mbps of real bandwidth (3 x Cambium PMP 450's, 6 x Cambium PMP100's) using 2.4GHz & 900MHz bands. The backhaul has not been built to that capacity yet. A licensed link will be added to the tower once the MikroTik BH is no longer keeping I never run MT BH's over 100Mbps if for no other reason then the latency. The individual subscriber modules (Cambium PMP 450) will easily provide 20Mbps connections (I was a beta tester for the 450's and pushed the AP up to 96Mbps with a couple of subscribers). Each AP is easily capable real a real world 50Mbps depending on conditions, number of clients and PPS of the connections. The Cambium is pretty awesome in ability but its at least 4 times more expensive when compared with MT.

A budget of 12x max works. This does not mean that you can always run at the full 12x often your at 10x or 8x as you build however when you reach 12x you need to split something. Furthermore some areas are different because of the type of people that live there.

If your not able to achieve budgets near 12x then you should defiantly look into QoS to make your network run smoother under higher loads and limit high speed connections that don't really benefit from the higher speeds. For example you can watch a Netflix video without buffering in HD with a 4Mbps stream of data... if you give it more then it will burst up to the higher speed and then drop way down. Graphing this type of data looks like high peeks and low valleys. There is no problem with this however if the situation changes and the connection wants to burst and cannot then you may end up with buffering (read unhappy customers). If you provide a constant stream of 4Mbps for this connection then the user experience is great. With limiting the needless bursts on a connection like this you can watch two Netflix streams on a 10Mbps connection and have a hand full of people playing online games on the same connection. When you scale this up to an entire AP or entire ISP then you can get better bandwidth budgets and provide better user experience to customers. Customers don't really want higher burst speeds (even if they think they do) they really want fast internet that always runs smoothly.

If you have a large network you may want to look into a traffic sharper (really expensive) or keep a good set of queues up to date on your MikroTiks (time consuming).

MIMO will get you nearly twice the throughput per channel. At your speeds, you get may more customers connected by using SISO on different polarities. Your situation is very different from mine.

If I use a dual chain MIMO AP. Then simply disable the chain0 on half CPEs and chain1 on other half CPEs, will I be able to reach 80-100 CPEs per AP? CPEs will be SXT lite5. AP will be RB912.

Even if I can do this, the numbers of client come to be about 360.

I am having trouble thinking of a sector/AP/CPE configuration that will allow more than 250 clients per tower. I need >1000 clients on one 5GHz tower. About 3-4 km radius. 5825-5875Mhz. 3 perfectly(?) separated 10Mhz channels.

Everything thing is fine if those 250 clients purchases 10mbps, but they won't. I will have majority subscribers in 1mbps. Some on 2mbps or 4mbps. 10 Mbps would be the premium service.

I am also looking at installing one CPE in a neighborhood and connecting 10+ people with a switch(ports isolated, traffic PPPoE). But this kills the flexibility of WISP deployment.

Using 3 10Mhz channels separated by 10Mhz, can I deploy a perfectly stable and "interference free" site? I expect about 120mbps throughput total. So I can sell about 400-600 Mbps with a decent uplink. BUT this is not possible if each client only buys 1mbps. There will be 200 CPEs per AP.

Another possible config is 6 sectors in ABCABC mode. SISO CPEs with MIMO APs. Comes to about 500 simultaneously active clients. Are there any disadvantages to this expect the high initial cost? How would one manage the collocation interference?

Edit:

According to the attached pdf, 20MHz 802.11 channels need to spaced apart by at least 20Mhz. I assumed that it would be 10MHz separation of 10Mhz channels. Is this true or you always need 20Mhz channel separation for that -40db separation?

If I use a dual chain MIMO AP. Then simply disable the chain0 on half CPEs and chain1 on other half CPEs, will I be able to reach 80-100 CPEs per AP? CPEs will be SXT lite5. AP will be RB912.

The AP only talks to one client at a time so disabling a chain will only slow down the communications which will have the side effect of reducing the number of client you can support per AP.

It is much much better to use more directional antennas on your AP and increase the number of AP's. Use 3 AP's for 100 people, one third on each AP or even better install a 6 sector array of 60º antennas. If you use multiple bands or large enough bands like 2.4GHz & 5.8GHz then you can get 12 AP's or more on a tower.

If you really want to put 100 stations on an AP it is possible but you your IP jitter will be really high (varied latency) and your customers will not have consistent service.

Even if I can do this, the numbers of client come to be about 360.

You should be able to get 360 customers with no more then 30 clients per AP (assuming perfect spread across the AP's) by using 2.4GHz and 5.8GHz. After that you can use the light licensed 3.65GHz. You can also use the license free 900 as well as 5.4GHz. (This info is assuming your in FCC land I don't know the other country rules).

I am having trouble thinking of a sector/AP/CPE configuration that will allow more than 250 clients per tower. I need >1000 clients on one 5GHz tower. About 3-4 km radius. 5825-5875Mhz. 3 perfectly(?) separated 10Mhz channels.

30MHz of channel width isn't really enough for 1000 customers anyway. You can transmit about 50Mbps per 10MHz channel and that is assuming perfect conditions. With an AP with clients on it you would be lucky to get 25Mbps out of the AP... 25MB devided by 333 customers is dialup modem speed... even with a 12 multiplier (one twelfth of the customers pulling full bandwith at a time) your still only looking at 0.72Mbps per customer. You will have to use more AP's and more then 30MHz of channel width.

I am also looking at installing one CPE in a neighborhood and connecting 10+ people with a switch(ports isolated, traffic PPPoE). But this kills the flexibility of WISP deployment.

You can have micropops. Use a OmniTik and a station. You can power them with one power injector. The station connect to your main tower and to the OmniTik. The customer at that location uses an ethernet connection and has PPPoE terminated at their home router. You could bridge the OmniTik & station so that other customers connecting to the OmniTik can also use PPPoE termination from the main tower.

Using 3 10Mhz channels separated by 10Mhz, can I deploy a perfectly stable and "interference free" site? I expect about 120mbps throughput total. So I can sell about 400-600 Mbps with a decent uplink. BUT this is not possible if each client only buys 1mbps. There will be 200 CPEs per AP.

I would estimate 25Mbps per 10MHz of real throughput (which may be high) then you can oversell that by 12x 25*3*12=900. If you sold all 10Mbps connections then that would be 90 customers on the tower, 30 on each AP. These budget numbers are close to reasonable but you don't want to expect filling these numbers perfectly, these are ceiling numbers that you need to not surpass.

Only frequency freely available here is 2400-2483.5 and 5825-5875. That is it. Licensing is very complex and not in reach of small players.

I will play around with different topologies. This is gonna be good.

How about using the H and V polarities for different APs? If the frequency is different, there should not be a problem. Some separation will be provided via the channels space and some via the H-V spacing.

I am having trouble thinking of a sector/AP/CPE configuration that will allow more than 250 clients per tower. I need >1000 clients on one 5GHz tower. About 3-4 km radius. 5825-5875Mhz. 3 perfectly(?) separated 10Mhz channels.

30MHz of channel width isn't really enough for 1000 customers anyway. You can transmit about 50Mbps per 10MHz channel and that is assuming perfect conditions. With an AP with clients on it you would be lucky to get 25Mbps out of the AP... 25MB devided by 333 customers is dialup modem speed... even with a 12 multiplier (one twelfth of the customers pulling full bandwith at a time) your still only looking at 0.72Mbps per customer. You will have to use more AP's and more then 30MHz of channel width.

I have 30MHz on 5.8Ghz + 40Mhz on 2.4Ghz. That should help.

What do you think about the pdf thing I said in the last post?

What are the EIRP limits for pmp links on each frequency band?

4 watts of EIRP.

There is a section in the pdf showing existing unlicensed frequencies.