We have a common WISP architecture: large router at the gateway to our service, branching off to multiple point-to-point links to towers serving individual communities.

Until now, we have been performing QOS and subscriber speed-limitation queueing in each individual tower. The downside is (of course) that every time we make a change or an improvement in the queueing rules, we have to duplicate the change in every tower.

Is there any good reason not to implement a single copy of the queueing rules in the gateway router and impose the per-subscriber speed limitations there? Do we lose significant capabilities in any tracing or tracking tools?

We do set speed limits in customer CPEs when they provide this capability, but some of the CPEs we use don't have a provision for this. We don't have any direct subscriber-to-subscriber traffic within our network, and if any crops up in the future that causes problems, we'll just throw a rule or two in each tower (which wouldn't change often if at all) to take care of that.

The main tradeoff is that you transport a whole bunch of packets through your network that you then throw away while they're still within your network. That's a waste of resources. Ideally - and this works well if your network is routed - you should shape and firewall traffic to the client at the Internet border so you don't haul traffic to the client you then throw out later at the customer edge, and shape and firewall traffic from the client at the CPE/tower so you don't haul traffic from the client you then throw out later at the Internet border.

I'd rather look into automation tools such as rancid so you can automate configuration roll outs and changes. That also gives you many other benefits, such as improved change management and configuration auditing.

That makes sense.

I realize I might start out each connection transporting and throwing away a bunch of traffic (especially torrents), but I understood that due to the underlying mechanisms of IP, the rate would self-adjust from outside my network as packets were seen to go missing at any faster rate. Is this a fair analysis?

Not quite. TCP works that way (or rather is supposed to work that way, and usually does with well implemented stacks), but IP does not. UDP specifically does not. Torrents usually run over UDP. They'll keep pumping away.

Quality of Service is about intelligently discarding packets when needed, so you want to apply QoS wherever you are discarding packets. If you are oversubscribing a backhaul, then you want QoS on the tower to make the most of the the backhaul. If the limitation is always at your internet connection, then just limiting there is fine.

I have also seen a few setups where service-based QoS is performed on the towers, then PCQ is performed at the core. Since the L7 filters for matching services can be CPU intensive, this distributes the load across all the towers (since each only has to process its local traffic), while PCQ happens at the egress point where the actual bandwidth constraint is.

What I found works best for bandwidth control queueing is to use two bandwidth controls at that same time. One bandwidth control in the customer CPE radio - and on the upstream side (my NOC) I use PfSense for bandwidth control - also. My PfSense uses radius to get the up/down speed for every connection.

Now here is what works really well - you set the transmitting side of the QOS bandwidth control to a slightly slower speed than the receiving side QOS on the other side of the wireless link. The indent is to bandwidth limit transmit data in both directions before you hit the wireless link.

Performing bandwidth control on a wireless link works - but performing bandwidth control prior to hitting the wireless link (in both directions) actually works better.

This way you are bandwidth rate-limiting traffic in both directions across a microwave link. This results in fewer wireless collisions and fewer wireless retries - which results in greater bandwidth for everybody else.

Tom Jones
A WISP in North Idaho