Page 1 of 1

Bandwidth not exceeding 75Mbit/s through Queue Tree & PCQ

Posted: Thu Sep 24, 2015 11:51 pm
by troy
A quick network diagram:

Queue tree (below) is on CCR2.

We did some testing both inside and outside the queue tree. Inside the queue tree, tests to CCR3 max out at 140Mbit/s, exactly where it should be. Outside the queue tree, bandwidth maxes out at about 160Mbit/s. The queue tree is set to 140Mbit/s to prevent the wireless link from getting saturated.

The problem, is that in practice, our bandwidth is capping out at about 75Mbit/s and our customers are complaining about it. People who should be getting 3Mbit/s are only getting 500-600k. Latency should be 25-30ms from the CPE to the edge, but during peak hours, it might be as high as 200ms.

I'm not sure what to look at with regards to figuring out why we're only able to get 75Mbit/s of customer traffic when we can get double that during testing.

Here's the queue tree. It's nothing special.
/queue tree
add limit-at=100M max-limit=140M name=total_down parent=global queue=default
add name=Bronze_Download packet-mark=Bronze_Download parent=total_down queue=pcq_bronze_down
add name=Gold_Download packet-mark=Gold_Download parent=total_down queue=pcq_gold_down
add name=Plat_Download packet-mark=Plat_Download parent=total_down queue=pcq_plat_down
add name=Silver_Download packet-mark=Silver_Download parent=total_down queue=pcq_silver_down
add limit-at=20M max-limit=30M name=total_up parent=global queue=default
add name=Bronze_Upload packet-mark=Bronze_Upload parent=total_up queue=pcq_bronze_up
add name=Gold_Upload packet-mark=Gold_Upload parent=total_up queue=pcq_gold_up
add name=Silver_Upload packet-mark=Silver_Upload parent=total_up queue=pcq_silver_up
add name=Plat_Upload packet-mark=Plat_Upload parent=total_up queue=pcq_plat_up

/queue type
add kind=pcq name=pcq_bronze_down pcq-burst-rate=1536k pcq-burst-threshold=512k pcq-burst-time=3m pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=1024k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_bronze_up pcq-burst-rate=256k pcq-burst-threshold=128k pcq-burst-time=3m pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=128k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_gold_down pcq-burst-rate=6144k pcq-burst-threshold=2048k pcq-burst-time=2m pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=4096k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_gold_up pcq-burst-rate=512k pcq-burst-threshold=256k pcq-burst-time=2m pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=384k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_plat_down pcq-burst-rate=12M pcq-burst-threshold=5M pcq-burst-time=3m pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=7M pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_plat_up pcq-burst-rate=768k pcq-burst-threshold=384k pcq-burst-time=3m pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=512k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_silver_down pcq-burst-rate=3072k pcq-burst-threshold=768k pcq-burst-time=4m pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=1536k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_silver_up pcq-burst-rate=512k pcq-burst-threshold=320k pcq-burst-time=3m pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=192k pcq-src-address6-mask=64 pcq-total-limit=30000

Re: Bandwidth not exceeding 75Mbit/s through Queue Tree & PCQ

Posted: Fri Sep 25, 2015 10:34 am
by bajodel
Maybe give some insight of the wireless link also.
In your speed tests I suppose you tested 140/160 mbit half duplex, but your real traffic is full duplex.

Re: Bandwidth not exceeding 75Mbit/s through Queue Tree & PCQ

Posted: Fri Sep 25, 2015 5:42 pm
by troy
Not sure where you're headed with that, but the wireless link is half duplex, just like what most people here are using. It's from another vendor, but that's beside the point.

It's been a long time since testing an unloaded link, but I want to say it was pushing right at 180 in each direction tested separately, and down to about 70Mbit/s when tested at a 1:1 ratio. When testing at a real world ratio of about 10:1, we were looking at something right around 160.

Last night, I broke up the queue tree a bit, and we passed just over 90Mbit/s. This is telling me more that the problem is with the Queue Tree on the CCR. The problem doing this, is that we no longer have a global queue to limit all traffic to 140Mbit/s.

So, why am I having this problem on the CCR? At another site, we have a queue tree on a RB1100 and it's passing nearly 200mbit/s without issue.

Re: Bandwidth not exceeding 75Mbit/s through Queue Tree & PCQ

Posted: Tue Nov 10, 2015 10:14 am
by RK
It looks like your queue tree has some incorrect/missing configurations.

Where is your pcq-limit setting?
Did you try using double or triple the default pcq-limit?

Re: Bandwidth not exceeding 75Mbit/s through Queue Tree & PCQ

Posted: Tue Nov 10, 2015 2:31 pm
by chechito
try this

in pcq queues a bigger 'limit' value and 'total limit' need to be the 'limit' value x number of users at least