Scenario Details

We want to pass our 16 Gbps traffic over 20 Gbps channel where every user will get 3 Mbps or 2 Mbps with using PCQ.

Scenario Details Breakup

40,0000 Queues one for each user with using Queue-type PCQ
16 Gbps Traffic Over 20 Gbps LACP Channel
3 Mbps or 2 Mbps for each user with using Queue-type PCQ

QUEUE TREE Issue Details

when we add rule in mangle to mark packet 10.x.x.x/22 and enable queue tree with that Marked traffic CCR interface max throughput becomes 600 Mbps up-to 700 Mbps Max even we put more load on it and add one more 20.x.x.x/22 it never increased. We had also tried to isolate marked traffic in 10.x.x.x/24 per Queue Tree but again when 4 Queue tree are created along with /24 subnetmask CCR total throughput 600 Mbps up-to 700 Mbps Max. But if i disable my all QUEUES-TREE my CCR got 2 Gbps up-to 2.2 Gbps traffic.

QUEUE SIMPLE Issue Details

We create mangle rules to mark traffic coming from 10.x.x.x/23 subnetmask and create a simple queue for each 10.x.x.x/23 traffic after this configuration we got max 1.5 Gbps traffic throughput and observe that every Simple Queue Average Max bandwidth throughput was 250 Mbps up-to 270 Mbps.But if i disable my all QUEUES my CCR got 2 Gbps up-to 2.2 Gbps traffic.

PS: In both mention scenarios my CPU utilization is 0% or just hit 8% utilization max.

IP-Connection Tracking=Disabled in Firewall

CCR is notoriously slow when it comes to processing firewall rules.

We saw the exact same on our CCR1036 when we started using mangle rules (also to mark packets). It can handle thousands of queues yes, but marking the packets to go into the queues, is really, really troublesome.

Your CCR will literally fall over with 16Gbps of traffic, and 40K Firewall rules (presumably one for each queue). Our CCR crashed at a few K firewall rules (mangle) and about 3Gbps of traffic.

Queue trees are also not recommended on the CCRs. MT recommends simple-queues. Kind of ironic, as you still need to SOMEHOW put traffic INTO the queue. MT essentially is assuming that everyone uses PPP and simple-queues attached to the interface, unfortunately, in reality, it doesn't always work that way, and the CCR is very slow in scenarios where each simple queue is not attached directly to an interface.

its the weakness of the mesh architecture between lots of cores. CPU usage isnt maxed out meaning that other cores are waiting for one core to finish its work. This is why mikrotik suggests using simple queues as they havent got multi threading down yet.

Dear Members,

Thanks for sharing technical experience , But some how we had tried to carter this with some tweaking which is still under observation.
When we digging this issue in more depth we have found that its due to Packets Drop in PCQ. We have made new Queue for physical interface with 120 Million packet throughput along with using PFIFO and MQ-PFIFO and tried both to set on physcial interface one by one but no luck....

Rather than using a physical interface, create a virtual one and tie the physical interface to it instead.

I think the best way to figure out which works better is to use the packet generator on a physical interface and virtual interface and see which one can use more CPU.