2600 queue's is nuts, work smarter not harder.
PCQ where you can, split the loading among many box's if the load is still too high
After I swapped the SFQ at PCQ performance of the hardware did not increase.
I have two servers connected at the same time, and I can flip freely move between them. One is a 2x SuperMicro Xeon 3GHz, the other is a Pentium D 3GHz MSI.
In the case of the first large packet loss begins at 70% CPU. Second calmly reaches 100% CPU and still works without noticeable problems.
CPU load looks very similar, both using SFQ and PCQ.
When configured with the SFQ had about 2600 queues. After moving to the PCQ queues have 20, 10 for download and upload, and a little more than 3,000 addresses in address lists (more due to the fact that some customers have 2 IP addresses - for SFQ queues I marked both the same packet mark).
Maybe am I doing something wrong?
So my setup looks like:
0 chain=postrouting action=mark-connection new-connection-mark=30Dconn passthrough=yes dst-address-list=30D
1 chain=postrouting action=mark-packet new-packet-mark=30D passthrough=no connection-mark=30Dconn
2 chain=postrouting action=mark-connection new-connection-mark=5Dconn passthrough=yes dst-address-list=5D
3 chain=postrouting action=mark-packet new-packet-mark=5D passthrough=no connection-mark=5Dconn
4 chain=postrouting action=mark-connection new-connection-mark=10Dconn passthrough=yes dst-address-list=10D
5 chain=postrouting action=mark-packet new-packet-mark=10D passthrough=no connection-mark=10Dconn
6 chain=postrouting action=mark-connection new-connection-mark=1Dconn passthrough=yes dst-address-list=1D
7 chain=postrouting action=mark-packet new-packet-mark=1D passthrough=no connection-mark=1Dconn
8 chain=postrouting action=mark-connection new-connection-mark=50Dconn passthrough=yes dst-address-list=50D
9 chain=postrouting action=mark-packet new-packet-mark=50D passthrough=no connection-mark=50Dconn
10 chain=postrouting action=mark-connection new-connection-mark=2Dconn passthrough=yes dst-address-list=2D
11 chain=postrouting action=mark-packet new-packet-mark=2D passthrough=no connection-mark=2Dconn
12 chain=postrouting action=mark-connection new-connection-mark=20Dconn passthrough=yes dst-address-list=20D
13 chain=postrouting action=mark-packet new-packet-mark=20D passthrough=no connection-mark=20Dconn
14 chain=postrouting action=mark-connection new-connection-mark=100Dconn passthrough=yes dst-address-list=100D
15 chain=postrouting action=mark-packet new-packet-mark=100D passthrough=no connection-mark=100Dconn
16 chain=postrouting action=mark-connection new-connection-mark=40Dconn passthrough=yes dst-address-list=40D
17 chain=postrouting action=mark-packet new-packet-mark=40D passthrough=no connection-mark=40Dconn
18 chain=postrouting action=mark-connection new-connection-mark=05Dconn passthrough=yes dst-address-list=05D
19 chain=postrouting action=mark-packet new-packet-mark=05D passthrough=no connection-mark=05Dconn
20 chain=prerouting action=mark-connection new-connection-mark=30Uconn passthrough=yes src-address-list=30U
21 chain=prerouting action=mark-packet new-packet-mark=30U passthrough=no connection-mark=30Uconn
22 chain=prerouting action=mark-connection new-connection-mark=5Uconn passthrough=yes src-address-list=5U
23 chain=prerouting action=mark-packet new-packet-mark=5U passthrough=no connection-mark=5Uconn
24 chain=prerouting action=mark-connection new-connection-mark=10Uconn passthrough=yes src-address-list=10U
25 chain=prerouting action=mark-packet new-packet-mark=10U passthrough=no connection-mark=10Uconn
26 chain=prerouting action=mark-connection new-connection-mark=20Uconn passthrough=yes src-address-list=20U
27 chain=prerouting action=mark-packet new-packet-mark=20U passthrough=no connection-mark=20Uconn
28 chain=prerouting action=mark-connection new-connection-mark=50Uconn passthrough=yes src-address-list=50U
29 chain=prerouting action=mark-packet new-packet-mark=50U passthrough=no connection-mark=50Uconn
30 chain=prerouting action=mark-connection new-connection-mark=2Uconn passthrough=yes src-address-list=2U
31 chain=prerouting action=mark-packet new-packet-mark=2U passthrough=no connection-mark=2Uconn
32 chain=prerouting action=mark-connection new-connection-mark=1Uconn passthrough=yes src-address-list=1U
33 chain=prerouting action=mark-packet new-packet-mark=1U passthrough=no connection-mark=1Uconn
34 chain=prerouting action=mark-connection new-connection-mark=100Uconn passthrough=yes src-address-list=100U
35 chain=prerouting action=mark-packet new-packet-mark=100U passthrough=no connection-mark=100Uconn
36 chain=prerouting action=mark-connection new-connection-mark=40Uconn passthrough=yes src-address-list=40U
37 chain=prerouting action=mark-packet new-packet-mark=40U passthrough=no connection-mark=40Uconn
38 chain=prerouting action=mark-connection new-connection-mark=05Uconn passthrough=yes src-address-list=05U
39 chain=prerouting action=mark-packet new-packet-mark=05U passthrough=no connection-mark=05Uconn
0 name="default" kind=pfifo pfifo-limit=50
1 name="ethernet-default" kind=pfifo pfifo-limit=50
2 name="wireless-default" kind=sfq sfq-perturb=5 sfq-allot=1514
3 name="synchronous-default" kind=red red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20 red-avg-packet=1000
4 name="hotspot-default" kind=sfq sfq-perturb=5 sfq-allot=1514
5 name="sfq" kind=sfq sfq-perturb=5 sfq-allot=1514
6 name="05D" kind=pcq pcq-rate=528000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=20000
7 name="1D" kind=pcq pcq-rate=1040000 pcq-limit=40 pcq-classifier=dst-address pcq-total-limit=20000
8 name="2D" kind=pcq pcq-rate=2064000 pcq-limit=30 pcq-classifier=dst-address pcq-total-limit=20000
9 name="5D" kind=pcq pcq-rate=5136000 pcq-limit=30 pcq-classifier=dst-address pcq-total-limit=40000
10 name="10D" kind=pcq pcq-rate=10000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=20000
11 name="20D" kind=pcq pcq-rate=20000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=20000
12 name="30D" kind=pcq pcq-rate=30000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=20000
13 name="40D" kind=pcq pcq-rate=40000000 pcq-limit=10 pcq-classifier=dst-address pcq-total-limit=20000
14 name="50D" kind=pcq pcq-rate=50000000 pcq-limit=10 pcq-classifier=dst-address pcq-total-limit=20000
15 name="100D" kind=pcq pcq-rate=100000000 pcq-limit=10 pcq-classifier=dst-address pcq-total-limit=20000
16 name="05U" kind=pcq pcq-rate=528000 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=20000
17 name="1U" kind=pcq pcq-rate=1040000 pcq-limit=40 pcq-classifier=src-address pcq-total-limit=20000
18 name="2U" kind=pcq pcq-rate=2064000 pcq-limit=30 pcq-classifier=src-address pcq-total-limit=20000
19 name="5U" kind=pcq pcq-rate=5136000 pcq-limit=30 pcq-classifier=src-address pcq-total-limit=40000
20 name="10U" kind=pcq pcq-rate=10000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=20000
21 name="20U" kind=pcq pcq-rate=20000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=20000
22 name="30U" kind=pcq pcq-rate=30000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=20000
23 name="40U" kind=pcq pcq-rate=40000000 pcq-limit=10 pcq-classifier=src-address pcq-total-limit=20000
24 name="50U" kind=pcq pcq-rate=50000000 pcq-limit=10 pcq-classifier=src-address pcq-total-limit=20000
25 name="100U" kind=pcq pcq-rate=100000000 pcq-limit=10 pcq-classifier=src-address pcq-total-limit=20000
26 name="default-small" kind=pfifo pfifo-limit=10
0 name="Download" parent=LAN limit-at=0 priority=8 max-limit=500M burst-limit=0 burst-threshold=0 burst-time=0s
1 name="Upload" parent=WAN limit-at=0 priority=8 max-limit=500M burst-limit=0 burst-threshold=0 burst-time=0s
2 name="Download 528kbps" parent=Download packet-mark=05D limit-at=0 queue=05D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
3 name="Download 1Mbps" parent=Download packet-mark=1D limit-at=0 queue=1D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
4 name="Download 2Mbps" parent=Download packet-mark=2D limit-at=0 queue=2D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
5 name="Download 5Mbps" parent=Download packet-mark=5D limit-at=0 queue=5D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
6 name="Download 10Mbps" parent=Download packet-mark=10D limit-at=0 queue=10D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
7 name="Download 20Mbps" parent=Download packet-mark=20D limit-at=0 queue=20D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
8 name="Download 30Mbps" parent=Download packet-mark=30D limit-at=0 queue=30D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
9 name="Download 40Mbps" parent=Download packet-mark=40D limit-at=0 queue=40D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
10 name="Download 50Mbps" parent=Download packet-mark=50D limit-at=0 queue=50D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
11 name="Download 100Mbps" parent=Download packet-mark=100D limit-at=0 queue=100D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
12 name="Upload dla 528kbps" parent=Upload packet-mark=05U limit-at=0 queue=05U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
13 name="Upload 1Mbps" parent=Upload packet-mark=1U limit-at=0 queue=1U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
14 name="Upload 2Mbps" parent=Upload packet-mark=2U limit-at=0 queue=2U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
15 name="Upload 5Mbps" parent=Upload packet-mark=5U limit-at=0 queue=5U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
16 name="Upload 10Mbps" parent=Upload packet-mark=10U limit-at=0 queue=10U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
17 name="Upload 20Mbps" parent=Upload packet-mark=20U limit-at=0 queue=20U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
18 name="Upload 30Mbps" parent=Upload packet-mark=30U limit-at=0 queue=30U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
19 name="Upload 40Mbps" parent=Upload packet-mark=40U limit-at=0 queue=40U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
20 name="Upload 50Mbps" parent=Upload packet-mark=50U limit-at=0 queue=50U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
21 name="Upload 100Mbps" parent=Upload packet-mark=100U limit-at=0 queue=100U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
Is this correct?