Two problems with shaper in 5.0RC4

jankad · Tue Nov 23, 2010 3:12 am

I noticed two problems:

1. List queue tree in Winbox is not always complete. In the lower right corner of the window, for example, "800 out of 2,672 items," even though it is not included in any display filter.
It helps to close the Winbox several times and log back in, sometimes the complete list of shows up until the third or fourth time

2. Does not always work in the CLI to add queues - often pops timeout

The same problems were in the RC3.

I have about 2600 queues and up to 300Mbps of traffic.
Shaper is working on MSI server with 3.0GHz PIV D processor but I tested it on SuperMicro with 2x Xeon 3GHz.
Multi-cpu is set to yes.

3.30 works fine but CPU utilization is up to 50% on MSI and 25% in SuperMicro for even 2-3 hour a day and during this time there are a small packet losses. It seems that this is the end capabilities of the hardware, hence the hope of SMP works fine in 5.0.

Chupaka · Tue Nov 23, 2010 2:29 pm

2600 queues is madness. look at PCQ

Chupaka · Tue Nov 23, 2010 2:30 pm

2. Does not always work in the CLI to add queues - often pops timeout

after timeout, make supout.rif and send to support@

jankad · Tue Nov 23, 2010 2:58 pm

2600 queues is madness. look at PCQ

Why madness?
I have 2600 queues on single shaper, two (download / upload) per customer.
Since when did mangle hashing with jumps to subchains it works without problems.

PCQ is not especially suited because I have dozens of different fares - from 512kbps through 1, 2, 5, 10Mbps to 30, 50 and 100Mbps.
Part is symmetrical, rest has a smaller part of the upload. In addition, manage marks and the queues by using the API directly from the billing system.

Tue Nov 23, 2010 3:10 pm

Of course, you can choose. Spend thousands of dollars on high tech hardware that can support such configuration, or offer PCQ with less choices for download speeds. Anyway, you are not getting what you think you are getting, the config with 2600 queues is very inefficient. Watch this: http://www.tiktube.com/index.php?video=247

jankad · Tue Nov 23, 2010 3:19 pm

Does this mean that MT can not handle this configuration?
I was hoping that 5.0 with Intel Core 2 Quad and Intel NIC with Receive Side Scaling can handle twice the number of queues and traffic at 900Mbps

Such hardware costs no more than $1000.

Tue Nov 23, 2010 3:51 pm

you do not understand how the queuing mechanism works, that's what I'm saying. your configuration is wrong. Watch the video, Janis explains it well.

martini · Tue Nov 23, 2010 8:42 pm

normis - you dont understand, question not about 2600 queue (i have shaper with 5120 queue), question about problem with showing a lot of stuff in winbox...
Have the same problem with interfaces or dhcp-leases, winbox not show all items, but in bottom i can see - "173 of 962 items" - it show only 173 item from 962 !!! If i close winbox and open again it might show all items or show 300 or 400 items )) and again if i reopen winbox it show all item.

jankad · Wed Nov 24, 2010 4:16 am

you do not understand how the queuing mechanism works, that's what I'm saying. your configuration is wrong. Watch the video, Janis explains it well.

Reconfigured the setup of my shaper for PCQ queues.
Now I have 20 PCQ queues (10 download and upload) and 20 of the address lists, in which added the IP addresses of customers according to their tariffs.
I'm curious how this configuration will behave in traffic peak.

Wed Nov 24, 2010 7:56 am

normis - you dont understand, question not about 2600 queue (i have shaper with 5120 queue), question about problem with showing a lot of stuff in winbox...
Have the same problem with interfaces or dhcp-leases, winbox not show all items, but in bottom i can see - "173 of 962 items" - it show only 173 item from 962 !!! If i close winbox and open again it might show all items or show 300 or 400 items )) and again if i reopen winbox it show all item.

Yes, I know that. But still, with 2600 queues, the last rules will not function properly.

Beccara · Wed Nov 24, 2010 9:44 am

2600 queue's is nuts, work smarter not harder.

PCQ where you can, split the loading among many box's if the load is still too high

mangust · Wed Nov 24, 2010 10:18 am

I have a similar problem.
Very often Tools/Graphing/Interface Graph not displayed Interface (for which statistics are collected).
So , I need to to watch it thru www.
This happens for all versions - 3.x, 4.x, 5.x ...
Maybe I just dnot know right way to "refrash" winbox window ...

jankad · Thu Nov 25, 2010 12:24 am

2600 queue's is nuts, work smarter not harder.

PCQ where you can, split the loading among many box's if the load is still too high

After I swapped the SFQ at PCQ performance of the hardware did not increase.

I have two servers connected at the same time, and I can flip freely move between them. One is a 2x SuperMicro Xeon 3GHz, the other is a Pentium D 3GHz MSI.
In the case of the first large packet loss begins at 70% CPU. Second calmly reaches 100% CPU and still works without noticeable problems.

CPU load looks very similar, both using SFQ and PCQ.

When configured with the SFQ had about 2600 queues. After moving to the PCQ queues have 20, 10 for download and upload, and a little more than 3,000 addresses in address lists (more due to the fact that some customers have 2 IP addresses - for SFQ queues I marked both the same packet mark).

Maybe am I doing something wrong?
So my setup looks like:

0 chain=postrouting action=mark-connection new-connection-mark=30Dconn passthrough=yes dst-address-list=30D 
1 chain=postrouting action=mark-packet new-packet-mark=30D passthrough=no connection-mark=30Dconn 
2 chain=postrouting action=mark-connection new-connection-mark=5Dconn passthrough=yes dst-address-list=5D 
3 chain=postrouting action=mark-packet new-packet-mark=5D passthrough=no connection-mark=5Dconn 
4 chain=postrouting action=mark-connection new-connection-mark=10Dconn passthrough=yes dst-address-list=10D 
5 chain=postrouting action=mark-packet new-packet-mark=10D passthrough=no connection-mark=10Dconn 
6 chain=postrouting action=mark-connection new-connection-mark=1Dconn passthrough=yes dst-address-list=1D 
7 chain=postrouting action=mark-packet new-packet-mark=1D passthrough=no connection-mark=1Dconn 
8 chain=postrouting action=mark-connection new-connection-mark=50Dconn passthrough=yes dst-address-list=50D 
9 chain=postrouting action=mark-packet new-packet-mark=50D passthrough=no connection-mark=50Dconn 
10 chain=postrouting action=mark-connection new-connection-mark=2Dconn passthrough=yes dst-address-list=2D 
11 chain=postrouting action=mark-packet new-packet-mark=2D passthrough=no connection-mark=2Dconn 
12 chain=postrouting action=mark-connection new-connection-mark=20Dconn passthrough=yes dst-address-list=20D 
13 chain=postrouting action=mark-packet new-packet-mark=20D passthrough=no connection-mark=20Dconn 
14 chain=postrouting action=mark-connection new-connection-mark=100Dconn passthrough=yes dst-address-list=100D 
15 chain=postrouting action=mark-packet new-packet-mark=100D passthrough=no connection-mark=100Dconn 
16 chain=postrouting action=mark-connection new-connection-mark=40Dconn passthrough=yes dst-address-list=40D 
17 chain=postrouting action=mark-packet new-packet-mark=40D passthrough=no connection-mark=40Dconn 
18 chain=postrouting action=mark-connection new-connection-mark=05Dconn passthrough=yes dst-address-list=05D 
19 chain=postrouting action=mark-packet new-packet-mark=05D passthrough=no connection-mark=05Dconn

20 chain=prerouting action=mark-connection new-connection-mark=30Uconn passthrough=yes src-address-list=30U 
21 chain=prerouting action=mark-packet new-packet-mark=30U passthrough=no connection-mark=30Uconn 
22 chain=prerouting action=mark-connection new-connection-mark=5Uconn passthrough=yes src-address-list=5U 
23 chain=prerouting action=mark-packet new-packet-mark=5U passthrough=no connection-mark=5Uconn 
24 chain=prerouting action=mark-connection new-connection-mark=10Uconn passthrough=yes src-address-list=10U 
25 chain=prerouting action=mark-packet new-packet-mark=10U passthrough=no connection-mark=10Uconn 
26 chain=prerouting action=mark-connection new-connection-mark=20Uconn passthrough=yes src-address-list=20U 
27 chain=prerouting action=mark-packet new-packet-mark=20U passthrough=no connection-mark=20Uconn 
28 chain=prerouting action=mark-connection new-connection-mark=50Uconn passthrough=yes src-address-list=50U 
29 chain=prerouting action=mark-packet new-packet-mark=50U passthrough=no connection-mark=50Uconn 
30 chain=prerouting action=mark-connection new-connection-mark=2Uconn passthrough=yes src-address-list=2U 
31 chain=prerouting action=mark-packet new-packet-mark=2U passthrough=no connection-mark=2Uconn 
32 chain=prerouting action=mark-connection new-connection-mark=1Uconn passthrough=yes src-address-list=1U 
33 chain=prerouting action=mark-packet new-packet-mark=1U passthrough=no connection-mark=1Uconn 
34 chain=prerouting action=mark-connection new-connection-mark=100Uconn passthrough=yes src-address-list=100U 
35 chain=prerouting action=mark-packet new-packet-mark=100U passthrough=no connection-mark=100Uconn 
36 chain=prerouting action=mark-connection new-connection-mark=40Uconn passthrough=yes src-address-list=40U 
37 chain=prerouting action=mark-packet new-packet-mark=40U passthrough=no connection-mark=40Uconn 
38 chain=prerouting action=mark-connection new-connection-mark=05Uconn passthrough=yes src-address-list=05U 
39 chain=prerouting action=mark-packet new-packet-mark=05U passthrough=no connection-mark=05Uconn

0 name="default" kind=pfifo pfifo-limit=50 
1 name="ethernet-default" kind=pfifo pfifo-limit=50 
2 name="wireless-default" kind=sfq sfq-perturb=5 sfq-allot=1514 
3 name="synchronous-default" kind=red red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20 red-avg-packet=1000 
4 name="hotspot-default" kind=sfq sfq-perturb=5 sfq-allot=1514 
5 name="sfq" kind=sfq sfq-perturb=5 sfq-allot=1514 
6 name="05D" kind=pcq pcq-rate=528000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=20000 
7 name="1D" kind=pcq pcq-rate=1040000 pcq-limit=40 pcq-classifier=dst-address pcq-total-limit=20000 
8 name="2D" kind=pcq pcq-rate=2064000 pcq-limit=30 pcq-classifier=dst-address pcq-total-limit=20000 
9 name="5D" kind=pcq pcq-rate=5136000 pcq-limit=30 pcq-classifier=dst-address pcq-total-limit=40000 
10 name="10D" kind=pcq pcq-rate=10000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=20000 
11 name="20D" kind=pcq pcq-rate=20000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=20000 
12 name="30D" kind=pcq pcq-rate=30000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=20000 
13 name="40D" kind=pcq pcq-rate=40000000 pcq-limit=10 pcq-classifier=dst-address pcq-total-limit=20000 
14 name="50D" kind=pcq pcq-rate=50000000 pcq-limit=10 pcq-classifier=dst-address pcq-total-limit=20000 
15 name="100D" kind=pcq pcq-rate=100000000 pcq-limit=10 pcq-classifier=dst-address pcq-total-limit=20000 
16 name="05U" kind=pcq pcq-rate=528000 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=20000 
17 name="1U" kind=pcq pcq-rate=1040000 pcq-limit=40 pcq-classifier=src-address pcq-total-limit=20000 
18 name="2U" kind=pcq pcq-rate=2064000 pcq-limit=30 pcq-classifier=src-address pcq-total-limit=20000 
19 name="5U" kind=pcq pcq-rate=5136000 pcq-limit=30 pcq-classifier=src-address pcq-total-limit=40000 
20 name="10U" kind=pcq pcq-rate=10000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=20000 
21 name="20U" kind=pcq pcq-rate=20000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=20000 
22 name="30U" kind=pcq pcq-rate=30000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=20000 
23 name="40U" kind=pcq pcq-rate=40000000 pcq-limit=10 pcq-classifier=src-address pcq-total-limit=20000 
24 name="50U" kind=pcq pcq-rate=50000000 pcq-limit=10 pcq-classifier=src-address pcq-total-limit=20000 
25 name="100U" kind=pcq pcq-rate=100000000 pcq-limit=10 pcq-classifier=src-address pcq-total-limit=20000 
26 name="default-small" kind=pfifo pfifo-limit=10

0 name="Download" parent=LAN limit-at=0 priority=8 max-limit=500M burst-limit=0 burst-threshold=0 burst-time=0s 
1 name="Upload" parent=WAN limit-at=0 priority=8 max-limit=500M burst-limit=0 burst-threshold=0 burst-time=0s 

2 name="Download 528kbps" parent=Download packet-mark=05D limit-at=0 queue=05D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
3 name="Download 1Mbps" parent=Download packet-mark=1D limit-at=0 queue=1D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
4 name="Download 2Mbps" parent=Download packet-mark=2D limit-at=0 queue=2D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
5 name="Download 5Mbps" parent=Download packet-mark=5D limit-at=0 queue=5D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
6 name="Download 10Mbps" parent=Download packet-mark=10D limit-at=0 queue=10D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
7 name="Download 20Mbps" parent=Download packet-mark=20D limit-at=0 queue=20D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
8 name="Download 30Mbps" parent=Download packet-mark=30D limit-at=0 queue=30D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
9 name="Download 40Mbps" parent=Download packet-mark=40D limit-at=0 queue=40D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
10 name="Download 50Mbps" parent=Download packet-mark=50D limit-at=0 queue=50D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
11 name="Download 100Mbps" parent=Download packet-mark=100D limit-at=0 queue=100D priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
12 name="Upload dla 528kbps" parent=Upload packet-mark=05U limit-at=0 queue=05U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
13 name="Upload 1Mbps" parent=Upload packet-mark=1U limit-at=0 queue=1U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
14 name="Upload 2Mbps" parent=Upload packet-mark=2U limit-at=0 queue=2U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
15 name="Upload 5Mbps" parent=Upload packet-mark=5U limit-at=0 queue=5U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
16 name="Upload 10Mbps" parent=Upload packet-mark=10U limit-at=0 queue=10U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
17 name="Upload 20Mbps" parent=Upload packet-mark=20U limit-at=0 queue=20U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
18 name="Upload 30Mbps" parent=Upload packet-mark=30U limit-at=0 queue=30U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
19 name="Upload 40Mbps" parent=Upload packet-mark=40U limit-at=0 queue=40U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
20 name="Upload 50Mbps" parent=Upload packet-mark=50U limit-at=0 queue=50U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 
21 name="Upload 100Mbps" parent=Upload packet-mark=100U limit-at=0 queue=100U priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

Is this correct?

Beccara · Thu Nov 25, 2010 12:32 am

Use PCQ to decrease the number of queues you need in total, what are you doing that require 2600 queues?

jankad · Thu Nov 25, 2010 12:48 am

In the configuration of the PCQ queues have only two main queues and twenty child queues exactly as pasted in the configuration above.

Previously, the configuration of the queue SFQ main idea lies in the fact that each customer may have a different tariff. For each client, the script generates two rules and two mangle queue. It looks like this:

/ip firewall mangle add chain=MIE5-in dst-address=195.177.64.227 action=mark-packet new-packet-mark=jankad.mie5.download passthrough=no comment=USR
/ip firewall mangle add chain=MIE5-out src-address=195.177.64.227 action=mark-packet new-packet-mark=jankad.mie5.upload passthrough=no comment=USR
/queue tree add name=USR-MIE5.jankad-download parent=Download packet-mark=jankad.mie5.download queue=sfq priority=8 max-limit=4096000
/queue tree add name=USR-MIE5.jankad-upload parent=Upload packet-mark=jankad.mie5.upload queue=sfq priority=8 max-limit=2064000

Besides, I had the chains to which jumped packets based on destination / source subnet. It looks like this:

/ip firewall mangle
add action=jump chain=prerouting comment="" disabled=no jump-target=MIE5-out \
    src-address=195.177.64.224/27
add action=jump chain=postrouting comment="" disabled=no dst-address=\
    195.177.64.224/27 jump-target=MIE5-in

In total I have about 50 subnets so have about 100 mangle rules: two for each subnet (one in postrouting and one in prerouting).
Then for each IP address have two mark rules and finally two queues for each customer.

BTW: even with the new configuration 5.0RC4 has a problem with displaying long lists. My address-list is not always complete.

Beccara · Thu Nov 25, 2010 1:59 am

If you have 50subnets that you need to apply the same mangle rules to then use an address list and only have 2 rules

jankad · Thu Nov 25, 2010 5:57 am

Why would I jump all the subnets to the same chain?

The idea is that I have 50 rules in prerouting and 50 in postrouting to jump to the chain in which they are usually mark rules for 32 IP addresess. In this way, the average number of tests is 50 / 2 * 32 / 2 = 400 for each packet traversing the firewall.

If all subnets jumps to the same chain average number of tests would be 50 / 2 * 1600 / 2 = 40000.

That was the configuration of the SFQ.

Now, using the PCQ average number of tests is 10 / 2 = 5 because I have defined 10 tariffs and thus 10 rules in the mangle.
Despite this performance has not increased or even declined.
Besides, from the viewpoint of customers, the situation got worse because, as everyone had their own SFQ was not dependent on other users with the same tariff as it is now. As the user does not generate traffic on the sea for their maximum tariffs do not have any loss or delay packets. If he generate it it was his responsibility.

In conclusion I am not convinced by the superiority of the PCQ in relation to the SFQ and still do not know why my configuration was wrong and PCQ is ok, since my work better.

Maybe it turns out that something's wrong in my PCQ configuration.

Thu Nov 25, 2010 9:12 am

to jankad:

a) in your mangle you have 2 rules per packet mark - i suggest to go for 3
1) mark connection if src-address-list=x
2) mark-connection if dst-address-list=x
3) mark-packet for this connection mark

b) think carefully about situations like this - 100D client are downloading from 05U client, what should be speed of this connection - 0,5 or 100M??? Choose one and in mangle place corresponding rules above others

c) PCQ finally works with subqueue rates that are higher than 10Mbps.

What's new in 5.0rc4 (2010-Nov-22 16:58):
*) fixed pcq queue type;

So without any hesitation i would go for an upgrade

d) if it is necessary it is possible to distribute traffic between customer classes based on limit-at and max-limit of queue

jankad · Thu Nov 25, 2010 10:03 am

to jankad:

a) in your mangle you have 2 rules per packet mark - i suggest to go for 3
1) mark connection if src-address-list=x
2) mark-connection if dst-address-list=x
3) mark-packet for this connection mark

b) think carefully about situations like this - 100D client are downloading from 05U client, what should be speed of this connection - 0,5 or 100M??? Choose one and in mangle place corresponding rules above others

My shaper is not router, so there is not traffic between customers. There is only traffic form/to internet.
Shaper works in bridge mode and is positioned at link between border router (Cisco with BPG) and access router (Layer 3 switch).

c) PCQ finally works with subqueue rates that are higher than 10Mbps.

What's new in 5.0rc4 (2010-Nov-22 16:58):
*) fixed pcq queue type;

So without any hesitation i would go for an upgrade

I allready use 5.0RC4.

smile__2006 · Sun Dec 19, 2010 2:33 am

-> jankad
try to set multi-cpu=no to resolve problem
PS use pcq instead of simple queues))!

Two problems with shaper in 5.0RC4

Two problems with shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems witch shaper in 5.0RC4

Re: Two problems with shaper in 5.0RC4

Who is online