PCQ implementation - what is wrong?

Bomber67 · November 17, 2013, 7:15pm

Hello,

I am trying to employ a “fair” sharing of bandwith between users at a customers location.
All users (unknown number) are NATed and on DHCP, total bandwidth for customer is 8000/2000 kbps.

What I’m looking for is a scheme where all users are granted a minimum of bandwith at any time, and where total bandwith is shared “fairly” between them.
Assuming 16 users (kinda wild shot, can be 5 and it can be 20) I was thinking they should be granted 512k down and 128k up.

Here are my setup:

/ip firewall mangle
add action=mark-connection chain=postrouting new-connection-mark=\
    conn_client_download out-interface=ether1
add action=mark-connection chain=prerouting in-interface=ether1 \
    new-connection-mark=conn_client_upload
add action=mark-packet chain=prerouting connection-mark=conn_client_download \
    new-packet-mark=packet_client_download passthrough=no
add action=mark-packet chain=prerouting connection-mark=conn_client_upload \
    new-packet-mark=packet_client_upload passthrough=no

/queue type
add kind=pcq name=PCQ_download pcq-classifier=dst-address pcq-rate=8M
add kind=pcq name=PCQ_upload pcq-classifier=src-address pcq-rate=2M

/queue tree
add limit-at=512k max-limit=8M name=Queue_download packet-mark=\
    packet_client_download parent=global queue=PCQ_download
add limit-at=128k max-limit=2M name=Queue_upload packet-mark=\
    packet_client_upload parent=global queue=PCQ_upload

What happens when I employ this is that the upload queue immediately turns red (saturated) and that download is significantly reduced.

Assuming a number of users (let’s say 5) are downloading/uploading at 4000/1000 at the moment, applying these queues shouldn’t restrict total, right?

derr12 · November 18, 2013, 7:07pm

Remove the limit on your queue types, the queue tree will be the one handling your speed limits. Other than that it could be that your packet marking is not working as expected.

I usually just mark traffic based on the source/destination IP or interface, for example;

26 ;;; Mark all remaining Upload.
chain=forward action=mark-packet new-packet-mark=p4_up passthrough=no
src-address-list=public out-bridge-port=ether1

27 ;;; Mark all remaining Download
chain=forward action=mark-packet new-packet-mark=p4_down passthrough=no
dst-address-list=public in-bridge-port=ether1

In my case, this MT is bridging. so you will want to taylor the rule for NAT.

derr12 · November 18, 2013, 7:11pm

As for the minimum bandwidth, you will want to create a simple queue and an additional PCQ queue type Here is what I use in a hotel, an RB1100 being a controller for 11 bridged wireless ap’s. Tho im doing some shaping, its a slightly more complicated version of what you are trying to do.

My mangles;

/ip firewall mangle
add action=mark-packet chain=forward comment="DNS Uploads" dst-port=53 new-packet-mark=p1_up passthrough=no protocol=tcp \
    src-address-list=public
add action=mark-packet chain=forward comment="DNS Downloads" dst-address-list=public new-packet-mark=p1_down passthrough=no \
    protocol=tcp src-port=53
add action=mark-packet chain=forward comment="DNS Uploads" dst-port=53 new-packet-mark=p1_up passthrough=no protocol=udp \
    src-address-list=public
add action=mark-packet chain=forward comment="DNS Downloads" dst-address-list=public new-packet-mark=p1_down passthrough=no \
    protocol=udp src-port=53
add action=mark-connection chain=forward comment="VOIP Connection mark SIP" dscp=26 layer7-protocol=sip new-connection-mark=\
    VOIP26 protocol=udp
add action=mark-packet chain=forward comment="SIP Uploads" connection-mark=VOIP26 new-packet-mark=p2_up passthrough=no \
    src-address-list=public
add action=mark-packet chain=forward comment="SIP Downloads" connection-mark=VOIP26 dst-address-list=public new-packet-mark=\
    p2_down passthrough=no
add action=mark-connection chain=forward comment="VOIP Connection mark RTP" dscp=46 layer7-protocol=RTP-accurate \
    new-connection-mark=VOIP46
add action=mark-packet chain=forward comment="RTP Uploads" connection-mark=VOIP46 new-packet-mark=p2_up passthrough=no \
    src-address-list=public
add action=mark-packet chain=forward comment="RTP Downloads" connection-mark=VOIP46 dst-address-list=public new-packet-mark=\
    p2_down passthrough=no
add action=mark-connection chain=forward comment="HTTP Download conn mark" dst-address-list=public new-connection-mark=http_down \
    protocol=tcp src-port=80,443,8080,81
add action=mark-connection chain=forward comment="HTTP upload conn mark" dst-port=80,443,8080,81 new-connection-mark=http_up \
    protocol=tcp src-address-list=public
add action=mark-packet chain=forward comment=HTTP-Download connection-mark=http_down new-packet-mark=p3_down passthrough=no
add action=mark-packet chain=forward comment=HTTP-Upload connection-mark=http_up new-packet-mark=p3_up passthrough=no
add action=mark-connection chain=forward comment="E-mail Connection" dst-port=25,110,143,465,585,587,993,995 \
    new-connection-mark=Email protocol=tcp
add action=mark-packet chain=forward comment="E-mail Upload" connection-mark=Email new-packet-mark=p3_up passthrough=no \
    src-address-list=public
add action=mark-packet chain=forward comment="E-mail Downloads" connection-mark=Email dst-address-list=public new-packet-mark=\
    p3_down passthrough=no
add action=mark-connection chain=forward comment="Xbox + PS3 Download tcp" new-connection-mark=xbox+ps3_down protocol=tcp \
    src-port=3074,5223,3074
add action=mark-connection chain=forward comment="Xbox + ps3 download UDP" dst-address-list=public new-connection-mark=\
    xbox+ps3_down protocol=udp src-port=88,3074,3478,3479,3658
add action=mark-connection chain=forward comment="Xbox+ps3 Up TCP" dst-port=3074,5223,3074 new-connection-mark=xbox+ps3_Up \
    protocol=tcp
add action=mark-connection chain=forward comment="Xbox + PS3 up UDP" dst-port=88,3074,3478,3479,3658 new-connection-mark=\
    xbox+ps3_Up protocol=udp
add action=mark-packet chain=forward comment="Xbox+ps3 Download" connection-mark=xbox+ps3_down dst-address-list=public \
    new-packet-mark=p3_down passthrough=no
add action=mark-packet chain=forward comment="Xbox&ps3 Upload" connection-mark=xbox+ps3_Up new-packet-mark=p3_up passthrough=no \
    src-address-list=public
add action=mark-connection chain=forward comment="Mark p2p with connection-mark" disabled=yes new-connection-mark=p2p_con p2p=\
    all-p2p
add action=mark-packet chain=forward comment="P2P up" connection-mark=p2p_con disabled=yes new-packet-mark=p5_up passthrough=no \
    src-address-list=public
add action=mark-packet chain=forward comment="p2p Down" connection-mark=p2p_con disabled=yes dst-address-list=public \
    new-packet-mark=p5_down passthrough=no
add action=mark-packet chain=forward comment="Mark all remaining Upload." new-packet-mark=p4_up out-bridge-port=ether1 \
    passthrough=no src-address-list=public
add action=mark-packet chain=forward comment="Mark all remaining Download" dst-address-list=public in-bridge-port=ether1 \
    new-packet-mark=p4_down passthrough=no

My queues;

/queue type
add kind=pcq name="3m per user down" pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-limit=35 pcq-rate=3M \
    pcq-src-address6-mask=64 pcq-total-limit=10000
add kind=pcq name="1m per user up" pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-limit=35 pcq-rate=1M \
    pcq-src-address6-mask=64 pcq-total-limit=10000
set 7 pcq-limit=35 pcq-total-limit=10000
set 8 pcq-limit=35 pcq-total-limit=10000
/queue simple
add name="user limit" queue="1m per user up/3m per user down" target=10.0.0.0/22
/queue tree
add max-limit=30M name=Uploads_Full parent=global priority=1 queue=pcq-upload-default
add limit-at=256k max-limit=30M name=UP_priority1 packet-mark=p1_up parent=Uploads_Full priority=1 queue=pcq-upload-default
add limit-at=768k max-limit=30M name=UP_priority2 packet-mark=p2_up parent=Uploads_Full priority=2 queue=pcq-upload-default
add limit-at=8M max-limit=29M name=UP_priority3 packet-mark=p3_up parent=Uploads_Full priority=3 queue=pcq-upload-default
add limit-at=2M max-limit=3M name=UP_priority4 packet-mark=p4_up parent=Uploads_Full priority=4 queue=pcq-upload-default
add limit-at=1M max-limit=1M name=UP_priority5 packet-mark=p5_up parent=Uploads_Full priority=5 queue=pcq-upload-default
add max-limit=30M name=Downloads_Full parent=global priority=1 queue=default
add limit-at=256k max-limit=30M name=DN_priority1 packet-mark=p1_down parent=Downloads_Full priority=1 queue=\
    pcq-download-default
add limit-at=768k max-limit=30M name=DN_priority2 packet-mark=p2_down parent=Downloads_Full priority=2 queue=\
    pcq-download-default
add limit-at=25M max-limit=29M name=DN_priority3 packet-mark=p3_down parent=Downloads_Full priority=3 queue=pcq-download-default
add limit-at=2M max-limit=3M name=DN_priority4 packet-mark=p4_down parent=Downloads_Full priority=4 queue=default
add limit-at=1M max-limit=1M name=DN_priority5 packet-mark=p5_down parent=Downloads_Full priority=5 queue=pcq-download-default

Bomber67 · November 18, 2013, 8:06pm

Thank you for your reply derr,

I think my mangling is ok, it’s something about the queues…
I must admit I don’t quite understand your setup, but basically you are shaping each user at 3/1 Mbps and then prioritizing traffic in the queue trees based on priority, right?
So simple queues and queue trees can live together like this, handling the same packet after eachother? I didn’t quite know that.

You are restricting your customers so they will never get more than 3/1, right?
I am trying to “share fairly” the bandwidth among the users. How do I do that?

By the phrase “remove the limit on your queue types”, do you mean I shall omit the PCQ-rate parameter or set it to let’s say 100M?

Bomber67 · November 18, 2013, 8:41pm

Will simply this do the trick?:

/queue type
add kind=pcq name=PCQ_download pcq-classifier=dst-address
add kind=pcq name=PCQ_upload pcq-classifier=src-address

/queue tree
add name=Queue_download packet-mark=packet_client_download parent=global queue=\
    PCQ_download
add name=Queue_upload packet-mark=packet_client_upload parent=global queue=\
    PCQ_upload

By omitting pcq-rate of the queue types, the bandwidth will be shared fairly between users, or do I misunderstand everything?