pcq works fine for download, but not for upload, why?

RouterOS General
1

Hi,
I have the typical pcq configuration:
add kind=pcq name=default_download pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=dst-address pcq-dst-address-mask=32
pcq-dst-address6-mask=64 pcq-limit=75 pcq-rate=6M pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=7000
add kind=pcq name=default_upload pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=src-address pcq-dst-address-mask=32
pcq-dst-address6-mask=64 pcq-limit=75 pcq-rate=400k pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=7000

This pcq are used in a tree:
Donwload branch:
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=500k max-limit=20M name=W2_D8 packet-mark=XXXX parent=Ether2 priority=8 \ queue=default_download

Upload branch:
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=50k max-limit=900k name=W2_U8 packet-mark=XXXXX2 parent=Upload_W2
priority=8 queue=default_upload

Question is, download branch works fine, and maxlimit is reached, but upload branch don’t works and is always 400k (pcq limit) and not 900k(branch limit)
why??

Thanks

2

Well you’re not giving a complete picture of your setup, so no one can really comment on why something is working or not. We need to see how you are marking your traffic with the firewall mangle, and all of your queues. The router can only limit the traffic it sends out of a given interface, not what it gets in on a given interface. Without knowing what that upload parent is tied to, it’s hard to comment on why it’s not working, and without seeing how you are marking traffic it’s hard to point to any issues.

3

Sorry… you are right.
I readed today for hours looking for my error. I will explain it better in a minutes, I think that my mangles rules are not right.

Thank you

4

Before writing code over and make you waste your time reading it, I have a basic question.

In my RouterOS I want to control the download and upload with PCQ (with Tree and Mangles obviously) for http, gamer, p2p, etc. by prioritys. In the famous:
http://mum.mikrotik.com/presentations/CZ09/QoS_Megis.pdf
is seen that in the mangle marking does in “forward”, and that same mark is used for Download Tree and Upload Tree.

I make however two marks, the first mark in the PREROUTING and I use it for Donwload Tree, and then back to mark with a different mark_name in same connection in the POSTROUTING, and use this new mark for the brand in the Upload Tree. I honestly do not remember because I did this at the time, I probably saw it somewhere.
Is it right or needed to dial twice?

I think that the reason that does not reach the MaxLimit of Upload tree branch, is because all the upload is considered a single client, not several, and therefore is never greater than the max limit of PCQ, and max limit of upload tree branch is ignored

Code summary: (I have two interfaces, one only for http trafic)

/queue type
add kind=pcq name=default_download pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=dst-address pcq-dst-address-mask=32 pcq-dst-address6-mask=64 pcq-limit=75 pcq-rate=12M pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=7000
add kind=pcq name=default_upload pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=src-address pcq-dst-address-mask=32 pcq-dst-address6-mask=64 pcq-limit=75 pcq-rate=600k pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=7000

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=9M name=Download packet-mark=“” parent=bridge1 priority=1
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=5M max-limit=5M name=Ether1 packet-mark=“” parent=Download priority=2
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=5M max-limit=5M name=Ether2 packet-mark=“” parent=Download priority=1
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=400k name=Upload_W1 packet-mark=“” parent=ether1 priority=1
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=400k name=Upload_W2 packet-mark=“” parent=ether2 priority=1

/* Intercafe eth1 for non-http */
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=2M max-limit=7M name=W1_D1 packet-mark=downoad_Winbox,downoad_dns,download_ping parent=Ether1 priority=1 queue=default_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=1M max-limit=15M name=W1_D8 packet-mark=download_p2p_packet_mark parent=Ether1 priority=8 queue=default_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=150k max-limit=400k name=W1_U1 packet-mark=dhcp,dns,winbox parent=Upload_W1 priority=1 queue=default_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=5k max-limit=500k name=W1_U8 packet-mark=p2p parent=Upload_W1 priority=8 queue=default_upload

/* Interface eth2 for http */
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=1M max-limit=4M name=W2_D1 packet-mark=download_games_packet_mark parent=Ether2 priority=1 queue=default_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=2M max-limit=20M name=W2_D4 packet-mark=download_httpmax_packet_mark parent=Ether2 priority=4 queue=default_download
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=1M max-limit=18M name=W2_D7 packet-mark=download_httpmax_p2p parent=Ether2 priority=7 queue=default_download

add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=150k max-limit=500k name=W2_U1 packet-mark=dhcp,dns,games_packet_mark parent=Upload_W2 priority=1 queue=default_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=300k max-limit=750k name=W2_U3 packet-mark=web parent=Upload_W2 priority=3 queue=default_upload
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=50k max-limit=600k name=W2_U8 packet-mark=web_file_p2p parent=Upload_W2 priority=8 queue=default_upload

/ip firewall mangle
add action=mark-packet chain=postrouting connection-bytes=0-50000 connection-mark=http_con disabled=no new-packet-mark=web out-interface=ether2 passthrough=no protocol=tcp
add action=mark-packet chain=postrouting connection-bytes=50000-0 connection-mark=http_con disabled=no new-packet-mark=web_file out-interface=ether2 passthrough=no protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=new disabled=no dst-port=80,443 new-connection-mark=http_con passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-bytes=0-200000 connection-mark=http_con disabled=no new-packet-mark=download_http_packet_mark passthrough=no protocol=tcp
add action=mark-packet chain=prerouting connection-bytes=200000-0 connection-mark=http_con disabled=no new-packet-mark=download_httpmax_packet_mark passthrough=no protocol=tcp

Where to mark packets in mangle?
5

Where you mark traffic really depends on what you are going for. The reason why people generally mark packets in forward is because it is after dst-nat has happened and before src-nat has happened. This means that you know where a packet has come from and where it is going to.
wiki.mikrotik.com/wiki/Packet_Flow#Diagram

The most important thing to know and remember about queues, and queue trees is that the router can only control traffic it sends out, it has no control over traffic it gets in on a given interface. What this piratically means is that in order to control your download traffic you need to have your queue that controls download on the LAN interface. In order to control upload traffic you need to have your queue on the WAN interface. I see that both your upload and download queues are on ether1 and ether2, I’m not sure if both of those are LANs or, if one is the WAN and one is the LAN.

You can use the global-x queues if desired to kind of get the same functionality, but I find it easier for myself to keep things separate by logical interface and control it on the interface level.

6

Thank you very much for your answer, I understand some things now.

After reading your response and look at the packet-flow:

  • It is wrong to mark packets in POSTROUTING for processing an upload queue using PCQ (classified by source). Being POSTROUTING after SRC-NAT would have a unique PCQ queue (Wan IP) and the traffic is not distributed equally among clients. Is right this?

  • Is it correct to use the bridge as a LAN interface in the download queue? (bridge has only one interface inside in my configuration)

  • In http://mum.mikrotik.com/presentations/CZ09/QoS_Megis.pdf page 32-34 explain the prioritization of traffic (P2P, VoIP, etc..), and specify that ‘prerouting’ mut be used, but not ‘forward’. I can not understand why.
    Should I also mark packets in ‘forward’ to control the upload?, Or what is the same for me and I really want to know, is there a unique point where mark packets serve to prioritize downstream traffic (P2P, VoIP, etc..) for download tree and also the upload wan traffic for upload tree? or mark twice is needed? (I use PCQ in all branch trees)

7

If anyone can clarify these questions, I’d be grateful.
thank you very much

8

First of all, there are many options here, and QoS is a very complex subject and it is hard to explain everything fully in a post. So keep that in mind.

A lot of this really comes down to doing what makes the most sense to you. There is no “right” or “wrong” place to mark packets when applying limits. It comes down to what you want to happen, and where they need to be marked in the chain to accomplish that. Hence why I choose the forward, along with putting a queue on the LAN and WAN interfaces. That setup helps me think about it logically the best and work out from there, and helps me to remember that if I want to limit upload, I put a queue on the WAN interface and mark traffic based on that. But I also don’t use PCQ in that situation, I use pfifo because I don’t care about dividing up the traffic into streams within the queue. Look at the packet flow diagram to see where a packet is and what you can do with it. Postrouting mangle does happen before src-nat, so it is possible to use it for an upload queue. Depending on how you want PCQ to divide up the streams, will tell you where you want to mark the packets.

The reason why I say it depends on how you want PCQ to divide up the traffic is because PCQ uses header information to make sub queues. So once a packet reaches the interface it’s leaving, some of that information that you were relying on being there to make a subqueue may not be available to it.

In my preferred method, yes, whatever the LAN interface is would be the download queue parent.

The reason why those pages say prerouting in mangle is because in that example they want to use the “global-x” queue as the parent, and therefore the packets need to be marked before that to be processed by it. When dealing with interfaces and putting queues on them directly, any chain is fine depending on what information you want to be available to the queue.

One thing to also keep in mind with queues and setting up priority over other kinds of packet marks is that it will only reorder packets at a time of congestion. Since when an internet link reaches around %90 traffic utilization, things start to not work correctly, in order to get the limits and behavior to do what you want, you need to have the maximum limit of your queue at around %90 of the actual link capacity to create a time of artificial congestion.

9

First of all, thank you very much for the long answer and time spent. It has been very useful.

Concretely my setup:

  • I have two wan of 10mb download and 1mb upload
  • I want to share it to several users so that they have a maximum of 5Mb download and 500kb upload, so I use PCQ to share the existing bandwidth equally (when it is not posible to reach 5Mb/500kb)
  • In addition to share/limit the bandwidth, I prioritize traffic (high for VoIP, low for P2P, etc.)
    In summary, a typical configuration with traffic prioritization + PCQ to share/limit download and upload bandwith.

So, I have three trees:

  • bridge1 (lan interfaces bridged) as parent for download treee
  • pppoe-out1 as parent for upload tree of wan1
  • pppoe-out2 as parent for upload tree of wan2

Each tree, has branches for every type of packet (http, VoIP, etc.)

So, What would be the appropriate place/places to mark the packets for this typical configuration?
Is there any relevant data that I has not written that you need to know for this?

10

Global-Out is where you are going to want to place the queues on in that case for upload since you want to have a per user limit/PCQ that will automatically divide up the bandwidth. The reason for that being is that the interface queue is after src-NAT so it will only see the src-address of the WAN interface. Global-Out will see the client’s IP address. In addition to this, when marking traffic for upload you will want to duplicate the upload marking twice, one for pppoe1 and one for pppoe2. Since Global-Out will not be aware of what interface something is going out of (you will only be feeding it the total upload of both interfaces), you will need to have two sub queues for Global out, one for pppoe1 and one for pppoe2, and have appropriate limits set there.

The mangle rule will contains something like this, narrowed down to mark how you want:

/ip firewall mangle
add action=mark-packet chain=forward out-interface=WAN1 new-packet-mark=wan1_upload passthrough=no
add action=mark-packet chain=forward out-interface=WAN2 new-packet-mark=wan2_upload passthrough=no

Given that since Global-out is after forward and postrouting, either place is fine to mark the packets for your upload.

For download, forward or postrouting again is where you want to mark packets. This is because return packets from servers will not have had their dst-address changed to the clients for their download.