Community discussions

MUM Europe 2020
 
gunther01
newbie
Topic Author
Posts: 39
Joined: Sun Aug 01, 2010 7:00 pm

Trying to set up PCQ

Fri Mar 11, 2011 10:51 pm

I have this working I believe now.. But I am not totally convinced it is correct. My head end router is running v4.11. It does NAT, 1:1, some firewall rules, and PCQ for our network. I would like to make sure I have the proper interfaces and settings for the PCQ if someone is willing to verify. I have read till I got confused from all the responses. Viewed/read the QOS on Tiktube half a dozen times, and one thing that is not clear to me is what to do when you use NAT.

My main questions.
Which interface would you use for your parent queues?
And do you use pre-routing, forward or what chain in your mangles while trying to do NAT on the same box?

We have seen where PCQ seems to over regulate (too slow) our customers when bandwidth is clearly available. And I am thinking that this is because of an improper set up in one of these areas.

I am also not terribly clear on the PCQ limits. But I am still reading up on those and trying to grasp what the best setting (calculation) should be with our customer counts.
 
gunther01
newbie
Topic Author
Posts: 39
Joined: Sun Aug 01, 2010 7:00 pm

Re: Trying to set up PCQ

Sun Mar 13, 2011 1:57 am

What, no body knows how to do this?? It's not like I asked which or what script do I enter in my mangle rules, or how to set up PCQ classifiers..

The one area that is NOT clear in the QOS video and PDF is what is what while using NAT. It's a quick 20 second blurb on the video with no real reference in the PDF. That and I am not sure if the QOS video is set up as a bridge or routed with no NAT. I could just be lost in the translation of it all I suppose.

I have global-out for my download parent, and the Interface to the Internet for the Upload.. Would that be correct? It does seem to work, but doesn't seem like it works correctly.

Searching for PCQ yields results for many,many, things that aren't related to my question. Some even claim it can't even be used in a NAT device??? But the video claims that it can
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Trying to set up PCQ

Sun Mar 13, 2011 2:37 am

What happens when is shown in the packet flow diagram.
http://wiki.mikrotik.com/wiki/Manual:Packet_Flow

Destination NAT happens after prerouting, and therefore before forward/input and postrouting. Destination NAT undoes source NAT. Therefore, marking based on real address for traffic to the client has to happen in forward, or later. Source NAT happens after postrouting, so you can mark whenever you want. That is, if you mark by IP at all. If everyone gets the same mark anyway I find it easier to mark by interface.

Global-in processing (queues that have that as a parent) happens after prerouting, global-out happens after postrouting, interface HTB (interfaces as parents) happens after that.

It's mainly a matter of preference and what you find easier to understand as there are several valid approaches. Personally on simple routers like yours I like to mark upload traffic in prerouting based on in-interface=LAN and use PCQ attached to global-in, and mark download traffic in postrouting based on out-interface=LAN and use PCQ attached to global-out. That works fine. Other people prefer to use interfaces.
One thing to keep in mind is that interface HTB queues only see packets leaving the interface, and never packets entering the specified interface (which is why you shouldn't use the Internet facing interface for upload). See the manual: http://wiki.mikrotik.com/wiki/Manual:Queue#Queues
In RouterOS, these hierarchical structures can be attached at 4 different places:

global-in: represents all the input interfaces in general (INGRESS queue). Queues attached to global-in apply to traffic that is received by the router before the packet filtering
global-out: represents all the output interfaces in general (EGRESS queue).
global-total: represents all input and output interfaces together (in other words it is aggregation of global-in and global-out). Used in case when customers have single limit for both, upload and download.
<interface name>: - represents one particular outgoing interface. Only traffic that is designated to go out via this interface will pass this HTB queue.
The PCQ limits are simply the number of packets any dynamic sub-queue can hold, the total-limit is the number of packets held for ALL dynamic sub-queues. It should be equal to the pcq-limit multiplied by the maximum number of simultaneous sub-queues you are expecting. If you have lots of RAM just assume that to be the total number of clients if you are using src-address and dst-address PCQ classifiers.

Hope that helps. Really, it's all in the packet flow diagram (which is easily the best and most valuable document in the wiki manual).
Specific answers require specific questions. When in doubt, post the output of "/ip address print detail", "/ip route print detail", "/interface print detail", "/ip firewall export", and an accurate network diagram.
 
gunther01
newbie
Topic Author
Posts: 39
Joined: Sun Aug 01, 2010 7:00 pm

Re: Trying to set up PCQ

Sun Mar 13, 2011 5:24 am

Thank you Fewi,
I think this is where I got really lost LOL.. But if I am reading this correctly you have two sets of mangles (or recommend that). One for download, and one for upload?? I think, and it's a big think my downloads are being regulated (they seem to be) but it's spuratic as all get out in it's regulation. I think maybe since I am not processing it at the right interface, or chain that is why. It seemed to work pretty well on the bench, but in a production environment it's gone down hill, and we are getting complaints. I can even see where when we have bandwidth available clients aren't downloading at their maximums most of the time. Any help in this is appreciated. And I will re-read your post and try to "grasp" it better/the best that I can. I do need or feel I should add that there is 1:1 NAT on this box also. We do use Private IP's and address list to set IP's to speeds, and this box is a routed head end box. If it helps any with your diagnosis.

I have this for current Mangle (shortened for ease, but no other rules enabled)

4 ;;; Mark Basic Traffic
chain=forward action=mark-connection new-connection-mark=Basic_client_connection passthrough=yes
src-address-list=Basic-Client
5 chain=forward action=mark-packet new-packet-mark=Basic-Client-Traffic passthrough=no
connection-mark=Basic_client_connection
6 ;;; Mark Standard Traffic
chain=forward action=mark-connection new-connection-mark=Standard_client_connection passthrough=yes
src-address-list=Standard-Client
7 chain=forward action=mark-packet new-packet-mark=Standard-Client-Traffic passthrough=no
connection-mark=Standard_client_connection
8 ;;; Mark Soho Traffic
chain=forward action=mark-connection new-connection-mark=SoHo_client_connection passthrough=yes
src-address-list=Soho-Client
9 chain=forward action=mark-packet new-packet-mark=Soho-Client-Traffic passthrough=no
connection-mark=SoHo_client_connection
10 ;;; Mark Pro Traffic
chain=forward action=mark-connection new-connection-mark=Pro_client_connection passthrough=yes
src-address-list=Pro-Client
11 chain=forward action=mark-packet new-packet-mark=Pro-Client-Traffic passthrough=no
connection-mark=Pro_client_connection
12 X ;;; Check for unmarked traffic
chain=forward action=log log-prefix=""

And this for queues..

0 name="default" kind=pfifo pfifo-limit=50
1 name="ethernet-default" kind=pfifo pfifo-limit=50
2 name="wireless-default" kind=sfq sfq-perturb=5 sfq-allot=1514
3 name="synchronous-default" kind=red red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20
red-avg-packet=1000
4 name="hotspot-default" kind=sfq sfq-perturb=5 sfq-allot=1514
5 name="Basic-Down" kind=pcq pcq-rate=768000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=10000
6 name="Basic-Up" kind=pcq pcq-rate=256000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=10000
7 name="Standard-Down" kind=pcq pcq-rate=4000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=10000
8 name="Standard-Up" kind=pcq pcq-rate=1000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=2000
9 name="SoHo-Down" kind=pcq pcq-rate=5000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=10000
10 name="SoHo-Up" kind=pcq pcq-rate=2000000 pcq-limit=20 pcq-classifier=src-address pcq-total-limit=2000
11 name="Pro-Down" kind=pcq pcq-rate=6000000 pcq-limit=20 pcq-classifier=dst-address pcq-total-limit=5000
12 name="Pro-Up" kind=pcq pcq-rate=3000000 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=2000
13 name="MainPCQ-Up" kind=pcq pcq-rate=15000000 pcq-limit=30 pcq-classifier=src-address pcq-total-limit=10000
14 name="Main-PCQ-Down" kind=pcq pcq-rate=15000000 pcq-limit=30 pcq-classifier=dst-address pcq-total-limit=10000
15 name="Large Download TCP" kind=pcq pcq-rate=6000000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=2000
16 name="Large Download UDP" kind=pcq pcq-rate=15000000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=2000
17 name="default-small" kind=pfifo pfifo-limit=10

And finally the tree:

0 name="Download" parent=global-out limit-at=0 priority=5 max-limit=20M burst-limit=0 burst-threshold=0
burst-time=0s
1 name="Upload" parent=ether8 limit-at=0 priority=8 max-limit=15M burst-limit=0 burst-threshold=0 burst-time=0s
2 name="Basic_Download" parent=Download packet-mark=Basic-Client-Traffic limit-at=4M queue=Basic-Down priority=5
max-limit=20M burst-limit=0 burst-threshold=0 burst-time=0s
3 name="Standard_Download" parent=Download packet-mark=Standard-Client-Traffic limit-at=5M queue=Standard-Down
priority=5 max-limit=20M burst-limit=0 burst-threshold=0 burst-time=0s
4 name="Soho_Download" parent=Download packet-mark=Soho-Client-Traffic limit-at=4M queue=SoHo-Down priority=5
max-limit=20M burst-limit=0 burst-threshold=0 burst-time=0s
5 name="Pro-Download" parent=Download packet-mark=Pro-Client-Traffic limit-at=7M queue=Pro-Down priority=5
max-limit=20M burst-limit=0 burst-threshold=0 burst-time=0s
6 name="Basic_Upload" parent=Upload packet-mark=Basic-Client-Traffic limit-at=0 queue=Basic-Up priority=8
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
7 name="Standard_Upload" parent=Upload packet-mark=Standard-Client-Traffic limit-at=0 queue=Standard-Up priority=8
max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s
8 name="Soho_Upload" parent=Upload packet-mark=Soho-Client-Traffic limit-at=0 queue=SoHo-Up priority=8 max-limit=0
burst-limit=0 burst-threshold=0 burst-time=0s
9 name="Pro_Upload" parent=Upload packet-mark=Pro-Client-Traffic limit-at=0 queue=Pro-Up priority=8 max-limit=0
burst-limit=0 burst-threshold=0 burst-time=0s
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Trying to set up PCQ

Sun Mar 13, 2011 5:36 am

That's hard to interpret information without a network diagram (what does ether8 connect to? It would be beneficial so see a diagram showing all interfaces of the router, what they connect to, and what their addressing is) and the address lists (do they contain private or public IPs?). By now you are asking for very specific help - basically if someone can write a rule set for you. That's fine and I'll try to help, but you'll have to also give way more specific information for that to be possible.

You're also using priorities without specifying both limit-at and max-limit. That makes no sense, as priority works based on both values.
Specific answers require specific questions. When in doubt, post the output of "/ip address print detail", "/ip route print detail", "/interface print detail", "/ip firewall export", and an accurate network diagram.
 
gunther01
newbie
Topic Author
Posts: 39
Joined: Sun Aug 01, 2010 7:00 pm

Re: Trying to set up PCQ

Sun Mar 13, 2011 6:01 am

That's useless information without a network diagram (what does ether8 connect to? It would be beneficial so see a diagram showing all interfaces of the router, what they connect to, and what their addressing is) and the address lists (do they contain private or public IPs?). By now you are asking for very specific help - basically if someone can write a rule set for you. That's fine and I'll try to help, but you'll have to also give way more specific information for that to be possible.

You're also using priorities without specifying both limit-at and max-limit. That makes no sense, as priority works based on both values. You may want to either read the queue manuals, or attend some training.
I didn't feel that I was asking for someone to write a rule. Sorry if you took it that way. Just pointers is all I am after.
Ether 8 is my Internet connection. (multiple public blocks)
Ether 4 is Ethernet to a wireless BH to supply my wireless network (Private IP block)
Ether 9 is a Server that is 1:1ed on this MT box for billing and monitoring (Private IP block)

Address lists have both Public and Private. I was unsure which IP to use in the case of 1:1 so I just used both to make sure we captured and slowed down the client in question. We use all privates internally for clients, unless they are 1:1ed at this head end box.
Priorities were changed a while back and I hadn't noticed until I sent you those lists. It was only on upload. All upload were one priority, and all download were another. All priorities have been set to 5 now. Upload parent has a maxlimit of 15 (because everything reads, you have to set a limit on the parent for the child queues to know where to start slowing down. Download parent has a high limit of 20, as do all children since we want any and all of our customers to be able to utilize our full pipe if available (within their PCQ limits). Download children have limit-at lower then high to facilitate allowable "extra" speeds if our pipe is full to our upper class clients.

I have read EVERY single manual/Wiki MT has ever posted for the past 2 years off and on while I "trained" to learn how to use this equipment and software. Please don't offend me because I can't quite figure out how to do this task since the manual doesn't explain it very well. Sorry but the "packet flow diagram" doesn't show me a clear cut example either. And reads in stereo.. I know it should show me, but I'm not a software engineer, nor a linux user for my whole life. I've used other products for the past 6 years, that worked pretty well until we needed more flexibility. I'm trying here..
 
gunther01
newbie
Topic Author
Posts: 39
Joined: Sun Aug 01, 2010 7:00 pm

Re: Trying to set up PCQ

Sun Mar 13, 2011 6:10 am

I can follow your first post. The only thing I am asking is if you have multiple Mangles. One for upload the other for download??
The QOS video claims you don't need to do that with Mangle, but maybe you do using NAT.
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Trying to set up PCQ

Sun Mar 13, 2011 6:43 am

NAT is irrelevant to this. Ignore the fact that you're using NAT.

You do not have to mark twice if you're using interface HTB, since each interface will only see outbound packets.
But there are many different approaches to this, and many things that work. Some stop working when you use them in combination with techniques such as proxies or load balancing, others are more efficient. The simpler approaches are less efficient, but uless you're starved for resources on the router that isn't a problem.
Below is an approach that is easy to understand and works for your situation.

Here is the pretend scenario: a router, two interfaces. Ether1 is the WAN interface, ether2 is the LAN interface. The LAN interface address is 192.168.1.1/24. The WAN interface address is 1.1.1.1/24 and all clients are 1:1 NATed to the Internet. As you will see below this doesn't matter.

192.168.1.10 and .11 are clients that have service A at 512/256 Kbps. .12 and .13 are clients that have service B at 768/512 Kbps. Both services are allowed to consume 10M/5M total for all the customers combined, per service.
/ip firewall address-list
add list=serviceA address=192.168.1.10
add list=serviceA address=192.168.1.11
add list=serviceB address=192.168.1.12
add list=serviceB address=192.168.1.13
/ip firewall mangle
add chain=prerouting in-interface=ether2 src-address-list=serviceA action=mark-packet new-packet-mark=serviceA-up
add chain=postrouting out-interface=ether2 dst-address-list=serviceA action=mark-packet new-packet-mark=serviceA-down
add chain=prerouting in-interface=ether2 src-address-list=serviceB action=mark-packet new-packet-mark=serviceB-up
add chain=postrouting out-interface=ether2 dst-address-list=serviceB action=mark-packet new-packet-mark=serviceB-down
/queue type
add type=pcq name=serviceA-up pcq-rate=256000 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=10000
add type=pcq name=serviceA-down pcq-rate=512000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=10000
add type=pcq name=serviceB-up pcq-rate=512000 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=10000
add type=pcq name=serviceB-up pcq-rate=768000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=10000
/queue tree
add parent=global-in packet-mark=serviceA-up queue=serviceA-up max-limit=5000000
add parent=global-out packet-mark=serviceA-down queue=serviceA-down max-limit=10000000
add parent=global-in packet-mark=serviceB-up queue=serviceB-up max-limit=5000000
add parent=global-out packet-mark=serviceB-down queue=serviceB-down max-limit=10000000
Again, there are many different ways to go about this. The above will work for the criteria you have posted so far, though.

My apologies if I was too abrasive. As you can see I had actually edited that post before you had posted the reply.
Specific answers require specific questions. When in doubt, post the output of "/ip address print detail", "/ip route print detail", "/interface print detail", "/ip firewall export", and an accurate network diagram.
 
gunther01
newbie
Topic Author
Posts: 39
Joined: Sun Aug 01, 2010 7:00 pm

Re: Trying to set up PCQ

Sun Mar 13, 2011 7:13 am

I follow your example, and I think I am getting a grasp on how the packet flows work.. Am I correct to assume then, that the packet flow diagram stays stationary in the flow? You would just move the in and out interface accordingly to "show" the flow? So in the case of an outbound packet from a client, you would actually be looking at the right side of the flow chart moving to the left, and an inbound packet from the Internet side, you would move left to right across the flow?

Also, in looking at your example, it doesn't look like you have any connection marks. Is that not needed in the case of your example then? Or were the marks more for some other configuration with the double QOS that was used in the QOS video? I'm guessing it was a method to not have to specify interfaces in the Mangle rules and use parents instead??

I too am sorry for snapping back. I had seen you changed it after I posted.. I really have tried to grasp how MT works, and just can't quite "get" some of it. Clear examples are very helpful.

It seems to me at this point, my download set up's shouldn't even work. But they do for the most part.. So that has me even more confused in some aspects. This is part of why I can't "get" MT most of the time. It shouldn't work, but it does. But it must not be "right".
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Trying to set up PCQ

Sun Mar 13, 2011 7:25 am

The diagram is stationary - truly so, regardless of upload and download. A packet from the client to the Internet is processed left to right, with an input interface of the LAN interface, and an output interface of the WAN interface. A packet from the client to the Internet is processed left to right, with an input interface of the WAN interface, and an output interface of the LAN interface. A packet is a packet. All packets are processed exactly according to that diagram.

Connection marks are, strictly speaking, irrelevant. Queue trees fire based on packet marks, so all you need are packet marks. Connection marks can be recalled for every packet that is part of a connection, so it can be very efficient to have a complicated rule that applies a connection mark, and then have very simple rules that apply packet marks based on connection marks already present. Efficient here means: cheap as far as CPU usage goes. My example doesn't use them because it makes things more complicated, they are just a different way to arrive at the final goal: applying packet marks. Whenever you see connection marks and packet marks, you can also write it with just packet marks - it's just possibly going to be more efficient to use connection marks. In my specific example it's actually equally resource intensive to use either approach.
Specific answers require specific questions. When in doubt, post the output of "/ip address print detail", "/ip route print detail", "/interface print detail", "/ip firewall export", and an accurate network diagram.
 
gunther01
newbie
Topic Author
Posts: 39
Joined: Sun Aug 01, 2010 7:00 pm

Re: Trying to set up PCQ

Sun Mar 13, 2011 7:50 am

Does Masquerade happen in DST-NAT then? I don't see how packets are being marked using the Private Src-Natted IP's using forward then. As it shows, and looks pretty clear the Src-NAT is after forward. I'm getting tired I know, so I'd better leave this alone for the night. I just can't quite follow the flow for some reason. I understand what you said in your last post. I think you may have duplicated yourself and maybe I didn't get what you really meant in the first few lines though.
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Trying to set up PCQ

Sun Mar 13, 2011 8:30 am

Masquerade is source NAT. But source and destination NAT are named for the direction the initial translation goes in. Source NAT means a LAN client initiates a connection and its source address gets translated. Return traffic for the same connection still needs to get its destination address translated, and that happens in destination NAT. Two sides of the same coin.
Specific answers require specific questions. When in doubt, post the output of "/ip address print detail", "/ip route print detail", "/interface print detail", "/ip firewall export", and an accurate network diagram.

Who is online

Users browsing this forum: Baidu [Spider] and 61 guests