I am having problems with a simple queue to shape bandwidth in a SOHO environment using a hEX. Can provide more detail if required, but basics are:
– config pretty much default quickset for a PPPoE connection
– average speeds from ISP approx 1.8Mbps up / 15Mbps down
– no current queuing or any strange firewall rules
Currently we do constant uploading (large design files to dropbox etc) which saturates our connection and ruins browsing and downloads. As a solution I limit this manually on each machine (nothing to do with the router) to keep the total background uploads below 1Mbps and this works fine. However, when adding the following simple queue:
keep the simple queue and and mangle / tree rules to prioritise dns and other small packets like I’ve seen suggested
play around with pcq-limit and pcq-total-limit on the rule types, but honestly I’m not really sure what’s going on there
switch to mangle and queue tree
Wanted to see if there was any advice on this tho, as I am trying my hardest but I’ve got to admit I’m in over my head. I’d ideally like to keep things simple if possible, and also not add rates to the queue types as I’d like any single client to be able to get access to the max bandwidth.
Below is a screenshot to illustrate. This is the simple queue as described above. Between 3 1/2 and “2 min ago” is when an upload is hitting the max-limit and downloads are restricted. This is when browsing is almost not possible. After that upload stops browsing goes back to normal.
Screen Shot 2017-09-27 at 4.58.22 pm.png
I tried changing upload/download to sfq (wireless-default). See screenshot below. This is a constant upload at max-limit and periods of download, everything seems to work as expected. Browsing and downloading feels slightly slower than without any queueing at all, but at least it doesn’t completely lag when the upload is hitting the max-limit.
Screen Shot 2017-09-27 at 5.40.15 pm 2.png
Hope this all makes sense. I’d be happy to leave it at that, but I’d just like to know what is going on, since lots of forum answers and tutorials seem to suggest simple queues with pcq-upload/download as a good basic bandwidth management policy, but I have been struggling with this for days.
You could try using queue trees on the WAN interface with a simple configuration that gives priority to TCP ACK’s.
Make a mangle table rule in the prerouting & output chains:
1: protocol=tcp | packet Size: 0-123 | tcp flags=ack | action=mark packet | new packet mark=tcpack | passthrough=no (important to uncheck passthrough)
Then make a parent queue for the WAN interface (name=“upstream”) and set max-limit=1500k (whatever your upstream speed is - and it’s better to make this slightly smaller than the true value)
Then make two more queues:
name=wan-tcp-ack: parent=upstream max-limit=640k limit-at=640k packet-mark=tcpack
name=wan-data parent=upstream packet-mark=no-mark
This will effectively guarantee 640Kbps of bandwidth to TCP acks which should be enough. Whenever the network isn’t doing anything, you can test to make sure it’s enough for a full-speed download. Do a speed test and watch this queue during the download test. If it turns red and maxes out, then you need to increase the max-limit and limit-at values a bit until it’s granting enough for a full-speed download.
Your suggestions make sense. I have been trying to learn about trees and mangling, and thinking of trying a stripped back version of what alaskanjackal posted here:
For now using anything other than pcq-upload-default on the upload queue works fine. I’ve found a few other forums mentioning this issue, it seems specifically to be a problem with high asymmetric lines and pcq-upload and there doesn’t seem to be a solution besides changing the queue type, setting pcq-rate or possibly tinkering with burst:
All of that is fine, and maybe they are best practice I don’t know, but for beginners like myself this is a stumbling block and time sink, since something seems broken. Why do the other queue types work and not pcq-upload-default in this scenario. Is there something we are misunderstanding? Is pcq-upload something that shouldn’t be used on asymmetric lines, or low Mbit bandwidth lines?
To reiterate, as mentioned in other forum posts, no matter how high or low you set the upload max-limit on the simple queue compared to line-rate – eg. you could have line rate up 1500k, simple queue upload max-limit 200k – this issue will occur, and download seems to almost get shaped down to the upload limit.
I’d suspect that the problem is due to the fact that queue trees parented to the WAN interface will not see what they need to see in order to work as expected.
Consider that the default setup for pcq-upload-default uses src-IP as the determining factor for creating/using the dynamic sub-queues. Okay, that’s all well and good, but parented to the WAN interface, a queue tree is going to receive the packets after they’ve been all the way through the packet flow diagram - meaning all firewall tables and chains have done their jobs - notably in this case: srcnat.
What this means is that in most common configurations, the src IP will be the router’s WAN IP for all connections - not the various internal IP addresses. Thus the PCQ will only really be using a single sub-queue.
You could try changing the criteria to use ports instead of IP addresses, which would make a separate sub-queue for each and every outbound socket going through the router (as opposed to a single queue per device in the downstream direction).
If you were to mark packets with a special mark that only gets applied to upload packets, then you could parent the queue tree to the global HTB at which point you should still have the un-natted source IPs to work with - but that seems a hassle to me. It’s much simpler to just prioritize small TCP packets with the ACK flag set and have done when working with queue trees.
You could likely use simple queues parented to the global HTB, which may work better for you, and would almost certainly get the desired behavior using the pcq-upload-default queue type in the upstream direction settings.
Thanks again for the quick and thoughtful response. I am going to have to go through this and try to understand everything properly, however I doubt I will get there quickly.
Just to clarify, I am not using a queue tree parented to the wan interface, I am using a simple queue on the network served by the wan interface. Maybe these are essentially the same thing. My simple queue follows step three from the the PCQ example in the wiki below:
However I am not setting pcq-rate since I am not trying to allocate specific bandwidth to each user, just share the connection between all users. This is documented in the PCQ rate examples diagram in the wiki also:
What I am trying to do is documented, but basically doesn’t work, seemingly for asymmetrical connections.
Using a wireless-default for upload seems to work best for me right now. I can then use sfq-download-default on download, but find I am getting better performance just using wireless-default there as well. Completely untested tho, just watching the packet graphs and seeing how things feel.
Yeah - my terminology got a little bit crossed, but since you’re using simple queues, they’re parented to ‘global’ by default anyway, so I went and checked the packet flow diagram to make sure of something before making this reply.
simple queues happen at the end of the input path and at the end of the postrouting path (incidentally, so does the global HTB for queue trees). This means that if you’re doing srcnat that the addresses have been natted before the queue gets a chance to see the traffic, or at least that’s my read of the packet flow diagram. If that’s true, then the PCQ won’t have multiple addresses to use for splitting traffic flows into separate subqueues. I’m definitely not an expert on Mikrotik’s queueing, so I could be wrong here, but that’s my take.
I’d say the easiest thing to do would be to just mark the TCP ACK packets and give them their own queue to go through. The oreder of simple queues in the list is important, like with firewall rules, so make sure the TCP queue comes before the data queue. I’d also say that you only need to use pcq in the downstream direction on the data queue. Use type default-small for the rest.
Hey ZeroByte. I also ended up having problems with downloads maxing the connection, and realised what I wanted to achieve was probably a bit too complicated for one single simple queue. Keeping in mind that my goal was QOS in a SOHO environment on a crappy adsl2+ plan, I’ve ended up essentially following your suggestions, but gone a bit further. As I mentioned I might try earlier, I’ve moved to a mangle / queue tree approach based on alaskanjackal’s rules over here https://forum.mikrotik.com/viewtopic.php?f=13&t=73214&start=50 which seemed solid. Also props to pcunite for initial posts.
This seems to be working well so far (too early to really tell tho). All individual clients allowed to fight for max upload / download and overall browsing experience super snappy even when there are large uploads and downloads maxing each way. I have found I needed to max-limit my downloads to almost a third of the connection speed to achieve this tho. I still seem to gravitate towards the default sfq queues, they just seem to do what I want how I want them to. I think I don’t really understand pcq.
See rules below, I know this is off topic, but let me know if anything looks off. Was thinking of adding jump chains but since this is all working at 0-2% cpu load and is pretty simple I don’t see any need to optimise further.
The HTTP-BIG concept intrigues me. Thanks for posting this!
One comment - I notice that you have a lot of rules duplicated in postrouting.
I’m not quite sure why you’ve done this, and my guess would be so that connections to/from the router itself will also be marked properly?
If that’s the case, then I’d recommend moving those to “output” instead of “postrouting”
The possible flows are as follows in mangle:
prerouting → input
prerouting → forward → postrouting
output → postrouting
Note that using postrouting will cause forwarded traffic (the vast majority of your traffic) to follow both chains every time, while moving them to output would accomplish the same thing but optimize things so that forwarded traffic doesn’t get double-checked.
In fact, I think you could probably then move all of the action=mark-packet rules into the postrouting chain since you’re using interface-parented queue trees now, and those would catch marked connections regardless of whether they came via forwarding or via output.
Hey, sorry for the delayed response, I didn’t realise you had replied til now.
Regarding the duplicate rules, I am just following the setup @alaskanjackal posted, but organising things a bit differently just so it is easier for me to read and understand what is going on. He has comments in his script linking to forum posts explaining certain mangle rules.
I will look back over his post, and look further into your recommendations and see what I can do.
Regarding effectiveness so far, it has been working really well. I just needed to play round with all the limits to find the right balance.
Sorry for reviving this old topic, but I’d love to have your opinion. I have the same problem (upload saturated, all traffic jammed). My big problem is that my ISP (the only one here) is not very good. Sometimes upload speed is not constant at all so putting a low limit would waste a lot of bandwidth, and putting it very high near the theoretical maximum would make the queue to mess everything.
Isn’t there another way to get around this? Something that measures the actual speed every now and there?
My previous router (had a TP-Link with Gargoyle) had a great tool on QoS called ACC, Active Congestion Controller, that did this for download performance https://www.gargoyle-router.com/wiki/doku.php?id=qos