No offense taken, it is a common issue with QoS where some traffic does not get handled and bypasses the queueing. This is part of the reason why I generally prefer interface-attached HTB, because then with a no-mark option, and all marks accounted for, you can be sure that you have actually properly assigned a queue for all potential traffic.
BTW, these interface attached queue trees (HTB) similarly catch MPLS as no-mark and therefore can do queueing of MPLS packets. (and, there are also ways to mark specific MPLS packets so that you can do full blown MPLS QoS)
BTW, is there a way of fixing the MikroTik wiki page on FastTrack (http://wiki.mikrotik.com/wiki/Manual:IP/Fasttrack) to reflect this behavior? Paragraph 2 on the page has the correct information:
Note that not all packets in a connection can be fasttracked, so it is likely to see some packets going through slow path even though connection is marked for fasttrack. This is the reason why fasttrack-connection is usually followed be identical action=accept rule. Fasttracked packets bypass firewall, connection tracking, simple queues, queue tree with parent=global, ip traffic-flow(restriction removed in 6.33), ip accounting, ipsec, hotspot universal client, vrf assignment, so it is up to administrator to make sure fasttrack does not interfere with other configuration;
The above quote from the wiki is completely accurate because it does not say that Fasttracked packets bypass queue tree with an interface as the parent (and they do not); however, later on the same page, there is a misleading warning:
Warning: Queues, firewall filter and mangle rules will not be applied for FastTracked traffic.
This not completely true due to the exception of interface-attached queue trees, and this warning is likely the reason why people have the misunderstanding that it is not possible to queue FastTrack’ed packets other than the regular hardware queue that everything goes through.
Yeah, I think that’s what was in my mind all along. I don’t think the forums moderators have any editorial power on the wiki - so perhaps a Mikrotik admin will see this post and update it to something like this:
Warning: Queues (except queue trees parented to interfaces), firewall filter and mangle rules will not be applied for FastTracked traffic.
The Packet flow diagram v6 certainly depicts the interface HTB as the final step prior to egress, but it doesn’t really show where fastpath fits into the flow… so perhaps that should be included in the updates.
At that thread, there is someone (IntrusDave) suggest using DSCP, and I found yours (I see your script based on IntrusDave’s QoS script).
If I’m using your script, Can I get same result ?
Yes you can with my script, you just have to add the rules to do this. I imagine if it is HTTP download, you’ll probably want to reduce it to ‘background’, so you would probably give it a DSCP value of 16. First you should back up your config, and then remove the existing queues and QoS rules created by the existing script so they don’t conflict, before installing mine. Then add the rule to de-prioritize HTTP download to ‘background’ class (ip precedence 2).
The idea of my script is to use existing DSCP on packets in a fasttrack-friendly way that still lets you keep your fasttrack rules. I don’t include rules to apply DSCP to packets that don’t have them because that will really depend on the user, each person may have different priorities in mind. Also, you need to bypass fasttrack for any packets that you need to apply DSCP tags to, so you lose more and more fasttrack benefits as you apply more and more DSCP tags. That is another reason why I do not include an ‘HTTP Download’ rule and things like that. A lot of QoS scripts will put in many rules that users do not really need, to prioritize things that they do not even use. Mine is kind of a ‘base setup’ that you add your own rules to based on the particular things that you either want to prioritize or de-prioritize.
I kind of see this happening in two different parts
The nuts and bolts of the QoS being provided by the QoS engine, which is my script
A ‘rule bank’ for applying DSCP tags to different types of traffic; users would pick rules from the ‘rule bank’ depending on what they needed and add them to the QoS system
My script is really the engine; there is no ‘rule bank’ currently, except you can pick and choose rules from other QoS scripts and apply them here, by placing them above the existing QoS rules and applying DSCP, and bypassing fasttrack for that same traffic so that the mangle rules work.
Very informative so thank you for that. I am confused though when you say that FastTrack bypasses mangle rules yet below you say that the traffic is still queued and rate-limited. So in that case how would I be able to mark the traffic that I want to prioritize?
Sorry, I missed this before. In order to mark the traffic you want to prioritize, you must create firewall rules that ‘accept’ this specific traffic that you want to mark before it hits the fasttrack rule, then your mangle will work because you will be effectively bypassing fasttrack for those packets only. Any traffic that hits the fasttrack rule will skip mangle and will therefore be treated as best-effort; any traffic that you either want to prioritize above best effort or de-prioritize below best effort must be mangled, and in order to mangle it, you must prevent this traffic from hitting the fasttrack rule by accepting it before it reaches that rule.
The first post includes a rule that automatically bypasses fasttrack if DSCP is non-zero, so for packets that are already tagged with DSCP, this will take care of all of them. You only need to create additional fasttrack bypass rules if there are packets that do not have DSCP applied that you either want to apply DSCP to, or simply apply the appropriate packet mark.
I would expect SFQ to be slightly more efficient in terms of CPU usage. SFQ is like PCQ but without the configuration options, if you are not going to tweak those, I believe there is little reason to use PCQ over SFQ.
Thanks for your help with this. I’ve gotten this to work pretty well but I’m having an issue where I’m trying to mark some packets as precedence 7 but they are going through as 6. I have listed what I have added to see if you can help determine why. I can see it in the log when I log 6 and 7 mangle rules.
Perhaps they are getting caught by the “prioritize ACKs” mangle rule?
I have my mangle rules above that one. I just looked and now I’m not getting any packets marked as “7” i must have something wrong somewhere. Maybe shouldn’t be postrouting?
I take that back. they won’t actually show up in the precedence 7 because I’m marking them earlier. however I do still have traffice from that ip showing up in precedence 6. So not sure why for that.
Mangle rule 19 above is already included with my script, I just included it so that you could see the relative positioning of my prioritization rules.
I mark the DSCP tags instead of directly marking the precedence so that I can use this tag on additional internal routers if I had them, but I could just as easily mark packet as ip_precedence_4 with the mangle rule listed as 18, then I wouldn’t really need the passthrough=yes on that setting (that approach is closer to what you are doing).
Also, look in the “Connections” tab of IP->Firewall to make sure the connection in question is getting marked properly. You should see it in there with the connection mark you are applying.
As you can see, it works for me and makes certain that I can browse the web on my PC without interfering with NetFlix viewing:
I’ll take a look at the connections tab. I hadn’t really looked at that yet. the traffic is going up on the firewall rule. thanks for your help. I’m going to look through your example to understand better.
That is going to be an ongoing problem. You might be able to work around it by using a DNS-based address list.
(I don’t know how Netflix uses addresses and DNS)
I have corrected the Wiki
It now states:
“Queues (except Queue Trees parented to interfaces), firewall filter and mangle rules will not be applied for FastTracked traffic”
