Page 1 of 1

fq_codel or cake in v7

Posted: Thu Oct 24, 2019 6:55 am
by mke
Hi, just wanted to start another topic for this now that beta is out.

Will there be core or extra packages for modern queuing libraries like fq_codel or cake now that the kernel has been updated?

I've used both the above on an edgerouter x as a soho user with a highly asymmetrical line and they work flawlessly, simply set and forget.

The only way I could get a stable qos set up on routeros with queue trees and sfq was to tune my uploads to well below 50%. With fq_codel / cake close to 95% works no worries.

Re: fq_codel or cake in v7

Posted: Thu Oct 24, 2019 11:31 am
by nkourtzis
+1 for cake. It is more advanced than fq_codel + htb. But it will require more work, to convert the queues from htb to the cake internal shaper.

Re: fq_codel or cake in v7

Posted: Fri Oct 25, 2019 5:03 am
by santyx32
+1000 fq_codel/cake/fq_codel_fast needs less reserved bandwidth to deal with bufferbloat

Re: fq_codel or cake in v7

Posted: Sat Oct 26, 2019 2:13 am
by mducharme
I agree with this too, but first I want to see their current feature set stabilized. The sooner that happens, the sooner that they can release v7 and people can start using it in production. New features like this can be added easily later. If they try and add all new features that have been requested in the last several years into the first release of v7 it will take forever until it is released. I would rather see v7 stable release in 4 months than wait 7-8 months for v7 stable release with fq_codel or cake.

Re: fq_codel or cake in v7

Posted: Thu Nov 07, 2019 8:42 pm
by hci
+1

CoDel etc. are badly needed.

https://en.wikipedia.org/wiki/CoDel

Re: fq_codel or cake in v7

Posted: Mon Nov 18, 2019 9:42 pm
by brotherdust
These features are already present in the kernel they chose to use for v7. What needs to be done is to "plumb" it into RouterOS and Winbox. Actually, there's MANY features in newer kernels that overlap with old v6 features that Mikrotik had to implement themselves. Depending on whether they want to keep their custom implementation or not, I imagine much of the work in v7 is removing Mikrotik implementations and using native kernel features. I expect it cake and fq_codel will become available in time. =) Be patient! At least they have STARTED on v7.

Re: fq_codel or cake in v7

Posted: Wed Dec 18, 2019 7:26 pm
by Steveocee
+1 for FQ_Codel I really want this feature in RouterOS. It is probably one of the only reasons why I look outside of the MikroTik product range.

Re: fq_codel or cake in v7

Posted: Thu Dec 19, 2019 11:15 am
by skylark
Fq_codel implementation would be a great improvement. Of course we cannot promise anything about new features, but your requests are noted!

Re: fq_codel or cake in v7

Posted: Fri Dec 20, 2019 5:40 pm
by Chupaka
🤞

Re: fq_codel or cake in v7

Posted: Sun Dec 22, 2019 1:47 pm
by UpRunTech
Well, if you use PCQ you have some knobs to twiddle. I have improved ADSL2+ responsiveness with PCQ in simple queues by making the buffers smaller than the default (which otherwise seem to work well with 50+MBit speeds). For example on an ADSL2 annex m line with max-limit=1400k/16500k these queues keep web browsing snappy enough even when the upload (especially) and downloads are busy.

/queue type
add kind=pcq name=pcq-upload-adsl pcq-classifier=src-address,src-port \
pcq-limit=2KiB pcq-total-limit=1000KiB
add kind=pcq name=pcq-download-adsl pcq-classifier=dst-address,dst-port \
pcq-limit=20KiB

From what I can tell fq-codel has some insight (because it's in the kernel) into how quickly data is moving through each tracked connection and can adjust the buffers accordingly to mimimise the time they hang around. With PCQ you can adjust the buffer sizes but I can't think of a way you could even use a script to adjust them dynamically.

The adhoc way to set this up is:
  • Set the queue max-limits to 90-95% of your measured saturated bit rate (using speed test for instance and use the peak values you see in the traffic plots in Winbox not what Speedtest says).
  • Adjust your pcq-limits downwards incrementally until you notice the responsiveness or pings get acceptable when the upload and downloads are saturated.

Re: fq_codel or cake in v7

Posted: Sun Dec 22, 2019 3:59 pm
by ivicask
Well, if you use PCQ you have some knobs to twiddle. I have improved ADSL2+ responsiveness with PCQ in simple queues by making the buffers smaller than the default (which otherwise seem to work well with 50+MBit speeds). For example on an ADSL2 annex m line with max-limit=1400k/16500k these queues keep web browsing snappy enough even when the upload (especially) and downloads are busy.

/queue type
add kind=pcq name=pcq-upload-adsl pcq-classifier=src-address,src-port \
pcq-limit=2KiB pcq-total-limit=1000KiB
add kind=pcq name=pcq-download-adsl pcq-classifier=dst-address,dst-port \
pcq-limit=20KiB

From what I can tell fq-codel has some insight (because it's in the kernel) into how quickly data is moving through each tracked connection and can adjust the buffers accordingly to mimimise the time they hang around. With PCQ you can adjust the buffer sizes but I can't think of a way you could even use a script to adjust them dynamically.

The adhoc way to set this up is:
  • Set the queue max-limits to 90-95% of your measured saturated bit rate (using speed test for instance and use the peak values you see in the traffic plots in Winbox not what Speedtest says).
  • Adjust your pcq-limits downwards incrementally until you notice the responsiveness or pings get acceptable when the upload and downloads are saturated.
Sounds good, doesnt work.

Re: fq_codel or cake in v7

Posted: Sun Dec 29, 2019 2:25 pm
by OutcomeTech
+1 for fq_codel and/or CAKE in ROS v7, as soon as reasonably possible. Thank you for listening!

Re: fq_codel or cake in v7

Posted: Sun Jan 12, 2020 1:36 am
by UpRunTech
Sounds good, doesnt work.
Did you even try it? It the modified PCQ improved my ADSL2 connection when I had it, I am using it still to good effect one some sites stuck with ADSL2 and someone on IRC has just used it with success on a 3M/512K connection.

Re: fq_codel or cake in v7

Posted: Thu Jan 16, 2020 5:12 pm
by Binser
+1 for fq_codel/cake

Re: fq_codel or cake in v7

Posted: Thu Jan 16, 2020 9:05 pm
by Note
plus 1000 for all of that stuff that will make our life better. Mikrotik plz stop all other and focus on that........ only.

Re: fq_codel or cake in v7

Posted: Thu Jan 23, 2020 12:52 am
by PtDragon
I would love to see:
CoDel
FQ_CoDel
Cake
Pie
FQ_Pie
and every other complete implementation of queues.
Those will not take much space but will help a lot with keeping buffers low.

Re: fq_codel or cake in v7

Posted: Mon Jan 27, 2020 10:37 pm
by santyx32
Well, if you use PCQ you have some knobs to twiddle. I have improved ADSL2+ responsiveness with PCQ in simple queues by making the buffers smaller than the default (which otherwise seem to work well with 50+MBit speeds). For example on an ADSL2 annex m line with max-limit=1400k/16500k these queues keep web browsing snappy enough even when the upload (especially) and downloads are busy.

/queue type
add kind=pcq name=pcq-upload-adsl pcq-classifier=src-address,src-port \
pcq-limit=2KiB pcq-total-limit=1000KiB
add kind=pcq name=pcq-download-adsl pcq-classifier=dst-address,dst-port \
pcq-limit=20KiB

From what I can tell fq-codel has some insight (because it's in the kernel) into how quickly data is moving through each tracked connection and can adjust the buffers accordingly to mimimise the time they hang around. With PCQ you can adjust the buffer sizes but I can't think of a way you could even use a script to adjust them dynamically.

The adhoc way to set this up is:
  • Set the queue max-limits to 90-95% of your measured saturated bit rate (using speed test for instance and use the peak values you see in the traffic plots in Winbox not what Speedtest says).
  • Adjust your pcq-limits downwards incrementally until you notice the responsiveness or pings get acceptable when the upload and downloads are saturated.
Sounds good, doesnt work.
I use a PCQ queue tree to keep my bufferbloat under load between 20-100ms compared to 300-1000ms without it. In the past I used a DD-WRT router with CAKE and it kept the bufferbloat at 30ms losing only 3mbps compared to 8mbps using PCQ.

PD 1: I use queue tree instead of simple queues because I have multiple bridge interfaces for guests and IOT.
PD 2: I have 97/72mbps fiber but my ISP think bufferbloat is an imaginary problem that only exists in my mind and everything is fine on their side :lol: .

Re: fq_codel or cake in v7

Posted: Sat Feb 01, 2020 3:59 am
by dtaht
I do also keep hoping for fq_codel or cake in miktrotik. However, it's not an "or" choice so much, but an informed one.

Wifi: fq_codel for 3 wifi chipsets (mt76, ath9k, and ath10k) have existed now for a couple years ( https://lwn.net/Articles/705884/ ) however a key feature for the ath10k (and hopefully a few other chipsets we know of) ("airtime queue limits") only just landed in mainline after baking in google's wifi and chromebooks for a few years ( http://flent-newark.bufferbloat.net/~d/ ... erface.pdf ). This implementation lives in the mac80211 layer of the kernel,
and is not a qdisc, per se. It was our hope that the techniques we created and documented for this implementation would make it into vastly more wifi, 5g, and ethernet over powerline designs.

elsewhere:

fq_codel is lightweight and runs well at "line rate" on ethernet, wifi, - pretty much everything - , and there are few manufacturers that have offloaded it into the ethernet or wifi firmware itself at this point. fq_codel (rfc8290) is pretty much the default on most linuxes now, and also is on all of apples products. Some form of restricting the buffering the lowest level firmware (as per linux's BQL or now, AQL) is also needed, and more than a few manufacturers and users have turned fq_codel on with hundreds of ms of buffering still stuck in the driver itself, expecting a better result. BQL is bog-standard in linux now, supporting - I've lost track - 50+? ethernet drivers - but not necessarily the ones mikrotik uses. Adding BQL to an ethernet driver is about 6 lines of code....

as for fq_pie, pie, codel, etc. In general we don't recommend codel be run standalone. If you want to construct a sfq + codel thing, ok... but seriously, use fq_codel even with that.

I'm not a one trick pony - I'd like there to be a low latency high speed and bufferbloat-free internet for everyone - I've long supported development of pie, also. It is arguably (so long as ecn is disabled) a better single queue AQM than codel is. pie is lighter weight than fq_codel is, also, but doesn't achieve anywhere near the same queue depth (16ms) that codel can (5ms). Pie keeps improving, with lots of new stuff landing in more recent kernels, including fq_pie, which just landed in linux 5.5. Some manufacturers (cough, cablemodems) have found it difficult to implement fq_codel in their devices, and hopefully fq_pie will prove a viable alternative. I'd really like some independent benchmarking of fq_pie vs cake and fq_codel, not just at line rate but in shaped modes. Cake in particular has docsis framing support, which the cable industry has thus far tried to ignore....

as for cake... well, after 5 years of user driven development, it was mainlined into linux 4.19, and the out of tree build for it goes back as far as 3.10. By default cake is about 2.5x more cpu intensive than fq_codel is, but it does a lot more - host + flow fairness, even through nat, ack-filtering, a better codel-like algorithm, etc. I like to think it's currently the gold standard for sqm-software shaping, but again, independent benchmarks would be just great. cake is also our research vehicle for even more improvements in sqm designs. Try it at line rate as a shaper if you can spare the cpu, but be prepared to fall back to a simpler htb + fq_codel shaper if you can't. Cake's biggest objective was to be easy to configure, and only takes one line of tc code to enable on egress, 3 on ingress, unlike the sqm-scripts or more complicated things.... and some more details can be had at the paper, here: https://arxiv.org/pdf/1804.07617.pdf

I think all these algorithms would be of value to mikrotik's customer base and the bufferbloat.net folk and mailing lists are always willing to help. Let me leave you with a laugh though, in my recent talk at linux.conf.au I used people, as packets, to try and explain tcp's behaviors better and the value of these algorthms.

https://blog.apnic.net/2020/01/22/buffe ... -over-yet/

somewhere on these threads fq_codel_fast was mentioned... that's an experimental version, with experimental "SCE" ( https://tools.ietf.org/html/draft-morton-tsvwg-sce-01 )support, and I would not be in favor of mikrotik shipping that. There is a competing standards attempt, called L4S, that works differently, and I really don't know where that's going to go at this point. fq_codel_fast without that feature does benchmark out at about 5% faster than fq_codel presently does, but that's pretty trivial in the scheme of things.

In summary - ship fq_codel for wifi, make sure you have BQL and AQL support underneath, ship fq_codel and cake, and feel free to play with fq_pie and pie. I look forward verymuch to playing with mikrotik's implementations one day soon.

Re: fq_codel or cake in v7

Posted: Sat Feb 01, 2020 5:36 am
by ivantirado
This is the way.

Re: fq_codel or cake in v7

Posted: Wed Feb 05, 2020 4:09 pm
by IPANetEngineer
This is the way.
Indeed :lol:

It would be nice to be able to run either fq_codel or cake in RouterOS for better shaping options. Please consider adding this MikroTik.

Re: fq_codel or cake in v7

Posted: Wed Feb 12, 2020 10:39 am
by HzMeister
Cake would be awesome in v7. I just did some testing with openwrt and it performed surprisingly well.

When mikrotik implements cake, hopefully they include an option for "per host fairness" (like in openwrt) which prevents any one client from hogging bandwidth.

edit: after some more time with cake on openwrt I'm considering going back to my custom queue tree on mikrotik. Don't get me wrong, cake is THE go-to solution for 99% of people who want great performance without a lot of skill or effort, but I prefer the more granular control of mangle and queue tree.

Re: fq_codel or cake in v7

Posted: Mon Feb 17, 2020 11:17 am
by mke
Wow everyone dtaht is here, and what an amazing response!!!

If you don't know now you know... https://github.com/dtaht

Please Tik admins read the above and reach out!

And Dave, thanks for so much for all your efforts.

Re: fq_codel or cake in v7

Posted: Wed Mar 25, 2020 9:50 am
by Steveocee
Gentle nudge.
I need a new router, choice is 4011 or ER4, one has some features I need and the other has SQM. Please make my decision easier!

Re: fq_codel or cake in v7

Posted: Thu Mar 26, 2020 10:16 pm
by chrismal
Gentle nudge.
I need a new router, choice is 4011 or ER4, one has some features I need and the other has SQM. Please make my decision easier!
I have the same situation.
Another Vote for SQM

Re: fq_codel or cake in v7

Posted: Sun Mar 29, 2020 4:11 am
by i4ko
-1
I have never had good experience with SQM as implemented in openwrt or ubnt ER. Practically most of the time SQM does hurt performance of TCP connections significantly. Mostly it does is introduce packet loss, and a lot of it. Now, the traffic I deal with is idiotic - sub-second bursts in the order of 12-25mb, then 8-9 seconds idle and then such a burst again. The receiving and sending devices are also idiotic, closed source systems that insist on keeping receive window at 16kb.
What works best is a poor man tcp optimization (traffic is passed via the router proxies) thus making the idiotic small receive window size only a problem in the last segment. The proxy in mtk is severely limited so a better proxy or a real tcp optimization engine (and ability to do automatic optimization in certain cases based on firewall rules will be great) is needed much more than SQM. If you set your PCQ properly you don't have problems. This is the setup that works best for me:
1. use a queue tree for the interface outgoing traffic
1. set a top PCQ to do all 4 criteria - dst-address, dst-port, src-address, src-port and mask at /32 for ipv4
2. calculate bandwidth delay product for 5ms at data rate and set as pcq-limit (.e.g 25mbps the value is 15)
3. calculate bandwidth delay product for 5ms at line rate and set as pcq-total-limit (e.g. 100mbps the value is 61) - the default of 2000 is way off
4. set max-limit to your data rate or 95% of it, though usually data rate is enough
5. use subqueues if needed but in this case the pcq-total-limit is calculated for the bandwidth of the outer queue data rate assumed as line rate for the inner, each inner subqueue needs its own pcq
6. do this as close to the source of traffic as possible

Re: fq_codel or cake in v7

Posted: Sun Apr 05, 2020 7:21 pm
by buraglio
Seconded. This would be a welcomed addition for many. Please consider it.
This is the way.
Indeed :lol:

It would be nice to be able to run either fq_codel or cake in RouterOS for better shaping options. Please consider adding this MikroTik.

Re: fq_codel or cake in v7

Posted: Thu Apr 30, 2020 8:21 pm
by santyx32
CAKE on DD-WRT worked much better than Mikrotik's SFQ queue tree but it's too unstable for home networking (wifi crashes, wan drops, etc), consider adding CAKE queue type instead of FQ-Codel since it provides the lowest fluctuations during heavy load, http://www.dslreports.com/speedtest/63079766 this is what I get with MK solution but the latency spikes up to 300ms while CAKE consistently mantained it under 130ms giving me A+ ratings.

Edit: this is my current config
arbol_colas.png

Re: fq_codel or cake in v7

Posted: Thu Jun 04, 2020 4:00 pm
by santyx32
ROS 7 is now running over Linux 5.6.3 kernel, so I guess it's the time for real SQM like CAKE or fq-codel to be available in ROS