I found an old thread (http://forum.mikrotik.com/t/fasttrack-friendly-qos-script/102401/1) that talks about using QoS with FastTrack enabled. That got me interested. After reading the thread and also reviewing packet flow documentation, I came to realize I can successfully use this approach with fq_codel.
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue tree
add bucket-size=0.01 max-limit=118M name=download packet-mark=no-mark parent=bridge1 queue=fq-codel
add bucket-size=0.01 max-limit=11M name=upload packet-mark=no-mark parent=ether1 queue=fq-codel
The key are interface queues that work along with FastTrack. Because interface-attached HTB works only for egress, I put my download queue on the bridge interface (BTW, I have two VLANs on top of it), so the queuing happens when Internet traffic leaves the router in LAN direction. The upload queue is on WAN ether1.
I then ran some tests and got the same good bufferbloat results as with a simple queue. Here is one sample: https://www.waveform.com/tools/bufferbloat?test-id=25e27c79-ac1a-42af-a700-4fb4aa8dc505. And CPU usage was much lower. It wasn’t too bad even before for my 120M/12M Internet connection, but now it got me thinking - how well would hAP ac2 handle a much faster WAN connection?
I pulled out a spare router, reset to all defaults, and ran some tests. I used iperf3 with 20 threads and 60 seconds run to simulate LAN to WAN heavy traffic. Client PC is on LAN side of hAP ac2, server PC is on WAN.
Same fq_codel configuration as above, only max-limit set to 1020M for both download and upload. Simulating a gigabit Internet service.
With FastTrack disabled, two CPU cores are maxed out at 100%, iperf reported ~560Mbps.
With FastTrack enabled, only one core reached 90-95%, overall utilization ~50%. Iperf speeds were 915-924Mbps. FastTrack makes a huge difference indeed.
I also ran continuous pings from client to server to see fq_codel working for the lack of a better test. The difference was also huge:
1ms unloaded.
With fq_codel average ping 3ms, 15ms max.
Without fq_codel (queues disabled) average ping 13ms, max 48ms. Fq_codel makes a difference even on such a fast connection.
It’s a big discovery for me. MikroTik’s clever solution - FastTrack - turns out to be even better than HW NAT in my case as every SOHO router with HW NAT I’ve seen can’t do SQM with it enabled. If you search online for best routers with SQM for gigabit Internet service, you will find most recommendations are around x86 or some other beefy solutions. Unless my test was flawed somehow, you can actually get pretty close to a gigabit with fq_codel on an inexpensive hAP ac2. Granted, I don’t have those kind of speeds, but this will certainly be on my list of recommendations for others.
It also means older low-performance MikroTik devices can do pretty well with fq_codel thanks to FastTrack. And even if you have a more powerful device like RB5009, the option to save some processing power is there. With container support coming up soon, there is always going to be a way to put that extra CPU to good use.
I want to point out that I don’t use WiFi on hAP ac2, which would definitely consume a good chunk of CPU.