I ended up ditching Queue Tree/Advanced QoS like http://forum.mikrotik.com/t/using-routeros-to-qos-your-network-2020-edition/66683/1 completely.
I use Simple queues like the following:
/queue simple
add bucket-size=0.01/0.01 burst-time=1s/1s dst=pppoe-out1 max-limit=205M/205M name=ISP1-QoS priority=1/1 queue=default/default target=""
End result is, I can download with 30 threads, watch 8K videos, call, game all simultaneously without any buffering/packet loss.
Borrowed the following from the above guide though:
# default queue behavior
set default kind=sfq