Community discussions

MikroTik App
 
User avatar
sergejs
MikroTik Support
MikroTik Support
Topic Author
Posts: 6694
Joined: Thu Mar 31, 2005 3:33 pm
Location: Riga, Latvia
Contact:

Queue bucket-size option explained

Thu May 12, 2016 1:41 pm

We have made documentation for the new Token Bucket option, it is new exciting and powerful feature for queues and bandwidth limitation. Detailed information and configuration example are here,
http://wiki.mikrotik.com/wiki/Manual:HT ... _Algorithm


Short diagram explaining Tocken Bucket algoritm,
Image
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2095
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Queue bucket-size option explained

Thu May 12, 2016 2:00 pm

Very interesting.

Thanks Mikrotik.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1764
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Queue bucket-size option explained

Fri May 13, 2016 11:22 am

After reading this i played around my test setup and realized that there setup "large parent queue bucket size, small child queue bucket size" are beneficial in situations when you have many child queue - now child queue that just started to use traffic in already saturated environment have much easier time to get initial traffic.

In short - queues got more responsive to usage changes, cause parent queue have an extra tokens for "unexpected expenses"

Also, i'm starting to thing old burst is starting to become redundant, this implementation with just one option is easier that old 3 option solution anyways.
 
freemannnn
Forum Veteran
Forum Veteran
Posts: 700
Joined: Sun Oct 13, 2013 7:29 pm

Re: Queue bucket-size option explained

Wed May 25, 2016 8:29 pm

Large Queue Bucket
Lets try to apply the same logic to a situation when bucket-size is at its maximal value:

/queue tree
add name=download parent=Local packet-mark=PC1-traffic max-limit=10M bucket-size=10
add name=upload parent=Public packet-mark=PC1-traffic max-limit=10M bucket-size=10
In this case bucket-size=10, so bucket-capacity= 10 x 10M = 100M

If the bucket is full (that is, client was not using full capacity of the queue for some time), the next 100Mb of traffic can pass through the queue at an unrestricted speed.

So you can have:

20Mbps transfer speed for 10s
60Mbps transfer burst for 2s
1Gbps transfer burst for approximately 100ms



i cant understand how seconds are calculated in the above example. why is 10s for 20mbps?
if 100mb traffic can pass through at an unrestricted speed, why is 20mbps for 10s?
is there an excel formula so we calculate easy how many mb or sec will be the burst result?
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: Queue bucket-size option explained

Wed Jun 01, 2016 6:08 pm

i cant understand how seconds are calculated in the above example. why is 10s for 20mbps?
if 100mb traffic can pass through at an unrestricted speed, why is 20mbps for 10s?
is there an excel formula so we calculate easy how many mb or sec will be the burst result?
Because in 10 seconds, you will have earned another 100M of tokens.

It only takes 5sec @ 20M/s to spend 100M of tokens....
BUT during that 5 sec, you also earned 50M more tokens....
That 50M takes 2.5 sec to spend, while earning another 25M of tokens...
etc...

If you want a formula, it's basically this:

Time to empty/full = absval(Bucket Size * Token Rate / (Data Rate - Token Rate))

Time means "fill from empty" if the data rate is below the token rate, and it means "empty from full" if data rate is greater than token rate.
If they're the same, it's divide by zero, so avoid that. ;)

Obviously this formula assumes a silly thing - a constant data rate.

Ultimately, though, it's not right to think in those terms - data rates are almost never constant, and it's impossible to predict whether a stream will "save up" its tokens or use them constantly, etc.
 
wispwest
Member
Member
Posts: 479
Joined: Tue May 19, 2009 3:48 am

Re: Queue bucket-size option explained

Fri Jun 17, 2016 7:17 pm

We are using Simple Queues for each customer, some Mikrotiks have 400-500 of them. We enter the customer name, IP address, then select a PCQ on the advanced tab to select the Download/Upload rate, the PCQ's have a burst timer where they can for example go to 7Mb for 40 minutes, then drops to around 1/2 the burst rate, around 3-4Mb/s. 

After upgrading firmware's we're noticing now the 'bucket size', which all are defaulted to 0.100, do we need to change anything to keep everything running 100% the same, or is 0.100 default not have any change on our existing setup?
 
User avatar
Cha0s
Forum Guru
Forum Guru
Posts: 1135
Joined: Tue Oct 11, 2005 4:53 pm

Re: Queue bucket-size option explained

Fri Jun 17, 2016 7:24 pm

http://wiki.mikrotik.com/wiki/Manual:HT ... diagram.29
bucket size (0..10, Default:0.1) - queue option was added in RouterOS v6.35, before that it was hard-coded to a value of "0.1".
As shown in the documentation, prior to v6.35, it was hard-coded to 0.1.
So after upgrade it shouldn't behave differently since it keeps the default value: 0.1.
 
Note
Frequent Visitor
Frequent Visitor
Posts: 78
Joined: Fri Jun 03, 2016 12:39 pm

Re: Queue bucket-size option explained

Wed Sep 28, 2016 11:44 am

Can u plz give me an example on how to use it on an 20/1 adsl connection and setting max limit to 10 in 1 of 8 queues in simple queue? What numbers offers me what?

ty in advance
 
Note
Frequent Visitor
Frequent Visitor
Posts: 78
Joined: Fri Jun 03, 2016 12:39 pm

Re: Queue bucket-size option explained

Fri Sep 30, 2016 12:17 pm

anyone not yet? good job.

I suspect that this feature is not working and u just shame to admit it. it is not bad if is not working. just someone write something, for us to stop waste our time.
 
damianivereigh
just joined
Posts: 15
Joined: Sun May 22, 2016 1:06 am

Re: Queue bucket-size option explained

Wed Oct 12, 2016 10:11 am

So given the above, if we want to have a setup with almost no burstiness (we have a downstream network that polices the traffic at 10ms intervals!), really I need the bucket size to be only big enough for one packet (1500 bytes). So at 100Mb would be a bucket size of 0.00012 (0.00012*10000000/8 = 1500). Pity you can't specify the size directly (in bytes).

Is that right?
 
damianivereigh
just joined
Posts: 15
Joined: Sun May 22, 2016 1:06 am

Re: Queue bucket-size option explained

Wed Oct 12, 2016 11:27 pm

I have just realised what this parameter actually is. It is really "bucket-time" - the amount of time it will take to fill the bucket up from empty when there is no packet flow. The number of seconds of token flow (which is set by the max-limit/limit-at etc) that can be stored in there (after which the tokens overflow). It also appears to have a lower limit of 0.001 - i.e. 1ms, which is good enough for what we want.
 
User avatar
omidkosari
Trainer
Trainer
Posts: 640
Joined: Fri Sep 01, 2006 4:18 pm
Location: Canada, Toronto

Re: Queue bucket-size option explained

Sun Oct 16, 2016 3:37 pm

Is there any benefit playing with Queue bucket-size to overcome DOS/DDOS and queuing 100% cpu usage ? If yes how ?
 
JulieHardy
just joined
Posts: 3
Joined: Thu Jan 05, 2017 10:25 am

Re: Queue bucket-size option explained

Thu Jan 05, 2017 10:35 am

Time means "fill from empty" if the data rate is below the token rate, and it means "empty from full" if data rate is greater than token rate.
 
Note
Frequent Visitor
Frequent Visitor
Posts: 78
Joined: Fri Jun 03, 2016 12:39 pm

Re: Queue bucket-size option explained

Tue May 16, 2017 12:40 pm

does bucket size work to unlimited queues? Or i have to set limits? if not, it will very good feature to work under unlimited queues........ i think.

set burst time only, unlimited the other values and borrow buckets from parent for ex.

does this work?
 
ajack46
newbie
Posts: 37
Joined: Tue Mar 28, 2017 9:08 am

Re: Queue bucket-size option explained

Tue Jun 20, 2017 12:21 pm

does bucket size work to unlimited queues? Or i have to set limits? if not, it will very good feature to work under unlimited queues........ i think.

set burst time only, unlimited the other values and borrow buckets from parent for ex.

does this work?
Does not the diagram say it has a burst limit and a burst limit at max, meaning they are giving out a range. And not something unlimited?
 
darkbreaker
just joined
Posts: 6
Joined: Thu Sep 14, 2017 6:22 pm

Re: Queue bucket-size option explained

Thu Sep 14, 2017 7:20 pm

Hi all!
I'm a newbie here, so I'll try to explain my problem in ROS therminology as close as I can. My problem is "a ghost" property of queue tree/simple queues. I actually spent 12+ hours yesterday reading WIKI, manual, example scripts etc. but I couldn't find a way to do a realy simple thing: how to set the "limit-at" value be higher then "max-limit" :-) Can anybody here enlight me so I can say : hey, i'm stupid, because otherwise my so far good opinion of ROS programmers will be very degraded. This post here is mainly reason to join forum, and here is what I don't like: this pic below. In this algorthm, <limit-at token rate> will never be engaged , or I'm blind. You tell me...
[admin@GoranmarGW] > /queue tree add name=test_queue max-limit=10M limit-at=20M parent=global
failure: -max-limit less than -limit
[admin@GoranmarGW] > 
And Mikrotik says:
We have made documentation for the new Token Bucket option, it is new exciting and powerful feature for queues and bandwidth limitation. Detailed information and configuration example are here,
http://wiki.mikrotik.com/wiki/Manual:HT ... _Algorithm

Short diagram explaining Tocken Bucket algoritm,
Image
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Queue bucket-size option explained

Fri Sep 15, 2017 10:12 am

Indeed, looks like 'Limit-at larger than max-limit?' is always 'no' :o
 
User avatar
strods
MikroTik Support
MikroTik Support
Posts: 1616
Joined: Wed Jul 16, 2014 7:22 am
Location: Riga, Latvia

Re: Queue bucket-size option explained

Fri Sep 15, 2017 10:16 am

Why do you want to set limit-at higher than max-limit?

Max limit - Max bandwidth client can use;
Limit at - Is the minimum bandwidth which should be available for client if there are multiple connections from different clients at the same time.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Queue bucket-size option explained

Fri Sep 15, 2017 11:35 am

But why is this case on the diagram then?
 
darkbreaker
just joined
Posts: 6
Joined: Thu Sep 14, 2017 6:22 pm

Re: Queue bucket-size option explained

Fri Sep 15, 2017 1:10 pm

Why do you want to set limit-at higher than max-limit?

Max limit - Max bandwidth client can use;
Limit at - Is the minimum bandwidth which should be available for client if there are multiple connections from different clients at the same time.
I dont ! My point is HUGE error in logic of the algorithm. But the irony is even better, since queues are behaving almost as digram shows. In my 10+ years experience with ROS, limit-at value never had definitive and predictable effect on queue beheviour. It seems that what ever I wanted to acomplish with queues, the best way was trial/error method, since documentation like this above is not of help. Sadly...
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Queue bucket-size option explained

Fri Mar 16, 2018 7:00 pm

Hi Dark breaker, imagine I cannot find clear basic descriptions of how to program simple queues, am not able to grasp Burst, through the available documentation and then I come across this thread TokenS. What is going to burst is my noggin LOL...... All to say is I echo your sentiments on the documentation being lacking. I know much of it is my lack of experience and knowledge but I can read..........
 
blk2001
just joined
Posts: 6
Joined: Tue Mar 27, 2018 8:20 am

Re: Queue bucket-size option explained

Tue Apr 10, 2018 8:04 am

So i have a question here: how bucket size will affect on accuracy of guarantees (such as limit-at and priority) for flow control? Will a larger bucket size damage the guarantees for higher priority queue due to token buffering of lower priority queue? Consider this example:

There are two queues: one for VoIP which needs a constant rate 1M, second for others which has rate 0-9M, the total bandwidth is 10M (suppose upper ISP dosen't provide buffering or burst), here we only consider upload.

/queue tree
add name=upload parent=Public max-limit=10M bucket-size=1
add name=voipup parent=upload packet-mark=voip priority=1 limit-at=1M max-limit=1M bucket-size=1
add name=othersup parent=upload packet-mark=others priority=8 limit-at=0 max-limit=9M bucket-size=1

1 second: queue voipup used 1M, queue othersup is idle, and its bucket filled with 9M tokens
2 second: queue voipup get 1M tokens, queue othersup get another new 9M tokens, so if othersup begins full speed upload, it will have 18M tokens, and then there are total 19M tokens but only 10M limit upload rate, total 19M data will be passed to ISP, they may be dropped or queued by ISP router, if they have a fair competition, voipup may lose nearly half of its guaranteed rate.

if we set bucket-size=0.1, othersup will have 9.9M tokens at 2 second, there are still some effects but much slighter.
 
carlabreto2
just joined
Posts: 3
Joined: Mon Apr 27, 2020 9:55 am

Re: Queue bucket-size option explained

Mon Apr 27, 2020 10:04 am

Excellent explanation, I have learned it perfectly. Thank you Mikrotik
 
nagylzs
Member
Member
Posts: 353
Joined: Sun May 26, 2019 2:08 pm

Re: Queue bucket-size option explained

Sun Dec 13, 2020 9:43 pm

We have made documentation for the new Token Bucket option, it is new exciting and powerful feature for queues and bandwidth limitation. Detailed information and configuration example are here,
http://wiki.mikrotik.com/wiki/Manual:HT ... _Algorithm
Can I suggest some changes to that wiki?

This line:

> If no, the packets stays at the start of packet waiting queue until the appropriate amount of tokens are available.

Should be spelled like this:

> If not, packets stay at the start of packet waiting queue until the appropriate amount of tokens are available.

At the mangle example, we should add "connection-state:new" condition. E.g.
/ip firewall mangle
add chain=forward action=mark-connection connection-mark=no-mark src-address=192.168.88.101 connection-state=new new-connection-mark=pc1_conn
add chain=forward action=mark-connection connection-mark=no-mark dst-address=192.168.88.101 connection-state=new new-connection-mark=pc1_conn
add chain=forward action=mark-packet connection-mark=pc1_conn new-packet-mark=pc1_traffic
I know that this is just an example, but I feel it is missing.

Great wiki by the way. It helped me to understand queue trees much better.
 
n21roadie
Forum Guru
Forum Guru
Posts: 1949
Joined: Fri Aug 07, 2009 10:36 pm
Location: Limerick,Ireland

Re: Queue bucket-size option explained

Thu Dec 17, 2020 5:08 pm

On the mange can in + out interfaces be used in place of src + dst address when PPPoE is used and both ethernet and wlan interfaces ports are bridged
 
HowardGutierrez
just joined
Posts: 1
Joined: Sat Feb 13, 2021 2:34 am

Re: Queue bucket-size option explained

Sat Feb 13, 2021 3:25 am

So i have a question here: how bucket size will affect on accuracy of guarantees (such as limit-at and priority) for flow control? Will a larger bucket size damage the guarantees for higher priority queue due to token buffering of lower priority queue? Consider this example:

There are two queues: one for VoIP which needs a constant rate 1M, second for others which has rate 0-9M, the total bandwidth is 10M (suppose upper ISP dosen't provide buffering or burst), here we only consider upload.

/queue tree
add name=upload parent=Public max-limit=10M bucket-size=1
add name=voipup parent=upload packet-mark=voip priority=1 limit-at=1M max-limit=1M bucket-size=1
add name=othersup parent=upload packet-mark=others priority=8 limit-at=0 max-limit=9M bucket-size=1

1 second: queue voipup used 1M, queue othersup is idle, and its bucket filled with 9M tokens
2 second: queue voipup get 1M tokens, queue othersup get another new 9M tokens of vlone x off white, so if othersup begins full speed upload, it will have 18M tokens, and then there are total 19M tokens but only 10M limit upload rate, total 19M data will be passed to ISP, they may be dropped or queued by ISP router, if they have a fair competition, voipup may lose nearly half of its guaranteed rate.

if we set bucket-size=0.1, othersup will have 9.9M tokens at 2 second, there are still some effects but much slighter.
I am also thinking the same question that how will we get to manage this?
 
dzo
newbie
Posts: 27
Joined: Mon Feb 04, 2019 2:53 pm

Re: Queue bucket-size option explained

Mon Aug 23, 2021 2:32 pm

We have made documentation for the new Token Bucket option, it is new exciting and powerful feature for queues and bandwidth limitation. Detailed information and configuration example are here,
http://wiki.mikrotik.com/wiki/Manual:HT ... _Algorithm


Short diagram explaining Tocken Bucket algoritm,
Image
I made some tests for me, and it looks like it would work not as expected:
parent queue.jpg
The tokens (extra-tokens because of limit-at) produced in my queue02 should be given as "parent queue token bucket" to my children queue03 and queue04 as the arrow down to the children-queue shows. But it doesn't! Queue04 and queue05 are limited by their max-limit, no tokens borrowed from the parent-queue. Bucket-size*max-limit as written on the childs token-bucket does not effect my test, the same with token-buket 9.100 or 0.100. queue03 and queue04 are limited by their own max-limit.

In fakt, the behavior I see is not what I would expect if I see in your diagram. Yes, there is a short explanation in the text of the website, but that is not shown in the diagram. Maybe you should make a field in the arrow of the parent tokens to the child token rate like this to make it more clear:
Bucket_size change.jpg
You do not have the required permissions to view the files attached to this post.
 
4lphanumeric
newbie
Posts: 27
Joined: Wed Jan 16, 2019 1:00 pm

Re: Queue bucket-size option explained

Fri Jul 21, 2023 6:43 pm

Any way to incorporate the bucket size option in hotspot? I don't think /ip/hotspot/profile rate-limit had this option implemented.

Who is online

Users browsing this forum: Ahrefs [Bot], GoogleOther [Bot], itamx, mkx and 67 guests