Community discussions

MikroTik App
 
void
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Fri Nov 07, 2008 5:28 pm

Strict priority queue

Sun Apr 11, 2010 11:27 am

So finally my QOS on my RB's are working as desired. Using limit-at and max-limit I can give priority to the different types of traffic during congestion periods. Now this only works if we have guaranteed bandwidth from our provider (which is very rare unless you pay a fortune).

Now what in the case of non-guaranteed bandwidth ? Wouldn't it be possible to implement a queue that has strict priority and has priority on all other traffic no matter what the available bandwith is ? Mainly for voip (sip) traffic this would be a great feature.

I was also thinking about creating a script that uses the bandwidth test tool to check bandwidth between 2 RB's every x minutes and adept the max-limit of the queues dynamically but this isn't a solid and good solution.

VOiD
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8394
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Strict priority queue

Wed Apr 14, 2010 1:13 pm

unfortunately, there are many pitfalls in such situations. for example, if you use ADSL modem, you cannot even know whether the upload link is congested - because 100 Mbps between the router and the modem are always almost free...
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
void
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Fri Nov 07, 2008 5:28 pm

Re: Strict priority queue

Wed Apr 14, 2010 2:44 pm

Chupaka,

That's the reason why I would like to have the ability to have a strict priority queue no matter what the available bandwith is.

With the current system priority only works if the max-limits are reached. That means that in case you have a 100Mbit non-guaranteed link and this links drops to 99Mbit the priority doesn't work and things get screwed up.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8394
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Strict priority queue

Wed Apr 14, 2010 2:59 pm

actually, priority works when limit-at is reached

and in general the situation is how HTB works. yes, it would be interesting to add possibility to force bypassing the HTB for some packets - but it's huge work, and for now we do not have something called 'system-test' or 'queue-test' packsge :(
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
changeip
Forum Guru
Forum Guru
Posts: 3819
Joined: Fri May 28, 2004 5:22 pm

Re: Strict priority queue

Wed Apr 14, 2010 7:09 pm

You run a car rental facility, but don't inventory cars. You've got 10 people in the blue collar line, and 10 people in the white collar line. Do you just send all 20 out of the door to get cars if the white collars are supposed to get them first?

How do you know when to give something priority if you dont know how full the lot is? You have to know the max limit in order to decide if you want to drop something else.

One question however, if you set max-limit at 1gbs and use priorities, does that still reorder packets in the outbound interface queue.
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com
 
void
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Fri Nov 07, 2008 5:28 pm

Re: Strict priority queue

Wed Apr 14, 2010 7:23 pm

You don't determine the priority when something is full, you determine it on beforehand. The only thing you need to decide when something is full is what you're going to drop.
I have a 100Kb stream that I decided that needs to have strict priority and I don't care about the 99,9... Mbit of the 100Mbit connection, not even when there is only 50Mbit available instead of 100Mbit.
 
changeip
Forum Guru
Forum Guru
Posts: 3819
Joined: Fri May 28, 2004 5:22 pm

Re: Strict priority queue

Wed Apr 14, 2010 7:30 pm

so can the limits be set to artificial values (1gbs) and priorities will be enforced in the interface queue?
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8394
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Strict priority queue

Wed Apr 14, 2010 7:31 pm

You run a car rental facility, but don't inventory cars. You've got 10 people in the blue collar line, and 10 people in the white collar line. Do you just send all 20 out of the door to get cars if the white collars are supposed to get them first?

How do you know when to give something priority if you dont know how full the lot is? You have to know the max limit in order to decide if you want to drop something else.
but let's suppose you have a gate that is passing one white car per minute - that's our queue. and now we get a blue one. so we want the blue car just go round all white cars and pass the gate without limitations =)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Sun Apr 25, 2010 9:37 pm

I think you could get reordering. If the different prio packets arrived at the same time? :) We need to discuss this.

And even then, if you send more packets upstream, the next device that has congestion, will drop whatever it decides, usually, in a .... crippled way :) Hahah. Unless your ISP has good packet queues themselves on the core / border, and you have no congestion up to that router. :) (Thanks to all upstream admins who do it properly).


Stop making examples with cars. Its not helping. Make examples with packets. And frames.


Just set that parents queue max-limit to a lower value. That's it. What exact value to use - that can be tested.
Last edited by NetworkPro on Thu May 06, 2010 8:12 pm, edited 1 time in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Strict priority queue

Tue Apr 27, 2010 8:30 am

And again we come to the same story about priorities.

Just to avoid any confusion - in MikroTik's QoS there are NO PACKET REORDERING, there are list of packets that will be dropped , but all remaining packets will leave QoS in the same relative order they came in.

Priority is not "Who goes trough first?" indicator it is more like "Who gets dropped last?" - higher priority packet has - less likely it will be dropped.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Tue Apr 27, 2010 10:04 am

I think theres a contradiction with the theory.

So let's say we have a packet queue with two leafs. One leaf is a priority one and for example 200kbps of packets will be processesd by it. This means they will go out first, no matter what, right? Because they have limit-at set at 200kbps. And the other leaf does not have limit-at set at all but it has a bigger queue size, so instead of dropping the packets from the second leaf, when the higher priority ones from the first leaf are sent out, they are queued. Delayed.

WIll these packets leave reordered?
Last edited by NetworkPro on Thu May 06, 2010 8:14 pm, edited 3 times in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Strict priority queue

Tue Apr 27, 2010 11:18 am

We are talking about real-time action here - there are no time to delay some packets (to let others first) , cause on next moment there will be next packets incoming. So what will you do then?

In real-time action you need to act fast - so each packet gets trough or gets dropped strait away. HTB is just a way to make that decision. Higher priority - less likely be discarded.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24609
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Strict priority queue

Tue Apr 27, 2010 11:34 am

To clarify.

Priority have no effect on packet order, but Queue types (of queues and interfaces) can cause packet reordering
No answer to your question? How to write posts
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Tue Apr 27, 2010 11:41 am

So when PRIORITY is working, the default action for non-priority queues is DROP insatead of QUEUE?
Last edited by NetworkPro on Thu May 06, 2010 8:15 pm, edited 1 time in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Tue Apr 27, 2010 12:00 pm

So http://wiki.mikrotik.com/wiki/Manual:Queue_Size How can this be explained better?
Last edited by NetworkPro on Thu May 06, 2010 8:15 pm, edited 1 time in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Strict priority queue

Tue Apr 27, 2010 12:42 pm

Queue size is one of the options of Queue Types - isn't it?
And if we look above we can see:
To clarify.
Priority have no effect on packet order, but Queue types (of queues and interfaces) can cause packet reordering
Priority, max-limit, limit-at are HTB options that indicate when and what traffic to drop. Queue types are not part of HTB, it is next step after HTB.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Tue Apr 27, 2010 2:54 pm

Macgaiver, do you think this can be summarised as a "manual" page or paragraph?
Last edited by NetworkPro on Thu May 06, 2010 8:16 pm, edited 1 time in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Tue Apr 27, 2010 3:03 pm

So if HTB drops packets, and if queues are after that, that means that we have tail drops when the queues get full

AND drops before that.

How can one really control what is dropped when we have so much mistic dropping of packets? :)
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
dssmiktik
Forum Veteran
Forum Veteran
Posts: 732
Joined: Fri Aug 17, 2007 8:42 am

Re: Strict priority queue

Tue Apr 27, 2010 9:38 pm

Could someone summarize these findings? I'm reading through this post, and I'm getting more confused. I read the HTB article on the Wiki, but some posts here seem to contradict it.
Doug
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Strict priority queue

Wed Apr 28, 2010 9:04 am

P.S. do you see this information from the source code?
No - experimentation over last 3-4 years. - MikroTik MTCTCE training + lots of questions to teacher
Macgaiver, do you think this can be summarized as a "manual" page or paragraph?
Well my guess is: the whole picture (with all possibilities) of QoS is way to complicated to put it all in one topic, to understand the whole you need to understand each part, then you will be able to combine those for your specific example. And most of the setups use only subset of 2-3 QoS feature at the same time, not the whole list, so I do not think summarized topic (at leaset how i imagine that) will help anyone.
Could someone summarize these findings? I'm reading through this post, and I'm getting more confused. I read the HTB article on the Wiki, but some posts here seem to contradict it.
Priority is a HTB option, that together with max-limit, limit-at and queue structure helps to determine how much actual traffic can be given to child queues. All traffic that is not allocated to the queues is dropped.

To ensure that communications is not "choppy" a buffer was introduced to every child queue - queue type (with specific queue size)
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Wed Apr 28, 2010 12:56 pm

My packet queus work. My QoS works. I build my business around it. I guess I just found out the working setup by experimentation, and I thought I knew all there was. But then I saw that HTB can drop packets without even trying to put them in the queue. And there is already a queue. So I'm asking all these question here.
Last edited by NetworkPro on Thu May 06, 2010 8:17 pm, edited 1 time in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Strict priority queue

Wed Apr 28, 2010 1:57 pm

I think you are making situation too dramatic. As far as I can see you just had 1 misunderstanding - you just made assumption that "priority" is rearranging packets so that first priority get trough first. This assumption doesn't change anything if you do not have double queue system (traffic shaped by type, and later shaped per client IP) on the same router.

I just notice that original topic was a little bit different ). Sorry, void!!!
So finally my QOS on my RB's are working as desired. Using limit-at and max-limit I can give priority to the different types of traffic during congestion periods. Now this only works if we have guaranteed bandwidth from our provider (which is very rare unless you pay a fortune).

Now what in the case of non-guaranteed bandwidth ? Wouldn't it be possible to implement a queue that has strict priority and has priority on all other traffic no matter what the available bandwith is ? Mainly for voip (sip) traffic this would be a great feature.

I was also thinking about creating a script that uses the bandwidth test tool to check bandwidth between 2 RB's every x minutes and adept the max-limit of the queues dynamically but this isn't a solid and good solution.
As far as i can see it - it is not possible. There are no way of knowing for sure that available throughput has changed. And rearrangement of packets in Real-time environment - when packets are coming and leaving all the time, even if implemented - will take a loots of CPU and memory.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
void
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Fri Nov 07, 2008 5:28 pm

Re: Strict priority queue

Wed Apr 28, 2010 6:43 pm

I just notice that original topic was a little bit different ). Sorry, void!!!
No problem.
As far as i can see it - it is not possible. There are no way of knowing for sure that available throughput has changed. And rearrangement of packets in Real-time environment - when packets are coming and leaving all the time, even if implemented - will take a loots of CPU and memory.
Well do we need to know what the throughput is in the first place ? If we know that certain packets have max priority why not send them directly and queue the others (this is already the case in HTB if I get it correct) ?
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Strict priority queue

Thu Apr 29, 2010 8:00 am

Well do we need to know what the throughput is in the first place ? If we know that certain packets have max priority why not send them directly and queue the others (this is already the case in HTB if I get it correct) ?
and here comes "Limit-at" - as we all know this amount is just given to the queue - even if parents do not have enough traffic.
So with limit-at you can build worst-case scenario, and with max-limits/priorities cover all better possibilities.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
void
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Fri Nov 07, 2008 5:28 pm

Re: Strict priority queue

Thu Apr 29, 2010 9:03 am

You are right macgaiver, thanks a lot ! I learned a lot through this thread.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24609
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Strict priority queue

Thu Apr 29, 2010 9:05 am

You are right macgaiver, thanks a lot ! I learned a lot through this thread.
don't forget to give karma :)
No answer to your question? How to write posts
 
void
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Fri Nov 07, 2008 5:28 pm

Re: Strict priority queue

Thu Apr 29, 2010 9:08 am

[don't forget to give karma :)
Normis, I already did 8)
 
User avatar
111111
Member Candidate
Member Candidate
Posts: 194
Joined: Thu Oct 05, 2006 1:39 am
Location: BG,SOFIA

Re: Strict priority queue

Thu Apr 29, 2010 10:50 am

Invest in speed and hardware, there is enough "Knowhow".
With making a temporary solutions, you will lose your clients
In God We Trust. All other must submit X.509 certificate!
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Thu Apr 29, 2010 11:20 am

Thanks for the recommendation, 111111. It is always good to invest in more speeds, that can be achieved with newer better hardware.

It is always a good idea for a router to have specially designed, optimised Packet Queues. If these queues are QoS-oriented - that's even better.

For example, on wireless interfaces, the default queue is not fifo. It's SQF that works a certain way. So a network admin for example can see what SFQ does and develop a huge Queue Tree that treats traffic in a way to serve the demand the right way.


Overprovisioning is done with high speed optical equipment and switches etc. The things that the big telecoms use. For them its easier to overprovision. And very often Layer 2 devices do not give you control over the packet queues.

When you can control them - use that. If you really are building a better packet queue - that will be a benefit to the QoS and QoE and you will actually gain more customers, when all the kids tell their friends that your service is great, for example.

So in order to be sure that you are helping the traffic with your queue, you have to understand what happens with it 100%. So far this is only possible if you look at the source code. No one has moved his ass to explain everything in complete detail.

I'm doing it with experimenting. My lab setups over the years have been from huge rooms to small on the table, to virtual, to live networks with live traffic. Day. Night. Morning.

But anyway. The protocols are built in a way to allow a business to exist without optimising the queues, but with buying hardware for better speeds and buying more bandwidth. So it's up to the manager of the company to decide which way to go - overprovision or control the traffic. Or overprovision first and control the traffic later or control traffic first and overprovision later.

In my opinion a quality business has control over everything. Traffic and packet queues included.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Mon May 03, 2010 10:10 am

So OK, let's take this example and try to solve it:

We have the simplest routing architecture and we want to prioritise only VoIP Outgoing, but it can take upto 99% or 100% of the entire bandwidth. And we want to let it do that.

Let's say we already have marked the packets - VoIP and all Else.

So we are building an HTB Queue Tree. WHat will that Quueue Tree look like?

+Queue Tree max-limit=100%
-->VoIP limit-at=100% max-limit=100% queue-type=fifo-10-packets
-->Else limit-at=0 max-limit=100% queue-type=fifo-10-packets

So let's say enough VoIP packets hit this queue/tree so that they take up 100%. And at the same time 500% Else traffic hits that HTB /queue tree. This is possible becase one Interface is 100Mbit or 1 Gigabit so it recieves a lot more than the Queue Tree can send out.

So let's drill down really deep as much as we can go with a surgical instrument. So what will happen ON A PER-PACKET LEVEL? How will this HTB / queue tree treat the traffic?
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
uldis
MikroTik Support
MikroTik Support
Posts: 3439
Joined: Mon May 31, 2004 2:55 pm

Re: Strict priority queue

Wed May 05, 2010 4:18 pm

I think some analogy is in the place:

Imagine a factory with conveyor belts. At the end of each conveyor belt there is a packing assembly, that pack each product unit into boxes and send out to export department. Each packing assembly have defined parameters:
1) buffer size - how many units of product it can hold unpacked in the assembly itself
2) packing speed - units per minute
3) packing method

Now imagine that these conveyor belts are unstoppable and all the product units that can't get into the packing assembly (cause buffer is already full) will be just dropped on the floor. As packing assembly is still working buffer periodically will have an empty spot that will be taken by most current product unit (cause all previous units are on the floor already)

These conveyor belts have different operators, that can simply forget about dropped units (UDP), or will slow down the conveyor belts speed and place dropped units back on the belt (TCP)

Also some of the packing assemblies have "turbo" button, that allow to increase packing speed for short amount of time, after that packing assembly needs to cool down

If your packing assemblies create more boxes than export department can handle, it has exactly the same type of system for putting product boxes into trucks/lorry.

factory - HTB
conveyor belt - traffic streams
packing assembly - queues
export department - Interface queue
buffer size - queue size
packing speed - limits
packing method - queue type
operators - protocols
"turbo" button - Burst
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Wed May 05, 2010 4:42 pm

Haha. Thanks. I will teach my kids that :)

Now point me to the current Linux HTB and queues source code :) I hope it's 100% theoretically the same as whats in RouterOS.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Wed May 05, 2010 5:43 pm

Contradictions contradictions contradictions :)

http://www.opalsoft.net/qos/DS-28.htm To what extent does this apply? So HTB is like RED? But RED is a queue type. So HTB is a Queue Type? Confusing :)
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
Eising
Member Candidate
Member Candidate
Posts: 272
Joined: Mon Oct 27, 2008 10:21 am
Location: Copenhagen, Denmark

Re: Strict priority queue

Wed May 05, 2010 7:03 pm

Yes, it is a bit contradicting, I will give you that. HTB is not a queuer, but a scheduler, and it works (d'oh) in a tree-like hierarchy.

Each node in the HTB tree is a separate queue, and those queues can have different queuing disciplines. Very roughly speaking, queuing has to do with how and when you drop your packets, while scheduling has many other purposes.

In your HTB setup, it is HTB's primary role to divide packets into different leaf queues where you normally implement a fifo queuing technique in. It also makes sure that bandwidth is distributed into the leafs in the manner configured.
Most of this is obvious, I assume.

The relationship with queuing techniques such as RED is twofold: Each leaf queue has a queuing algorithm attached. Normally, it is some kind of FIFO, but this could be set to any other algorithm, such as SFQ, PCQ or RED. Using any of these queuing techniques allows you to make sure that bandwidth within the leaf-queue is distributed in a way that differs from FIFO. SFQ and PCQ will allow you distribute the bandwidth within the leaf queue fairly, and avoid one connection dominating a leaf queue, while RED will, if configured properly, try to avoid congestion before the actual packet loss occurs. Besides the queuing within the leaf-queues, the data will always end up on the interface queue, and it is possible to once again implement a queuing algorithm here.

I hope this clarifies things a bit, and if I have just stated things that were apparent to everybody already, I apologize.
The road to hell is paved with good intentions.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Wed May 05, 2010 7:36 pm

Danke, Eising. Very very good post.

So about my example three posts above - anyone can tell me exactly what happens to my packets? What happens to my VoIP prioritiesd packets? I don't want to have loss with those :) And what will happen to the others, non-prioritised ones?

The packet is put in which queue first? or in what type of buffer? where it is evaluated how? based on what merit exactly? What does "HTB" do to it? Can it decide to discard it? I hope not. I hope it decides to queue it. Actually I hope it does not queue my VoIP packet - it needs to get out the interface without any delay whatsoever. SO I hope it waits for the current packet beign transmitted (Serialised) and then it launches my VoIP packets right afterthat with a lot of jet fuel. But what happens to the low priority packets at this time? I hope they get queued and not dropped. If they are dropped - that should happen at the end of the queue - the so-called "tail drop" and not before they have a chance to get queued. etc etc. This stuff needs to be clear :)

I still have not found the "TC" .c source file or the part of the Linux Kernel with that stuff in it. Anyne can help and point us in the right C source file?
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
Eising
Member Candidate
Member Candidate
Posts: 272
Joined: Mon Oct 27, 2008 10:21 am
Location: Copenhagen, Denmark

Re: Strict priority queue

Thu May 06, 2010 1:30 am

So OK, let's take this example and try to solve it:

We have the simplest routing architecture and we want to prioritise only VoIP Outgoing, but it can take upto 99% or 100% of the entire bandwidth. And we want to let it do that.

Let's say we already have marked the packets - VoIP and all Else.

So we are building an HTB Queue Tree. WHat will that Quueue Tree look like?

+Queue Tree max-limit=100%
-->VoIP limit-at=100% max-limit=100% queue-type=fifo-10-packets
-->Else limit-at=0 max-limit=100% queue-type=fifo-10-packets

So let's say enough VoIP packets hit this queue/tree so that they take up 100%. And at the same time 500% Else traffic hits that HTB /queue tree. This is possible becase one Interface is 100Mbit or 1 Gigabit so it recieves a lot more than the Queue Tree can send out.

So let's drill down really deep as much as we can go with a surgical instrument. So what will happen ON A PER-PACKET LEVEL? How will this HTB / queue tree treat the traffic?
Let me see if I can answer that. Let's have a realistic queue tree:
Parent, max-limit=4Mbit/s parent's parent is the interface queue
  |
  --> Unmarked traffic, prio 7, max limit set to 100% (or 0 i RouterOS), pfifo
  |
  --> VoIP traffic, prio 6 (aka one better than unmarked), no max limit, as before, pfifo
  • Before traffic starts, there are no packets in any of the queues.
  • When the unmarked queue is full, and VoIP traffic exists, that queue will lend traffic to the VoIP queue, since it has better priority.
  • When the VoIP queue is full, new flows will be dropped
Another way of understanding this is looking at the two performance stats of the queues, lends and borrows.
Bandwidth will always be distributed downwards in a queue. For instance a parent queue with 4Mbit/s available bandwidth will give bandwidth to demanding children. The Unmarked queue will take (borrow) bandwidth from the parent when required. The VoIP queue will also borrow from from this, but if there is no bandwidth available, it will borrow it from the queue that has the lowest priority (highest number). When a queue cannot borrow any more bandwidth, it will drop packets.
It may be desirable to set up a limit-at parameter to ensure that a queue will not end up at 0 rate if no more bandwidth is available.

I hope this helps you forward.

If you want to see the source code of linux' tc program, it is part of the iproute2 suite, and can be fetched here: http://www.linuxfoundation.org/collabor ... g/iproute2
The road to hell is paved with good intentions.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Strict priority queue

Thu May 06, 2010 8:18 am

1)So about my example three posts above - anyone can tell me exactly what happens to my packets?
2)What happens to my VoIP prioritiesd packets? I don't want to have loss with those :)
3)And what will happen to the others, non-prioritised ones?
4)The packet is put in which queue first? or in what type of buffer?
5) where it is evaluated how? based on what merit exactly?
6)What does "HTB" do to it? Can it decide to discard it?
7)I still have not found the "TC" .c source file or the part of the Linux Kernel with that stuff in it. Anyone can help and point us in the right C source file?

1) Packets can be
a) accepted ("packed in to box and sent to export department")
b) buffered/scheduled ("in packing assembly buffer")
c) dropped (on the floor)

2) in your example VoIP traffic have good chance to be accepted till it reaches 100% + 10 packets can be buffered
3) in your example non-prioritized traffic probably will be dropped + 10 packets can be buffered

4) 5) 6) packets just arrive to HTB, in the same order as they travel trough wire, and then HTB gets it all done the way uldis posted

7) I think you will only be able to get original HTB from http://luxik.cdi.cz/~devik/qos/htb/
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Thu May 06, 2010 8:32 pm

I was also thinking about creating a script that uses the bandwidth test tool to check bandwidth between 2 RB's every x minutes and adept the max-limit of the queues dynamically but this isn't a solid and good solution.
Inspired by cFosSpeed, I see a clear way to implement something in RouterOS. This feature would kick ass, I hope MT see how it will make a lot of cash for them, feelin' rich MT? :). It can also be done with a tool that is run on a separate host that communicates with a server and checks for loss between the two. By ICMP and by TCP RTT and the way TCP Vegas does it and the way the new uTP protocol does it - this communcation will need to have high priority in RouterOS. If only ICMP is used - this can be scripted in bash and in RouterOS. If loss occurs - the max-limit is lowered. This feature can be put in a new binary and in a new npk called "prioritiser" or "WAN optimiser". This new binary can be made outside MT, so that the MT guys would be free to make all the other stuff they have their hands full with already. :) This feature will be useful for example in cases when the available bandwidth changes slowly. So it will adjust the max-limit to a lower value than what is actually avaliable. So we will still have wasted bandwidth but at least we will have better chace for our priority packets. So its either joy and happines by this method or its back to the dark ages and sure death by setting youtself on fire while drowning.
unfortunately, there are many pitfalls in such situations. for example, if you use ADSL modem..
If you have something other than Ethernet - you have serialisation delay and crazy loss and delay due to ATM etc etc, due to noise, etc etc. so there are soooo many variables to work with. Remember the adsl-optimiser that claimed to be able to delay the packets properly, to compensate for the ATM BS on ADSL? I wonder if RouterOS HTB and queues work in a different way when you have different serialisation delay caused by your interface. In my opinion MikroTik have to make an integrated ADSL client interface so that transmission over it is controlled the best way possible.
I would like to have the ability to have a strict priority queue no matter what the available bandwith is.
so can the limits be set to artificial values (1gbs) and priorities will be enforced in the interface queue?
You can. Your queue will be able to send out the priority packets strictly before the rest. The problem occurs in the next device. It will treat all packets equally or it will have other priority packets - some client that is paying tons more cash for example. Or there will be enough resources but the device will be misconfigured (very often). Because in the case you send out all priority packets first and after them tons and tons of non-prioirty packets, the upstream device will need to drop some. Which ones will it drop? Usually it will choose which ones to drop in a dumb-ass crippling way. This is why DiffServ is a (not guaranteed) standard.
With the current system priority only works if the max-limits are reached. That means that in case you have a 100Mbit non-guaranteed link and this links drops to 99Mbit the priority doesn't work and things get screwed up.

If the upstream device has smart queues and small queue sizes, things will be not so bad. But I dont trust ANY upstream device, even if it was configured/owned by the Pope. So usually the max-limit is set to 98999k and that 1Mbit is left unused, to achieve queue control. And that control is still some of the time :). Since there will be times that the usptream device will be able to forward even smaller amounts of your traffic... But this is The Internet, it is defined as not guaranteed. So I'ts not your problem. If you can compare an upstream provider to another, if you have access to many ISPs routers (like me) and can do special tests to see where the loss occurs and the delays etc - you can make the decision to switch upstream providers or to make a new deal with your current one. Or to build your own link/rent your own dedicated optic to an even upper-stream provider. From which you can profit by selling "guaranteed" bandwidth and you will be able to mess up your own "upstream" devices :) There are other ways to help deal with these problems that I have implemented extremely successfully in a dozen of networks :).
actually, priority works when limit-at is reached
I hope you mean the "HTB feature" "Priority". Because one can achieve "piroirty" by bulding his Queue Tree with the limit-ats and the max-limits in a way to give certain leafs "priority" (limit-at). And I think void talks about doing it this way in his first post. In any case, I ask everyone who use this word to specify whether he is talking about "HTB Priority feature" or priority by limit-at or what exactly? Priority that needs Not be defined but just needs to be achieved? (as most users ask for - they just need it achieved). And @MT - could you please make a destinction between these two in the documentation and in the sofware, maybe start using a new word or phrase? As far as I see it right now - one is the "THB Priority feature" from 1 to 8 and the other is "limit-at". So what exactly is the practical difference between the two? How to take advatage of both, in an optimal way?
but let's suppose you have a gate that is passing one white car per minute - that's our queue. and now we get a blue one. so we want the blue car just go round all white cars and pass the gate without limitations =)
My VoIP example should do that. And the price would be that a white car would be made to disappear by Harry Potter (discarded packet) so that the blue one can take its place for immedaite transmission. Also that could happen if the blue car was missed in mangle and it travels out the same interface, in which case the decision if a white car should be dropped or not is made upstream, and in which case this decision could drop/delay a blue car.
Just to avoid any confusion - in MikroTik's QoS there are NO PACKET REORDERING, there are list of packets that will be dropped , but all remaining packets will leave QoS in the same relative order they came in.
What if these packets came in different interfaces?

So let's assume that on 100Mbit FastEthernet, packets are sent one by one, but so fast that in one second time - maybe about 7k-10k pps.

So we have 7k packets coming in the Local interface. Let's assume we number our incoming packets. Let's say we recognise (mangle) 2-3 VoIP packets. They came in as packet number 3, number 666 and number 6666.

So let's say we want to send out VoIP packet number 3 before non-VoIP packet number 2. For that "reordering" to be possible, the outgoing interface serialisation must be slower than FastEthernet. And for RouterOS to know that, it must be a local interface for example a 10Mbit Etherent card. Also the outgoing queues probably need to be bigger. Someone can wrap his brain around this situtation and tell us how could this happen or how and why this will not happen ? :)
When the unmarked queue is full, and VoIP traffic exists, that queue will lend traffic to the VoIP queue, since it has better priority.
At this point what happens with the lower prio packet that would get this bandwidth if there were no VoIP packets? It gets queued? Or it gets discarded? Because the VoIP one "overwrites" it? Such are the important qeustions that are not (definitely) answered in the conveyr belt example. Or in the Manual. Or anywhere.
The VoIP queue will also borrow...... When a queue cannot borrow any more bandwidth, it will drop packets
Which packets will it drop? The ones in front of the queue or the ones at the back of the queue? :) What if we want it to drop from the middle of the queue?
It may be desirable to set up a limit-at parameter to ensure that a queue will not end up at 0 rate if no more bandwidth is available.
What if this queue is for the really low priority traffic? Like p2p traffic on a business network ? :) Will the Low-prio traffic from my VoIP example find itself all discarded when the VoIP traffic takes full 100%?
7) I think you will only be able to get original HTB from http://luxik.cdi.cz/~devik/qos/htb/
The .diff files are the complete source?

I found something here : http://git.kernel.org/?p=network/iprout ... ;a=summary the Snapshot link
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1370
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Strict priority queue

Sun May 09, 2010 11:22 pm

OK let me clarify a bit,

a quote from http://wiki.mikrotik.com/wiki/Manual:HTB
Priority

We already know that limit-at (CIR) to all queues will be given out no matter what.

Priority is responsible for distribution of remaining parent queues traffic to child queues so that they are able to reach max-limit

Queue with higher priority will reach its max-limit before the queue with lower priority. 8 is the lowest priority, 1 is the highest.

Make a note that priority only works:

* for leaf queues - priority in inner queue have no meaning.
* if max-limit is specified (not 0)
This HTB "Priority" feature gives bandwidth.

Where as when one says "strict priority queue" - one should mean - a queue (tree) that sends priority packets out, while non-priority packets are put in a queue to be sent after the priority ones. Strict - this means that the priority packets will never wait for a non-priority packet. Even if we have huge loss with the non-priority stuff.

So a lot of answers can be given in this topic. A lot of clarity can be given.

Maybe even as a flash movie, the Janis factory example http://forum.mikrotik.com/viewtopic.php ... 85#p206685 could be used as a base. And still - more clarity is needed for how, when packets are decided to be dropped, to be queued, etc. - informaction that can be shown graphically.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
alaskanjackal
newbie
Posts: 25
Joined: Tue Sep 29, 2015 1:29 pm

Re: Strict priority queue

Fri Mar 24, 2017 10:11 am

I was also thinking about creating a script that uses the bandwidth test tool to check bandwidth between 2 RB's every x minutes and adept the max-limit of the queues dynamically but this isn't a solid and good solution.
Inspired by cFosSpeed, I see a clear way to implement something in RouterOS. This feature would kick ass, I hope MT see how it will make a lot of cash for them, feelin' rich MT? :). It can also be done with a tool that is run on a separate host that communicates with a server and checks for loss between the two. By ICMP and by TCP RTT and the way TCP Vegas does it and the way the new uTP protocol does it - this communcation will need to have high priority in RouterOS. If only ICMP is used - this can be scripted in bash and in RouterOS. If loss occurs - the max-limit is lowered. This feature can be put in a new binary and in a new npk called "prioritiser" or "WAN optimiser". This new binary can be made outside MT, so that the MT guys would be free to make all the other stuff they have their hands full with already. :) This feature will be useful for example in cases when the available bandwidth changes slowly. So it will adjust the max-limit to a lower value than what is actually avaliable. So we will still have wasted bandwidth but at least we will have better chace for our priority packets. So its either joy and happines by this method or its back to the dark ages and sure death by setting youtself on fire while drowning.
Sounds like the "active congestion control" feature on Gargoyle (an OpenWrt distro): https://www.gargoyle-router.com/phpbb/v ... ?f=7&t=754

If I'm understanding it right, the final couple of pages in Muqatil's slideshow here have a working Mikrotik script that accomplishes something similar: viewtopic.php?t=48056#p243897

I may mess with it in the next week or so and see how it performs.

CoDel sounds intriguing, but I don't think it'll work unless I have control of the slow link. As an end user (not an ISP) with a WISP that has very variable speeds (~4mbps to ~12mbps, depending on the second) with a RouterBoard as my gateway but plugged into their router/modem via Fast Ethernet, the CoDel algorithm won't have any idea what kind of bufferbloat/queueing is taking place on the modem, so it can't adjust anything to compensate. A script like described above is probably the best I can hope for.

Other options are this: viewtopic.php?t=58205, but I don't really see how that would do anything if done on my router, or post #4 here: https://forum.openwrt.org/viewtopic.php?id=26086, which has potential if possible to do on MT.

Who is online

Users browsing this forum: No registered users and 79 guests