Community discussions

 
KimC
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 95
Joined: Sun Jun 06, 2004 3:16 pm
Location: Denmark

Real world QOS - tips and tricks

Sat Feb 07, 2009 1:26 pm

Hello guys and girls !

We found (as many others did before us), that voip on a large unprioritized WLAN with many MT-boxes is at least difficult ;-) Random jitter, disconnections, lost voice in one direction etc. and many other problems are daily issues. Therefore we had to implement qos.

After countless versions, the solution has now converged to something very efficient. Voip is now ok 24/7, and generally, the lan has never performed so well before (very fast surf and completely symetrical bandwidth). Satisfying for everybody :-) I will not give you every detail, but in stead line op a number of things that really should be somewhere in the manual. Hopefully, this thread will end up as a comprehensive wiki-page, so many others with similar problems will be able to find a faster path to success.

1. Limit users (if you must) as close to the user as possible. PCQ is perfect for this job. Failing to do this gives large amounts of undesired retransmits.

2. Make a relatively simple priority scheme (not more than 4 categories), and use this consistently all over the network.

3. Never identify a packet more than once: At the traffic entry points only. The identification algorithms are not 100% failsafe and requires lots of cpu on routers with heavy traffic, and you wil typically get udp "out of order" if you forget this rule. At the same time, you will be able to save a lot of cpu-power for the much more important task of transporting the packets, when you get dscp-tagging working.

4. Pay attention to queue type and size for both interface queues and your own queue trees and simple queues. "Default" will almost always be wrong, when your traffic gets above dsl-capacity. If you forget this, you will suffer from "hidden" packet loss, since MT´s interface queues doesn't tell you about this problem. And remember: Udp behaves VERY different from tcp and requires special treatment.

5. I found, that on all routers except "last hop" routers, a simple global-total HTB is the best choice. Read the wiki carefully when setting it up.

6. All routers should have a queue tree - also a simple bridging repeater. If you don't do this, you will again suffer from "unexplainable" problems with "hidden" losses.

7. Remember to prioritize ALL traffic on the net (remember the output chain on MT-routers). All unprioritized traffic gets automatically highest priority, which is very undesired together with voip.

8. Using a protocol analyzer to evaluate your modifications, ensures that you move in the right direction during your development.

And most important: Read a lot, and don't give up if the first attempts don't work ! These pages are easy accessible, but they suffer from quality, as there are many novices on the forum. I found, that cisco is a very important source in this area. They may have a different way to describe things, but they are specialists, and they have VERY comprehensive manuals. And the general principles of qos are all the same, no matter of the software manufacturers name.

And then two questions:

- How do you properly detect Skype supernodes ??? The L7 regex avaliable gives too many false positives

- How do you detect Youtube traffic (forget about address lists - Google owns half of the internet) ??? I think that this low priority traffic is at least half of our http-traffic.

I hope you find this useful, and hopefully we will get a qos-wiki with all the practical stuff included.

Regards Kim
 
doush
Long time Member
Long time Member
Posts: 625
Joined: Thu Jun 04, 2009 3:11 pm

Re: Real world QOS - tips and tricks

Sat Feb 07, 2009 8:19 pm

Good Work. Im looking forward for the wiki entry aswell.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Real world QOS - tips and tricks

Sat Feb 07, 2009 8:21 pm

1. what queue types to use?

2. what do you mean "last hop"
- How do you properly detect Skype supernodes ??? The L7 regex avaliable gives too many false positives
You could distribute this .reg file to your customers and make them update their Skype before using the .reg, to the latest versions, with clean reinstalling - first uninstall, then install newewst version:
REGEDIT4

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Skype\Phone]
"DisableSupernode"=dword:00000001

[/i]
- How do you detect Youtube traffic (forget about address lists - Google owns half of the internet) ??? I think that this low priority traffic is at least half of our http-traffic.

Use Wireshark to detect something unique of a youtube connection, create a L7 pattern, and use mangle to 1. identify a packet that is known to be from a youtube connection 2. mark the connection of that packet 3. mark all the packets of the connection 4. include the packets into the queue tree you want. Additionally, to improve YouTube and other popular tubes quality, you could put a videocache Linux box(es) on your network: http://cachevideos.com/
I hope you find this useful, and hopefully we will get a qos-wiki with all the practical stuff included.

Regards Kim
Last edited by NetworkPro on Mon Apr 09, 2012 6:25 am, edited 2 times in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
KimC
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 95
Joined: Sun Jun 06, 2004 3:16 pm
Location: Denmark

Re: Real world QOS - tips and tricks

Sat Feb 07, 2009 11:32 pm

Networkpro,

1. The best queue types. Yoy really need to RTFM. However, the most important is to watch and try. I found, that sfq is useless when coping with high rates. Pfifo is best with voip, but you need to watch for dropped packets. Red seems to be best as general purpose. In any case: Adapt queue size to your network

2. Global-total and duplex. A wlan is 100 % half duplex ! Full duplex is a dream. Priority helps the packets to slip through both ways. As I said, with good qos, the network "seems to be" symmetrical.

3. Wireshark. RTFM - and keep focus on your problems. If it is udp, you will find lots of tools.

4. Url's: My best advice is www.google.com and some well chosen keywords. An other direction is to pick a couple of capable routers, and browse through the manuals. You wil be surprised...

5. Youtube. Forget about caches. Just imagine, how much storage this will require. And external bandwidth is not really the problem - the key question is how to seperate a video download from simple surf. It doesn't matter, that you have to wait a couple of sec's before the video plays, bút it is very annoying to wait a couple of sec's when you want to flip a page in your newspaper.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Real world QOS - tips and tricks

Sun Feb 08, 2009 2:18 am

You did all this and got good results with QoS and you still think that giving video lower priority than websites will have a big impact?

Tell me more about the network you implemented all this into? What is the internet uplink?

What happens when for example a link that you put QoS on fails to meet the max-limit you set, because of ... noisy Wi-Fi for example or ... wind screwing with the outdoor equipment causing 5% packet loss from time to time ? Or traffic being sent over ADSL? (http://www.adsl-optimizer.dk ) How do you deal with these problems?

Regards.
Last edited by NetworkPro on Mon Apr 09, 2012 6:27 am, edited 1 time in total.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
KimC
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 95
Joined: Sun Jun 06, 2004 3:16 pm
Location: Denmark

Re: Real world QOS - tips and tricks

Mon Feb 09, 2009 12:10 am

about 1: Last time when I tested with pfifo - the packet drops just happened
You only use pfifo for voip - and increase buffer until it don't drop packets anymore. Remember, that one voip-connection at most is 160 Kbit total, so you need a really large network to come into trouble, if your identification Works... And read http://www.mikrotik.com/testdocs/ros/3.0/qos/queue.php and http://wiki.mikrotik.com/wiki/Category:Manual - there you will find lots of info.
about 5: cache is good. no money to invest in harddrives?
Nope - we are talking terrabytes... And the problem is real. More than half of our http-traffic is Youtube. We must be able to give normal surf priority over video download.
And the killer question: [/b] What happens when for example a link that you put QoS on fails to meet the max-limit you set, because of ... noisy Wi-Fi for example or ... wind screwing with the outdoor equipment causing 5% packet loss from time to time ? Or traffic being sent over ADSL? (http://www.adsl-optimizer.dk ) How do you deal with these problems?
I don't have a good answer to your problem with a failing link. You must assume, that your links are in reasonable shape. However, if yout priority scheme is working, the most important traffic will pass through first, no matter of the speed - both ways. We don't work with dsl-speeds, but speed in not important, except that lower speeds requires more attention to bursting.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Real world QOS - tips and tricks

Mon Feb 09, 2009 1:36 am

However, if your priority scheme is working, the most important traffic will pass through first, no matter of the speed - both ways.
:shock: Explain, how, why?

I thought that when there are link problems traffic acts similar to as if we have set max-limit way above to what our real speeds are, therefore causing the packet queue to queue and delay the priority packets and misbehave altogether generally... ? How come the priority traffic will pass first? What kind of priority scheme would do that ?!?!? That would solve a lot of QoS problems worldwide if this worked.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Real world QOS - tips and tricks

Mon Apr 09, 2012 6:31 am

Hello

During this time I became an authority on MirkroTik QoS.

Some of the information above given by KimC is not true.

KimC, if you can - correct yourself, edit your posts.

Regards.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
phendry
Member Candidate
Member Candidate
Posts: 258
Joined: Fri May 28, 2004 4:42 pm

Re: Real world QOS - tips and tricks

Mon Apr 09, 2012 6:51 pm

During this time I became an authority on MirkroTik QoS.
Being any expert, do you have any recommendations for getting QoS to work properly on an N-Streme link?
 
doush
Long time Member
Long time Member
Posts: 625
Joined: Thu Jun 04, 2009 3:11 pm

Re: Real world QOS - tips and tricks

Mon Apr 09, 2012 10:17 pm

Do you really need to do all these things even if none of your links (backbone, POP PtPs) are congested ?

If nothing is congested, then everything flows through easily. right ?
 
KimC
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 95
Joined: Sun Jun 06, 2004 3:16 pm
Location: Denmark

Re: Real world QOS - tips and tricks

Mon Apr 09, 2012 11:08 pm

It's a very long time ago, I wrote this post.

Suddenly, I got a hint, that I should correct my errors :-)

Well, I can say so much, that the general scheme desribed still in the original post still holds true in our large network (+3000 private households and +1000 ip-phones). However, reading through the conversation, I must agree, that some minor details aren't as clear as when I wrote it.

* We still use sfq for most queues, and pfifo for voip. But I'm not so sure, that it is as important as when I wrote it.
* When you have a partially link failure, nothing using HTB works, unless you have a really low MaxLimit. I am experimenting with the nv2 qos, where a better priority scheme should be implemented, but I haven't concluded anything yet. Also, we are using more and more ubnt-hardware for links. It's easier, cheaper, maybe faster but not better, and you loose a little control over your links.

NetworkPro, what did you have in mind ??

Regards Kim C
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Real world QOS - tips and tricks

Tue Apr 10, 2012 10:46 pm

Why not use the newly rewritten PCQ instead of SFQ ?
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: Real world QOS - tips and tricks

Mon Oct 08, 2012 5:10 pm

During this time I became an authority on MirkroTik QoS.
Being any expert, do you have any recommendations for getting QoS to work properly on an N-Streme link?
If there is queueing (latency) in your particular link - for reasons of aggregations and such - then I recommend marking a little bit of priority packets and using the built in queues. Are you asking for NV2 ?

I don't like the latency though, if it is nstreme - turn down the aggregation ..
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service

Who is online

Users browsing this forum: No registered users and 48 guests