Community discussions

MikroTik App
 
troy
Member
Member
Topic Author
Posts: 320
Joined: Thu Jun 30, 2005 6:47 pm

Bandwidth Management & QOS - Is it possible?

Sat Nov 16, 2013 4:00 pm

For bandwidth management, we've been using a Queue Tree with PCQ. We have 3 different bandwidth packages, and each user (by IP) is in an appropriate address list. The parent queue limits bandwidth to 40/8 (off a Comcast 50/10 connection). With peak usage at just under 30M, this has been working awesome.

To give you an idea of how awesome this is, the SMC cable modem started acting up, so we had to switch to a backup link (16/2). After adjusting the queues accordingly, customers were still happy with overall performance, even though we flatlined the bandwidth for 2 days:
foobar.gif
With regards to QOS on MT, I consider it to be tried and true... I can transfer massive amounts of data, while the family watches streaming moovies, while I'm playing games and talking on the phone. I have to set the top-level parent queue to about 90% of my actual capacity, but it works awesome!

The one thing I have not yet been able to figure out how to implement, is bandwidth management and QOS at the same time for 100's or even 1000's of users.

Here's what I'm trying to accomplish:

Bandwidth packages: 768k, 1.5M, 3M, and eventually a 6M and 12M package, with QOS to prioritize by type of traffic.

If Normis wants to use VOIP on his 768k package, the quality of his call should not suffer because everyone else is watching streaming moovies.

When Janisk browses the web, it should be snappy and responsive, even if the network is completely congested with other traffic.

And when Sergejs wants to waste some time playing an online game, he should not lag out because someone's downloading a thousand different torrents.

---

So, how can I do both bandwidth management and QOS at the same time?

I read that in ROS6, one can use PCQ in simple queues, which would leave the Queue Tree open for QOS, but for the life of me, I can't figure out how to re-create my PCQ Queue Tree in Simple Queues. If there's a working example out there, I've not yet found it.

Can someone help me translate this queue tree into Simple Queues?

ros code

/queue tree
add disabled=yes limit-at=34M max-limit=40M name=total_down parent=global-out
add disabled=yes limit-at=5M max-limit=8M name=total_up parent=global-out
add disabled=yes name=Bronze_Upload packet-mark=Bronze_Upload parent=total_up queue=pcq_bronze_up
add disabled=yes name=Bronze_Download packet-mark=Bronze_Download parent=total_down queue=pcq_bronze_down
add disabled=yes name=Silver_Upload packet-mark=Silver_Upload parent=total_up queue=pcq_silver_up
add disabled=yes name=Silver_Download packet-mark=Silver_Download parent=total_down queue=pcq_silver_down
add disabled=yes name=Gold_Upload packet-mark=Gold_Upload parent=total_up queue=pcq_gold_up
add disabled=yes name=Gold_Download packet-mark=Gold_Download parent=total_down queue=pcq_gold_down
add disabled=yes name=Plat_Download packet-mark=Plat_Download parent=total_down queue=pcq_plat_down
add disabled=yes name=Plat_Upload packet-mark=Plat_Upload parent=total_up queue=pcq_plat_up
PS... WTF is this "can't post about MOOVIES" ??? don't we all have customers who watch MOOVIES on the internet? Doesn't this use up bandwdith and affect network performance? Isn't this a VALID topic of conversation? Isn't it a little pedantic and childish for MT to put a freaking word filter on this?

Thanks...
You do not have the required permissions to view the files attached to this post.
 
derr12
Member
Member
Posts: 411
Joined: Fri May 01, 2009 11:32 pm

Re: Bandwidth Management & QOS - Is it possible?

Mon Nov 18, 2013 9:33 pm

Im my experience, i would create your speed "tiers" in simple queues. you wont need to set mangle rules for for individual speed limits.

example PCQ queue type to set a 3m/1m package;
/queue type
add kind=pcq name="3m per user down" pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-limit=35 pcq-rate=3M pcq-src-address6-mask=64 pcq-total-limit=10000
add kind=pcq name="1m per user up" pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-limit=35 pcq-rate=1M pcq-src-address6-mask=64 pcq-total-limit=10000
and then a simple queue for that speeds subnet;
/queue simple
add name="user limit" queue="1m per user up/3m per user down" target=10.0.0.0/22

Then for the more complicated stuff; use the mangle + queue tree to prioritize traffic types. Here is an example qos scheme I use, there is some basic and a bit of l7 stuff sprinkled in there.
/ip firewall mangle
add action=mark-packet chain=forward comment="DNS Uploads" dst-port=53 new-packet-mark=p1_up passthrough=no protocol=tcp src-address-list=public
add action=mark-packet chain=forward comment="DNS Downloads" dst-address-list=public new-packet-mark=p1_down passthrough=no protocol=tcp src-port=53
add action=mark-packet chain=forward comment="DNS Uploads" dst-port=53 new-packet-mark=p1_up passthrough=no protocol=udp src-address-list=public
add action=mark-packet chain=forward comment="DNS Downloads" dst-address-list=public new-packet-mark=p1_down passthrough=no protocol=udp src-port=53
add action=mark-connection chain=forward comment="VOIP Connection mark SIP" dscp=26 layer7-protocol=sip new-connection-mark=VOIP26 protocol=udp
add action=mark-packet chain=forward comment="SIP Uploads" connection-mark=VOIP26 new-packet-mark=p2_up passthrough=no src-address-list=public
add action=mark-packet chain=forward comment="SIP Downloads" connection-mark=VOIP26 dst-address-list=public new-packet-mark=p2_down passthrough=no
add action=mark-connection chain=forward comment="VOIP Connection mark RTP" dscp=46 layer7-protocol=RTP-accurate new-connection-mark=VOIP46
add action=mark-packet chain=forward comment="RTP Uploads" connection-mark=VOIP46 new-packet-mark=p2_up passthrough=no src-address-list=public
add action=mark-packet chain=forward comment="RTP Downloads" connection-mark=VOIP46 dst-address-list=public new-packet-mark=p2_down passthrough=no
add action=mark-connection chain=forward comment="HTTP Download conn mark" dst-address-list=public new-connection-mark=http_down protocol=tcp src-port=80,443,8080,81
add action=mark-connection chain=forward comment="HTTP upload conn mark" dst-port=80,443,8080,81 new-connection-mark=http_up protocol=tcp src-address-list=public
add action=mark-packet chain=forward comment=HTTP-Download connection-mark=http_down new-packet-mark=p3_down passthrough=no
add action=mark-packet chain=forward comment=HTTP-Upload connection-mark=http_up new-packet-mark=p3_up passthrough=no
add action=mark-connection chain=forward comment="E-mail Connection" dst-port=25,110,143,465,585,587,993,995 new-connection-mark=Email protocol=tcp
add action=mark-packet chain=forward comment="E-mail Upload" connection-mark=Email new-packet-mark=p3_up passthrough=no src-address-list=public
add action=mark-packet chain=forward comment="E-mail Downloads" connection-mark=Email dst-address-list=public new-packet-mark=p3_down passthrough=no
add action=mark-connection chain=forward comment="Xbox + PS3 Download tcp" new-connection-mark=xbox+ps3_down protocol=tcp src-port=3074,5223,3074
add action=mark-connection chain=forward comment="Xbox + ps3 download UDP" dst-address-list=public new-connection-mark=xbox+ps3_down protocol=udp src-port=88,3074,3478,3479,3658
add action=mark-connection chain=forward comment="Xbox+ps3 Up TCP" dst-port=3074,5223,3074 new-connection-mark=xbox+ps3_Up protocol=tcp
add action=mark-connection chain=forward comment="Xbox + PS3 up UDP" dst-port=88,3074,3478,3479,3658 new-connection-mark=xbox+ps3_Up protocol=udp
add action=mark-packet chain=forward comment="Xbox+ps3 Download" connection-mark=xbox+ps3_down dst-address-list=public new-packet-mark=p3_down passthrough=no
add action=mark-packet chain=forward comment="Xbox&ps3 Upload" connection-mark=xbox+ps3_Up new-packet-mark=p3_up passthrough=no src-address-list=public
add action=mark-connection chain=forward comment="Mark p2p with connection-mark" disabled=yes new-connection-mark=p2p_con p2p=all-p2p
add action=mark-packet chain=forward comment="P2P up" connection-mark=p2p_con disabled=yes new-packet-mark=p5_up passthrough=no src-address-list=public
add action=mark-packet chain=forward comment="p2p Down" connection-mark=p2p_con disabled=yes dst-address-list=public new-packet-mark=p5_down passthrough=no
add action=mark-packet chain=forward comment="Mark all remaining Upload." new-packet-mark=p4_up out-bridge-port=ether1 passthrough=no src-address-list=public
add action=mark-packet chain=forward comment="Mark all remaining Download" dst-address-list=public in-bridge-port=ether1 new-packet-mark=p4_down passthrough=no
/queue tree
add max-limit=30M name=Uploads_Full parent=global priority=1 queue=pcq-upload-default
add limit-at=256k max-limit=30M name=UP_priority1 packet-mark=p1_up parent=Uploads_Full priority=1 queue=pcq-upload-default
add limit-at=768k max-limit=30M name=UP_priority2 packet-mark=p2_up parent=Uploads_Full priority=2 queue=pcq-upload-default
add limit-at=8M max-limit=29M name=UP_priority3 packet-mark=p3_up parent=Uploads_Full priority=3 queue=pcq-upload-default
add limit-at=2M max-limit=3M name=UP_priority4 packet-mark=p4_up parent=Uploads_Full priority=4 queue=pcq-upload-default
add limit-at=1M max-limit=1M name=UP_priority5 packet-mark=p5_up parent=Uploads_Full priority=5 queue=pcq-upload-default
add max-limit=30M name=Downloads_Full parent=global priority=1 queue=default
add limit-at=256k max-limit=30M name=DN_priority1 packet-mark=p1_down parent=Downloads_Full priority=1 queue=pcq-download-default
add limit-at=768k max-limit=30M name=DN_priority2 packet-mark=p2_down parent=Downloads_Full priority=2 queue=pcq-download-default
add limit-at=25M max-limit=29M name=DN_priority3 packet-mark=p3_down parent=Downloads_Full priority=3 queue=pcq-download-default
add limit-at=2M max-limit=3M name=DN_priority4 packet-mark=p4_down parent=Downloads_Full priority=4 queue=default
add limit-at=1M max-limit=1M name=DN_priority5 packet-mark=p5_down parent=Downloads_Full priority=5 queue=pcq-download-default
 
troy
Member
Member
Topic Author
Posts: 320
Joined: Thu Jun 30, 2005 6:47 pm

Re: Bandwidth Management & QOS - Is it possible?

Wed Nov 20, 2013 9:19 pm

Seriously, I'm lost... not sure why all this isn't working.

Here, I mangle. It works and works great. (there are actually some other rules that catch unlisted addresses and gives them a default packet-mark)

ros code

/ip firewall mangle
add action=mark-packet chain=prerouting in-interface=LAN_Bridge new-packet-mark=Gold packet-mark=no-mark src-address-list=Gold
add action=mark-packet chain=postrouting dst-address-list=Gold new-packet-mark=Gold out-interface=LAN_Bridge packet-mark=no-mark
add action=mark-packet chain=prerouting in-interface=LAN_Bridge new-packet-mark=Silver packet-mark=no-mark src-address-list=Silver
add action=mark-packet chain=postrouting dst-address-list=Silver new-packet-mark=Silver out-interface=LAN_Bridge packet-mark=no-mark
add action=mark-packet chain=prerouting in-interface=LAN_Bridge new-packet-mark=Bronze packet-mark=no-mark src-address-list=Bronze
add action=mark-packet chain=postrouting dst-address-list=Bronze new-packet-mark=Bronze out-interface=LAN_Bridge packet-mark=no-mark
Then, I define some queue types. These are the same queue types I use with a Queue Tree, and they work flawlessly.

ros code

/queue type
add kind=pcq name=pcq_bronze_down pcq-burst-rate=1152k pcq-burst-threshold=512k pcq-burst-time=2m pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=768k pcq-src-address6-mask=64
add kind=pcq name=pcq_bronze_up pcq-burst-rate=192k pcq-burst-threshold=64k pcq-burst-time=3m pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=128k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_silver_up pcq-burst-rate=384k pcq-burst-threshold=128k pcq-burst-time=3m pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=256k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_silver_down pcq-burst-rate=2304k pcq-burst-threshold=758k pcq-burst-time=3m pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=1536k pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_gold_down pcq-burst-rate=4M pcq-burst-threshold=2M pcq-burst-time=3m pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=3M pcq-src-address6-mask=64 pcq-total-limit=30000
add kind=pcq name=pcq_gold_up pcq-burst-rate=768k pcq-burst-threshold=384k pcq-burst-time=2m pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=512k pcq-src-address6-mask=64 pcq-total-limit=30000
Finally, the simple queues. They appear to work, though I get a dozen or more queues in Gold, even tough I only have 5 addresses in the last and getting marked. In addition, when testing from an address that should be in one of the queues, I do not get throttled at all.

ros code

/queue simple
add limit-at=16M/16M max-limit=20M/20M name=Total_Bandwidth queue=ethernet-default/ethernet-default target=LAN_Bridge
add name=Gold packet-marks=Gold parent=Total_Bandwidth queue=pcq_gold_down/pcq_gold_up target=LAN_Bridge
add name=Silver packet-marks=Silver parent=Total_Bandwidth queue=pcq_silver_down/pcq_silver_up target=LAN_Bridge
add name=Bronze packet-marks=Bronze parent=Total_Bandwidth queue=pcq_bronze_down/pcq_bronze_up target=LAN_Bridge
If anyone can help me out, I'd sure appreciate it!
 
whoknew
Member Candidate
Member Candidate
Posts: 153
Joined: Wed Oct 13, 2010 8:51 pm

Re: Bandwidth Management & QOS - Is it possible?

Wed Nov 20, 2013 10:09 pm

derr12,

can you elaborate what you have in the address-list=public.

I too only use pcq for individual bandwidth queues but I'm look to give ea better browsing experience to my users.

Thanks.
 
troy
Member
Member
Topic Author
Posts: 320
Joined: Thu Jun 30, 2005 6:47 pm

Re: Bandwidth Management & QOS - Is it possible?

Thu Nov 21, 2013 1:17 am

WhoKnew,

I'd imagine that the address list you're asking about, is a catch-all list that includes the IPs and/or subnets all his clients are on.

This is how I do it, anyways... list for all subnets on the network, different lists for different nat pools, list for different bandwidth classes, list for infrastructure devices, list for CPE devices, list for client IPs, etc...

With the exception of 1:1 NAT and some port forwarding, I do use hardly any IP addresses in my firewall/filter/nat/mangle configuration. For example, here's a port forward I use to get into one device:

ros code

/ip firewall nat
add action=dst-nat chain=dstnat dst-address-list=ADMIN_NAT dst-port=1733 \
    protocol=tcp to-addresses=10.xx.yy.zz to-ports=443
 
whoknew
Member Candidate
Member Candidate
Posts: 153
Joined: Wed Oct 13, 2010 8:51 pm

Re: Bandwidth Management & QOS - Is it possible?

Thu Nov 21, 2013 4:20 pm

Troy, I figured it was his internal lan subnets, wasn't for sure, I usually do not use address list.

I am thinking of giving this a go as well, I will use a simple queue to limit the total bandwidth download and upload, then begin using the queue tree for QoS. Why does DNS take such a high priority? Also the P2P does that really catch much anymore? It seems like the http rule (80,442,8080,81) would catch the torrent traffic. Also the marking of the all remaining wouldn't that catch UDP gaming traffic last?

Also the xbox+playstation would only apply in the event that you are using 1:1 nat on every internal connection?
 
troy
Member
Member
Topic Author
Posts: 320
Joined: Thu Jun 30, 2005 6:47 pm

Re: Bandwidth Management & QOS - Is it possible?

Thu Nov 21, 2013 6:00 pm

I don't have the expertise to go into depth about QOS (yet), but DNS is a pretty critical service. Everything else depends on it.

For the same reason, ICMP and certain types/sizes of TCP packets should have super-high priority.

When you want to pull up a web page, even a few hundred milliseconds can make a difference in the customer experience. If web pages take more than a second to start loading, it's likely a DNS issue, easily resolved by making sure that DNS traffic runs as fast as possible.

I really wish someone had a solution for torrents over port 80/443/etc... If, as an industry, we would have left the P2P crowd alone instead of trying to block them, we wouldn't have this problem. I'd much rather let them do their thing as long as it doesn't interfere with more interactive traffic. When I identify heavy torrent users, I encourage them to enable a scheduler to limit their day/evening traffic, but let it go wide-open at night. They actually thank me for it, since it also improves their own online experience.

For the gamers, I don't know what the current platforms/engines are doing, but I remember playing things like Quake, UT, HL, and others... it was all about small control messaging moving across the network in a fast, orderly manner. It was not a bandwidth hog. So, if you want to keep your gamers happy, give them high priority, but low bandwidth. Same way you would do for VOIP... higher priority with just enough bandwidth to get the job done.

--

UGH! Still nobody to help me with replacing my Queue Tree with simple queues? Please! There's gotta be someone out there smart enough to figure this out!
 
Zod
Frequent Visitor
Frequent Visitor
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: Bandwidth Management & QOS - Is it possible?

Tue Nov 26, 2013 9:47 pm

In addition, when testing from an address that should be in one of the queues, I do not get throttled at all.
Reason Simple Queues are not working is right here
chain=postrouting
From the manual:
.... Take look at the RouterOS packet flow diagram. It is necessary to mark packets before the simple queues (before global-in HTB queue) or else target's download limitation will not work. The only mangle chain before global-in is <b>prerouting</b>.
 
troy
Member
Member
Topic Author
Posts: 320
Joined: Thu Jun 30, 2005 6:47 pm

Re: Bandwidth Management & QOS - Is it possible?

Wed Nov 27, 2013 1:23 am

Thanks for taking a look man, I do appreciate it, but I've already been there. Following the link, which takes you to this page, you can find exactly one diagram that shows Simple Queues, and has a description stating that this is how it works in ROS6.

Image

Looking at this, Postrouting does do mangle before simple queue, and there is no mangle in prerouting.

I see 3 options: 1) I'm missing something, 2) the queue manual is wrong, or 3) the packet flow diagram is wrong.

Where to look next?
 
Zod
Frequent Visitor
Frequent Visitor
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: Bandwidth Management & QOS - Is it possible?

Wed Nov 27, 2013 1:58 pm

Are you using ROS 6.x ?
 
troy
Member
Member
Topic Author
Posts: 320
Joined: Thu Jun 30, 2005 6:47 pm

Re: Bandwidth Management & QOS - Is it possible?

Wed Nov 27, 2013 2:12 pm

As indicated in the OP, I am.

Was your earlier reply based on an earlier version?
 
Zod
Frequent Visitor
Frequent Visitor
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: Bandwidth Management & QOS - Is it possible?

Wed Nov 27, 2013 8:01 pm

Yes. Sorry, I missed that. I have not rolled 6.x out, or even benched it yet - I'm not an 'early adopter' kind of guy.
 
an3
just joined
Posts: 1
Joined: Thu Jun 29, 2017 5:57 am

Re: Bandwidth Management & QOS - Is it possible?

Thu Jun 29, 2017 6:24 am

Hi, can you solve this problem, using a "QOS" queue into a " bandwidth " Queue ?

Thanks
Andres

Who is online

Users browsing this forum: Ahrefs [Bot], cddaraa, Google [Bot], GoogleOther [Bot], onion83 and 39 guests