Community discussions

MikroTik App
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Using RouterOS to QoS your network - 2020 Edition

Fri May 31, 2013 9:43 pm

Title:
Using RouterOS to QoS your network - 2020 Edition

Welcome:
The following article is a high-level introduction to a QoS implementation using MikroTik RouterOS. Quality of Service is a large topic. So, this short article will not attempt to explain all edge cases, compare the many algorithms, or provide deep context on packet prioritization. However, it is possible to achieve good - even fantastic results - by creating simple classifications and actions on the most common traffic flows. The configuration presented here is suitable for small business, home, ip telephony, and gaming environments where a single device is providing QoS management.

Why prioritize traffic?
Generally speaking because there is network contention. This occurs most commonly because two or more applications are requesting enough data to exceed the interface. Maybe you want to plan ahead knowing there will be congestion. Even when individual applications and protocols are managing themselves well, they are not aware of the effect they are having on the rest of the network. QoS is then a network governor watching all packet flows and making good decision for everyone. Since network interfaces operate in a serial manner, interactive traffic will be waiting on the many packets ahead of it from big bulky traffic. Even if you could afford to add more Internet connections and more routers, it is still possible to overwhelm them. Therefore, prioritizing your network is a QoS mechanism to manage the different types of traffic flows.

Traffic Types:
You need to classify at least three: interactive, network, and then bulky. For the purposes of this article, VoIP packets are interactive traffic and considered the most important. The network traffic consists of DNS, ICMP, and ACK packets. For the bulky category we have HTTP and QUIC. We also have a catch all for everything else which gets the lowest priority. When our highly interactive traffic is occurring we will ensure it is never impeded by the other types. Indeed, all other traffic types will have secondary importance for the duration of VoIP packet flows, but only when the network is under threat of congestion. Use our model as a guide and create your own categories.

How to identify Traffic Types:
There are actually quite a few ways. Some applications have standard port numbers like DNS. Maybe you have equipment setting DSCP bits for you. You could also check ip addresses, mac addresses, or even VLAN IDs to know the importance of packets coming from those locations. It is also possible to check byte rates to identify streaming traffic. Knowing the types of applications in use and their bandwidth requirements will help you to correctly know what is important or at least what category it should go in.

TrafficTypes.png

How Interfaces and QoS work together:
It is helpful to understand a little bit about interfaces, queuing, and shaping before we jump right into the implementations. Think of interfaces as buckets and packets as different color liquids. These buckets have a drain port at the bottom to let out the liquid. Nozzles are pouring red, green, and blue liquid into our bucket. Thus we have two considerations: the speed of the drain port and the size of the bucket.

If five packets arrive every 5 minutes, it is easy to reason that our bucket can handle that just fine. But if 10,000 packets arrive every second we are going to have a problem. We could speed up the port, say with a 100GbE interface. But there are side-effects to doing that and it is not always affordable to go with faster interfaces. We could get a bigger bucket. A bucket so big that it could hold an elephant. Unfortunately we have the situation to where the last drop of red may take too long waiting for all that blue to drip out. No matter the speed of our drain port or the size of the bucket, it can be so utilized that it can not keep up with incoming data. Our bucket can overflow, throwing away some of the packets.

When we QoS packets, we do make use of port and bucket characteristics, but we also notify those pouring nozzles to release in a more responsible way for our capacity. If they don't, then we take matters into our own hands to ensure the packets we care about the most don't overflow. With QoS this is done by dropping packets. Naturally, some packets can't be dropped without effecting the application experience. We plan accordingly.

Disclaimer:
What follows is my best understanding of how to implement the stated goals. FastTrack must not be enabled. Feedback from MikroTik as well as fellow forum members is required to make this an accurate document. Please suggest changes that should be made. Let's make this issue a commonly understood one. Special thanks to bharrisau for testing and feedback.
You do not have the required permissions to view the files attached to this post.
Last edited by pcunite on Thu Apr 16, 2020 10:30 pm, edited 22 times in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri May 31, 2013 9:43 pm

Implementing traffic prioritization (QoS) with RouterOS

To turn on the QoS capabilities of RouterOS, we implement two things: marking and then queuing.

The Marking Stage

How to Mark the Traffic Types:
RouterOS supplies the Mangle feature to mark packets. What you decide to mark is up to personal and business decisions. Here is a sample starting point. It can be appropriate to mark items, interesting to you, that will ultimately go into the same queue. This is useful for network monitoring purposes. The POP3 mark is an example of that.

Take time to get your marking correct. Test to ensure you are seeing the totals move as you expect. At this stage, we are only marking items. We will use another command to take actions on these marks.

/ip firewall mangle
# Identify DNS on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes comment="DNS"
add chain=prerouting  action=mark-packet     connection-mark=DNS  new-packet-mark=DNS passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes
add chain=postrouting action=mark-packet     connection-mark=DNS  new-packet-mark=DNS passthrough=no

# Identify VoIP
add chain=prerouting  action=mark-connection new-connection-mark=VOIP port=5060-5062,10000-10050 protocol=udp passthrough=yes comment="VOIP"
add chain=prerouting  action=mark-packet     connection-mark=VOIP new-packet-mark=VOIP passthrough=no

# Identify HTTP/3 and Google's QUIC
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=QUIC port=80,443 protocol=udp passthrough=yes comment="QUIC"
add chain=prerouting  action=mark-packet     connection-mark=QUIC new-packet-mark=QUIC passthrough=no

# Identify UPD. Useful for further analysis. Should it be considered high priority or put in the catchall? You decide.
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=UDP protocol=udp passthrough=yes comment="UDP"
add chain=prerouting  action=mark-packet     connection-mark=UDP  new-packet-mark=UDP passthrough=no

# Identify PING on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes comment="ICMP"
add chain=prerouting  action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes
add chain=postrouting action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no

# Identify Acknowledgment packets
add chain=postrouting action=mark-packet     new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no comment="ACK"
add chain=prerouting  action=mark-packet     new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no

# Identify HTTP traffic but move it to a Streaming mark if necessary.
add chain=prerouting  action=mark-connection connection-mark=no-mark  connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp passthrough=yes comment="HTTP"
add chain=prerouting  action=mark-connection connection-bytes=5M-0    connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG protocol=tcp passthrough=yes
add chain=prerouting  action=mark-packet     connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add chain=prerouting  action=mark-packet     connection-mark=HTTP     new-packet-mark=HTTP passthrough=no

# Email goes to the catchall
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=POP3 port=995,465,587 protocol=tcp passthrough=yes comment="OTHER"
add chain=prerouting  action=mark-packet     connection-mark=POP3 new-packet-mark=OTHER passthrough=no

# Unknown goes to the catchall
add chain=prerouting  action=mark-connection connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add chain=prerouting  action=mark-packet     connection-mark=OTHER   new-packet-mark=OTHER passthrough=no


The Queuing Stage

How to act on Traffic Marks:
RouterOS supplies the Queue Tree structure that enable us to act on marks. This is how we truly classify the packet flows on the network. A whole book could be written on what is occurring here. There are many options one could use to dial in a very custom Queue Tree. The purpose of this article, however, is to present a simple yet very effective implementation. A few things do need to be understood.

Max-limit:
In order for queuing to occur in our equipment, and thus give us the control on packet flows, we have to set our interfaces to operate at 10% the rate of our ISP connection. This is only a starting number and is dependent upon your CPU speed and simultaneous connections. Apply to both the upload and download links. This way buffering always occurs inside of our equipment. The max-limit parameter is required for the algorithms to function and must not be 0. In our example, we have 100M service, so we have set it to 90M.

Limit-at:
This option is not something you will use commonly and is therefore recommended to leave it at 0 (disabled). However, there is a very special situation where you must enable it. Read the Protection with Limit-at section to learn more.

bucket-size:
During congestion, this value sets the amount of tokens to accrue before the chosen queue type takes effect. This is an equation and means that after Max-limit is reached (bucket-size * max-limit) worth of bytes will be engaged by the queue type. For our purposes, we only want a small amount of time addressing packets going over the limit, enough to smooth out any protocol windowing.


/queue tree

# DOWN
add name=DOWN max-limit=90M parent=LAN bucket-size=0.01 queue=default

add name="1. VOIP"     packet-mark=VOIP     parent=DOWN priority=1 queue=default
add name="2. DNS"      packet-mark=DNS      parent=DOWN priority=2 queue=default
add name="3. ACK"      packet-mark=ACK      parent=DOWN priority=3 queue=default
add name="4. UDP"      packet-mark=UDP      parent=DOWN priority=3 queue=default
add name="5. ICMP"     packet-mark=ICMP     parent=DOWN priority=4 queue=default
add name="6. HTTP"     packet-mark=HTTP     parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC"     packet-mark=QUIC     parent=DOWN priority=7 queue=default
add name="9. OTHER"    packet-mark=OTHER    parent=DOWN priority=8 queue=default


# UP
add name=UP max-limit=90M parent=WAN bucket-size=0.01 queue=default

add name="1. VOIP_"     packet-mark=VOIP     parent=UP priority=1 queue=default
add name="2. DNS_"      packet-mark=DNS      parent=UP priority=2 queue=default
add name="3. ACK_"      packet-mark=ACK      parent=UP priority=3 queue=default
add name="4. UDP_"      packet-mark=UDP      parent=UP priority=3 queue=default
add name="5. ICMP_"     packet-mark=ICMP     parent=UP priority=4 queue=default
add name="6. HTTP_"     packet-mark=HTTP     parent=UP priority=5 queue=default
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="8. QUIC_"     packet-mark=QUIC     parent=UP priority=7 queue=default
add name="9. OTHER_"    packet-mark=OTHER    parent=UP priority=8 queue=default
Last edited by pcunite on Mon Aug 31, 2020 9:31 pm, edited 33 times in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri May 31, 2013 9:44 pm

Additional Information:
Since QoS is such a big topic, additional information is presented here for those who want to dive a little deeper. While the example configuration is simple to understand and explain, we should go further in the real world. Also read the Bufferbloat article.

Protection with Limit-at:
In the event that traffic volume, for one particular queue, reaches at least 51% of the Max-limit value, that traffic will be subject to packet prioritization. This means packets might get dropped. This is normal and the whole point of what we are doing. But, what if that queue is your high priority queue? There is a way to carve out some protection for these special packets.

To illustrate how this is done, we'll demonstrate with VoIP packets flowing in our VOIP queue. Each VoIP call is about 86Kb, both directions. Let's say we have to protect 700 active calls to support a business contract. At 86Kb, that works out to be about 60M, which is indeed over 51% of our 90M Max-limit. Thus we set a Limit-at of 60M, on the VOIP queue only (both up & down). This will prevent those packets from ever being dropped. If you only ever expect 10 VoIP calls then Limit-at would be unnecessary.

Protection for low priority queues with Limit-at:
Why offer any protection for a low priority queue? If you have enough bandwidth to go around, it is a very good idea to carve out a minimum amount of bandwidth for busy queues. Otherwise, all higher priority queues will take nearly everything under ultra-high contention events. Thus, you could ensure at least 10M or more if you can afford it. QUIC and HTTP_BIG are queues that could use some level of protection.

Limit-at Qualifiers:

  • You can't protect more bandwidth than you actually have.
  • Protection for one queue (only when under pressure) naturally forces the other queues to battle over what's left.
  • Attempting to protect more than 90% is risky. Better to increase bandwidth instead.
  • Not everything on your network should be high priority. If so, then in reality, you only have one queue.

Implement like so:

# Set protection on VOIP queue, both directions. Also some for HTTP_BIG.
/queue tree 
add limit-at=60M max-limit=90M name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add limit-at=60M max-limit=90M name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add limit-at=10M max-limit=90M name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=7 queue=default
add limit-at=10M max-limit=90M name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=7 queue=default


Optimizing the HTTP, HTTP_BIG, and QUIC queues:
Any queue that frequently hits the Max-limit value and has packets using global synchronization, Cubic, NewReno, or similar congestion methods, will benefit from a better queuing mechanism. Therefore, it is recommended to use RED (Random Early Detection) on these queue types. Note that WRED is an often mentioned improved version that looks at priority data in the packet. However, since we are placing packets in their own special queue anyway, we have already classified what is important.

Implement like so:

/queue type

# default queue behavior
set default kind=sfq

# queue behavior for streaming type traffic
add kind=red red-avg-packet=1514 name=redCustom

# example of how to use red, optionally set for all bulky traffic types
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=redCustom
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=redCustom
Last edited by pcunite on Sun Apr 19, 2020 8:29 pm, edited 37 times in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri May 31, 2013 9:45 pm

How Buffering and Bufferbloat in ISP supplied equipment affects QoS and latency sensitive protocols

Understanding this phenomenon is really tricky. It helps to know how applications, network controllers, protocols, and memory all behave in isolation. As networks have grown in size and complexity, all these elements have accidentally worked together to increase Latency, the enemy of networking.

Applications:
Software applications send their data to receivers in chunks or packets about 1Kb in size. If Application 1A sends a packet destined for Application 1B, it waits for a moment, to see if it gets a reply. During this wait, other applications may try to say something. Now we have an interleaving amount of traffic coming and going. To make full use of this highway (bandwidth), applications start slowly sending data. As they get acknowledgment replies back that they are being heard, they increase the rate of packets they send. Applications try to send as much as they can.

Controllers:
Network interfaces, whether exposed as a Cat5-7, fiber ports, or wifi, are connected to a controller that is ultimately managed by low level software. This implies memory, and is what is used to hold the incoming bits that eventually make up an individual packet. As the bits come in, one by one, a whole packet is realized as it comes in off the wire. The first packet might belong to Application 1. The next packet might belong to Application 2. The next 30 packets might all belong to Application 3. A natural question might then be: how many packets should network controllers, routers, and switches store?

Protocols:
When a sending application does not receive a reply (ACK packet), within a certain time window, it begins to slow down its sending rate. It will slow down until it is heard, at which point is starts increasing again. It can enter a timeout period if it is never heard. If this happens to a graphical app, it may appear to freeze. If a file download is in progress, timeouts make the network seem "slow" or even "down". The reply window and timeouts work well when there is one type application on the network. But today, there is almost never one type of application or even one type of packet flow.

Memory:
Memory is plentiful these days. Since packets might only be 1Kb in size, it is cheap to hold a lot of them. But it stands to reason that at some point, room for holding packets is going to run out. When that happens, the network controllers start to drop packets and their buffers overflow. As applications realize they are not getting acknowledgments they begin to slow down too freeing up memory pressure. The cycle is then repeated.

PacketPath.png

Everything described so far is correct behavior, as designed, and works well under certain conditions. However, conditions and what is expected of a network have changed. Networks are not used to simply download things or view web pages any more. Increasingly, there is interactive traffic.

The interactive packet:
As packets get stored by the network, every so often a tiny little packet needs to be sent. These come from applications that want to send and reply in a loop with small amounts of data indicating state. Yet, the network is completely full causing any new packet to wait behind long lines of bulky traffic. Because of this, controllers are overflowing their buffers all along the way. Our small packets are getting dropped too. We are now experiencing Latency.

Latency:
The amount of time for a packet to get its reply. This is what makes a network feel fast or slow. There will always be a delay but how much is too much? For certain applications, that is about 150ms. If these packets are for voice or game data, the end user will think the network is unstable. Yet, big downloads, running in the background or on other system, finish and no one perceives any issues.

Buffer collapse:
Because of the way networks store packets in buffers, sending applications get bandwidth and priority that should not be available to them. When buffers along the way are seemly accepting everything being sent, applications increase their packet flow rates, never leaving room for interactive applications. At some point in time, random buffers begin to be completely maxed out. Packets get dropped, acknowledgments are not sent. Sending applications slow down. Unfortunately, they all do this at once. Then all applications ramp back up, then they all slow down again. The network is weaving back and forth inefficiently. This affects bulky traffic, and most certainly never properly allows for interactive traffic to work as needed.

Solution:
On the ISP side, buffers sized appropriately and matched to processor, controller, and bandwidth sizes and speeds. On the client side, Active Queue Management techniques as shown in this article series. The ISP may never know what you consider to be your most important traffic, so for both your up-link and down-link interfaces, you must ensure to manage your own buffers, control what overflows, and determine which packets should be dropped to signal packet rates.
You do not have the required permissions to view the files attached to this post.
Last edited by pcunite on Thu Jan 09, 2020 1:21 am, edited 14 times in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Jun 01, 2013 6:02 pm

Reserved
Last edited by pcunite on Wed Jan 08, 2020 7:19 pm, edited 19 times in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Jun 01, 2013 6:02 pm

Reserved
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Jun 04, 2013 4:17 pm

great info, but it's better to use http://wiki.mikrotik.com for such articles - forum is more for questions, not for tutorials :)
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Jun 04, 2013 4:22 pm

great info, but it's better to use http://wiki.mikrotik.com for such articles - forum is more for questions, not for tutorials :)
Thank you. My intention was to perfect this post and then have it accepted by MikroTik after all the experts had confirmed it. Then, as you say, have it posted in the Wiki.
Last edited by pcunite on Tue Jun 04, 2013 10:26 pm, edited 1 time in total.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 26894
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia
Contact:

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Jun 04, 2013 4:24 pm

great info, but it's better to use http://wiki.mikrotik.com for such articles - forum is more for questions, not for tutorials :)
Thank you. My intension was to perfect this post and then have it accepted by MikroTik after all the experts had confirmed it. Then, as you say, have it posted in the Wiki.
Good idea. Waiting for next parts.
 
deejayq
Member Candidate
Member Candidate
Posts: 195
Joined: Wed Feb 23, 2011 8:33 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Mon Jul 01, 2013 1:30 pm

# HELP! I don't know how to mark what is left?
just add two rules at the bottom of the script
first mark-connection with dst-address=0.0.0.0/0 (nothing else)
second mark-packet with connection mark you set on the first rule (BULK i think you named both marks).
 
theprism
newbie
Posts: 29
Joined: Sun Sep 16, 2012 4:11 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sun Jul 14, 2013 3:48 am

It doesn't work.

Here's what I have:
****************************************
/ip firewall mangle

add chain=forward action=mark-connection src-address=192.168.151.7 new-connection-mark=VOIP
add chain=forward action=mark-packet passthrough=yes connection-mark=VOIP new-packet-mark=VOIP
add chain=forward action=mark-connection dst-address=192.168.151.7 new-connection-mark=VOIP
add chain=forward action=mark-packet passthrough=no connection-mark=VOIP new-packet-mark=VOIP

add chain=forward action=mark-connection dst-address-list=IPTVlist new-connection-mark=IPTV
add chain=forward action=mark-packet passthrough=yes connection-mark=IPTV new-packet-mark=IPTV
add chain=forward action=mark-connection src-address-list=IPTVlist new-connection-mark=IPTV
add chain=forward action=mark-packet passthrough=no connection-mark=IPTV new-packet-mark=IPTV

add chain=forward action=mark-connection src-address=0.0.0.0/0 new-connection-mark=BULK
add chain=forward action=mark-packet passthrough=yes connection-mark=BULK new-packet-mark=BULK
add chain=forward action=mark-connection dst-address=0.0.0.0/0 new-connection-mark=BULK
add chain=forward action=mark-packet passthrough=no connection-mark=BULK new-packet-mark=BULK

/queue tree

add name="TOTAL_U" parent=pppoe-out1 queue=default priority=8 limit-at=0 max-limit=680k
add name="TOTAL_D" parent=bridge-homelan queue=default priority=8 limit-at=0 max-limit=4300k

add name="VOIP_U" parent=TOTAL_U packet-mark=VOIP queue=default priority=1
add name="VOIP_D" parent=TOTAL_D packet-mark=VOIP queue=default priority=1

add name="IPTV_U" parent=TOTAL_U packet-mark=IPTV queue=default priority=2
add name="IPTV_D" parent=TOTAL_D packet-mark=IPTV queue=default priority=2

add name="BULK_U" parent=TOTAL_U packet-mark=BULK queue=default priority=8
add name="BULK_D" parent=TOTAL_D packet-mark=BULK queue=default priority=8

/ip firewall address-list
add address=184.84.243.193 list=IPTVlist
add address=4.27.18.126 list=IPTVlist
****************************************

In mangle I see exactly same counts for src and dst on each marking portion, which makes me think that mangles shouldn't be configured like that.
/ip firewall mangle screenshot is attached

Also prioritization doesn't work, since if I start my IPTV only, it works fine. Now, while IPTV is running, if I start a download, my IPTV is screwd. Image is distorted and become impossible to view and hear the audio.
/queue tree screenshot is attached

Any solutions?

Thanks,
T.P.
You do not have the required permissions to view the files attached to this post.
 
WiPoint
just joined
Posts: 5
Joined: Fri Nov 16, 2012 6:47 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Aug 29, 2013 3:05 pm

Hi from your Web Mangle Mark up rule MAX packet size has to to be in the range of 0-65535. Any advice on what to set it as?
ROS6.2 RB1100AHx2
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Aug 31, 2013 12:56 am

We need someone from MikroTik (or popular online blogger) to help with these rules. These are what I was able to create based on documentation. Qos is a very important feature and one of the main reasons someone would want one of these advanced routers. Let's hope they chime in and we can build a good default script.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Aug 31, 2013 6:15 am

Hello everyone. I've tested and updated the script. It now works correctly on RouterOS 6.1. Note that ether1 is WAN and ether2 is LAN. Adjust those as necessary for your environment.

I would appreciate if someone could tell me how to mark big downloads over HTTP traffic. Currently, the script marks port 80 so everything HTTP gets too much priority. The idea situation would be to let short bursts of HTTP traffic get high priority and the big long downloads get less.
 
User avatar
whiely
just joined
Posts: 1
Joined: Wed Apr 27, 2011 2:33 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sun Sep 01, 2013 11:27 pm

Hello everyone. I've tested and updated the script. It now works correctly on RouterOS 6.1. Note that ether1 is WAN and ether2 is LAN. Adjust those as necessary for your environment.

I would appreciate if someone could tell me how to mark big downloads over HTTP traffic. Currently, the script marks port 80 so everything HTTP gets too much priority. The idea situation would be to let short bursts of HTTP traffic get high priority and the big long downloads get less.
with connection byte ?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Sep 05, 2013 3:14 am

>> how to mark big downloads over HTTP traffic?

With connection byte?
Thank you, the answer is to use connection-bytes and connection-rate.
 
TikUser
newbie
Posts: 48
Joined: Thu Jul 04, 2013 2:40 pm
Location: EU

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Sep 12, 2013 12:15 am

I’ve watched these 2 videos: HR13: QoS on RouterOS v6 and MUM US11: QoS workshop. In the second video (18:15min) Janis says: "Prerouting is a sum of (mangle) input and (mangle) forward. Both chains are together in here. Postrouting is a sum of (mangle) forward and (mangle) output."

Prerouting = input + forward
Postrouting = output + forward

In your Option 2 configuration You used prerouting and postrouting together. Will these two mangles come into conflict (because of 2 forwards)? If I’ve understood it correctly, in mangle postrouting we mark traffic that goes from the router (output) and through the router (forward). How will this affect the marked traffic from the prerouting and meant for forward? Will it be remarked?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Sep 12, 2013 12:55 am

Prerouting = input + forward
Postrouting = output + forward

In your Option 2 configuration You used prerouting and postrouting together. Will these two mangles come into conflict (because of 2 forwards)? If I’ve understood it correctly, in mangle postrouting we mark traffic that goes from the router (output) and through the router (forward). How will this affect the marked traffic from the prerouting and meant for forward? Will it be remarked?
I don't think the 2011 talk was referring to v6, maybe it was. But Prerouting not = input + forward. Look at the 3rd image here. The reason I use pre and post on DNS traffic is because the forward chain is not able to mark traffic sourced from the router itself and also traffic coming into the router from LAN.

Maybe there are other ways of accomplishing the same thing. Perhaps your question will reveal a better way. At the moment, this is all I know. However, when I was watching traffic from the connections menu, doing what I'm doing now was the only way to see it show up marked.
 
TikUser
newbie
Posts: 48
Joined: Thu Jul 04, 2013 2:40 pm
Location: EU

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Sep 12, 2013 1:48 am

I believe the presentation (Packet Flow and QoS v6) in the 1st video (from 7:16min) could help clarify the situation. This is for ROS v6.

All traffic from the Input Interface goes to Chain Prerouting. If we choose to mark it in Prerouting, from there on, all traffic going to Input and Forward is marked. So, in a way, Prerouting is a sum of Forward and Input.
The Chain Output will remain unmarked. Maybe this is the answer on how to mark traffic sourced from the router itself. Mark it in the mangle Chain Output?
 
jandafields
Forum Guru
Forum Guru
Posts: 1515
Joined: Mon Sep 19, 2005 6:12 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Mon Oct 07, 2013 6:15 am

Looking at OPTION 2 above....

In Queue A, you have voip = priority 1
In Queue B, you have ack = priority 1
In Queue C, you have http = priority 1

Since the parent queues don't have a priority, what actually makes "queue A priority 1 (voip)" a higher priority than "queue C priority 1 (http)" since they are both priority 1?

Isn't there a setting that must tell which parent has a higher priority than another parent, or how will it know?

Or, if priority is the wrong term, the real question is: What keeps queue C from using up all the bandwidth and not leaving any for queue A, since they are both maxed at 900k/4M?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Mon Oct 07, 2013 3:53 pm

Looking at OPTION 2 above....

In Queue A, you have voip = priority 1
In Queue B, you have ack = priority 1
In Queue C, you have http = priority 1
...
Or, if priority is the wrong term, the real question is: What keeps queue C from using up all the bandwidth and not leaving any for queue A, since they are both maxed at 900k/4M?
You are correct to wonder, and indeed there is something at issue here. I'm still perfecting this script. Basically what is happening currently, and why this script works for small networks is that: since Queue A is never full (never exceeds capacity) nothing is ever prioritized (dropped) within it. Queue C will always be exceeding capacity and thus traffic marked to go there will indeed be dropped (prioritized) at times. But as you've correctly noticed only at 900k/4M. This means that Queue A gets a 10% window of head room before ISP equipment starts to buffer.

What is the fix for this? I'm still researching but I think it must be a dynamic script using connection-rate and other actions. Let's review some wants.

1. Don't ever drop SIP and RTP (VoIP) packets.
2. Don't limit (drop) HTTP traffic until we absolutely have to.

To achieve the above we need to know how many SIP and HTTP (or random BULK) sessions are open (or their byte usage) at any given time. Thus a dynamic script ... when many SIP sessions are open we should increase the limits for HTTP (maybe only 100k/1M), when there are no SIP sessions, automatically allow HTTP to have 900k/4M usage again.

I've created this thread for people that know to tell us how to do this. So far, they've not see this thread and replied. But I do thank you for noticing.
 
jandafields
Forum Guru
Forum Guru
Posts: 1515
Joined: Mon Sep 19, 2005 6:12 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Mon Oct 07, 2013 4:21 pm

Ok, this command should tell you how many sip sessions are open. You might need to change dst-address to src-address, depending on the sip traffic:
/ip firewall connection print count-only where protocol="udp" and dst-address~":50.." 

~":50.." will match any port between 5000 and 5999

We could run that every xx seconds, then use the number returned by that command and then limit the other queues accordingly as we know that every sip session is probably around 100kbps in both directions.
 
User avatar
Craftylover
just joined
Posts: 1
Joined: Fri Jun 21, 2013 7:22 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Oct 15, 2013 4:27 am

Hello everyone.
It seems as though many of the QOS examples listed in the forum are individually tailored to a certain users needs and/or a particular application -this one just happens to be generic enough to make a little sense to me (completely new at this). :)
Might there be a way to add anything to this configuration that would help prioritize streaming video for a home network? Looks as though some of the Netflix traffic for example gets lost in the fray when there is heavy traffic marked as HTTP and the quality tends to suffer. I have seen some stuff pointing to L7 pattern matching, but I can honestly say that I do not know how to implement.
Any pointers would be appreciated.
Thank you.
 
ingjaab
just joined
Posts: 1
Joined: Tue Feb 22, 2011 8:56 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Oct 15, 2013 11:17 am

hello how would qos, wan, lan1, lan2 , LAN3, lan4, lan5, ... ..........., yes core has 12 ether

thanks
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Oct 15, 2013 6:23 pm

Might there be a way to add anything to this configuration that would help prioritize streaming video for a home network? Looks as though some of the Netflix traffic for example gets lost in the fray when there is heavy traffic marked as HTTP and the quality tends to suffer. I have seen some stuff pointing to L7 pattern matching, but I can honestly say that I do not know how to implement.
Great question. You must understand that the best way to Qos a network is to have enough bandwidth to work with. People want everything ... but that is a tall order.

Want: VoIP is perfect, file downloads are max speed, video streaming is perfect ... all at the same time!

Wow! I want that too but to achieve this you must have at least 10mb down. Why? Because streaming video is 5mb down! How many users do you have? It gets worse from there.

Currently, the script does find and mark video traffic as "HTTP_BIG". To have what you want you have two options. Keep the script as is but put "HTTP_BIG" into a bigger queue and ensure internet speed is 10mb down and hope that only one user is streaming video.

Or you need to separate video streaming from big downloads. Thus you could use L7 matching or IP's from hulu, YouTube, etc. Once you've separated out your traffic types I would put "VIDEO" into PCC queues. The more users streaming the better PCC will segment them. Naturally, you'll want 5MB per user ... but that is your problem.

You might also tweak what the script marks as "HTTP_BIG". Some sites (like pinterest) have so many images on a single page it looks like streaming video for a short time.
 
whowantstwoknow
just joined
Posts: 4
Joined: Fri Oct 11, 2013 1:55 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Oct 23, 2013 2:36 am

Hi there,

In my recent thread, part of my goal is to route VOIP traffic from the netgear TA612V through the routerOS router. Came across this thread, now forgive the noob question, but in the context of the scripts in this thread and the interfaces I have (see below), which interfaces are my ether-LAN and ether-WAN?

[admin@MikroTik] /interface> print
Flags: D - dynamic, X - disabled, R - running, S - slave
# NAME TYPE MTU L2MTU MAX-L2MTU
0 R ether1-gateway ether 1500 1598 4074
1 R ether2-master-local ether 1500 1598 4074
2 R ether3-slave-local ether 1500 1598 4074
3 ether4-slave-local ether 1500 1598 4074
4 ether5-slave-local ether 1500 1598 4074
5 R wlan1 wlan 1500 2290
6 R bridge-local bridge 1500 1598

Thanks in advance
W.
 
2fast4youbr
Member Candidate
Member Candidate
Posts: 113
Joined: Mon Apr 15, 2013 10:39 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Nov 22, 2013 10:19 pm

Just did and when a softphone from outside company tries to connect, we can listen anything...

any tips ?
 
2fast4youbr
Member Candidate
Member Candidate
Posts: 113
Joined: Mon Apr 15, 2013 10:39 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Nov 26, 2013 11:37 pm

Cant wait for version 3....

good job, working nice here!
 
arpiska
just joined
Posts: 22
Joined: Thu Jun 09, 2011 2:54 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Jan 23, 2014 2:07 pm

Please help me how to change this solution, if I have 2 WAN and several LAN interface?
- wan1 60M/3M , wan2 21M/5M
- vlan1, vlan2, vlan3... as local ethernet
 
TikUser
newbie
Posts: 48
Joined: Thu Jul 04, 2013 2:40 pm
Location: EU

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Mon Feb 10, 2014 3:20 am

[Solved]
Last edited by TikUser on Thu Feb 20, 2014 12:17 pm, edited 1 time in total.
 
mdkberry
Frequent Visitor
Frequent Visitor
Posts: 70
Joined: Tue Jan 14, 2014 4:39 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Feb 14, 2014 7:33 am

Hi pcunite

great, post and pretty much the only thing out there on VoIP QoS at this level that I can find for Router OS.
have you progressed any yet. There seem to be some key issues still outstanding.

I am about to setup QoS for a VoIP network on an RB2011 which will be competing with other subnet LANS for a 5mbps up and 5mbps down EFM service and I really need to nail the VoIP QoS. Any further reading recommendations... other than who the best support person is to pay to do it at Mikrotik.

Rgds
Mark
 
Urban!
just joined
Posts: 4
Joined: Fri Sep 28, 2007 4:50 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Feb 14, 2014 3:40 pm

hi pcunite

im using 12M up and 12M Down

how to set the upload rates and download rates in Voip

# The secret to ensuring VoIP quality (or any UDP traffic) is to put it into
# a queue that will never be full and thus never prioritize (drop) packets.
add name="LEVEL_A_UP" parent=ether-WAN queue=default max-limit=900k
add name="LEVEL_A_DOWN" parent=ether-LAN queue=default max-limit=4M

# Next, create a queue for high priority traffic.
add name="LEVEL_B_UP" parent=ether-WAN queue=default max-limit=900k
add name="LEVEL_B_DOWN" parent=ether-LAN queue=default max-limit=4M

# Finally, create a queues for traffic that normally exceeds levels.
add name="LEVEL_C_UP" parent=ether-WAN queue=default max-limit=900k
add name="LEVEL_C_DOWN" parent=ether-LAN queue=default max-limit=4M



thanks
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Feb 14, 2014 3:49 pm

Great post and pretty much the only thing out there on VoIP QoS at this level that I can find for Router OS.
Have you progressed any yet? There seem to be some key issues still outstanding.

Any further reading recommendations ... other than who the best support person is to pay to do it at Mikrotik?
There are a couple of people on the board here who you could pay: efaden, sdischer, or IPANetEngineer. They know what they're doing. I was hoping this post would bring out the VoIP experts but sadly they've not seen it yet. This is something we've all got to get nailed down.
Last edited by pcunite on Fri Feb 14, 2014 8:13 pm, edited 2 times in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Feb 14, 2014 4:03 pm


Hi pcunite. I'm have 12M/12M. How to set the upload rates and download rates for Voip? Thanks.
Make sure you understand how buffering works. Next, it all depends on how many active phone calls you want to support. A single VoIP call is 90kb both directions ... up and down.

With this knowledge, first we chop off 1M/1M (90%) so that the cable/adsl modem does not buffer anything. Next, let's allow for 10 active phone calls: 90kb x 10 = 900kb, which is about 1M. So, 1M for buffering, and 1M for VoIP means we set max-limit to equal 12 - 2 = 10M.

ros code

add name="LEVEL_A_UP"   parent=ether-WAN queue=default max-limit=10M
add name="LEVEL_A_DOWN" parent=ether-LAN queue=default max-limit=10M
add name="LEVEL_B_UP"   parent=ether-WAN queue=default max-limit=10M
add name="LEVEL_B_DOWN" parent=ether-LAN queue=default max-limit=10M
add name="LEVEL_C_UP"   parent=ether-WAN queue=default max-limit=10M
add name="LEVEL_C_DOWN" parent=ether-LAN queue=default max-limit=10M
Just so you know, my VoIP script is not perfect. It works, however. Until you really know how to master segmenting traffic you'll at least have VoIP calls working correctly.
 
mdkberry
Frequent Visitor
Frequent Visitor
Posts: 70
Joined: Tue Jan 14, 2014 4:39 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sun Feb 16, 2014 4:05 am

I would like to adapt pcunite's script to my scenario. and would welcome any suggestions.

my router is RB2011 routerOS 6.7

ISP service is 5mbps up and 5mbps down on EFM (4 wire symmetrical).
this is connected on Eth1 of the router.

I have a VoIP PBX which will make 25 calls max at a time.
this is on the Voice subnet x.x.20.x which is connected to Eth2

the Data LAN is on x.x.10.x connected to Eth3

another subnet is on x.x.88.x connected to Eth4 (but bridged with Eth3)

and there is a WLAN1 interface with x.x.30.x subnet on it.

what I am looking at currently with this is to QoS VoIP traffic on x.x.20.x with a set bandwidth of 2.5mbps each way up and down. to guarantee the VoIP service.and then allow the other subnets to compete for the other 2.5mps ideally using the same queues as in the example from pcunite. Though it would be nice if they could use the remainder of the 5mbps when VoIP isnt being used to the max. In the Cisco configs I have used before the way I dealt with this was to assign QoS as a percentage of bandwidth use and that worked well. I dont see a way to do that with Mikrotiks.

another problem I can't work out yet is how to associate multiple interfaces to a queue, which I will need in this example since I have seperate LANs on seperate interfaces, but they need to use the same queue. I can't bridge the interfaces and apply it there, because two of the subnets have DHCP services and that all stops working when you add bridges in.

EDIT: here are some solutions for my situation so far,

-I have been able to pretty much stick to the script provided by pcunite.
-I included the chains for my PBX using 192.168.20.2 in src-address and another one for dst-address and set ports to 5060, 5061, 10000-20000 on both of those. These are associated to Level A queue.
-all the mangle chains are on prerouting or postrouting including the above ones, instead of forward, though not sure what difference that makes I would be interested to hear comments on that, it seems to work.
-the LEVEL_DOWN queues are all set to parent 'global' resolving the NAT fine, and also multiple subnets/interfaces issue.
-the LEVEL_UP queues set to parent 'ether1-gateway' (the WAN link)
-copy and pasting the script gave me a few issues not sure why, it may have been me accidently clicking fields before saving but it didnt seem to take properly into RouterOS 6.7 and I had to double check some of the mangles to fix a few fields that got dropped randomly, as per my post later on in this thread.
- I have set the Level A queue to 4M/4M, Level B to 3M/3M and Level C to 3M/3M though I am thinking of putting a Queue above Level B and C to max-limit them collectively to 3M to allow the 2M of voip always to be available to Level A. Have to think about that as it may still cause problems for the Level B traffic which can be from my PBX. From my understanding of VoIP so far it is more important to get the ACK and SIP traffic without packet loss than the RTP media traffic which can handle it a little, though it needs to stay below <5% loss.
Last edited by mdkberry on Tue Feb 18, 2014 7:39 am, edited 1 time in total.
 
mdkberry
Frequent Visitor
Frequent Visitor
Posts: 70
Joined: Tue Jan 14, 2014 4:39 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Feb 18, 2014 2:38 am

Hi pcunite

in testing your script using YouTube media , I am finding that instead of port 80 traffic getting marked with HTTP or HTTP_BIG mark
it is ending up getting marked by the UDP rule in mangle.
if I disable UDP in mangle then it correctly marks the traffic with HTTP or HTTP_BIG

any thoughts on this?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Feb 18, 2014 4:22 am

In testing your script using YouTube media, I am finding that instead of port 80 traffic getting marked with HTTP or HTTP_BIG mark it is ending up getting marked by the UDP rule in mangle. If I disable UDP in mangle then it correctly marks the traffic with HTTP or HTTP_BIG.
The order of the rules is important somewhat ... but let me see what you have and I can hopefully spot the error.
 
mdkberry
Frequent Visitor
Frequent Visitor
Posts: 70
Joined: Tue Jan 14, 2014 4:39 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Feb 18, 2014 7:15 am

my bad...somehow when I copy-pasted it from your script into the terminal on my router, the UDP mangle didnt copy the protocol for just that mangle.
I just fixed it and everything looks to be ok.

another question, when I copy-pasted, all the mark-connection mangle rules defaulted to "passthrough=yes" should these be set to no?
I can see mark-packet should be set to no, but wasnt clear about the mark-connection default state.
 
TikUser
newbie
Posts: 48
Joined: Thu Jul 04, 2013 2:40 pm
Location: EU

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Feb 18, 2014 12:52 pm

Hello pcunite! I've noticed something interesting in your rules. Have You used the examples from this Mikrotik wiki for HTTP and HTTP_BIG?
The rule structure is pretty much the same. You are using the same connection rate and connection bytes values. Everything seems fine, except your comment(explanation) for this rules. The author of the wiki talks about 4Mb (500kB), and You about 5MB. The same values, but different explanation.
4Mb (little letter b), 5MB (big letter b).
Mb= megabit
MB= megabyte
This is a huge difference. :D
 
SyCo
just joined
Posts: 2
Joined: Mon Feb 10, 2014 8:41 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Feb 18, 2014 2:44 pm

Hello! I've noticed something interesting in your rules. [...]
Nice find! Like you've said it's only in the comments.
I see also that the rule use the correct value of 500kB -> connection-bytes=500000-0

Then we simply need to change the comments ;-) BTW on that same line there is also a little typo where it's written "tranfers" instead of "tranSfers".
 
mdkberry
Frequent Visitor
Frequent Visitor
Posts: 70
Joined: Tue Jan 14, 2014 4:39 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Feb 19, 2014 6:38 am

another router setup using the same script and I am seeing HTTP_D traffic which is supposed to be priority 1 being dropped before the other traffic in Level C which is lower priority.
any thoughts? piccy attached.
dropped_packets.tiff
You do not have the required permissions to view the files attached to this post.
 
TikUser
newbie
Posts: 48
Joined: Thu Jul 04, 2013 2:40 pm
Location: EU

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Feb 19, 2014 12:52 pm

another router setup using the same script and I am seeing HTTP_D traffic which is supposed to be priority 1 being dropped before the other traffic in Level C which is lower priority.
any thoughts? piccy attached.
dropped_packets.tiff
I. Reduce the number of Queue Trees to 2. One for download & one upload. Your goal is to create a unified hierarchy.
Like this:
Queue Tree - download(the same for upload).
1. DNS - Priority 1
2. ICMP - Priority 1
3. VOIP -Priority 2
4. ACK - Priority 3
5. HTTP - Priority 4
6. HTTP_BIG - Priority 5
7. UDP - Priority 7
8. OTHER - Priority 8

II. Or rearrange the Queue Tree. Look at the picture in the attachment. The picture isn’t something, but it serves the purpose. :D
Merge level B & C into one queue tree. VOIP (level A) remains in its queue. At the top create the total download bandwidth queue, with VOIP (level A) and Everything else (level B+ level C) as its children.
Read this Mikrotik wiki for more information.

P.s. For the Queue Tree - download, You also need to use the HTB Interface as parent. NOT global. Global is not good for this examples. It is used for another type of QoS.
You do not have the required permissions to view the files attached to this post.
 
TikUser
newbie
Posts: 48
Joined: Thu Jul 04, 2013 2:40 pm
Location: EU

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Feb 19, 2014 8:53 pm

another problem I can't work out yet is how to associate multiple interfaces to a queue, which I will need in this example since I have seperate LANs on seperate interfaces, but they need to use the same queue. I can't bridge the interfaces and apply it there, because two of the subnets have DHCP services and that all stops working when you add bridges in.
The pcunite examples won't work well in Your case. You need to create another set of mangle and Queue Tree rules.
- Replace prerouting with the forward chain for traffic going through the router.
- Use src-address-list(your network) + out. interface(WAN-Internet) for marking the connections going through the router(forward chain) to the internet and backwards.
- Create separate mangle rules for marking the download(Internet->LAN) and upload(LAN->Internet) packets from these connections.

- Replace postrouting with the output chain for traffic from the router to the internet. If needed, use out. interface option in order to make the connections in the output chain more precise.
- Create separate mangle rules for marking the download and upload packets from these connections. Which means: Packets coming from the internet to the router (Internet->router), and packets going from the router to the internet (router->Internet).

"We think of Connections as "Both Upload and Download" since most connections have packets in both directions. We think of Packets as either "Download" or "Upload"." Quoted from: NetworkPro on Quality of Service
Watch out how you order the rules!

- Now you can use parent global in Queue Tree for creating the download and upload tree. You must not use the HTB Interface (ether1, ether2,...) in Queue Tree as the parent for the download and upload queue, because You have more than 1 active LAN interfaces, which, as You said, mustn't be bridged.

This is out-of-my-head solution. :D Not tested. Test it and adapt it to your needs. This is not the only possibility. For every problem, there are a couple of solutions.

And most important: You need to understand the packet flow, Queue Tree and prerouting, forward, output, postrouting chain in order to know where to apply what, and why. Until You know what You are doing, a router without prioritization is a better solution than a router with a false implemented prioritization.

Search the Mikrotik forum and You will find the answer to: “another question, when I copy-pasted, all the mark-connection mangle rules defaulted to "passthrough=yes" should these be set to no? I can see mark-packet should be set to no, but wasnt clear about the mark-connection default state.“

In most of the cases, when You mark the connections, use passthrough=yes. When You mark the packets, use passthrough=no.
Last edited by TikUser on Thu Feb 20, 2014 3:26 am, edited 7 times in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Feb 19, 2014 9:47 pm

I am seeing HTTP_D traffic (which is priority 1) being dropped before other traffic in Level C, which has a lower priority.
Notes:
To QoS traffic with a MikroTik you have to be an expert and really understand their documentation. I am not an expert. Please note that my QoS script is for VoIP packet protection. I don't ever want to drop a VoIP packet ... ever ... never. You need to do extensive reading on the subject if you want to design a QoS system that differs from the one I've created.

Terms:
priority means 'drop packets' WHEN needed.
max-limit means to drop packets ONLY when exceeded.

Explanation:
If over 1M of HTTP_D traffic appears then they'll get dropped. I don't know how many PC's you have on your network, but it is real easy to exceed 1M from casual browsing. If you have over 1M of HTTP_D and HTTP_BIG_D at the exact same time then priority is calculated between the two.
 
TecnoNet
just joined
Posts: 2
Joined: Sat Feb 22, 2014 9:51 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Feb 22, 2014 10:05 am

Note: ehter1--REAL--MAIN is the internet from the ISP,
and in this QOS we Limit the HEAVY DOWNLOAD DATA and leave the rest :D
You do not have the required permissions to view the files attached to this post.
 
mdkberry
Frequent Visitor
Frequent Visitor
Posts: 70
Joined: Tue Jan 14, 2014 4:39 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Feb 26, 2014 1:07 am

thanks for all the input those that have posted that really helps. I will test it TikUser ideas out and let you know how it goes.

one thing I noticed when I tried pcunite's script tailored in the way I mentioned. once the PBX started distributing calls outbound
all the VoIP traffic ended up marked with PBX_UDP up or down as I expected, but all the actual queue traffic was in Level B under ACK and UDP. hardly any showed up in Level A VoIP despite it being marked from the PBX server fine.

anyway I had to strip out all the marking and queue info anyway as it wasnt working for my setup.

pcunite, in answer to your comment about 1M being a low limit the voice network is setup to have only phones on it, and minimal HTTP traffic should be on it. it was working ok and no prioritising was going on. that idea was really to capture anyone accidently plugging into a voice network floor port and surfing the voice network into a grinding halt.

to be continued...
 
RB951G2HnD
newbie
Posts: 30
Joined: Mon Oct 21, 2013 9:52 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Mon Mar 24, 2014 8:11 pm

TecnoNet,

Thank you for a good QoS config script. It seems that it's working fine except P2P. In our days the most of P2P clients are using encryption and that is the reason why a simple indicated P2P option is not working in a Mikrotik ROS. I think that to make this QoS config script working with torrent's traffic we need to specify incoming ports in P2P client. For example to use incoming ports from 6881 to 6891. If it so then we need to change Mangle rule for P2P. How to relate this rule with the dedicated ports?

ros code

add action=mark-connection chain=postrouting comment="P2P UP" new-connection-mark=p2pupconn out-interface=ether1 p2p=all-p2p
add action=mark-packet chain=postrouting connection-mark=p2pupconn new-packet-mark=p2pup passthrough=no
add action=mark-connection chain=prerouting comment="P2P DOWN" in-interface=ether1 new-connection-mark=p2pdownconn p2p=all-p2p
add action=mark-packet chain=prerouting connection-mark=p2pdownconn new-packet-mark=p2pdown passthrough=no
 
RB951G2HnD
newbie
Posts: 30
Joined: Mon Oct 21, 2013 9:52 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Mar 26, 2014 10:36 am

No QoS config provided by TecnoNet is not working because all traffic marked as 'other' including http and etc. So the attached config file is wrong.
 
RB951G2HnD
newbie
Posts: 30
Joined: Mon Oct 21, 2013 9:52 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Mar 28, 2014 5:36 am

Yesterday I was able to slightly adjust QoS priorities. But somehow torrent traffic clogs channel so that for example in parallel to watch youtube is not possible. Torrent traffic is marked in the Mangle rules correctly. Please prompt me where to look and what needs to be corrected into config?

ros code

/queue tree
add max-limit=2800k name=LEVEL_A_UP parent=ether1 priority=7 queue=default
add max-limit=2800k name=LEVEL_A_DOWN parent=ether2 priority=7 queue=default
add max-limit=2800k name=LEVEL_B_UP parent=ether1 priority=7 queue=default
add max-limit=2800k name=LEVEL_B_DOWN parent=ether2 priority=7 queue=default
add max-limit=2800k name=LEVEL_C_UP parent=ether1 queue=default
add max-limit=2800k name=LEVEL_C_DOWN parent=ether2 queue=default
add name=ACK_U packet-mark=ACK parent=LEVEL_A_UP priority=1 queue=default
add name=ACK_D packet-mark=ACK parent=LEVEL_A_DOWN priority=1 queue=default
add name=DNS_U packet-mark=DNS parent=LEVEL_A_UP priority=2 queue=default
add name=DNS_D packet-mark=DNS parent=LEVEL_A_DOWN priority=2 queue=default
add name=ICMP_U packet-mark=ICMP parent=LEVEL_A_UP priority=3 queue=default
add name=ICMP_D packet-mark=ICMP parent=LEVEL_A_DOWN priority=3 queue=default
add name=UDP_U packet-mark=UDP parent=LEVEL_B_UP priority=1 queue=default
add name=UDP_D packet-mark=UDP parent=LEVEL_B_DOWN priority=1 queue=default
add name=OTHER_U packet-mark=OTHER parent=LEVEL_C_UP priority=1 queue=default
add name=OTHER_D packet-mark=OTHER parent=LEVEL_C_DOWN priority=1 queue=default
add name=HTTP_U packet-mark=HTTP parent=LEVEL_B_UP priority=3 queue=default
add name=HTTP_D packet-mark=HTTP parent=LEVEL_B_DOWN priority=3 queue=default
add name=HTTP_BIG_U packet-mark=HTTP_BIG parent=LEVEL_B_UP priority=3 queue=default
add name=HTTP_BIG_D packet-mark=HTTP_BIG parent=LEVEL_B_DOWN priority=3 queue=default
add name=E-MAIL_U packet-mark=E-MAIL parent=LEVEL_B_UP priority=2 queue=default
add name=E-MAIL_D packet-mark=E-MAIL parent=LEVEL_B_DOWN priority=2 queue=default
add name=TCP_TORRENT_U packet-mark=TCP_TORRENT parent=LEVEL_C_UP priority=2 queue=default
add name=TCP_TORRENT_D packet-mark=TCP_TORRENT parent=LEVEL_C_DOWN priority=2 queue=default
add name=UDP_TORRENT_U packet-mark=UDP_TORRENT parent=LEVEL_C_UP priority=2 queue=default
add name=UDP_TORRENT_D packet-mark=UDP_TORRENT parent=LEVEL_C_DOWN priority=2 queue=default
/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=UDP_TORRENT connection-state=new new-connection-mark=UDP_TORRENT port=6881-6891 protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP_TORRENT new-packet-mark=UDP_TORRENT passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=E-MAIL connection-state=new new-connection-mark=E-MAIL port=110,995,143,993,25,465,587,2525 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=E-MAIL new-packet-mark=E-MAIL passthrough=no
add action=mark-connection chain=prerouting comment=HTTP connection-mark=!HTTP_BIG connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=500000-0 connection-mark=HTTP connection-rate=200k-100M new-connection-mark=HTTP_BIG protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=TCP_TORRENT connection-state=new new-connection-mark=TCP_TORRENT port=6881-6891 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=TCP_TORRENT new-packet-mark=TCP_TORRENT passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-mark=no-mark new-connection-mark=OTHER
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no
 
Rudios
Forum Veteran
Forum Veteran
Posts: 977
Joined: Mon Mar 11, 2013 12:58 pm
Location: The Netherlands

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Mar 28, 2014 8:22 am

I'm a little curious why you have some rules twice

ros code

/ip firewall mangle
add chain=forward action=mark-connection protocol=udp   src-address=192.168.100.5 connection-state=new new-connection-mark="VOIP" comment="IP-PBX"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
add chain=forward action=mark-connection protocol=udp   dst-address=192.168.100.5 connection-state=new new-connection-mark="VOIP"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
 
You are marking packets twice based on the same options. In my opinion rule 2 can be removed.
Then first there are two rules marking the connection and setting connection-marks. Then use the connection-mark to mark the packets in one single rule.
 
mdkberry
Frequent Visitor
Frequent Visitor
Posts: 70
Joined: Tue Jan 14, 2014 4:39 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu May 22, 2014 6:30 am

Ok been a while since I was on this thread but I have tried the above setup and run into a strange problem where marking the packets stops some web sites being available. I am not sure what is causing it and hoped someone could look at the below Mangle rules and tell me.

I have enabled everything except HTTP, HTTP_BIG and one I called 'THE_REST' because when I enabled these no one could access various web sites, such as http://www.adobe.com http://www.sybase.com and a host of others. It was always the same web sites and lots of them, until I disabled those three rules then it works fine, I am not sure if the other traffic like ACK or DNS is having a problem but no one has complained and it has been up a couple of days like this now.

I assume this is because those disabled rules are stopping the traffic flow passing to the dynamic rule set that appeared after I configured the PPPoE TPG fibre gateway. I have added those two dynamic rules in at the end of this post.

should just marking traffic cause it to stop going out the normal route? or have I done something wrong that I cant see here?

my mangle rules:
0   ;;; VOIP - Voip traffic FROM PBX (5060-5061, 10000-20000 udp)
     chain=forward action=mark-connection new-connection-mark=VOIP passthrough=yes connection-state=new protocol=udp
     src-address=10.10.20.5 port=5060-5061,10000-20000

 1   chain=forward action=mark-packet new-packet-mark=VOIP passthrough=no connection-mark=VOIP

 2   ;;; VOIP - Voip traffic TO PBX (5060-5061, 10000-20000 udp)
     chain=output action=mark-connection new-connection-mark=VOIP passthrough=yes connection-state=new protocol=udp
     dst-address=10.10.20.5 port=5060-5061,10000-20000

 3   chain=output action=mark-packet new-packet-mark=VOIP passthrough=no connection-mark=VOIP

 4   ;;; V_GENERAL - other traffic FROM voice network
     chain=forward action=mark-connection new-connection-mark=V_GENERAL passthrough=yes src-address-list=VoIP_Network
     connection-mark=no-mark

 5   chain=forward action=mark-packet new-packet-mark=V_GENERAL passthrough=no connection-mark=V_GENERAL

 6   ;;; V_GENERAL - other traffic TO voice network
     chain=output action=mark-connection new-connection-mark=V_GENERAL passthrough=yes dst-address-list=VoIP_Network
     connection-mark=no-mark

 7   chain=output action=mark-packet new-packet-mark=V_GENERAL passthrough=no connection-mark=V_GENERAL

 8   ;;; DNS
     chain=forward action=mark-connection new-connection-mark=DNS passthrough=yes connection-state=new protocol=udp port=53

 9   chain=forward action=mark-packet new-packet-mark=DNS passthrough=no connection-mark=DNS

10   chain=output action=mark-connection new-connection-mark=DNS passthrough=yes connection-state=new protocol=udp port=53

11   chain=output action=mark-packet new-packet-mark=DNS passthrough=no connection-mark=DNS

12   ;;; UDP
     chain=forward action=mark-connection new-connection-mark=UDP passthrough=yes connection-state=new protocol=udp

13   chain=forward action=mark-packet new-packet-mark=UDP passthrough=no connection-mark=UDP

14   ;;; ICMP
     chain=forward action=mark-connection new-connection-mark=ICMP passthrough=yes connection-state=new protocol=icmp

15   chain=forward action=mark-packet new-packet-mark=ICMP passthrough=no connection-mark=ICMP

16   chain=output action=mark-connection new-connection-mark=ICMP passthrough=yes connection-state=new protocol=icmp

17   chain=output action=mark-packet new-packet-mark=ICMP passthrough=no connection-mark=ICMP

18   ;;; ACK
     chain=output action=mark-packet new-packet-mark=ACK passthrough=no tcp-flags=ack protocol=tcp packet-size=0-123

19   chain=forward action=mark-packet new-packet-mark=ACK passthrough=no tcp-flags=ack protocol=tcp packet-size=0-123

20 X ;;; HTTP
     chain=forward action=mark-connection new-connection-mark=HTTP passthrough=yes connection-state=new protocol=tcp
     port=80,443 connection-mark=!HTTP_BIG

21 X chain=forward action=mark-connection new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp connection-mark=HTTP
     connection-bytes=500000-0 connection-rate=200k-100M

22 X chain=forward action=mark-packet new-packet-mark=HTTP_BIG passthrough=no connection-mark=HTTP_BIG

23 X chain=forward action=mark-packet new-packet-mark=HTTP passthrough=no connection-mark=HTTP

24 X ;;; THE_REST
     chain=forward action=mark-connection new-connection-mark=THE_REST passthrough=yes connection-mark=no-mark

25 X chain=forward action=mark-packet new-packet-mark=THE_REST passthrough=no connection-mark=THE_REST


Below are the dynamic rules in place from the PPPoE gateway , they are the last two rules in my mangle list but dont show when doing a print from command line.

chain=forward
protocol= tcp
in interface = all ppp
TCP MSS = 1441-65535
TCP Flags = syn
action = change MSS
new TCP MSS = 1440
(no traffic through this as yet)

chain=forward
protocol = tcp
out.interface = all ppp
TCP MSS = 1441-65535
TCP Flags = syn
action = change MSS
new TCP MSS = 1440
passthrough= ticked
(quite a lot of traffic has passed through this rule)
Last edited by mdkberry on Wed May 28, 2014 4:21 am, edited 1 time in total.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sun May 25, 2014 4:25 pm

I'm a little curious why you have some rules twice.

ros code

/ip firewall mangle
add chain=forward action=mark-connection protocol=udp   src-address=192.168.100.5 connection-state=new new-connection-mark="VOIP" comment="IP-PBX"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
add chain=forward action=mark-connection protocol=udp   dst-address=192.168.100.5 connection-state=new new-connection-mark="VOIP"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
You are marking packets twice based on the same options. In my opinion rule 2 can be removed.
Then first there are two rules marking the connection and setting connection-marks. Then use the connection-mark to mark the packets in one single rule.
This section is from Option 1. It is marking "src" and then "dst" to the same ip address. Also, unless MikroTik documentation has changed, one must mark the connection and then the packet for performance.
 
TikUser
newbie
Posts: 48
Joined: Thu Jul 04, 2013 2:40 pm
Location: EU

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu May 29, 2014 6:44 pm

Hey mdkberry,
it would be good to give us a visual scheme of your network. So, we could easily see what goes where. :-D
First, do you want to do QoS for traffic going/coming to/from the internet, or for all the traffic?
If it's the Internet, then your mangle rules need to be more precise.
Example (imprecise rule):
add action=mark-connection chain=forward connection-state=new new-connection-mark=UDP protocol=udp
This rule catches ALL the udp traffic going through your router. Not only internet udp traffic. If you have users on ether3, ether4 and ether 5 and they are communicating with each other, then this rule will also catch their udp inside the network traffic.

You said that your internet speed is 5Mb up/down. If your network is connected with Ethernet cables, then you have at least 100Mb of available bandwidth inside your network. QoS for inside traffic is NOT needed, because you have more than enough bandwidth.
If your internet connection is on ether1, then change the rule:
add action=mark-connection chain=forward connection-state=new new-connection-mark=UDP out-interface=ether1 protocol=udp
add action=mark-packet chain=forward connection-mark=UDP new-packet-mark=UDP passthrough=no
With this rule you are cathing all the udp connections going to the internet. You don't need to specify the mark-packet rule, because it uses the data from the mark-connection rule.

Why are you using output chain for the VoIP traffic to PBX and other traffic to voice network? Output chain means only traffic that is generated by the Mikrotik router.
If you have a scheme: PBX VOIP <-> Mikrotik-router <-> Internet, then the output chain will NOT catch this traffic, unless you have modified the ROS to redirect the VoIP traffic to the router itself and then to other interfaces.
Try with this rules:
add action=mark-connection chain=forward connection-state=new new-connection-mark=VOIP out-interface=ether1 port=5060-5061,10000-20000 protocol=udp src-address=10.10.20.5 
add action=mark-connection chain=forward connection-state=new dst-address=10.10.20.5 in-interface=ether1 new-connection-mark=VOIP port=5060-5061,10000-20000 protocol=udp
First mark-connection rule catches voip connections initiated from 10.10.20.5 and going to the internet(ether1). Second mark-connection rule catches connections initiated from the internet(ether1) and going to 10.10.20.5.
If on 10.10.20.5 you only have the PBX, you can remove ports and protocol from these rules. Also replace the connection-mark=no-mark with connection-state=new in rules 4 and 6.

If I were at your place, I would remove the udp rules 12 and 13 completely because, for your configuration, they don’t make any sense.
And I would integrate the ACK rules with the HTTP and HTTP_BIG rules. Like this:
add action=mark-packet chain=forward connection-mark=HTTP_BIG new-packet-mark=ACK-HTTP-BIG packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=forward connection-mark=HTTP new-packet-mark=ACK-HTTP packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
These rules catch the ack packets from HTTP & HTTP_BIG connections. Put them AFTER the HTTP & HTTP_BIG mark-connection rules, and BEFORE the HTTP & HTTP_BIG mark-packet rules.
Example:
1. HTTP mark-connection rule
2. HTTP_BIG mark-connection rule
3. ACK-HTTP-BIG mark-packet rule
4. ACK-HTTP mark-packet rule
5. HTTP_BIG mark-packet rule
6. HTTP mark-packet rule

About the PPPOE. You will need to play with this. :-D I don't have a PPPOE configuration (yet), so I can't help you in this field.
- Disable all the static mangle rules.
- Check if traffic is flowing through both dynamic pppoe rules.
- Change the MSS size, if needed.
- Change the Max UDP Packet size in DNS Settings, if needed.
- Try with putting the dynamic mangle rules above all static mangle rules.
- Search this forum for more information about ppp dynamic rules and mss size.
 
beb
just joined
Posts: 19
Joined: Tue Nov 13, 2012 4:17 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Jun 26, 2014 3:58 am

How Buffering and bufferbloat in ISP supplied equipment affects Qos and latency sensitive protocols ...
Thank you for this great explanation!

Am I correct in my conclusion that without a reasonable estimate of my wan bandwidth, it's impossible to implement qos?

Specifically - I have Comcast (cable) and the download speed varies between 60Mbps and 5Mbps, depending on the time-of-day, phase of the moon, and Comcast's share price. If I understand correctly, if I want to implement qos I need to set the maximum limit to <5Mbps. This of course suffices for VOIP, but gives me much less than available bandwidth most of the time.

A bit sobering, but at least I am not wasting my time trying to fix something I cannot.

Correct?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Jun 27, 2014 9:09 pm

Am I correct in my conclusion that without a reasonable estimate of my wan bandwidth, it's impossible to implement qos? Specifically - I have Comcast (cable) and the download speed varies between 60Mbps and 5Mbps
That is quite a variance! Yes, you are correct ... you can not use "static Qos" techniques to maintain your traffic integrity. There are "dynamic Qos" ways of doing things but that is beyond the scope of this thread and my expertise.
 
war3boy
just joined
Posts: 7
Joined: Sun Jun 29, 2014 1:38 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sun Jun 29, 2014 4:16 pm

Hi Guys,

Need some light for the Latency QoS. I try to shape but seem it failed to do so.
I knew we can use bandwidth control to guarantee the bandwidth for gaming , but i wish can fully utilize the bandwidth when the game is not in play.
But when the game is in play, the QoS will do it work to keep the game latency as usual.

ros code

# jun/29/2014 21:38:22 by RouterOS 6.12
# software id = AF8C-ZI5T
#
/ip firewall mangle
#Skype Layer7 Marking
add action=mark-packet chain=prerouting comment=Skype in-interface=PPPoE-UniFi layer7-protocol=skypetoskype new-packet-mark=QoS_1_In passthrough=no
add action=mark-packet chain=postrouting layer7-protocol=skypetoskype new-packet-mark=QoS_1_Out out-interface=PPPoE-UniFi passthrough=no

#VPN Layer 7 Marking
add action=mark-packet chain=prerouting comment=VPN in-interface=PPPoE-UniFi new-packet-mark=QoS_2_In passthrough=no protocol=gre
add action=mark-packet chain=postrouting new-packet-mark=QoS_2_Out out-interface=PPPoE-UniFi passthrough=no protocol=gre

#QoS_4 Steaming Layer 7 Marking
add action=mark-packet chain=prerouting comment="------------QoS_4 [Streaming_Services]------------" in-interface=PPPoE-UniFi layer7-protocol=Streaming new-packet-mark=QoS_4_In passthrough=no
add action=mark-packet chain=postrouting layer7-protocol=Streaming new-packet-mark=QoS_4_Out out-interface=PPPoE-UniFi passthrough=no
add action=mark-packet chain=prerouting in-interface=PPPoE-UniFi layer7-protocol=video new-packet-mark=QoS_4_In passthrough=no
add action=mark-packet chain=postrouting layer7-protocol=video new-packet-mark=QoS_4_Out out-interface=PPPoE-UniFi passthrough=no

#QoS_8 Torrent Layer 7 Marking
add action=mark-packet chain=prerouting comment="------------QoS_8 [Torrent_Services]------------" in-interface=PPPoE-UniFi layer7-protocol=Torrent-wwws new-packet-mark=QoS_8_In passthrough=no
add action=mark-packet chain=postrouting layer7-protocol=Torrent-wwws new-packet-mark=QoS_8_Out out-interface=PPPoE-UniFi passthrough=no
add action=mark-packet chain=prerouting in-interface=PPPoE-UniFi layer7-protocol=torrentDNS new-packet-mark=QoS_8_In passthrough=no
add action=mark-packet chain=postrouting layer7-protocol=torrentDNS new-packet-mark=QoS_8_Out out-interface=PPPoE-UniFi passthrough=no
add action=mark-packet chain=prerouting in-interface=PPPoE-UniFi new-packet-mark=QoS_8_In p2p=all-p2p passthrough=no
add action=mark-packet chain=postrouting new-packet-mark=QoS_8_Out out-interface=PPPoE-UniFi p2p=all-p2p passthrough=no

#QoS_1 Steam
add action=mark-packet chain=prerouting comment="------------QoS_1 [Steam]------------" in-interface=PPPoE-UniFi new-packet-mark=QoS_1_In passthrough=no protocol=tcp src-port=27014-27050
add action=mark-packet chain=postrouting dst-port=27014-27050 new-packet-mark=QoS_1_Out out-interface=PPPoE-UniFi passthrough=no protocol=tcp
add action=mark-packet chain=prerouting in-interface=PPPoE-UniFi new-packet-mark=QoS_1_In passthrough=no protocol=udp src-port=3478,4379,4380,28960,27000-27030
add action=mark-packet chain=postrouting dst-port=3478,4379,4380,28960,27000-27030 new-packet-mark=QoS_1_Out out-interface=PPPoE-UniFi passthrough=no protocol=udp

#QoS_2 DNS
add action=mark-packet chain=prerouting comment="------------QoS_2 [DNS+NTP_PPTP_Services]------------" in-interface=PPPoE-UniFi new-packet-mark=QoS_2_In passthrough=no protocol=udp src-port=53,123,1723
add action=mark-packet chain=postrouting dst-port=53,123,1723 new-packet-mark=QoS_2_Out out-interface=PPPoE-UniFi passthrough=no protocol=udp
add action=mark-packet chain=prerouting connection-state=new in-interface=PPPoE-UniFi new-packet-mark=QoS_2_In passthrough=no protocol=tcp src-port=52,123,1723
add action=mark-packet chain=postrouting dst-port=52,123,1723 new-packet-mark=QoS_2_Out out-interface=PPPoE-UniFi passthrough=no protocol=tcp

#QoS_3 WWW Services + General_Services
add action=mark-packet chain=prerouting comment="------------QoS_3 [WWW_Services]------------" connection-bytes=0-1500000 in-interface=PPPoE-UniFi new-packet-mark=QoS_3_In passthrough=no protocol=tcp src-port=80,443
add action=mark-packet chain=postrouting connection-bytes=0-1500000 dst-port=80,443 new-packet-mark=QoS_3_Out out-interface=PPPoE-UniFi passthrough=no protocol=tcp
add action=mark-packet chain=prerouting comment="------------QoS_3 [General_Services]------------" in-interface=PPPoE-UniFi new-packet-mark=QoS_3_In passthrough=no protocol=tcp src-port=20,21,22,23,25,110,143,465,587,993,995,2525,3535
add action=mark-packet chain=postrouting dst-port=20,21,22,23,25,110,143,465,587,993,995,2525,3535 new-packet-mark=QoS_3_Out out-interface=PPPoE-UniFi passthrough=no protocol=tcp

#QoS_5 WWW_Heavy
add action=mark-packet chain=prerouting comment="------------QoS_5 [WWW_Services_Heavy]------------" connection-bytes=1500000-0 in-interface=PPPoE-UniFi new-packet-mark=QoS_5_In passthrough=no protocol=tcp src-port=80,443
add action=mark-packet chain=postrouting connection-bytes=1500000-0 dst-port=80,443 new-packet-mark=QoS_5_Out out-interface=PPPoE-UniFi passthrough=no protocol=tcp

#QoS_8 Other BULK
add action=mark-packet chain=prerouting comment="------------QoS_8 [Other_Services]------------" in-interface=PPPoE-UniFi new-packet-mark=QoS_8_In passthrough=no protocol=tcp
add action=mark-packet chain=postrouting new-packet-mark=QoS_8_Out out-interface=PPPoE-UniFi passthrough=no protocol=tcp
add action=mark-packet chain=prerouting in-interface=PPPoE-UniFi new-packet-mark=QoS_8_In passthrough=no protocol=udp
add action=mark-packet chain=postrouting new-packet-mark=QoS_8_Out out-interface=PPPoE-UniFi passthrough=no protocol=udp
add action=mark-packet chain=prerouting in-interface=PPPoE-UniFi new-packet-mark=QoS_8_In passthrough=no
add action=mark-packet chain=postrouting new-packet-mark=QoS_8_Out out-interface=PPPoE-UniFi passthrough=no

ros code

# jun/29/2014 21:27:09 by RouterOS 6.12
# software id = AF8C-ZI5T
#
/queue tree

#Upload
add max-limit=4700k name=QoS_Global_Upload parent=global queue=default
add limit-at=512k max-limit=3M name="QoS_1_Out(Steam)" packet-mark=QoS_1_Out parent=QoS_Global_Upload priority=1 queue=default
add limit-at=100k max-limit=1M name="QoS_2_Out(DNS+NTP+PPTP)" packet-mark=QoS_2_Out parent=QoS_Global_Upload priority=2 queue=default
add limit-at=1500k max-limit=4M name="QoS_3_Out(WWW)" packet-mark=QoS_3_Out parent=QoS_Global_Upload priority=3 queue=default
add burst-limit=3800k burst-time=3s max-limit=3M name="QoS_4_Out(Streaming)" packet-mark=QoS_4_Out parent=QoS_Global_Upload priority=4 queue=default
add burst-limit=3800k burst-time=3s max-limit=3M name="QoS_5_Out(WWW_Heavy)" packet-mark=QoS_5_Out parent=QoS_Global_Upload priority=5 queue=default
add burst-limit=4300k burst-time=3s max-limit=3500k name="QoS_8_Out(Default)" packet-mark=QoS_8_Out parent=QoS_Global_Upload queue=pcq-upload-default

#Download
add max-limit=4800k name=QoS_Global_Download parent=global queue=default
add limit-at=512k max-limit=3M name="QoS_1_In(Steam)" packet-mark=QoS_1_In parent=QoS_Global_Download priority=1 queue=default
add limit-at=100k max-limit=500k name="QoS_2_In(DNS+NTP+PPTP)" packet-mark=QoS_2_In parent=QoS_Global_Download priority=2 queue=default
add burst-limit=3M burst-threshold=2M burst-time=5s limit-at=1500k max-limit=2M name="QoS_3_In(WWW)" packet-mark=QoS_3_In parent=QoS_Global_Download priority=3 queue=default
add burst-limit=2500k burst-threshold=1M burst-time=5s max-limit=1M name="QoS_4_In(Streaming)" packet-mark=QoS_4_In parent=QoS_Global_Download priority=4 queue=default
add burst-limit=3800k burst-time=2s max-limit=1M name="QoS_5_In(WWW_Heavy)" packet-mark=QoS_5_In parent=QoS_Global_Download priority=5 queue=default
add burst-limit=4300k burst-threshold=4M burst-time=3s max-limit=3M name="QoS_8_In(Default+Torrent)" packet-mark=QoS_8_In parent=QoS_Global_Download queue=pcq-download-default

 
ceather93
just joined
Posts: 1
Joined: Fri Sep 12, 2014 5:50 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Sep 12, 2014 5:56 am

Hi,

I am getting a bit confused with this. Wondering if someone could help me out.

I am trying to achieve the following in terms of QoS - in that order

1) All WAN related UDP connections
2) All Connections from and to Certain Address List
3) All Connections from and to a different address List
4) All Other Traffic
5) File Sharing/Torrents

I think I am stumbling on the packet flow part and why you need to set a limit on the LAN side of things in the queuing?
Other question is for the mangle do you need to make a separate rule for both outgoing and incoming traffic eg:
 2 X ;;; LYNC | Mark new source connections
     chain=forward action=mark-connection new-connection-mark=Lync-Connected
     passthrough=yes connection-state=new src-address-list=Lync in-interface=TPG PPPoe

 3 X ;;; LYNC | Mark Packet
     chain=forward action=mark-packet new-packet-mark=Lync_Packet passthrough=yes
     connection-mark=Lync-Connected

 4 X ;;; LYNC | Mark new dest connections
     chain=forward action=mark-connection new-connection-mark=Lync-Connected
     passthrough=yes connection-state=new dst-address-list=Lync
     out-interface=TPG PPPoe

 5 X ;;; LYNC | Mark Packet
     chain=forward action=mark-packet new-packet-mark=Lync_Packet passthrough=no
     connection-mark=Lync-Connected
Interaces:

Bridge Called 'Bridge' (Ether2 in it - connected to switch)
PPPoE Client called 'PPPoE' assigned to Ether1
 
vamose
just joined
Posts: 6
Joined: Tue Dec 23, 2014 11:45 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Mar 03, 2015 7:51 pm

Guys,

I bought the Mikrotik router and used AP and PPOE settings to enable the router and my ISP provider.

I have 10MBs service, but when I use speediest websites such as testmy.net net performance maxes out at 3.2MBps.

I need your help with streamlining my configuration and with QOS. The requirement for QOS is as per the priority listed, can any one help me.

VOIP
Streaming video content - You tube
SSH
FTP
Downloading
Torrent downloads.
 
toxicfusion
Member
Member
Posts: 324
Joined: Mon Jan 14, 2013 6:02 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Mar 17, 2015 8:57 pm

Hi

I've followed this guide... and have made adjustments to try and fix issues.

However, when ever I have queue tree rules created for 'no-mark'. The router is ALSO clamping the LAN network file transfer speeds... not cool.

/queue tree
add max-limit=4M name=LEVEL_A_UP parent=ether1_wan01 queue=default
add max-limit=22M name=LEVEL_A_DOWN parent=TRNK_LAN queue=default
add limit-at=3799k max-limit=3880k name=VOIP_UP packet-mark=VOIP parent=\
LEVEL_A_UP priority=1 queue=default
add limit-at=20M max-limit=22M name=VOIP_DWN packet-mark=VOIP parent=\
LEVEL_A_DOWN priority=1 queue=default
add name=DNS_UP packet-mark=DNS parent=LEVEL_A_UP priority=2 queue=default
add name=DNS_DWN packet-mark=DNS parent=LEVEL_A_DOWN priority=2 queue=default
add max-limit=60M name=LEVEL_B_DOWN parent=TRNK_LAN queue=default
add max-limit=3800k name=LEVEL_B_UP parent=ether1_wan01 queue=default
add limit-at=2600k max-limit=3M name=HTTP_UP packet-mark=HTTP parent=LEVEL_B_UP \
priority=2 queue=default
add limit-at=59M max-limit=60M name=HTTP_DWN packet-mark=HTTP parent=\
LEVEL_B_DOWN priority=2 queue=default
add limit-at=2800k max-limit=3M name=HTTP_Traff packet-mark=HTTP_Traff parent=\
LEVEL_B_UP priority=2 queue=default
add limit-at=2800k max-limit=3M name=HTTPS_up packet-mark=HTTPS parent=\
LEVEL_B_UP priority=2 queue=default
add limit-at=58M max-limit=60M name=HTTPS_Down packet-mark=HTTPS parent=\
LEVEL_B_DOWN priority=2 queue=default
add limit-at=55M max-limit=58M name=IPSEC_Down packet-mark=IPSEC parent=\
LEVEL_B_DOWN priority=2 queue=default
add limit-at=2800k max-limit=3M name=IPSEC_OUT_IN packet-mark=IPSEC parent=\
LEVEL_B_UP priority=3 queue=default
add disabled=yes limit-at=58M max-limit=60M name=misc_down packet-mark=no-mark \
parent=LEVEL_B_DOWN priority=3 queue=default
 
kivimart
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Thu Oct 10, 2013 3:06 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Thu Mar 19, 2015 10:46 pm

This is my queue tree, no limits set only trying to learn how to mark packet and queue then then later set limits.

I do not use VOIP in my setup...


If you post your Mangle rules there will be easier to help you with your problem on local traffic hitting queues.
I am no expert on queues but will help you as much as i can.

My routerboard is a RB850Gx2

Gateway interface Name=GW1


/ip firewall layer7-protocol
add name=bittorrent regexp="^(\\x13bittorrent protocol|azver\\x01\$|get /scrape\\\?info_hash=ge\
t /announce\\\?info_hash=|get /client/bitcomet/|GET /data\\\?fid=)|d1:ad2:id20:|\\x08'7P\\)\
[RP]"
add name=streaming regexp=videoplayback|video|stream
add name=streaming-audio regexp=audioplayback|audio

/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS-IN new-connection-mark=DNS port=53 \
protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=output comment=DNS-UT new-connection-mark=DNS_UP port=53 \
protocol=udp
add action=mark-packet chain=output connection-mark=DNS_UP new-packet-mark=DNS_UP passthrough=\
no
add action=mark-connection chain=prerouting comment=ICMP-IN in-interface=GW1 \
new-connection-mark=ICMP-CON protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP-CON new-packet-mark=ICMP \
passthrough=no
add action=mark-connection chain=postrouting comment=ICMP-UT new-connection-mark=ICMP_UP-CON \
out-interface=GW1 protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP_UP-CON new-packet-mark=ICMP_UP \
passthrough=no
add action=mark-connection chain=prerouting comment="WinBox, Sniffer" new-connection-mark=\
WinBox-CON port=8291,8081 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=WinBox-CON new-packet-mark=WinBox \
passthrough=no
add action=mark-connection chain=postrouting comment="WinBox, Sniffer-UT" new-connection-mark=\
WinBox-CON_UP port=8291,8081 protocol=tcp
add action=mark-packet chain=postrouting connection-mark=WinBox-CON_UP new-packet-mark=\
WinBox_UP passthrough=no
add action=mark-connection chain=prerouting comment=SlingBox-IN in-interface=GW1 \
new-connection-mark=SLINGBOX-CON port=5001,5757 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=SLINGBOX-CON new-packet-mark=SLINGBOX \
passthrough=no
add action=mark-connection chain=postrouting comment=SlingBox-UT new-connection-mark=\
SLINGBOX_UP out-interface=GW1 port=5001 protocol=tcp
add action=mark-packet chain=postrouting connection-mark=SLINGBOX_UP new-packet-mark=\
SLINGBOX_UP passthrough=no
add action=mark-connection chain=prerouting comment=Stream-AUDIO layer7-protocol=\
streaming-audio new-connection-mark=STREAM-CON_AUDIO
add action=mark-packet chain=prerouting connection-mark=STREAM-CON_AUDIO new-packet-mark=\
STREAM_AUDIO passthrough=no
add action=mark-connection chain=prerouting comment=Stream-VIDEO layer7-protocol=streaming \
new-connection-mark=STREAM-CON
add action=mark-packet chain=prerouting connection-mark=STREAM-CON new-packet-mark=STREAM \
passthrough=no
add action=mark-connection chain=prerouting comment=HTTP-IN connection-mark=!HTTP_BIG \
in-interface=GW1 new-connection-mark=HTTP port=80,443,8080 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=500000-0 connection-mark=HTTP \
new-connection-mark=HTTP_BIG protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=ACK-HTTP-BIG \
packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=ACK-HTTP \
packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG \
passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=\
no
add action=mark-connection chain=postrouting comment=HTTP_UT connection-mark=!HTTP_BIG_UT \
new-connection-mark=HTTP_UT out-interface=GW1 port=80,443,8080 protocol=tcp
add action=mark-connection chain=postrouting connection-bytes=500000-0 connection-mark=HTTP_UT \
new-connection-mark=HTTP_BIG_UT protocol=tcp
add action=mark-packet chain=postrouting connection-mark=HTTP_BIG_UT new-packet-mark=\
ACK-HTTP-BIG_UP packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting connection-mark=HTTP_UT new-packet-mark=ACK-HTTP_UT \
packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting connection-mark=HTTP_BIG_UT new-packet-mark=\
HTTP_BIG_UT passthrough=no
add action=mark-packet chain=postrouting connection-mark=HTTP_UT new-packet-mark=HTTP_UT \
passthrough=no
add action=mark-connection chain=prerouting comment=SNMP new-connection-mark=SNMP-CON port=161 \
protocol=udp
add action=mark-packet chain=prerouting connection-mark=SNMP-CON new-packet-mark=SNMP
add action=mark-connection chain=postrouting comment=SNMP_UP new-connection-mark=SNMP-CON_UP \
port=161 protocol=udp
add action=mark-packet chain=postrouting connection-mark=SNMP-CON_UP new-packet-mark=SNMP_UP
add action=mark-connection chain=prerouting comment=BITTORRENT layer7-protocol=bittorrent \
new-connection-mark=BITTORRENT
add action=mark-packet chain=prerouting connection-mark=BITTORRENT new-packet-mark=BITTORRENT \
passthrough=no
add action=mark-connection chain=postrouting comment=BITTORRENT_UP layer7-protocol=bittorrent \
new-connection-mark=BITTORRENT_UP out-interface=GW1
add action=mark-packet chain=postrouting connection-mark=BITTORRENT_UP new-packet-mark=\
BITTORRENT_UP passthrough=no
add action=mark-connection chain=prerouting comment=THE_REST connection-mark=no-mark \
in-interface=GW1 new-connection-mark=THE_REST-CON
add action=mark-packet chain=prerouting connection-mark=THE_REST-CON new-packet-mark=THE_REST \
packet-mark=!Work passthrough=no
add action=mark-connection chain=postrouting comment=THE_REST-UP connection-mark=no-mark \
new-connection-mark=THE_REST_UP-CON out-interface=GW1
add action=mark-packet chain=postrouting connection-mark=THE_REST_UP-CON new-packet-mark=\
THE_REST_UP passthrough=no

/queue tree
add name=Down parent=global queue=default
add name=Up parent=global queue=default
add name=THE_REST packet-mark=THE_REST parent=Down priority=7 queue=default
add name=THE_REST_UP packet-mark=THE_REST_UP parent=Up priority=7 queue=default
add name=HTTP packet-mark=HTTP parent=Down priority=3 queue=default
add name=HTTP_UP packet-mark=HTTP_UT parent=Up priority=3 queue=default
add name=HTTP_BIG_UP packet-mark=HTTP_BIG_UT parent=Up priority=6 queue=default
add name=HTTP_BIG packet-mark=HTTP_BIG parent=Down priority=7 queue=default
add name=ACK packet-mark=ACK-HTTP parent=Down priority=1 queue=default
add name=ACK_UP packet-mark=ACK-HTTP_UT parent=Up priority=1 queue=default
add name=DNS packet-mark=DNS parent=Down priority=2 queue=default
add name=DNS_UP packet-mark=DNS_UP parent=Up priority=2 queue=default
add name=ICMP packet-mark=ICMP parent=Down priority=2 queue=default
add name=ICMP_UP packet-mark=ICMP_UP parent=Up priority=2 queue=default
add name=SLINGBOX packet-mark=SLINGBOX parent=Down priority=6 queue=default
add max-limit=128k name=SLINGBOX_UP packet-mark=SLINGBOX_UP parent=Up queue=default
add name=BITTORRENT packet-mark=BITTORRENT parent=Down queue=default
add max-limit=64k name=BITTORRENT_UP packet-mark=BITTORRENT_UP parent=Up queue=default
add name=Stream packet-mark=STREAM parent=Down priority=5 queue=pcq-download-default
add name=STREAM_UP packet-mark=STREAM_UP parent=Up priority=4 queue=default
add name=STREAM-AUDIO packet-mark=STREAM_AUDIO parent=Down priority=4 queue=default
add name=WinBox packet-mark=WinBox parent=Down priority=2 queue=default
add name=WinBox_UP packet-mark=WinBox_UP parent=Up priority=2 queue=default
add name=ACK-HTTP-BIG packet-mark=ACK-HTTP-BIG parent=Down priority=1 queue=default
add name=ACK-HTTP-BIG_UP packet-mark=ACK-HTTP-BIG_UP parent=Up priority=1 queue=default
add name=SNMP packet-mark=SNMP parent=Down priority=2 queue=default
add name=SNMP_UP packet-mark=SNMP_UP parent=Up priority=2 queue=default

--------------------
 
toxicfusion
Member
Member
Posts: 324
Joined: Mon Jan 14, 2013 6:02 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Mar 27, 2015 8:04 pm

Below are my mangle rules

I use similar / same QoS for customers and they experience same issues on local UPLOAD traffic being clamped. meaning, if I do a file transfer over the network from one computer to another -- that transfer gets capped.

Mangle rules are essentially same for myself and i use for clients

/ip firewall mangle
add action=mark-connection chain=prerouting comment=VOIP_TRAFFIC \
new-connection-mark=VOIP protocol=udp src-port=5060-5099
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=\
VOIP passthrough=no protocol=udp src-port=5060-5099
add action=mark-connection chain=prerouting comment=DNS dst-port=53 \
new-connection-mark=DNS protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS dst-port=53 \
new-packet-mark=DNS passthrough=no protocol=udp
add action=mark-connection chain=postrouting connection-mark=DNS \
connection-state=new dst-port=53 new-connection-mark=DNS protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS dst-port=53 \
new-packet-mark=DNS packet-mark=DNS passthrough=no protocol=udp
add action=mark-connection chain=prerouting comment="HTTP TRAFFIC" dst-port=80 \
new-connection-mark=HTTP protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP dst-port=80 \
new-packet-mark=HTTP passthrough=no protocol=tcp
add action=mark-connection chain=prerouting comment=HTTPS_Traffic dst-port=443 \
new-connection-mark=HTTPS protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTPS dst-port=443 \
new-packet-mark=HTTPS passthrough=no protocol=tcp
add action=mark-connection chain=postrouting comment="IPSEC TRAFFIC" \
new-connection-mark=IPSEC protocol=gre
add action=mark-packet chain=postrouting connection-mark=IPSEC new-packet-mark=\
IPSEC passthrough=no protocol=gre
add action=mark-connection chain=postrouting new-connection-mark=IPSEC \
protocol=ipsec-esp
add action=mark-packet chain=postrouting connection-mark=IPSEC new-packet-mark=\
IPSEC passthrough=no protocol=ipsec-esp
add action=mark-connection chain=postrouting dst-port=500 new-connection-mark=\
IPSEC protocol=udp
add action=mark-packet chain=postrouting connection-mark=IPSEC dst-port=500 \
new-packet-mark=IPSEC protocol=udp
add action=mark-connection chain=prerouting comment=OTHER in-interface=\
ether1_wan01 new-connection-mark=no-mark
add action=mark-packet chain=prerouting connection-mark=no-mark in-interface=\
ether1_wan01 new-packet-mark=no-mark passthrough=no
add action=mark-connection chain=postrouting comment=TORRENTS \
new-connection-mark=p2p_conns p2p=all-p2p
add action=mark-packet chain=postrouting connection-mark=p2p_conns \
new-packet-mark=p2p_traff p2p=all-p2p passthrough=no
 
kivimart
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Thu Oct 10, 2013 3:06 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Mar 27, 2015 8:17 pm

I put my master interface in a bridge and the problem off cap local traffic went away.
 
toxicfusion
Member
Member
Posts: 324
Joined: Mon Jan 14, 2013 6:02 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Apr 01, 2015 5:17 pm

Thanks to know!

I have RB2011 routers out in field with similar configuration, except VLAN's and they're using BRIDGES.

So they might be fine.

It might just be an issue with this RB1100AH as I'm using 'router on stick' configuration with VLAN's assigned to ether-trunks

I'm worried that If I put the trunks into a bridge, it will have a huge negative impact on performance?
 
EngineerAustin
just joined
Posts: 6
Joined: Sat Jun 20, 2015 6:57 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Sat Jun 20, 2015 7:41 pm

I am new to VOIP prioritization and I need a bit of advice and it looks like I'm at the correct forum.

I have a multiple site network with multiple subnet vlans:

Main Location Connections:
1) 50/50Mb fiber internet, protected with sonicwall 2600 firewall (not my choice of equipment, but what I have to use)
2) 20/20Mb fiber ethernet MPLS provided by ISP to circuit to remote locations

I have a mikrotik router that I have configured at main to do NAT & Routing with vlans.
the way I have configured the MikroTik is this:
1 subnet for VOIP VLAN network 51.0/24 DHCP on MikroTik VLAN Bridge, 1 access port for local network switch. PBX onsite.
1 subnet for local computer DATA VLAN network 50.0/24 DHCP on MikroTik VLAN Bridge, 1 access port, for Local Network Switch computers, printers, and all other
3 x subnet for remote sites DATA VLAN network 40.0/24, 30.0/24, 20.0/24 DHCP on MikroTik VLAN for computers, printers, and all other at remote sties that need to communicate with each other site
1 MPLS (ether5) Trunk Port with 4 DATA Vlans & 1 VOIP VLAN51 (20/20Mb to MPLS ether switch)

The remote sites are all identical and look like this:
1) access port for VOIP VLAN 51 (ether1)
2) access port for MAIN Data VLAN 51 (ether2)
3) access port for Local Network Vlan, 40, 30, 20 (ether3)
4) ether4 spare
5) MPLS (ether5) Trunk Port with 4 DATA Vlans & 1 VOIP VLAN51 (5/5Mb to MPLS ether switch)

I need QoS for VLAN 51 prioritized over all other traffic except primary default network traffic for routing.

Am I at the right spot? Please contact me if you can help or check over my setup and help with QoS.

Thank you for your time in this. This forum seemed to be pointing me in the correct direction with QoS, but I'm a little confused how to prioritize VLAN 51, at remote sites and then at the main location with different limits?

Also, I was wondering if I could just prioritize all VLAN 51 VOIP subnet, or if I needed to do port prioritization too.

Any guidance would be greatly appreciated.

Sincerely,

Lynn
 
alaskanjackal
newbie
Posts: 27
Joined: Tue Sep 29, 2015 1:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Sep 29, 2015 1:57 pm

I spent a gazillion hours tonight Googling and working through a bunch of different threads on the topic of QOS. (I'm new to Mikrotik and was going to wade in slowly, but I encountered some choppy voice on a test call today and thought I'd try to clean that up.)

I started with the script posted upthread by pcunite, but after reading some of the documentation, I made some changes and thought I'd post them here to share (in case it helps anyone, or in case someone would like to fix any massive errors I've created).

One of the issues with pcunite's script is that you have to back off from the rated line speed by 10% or so in order to give Queue A (the one with the VoIP markings) a chance to breathe, since there doesn't appear to be any logic that actually prioritizes Queue A over Queue C--it just basically relies on the fact that Queue C can't quite use the entire connection to prevent Queue A's packets from queuing at the ISP edge (in my case, the cable modem).

I think I've encountered a way to get a bit closer to the actual maximum throughput and let HTTP and other bulk downloads use the full connection without compromising VoIP (and other priority) packets. This is done by making all of the traffic classes child queues of a single parent queue and then ordering these child queues in priority--my understanding in the Mikrotik wiki documentation is that this should work.

I did some testing and it seems to work when I forced the link to saturation with HTTP traffic (there was no audible VoIP packet loss, and HTTP traffic showed a small amount of packets in the queue while the VoIP queue showed traffic and no queuing, indicating they were passed through without delay).

In my case, I'm on a 50m down, 3m up cable connection, and my ISP does ever so slightly overprovision the connection (~51mbps down and ~3.2mbps up, and they do a pretty good job of giving me my rated speed 24/7), so I wrote the script to push me right to the edge of 50M/3M, and it seems to work OK and gives me just barely shy of those figures to my workstation all while keeping the VoIP connection clean and ICMP pings to nearby servers in the single digits. That said, this is really just a test and a draft and a quick hack of the script (plus an attempt to remove a few of the seemingly redundant lines), so for those just stumbling upon this thread from Google, don't put this into production without it being vetted by the more experienced community members here.

I also threw in a few extra things--some lines (with limit-at=) to guarantee bandwidth for at least a couple of concurrent calls--this may be unnecessary since it's first in the priority list, but I figured it couldn't hurt. I also capped the bandwidth of VoIP and other priority traffic to a shade less than my actual connection, just in case for some reason something happens and a UDP flood attempts to monopolize my upstream bandwidth--there will be a little left over for HTTP and other traffic.

One thing--both the original script by pcunite and this one seem to fail to properly mark the HTTP_BIG traffic. I don't know if there have been any changes in RouterOS since pcunite last updated his script that may affect that, but I spent a good two hours working on trying to get that to work to no avail. Ultimately, I decided that as long as VoIP traffic is properly classified and prioritized and ACK packets are given priority on the uplink (which helps with the browser feeling responsive even on a saturated uplink), the need to separately classify bulk uploads is mitigated. Perhaps someone can troubleshoot that further.

Oh, one other thing--I did change the VoIP mangle rules to go by DSCP markings rather than ports, since my VoIP infrastructure does mark packets, but if yours doesn't, you'll want to uncomment the port-based matching lines and delete the ones that reference DSCP.

Anyway, it seems to be working here, but there's a very good chance I just got lucky in my testing and it really isn't working at all or something, so hopefully someone will come along to stop me from spreading misinformation if that's the case...but I hope this helps in some way.

# Date: September 29, 2015
# Version: 1.3b1, based on work by pcunite
# Tested with RouterOS 6.32.1
# Rename ether1-gateway and bridge-local to match your environment



###############################################################################
# Mangle
#
# Using prerouting/postrouting since we don't have dst or src checks.
#
/ip firewall mangle
###############################################################################

# DNS requests. Mark in two places because DNS is sent out by the router (itself) too.
add chain=prerouting action=mark-connection protocol=udp port=53 connection-state=new new-connection-mark="DNS" comment="DNS"
add chain=postrouting action=mark-connection protocol=udp port=53 connection-state=new new-connection-mark="DNS"
add chain=postrouting action=mark-packet passthrough=no connection-mark="DNS" new-packet-mark="DNS"

# Mark all VoIP TCP traffic based on DSCP marking 46 and 26.
add chain=prerouting action=mark-connection protocol=tcp dscp=46 new-connection-mark="VOIP" comment="VOIP"
add chain=prerouting action=mark-connection protocol=udp dscp=46 new-connection-mark="VOIP"
add chain=prerouting action=mark-connection protocol=tcp dscp=26 new-connection-mark="VOIP"
add chain=prerouting action=mark-connection protocol=udp dscp=26 new-connection-mark="VOIP"
add chain=prerouting action=mark-packet passthrough=no connection-mark="VOIP" new-packet-mark="VOIP"

# Mark all VoIP traffic. We've set all our equiptment to use SIP 5060,5061 and RTP 10000-20000.
# add chain=prerouting action=mark-connection protocol=udp port=5060,5061,10000-20000 new-connection-mark="VOIP" comment="VOIP"
# add chain=prerouting action=mark-packet passthrough=no connection-mark="VOIP" new-packet-mark="VOIP"

# Mark all UDP traffic. Mark different UDP streams if you want more granularity.
add chain=prerouting action=mark-connection protocol=udp connection-state=new new-connection-mark="UDP" comment="UDP"
add chain=prerouting action=mark-packet passthrough=no connection-mark="UDP" new-packet-mark="UDP"

# Ping replies. Mark in two places because ICMP is sent out by the router (itself) too.
add chain=prerouting action=mark-connection protocol=icmp connection-state=new new-connection-mark="ICMP" comment="ICMP"
add chain=postrouting action=mark-connection protocol=icmp connection-state=new new-connection-mark="ICMP"
add chain=postrouting action=mark-packet passthrough=no connection-mark="ICMP" new-packet-mark="ICMP"

# ACK traffic. Based on viewtopic.php?f=2&t=67965
add chain=postrouting action=mark-packet passthrough=no protocol=tcp tcp-flags=ack packet-size=0-123 new-packet-mark="ACK" comment="ACK"
add chain=prerouting action=mark-packet passthrough=no protocol=tcp tcp-flags=ack packet-size=0-123 new-packet-mark="ACK"

# Mark all new HTTP(s) connections with "HTTP" if they have not previously been marked as "HTTP_BIG".
# If the current mark of "HTTP" tranfers more than 5MB and at a rate of 200k+ then mark it as "HTTP_BIG" for the duration of the TCP session.
add chain=prerouting action=mark-connection protocol=tcp new-connection-mark="HTTP" connection-state=new port=80,443 comment="HTTP"
add chain=prerouting action=mark-connection protocol=tcp connection-mark="HTTP" new-connection-mark="HTTP_BIG" connection-bytes=500000-0 connection-rate=200k-1000M
add chain=prerouting action=mark-packet passthrough=no connection-mark="HTTP_BIG" new-packet-mark="HTTP_BIG"
add chain=prerouting action=mark-packet passthrough=no connection-mark="HTTP" new-packet-mark="HTTP"

# Mark everything else that has no mark applied.
add chain=prerouting action=mark-connection connection-mark=no-mark new-connection-mark="OTHER" comment="OTHER"
add chain=prerouting action=mark-packet passthrough=no connection-mark="OTHER" new-packet-mark="OTHER"



###############################################################################
# HTB Queue Tree a unidirectional queue
#
# Based on 50M down / 3M up connection (measured at 50400k/50.4M down, 3008k/3.0M up).
# Replace with your own values as needed and adjust downward if connection is unreliable.
#
# Notes:
# priority means 'drop packets' WHEN needed.
# When limit-at=0 priority starts when max-limit is reached.
# When limit-at=123 priority starts when limit-at is reached.
#
# The priority option applies to children not parents. Parent is for setting
# overall limits. Therefore use limit-at and max-limit on the children if
# you want more granularity.
#
# max-limit must always be set or priority will not happen.
#
# Tips for TCP (not VoIP) SOHO network:
# limit-at = Total bandwidth / max hosts
# max-limit = Total bandwidth / min hosts
#
/queue tree
###############################################################################

# The secret to ensuring VoIP quality (or any UDP traffic) is to put it into
# a queue that will never be full and thus never prioritize (drop) packets.
add name="MASTER_UP" parent=ether1-gateway queue=default max-limit=3000k
add name="MASTER_DOWN" parent=bridge-local queue=default max-limit=50000k

add name="VOIP_U" parent="MASTER_UP" packet-mark="VOIP" queue=default limit-at=200k max-limit=2850k priority=1
add name="VOIP_D" parent="MASTER_DOWN" packet-mark="VOIP" queue=default limit-at=200k max-limit=45000k priority=1
add name="ACK_U" parent="MASTER_UP" packet-mark="ACK" queue=default limit-at=200k max-limit=2850k priority=2
add name="ACK_D" parent="MASTER_DOWN" packet-mark="ACK" queue=default limit-at=2000k max-limit=45000k priority=2
add name="DNS_U" parent="MASTER_UP" packet-mark="DNS" queue=default limit-at=200k max-limit=2850k priority=3
add name="DNS_D" parent="MASTER_DOWN" packet-mark="DNS" queue=default limit-at=2000k max-limit=45000k priority=3
add name="UDP_U" parent="MASTER_UP" packet-mark="UDP" queue=default limit-at=200k max-limit=2850k priority=4
add name="UDP_D" parent="MASTER_DOWN" packet-mark="UDP" queue=default limit-at=2000k max-limit=45000k priority=4
add name="ICMP_U" parent="MASTER_UP" packet-mark="ICMP" queue=default limit-at=200k max-limit=2850k priority=5
add name="ICMP_D" parent="MASTER_DOWN" packet-mark="ICMP" queue=default limit-at=2000k max-limit=45000k priority=5
add name="HTTP_U" parent="MASTER_UP" packet-mark="HTTP" queue=default limit-at=200k max-limit=3000k priority=6
add name="HTTP_D" parent="MASTER_DOWN" packet-mark="HTTP" queue=default limit-at=2000k max-limit=50000k priority=6
add name="HTTP_BIG_U" parent="MASTER_UP" packet-mark="HTTP_BIG" queue=default limit-at=200k max-limit=3000k priority=7
add name="HTTP_BIG_D" parent="MASTER_DOWN" packet-mark="HTTP_BIG" queue=default limit-at=2000k max-limit=50000k priority=7
add name="OTHER_U" parent="MASTER_UP" packet-mark="OTHER" queue=default limit-at=200k max-limit=3000k priority=8
add name="OTHER_D" parent="MASTER_DOWN" packet-mark="OTHER" queue=default limit-at=2000k max-limit=50000k priority=8
 
freemannnn
Forum Veteran
Forum Veteran
Posts: 700
Joined: Sun Oct 13, 2013 7:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Sep 29, 2015 7:56 pm

dear "alaskanjackal" giving higher priority to udp packets is not good when u download torrents, right?
i think i will place them under http_big
 
alaskanjackal
newbie
Posts: 27
Joined: Tue Sep 29, 2015 1:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Sep 29, 2015 8:35 pm

dear "alaskanjackal" giving higher priority to udp packets is not good when u download torrents, right?
i think i will place them under http_big
Perhaps not. I don't (often) torrent, so that isn't a concern. But that was in pcunite's original script, and while prioritizing UDP traffic like that wouldn't necessarily have been something I would have thought to do if I were writing a script myself, I saw no reason to change it.

There are other example scripts on the forums here that do a better job of classifying and deprioritizing torrent traffic, though none of the scripts I came across are perfect--most are just suggestions/ideas that require some heavy editing to get working. Despite its limited scope, the ones brought up in this thread are the most functional, at least from what I saw.
 
alaskanjackal
newbie
Posts: 27
Joined: Tue Sep 29, 2015 1:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Sep 30, 2015 2:00 am

OK, so I just did a 1-hour long test (playing music through the line) while completely saturating my connection with multiple concurrent speed tests, uploads, downloads, and several very large Linux torrents going at once, and aside from one 20-second episode during the whole hour of some lightly choppy upstream audio, the VoIP call performed flawlessly.

I realized that shaping UDP downstream is pointless because it won't cause any slowdown on the transmission, so there is no point in throwing away UDP packets that have managed to make it as far as my edge router. Perhaps it's better just to not mark downstream UDP traffic or to put it in a separate (unlimited) queue or something, so I'll mess with that later.

Also, there's still some weirdness going on with HTTP and HTTP_BIG--stuff that isn't big is going into BIG and stuff that is BIG is not, and then other stuff that should go into BIG gets left over and goes into OTHER. I'm not super worried about it, as the setup will likely work fine for daily use (and still prioritize VoIP, which is the main goal here), but it causes some unpredictable weirdness in choking/throttling larger uploads for no reason (occasionally, suddenly all upstream HTTP traffic will zero out for several seconds or even longer until it eases back to link speed). I probably need a better understanding of mangling and queue priorities in Mikrotik to diagnose that much more.
 
freemannnn
Forum Veteran
Forum Veteran
Posts: 700
Joined: Sun Oct 13, 2013 7:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Sep 30, 2015 8:39 am

connection-rate is broken from firmware 6.28 and after. i dont understand why mikrotik dont fix it.
http://forum.mikrotik.com/viewtopic.php ... te#p501644
downgrade to 6.28 and you will see that it works!
thank you for the nice qos script
 
alaskanjackal
newbie
Posts: 27
Joined: Tue Sep 29, 2015 1:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Oct 02, 2015 10:03 pm

connection-rate is broken from firmware 6.28 and after. i dont understand why mikrotik dont fix it.
http://forum.mikrotik.com/viewtopic.php ... te#p501644
downgrade to 6.28 and you will see that it works!
thank you for the nice qos script
Thanks. I dropped the connection-rate parameter from Mangle and it works now and properly classifies large transfers in HTTP_BIG once 5MB of data has been transferred. This may have negative consequences for some long-lived HTTP TCP connections that don't use up much speed but may, over time, use more than 5MB of data, but I'm thinking it all probably makes very little practical difference in my network.

I also made a couple of additional changes just now: I separated out VOIP into VOIP-TCP and VOIP-UDP. I then put VOIP-UDP_D into a separate queue with a parent of my inbound interface (rather than MASTER_DOWN) without a limit. I also moved the other UDP_D outside of MASTER_DOWN with no limit. I think this is probably good practice, since, as I mentioned above, there's no point in dropping inbound UDP packets if they've already made it this far. I left VOIP-TCP_D in the MASTER_DOWN queue with a limit as well as both VOIP-UDP_U and VOIP-TCP_U under MASTER_UP with limits (very unlikely that I'll ever have more than a few dozen Kbps in VoIP traffic, but you never know what might happen, so why not).

Seems to be working, but I'll play with it a bit more before posting a revised script.
 
haffari
just joined
Posts: 1
Joined: Mon Dec 28, 2015 11:15 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Mon Dec 28, 2015 11:38 pm

Implementing VoIP traffic prioritization (Qos) with RouterOS v6
Option 3 ... coming soon
Hi
Waiting for next parts....................................... :D
 
freemannnn
Forum Veteran
Forum Veteran
Posts: 700
Joined: Sun Oct 13, 2013 7:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Feb 24, 2016 11:55 am

is there a newer version of this qos script? (1.3b1 is the latest i found)
can we use simple queues (pcq or sfq) instead of queue tree?
 
MikroTikOkay
just joined
Posts: 3
Joined: Wed Feb 10, 2016 11:02 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 07, 2016 5:34 pm

connection-rate is broken from firmware 6.28 and after. i dont understand why mikrotik dont fix it.
http://forum.mikrotik.com/viewtopic.php ... te#p501644
downgrade to 6.28 and you will see that it works!
thank you for the nice qos script
Thanks. I dropped the connection-rate parameter from Mangle and it works now and properly classifies large transfers in HTTP_BIG once 5MB of data has been transferred. This may have negative consequences for some long-lived HTTP TCP connections that don't use up much speed but may, over time, use more than 5MB of data, but I'm thinking it all probably makes very little practical difference in my network.

I also made a couple of additional changes just now: I separated out VOIP into VOIP-TCP and VOIP-UDP. I then put VOIP-UDP_D into a separate queue with a parent of my inbound interface (rather than MASTER_DOWN) without a limit. I also moved the other UDP_D outside of MASTER_DOWN with no limit. I think this is probably good practice, since, as I mentioned above, there's no point in dropping inbound UDP packets if they've already made it this far. I left VOIP-TCP_D in the MASTER_DOWN queue with a limit as well as both VOIP-UDP_U and VOIP-TCP_U under MASTER_UP with limits (very unlikely that I'll ever have more than a few dozen Kbps in VoIP traffic, but you never know what might happen, so why not).

Seems to be working, but I'll play with it a bit more before posting a revised script.
alaskanjackal,

Would you mind sharing your revised script? Thanks!
 
porch
just joined
Posts: 7
Joined: Sat Mar 12, 2016 9:51 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Sat Mar 12, 2016 10:24 am

Had anyone got Option 2 to do anything? I have a 951Ui-2HnD 6.33.3 running in a small office with 5 VOIP phones connected to an internet VIOP server. Bandwidth is 8Mbs/1Mbs.
I have Option 2 installed set for 6Mbs/600Kbs. It's tagging the packets right and the queues look good, but nothing happens. Speedtest.net reports the speed being 8Mbs/1Mbs, and if I run Speedtest.net while on a call, it gets all choppy.
So it's not really doing anything.
I have spent the last 4 hours trying to get this to work, and bricked one 951 to the point that netinstall won't see it trying to get this to work.
HELP!!!!!
 
MosquitoCR
just joined
Posts: 2
Joined: Thu Feb 05, 2015 8:14 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Aug 01, 2016 4:49 am

alaskanjackal,

Please send me the updated MK script .
 
IntrusDave
Forum Guru
Forum Guru
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Aug 01, 2016 7:54 am

 
User avatar
nichky
Forum Guru
Forum Guru
Posts: 1389
Joined: Tue Jun 23, 2015 2:35 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Sep 05, 2016 12:08 am

Does anyone know what is happen with option 3? Does it come out?
appreciate you


Thanks
 
User avatar
mrjurek
just joined
Posts: 1
Joined: Sun Dec 18, 2016 11:46 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Dec 19, 2016 12:01 am

alaskanjackal,

Would you mind sharing your new script?
(My mail: mrjurek@poczta.onet.pl)

Thanks.
 
mltobing
just joined
Posts: 11
Joined: Thu Dec 15, 2016 9:19 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Dec 21, 2016 4:33 am

Hello,

I am still using RouterOS 6.27 because connection-rate bug. Is it no fixed in new version ?
If already fixed, please tell me in which version.

Thanks
 
freemannnn
Forum Veteran
Forum Veteran
Posts: 700
Joined: Sun Oct 13, 2013 7:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Dec 21, 2016 9:31 am

is already fix after 6.31 or 6.33. i cant remember exactly
u should install and test bugfix version 6.36.4
 
49er
Member
Member
Posts: 409
Joined: Tue Sep 27, 2011 7:55 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Dec 21, 2016 12:39 pm

Hi,
This is a very interesting topic.
I want to use it on a transparent mikrotik device.
So no router.
Is that also possible?
And than make difference in up and download
 
satskiy
just joined
Posts: 6
Joined: Wed Dec 21, 2016 4:30 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Thu Dec 22, 2016 9:40 am

if i hawe spf as WAN and all lan+ wifi are in bridge
then
Ether-WAN= SPF
Ether-LAN=Bridge right ?
 
mltobing
just joined
Posts: 11
Joined: Thu Dec 15, 2016 9:19 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Dec 26, 2016 4:22 am

is already fix after 6.31 or 6.33. i cant remember exactly
u should install and test bugfix version 6.36.4
RouterOS v6.36.4 working properly, then upgrade to 6.37.3. Bug fix confirmed

Thanks freemannnn
 
alaskanjackal
newbie
Posts: 27
Joined: Tue Sep 29, 2015 1:29 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 20, 2017 1:12 am

alaskanjackal,

Would you mind sharing your new script?
(My mail: mrjurek@poczta.onet.pl)

Thanks.
Sorry for my long absence here.

Here's an updated version of the script (from another device I have deployed, so I think the interface names might be different than my first script and the connection speed is also different--adjust as necessary) that's been running for >1y without any issues.

I do see the report that connection-rate may be fixed in a newer release, but I have not re-incorporated it into my scripts. Perhaps I will someday. For now, here is my latest operating script:

# Date: March 19, 2017
# Version: 1.4, based on work by pcunite
# Tested with RouterOS 6.38.5
# Rename ether1-gateway and bridge-local to match your environment



###############################################################################
# Mangle
#
# Using prerouting/postrouting since we don't have dst or src checks.
#
/ip firewall mangle
###############################################################################

# DNS requests. Mark in two places because DNS is sent out by the router (itself) too.
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no

# Mark all VoIP traffic based on DSCP marking 46 and 26.
add action=mark-connection chain=prerouting comment=VOIP dscp=46 new-connection-mark=VOIP-TCP passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting dscp=46 new-connection-mark=VOIP-UDP passthrough=yes protocol=udp
add action=mark-connection chain=prerouting new-connection-mark=VOIP-UDP passthrough=yes port=4500 protocol=udp
add action=mark-connection chain=prerouting dscp=26 new-connection-mark=VOIP-TCP passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting dscp=26 new-connection-mark=VOIP-UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP-TCP new-packet-mark=VOIP-TCP passthrough=no
add action=mark-packet chain=prerouting connection-mark=VOIP-UDP new-packet-mark=VOIP-UDP passthrough=no

# Mark all UDP traffic. Mark different UDP streams if you want more granularity.
add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no

# Ping replies. Mark in two places because ICMP is sent out by the router (itself) too.
add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no

# ACK traffic. Based on viewtopic.php?f=2&t=67965
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack

# Mark all new HTTP(s) connections with "HTTP" if they have not previously been marked as "HTTP_BIG".
# If the current mark of "HTTP" tranfers more than 5MB and at a rate of 200k+ then mark it as "HTTP_BIG" for the duration of the TCP session.
add action=mark-connection chain=prerouting comment=HTTP connection-state=new new-connection-mark=HTTP passthrough=yes port=80,443,8080 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=500000-0 connection-mark=HTTP new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no

# Mark everything else that has no mark applied.
add action=mark-connection chain=prerouting comment=OTHER connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no

###############################################################################
# HTB Queue Tree a unidirectional queue
#
# Based on 50M down / 5M up connection (measured at 54000k/54.0M down, 5600k/5.6M up).
# Replace with your own values as needed and adjust downward if connection is unreliable.
#
# Notes:
# priority means 'drop packets' WHEN needed.
# When limit-at=0 priority starts when max-limit is reached.
# When limit-at=123 priority starts when limit-at is reached.
#
# The priority option applies to children not parents. Parent is for setting
# overall limits. Therefore use limit-at and max-limit on the children if
# you want more granularity.
#
# max-limit must always be set or priority will not happen.
#
# Tips for TCP (not VoIP) SOHO network:
# limit-at = Total bandwidth / max hosts
# max-limit = Total bandwidth / min hosts
#
/queue tree
###############################################################################

# The secret to ensuring VoIP quality (or any UDP traffic) is to put it into
# a queue that will never be full and thus never prioritize (drop) packets.

add max-limit=5500k name=MASTER_UP parent=ether1-gateway queue=default
add max-limit=53M name=MASTER_DOWN parent=bridge-local queue=default
add limit-at=200k max-limit=5M name=VOIP-TCP_U packet-mark=VOIP-TCP parent=MASTER_UP priority=1 queue=default
add limit-at=200k max-limit=45M name=VOIP-TCP_D packet-mark=VOIP-TCP parent=MASTER_DOWN priority=1 queue=default
add limit-at=200k max-limit=5M name=ACK_U packet-mark=ACK parent=MASTER_UP priority=2 queue=default
add limit-at=4M max-limit=45M name=ACK_D packet-mark=ACK parent=MASTER_DOWN priority=2 queue=default
add limit-at=200k max-limit=5M name=DNS_U packet-mark=DNS parent=MASTER_UP priority=3 queue=default
add limit-at=4M max-limit=45M name=DNS_D packet-mark=DNS parent=MASTER_DOWN priority=3 queue=default
add limit-at=200k max-limit=5500k name=UDP_U packet-mark=UDP parent=MASTER_UP priority=4 queue=default
add name=UDP_D packet-mark=UDP parent=bridge-local priority=4 queue=default
add limit-at=200k max-limit=5M name=ICMP_U packet-mark=ICMP parent=MASTER_UP priority=5 queue=default
add limit-at=4M max-limit=45M name=ICMP_D packet-mark=ICMP parent=MASTER_DOWN priority=5 queue=default
add limit-at=200k max-limit=5500k name=HTTP_U packet-mark=HTTP parent=MASTER_UP priority=6 queue=default
add limit-at=4M max-limit=53M name=HTTP_D packet-mark=HTTP parent=MASTER_DOWN priority=6 queue=default
add limit-at=200k max-limit=5500k name=HTTP_BIG_U packet-mark=HTTP_BIG parent=MASTER_UP priority=7 queue=default
add limit-at=4M max-limit=53M name=HTTP_BIG_D packet-mark=HTTP_BIG parent=MASTER_DOWN priority=7 queue=default
add limit-at=200k max-limit=5500k name=OTHER_U packet-mark=OTHER parent=MASTER_UP queue=default
add limit-at=4M max-limit=53M name=OTHER_D packet-mark=OTHER parent=MASTER_DOWN queue=default
add name=VOIP-UDP_D packet-mark=VOIP-UDP parent=bridge-local priority=1 queue=default
add limit-at=200k max-limit=5M name=VOIP-UDP packet-mark=VOIP-UDP parent=MASTER_UP priority=1 queue=default

Note that the limit-at amounts could probably use some playing around with, and (as I mentioned above) connection-rate could probably be re-implemented. But this ruleset seems to be working pretty bulletproof for me--allowing me access to very, very near my max possible throughput while preventing any sort of connection issues on VoIP calls.
 
User avatar
nichky
Forum Guru
Forum Guru
Posts: 1389
Joined: Tue Jun 23, 2015 2:35 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Apr 19, 2017 12:11 am

Can i implement PCQ on that script? i mean..for e.g.

add limit-at=200k max-limit=5M name=ACK_U packet-mark=ACK parent=MASTER_UP priority=2 queue=PCQ_U

And also i would like to specified any IP_Subnet for e.g. (address list) can i do this?

if it is posible will be exselent :))

Thanks a lot
 
User avatar
nichky
Forum Guru
Forum Guru
Posts: 1389
Joined: Tue Jun 23, 2015 2:35 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Thu Apr 20, 2017 4:47 am

just I'm wondering it' possible to do, on second part of QoS on this post, i wold like to do with Address List.Can anyone help me?

Thanks
 
korcom
just joined
Posts: 17
Joined: Fri Aug 04, 2017 3:38 pm
Location: Johannesburg
Contact:

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Aug 07, 2017 10:48 am

@alaskanjackal your scripts are incredible.

How would I implement it with 2 WANs both with different speeds?

Thank you
 
marwooj
newbie
Posts: 40
Joined: Mon Nov 06, 2017 10:44 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Mar 07, 2018 7:23 pm

Implementing VoIP traffic prioritization (Qos) with RouterOS v6 – Option 2

If I have s2s ipsec VPN will it go into "OTHER"?
 
marwooj
newbie
Posts: 40
Joined: Mon Nov 06, 2017 10:44 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Wed Mar 07, 2018 7:36 pm

Implementing VoIP traffic prioritization (Qos) with RouterOS v6 – Option 2
I also do not understand how LEVEL_A_ gets priority over LEVEL_B_ and then over LEVEL_C_.
Would somebody be so kind explain this?
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sun Mar 11, 2018 10:55 pm

Implementing VoIP traffic prioritization (Qos) with RouterOS v6 – Option 2

If I have s2s ipsec VPN will it go into "OTHER"?
In my opinion, the only way to understand how the traffic prioritization works is to try to implement a simple one on your own and debug it, not to try to understand someone else's complex example like the one above. Once you understand how it works using your simple case, reading the complex example becomes much easier.

So as I wrote in the other thread, you have to prioritize VoIP traffic among everything you send through the uplink. To do that, you have to set a DSCP mark (better to use the standard one, i.e. 46, for VoIP media and maybe also signalling) as DSCP is the only attribute of the packet which remains unchanged after IPsec encryption and encapsulation.

To answer your question - when talking about the classifier rules in the example above, the IPSec transport packets would definitely fall into the OTHER group, while the packets before encryption and encapsulation could fall into any of the groups depending on the criteria. So if you would use that script without modification of the classifier rules, you would prioritize VoIP traffic among everything that is going to be sent encrypted, but that would be useless because you would give the lowest (OTHER) priority to all the encrypted traffic.

So the whole implementation would have to include
  • for the upload direction:
    • mangle rules setting DSCP value for anything which is going to be sent via the IPsec tunnel (it is possible that the phones already do that, but you also need to eventually remove priority marking from other than VoIP packets should it eventually be set on them)
    • mangle rules marking connections and packets going to be sent via the uplink, including the IPsec transport packets, based on DSCP and other criteria
    • queue setup prioritizing the packets going to be sent via the uplink based on the marks assigned in the step above
  • for the download direction:
    • mangle rules marking connections and packets received based on protocol type and, possibly, DSCP marking (if it survives transit through internet as stated earlier)
    • queue setup throttling the low priority packets in such a way that the necessary amount of download bandwidth is kept free for the priority packets.
The download margin you have to reserve for the priority packets will differ significantly depending on whether the DSCP markings survive the transit or not. If they do, you may keep the margin only as big as to accommodate the incoming VoIP traffic; if they don't, you have to limit the IPsec transport bandwidth as a whole on the remote site upload and reserve enough space for all of it at local download because in such case, there is no way to freely distribute the "non-VoIP" download bandwidth between the tunnelled and non-tunnelled traffic.
 
marwooj
newbie
Posts: 40
Joined: Mon Nov 06, 2017 10:44 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 12, 2018 1:47 pm

I think I will forget about putting VoIP into VPN, this definitely sounds to complicated. And I will try to tune the example I have already implemented. Just after I understood how LEVEL_A_ gets priority over LEVEL_B_ and then over LEVEL_C_ :-)
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 12, 2018 2:55 pm

I think I will forget about putting VoIP into VPN, this definitely sounds to complicated. And I will try to tune the example I have already implemented. Just after I understood how LEVEL_A_ gets priority over LEVEL_B_ and then over LEVEL_C_ :-)
Funny. To me the idea of setting the DSCP field doesn't sound complicated at all as it is basically a single mangle rule. What may be complicated is to distinguish VoIP packets from non-VoIP ones but if you reserve an address range or subnet on each site for your VoIP devices, or if you can ask the VoIP devices to set the DSCP themselves, it is also not very complex. Softphones are a problem unless they can set DSCP themselves because they cannot be identified by IP address and the PCs they run at generate both kinds of traffic.

I mean, once you pushstart the prioritization as such, adding VoIP needs just a tiny bit compared to the overall effort.

What I still have to try is to practically deploy the queues, I didn't have the need so far.

But I've had a look at the setup with LEVEL_X and it seems to me that LEVEL_A and LEVEL_B have no priority over each other, they simply each have their dedicated share of the total bandwidth as they have a common parent queue which doesn't have any bandwidth limits set. So both LEVEL_A and LEVEL_B may reach their limits simultaneously, so the summary traffic may be 7800k up and 82M down, but none of them may "borrow" the other one's bandwidth even if it is completely unused. So GROUP_A and GROUP_B would be more appropriate names.
 
marwooj
newbie
Posts: 40
Joined: Mon Nov 06, 2017 10:44 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 12, 2018 3:10 pm

Looks like my PBX boxes are adding DSCP = 40 already, I just afraid that by putting this traffic into VPN will add unnecessary overheat and I will end up with having bigger disaster that I have now.

So to make LEVEL_B and LEVEL_C always lower priority than LEVEL_A I will change parent of B and C to A?
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 12, 2018 3:39 pm

Looks like my PBX boxes are adding DSCP = 40 already, I just afraid that by putting this traffic into VPN will add unnecessary overheat and I will end up with having bigger disaster that I have now.
Do you use VoIP phones or analog/ISDN phones and only the PBXes would talk VoIP to each other? If VoIP phones, do the PBXes tell the phones to send media to each other directly or do they force themselves into the media path? All that plays a role - if VoIP phones send media directly to each other, they must set DSCP or you must do it on their behalf when forwarding the packets to the remote site.

So to make LEVEL_B and LEVEL_C always lower priority than LEVEL_A I will change parent of B and C to A?
Yes but not only, you must also give the LEVEL_B queue the lowest priority among all the other child queues of LEVEL_A. And LEVEL_C must have the lowest priority among all child queues of LEVEL_B.

The queues play two roles - one is speed limitation and another one is prioritization. Speed limit doesn't allow more data to be sent even if bandwidth is available. Priority determines the order of searching through the queues at each level for the next packet to be sent once the "line" gets free after the previous packet has been sent.

On the upload, it is basically enough for QoS to do the prioritization, speed limitation is only necessary if you want to guarantee some minimum bandwidth to individual users.

On the download, you cannot actually affect the priority with which the ISP is sending the packets down your connection, so you must make use of the TCP's flow control - if you limit the speed with which received TCP packets are delivered from WAN to LAN, the remote senders limit their sending speed accordingly as they wait for ACKs from the recipients before sending another burst of packets. There is no point in throttling incoming UDP as it has no effect on the sender. So the trick is to limit the bandwidth for TCP in download direction and leave the rest for the peak amount of UDP you can expect (which is some 100k per audio RTP stream, but you have to inspect your traffic to see what else on top of RTP, DNS responses, NTP responses arrives to you as UDP). You'll likely also see some icmp there.
 
marwooj
newbie
Posts: 40
Joined: Mon Nov 06, 2017 10:44 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 12, 2018 3:59 pm

I also have ipsec s2s not for the purpose of VoIP, to completely different site. It is mostly RDP.
Since IPsec s2s is also UDP (I think), how I can put it "to not disturb" my VoIP but not to go into LEVEL_C / OTHER?
Last edited by marwooj on Tue Mar 13, 2018 10:23 am, edited 1 time in total.
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Mar 12, 2018 4:43 pm

I also have ipsec s2s not for the purpose of VoIP, to completely different site. It is moatly RDP.
Since IPsec s2s is also UDP (I think), how I can put it "to not disturb" my VoIP but not to go into LEVEL_C / OTHER?
That's the role of DSCP as mentioned in the other (s2s) thread. The "plaintext" traffic on the WAN can be classified as "TCP" and "other protocols" directly; the IPsec transport packets will either be ESP ones or UDP ones (depending on existence of NAT in the path) with a particular source and destination; whether "TCP" or "other" is encrypted inside them can only be determined if the sending side sets the DSCP field and the transit through internet doesn't destroy it completely.

On upload, you have to send the packets to the IPsec machinery without any throttling, only set their DSCP. The outgoing prioritization will take place on the plaintext and IPsec transport packets together, where the DSCP shows to which group or level the IPsec transport packets belong.

On download, there is a special situation - the IPsec transport packet and its decapsulated an decrypted payload are treated as two separate packets which came in via the same interface. As you do not want to queue the same actual packet twice, you can
  • either inspect the remainders of DSCP (if any) on the IPsec transport packets and choose a queue for them on that basis, but in such case you have to exclude from the queueing the decapsulated an decrypted payload
  • or exclude from queueing the IPsec transport packets, but in such case you have to inspect the protocol type and other attributes of the decapsulated and decrypted packets and choose the queue for them on that basis
I am afraid that "exclude from queueing" actually means to put them into a topmost queue with highest priority and no speed limits.

Classification of IPsec transport packets by DSCP in download saves CPU because the classification rules are simpler as the class has already been determined by the sending Mikrotik.
 
marwooj
newbie
Posts: 40
Joined: Mon Nov 06, 2017 10:44 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Mar 13, 2018 11:19 am

So to make LEVEL_B and LEVEL_C always lower priority than LEVEL_A I will change parent of B and C to A?
Yes but not only, you must also give the LEVEL_B queue the lowest priority among all the other child queues of LEVEL_A. And LEVEL_C must have the lowest priority among all child queues of LEVEL_B.
Should LEVEL_A as a parent of everything have
priority=1
?
Does priority is valid only among members of the same branch. So all children in sub branch will always step down to traffic if they parent priority is smaller than priority of an other branches on the same level. Does not mutter they have 1 but there is another purrent that have 7 ant they parrent has 8?

Regarding bandwidth, how to make C not to stal everything from B, and then B not to eat everything of A.
Do I need to change theirs max-limit to be smaller from each other?


and the example of proper "do not disturb my VoIP" should look like:
add name="LEVEL_A_UP"   parent=ether-WAN  queue=default max-limit=900k [b]priority=1
[/b]add name="LEVEL_A_DOWN" parent=bridge-LAN queue=default max-limit=4M [b]priority=1
[/b]add name="VOIP_U"       parent="LEVEL_A_UP"    packet-mark="VOIP"     queue=default priority=1
add name="VOIP_D"       parent="LEVEL_A_DOWN"  packet-mark="VOIP"     queue=default priority=1

add name="LEVEL_B_UP"   parent=LEVEL_A_UP  queue=default max-limit=[b]something smaler than parent A[/b] [b]priority=2
[/b]add name="LEVEL_B_DOWN" parent=LEVEL_A_DOWN queue=default max-limit=something smaler than parent A [b]priority=2
[/b]
add name="ACK_U"        parent="LEVEL_B_UP"    packet-mark="ACK"      queue=default priority=1
add name="ACK_D"        parent="LEVEL_B_DOWN"  packet-mark="ACK"      queue=default priority=1
add name="SOMETHING"        parent="LEVEL_B_UP"    packet-mark="SOMETHING"      queue=default [b]priority=7
[/b]add name="SOMETHING"        parent="LEVEL_B_DOWN"  packet-mark="SOMETHING"      queue=default [b]priority=7
[/b]
add name="LEVEL_C_UP"   parent=[b]LEVEL_B_UP[/b]  queue=default max-limit=[b]something smaler than parent B[/b] [b]priority=8
[/b]add name="LEVEL_C_DOWN" parent=[b]LEVEL_B_UP[/b] queue=default max-limit=[b]something smaler than parent B[/b] [b]priority=8
[/b]
add name="HTTP_U"       parent="LEVEL_C_UP"    packet-mark="HTTP"     queue=default [b]priority=1
[/b]add name="HTTP_D"       parent="LEVEL_C_DOWN"  packet-mark="HTTP"     queue=default [b]priority=1
[/b]
add name="OTHER_U"      parent="LEVEL_C_UP"    packet-mark="OTHER"    queue=default priority=8
add name="OTHER_D"      parent="LEVEL_C_DOWN"  packet-mark="OTHER"    queue=default priority=8
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Mar 13, 2018 12:01 pm

So to make LEVEL_B and LEVEL_C always lower priority than LEVEL_A I will change parent of B and C to A?
Yes but not only, you must also give the LEVEL_B queue the lowest priority among all the other child queues of LEVEL_A. And LEVEL_C must have the lowest priority among all child queues of LEVEL_B.
Should LEVEL_A as a parent of everything have
priority=1
?
Does priority is valid only among members of the same branch. So all children in sub branch will always step down to traffic if they parent priority is smaller than priority of an other branches on the same level. Does not mutter they have 1 but there is another purrent that have 7 ant they parrent has 8?

Regarding bandwidth, how to make C not to stal everything from B, and then B not to eat everything of A.
Do I need to change theirs max-limit to be smaller from each other?


and the example of proper "do not disturb my VoIP" should look like:
add name="LEVEL_A_UP"   parent=ether-WAN  queue=default max-limit=900k [b]priority=1
[/b]add name="LEVEL_A_DOWN" parent=bridge-LAN queue=default max-limit=4M [b]priority=1
[/b]add name="VOIP_U"       parent="LEVEL_A_UP"    packet-mark="VOIP"     queue=default priority=1
add name="VOIP_D"       parent="LEVEL_A_DOWN"  packet-mark="VOIP"     queue=default priority=1

add name="LEVEL_B_UP"   parent=LEVEL_A_UP  queue=default max-limit=[b]something smaler than parent A[/b] [b]priority=2
[/b]add name="LEVEL_B_DOWN" parent=LEVEL_A_DOWN queue=default max-limit=something smaler than parent A [b]priority=2
[/b]
add name="ACK_U"        parent="LEVEL_B_UP"    packet-mark="ACK"      queue=default priority=1
add name="ACK_D"        parent="LEVEL_B_DOWN"  packet-mark="ACK"      queue=default priority=1
add name="SOMETHING"        parent="LEVEL_B_UP"    packet-mark="SOMETHING"      queue=default [b]priority=7
[/b]add name="SOMETHING"        parent="LEVEL_B_DOWN"  packet-mark="SOMETHING"      queue=default [b]priority=7
[/b]
add name="LEVEL_C_UP"   parent=[b]LEVEL_B_UP[/b]  queue=default max-limit=[b]something smaler than parent B[/b] [b]priority=8
[/b]add name="LEVEL_C_DOWN" parent=[b]LEVEL_B_UP[/b] queue=default max-limit=[b]something smaler than parent B[/b] [b]priority=8
[/b]
add name="HTTP_U"       parent="LEVEL_C_UP"    packet-mark="HTTP"     queue=default [b]priority=1
[/b]add name="HTTP_D"       parent="LEVEL_C_DOWN"  packet-mark="HTTP"     queue=default [b]priority=1
[/b]
add name="OTHER_U"      parent="LEVEL_C_UP"    packet-mark="OTHER"    queue=default priority=8
add name="OTHER_D"      parent="LEVEL_C_DOWN"  packet-mark="OTHER"    queue=default priority=8
I don't like to provide theoretical-only answers and as I've stated several times, I didn't have a strong enough reason to test this practically so far. So already my previous response was wrong, as the manual says the following:
priority (1..8 ) : Prioritize one child queue over other child queue. Does not work on parent queues (if queue has at least one child). One is the highest, eight is the lowest priority. Child queue with higher priority will have chance to reach its max-limit before child with lower priority. Priority have nothing to do with bursts.

I have assumed that the priority works among all children of the same queue and recursively, but it seems not to be the case. So either you have to split the total available banwidth into subbands which do not affect each other, or you have to manage everything within the 8 priority levels of the common band. I just remind that in download direction there is basically no other way than throttling the traffic which can be throttled to leave a "fast lane" always free for the real-time traffic.
 
SamWCL
Frequent Visitor
Frequent Visitor
Posts: 75
Joined: Mon Apr 20, 2009 1:18 pm
Location: Nelson, NZ

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Mar 27, 2018 10:09 am

Did anyone figure out MPLS/VPLS QoS using DSCP/EXP bits?
 
mixig
Member
Member
Posts: 316
Joined: Thu Oct 27, 2011 2:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Nov 06, 2018 9:29 pm

I'm a little curious why you have some rules twice

ros code

/ip firewall mangle
add chain=forward action=mark-connection protocol=udp   src-address=192.168.100.5 connection-state=new new-connection-mark="VOIP" comment="IP-PBX"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
add chain=forward action=mark-connection protocol=udp   dst-address=192.168.100.5 connection-state=new new-connection-mark="VOIP"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
 
You are marking packets twice based on the same options. In my opinion rule 2 can be removed.
Then first there are two rules marking the connection and setting connection-marks. Then use the connection-mark to mark the packets in one single rule.

This two rules are not the same... src/dst address are in use :)
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Jan 08, 2019 10:31 am

Should this topic be moved to Useful user articles forum?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed Jan 09, 2019 12:18 am

Should this topic be moved to Useful user articles forum?

I would like to rewrite it, make it a little easier to follow.
 
pegasus123
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Tue Jul 24, 2018 7:02 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Jan 25, 2019 12:27 pm

I'm a little curious why you have some rules twice

ros code

/ip firewall mangle
add chain=forward action=mark-connection protocol=udp   src-address=192.168.100.5 connection-state=new new-connection-mark="VOIP" comment="IP-PBX"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
add chain=forward action=mark-connection protocol=udp   dst-address=192.168.100.5 connection-state=new new-connection-mark="VOIP"
add chain=forward action=mark-packet     passthrough=no connection-mark="VOIP"    new-packet-mark="VOIP"
 
You are marking packets twice based on the same options. In my opinion rule 2 can be removed.
Then first there are two rules marking the connection and setting connection-marks. Then use the connection-mark to mark the packets in one single rule.

This two rules are not the same... src/dst address are in use :)
isn't it enough to just mark it once? once a packet is read, it will be marked with connection mark, connection mark works both ways upload and download in which case you can mark the packet and be used in queue tree.

is my understanding correct?
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Jan 25, 2019 4:33 pm

is my understanding correct?
Your understanding is correct in terms that the rule translating connection-mark into packet-mark may be there only once (as the last one after the two assigning the connection-mark). Regarding the need for two rules assigning the connection-mark, it is a more complex question.

One approach would be to have only one such rule (for src- or dst-address) and say that if a single packet in the "wrong" direction doesn't get connection-marked, it causes no harm as the first subsequent packet in the opposite direction will fix this. The other approach, however, is to save CPU by having the connection-mark->packet-mark translation rule before the connection-marking rules so that these rules would only handle packets belonging to not yet marked connections. And this is usually combined with allowing these rules to handle only the initial packet of each connection (connection-state=new), so then you need to use both rules because you don't know in advance which RTP packet will be the first one in a given call.

And of course, the translation rules have to be doubled in the latter case - the first one handles packets belonging to already marked connections early in the chain, and the other one has to be there to handle the translation for the initial packets after the connection-mark has been just assigned.
 
pegasus123
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Tue Jul 24, 2018 7:02 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Jan 26, 2019 5:49 am

is my understanding correct?
Your understanding is correct in terms that the rule translating connection-mark into packet-mark may be there only once (as the last one after the two assigning the connection-mark). Regarding the need for two rules assigning the connection-mark, it is a more complex question.

One approach would be to have only one such rule (for src- or dst-address) and say that if a single packet in the "wrong" direction doesn't get connection-marked, it causes no harm as the first subsequent packet in the opposite direction will fix this. The other approach, however, is to save CPU by having the connection-mark->packet-mark translation rule before the connection-marking rules so that these rules would only handle packets belonging to not yet marked connections. And this is usually combined with allowing these rules to handle only the initial packet of each connection (connection-state=new), so then you need to use both rules because you don't know in advance which RTP packet will be the first one in a given call.

And of course, the translation rules have to be doubled in the latter case - the first one handles packets belonging to already marked connections early in the chain, and the other one has to be there to handle the translation for the initial packets after the connection-mark has been just assigned.
Thanks for the detailed explanation!
 
User avatar
denisun
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Wed Jul 16, 2014 6:38 pm
Location: Greece

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Feb 08, 2019 8:56 am

I am also writing here because I have not found a solution.
Has anyone managed to give priority to VoIP, and to work with no problem, with full load on the line?
My problem is this.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Feb 08, 2019 3:13 pm

I am also writing here because I have not found a solution. Has anyone managed to give priority to VoIP, and to work with no problem, with full load on the line? My problem is this.

Someday, I hope to do another write up on this subject, when I get time. I had hoped that one would not need to be an expert to get this correct, but as of 2019, it still does. However, I think a better article would help.

From the moment that audio transmits from your equipment, until it gets back, there must not be greater than a 150ms interruption (or thereabout) otherwise you will notice the delay. Now, think about everything that an audio packet has to go through to make that happen. When you understand that, you'll have solved your VoIP issue.

The permanent fix, a separate Internet service line. Can't do that? Then fast hardware performing QoS where VoIP packets get the best treatment when bad things happen (and they will).

:-)
 
User avatar
denisun
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Wed Jul 16, 2014 6:38 pm
Location: Greece

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Feb 08, 2019 4:13 pm

I am also writing here because I have not found a solution. Has anyone managed to give priority to VoIP, and to work with no problem, with full load on the line? My problem is this.

Someday, I hope to do another write up on this subject, when I get time. I had hoped that one would not need to be an expert to get this correct, but as of 2019, it still does. However, I think a better article would help.

From the moment that audio transmits from your equipment, until it gets back, there must not be greater than a 150ms interruption (or thereabout) otherwise you will notice the delay. Now, think about everything that an audio packet has to go through to make that happen. When you understand that, you'll have solved your VoIP issue.

The permanent fix, a separate Internet service line. Can't do that? Then fast hardware performing QoS where VoIP packets get the best treatment when bad things happen (and they will).

:-)
I have a CRS109-8G.
I can not believe that my provider's simple modem/router equipment work perfect with VoIP packets and full load
and the MT have problem.
My question is...
- Have i somewhere wrong in mangles, tree or queue types?
- Is MT hw low specifications and can't process the load?
- Is ROS bug?
With ubiquity the result is much perfect.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Feb 08, 2019 4:50 pm

I have a CRS109-8G. With ubiquity the result is much better.

I would not use the CRS109 for QoS tasks. Too under powered in my opinion. However, there are many variables.
 
User avatar
denisun
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Wed Jul 16, 2014 6:38 pm
Location: Greece

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Fri Feb 08, 2019 7:26 pm

I have a CRS109-8G. With ubiquity the result is much better.

I would not use the CRS109 for QoS tasks. Too under powered in my opinion. However, there are many variables.
I don't believe that the problem is the hw specifications of CRC.
My ISPs equipment cost ~20-50e and work perfect with traffic (VoIP and dw).
 
bolean
just joined
Posts: 7
Joined: Fri Apr 13, 2018 2:45 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Sat Feb 23, 2019 3:08 pm

version 3, running smooth

Image
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Sat Feb 23, 2019 6:43 pm

Version 3, running smooth

@bolean,

How does this test compared to what you're doing?

Image

/queue tree

# DOWN
add name=DOWN max-limit=90M parent=BR_LAN queue=default

add name="1. VOIP"     packet-mark=VOIP     parent=DOWN priority=1 queue=default
add name="2. DNS"      packet-mark=DNS      parent=DOWN priority=2 queue=default
add name="3. ACK"      packet-mark=ACK      parent=DOWN priority=3 queue=default
add name="4. UDP"      packet-mark=UDP      parent=DOWN priority=3 queue=default
add name="5. ICMP"     packet-mark=ICMP     parent=DOWN priority=4 queue=default
add name="6. HTTP"     packet-mark=HTTP     parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC"     packet-mark=QUIC     parent=DOWN priority=7 queue=default
add name="9. OTHER"    packet-mark=OTHER    parent=DOWN priority=8 queue=default


# UP
add name=UP max-limit=90M parent=ether1 queue=default

add name="1. VOIP_"     packet-mark=VOIP     parent=UP priority=1 queue=default
add name="2. DNS_"      packet-mark=DNS      parent=UP priority=2 queue=default
add name="3. ACK_"      packet-mark=ACK      parent=UP priority=3 queue=default
add name="4. UDP_"      packet-mark=UDP      parent=UP priority=3 queue=default
add name="5. ICMP_"     packet-mark=ICMP     parent=UP priority=4 queue=default
add name="6. HTTP_"     packet-mark=HTTP     parent=UP priority=5 queue=default
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="8. QUIC_"     packet-mark=QUIC     parent=UP priority=7 queue=default
add name="9. OTHER_"    packet-mark=OTHER    parent=UP priority=8 queue=default
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Tue Dec 31, 2019 2:28 am

I have a question to ask the admins (how do we private message you and talk about the forums)? I am planning on rewriting this article. What is the best course of action to maintain the link (which is pinned and also maybe linked elsewhere)? I would like for all posts to be deleted except for the first six ones (I'm going to go back and edit my own). Is there a better recommendation?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 21818
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jan 05, 2020 9:24 pm

I would hope the admins give you another thread to start that only you can edit.
Folks could use your original thread to comment and keep the conversation going leaving your new thread as the more polished pre-wiki draft to tweak.
 
User avatar
digin4
just joined
Posts: 13
Joined: Tue Jan 22, 2019 8:18 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 07, 2020 11:27 pm

How to add some TCP ports and prioritize them as VOIP? can I simply do this and it will be enough?:
/ip firewall mangle
add chain=prerouting  action=mark-connection new-connection-mark=VOIP port=4244,5222,5223,5228,5242,50318,59234,1119,3724,6113 protocol=tcp passthrough=yes comment="VOIP"
Thank you!
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jan 09, 2020 1:26 am

How to add some TCP ports and prioritize them as VOIP? can I simply do this and it will be enough?

Please study the example more closely. VoIP packets on your network will not be the same as others. Using Mangle, you will choose what you need to mark. It could be via IP Address, standard SIP or RTP ports, or other metrics unique to your environment.
 
gcser
newbie
Posts: 29
Joined: Sun May 14, 2006 1:03 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jan 09, 2020 2:48 pm

Hi Everybody,

I have the following scenario (only for testing the limiter, this is not a production environment):
PC->CCR eth3 (vlan id: 2205 untagged)
CCR eth2->RB600eth3 (2205, 2206, 2207 vlans configured, all traffic are tagged)
RB600 eth2->Server (vlan id: 2205 untagged).

On th CCR I have a bridge containing 2 ports, with VLAN filtering enabled. I have 3 VLANs on it (2205, 2206, 2207), all traffic are tagged on eth2 interface, on the eth3 interface the 2205 vlan is untagged, pvid set to 2205. I connected my test pc to eth3.
On the RB600 I the the same situation (I would like to simulate a trunk between the 2 MTs). I connected a server pc to the RB600.
I can ping the server, I can download files from it, the network works as expected.

I would like to make a traffice shaping/priorization based on VLAN-s.
On the CCR, at the bridge configuration I checked the "use IP firewall" and the "user IP firewall for VLAN" checkboxes.
I created a bridge filter rule: chain: forward, MAC-protocol-num=8100 (vlan). On Advanced tab I filled the VLAN ID, on the action tab: mark packet with mark "office-traffic"
After that, I created a queue (in queue tree), set the packet mark to "office-traffic", and made a bandwith limit to 10mbit/sec.

My problem is, that if I make a download from the server, the bandwidth limit is not applied. If I remove the vlan ID from the rule, and set the MAC-protocol back to "ip", the limiter works as expected.
If I set the VLAN tag, in the queue I can see only a ~700kbit/sec traffic. If I remove the vlan filter from the rule, the queue shows 10mbit/sec traffic.

What doing I wrong? It seems to me, that the MAC-protocol =vlan not capturing all the packets... Is it possible?
 
Tabalugach
just joined
Posts: 2
Joined: Sat Dec 08, 2018 10:58 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jan 19, 2020 7:39 pm

Hi Everybody,

I setup queue tree like this :
/queue tree

# DOWN
add name=DOWN max-limit=9M parent=LAN queue=default

add name="1. Gaming"     packet-mark=Gaming parent=DOWN priority=1 queue=default
add name="2. DNS"      packet-mark=DNS      parent=DOWN priority=2 queue=default
add name="3. ACK"      packet-mark=ACK      parent=DOWN priority=3 queue=default
add name="4. UDP"      packet-mark=UDP      parent=DOWN priority=3 queue=default
add name="5. ICMP"     packet-mark=ICMP     parent=DOWN priority=4 queue=default
add name="6. HTTP"     packet-mark=HTTP     parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC"     packet-mark=QUIC     parent=DOWN priority=7 queue=default
add name="9. OTHER"    packet-mark=OTHER    parent=DOWN priority=8 queue=default


# UP
add name=UP max-limit=540k parent=WAN queue=default

add name="1. Gaming_"     packet-mark=Gaming parent=UP priority=1 queue=default
add name="2. DNS_"      packet-mark=DNS      parent=UP priority=2 queue=default
add name="3. ACK_"      packet-mark=ACK      parent=UP priority=3 queue=default
add name="4. UDP_"      packet-mark=UDP      parent=UP priority=3 queue=default
add name="5. ICMP_"     packet-mark=ICMP     parent=UP priority=4 queue=default
add name="6. HTTP_"     packet-mark=HTTP     parent=UP priority=5 queue=default
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="8. QUIC_"     packet-mark=QUIC     parent=UP priority=7 queue=default
add name="9. OTHER_"    packet-mark=OTHER    parent=UP priority=8 queue=default
I only mark two things, one is Apex Legends and 2nd is everything else.
Apex Legends is correctly marked and it's go to gaming queue, everything else go to OTHER queue.

But still when i hit speed test (https://www.speedtest.net/), when upload start, apex will lost connection.

How is this possible that priority=8 overcomes priority=1?

Another thing which I don't understand is if I setup :

#DOWN
add name=DOWN max-limit=1M parent=LAN queue=default
# UP
add name=UP max-limit=100k parent=WAN queue=default

I will still get :

Download Mbps 9.68
Upload Mbps 0.56

How is this possible?

Please help!
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1742
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Jan 20, 2020 6:48 pm

You need to use 'limit-at' to guarantee bandwidth instead of 'max-limit' try that and see if you get different results.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Jan 20, 2020 10:00 pm

#DOWN
add name=DOWN max-limit=1M parent=LAN queue=default
# UP
add name=UP max-limit=100k parent=WAN queue=default

I will still get : Download Mbps 9.68, Upload Mbps 0.56

How is this possible?

You can not use Fast Track and Queues Tree together.
 
FRANKNunoz
just joined
Posts: 10
Joined: Mon Jan 20, 2020 9:38 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Mon Jan 20, 2020 10:39 pm

version 3, running smooth

Image
friend how is your connection dial and packet marking in mangrove?
 
Tabalugach
just joined
Posts: 2
Joined: Sat Dec 08, 2018 10:58 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 21, 2020 2:11 am

You can not use Fast Track and Queues Tree together.
I found it today! Thank you!
Btw, do you prefer to put highest priority 1 (in my situation game : Apex) to fast track?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 21, 2020 4:57 am

do you prefer to put highest priority 1 (in my situation game : Apex) to fast track?

Fast Track is CPU usage mitigation technique. Queuing is a bandwidth utilization technique. Different goals. If the CPU can handle it, you need to use Queue technique only.
 
theprism
newbie
Posts: 29
Joined: Sun Sep 16, 2012 4:11 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Jan 24, 2020 2:04 pm

Hello everyone. I've tested and updated the script. It now works correctly on RouterOS 6.1. Note that ether1 is WAN and ether2 is LAN. Adjust those as necessary for your environment.

I would appreciate if someone could tell me how to mark big downloads over HTTP traffic. Currently, the script marks port 80 so everything HTTP gets too much priority. The idea situation would be to let short bursts of HTTP traffic get high priority and the big long downloads get less.
with connection byte ?

OK for WAN, but what about LAN in cases when more than 1 interface is used?
For example, I use ether2+wlan1+wlan2 in homebridge, but I also have ether3+wlan1-guest in guestbridge. How I identify them both together as LAN in the QoS rules and don't let one of them to take the whole bandwidth if both are traffic-hungry?

Thanks,
T.P.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Jan 25, 2020 10:00 pm

what about when there are several LAN interfaces?

There are several ways, in RouterOS, to combine several things into one. Maybe VLAN, maybe interface lists, maybe address lists. Its up to you. Then you simply mangle them and send them to the queue.
 
louisren
just joined
Posts: 3
Joined: Mon Sep 10, 2018 1:53 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jan 30, 2020 4:41 am

If anyone can add a tutorial on how to conduct QoS on CRS328-24P-4S+RM that would be great...........
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 3115
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jan 30, 2020 6:49 am

If anyone can add a tutorial on how to conduct QoS on CRS328-24P-4S+RM that would be great...........

maybe when MikroTik enable DSCP and TOS remarking

maybe when MikroTik enable 4 or 8 hardware queues per port

all of this in hardware switch ASIC to do it at wire-speed without almost CPU usage

then will be nice to do QoS on MikroTik switch's, but today is not
 
helper
just joined
Posts: 5
Joined: Thu Jan 02, 2020 12:10 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Feb 04, 2020 1:45 pm

Thanks alot .....Trying
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Feb 04, 2020 2:00 pm

Shouldn't the topic be moved to viewforum.php?f=23?
 
bharrisau
just joined
Posts: 6
Joined: Wed Feb 05, 2020 8:35 am

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Feb 05, 2020 9:02 am

Did a pile of testing today with Flent measuring bufferbloat and performance. The final system that gave the best results (no increase in latency when limited to 90% of line bandwidth) is below.

  • Set the two parent queues (UP and DOWN) to have a bucket-size of 0.005. This means that up to 5ms of token can accumulate.
  • Create a bulkUp queue of kind PCQ, set the pcq-limit to 11*[upload rate in Mbps] (100ms of upload bandwidth) and the pcq-total-limit to 10 times that. Select all 4 classifier options.
  • Create a bulkDown queue of kind sqf. (Using RED gave similar performance, but multiple downloads were less fair)
  • Change the default queue to sfq. (This will only effect something if the interactive traffic is saturating the link, which is a bad state to get into)
  • Set the HTTP/HTTP_BIG/QUIC/OTHER queues to be bulkUp in the UP direction, and bulkDown in the DOWN direction.

Rationale for the 0.005 size is to copy CoDel as much as possible. We set bulkUp to 100ms of data to smooth out throughput, 100ms of latency for bulk traffic isn't expected to be noticeable in practice. In the down direction we have a much smaller buffering capacity to ensure we are dropping early - the down direction isn't controlled as well so early drops are the only way to avoid bloat at the ISP end. RED and SFQ work well here - I had more fairness during testing using SFQ.

An example of 100ms of data setting for my 40 Mbps upload is pcq-limit = 440. Because pcq-limit is a value in KiB, multiplying the Mbps by 11 gives us approximately 100ms of data transfer at 90% of the line speed.

Attached image is my final Flent test. I'm testing from Australia to Europe so the baseline latency is pretty high, upload and download limits in router are 90% of the line speeds (90/36). At the start of this test I also did a simultaneous speedtest.com test to see how fair things are.
You do not have the required permissions to view the files attached to this post.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Feb 05, 2020 3:12 pm

Shouldn't the topic be moved to viewforum.php?f=23?

I'm okay with that. Please keep the original url and redirect it to whatever the new one will be. Also, remove many of the old posts that don't advance the topic, since the new 2020 edition for example.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Feb 05, 2020 3:16 pm

  • Set the two parent queues (UP and DOWN) to have a bucket-size of 0.005.
  • Create a bulkUp queue of kind PCQ, set the pcq-limit to 11*[upload rate in Mbps] (100ms of upload bandwidth) and the pcq-total-limit to 10 times that. Select all 4 classifier options.
  • Create a bulkDown queue of kind sqf.
  • Change the default queue to sfq.
  • Set the HTTP/HTTP_BIG/QUIC/OTHER queues to be bulkUp in the UP direction, and bulkDown in the DOWN direction.

Wow, thank you for doing this. To remove any ambiguity, would you please post your queue config (between code tags) so I can study it? How well did my reference implementation do for you? I want to make sure that your config is not too specific to your needs as we do need something general that people can start and learn from.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Feb 06, 2020 10:14 pm

I'm okay with that. Please keep the original url and redirect it to whatever the new one will be.
Moved. Topic ID is the same, so all links are still valid.
 
User avatar
hknet
Member Candidate
Member Candidate
Posts: 128
Joined: Sun Jul 17, 2016 6:05 pm
Location: Vienna, Austria
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Feb 07, 2020 3:52 am

as this is trending and the initial target if I understand it correctly was VoIP priority, here is a small example on how we do it.

the assumption is like this: download-link can be oversaturated, though this is not desireable we usually can't do much about it if incoming packets are thrown at the CPE, therefore the provider-end is assumed to also hold it's end of the bargain and do their VoIP prio right. in our situation we are the provider end ourselves and yes our BRAS do this quite well.

so the real job of the mikrotik at the customer-site is handle the upload well.
also we assume VoIP packets are well marked using DSCP-markings (usually all VoIP equipment does this quite well).

therefore we first build a DSCP-marking to packet-marking within MT setup and we slide in a marking for remote-winbox-connects also:
/ip firewall mangle add action=mark-packet chain=postrouting comment="mgmt prio winbox" \
dst-address=<dudeserver-mgmt-IP> new-packet-mark=dscp.48 passthrough=no protocol=tcp src-port=8291
/ip firewall mangle add action=mark-packet chain=postrouting comment=dscp.0_no-marking disabled=no \
dscp=0 new-packet-mark=dscp.0 passthrough=no
/ip firewall mangle add action=mark-packet chain=postrouting comment=dscp.46_VoIP-marking disabled=no \
dscp=46 new-packet-mark=dscp.46 passthrough=no
/ip firewall mangle add action=mark-packet chain=postrouting comment=dscp.48_network-mark disabled=no \
dscp=48 new-packet-mark=dscp.48 passthrough=no
:for x from 1 to 45 do={/ip firewall mangle add action=mark-packet chain=postrouting \
comment=dscp.1-45 disabled=no dscp=$x new-packet-mark=dscp.other passthrough=no}
/ip firewall mangle add action=mark-packet chain=postrouting comment=dscp.47 disabled=no \
dscp=47 new-packet-mark=dscp.other passthrough=no
:for x from 49 to 63 do={/ip firewall mangle add action=mark-packet chain=postrouting \
comment=dscp.49-63 disabled=no dscp=$x new-packet-mark=dscp.other passthrough=no}
and as you sure do provide IPv6 connectivity to your customers we do this very same thing for IPv6 packets:
/ipv6 firewall mangle add action=mark-packet chain=postrouting comment="mgmt prio winbox" \
dst-address=<dudeserver-mgmt-IP> new-packet-mark=dscp.48 passthrough=no protocol=tcp src-port=8291
/ipv6 firewall mangle add action=mark-packet chain=postrouting comment=dscp.0_no-marking disabled=no \
dscp=0 new-packet-mark=dscp.0 passthrough=no
/ipv6 firewall mangle add action=mark-packet chain=postrouting comment=dscp.46_VoIP-marking disabled=no \
dscp=46 new-packet-mark=dscp.46 passthrough=no
/ipv6 firewall mangle add action=mark-packet chain=postrouting comment=dscp.48_network-mark disabled=no \
dscp=48 new-packet-mark=dscp.48 passthrough=no
:for x from 1 to 45 do={/ipv6 firewall mangle add action=mark-packet chain=postrouting \
comment=dscp.1-45 disabled=no dscp=$x new-packet-mark=dscp.other passthrough=no}
/ipv6 firewall mangle add action=mark-packet chain=postrouting comment=dscp.47 disabled=no \
dscp=47 new-packet-mark=dscp.other passthrough=no
:for x from 49 to 63 do={/ipv6 firewall mangle add action=mark-packet chain=postrouting \
comment=dscp.49-63 disabled=no dscp=$x new-packet-mark=dscp.other passthrough=no}
now we have packets marked for further use in our queues.
the basic queue max-limit depends on the upload speed, usually we automate this and take for example 90% of the theoretically available bandwidth (as we have IP/PPP/.. overhead to accomodate):
/queue tree
add max-limit=666k name=line01 parent=ppp-out-line1 queue=default
then we add priority based on the previously created dscp-markings:
add name="line01_dscp-48-prio=2" packet-mark=dscp.48 parent=line01 priority=2
add name="line01_dscp-46-prio=3" packet-mark=dscp.46 parent=line01 priority=3
add name="line01_dscp-other-prio=7" packet-mark=dscp.other parent=line01 priority=7
add name="line01_dscp-none_besteffort" packet-mark=dscp.0 parent=line01
if you have a non-overhead-connection (eg. ethernet-connection) to your provider, you could do this (assuming the uplink is on ether2):
/queue tree
add max-limit=1000M name=ether2-queue parent=ether2 queue=default
add name="ether2_dscp-48-prio=2" packet-mark=dscp.48 parent=ether2-queue priority=2
add name="ether2_dscp-46-prio=3" packet-mark=dscp.46 parent=ether2-queue priority=3
add name="ether2_dscp-other-prio=7" packet-mark=dscp.other parent=ether2-queue priority=7
add name="ether2_dscp-none_besteffort" packet-mark=dscp.0 parent=ether2-queue
now in case you're wondering how the provider-end holds up to their promise - we do this on ciscos like this (simplified but also working if you want to try it):
class-map match-any MEDIA
 match dscp ef
class-map match-all VOICE
  description express forward = VoIP
 match ip dscp ef
class-map match-any SIGNALING
  description VoIP Signalling
 match ip dscp cs3
 match dscp cs3
 match dscp af31
now we have kind of what is a marking in mikrotiks for packets matching those classes in dscp.
we then need a policy what to do:
policy-map VOIP1M
 description VoIP Prio for 1Mbit
 class SIGNALING
  bandwidth 100
 class MEDIA
  bandwidth 1000
 class class-default
  fair-queue
we simpy define the bandwidth we want to be guaranteed downstream to the CPE and then
we also need to create a policy for the interface towards the user limiting the bandwidth to their expected bandwidth:
policy-map 10MVOIP
 class class-default
  shape average 10000000
   service-policy VOIP1M
this shapes the connection to 10Mbps and hands over the calculation to the before defined 1Mbps VoIP prio.
in your radius-server there then you add a pair like this to the user-account: cisco-avpair += ip:sub-qos-policy-out=10MVOIP

the result on the provider-end looks like this (here for a 30Mbps access-service):
sh policy-map interface Vi2.17

 Virtual-Access2.17

 SSS session identifier 447 -

  Service-policy output: 30MVOIP

    Class-map: class-default (match-any)
      7255033 packets, 949096170 bytes
      30 second offered rate 0000 bps, drop rate 0000 bps
      Match: any
      Queueing
      queue limit 64 packets
      (queue depth/total drops/no-buffer drops) 0/1401/0
      (pkts output/bytes output) 5906077/903171025
      shape (average) cir 30000000, bc 120000, be 120000
      target shape rate 30000000

      Service-policy : VOIP1M

        Class-map: SIGNALING (match-any)
          614466 packets, 396421487 bytes
          30 second offered rate 0000 bps, drop rate 0000 bps
          Match: ip dscp cs3 (24)
          Match:  dscp cs3 (24)
          Match:  dscp af31 (26)
          Queueing
          queue limit 64 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 614468/396422741
          bandwidth 100 kbps

        Class-map: MEDIA (match-any)
          453497 packets, 102353156 bytes
          30 second offered rate 0000 bps, drop rate 0000 bps
          Match:  dscp ef (46)
          Queueing
          queue limit 64 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 453497/102353156
          bandwidth 1000 kbps

        Class-map: class-default (match-any)
          6187064 packets, 450322568 bytes
          30 second offered rate 0000 bps, drop rate 0000 bps
          Match: any
          Queueing
          queue limit 64 packets
          (queue depth/total drops/no-buffer drops/flowdrops) 0/1401/0/1401
          (pkts output/bytes output) 4838112/404395128
          Fair-queue: per-flow queue limit 16 packets
the drops if bandwidth is exceeded happen in the default-class (as expected).

at first DSCP might seem uncontrollable (and for some users who choose their own dscp-markings at their devices it is), but if your users also want to have the QoS for their VoIP calls and they follow the guidelines to simply use DSCP and leave the rest alone this usually works out very well and everybody gets what they want - good service.

maybe this helps someone.
 
bharrisau
just joined
Posts: 6
Joined: Wed Feb 05, 2020 8:35 am

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Feb 07, 2020 7:29 am

Wow, thank you for doing this. To remove any ambiguity, would you please post your queue config (between code tags) so I can study it? How well did my reference implementation do for you? I want to make sure that your config is not too specific to your needs as we do need something general that people can start and learn from.
Need text here for the code block to work.
/queue type
set 0 kind=sfq
add kind=pcq name=bulkUp pcq-classifier=src-address,dst-address,src-port,dst-port pcq-limit=450KiB pcq-total-limit=4500KiB
add kind=sfq name=bulkDown

/queue tree
add bucket-size=0.005 max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=bulkDown
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=bulkDown
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=bulkDown
add name="9. OTHER" packet-mark=OTHER parent=DOWN priority=8 queue=bulkDown
add bucket-size=0.005 max-limit=36M name=UP parent=WAN1 queue=default
add name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=bulkUp
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=bulkUp
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=bulkUp
add name="9. OTHER_" packet-mark=OTHER parent=UP priority=8 queue=bulkUp

I'll re-run the tests next week with a baseline and the reference implementation to compare. Just trying to get a test server set up closer so there is less natural latency variability.

I'm also keen to test out the "Layer Cake" style system where instead of 9 categories you just have 3 - 'priority', 'sparse' and 'bulk'. VoIP becomes priority (maybe DNS and ACK too), and the rest of the connections are classified as either sparse or bulk depending on their connection-rate/packet-size/not-sure-yet. Cake is making them 'bulk' when the queue for that connection is not emptying, they are 'sparse' when packet/bit rate is low enough to not queue any packets (at least that is what I'm seeing from a quick scan through the code). I don't have access to the PCQ data at the mangle stage, so I need to experiment with a different heuristics - similar to your HTTP vs HTTP_BIG heuristic. Thinking something along the lines of this - with tweaking based on experiments next week. Mainly using connection-rate, and including some dst-limit to work around the fact that rate is calculated every second.

/ip firewall mangle

add action=mark-connection chain=prerouting comment="SPARSE by default" connection-mark=no-mark new-connection-mark=SPARSE passthrough=yes
add action=mark-connection chain=postrouting connection-mark=no-mark new-connection-mark=SPARSE passthrough=yes
add action=mark-connection chain=prerouting comment="SPARSE low rate" new-connection-mark=SPARSE connection-rate=0-1000 passthrough=yes
add action=mark-connection chain=postrouting new-connection-mark=SPARSE connection-rate=0-1000 passthrough=yes

add action=mark-connection chain=prerouting comment="SPARSE first packets when idle or new" new-connection-mark=SPARSE_FORCE connection-rate=0-1 dst-limit=1,1,addresses-and-dst-port/1m40s passthrough=yes
add action=mark-connection chain=postrouting new-connection-mark=SPARSE_FORCE connection-rate=0-1 dst-limit=1,1,addresses-and-dst-port/1m40s passthrough=yes
add action=mark-connection chain=prerouting comment="BULK large packets in idle or new" connection-mark=SPARSE new-connection-mark=BULK packet-size=!0-500 connection-rate=0-1 passthrough=yes
add action=mark-connection chain=postrouting connection-mark=SPARSE new-connection-mark=BULK packet-size=!0-500 connection-rate=0-1 passthrough=yes
add action=mark-connection chain=prerouting comment="BULK high rate" new-connection-mark=BULK connection-rate=!0-10000 passthrough=yes
add action=mark-connection chain=postrouting new-connection-mark=BULK connection-rate=!0-10000 passthrough=yes

add action=mark-connection chain=prerouting connection-mark=SPARSE_FORCE new-connection-mark=SPARSE passthrough=yes
add action=mark-connection chain=postrouting connection-mark=SPARSE_FORCE new-connection-mark=SPARSE passthrough=yes
add action=mark-packet chain=prerouting connection-mark=BULK new-packet-mark=BULK passthrough=no
add action=mark-packet chain=postrouting connection-mark=BULK new-packet-mark=BULK passthrough=no
add action=mark-packet chain=prerouting connection-mark=SPARSE new-packet-mark=SPARSE passthrough=no
add action=mark-packet chain=postrouting connection-mark=SPARSE new-packet-mark=SPARSE passthrough=no
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 3115
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Feb 09, 2020 3:37 am

nice job guys, Thanks a lot 8)
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1742
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Feb 09, 2020 4:38 pm

Agreed, this is great work. :)
 
Gombeen666
Member Candidate
Member Candidate
Posts: 224
Joined: Tue Jun 25, 2019 5:33 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Feb 09, 2020 5:57 pm

I get error "input does not match any value of parent"

Anyone got a complete file to test please?
 
bharrisau
just joined
Posts: 6
Joined: Wed Feb 05, 2020 8:35 am

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Feb 10, 2020 7:55 am

I've done some testing with a closer server. Four cases considered: baseline, QOS2020, QOS2020-Optimised (my recommended changes), mikroCAKE (pseudo-CAKE implementation).

Graphs of the bandwidth vs latency, and a latency distribution plot. All tests were Perth to Sydney, 30 seconds of baseline either side of a 120 second test. Local connection is 100/40 Mbps fibre, remote connection is 1000/250 Mbps OVH dedicated server. I've used 90% of the rated line speed locally for the settings.

Configuration exports below

Baseline
/ip firewall mangle
/queue type
/queue tree
add max-limit=36M name=UP packet-mark=no-mark parent=WAN1 queue=default
add max-limit=90M name=DOWN packet-mark=no-mark parent=BR1 queue=default
QOS2020
/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP new-connection-mark=VOIP passthrough=yes port=5060-5062,10000-20000 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no
add action=mark-connection chain=prerouting comment=QUIC connection-state=new new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new new-connection-mark=HTTP passthrough=\
    yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=\
    HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=new new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no

/queue type
add kind=red name=customRED red-avg-packet=1514

/queue tree
add max-limit=36M name=UP parent=WAN1 queue=default
add max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=customRED
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=customRED
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=customRED
add name="9. OTHER" packet-mark=OTHER parent=DOWN queue=default
add name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=customRED
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=customRED
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=customRED
add name="9. OTHER_" packet-mark=OTHER parent=UP queue=default
QOS2020 - Optimised
# Mangle is as above
/queue type
set 0 kind=sfq
add kind=pcq name=bulkUp pcq-classifier=src-address,dst-address,src-port,dst-port pcq-limit=450KiB pcq-total-limit=4500KiB

/queue tree
add bucket-size=0.005 max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=default
add name="9. OTHER" packet-mark=OTHER parent=DOWN priority=8 queue=default
add bucket-size=0.005 max-limit=36M name=UP parent=WAN1 queue=default
add name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=bulkUp
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=bulkUp
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=bulkUp
add name="9. OTHER_" packet-mark=OTHER parent=UP priority=8 queue=bulkUp
mikroCAKE
/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=PRIO passthrough=yes port=53 protocol=udp
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=PRIO passthrough=yes port=53 protocol=udp
add action=mark-connection chain=prerouting comment=VOIP new-connection-mark=PRIO passthrough=yes port=5060-5062,10000-20000 protocol=udp
add action=mark-packet chain=postrouting connection-mark=PRIO new-packet-mark=PRIO passthrough=no
add action=mark-packet chain=prerouting connection-mark=PRIO new-packet-mark=PRIO passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=PRIO packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=PRIO packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment=Sparse connection-rate=0-1 dst-limit=5,2,addresses-and-dst-port/1m40s new-packet-mark=\
    SPARSE passthrough=no
add action=mark-packet chain=postrouting connection-rate=1-10k new-packet-mark=SPARSE passthrough=no
add action=mark-packet chain=postrouting comment=Bulk new-packet-mark=BULK passthrough=no

/queue type
set 0 kind=sfq
add kind=pcq name=bulkUp pcq-classifier=src-address,dst-address,src-port,dst-port pcq-limit=450KiB pcq-total-limit=4500KiB

/queue tree
add bucket-size=0.005 max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. Priority" packet-mark=PRIO parent=DOWN priority=1 queue=default
add name="2. Sparse" packet-mark=SPARSE parent=DOWN priority=2 queue=default
add name="3. Bulk" packet-mark=BULK parent=DOWN priority=3 queue=default
add bucket-size=0.005 max-limit=36M name=UP parent=WAN1 queue=default
add name="1. Priority_" packet-mark=PRIO parent=UP priority=1 queue=default
add name="2. Sparse_" packet-mark=SPARSE parent=UP priority=2 queue=default
add name="3. Bulk_" packet-mark=BULK parent=UP priority=3 queue=bulkUp
You do not have the required permissions to view the files attached to this post.
 
erhtun
newbie
Posts: 26
Joined: Thu May 31, 2018 12:34 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Feb 25, 2020 5:28 pm

I've done some testing with a closer server. Four cases considered: baseline, QOS2020, QOS2020-Optimised (my recommended changes), mikroCAKE (pseudo-CAKE implementation).

Graphs of the bandwidth vs latency, and a latency distribution plot. All tests were Perth to Sydney, 30 seconds of baseline either side of a 120 second test. Local connection is 100/40 Mbps fibre, remote connection is 1000/250 Mbps OVH dedicated server. I've used 90% of the rated line speed locally for the settings.

Configuration exports below

Baseline
/ip firewall mangle
/queue type
/queue tree
add max-limit=36M name=UP packet-mark=no-mark parent=WAN1 queue=default
add max-limit=90M name=DOWN packet-mark=no-mark parent=BR1 queue=default
QOS2020
/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP new-connection-mark=VOIP passthrough=yes port=5060-5062,10000-20000 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no
add action=mark-connection chain=prerouting comment=QUIC connection-state=new new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new new-connection-mark=HTTP passthrough=\
    yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=\
    HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=new new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no

/queue type
add kind=red name=customRED red-avg-packet=1514

/queue tree
add max-limit=36M name=UP parent=WAN1 queue=default
add max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=customRED
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=customRED
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=customRED
add name="9. OTHER" packet-mark=OTHER parent=DOWN queue=default
add name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=customRED
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=customRED
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=customRED
add name="9. OTHER_" packet-mark=OTHER parent=UP queue=default
QOS2020 - Optimised
# Mangle is as above
/queue type
set 0 kind=sfq
add kind=pcq name=bulkUp pcq-classifier=src-address,dst-address,src-port,dst-port pcq-limit=450KiB pcq-total-limit=4500KiB

/queue tree
add bucket-size=0.005 max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=default
add name="9. OTHER" packet-mark=OTHER parent=DOWN priority=8 queue=default
add bucket-size=0.005 max-limit=36M name=UP parent=WAN1 queue=default
add name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=bulkUp
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=bulkUp
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=bulkUp
add name="9. OTHER_" packet-mark=OTHER parent=UP priority=8 queue=bulkUp
mikroCAKE
/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=PRIO passthrough=yes port=53 protocol=udp
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=PRIO passthrough=yes port=53 protocol=udp
add action=mark-connection chain=prerouting comment=VOIP new-connection-mark=PRIO passthrough=yes port=5060-5062,10000-20000 protocol=udp
add action=mark-packet chain=postrouting connection-mark=PRIO new-packet-mark=PRIO passthrough=no
add action=mark-packet chain=prerouting connection-mark=PRIO new-packet-mark=PRIO passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=PRIO packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=PRIO packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment=Sparse connection-rate=0-1 dst-limit=5,2,addresses-and-dst-port/1m40s new-packet-mark=\
    SPARSE passthrough=no
add action=mark-packet chain=postrouting connection-rate=1-10k new-packet-mark=SPARSE passthrough=no
add action=mark-packet chain=postrouting comment=Bulk new-packet-mark=BULK passthrough=no

/queue type
set 0 kind=sfq
add kind=pcq name=bulkUp pcq-classifier=src-address,dst-address,src-port,dst-port pcq-limit=450KiB pcq-total-limit=4500KiB

/queue tree
add bucket-size=0.005 max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. Priority" packet-mark=PRIO parent=DOWN priority=1 queue=default
add name="2. Sparse" packet-mark=SPARSE parent=DOWN priority=2 queue=default
add name="3. Bulk" packet-mark=BULK parent=DOWN priority=3 queue=default
add bucket-size=0.005 max-limit=36M name=UP parent=WAN1 queue=default
add name="1. Priority_" packet-mark=PRIO parent=UP priority=1 queue=default
add name="2. Sparse_" packet-mark=SPARSE parent=UP priority=2 queue=default
add name="3. Bulk_" packet-mark=BULK parent=UP priority=3 queue=bulkUp
Hi, I am trying your configuration. Until now there are too much drops ACK upload packets than HTTP or HTTP_BIG packets. Download ACK packets are not dropped. Is this normal?

Also I want to ask a question aboud bulkUp pcq queue size and total queue size. I have set upload max-limit to 4M. What I should set queue size and total queue size in bulkUp queue type for my connection?
 
zlsd
just joined
Posts: 1
Joined: Sat Feb 29, 2020 11:42 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sat Feb 29, 2020 11:56 am

Why don't you use "PCQ" instead of the default queue?
Could you create a rule that would reduce the priority for a large number of connections for applications like windows update?
Why is "SFQ" better for download than "PCQ"?
Last edited by zlsd on Sun Mar 01, 2020 12:58 pm, edited 1 time in total.
 
KingKeA
just joined
Posts: 1
Joined: Sun Mar 15, 2020 2:23 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Mar 15, 2020 4:37 pm

Thank you so much for this!

I have 940mbps/35mbps cable using a RB4011 router. When applying the optimized QOS, I see better results at dslreport's bufferbloat test, but my overall throughput is much lower, around 500mbps. Is there a way to have QOS and still have close to line speeds with this model?

Thank you again!
 
purplecloud
just joined
Posts: 9
Joined: Tue Jan 30, 2018 3:29 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Mar 29, 2020 1:04 pm

Thank you so much for this!

I have 940mbps/35mbps cable using a RB4011 router. When applying the optimized QOS, I see better results at dslreport's bufferbloat test, but my overall throughput is much lower, around 500mbps. Is there a way to have QOS and still have close to line speeds with this model?

Thank you again!
I second this question. I have similar problems. Would it be similar with QOS?
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3341
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Apr 09, 2020 4:10 pm

How to Mark the Traffic Types:
Nice thread. :mrgreen:
I have not read all in detail yet, but there are on thing I do miss or not understand.
Do you need to mark the connection before mark the packet? Yes/No/Why?
Please add some words about this in your second post where you talk about marking.
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3341
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Thu Apr 09, 2020 4:29 pm

I have a question to ask the admins (how do we private message you and talk about the forums)? I am planning on rewriting this article. What is the best course of action to maintain the link (which is pinned and also maybe linked elsewhere)? I would like for all posts to be deleted except for the first six ones (I'm going to go back and edit my own). Is there a better recommendation?
phpBB that this forum is based on do support private message user/user user/admin etc, but for some reason MikroTik has turned this function off. It would be nice to have it one, since there are times you like to exchange private information that are not for all to read.
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Apr 09, 2020 5:01 pm

Do you need to mark the connection before mark the packet? Yes/No/Why?
Before @pcunite addresses that, a quick answer:
Connection marking is a way to give all packets of the same "connection" (a TCP session, or a bi-directional UDP stream) the same treatment without need to analyse all the selection criteria for each individual packet. So usually the processing of the initial packet of a new connection results in choosing a treatment for the whole connection, and all connections whose packets should be treated that way are labeled with the same connection mark. Whenever a packet arrives to the firewall, the connection tracking module matches it to the list of existing connections, and if it finds a match, the connection mark is retrieved from the connection context (along with e.g. the eventual NAT treatment which is, however, applied directly by the connection tracking module) and attached as a metafield to the packet, so that further layers of the firewall (and queueing) can refer to it instead of re-evaluating all the other conditions again. Routing cannot refer to connection-mark directly but needs a routing-mark, so a rule must be used to translate the former into the latter for each packet; queues need a packet-mark so a translation rule is needed too. NAT rules can refer directly to connection-mark because they're an organic part of the firewall; routing and queueing are not. A connection mark may be changed during the lifetime of the connection, but each connection may only bear a single connection mark at any given time, so policy routing along with queueing is a bit complex, yet doable. The change of connection mark in context of QoS is useful to de-prioritize large downloads to allow quick response for small-volume exchanges.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Thu Apr 09, 2020 8:23 pm

Do you need to mark the connection before mark the packet? Yes/No/Why? Please add some words about this in your second post where you talk about marking.

How to use RouterOS to accomplish a task vs. how RouterOS itself works are two different concepts. I only focus on the former. However, everyone is welcome to discuss. The examples show what you need to do in order to facilitate QoS.
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3341
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Apr 09, 2020 8:56 pm

I am trying to learning, so bear with me.

I could mark the HTTP packets directly by setting TCP and port 80 and select mark packet, without marking connection?
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Apr 09, 2020 9:01 pm

I could mark the HTTP packets directly by setting TCP and port 80 and select mark packet, without marking connection?
Yes, that would work too. The only drawback is that matching two items (protocol & port) takes more CPU than matching just one (connection mark), and if you start thinking about discrimination between several categories of http connections, so you start matching e.g. address lists, the amount of CPU power spent on handling every single packet in the firewall grows even higher.
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3341
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Apr 09, 2020 11:00 pm

Thanks for the explanation, some clearer now :)
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Apr 16, 2020 10:46 pm

Set parent queues to have a bucket-size of 0.005. Changed the default queue to sfq. (Using red gave similar performance, but multiple downloads seemed less fair). Rationale for the 0.005 size is to copy CoDel as much as possible.

@bharrisau,

I've have tested your feedback and have made changes to the original post to include bucket-size recommendations, which indeed was helpful. However, I do find SFQ to be situational dependent. Yes, it drops very quickly and responds well to speed tests, but it does not handle global synchronization well, which is a real problem in the wild. Thus, I still show the recommendation for using RED, howbeit, as you did for all queues, which is a cleaner config example. There is still tweaking here to do. More testing is needed under real workloads.
 
bharrisau
just joined
Posts: 6
Joined: Wed Feb 05, 2020 8:35 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Apr 18, 2020 9:55 am

@pcunite
No worries. I'll see if I can rerun the test graphs with the updated baseline when office reopens.
 
Dude2048
Member Candidate
Member Candidate
Posts: 212
Joined: Thu Sep 01, 2016 4:04 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Apr 18, 2020 10:16 am

@pcunite
Thanks for the hard work.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Apr 19, 2020 8:33 pm

@pcunite
No worries. I'll see if I can rerun the test graphs with the updated baseline when office reopens.

I've seen some weird packet issues when using RED for the defaults. So, I've gone back to SFQ for default but use RED for the bulky flows. The behavior of RED as default causes the VoIP queue to drop a single packet here and there when it does not appear to be under pressure. Perhaps there is something else going on, but I never want a packet to drop there, so SFQ (or probably any other type) prevents that behavior.
 
zx128k
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Wed Oct 16, 2013 12:24 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Apr 24, 2020 11:52 am

Guys, can anyone share QOS configuration which prioritizes gaming, browsing and some other important packets over downloads/uploads and video traffics, please?
 
User avatar
digin4
just joined
Posts: 13
Joined: Tue Jan 22, 2019 8:18 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Apr 25, 2020 2:20 pm

Does it make sense to use RED for VPN queue running on UDP only? the congestion methods mentioned all seem to apply to TCP exclusively, but in the guide it's mentioned that it can be applied to QUIC which uses UDP so I gues this means it's fine to use it on UDP VPN?
 
bharrisau
just joined
Posts: 6
Joined: Wed Feb 05, 2020 8:35 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Apr 26, 2020 3:21 am

the congestion methods mentioned all seem to apply to TCP exclusively
TCP and QUIC are stream protocols - as in you have a large amount of data and you will send multiple packets to transfer it. What the connection methods do is to drop some packets as a signal that the stream should slow down. So we are using RED for example to say 'speed up' or 'slow down'. TCP and QUIC have built in mechanisms to resend lost data so it doesn't cause much issue to drop data.

UDP is just a packet format. Depending on the application it will behave differently. Some applications don't have and contention management so dropping packets doesn't cause it to slow down (it actually speeds up because it has to send everything twice). For example VoIP and gaming is usually a fixed rate, and if you drop a packet it causes stuttering or lag. BitTorrent is also UDP, but uses dropped packets to slow the stream down.

So the simple answer is - the router doesn't care if it is TCP or UDP. All TCP connections can be slowed by dropping packets. Some UDP can be slowed, some UDP you don't want to drop packets. It is based on the application though, and not anything to do with UDP.
 
jaguar666
just joined
Posts: 3
Joined: Mon May 04, 2020 1:27 am

Re: Using RouterOS to QoS your network - 2020 Edition

Mon May 04, 2020 7:30 pm

@bharrisau, @pcunite and to all experts out there

Is it recommended to use 2 QOS servers, 1 is for the 2020 QOS-Optimized server (as main bandwidth management) and the other one is for the microCAKE and this will be running inside PPPoE/ Hotspot server?
 
santyx32
Member Candidate
Member Candidate
Posts: 215
Joined: Fri Oct 25, 2019 2:17 am

Re: Using RouterOS to QoS your network - 2020 Edition

Mon May 04, 2020 8:10 pm

@bharrisau I'm using your QOS2020 Optimized script and it's working awesome but on my home network I have two bridges, one for the main network and the other for a guest wifi, how should I proceed?
 
aletrevizoli
just joined
Posts: 3
Joined: Tue Dec 07, 2010 7:42 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Mon May 04, 2020 11:40 pm

I have an BIG doubt.....My network is distributed, I have dozens of MK routers with OSPF on network and on each site I have an OLT and one Router that is PPPoE server.
I have too two places that arrive Internet(with two or more Carrier Link), on each place I have an MK to control this links.
So, where I should use the QOS? in all routers, In PPPoE servers or in Last router (link control).
Thanks a lot!
Very good work!
 
kenyloveg
Frequent Visitor
Frequent Visitor
Posts: 89
Joined: Tue Jul 14, 2009 3:25 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat May 09, 2020 4:48 pm

My network only have two devices connect to one HAP AC2 through 2 RJ45 cable.(one is an openwrt router, and other is windows 7 computer)
What is the most effecient way to make windows 7 computer the higher priority than the other?
Maybe something like mac/ether port based QoS can be easily set up? (for ex, ether2 in bridge1 is higher priority than ether3 in bridge1)
Thanks.
 
jaguar666
just joined
Posts: 3
Joined: Mon May 04, 2020 1:27 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat May 09, 2020 9:32 pm

My network only have two devices connect to one HAP AC2 through 2 RJ45 cable.(one is an openwrt router, and other is windows 7 computer)
What is the most effecient way to make windows 7 computer the higher priority than the other?
Maybe something like mac/ether port based QoS can be easily set up? (for ex, ether2 in bridge1 is higher priority than ether3 in bridge1)
Thanks.
You may create a simple queues entry specifying the target IP or the entire interface you want to prioritize.
 
santyx32
Member Candidate
Member Candidate
Posts: 215
Joined: Fri Oct 25, 2019 2:17 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat May 09, 2020 10:26 pm

You may create a simple queues entry specifying the target IP or the entire interface you want to prioritize.
Yeah, that would get the job done for OP but beware that doing that on a wireless network will also limit LAN-LAN speed. I say this since I've seen lots of tutorials suggesting people do this.
 
santyx32
Member Candidate
Member Candidate
Posts: 215
Joined: Fri Oct 25, 2019 2:17 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sun May 10, 2020 12:13 am

This is my config but I can't get it to work
/ip firewall mangle
add action=mark-connection chain=forward comment=MAIN-DOWNLOAD connection-state=new in-interface=bridge new-connection-mark=main-download passthrough=yes
add action=mark-packet chain=forward connection-mark=main-download new-packet-mark=main-download passthrough=yes
add action=mark-connection chain=forward comment=GUEST-DOWNLOAD connection-state=new in-interface=bridge-invitados new-connection-mark=guest-download passthrough=yes
add action=mark-packet chain=forward connection-mark=guest-download new-packet-mark=guest-download passthrough=yes
add action=mark-connection chain=forward comment=MAIN-UPLOAD connection-state=new new-connection-mark=main-upload out-interface=bridge passthrough=yes
add action=mark-packet chain=forward connection-mark=main-upload new-packet-mark=main-upload passthrough=yes
add action=mark-connection chain=forward comment=GUEST-UPLOAD connection-state=new new-connection-mark=guest-upload out-interface=bridge-invitados passthrough=yes
add action=mark-packet chain=forward connection-mark=guest-upload new-packet-mark=guest-upload passthrough=yes
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP new-connection-mark=VOIP passthrough=yes port=5060-5062,10000-20000 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no
add action=mark-connection chain=prerouting comment=QUIC connection-state=new new-connection-mark=QUIC passthrough=yes port=80,443 protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new new-connection-mark=HTTP passthrough=yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=new new-connection-mark=POP3 passthrough=yes port=995,465,587 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no


/queue tree
add max-limit=68M name=GLOBAL-DOWN parent=global queue=default
add bucket-size=0.005 max-limit=68M name=MAIN-DOWN packet-mark=main-download parent=GLOBAL-DOWN priority=1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=MAIN-DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=MAIN-DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=MAIN-DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=MAIN-DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=MAIN-DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=MAIN-DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=MAIN-DOWN priority=6 queue=default
add name="8. QUIC" packet-mark=QUIC parent=MAIN-DOWN priority=7 queue=default
add name="9. OTHER" packet-mark=OTHER parent=MAIN-DOWN queue=default
add bucket-size=0.005 max-limit=20M name=GUEST-DOWN packet-mark=guest-download parent=GLOBAL-DOWN priority=2 queue=default
add name="1. VOIP_GD" packet-mark=VOIP parent=GUEST-DOWN priority=1 queue=default
add name="2. DNS_GD" packet-mark=DNS parent=GUEST-DOWN priority=2 queue=default
add name="3. ACK_GD" packet-mark=ACK parent=GUEST-DOWN priority=3 queue=default
add name="4. UDP_GD" packet-mark=UDP parent=GUEST-DOWN priority=3 queue=default
add name="5. ICMP_GD" packet-mark=ICMP parent=GUEST-DOWN priority=4 queue=default
add name="6. HTTP_GD" packet-mark=HTTP parent=GUEST-DOWN priority=5 queue=default
add name="7. HTTP_BIG_GD" packet-mark=HTTP_BIG parent=GUEST-DOWN priority=6 queue=default
add name="8. QUIC_GD" packet-mark=QUIC parent=GUEST-DOWN priority=7 queue=default
add max-limit=70M name=GLOBAL-UP parent=global queue=default
add bucket-size=0.005 max-limit=70M name=MAIN-UP packet-mark=main-upload parent=GLOBAL-UP priority=1 queue=default
add name="1. VOIP_" packet-mark=VOIP parent=MAIN-UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=MAIN-UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=MAIN-UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=MAIN-UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=MAIN-UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=MAIN-UP priority=5 queue=bulkUp-Main
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=MAIN-UP priority=6 queue=bulkUp-Main
add name="8. QUIC_" packet-mark=QUIC parent=MAIN-UP priority=7 queue=bulkUp-Main
add name="9. OTHER_" packet-mark=OTHER parent=MAIN-UP queue=bulkUp-Main
add bucket-size=0.005 max-limit=20M name=GUEST-UP packet-mark=guest-upload parent=GLOBAL-UP priority=2 queue=default
add name="1. VOIP_GU" packet-mark=VOIP parent=GUEST-UP priority=1 queue=default
add name="2. DNS_GU" packet-mark=DNS parent=GUEST-UP priority=2 queue=default
add name="3. ACK_GU" packet-mark=ACK parent=GUEST-UP priority=3 queue=default
add name="4. UDP_GU" packet-mark=UDP parent=GUEST-UP priority=3 queue=default
add name="5. ICMP_GU" packet-mark=ICMP parent=GUEST-UP priority=4 queue=default
add name="6. HTTP_GU" packet-mark=HTTP parent=GUEST-UP priority=5 queue=bulkUp-Guest
add name="7. HTTP_BIG_GU" packet-mark=HTTP_BIG parent=GUEST-UP priority=6 queue=bulkUp-Guest
add name="8. QUIC_GU" packet-mark=QUIC parent=GUEST-UP priority=7 queue=bulkUp-Guest
add name="9. OTHER_GU" packet-mark=OTHER parent=GUEST-UP queue=bulkUp-Guest
 
kenyloveg
Frequent Visitor
Frequent Visitor
Posts: 89
Joined: Tue Jul 14, 2009 3:25 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun May 10, 2020 6:19 pm

My network only have two devices connect to one HAP AC2 through 2 RJ45 cable.(one is an openwrt router, and other is windows 7 computer)
What is the most effecient way to make windows 7 computer the higher priority than the other?
Maybe something like mac/ether port based QoS can be easily set up? (for ex, ether2 in bridge1 is higher priority than ether3 in bridge1)
Thanks.
You may create a simple queues entry specifying the target IP or the entire interface you want to prioritize.
Thanks for your replay.
Can you discribe this step more detailed?
I have pppoe-out1 for wan, bridge1 (includ port ether3 and ether4) for internal lan, and deive A connected to ether3, device B to ether4.
I want deivce A get higher priority than device B, just play games regardless the bandwidth.
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: Using RouterOS to QoS your network - 2020 Edition

Fri May 15, 2020 8:56 pm

I ve used Queue Trees mostly with Global Parent.
My question is, why do we use as Parent the Lan Interface for the Download Traffic and the WAN for the Upload Traffic ?
 
dmuzikero
just joined
Posts: 1
Joined: Tue May 19, 2020 4:55 pm

Re: Using RouterOS to prioritize (Qos) traffic for a Class C net

Wed May 20, 2020 11:24 am

version 3, running smooth

Image
Hi can u share script? thanks man :D
 
mikegleasonjr
Frequent Visitor
Frequent Visitor
Posts: 59
Joined: Tue Aug 07, 2018 3:14 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat May 23, 2020 2:48 am

Hi, thanks for the ver. 2020 of the guide. I learned a ton while implementing it and playing with it.

But for me there was a few issues. Mainly regarding the lack of flexibility and problems about incoming packets not being accounted into the queues (even if properly marked by the mangle rules). I'm no expert, only a software engineer so feel free to comment on my findings.

1. lack of flexibility

Having the queue tree main parents (in and out) using interfaces as parents (LAN & WAN) assumes the router is an "internet gateway" only router and that all the traffic would only be internet traffic. What if I want to track (or not track) forwarded traffic between the lan interfaces, etc ? What if I want non-internet related queues at the same time ? I wanted a common root for the Internet traffic, not 2 nodes at the root (in and out).

2. packets not going through the queues

When I was checking a ping and a pong packets going out of the router itself (not forwarded), they would be properly marked by the mangle rules but only the outgoing ones would be accounted in the queues. Since the incoming queue subtree has the LAN as parent, the incoming pong packet would not be accounted (since it never reaches the LAN). You can try and reproduce by yourself by ticking the log checkbox in the mangle rules for icmp and by doing a ping from the router to the outside. Ping might not be a big deal but my DNS traffic were doing the same, because the router makes the queries (in my case). Unless I missed something, that's what I observed.

My solution

I came up which seems the be one of the 2 main approach I saw on the Internet. Mark packets irrespective of their direction (this 2020 guide) or mark separately the ins from outs packets (my final solution). I was able to fix the 2 problems I mentioned earlier with this:

Mangle rules (wlan-2G-client is my wan)

/ip firewall mangle
    add action=mark-connection chain=postrouting comment=icmp connection-state=new new-connection-mark=icmp-conn out-interface=wlan-2G-client passthrough=yes protocol=icmp
    add action=mark-packet chain=postrouting connection-mark=icmp-conn new-packet-mark=icmp-out-pk out-interface=wlan-2G-client passthrough=no
    add action=mark-packet chain=prerouting connection-mark=icmp-conn in-interface=wlan-2G-client new-packet-mark=icmp-in-pk passthrough=no
    add action=mark-connection chain=postrouting comment=dns connection-state=new dst-port=53 new-connection-mark=dns-conn out-interface=wlan-2G-client passthrough=yes protocol=udp
    add action=mark-connection chain=postrouting connection-state=new dst-port=53 new-connection-mark=dns-conn out-interface=wlan-2G-client passthrough=yes protocol=tcp
    add action=mark-packet chain=postrouting connection-mark=dns-conn new-packet-mark=dns-out-pk out-interface=wlan-2G-client passthrough=no
    add action=mark-packet chain=prerouting connection-mark=dns-conn in-interface=wlan-2G-client new-packet-mark=dns-in-pk passthrough=no
    add action=mark-connection chain=postrouting comment=voip connection-state=new dst-port=5060,5061 new-connection-mark=voip-conn out-interface=wlan-2G-client passthrough=yes protocol=udp
    add action=mark-connection chain=postrouting connection-state=new dst-port=5060,5061 new-connection-mark=voip-conn out-interface=wlan-2G-client passthrough=yes protocol=tcp
    add action=mark-packet chain=postrouting connection-mark=voip-conn new-packet-mark=voip-out-pk out-interface=wlan-2G-client passthrough=no
    add action=mark-packet chain=prerouting connection-mark=voip-conn in-interface=wlan-2G-client new-packet-mark=voip-in-pk passthrough=no
    add action=mark-connection chain=postrouting comment=zoom connection-state=new dst-port=3478,3479,8801-8810,5090 new-connection-mark=zoom-conn out-interface=wlan-2G-client passthrough=yes protocol=udp
    add action=mark-connection chain=postrouting connection-state=new dst-port=8801,8802,5090,5091 new-connection-mark=zoom-conn out-interface=wlan-2G-client passthrough=yes protocol=tcp
    add action=mark-packet chain=postrouting connection-mark=zoom-conn new-packet-mark=zoom-out-pk out-interface=wlan-2G-client passthrough=no
    add action=mark-packet chain=prerouting connection-mark=zoom-conn in-interface=wlan-2G-client new-packet-mark=zoom-in-pk passthrough=no
    add action=mark-connection chain=postrouting comment=rocket-league connection-state=new dst-port=7000-8800,8811-8887,8889-9000 new-connection-mark=rocketleague-conn out-interface=wlan-2G-client passthrough=yes protocol=udp
    add action=mark-packet chain=postrouting connection-mark=rocketleague-conn new-packet-mark=rocketleague-out-pk out-interface=wlan-2G-client passthrough=no
    add action=mark-packet chain=prerouting connection-mark=rocketleague-conn in-interface=wlan-2G-client new-packet-mark=rocketleague-in-pk passthrough=no
    add action=mark-connection chain=postrouting comment=other connection-state=new new-connection-mark=other-conn out-interface=wlan-2G-client passthrough=yes
    add action=mark-packet chain=postrouting connection-mark=other-conn new-packet-mark=other-out-pk out-interface=wlan-2G-client passthrough=no
    add action=mark-packet chain=prerouting connection-mark=other-conn in-interface=wlan-2G-client new-packet-mark=other-in-pk passthrough=no

Queues

/queue tree
    add max-limit=22M name=internet parent=global queue=ethernet-default
    add max-limit=10M name=internet-in parent=internet queue=ethernet-default
    add max-limit=12M name=internet-out parent=internet queue=ethernet-default
    add name=zoom-in packet-mark=zoom-in-pk parent=internet-in priority=4 queue=ethernet-default
    add name=zoom-out packet-mark=zoom-out-pk parent=internet-out priority=4 queue=ethernet-default
    add name=icmp-out packet-mark=icmp-out-pk parent=internet-out priority=2 queue=ethernet-default
    add name=icmp-in packet-mark=icmp-in-pk parent=internet-in priority=2 queue=ethernet-default
    add name=dns-out packet-mark=dns-out-pk parent=internet-out priority=2 queue=ethernet-default
    add name=dns-in packet-mark=dns-in-pk parent=internet-in priority=2 queue=ethernet-default
    add name=voip-in packet-mark=voip-in-pk parent=internet-in priority=2 queue=ethernet-default
    add name=voip-out packet-mark=voip-out-pk parent=internet-out priority=2 queue=ethernet-default
    add name=rocketleague-in packet-mark=rocketleague-in-pk parent=internet-in priority=4 queue=ethernet-default
    add name=rocketleague-out packet-mark=rocketleague-out-pk parent=internet-out priority=4 queue=ethernet-default
    add bucket-size=0.01 max-limit=10M name=other-out packet-mark=other-out-pk parent=internet-out queue=pcq-upload-default
    add max-limit=8M name=other-in packet-mark=other-in-pk parent=internet-in queue=pcq-download-default

A picture is worth a thousand words

winbox.png

With this setup, I am able to track any traffic that goes out of my wan port, whether or not it originates from the router itself or not. No more, no less.

Hope this helps someone.
You do not have the required permissions to view the files attached to this post.
 
jamesbutterfill
just joined
Posts: 1
Joined: Fri Jun 05, 2020 5:13 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Jun 05, 2020 5:18 pm

Hi All. Firstly thanks for all this hard work. I've been having a bit of an issue though. I have the LHG Cat6 LTE Mikrotik router but my speed on a day by day basis varies significantly with downloads varying from 20-80mbits depending on environmental/network conditions. My upload can vary from 2-12mbits too. So what speeds should I specify without inhibiting performance on the days when the UL/DL rates are much higher?
Last edited by jamesbutterfill on Fri Jun 05, 2020 5:36 pm, edited 1 time in total.
 
WayneF
newbie
Posts: 34
Joined: Sat May 23, 2020 4:51 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jun 11, 2020 2:26 am

How should I go about choosing sensible max limits for the queues.

I have a X Mpbs down and a Y Mbps up; well so the ISP claims. When I actually do tests I hardly ever see those results when using speedtest.net,
using various test servers, it seems to vary as most test servers fail to deliver the speeds.

Should I be setting the limit's to more practical values, values I most typically see?

Also should one be considering the Maximum WiFi Speeds and use different queues for the WiFi and have different Max values?
(to me this makes sense as I cannot get the maximum values over WiFi; my crude testing I could see icmp response time increases when I
sent packets over the WiFi while also doing a speedtest over the WiFi).


Also how does one establish the WiFi max Speed using Mac to Mikrotik?

Here is my crude testing strategy:
1) I start sending (icmp packets) pings continuously to googles dns server 8.8.8.8 - using the Mikrotik Ping tool
2) then I do a speedtest over WiFi to the fastest server I can find.
3) record the icmp response times and dropped packets in the queues.

[Download test]
No increase in response times.
No dropped packets. (Which is expect as my WiFi cannot get close to the Max Download of X Mpbs - 10%.)

[Upload test]
No increase in response times.
And some dropped packets for the "OTHER" entry.

When I ping over WiFi the icmp response times are affected as I am maximising the WiFi throughput I assume.
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jun 18, 2020 7:54 am

TCP uses a three-way handshake to establish a reliable connection, why prioritize the ack traffic only??
Image
Last edited by kswong on Sun Jan 23, 2022 10:34 pm, edited 28 times in total.
 
erhtun
newbie
Posts: 26
Joined: Thu May 31, 2018 12:34 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jun 25, 2020 12:43 pm

@pcunite, @kswong what is your queue > interface types for Wan interface(ether1) and wifi interfaces?
 
sbernesto
just joined
Posts: 1
Joined: Fri Mar 30, 2018 8:20 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jun 25, 2020 10:00 pm

Hi,
Why is it necessary to deactivate the filter fasttrack connection?
or only in this particular configuration is it required or does it apply to all config shared?
TCP uses a three-way handshake to establish a reliable connection, why prioritize the ack traffic only??
Image

My setup able to track the TCP 3-way & 4-way handshake traffic ...
Image

Mangle rules
/ip firewall mangle
add action=mark-packet chain=prerouting comment="FIN / SYN / ACK / RST in" in-interface-list=WAN new-packet-mark=FIN-I packet-size=0-64 passthrough= no protocol=tcp tcp-flags=fin
add action=mark-packet chain=prerouting in-interface-list=WAN new-packet-mark=SYN-I packet-size=0-64 passthrough=no protocol=tcp tcp-flags=syn
add action=mark-packet chain=prerouting in-interface-list=WAN new-packet-mark=RST-I packet-size=0-64 passthrough=no protocol=tcp tcp-flags=rst
add action=mark-packet chain=prerouting in-interface-list=WAN new-packet-mark=ACK-I packet-size=0-64 passthrough=no protocol=tcp tcp-flags=ack,!psh
add action=mark-packet chain=postrouting comment="FIN / SYN / ACK / RST out" new-packet-mark=FIN-O out-interface-list=WAN packet-size=0-64 passthrough=no protocol=tcp tcp-flags=fin
add action=mark-packet chain=postrouting new-packet-mark=SYN-O out-interface-list=WAN packet-size=0-64 passthrough=no protocol=tcp tcp-flags=syn
add action=mark-packet chain=postrouting new-packet-mark=RST-O out-interface-list=WAN packet-size=0-64 passthrough=no protocol=tcp tcp-flags=rst
add action=mark-packet chain=postrouting new-packet-mark=ACK-O out-interface-list=WAN packet-size=0-64 passthrough=no protocol=tcp tcp-flags=ack,!psh
add action=mark-connection chain=prerouting comment="ICMP in" connection-mark=no-mark new-connection-mark=CM-ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=CM-ICMP in-interface-list=WAN new-packet-mark=ICMP-I passthrough=no
add action=mark-connection chain=postrouting comment="ICMP out" connection-mark=no-mark new-connection-mark=CM-ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=CM-ICMP new-packet-mark=ICMP-O out-interface-list=WAN passthrough=no
add action=mark-connection chain=prerouting comment="DNS in" connection-mark= no-mark dst-port=53,853 new-connection-mark=CM-DNS passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-port= 53,853 new-connection-mark=CM-DNS passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=CM-DNS in-interface-list=WAN new-packet-mark=DNS-I passthrough=no
add action=mark-connection chain=postrouting comment="DNS out" connection-mark=no-mark dst-port=53,853 new-connection-mark=CM-DNS passthrough=yes protocol=tcp
add action=mark-connection chain=postrouting connection-mark=no-mark dst-port=53,853 new-connection-mark=CM-DNS passthrough=yes protocol=udp
add action=mark-packet chain=postrouting connection-mark=CM-DNS new-packet-mark=DNS-O out-interface-list=WAN passthrough=no
add action=mark-connection chain=prerouting comment="VPN in" connection-mark= no-mark dst-port=1723 new-connection-mark=CM-VPN passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting connection-mark=no-mark dst-port= 500,1194,1701,4500 new-connection-mark=CM-VPN passthrough=yes protocol= udp
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=CM-VPN passthrough=yes protocol=gre
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=CM-VPN passthrough=yes protocol=ipsec-esp
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=CM-VPN passthrough=yes protocol=ipsec-ah
add action=mark-packet chain=prerouting connection-mark=CM-VPN in-interface-list=WAN new-packet-mark=VPN-I passthrough=no
add action=mark-connection chain=postrouting comment="VPN out" connection-mark=no-mark dst-port=1723 new-connection-mark=CM-VPN passthrough=yes protocol=tcp
add action=mark-connection chain=postrouting connection-mark=no-mark dst-port=500,1194,1701,4500 new-connection-mark=CM-VPN passthrough=yes protocol=udp
add action=mark-connection chain=postrouting connection-mark=no-mark new-connection-mark=CM-VPN passthrough=yes protocol=gre
add action=mark-connection chain=postrouting connection-mark=no-mark new-connection-mark=CM-VPN passthrough=yes protocol=ipsec-esp
add action=mark-connection chain=postrouting connection-mark=no-mark new-connection-mark=CM-VPN passthrough=yes protocol=ipsec-ah
add action=mark-packet chain=postrouting connection-mark=CM-VPN new-packet-mark=VPN-O out-interface-list=WAN passthrough=no
add action=mark-connection chain=prerouting comment="VOIP in" connection-mark=no-mark dscp=26 new-connection-mark=CM-VOIP passthrough= yes
add action=mark-connection chain=prerouting connection-mark=no-mark dscp=46 new-connection-mark=CM-VOIP passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark dst-port= 3478-3479,5060-5061,8801-8802 new-connection-mark=CM-VOIP passthrough= yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=CM-VOIP in-interface-list=WAN new-packet-mark=VOIP-I passthrough=no
add action=mark-connection chain=postrouting comment="VOIP out" connection-mark=no-mark dscp=26 new-connection-mark=CM-VOIP passthrough= yes
add action=mark-connection chain=postrouting connection-mark=no-mark dscp=46 new-connection-mark=CM-VOIP passthrough=yes
add action=mark-connection chain=postrouting connection-mark=no-mark dst-port=3478-3479,5060-5061,8801-8802 new-connection-mark=CM-VOIP passthrough=yes protocol=udp
add action=mark-packet chain=postrouting connection-mark=CM-VOIP new-packet-mark=VOIP-O out-interface-list=WAN passthrough=no
add action=mark-connection chain=prerouting comment="EMAIL in" connection-mark=no-mark dst-port=25,110,143,465,587,993,995 new-connection-mark=CM-EMAIL passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=CM-EMAIL in-interface-list=WAN new-packet-mark=EMAIL-I passthrough=no
add action=mark-connection chain=postrouting comment="EMAIL out" connection-mark=no-mark dst-port=25,110,143,465,587,993,995 new-connection-mark=CM-EMAIL passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting connection-mark=CM-EMAIL new-packet-mark=EMAIL-O out-interface-list=WAN passthrough=no
add action=mark-connection chain=prerouting comment="QUIC in" connection-mark=no-mark dst-port=80,443 new-connection-mark=CM-QUIC passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=CM-QUIC in-interface-list=WAN new-packet-mark=QUIC-I passthrough=no
add action=mark-connection chain=postrouting comment="QUIC out" connection-mark=no-mark dst-port=80,443 new-connection-mark=CM-QUIC passthrough=yes protocol=udp
add action=mark-packet chain=postrouting connection-mark=CM-QUIC new-packet-mark=QUIC-O out-interface-list=WAN passthrough=no
add action=mark-connection chain=prerouting comment="HTTP in" connection-mark=no-mark dst-port=80,443,8080,8443 new-connection-mark= CM-HTTP passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=1000000-0 connection-mark=CM-HTTP connection-rate=!0-8k new-connection-mark= CM-HTTP-L passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=CM-HTTP-L in-interface-list=WAN new-packet-mark=HTTP-B-I passthrough=no
add action=mark-packet chain=prerouting connection-mark=CM-HTTP in-interface-list=WAN new-packet-mark=HTTP-I passthrough=no
add action=mark-connection chain=postrouting comment="HTTP out" connection-mark=no-mark dst-port=80,443,8080,8443 new-connection-mark= CM-HTTP passthrough=yes protocol=tcp
add action=mark-connection chain=postrouting connection-bytes=1000000-0 connection-mark=CM-HTTP connection-rate=!0-8k new-connection-mark= CM-HTTP-L passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting connection-mark=CM-HTTP-L new-packet-mark=HTTP-B-O out-interface-list=WAN passthrough=no
add action=mark-packet chain=postrouting connection-mark=CM-HTTP new-packet-mark=HTTP-O out-interface-list=WAN passthrough=no
add action=mark-connection chain=prerouting comment="OTHER in" connection-mark=no-mark new-connection-mark=CM-OTHER passthrough=yes
add action=mark-connection chain=prerouting connection-mark=CM-OTHER new-connection-mark=CM-OTHER-L packet-size=!0-400 passthrough=yes
add action=mark-packet chain=prerouting connection-mark=CM-OTHER-L in-interface-list=WAN new-packet-mark=OTHER-B-I passthrough=no
add action=mark-packet chain=prerouting connection-mark=CM-OTHER in-interface-list=WAN new-packet-mark=OTHER-I passthrough=no
add action=mark-connection chain=postrouting comment="OTHER out" connection-mark=no-mark new-connection-mark=CM-OTHER passthrough=yes
add action=mark-connection chain=postrouting connection-mark=CM-OTHER new-connection-mark=CM-OTHER-L packet-size=!0-400 passthrough=yes
add action=mark-packet chain=postrouting connection-mark=CM-OTHER-L new-packet-mark=OTHER-B-O out-interface-list=WAN passthrough=no
add action=mark-packet chain=postrouting connection-mark=CM-OTHER new-packet-mark=OTHER-O out-interface-list=WAN passthrough=no

Image

Queue tree
/queue tree
add bucket-size=0.01 max-limit=210M name=INTERNET parent=global queue=default
add bucket-size=0.01 max-limit=200M name=IN parent=INTERNET queue=default
add bucket-size=0.01 max-limit=10M name=OUT parent=INTERNET queue=default
add name=DNS_IN packet-mark=DNS-I parent=IN priority=1 queue=default
add name=NETWORK_IN packet-mark= FIN-I,SYN-I,RST-I,ACK-I,ICMP-I parent=IN priority=3 queue=default
add name=VOIP_IN packet-mark=VOIP-I parent=IN priority=2 queue=default
add name=OTHER_IN packet-mark=OTHER-I,VPN-I parent=IN priority=4 queue=default
add name=HTTP_IN packet-mark=HTTP-I,EMAIL-I parent=IN priority=5 queue=default
add name=HTTP_BIG_IN packet-mark=HTTP-B-I parent= IN priority=6 queue=default
add name=QUIC_IN packet-mark=QUIC-I parent=IN priority=7 queue=default
add name=OTHER_BIG_IN packet-mark=OTHER-B-I parent=IN queue=default
add name=DNS_OUT packet-mark=DNS-O parent=OUT priority=1 queue=default
add name=NETWORK_OUT packet-mark= FIN-O,SYN-O,RST-O,ACK-O,ICMP-O parent=OUT priority=3 queue=default
add name=VOIP_OUT packet-mark=VOIP-O parent=OUT priority=2 queue=default
add name=OTHER_OUT packet-mark=OTHER-O,VPN-O parent=OUT priority=4 queue=default
add name=HTTP_OUT packet-mark=HTTP-O,EMAIL-O parent=OUT priority=5 queue=default
add name=HTTP_BIG_OUT packet-mark=HTTP-B-O parent=OUT priority=6 queue=default
add name=QUIC_OUT packet-mark=QUIC-O parent=OUT priority=7 queue=default
add name=OTHER_BIG_OUT packet-mark=OTHER-B-O parent=OUT queue=default

Make sure that the fasttrack connection is turned off
Image
 
bharrisau
just joined
Posts: 6
Joined: Wed Feb 05, 2020 8:35 am

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Jun 26, 2020 3:37 pm

TCP uses a three-way handshake to establish a reliable connection, why prioritize the ack traffic only??
We aren't trying to prioritize the handshake. The handshake is only 3 small packets to establish the TCP connection.

Once you establish the connection and then start downloading at full speed, your computer need to tell the remote end that is successfully received the data. It does this by sending ACK packets. By prioritizing these ACK packets we can ensure that uploads on a connection do not impact downloads. If we didn't prioritize the ACK then the downloads would be limited because the remote computer would be waiting on the ACK until is sends more data.

Hope that clarifies somewhat.
 
erhtun
newbie
Posts: 26
Joined: Thu May 31, 2018 12:34 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Jun 27, 2020 12:21 am

TCP uses a three-way handshake to establish a reliable connection, why prioritize the ack traffic only??
We aren't trying to prioritize the handshake. The handshake is only 3 small packets to establish the TCP connection.

Once you establish the connection and then start downloading at full speed, your computer need to tell the remote end that is successfully received the data. It does this by sending ACK packets. By prioritizing these ACK packets we can ensure that uploads on a connection do not impact downloads. If we didn't prioritize the ACK then the downloads would be limited because the remote computer would be waiting on the ACK until is sends more data.

Hope that clarifies somewhat.

I have 100M Down / 5M Up cable connection. I don't have voip calls. I rarely upload files. So under this circumstances should I prioritize ACK packets?

Another question I want to ask is about "parent" settings in queue tree. When to use parent=global or parent=interface(ether1, bridge etc.) for optimum performance? Why?
 
jaguar666
just joined
Posts: 3
Joined: Mon May 04, 2020 1:27 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Jun 27, 2020 7:09 am

Why is it necessary to deactivate the filter fasttrack connection?
The mangle rules and queue tree will not act accordingly if you enable fasttrack connections in /ip firewall filter (default config).
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jun 28, 2020 4:00 pm

# Identify VoIP
add chain=prerouting  action=mark-connection new-connection-mark=VOIP port=5060-5062,10000-20000 protocol=udp passthrough=yes comment="VOIP"
add chain=prerouting  action=mark-packet     connection-mark=VOIP new-packet-mark=VOIP passthrough=no
Does this account for all VoIP like VoWiFi, Google Duo, FaceTime etc?

Is this correct for ICMPv6 marking? Of course, I used the same marking name "ICMP" from IPv4 rules.
# Identify PING on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmpv6 passthrough=yes comment="ICMP"
add chain=prerouting  action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmpv6 passthrough=yes
add chain=postrouting action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no
Last edited by DarkNate on Fri Jul 17, 2020 3:14 am, edited 1 time in total.
 
dikrek
just joined
Posts: 4
Joined: Mon Jun 29, 2020 2:04 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jul 02, 2020 9:52 pm

How would one use this approach (say, MikroCake) with LTE? The speeds are variable, but I'd argue it would still benefit from ACK prioritization etc.

Anyone care to show a sample?
 
User avatar
digin4
just joined
Posts: 13
Joined: Tue Jan 22, 2019 8:18 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jul 12, 2020 5:58 pm

Is it possible to prioritize DoH (DNS over HTTPS) traffic such as 1.1.1.1?
I created a src-list with all the IP address of DoH providers, then added this mangle rule:
add action=mark-packet chain=forward new-packet-mark=DoH passthrough=no src-address-list=DoH comment="DoH"
But the packets aren't getting marked at all, what am I doing wrong?
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Jul 13, 2020 12:13 am

Do you really want to mark the responses from the DoH servers? If you actually want to mark the queries, the rule must match on dst-address-list, not the src one. Regardless that - if a rule's statistics doesn't count at all although you know that matching packets do exist, that rule is likely shadowed by some other one before (above) it, which matches the same packets and has passthrough=no (implicitly or explicitly).
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jul 16, 2020 2:19 am

Do you really want to mark the responses from the DoH servers? If you actually want to mark the queries, the rule must match on dst-address-list, not the src one. Regardless that - if a rule's statistics doesn't count at all although you know that matching packets do exist, that rule is likely shadowed by some other one before (above) it, which matches the same packets and has passthrough=no (implicitly or explicitly).
I tried this, but it is not working.

add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new dst-address-list=!RFC1918 new-connection-mark=HTTP passthrough=yes port=80,443 protocol=\
    tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=3M-100M dst-address-list=!RFC1918 new-connection-mark=HTTP_BIG passthrough=yes \
    protocol=tcp
add action=mark-packet chain=forward comment=DoH connection-mark=HTTP dst-address-list=DoH log=yes log-prefix=DoHLogging new-packet-mark=DNS passthrough=no tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
 
User avatar
digin4
just joined
Posts: 13
Joined: Tue Jan 22, 2019 8:18 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jul 16, 2020 10:22 am

This is what I'm using and it's working:
# Identify DoH
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=DoH passthrough=yes dst-address-list=DoH comment="DoH"
add chain=prerouting  action=mark-packet     connection-mark=DoH  new-packet-mark=DoH passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=DoH passthrough=yes dst-address-list=DoH
add chain=postrouting action=mark-packet     connection-mark=DoH  new-packet-mark=DoH passthrough=no
Do you really want to mark the responses from the DoH servers? If you actually want to mark the queries, the rule must match on dst-address-list, not the src one. Regardless that - if a rule's statistics doesn't count at all although you know that matching packets do exist, that rule is likely shadowed by some other one before (above) it, which matches the same packets and has passthrough=no (implicitly or explicitly).
I tried this, but it is not working.

add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new dst-address-list=!RFC1918 new-connection-mark=HTTP passthrough=yes port=80,443 protocol=\
    tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=3M-100M dst-address-list=!RFC1918 new-connection-mark=HTTP_BIG passthrough=yes \
    protocol=tcp
add action=mark-packet chain=forward comment=DoH connection-mark=HTTP dst-address-list=DoH log=yes log-prefix=DoHLogging new-packet-mark=DNS passthrough=no tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jul 16, 2020 2:11 pm

This is what I'm using and it's working:
# Identify DoH
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=DoH passthrough=yes dst-address-list=DoH comment="DoH"
add chain=prerouting  action=mark-packet     connection-mark=DoH  new-packet-mark=DoH passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=DoH passthrough=yes dst-address-list=DoH
add chain=postrouting action=mark-packet     connection-mark=DoH  new-packet-mark=DoH passthrough=no
Do you really want to mark the responses from the DoH servers? If you actually want to mark the queries, the rule must match on dst-address-list, not the src one. Regardless that - if a rule's statistics doesn't count at all although you know that matching packets do exist, that rule is likely shadowed by some other one before (above) it, which matches the same packets and has passthrough=no (implicitly or explicitly).
I tried this, but it is not working.

add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new dst-address-list=!RFC1918 new-connection-mark=HTTP passthrough=yes port=80,443 protocol=\
    tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=3M-100M dst-address-list=!RFC1918 new-connection-mark=HTTP_BIG passthrough=yes \
    protocol=tcp
add action=mark-packet chain=forward comment=DoH connection-mark=HTTP dst-address-list=DoH log=yes log-prefix=DoHLogging new-packet-mark=DNS passthrough=no tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
Thanks that seems to be working!

I've added TCP protocol and port 53, 443 and 853 to the mangle rule to ensure it only deals with specific connectivity to DoH address list. A more cleaner approach.

Why port 53, 853? It is to help catch DoH traffic from my client devices who may run independently for DoH queries.
Source: https://en.wikipedia.org/wiki/DNS_over_ ... _scenarios
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Jul 17, 2020 4:00 am

@pcunite

Does "connection-state" matter? If so why is VoIP lacking connection state?

Should we only mark the "new" connections? What about established and related?

When I look through connection tracking I can see a lot of what I'd assume to be "related" connections that are not being marked by a mangle.
 
smileymattj
newbie
Posts: 33
Joined: Tue Apr 19, 2011 10:12 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Jul 17, 2020 6:10 am

First off, nice work pcunite. That's a lot of effort you put in and very well documented.

Here is something I was thinking of. If you trust the DSCP markings applied by the devices on your network. Typically you can unless your devices are compromised. You could first check to see if the device is already prioritizing it's packets with a DSCP and use that first.

Also for the OTHER, I set it to no-mark and took out the mangle for those. Should make it slightly more efficient.
/ip setting set allow-fast-path=no


# Place at very top

/ip firewall mangle

add action=mark-connection chain=prerouting dscp=0 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=1 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=2 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=3 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=4 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=5 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=6 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=7 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=prerouting dscp=8 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=9 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=10 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=11 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=12 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=13 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=14 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=15 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=prerouting dscp=16 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=17 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=18 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=19 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=20 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=21 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=22 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=23 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=prerouting dscp=24 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=25 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=26 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=27 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=28 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=29 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=30 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=31 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=prerouting dscp=32 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=33 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=34 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=35 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=36 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=37 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=38 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=39 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=prerouting dscp=40 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=41 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=42 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=43 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=44 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=45 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=46 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=47 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=prerouting dscp=48 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=prerouting dscp=49 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=prerouting dscp=50 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=prerouting dscp=51 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=prerouting dscp=52 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=prerouting dscp=53 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=prerouting dscp=54 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=prerouting dscp=55 comment=TOS_6 new-connection-mark=QUIC
# add action=mark-connection chain=prerouting dscp=56 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=prerouting dscp=57 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=prerouting dscp=58 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=prerouting dscp=59 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=prerouting dscp=60 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=prerouting dscp=61 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=prerouting dscp=62 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=prerouting dscp=63 comment=TOS_7 new-connection-mark=OTHER

add action=mark-connection chain=postrouting dscp=0 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=1 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=2 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=3 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=4 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=5 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=6 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=7 comment=TOS_0 new-connection-mark=VOIP
add action=mark-connection chain=postrouting dscp=8 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=9 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=10 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=11 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=12 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=13 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=14 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=15 comment=TOS_1 new-connection-mark=DNS
add action=mark-connection chain=postrouting dscp=16 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=17 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=18 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=19 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=20 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=21 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=22 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=23 comment=TOS_2 new-connection-mark=ACK
add action=mark-connection chain=postrouting dscp=24 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=25 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=26 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=27 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=28 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=29 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=30 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=31 comment=TOS_3 new-connection-mark=ICMP
add action=mark-connection chain=postrouting dscp=32 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=33 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=34 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=35 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=36 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=37 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=38 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=39 comment=TOS_4 new-connection-mark=HTTP
add action=mark-connection chain=postrouting dscp=40 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=41 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=42 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=43 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=44 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=45 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=46 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=47 comment=TOS_5 new-connection-mark=HTTP_BIG
add action=mark-connection chain=postrouting dscp=48 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=postrouting dscp=49 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=postrouting dscp=50 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=postrouting dscp=51 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=postrouting dscp=52 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=postrouting dscp=53 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=postrouting dscp=54 comment=TOS_6 new-connection-mark=QUIC
add action=mark-connection chain=postrouting dscp=55 comment=TOS_6 new-connection-mark=QUIC
# add action=mark-connection chain=postrouting dscp=56 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=postrouting dscp=57 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=postrouting dscp=58 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=postrouting dscp=59 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=postrouting dscp=60 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=postrouting dscp=61 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=postrouting dscp=62 comment=TOS_7 new-connection-mark=OTHER
# add action=mark-connection chain=postrouting dscp=63 comment=TOS_7 new-connection-mark=OTHER


# Insert other/custom capture rules here




# pcunite's mangle

# Identify DNS on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes comment="DNS"
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes comment="DNS"

# Identify VoIP
add chain=prerouting  action=mark-connection new-connection-mark=VOIP port=5060-5062,10000-20000 protocol=udp passthrough=yes comment="VOIP"

# Identify HTTP/3 and Google's QUIC
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=QUIC port=80,443 protocol=udp passthrough=yes comment="QUIC"

# Identify UPD. Useful for further analysis. Should it be considered high priority or put in the catchall? You decide.
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=UDP protocol=udp passthrough=yes comment="UDP"

# Identify PING on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes comment="ICMP"
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes comment="ICMP"

# Identify Acknowledgment packets
add chain=postrouting action=mark-packet new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no comment="ACK"
add chain=prerouting  action=mark-packet new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no comment="ACK"

# Identify HTTP traffic but move it to a Streaming mark if necessary.
add chain=prerouting  action=mark-connection connection-mark=no-mark connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp passthrough=yes comment="HTTP"
add chain=prerouting  action=mark-connection connection-bytes=5M-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG protocol=tcp passthrough=yes comment="HTTP_BIG"


# Mark Packets
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no comment=VOIP
add action=mark-packet chain=postrouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no comment=VOIP

add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no comment=DNS
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no comment=DNS

add action=mark-packet chain=prerouting connection-mark=ACK new-packet-mark=ACK passthrough=no comment=ACK
add action=mark-packet chain=postrouting connection-mark=ACK new-packet-mark=ACK passthrough=no comment=ACK

add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no comment=UDP
add action=mark-packet chain=postrouting connection-mark=UDP new-packet-mark=UDP passthrough=no comment=UDP

add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no comment=ICMP
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no comment=ICMP

add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no comment=HTTP
add action=mark-packet chain=postrouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no comment=HTTP

add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no comment=HTTP_BIG
add action=mark-packet chain=postrouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no comment=HTTP_BIG

add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no comment=QUIC
add action=mark-packet chain=postrouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no comment=QUIC


/queue tree

# DOWN
add name=DOWN max-limit=900M parent=LAN bucket-size=0.01 queue=default

add name="1. VOIP"     packet-mark=VOIP     parent=DOWN priority=1 queue=default
add name="2. DNS"      packet-mark=DNS      parent=DOWN priority=2 queue=default
add name="3. ACK"      packet-mark=ACK      parent=DOWN priority=3 queue=default
add name="4. UDP"      packet-mark=UDP      parent=DOWN priority=3 queue=default
add name="5. ICMP"     packet-mark=ICMP     parent=DOWN priority=4 queue=default
add name="6. HTTP"     packet-mark=HTTP     parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC"     packet-mark=QUIC     parent=DOWN priority=7 queue=default
add name="9. OTHER"    packet-mark=no-mark  parent=DOWN priority=8 queue=default


# UP
add name=UP max-limit=900M parent=WAN bucket-size=0.01 queue=default

add name="1. VOIP_"     packet-mark=VOIP     parent=UP priority=1 queue=default
add name="2. DNS_"      packet-mark=DNS      parent=UP priority=2 queue=default
add name="3. ACK_"      packet-mark=ACK      parent=UP priority=3 queue=default
add name="4. UDP_"      packet-mark=UDP      parent=UP priority=3 queue=default
add name="5. ICMP_"     packet-mark=ICMP     parent=UP priority=4 queue=default
add name="6. HTTP_"     packet-mark=HTTP     parent=UP priority=5 queue=default
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="8. QUIC_"     packet-mark=QUIC     parent=UP priority=7 queue=default
add name="9. OTHER_"    packet-mark=no-mark  parent=UP priority=8 queue=default

 
tawh
just joined
Posts: 17
Joined: Sun Jun 30, 2013 2:11 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jul 19, 2020 12:49 pm

Backgound:
I have built a MPLS network over full mesh EOIP tunnels on all edge routers. I also configured VPLS tunnels in the MPLS network. all network function are worked as expected.

Problem:
There are both MPLS and non-MPLS traffics running over the EOIP tunnels, sometimes the non-MPLS traffic are so huge that some of the MPLS packets get dropped.

I want the priority all VPLS traffic (or generalized as all MPLS packets) to take higher precedence over non-MPLS traffics over the EOIP tunnels. However, I cannot figure out a mangle rule to identify the MPLS/VPLS packets.

Question:
Is there anything I missed or is there any alternative way to achieve this goal?
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jul 19, 2020 4:19 pm

MPLS packets are invisible to the ip firewall rules (unless you set use-ip-firewall=yes in /interface bridge settings, which affects a lot of things so don't do that unless you absolutely have to). So this topic suggests two approaches - to assign some packet-mark to all IP traffic, and let the queue for MPLS traffic be chosen by an absence of any packet-mark, or to connect the EoIP interfaces to dedicated bridges, and use /interface bridge filter rules rather than /ip firewall mangle ones to assign packet marks to the MPLS packets.
 
tawh
just joined
Posts: 17
Joined: Sun Jun 30, 2013 2:11 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jul 19, 2020 10:48 pm

MPLS packets are invisible to the ip firewall rules (unless you set use-ip-firewall=yes in /interface bridge settings, which affects a lot of things so don't do that unless you absolutely have to). So this topic suggests two approaches - to assign some packet-mark to all IP traffic, and let the queue for MPLS traffic be chosen by an absence of any packet-mark, or to connect the EoIP interfaces to dedicated bridges, and use /interface bridge filter rules rather than /ip firewall mangle ones to assign packet marks to the MPLS packets.
Thanks for your reply.
I tried the solution proposed by your reference link, I found I cannot work out the expected result. I suspect the main problem is that in my case the ingress MPLS router and the edge router are at the same router, while there were two separate routers in the reference link.
I further investigated the packet flow diagram from Mikrotik, In my case, the packets reaches my router with VLAN tag, then it bridges to my VPLS interface. since my VPLS circuit is further encapsulated in EOIP, as a result, the following flow shall be seen (This diagram is derived from 4.3 and 4.4 of https://wiki.mikrotik.com/wiki/Manual:Packet_Flow):

Image

I believe the packet marking at the early stage (step 8 to step 9) of this flow didn't inherit to the VPLS (step 13) and the final EOIP packets (step 21).

Observation:
(i) I created bridge rule to add packet marks "abc" to all VLAN1 packets
(ii) I created a queue tree which selects packet mark "abc" and "eoip" interface as parent (step 18). NO packets were queued
(iii) I changed the parent interface to the "vpls" interface (step 10), packets were queued.

Can anyone review whether the packet flow is correct and any solution to this scenario? any help is greatly appreciated.
 
User avatar
sindy
Forum Guru
Forum Guru
Posts: 11053
Joined: Mon Dec 04, 2017 9:19 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Jul 20, 2020 1:10 pm

I found I cannot work out the expected result
Can you please create a dedicated topic and place a link to it here? I think your case is quite specific and should be treated separately.
 
ivicask
Member
Member
Posts: 438
Joined: Tue Jul 07, 2015 2:40 pm
Location: Croatia, Zagreb

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jul 21, 2020 12:58 pm

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.
You do not have the required permissions to view the files attached to this post.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jul 22, 2020 6:33 pm

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.
The reason why he marked connections instead of packets is to reduce load on the CPU.

I've encountered these anamolies you described as well. I've made a lot of modifications to the mangle rules since though.

One thing I'm not sure about is the priority of the mangle rules, like should ACK be marked before UDP/ICMP/QUIC or what? Should mark-packet http_big come first before mark-parket HTTP?
Maybe you know something about the mangle rules priority.


I also added established/related connection state when I discovered some connections would escape the mangel rules via connections tab. DoH traffic is also marked via address-list, DNS over UDP 443 is marked especially for Google Chrome as it uses that port for DNS queries sometimes, something I caught from the connections tab.

Here's mine:
add action=mark-connection chain=prerouting comment=DNS connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    src-address-list=""
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=DoH connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment="DoH (UDP 443)" connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=VOIP passthrough=yes port=\
    5060-5062,10000-15251,15253-20000,3478-3497,16384-16387,16393-16402,19302-19309 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no tcp-flags=""
add action=mark-connection chain=prerouting comment=QUIC connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=HTTP \
    passthrough=yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-300M connection-state=established,related,new dst-address-list=!not_in_internet \
    new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=OTHER packet-mark=no-mark \
    passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER packet-mark=no-mark passthrough=no

I've taken advantage of the 5 second bandwidth burst from my ISP that maxed out at 300M that I've seen so far. So I added burst for everything. I capped individual traffic types to 130M to prevent bufferbloat. That extra 300M for parent queue gives it that extra headroom.

The queue tree:
/queue tree
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=DOWN parent=bridge queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS" packet-mark=DNS parent=DOWN priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP" packet-mark=VOIP parent=DOWN priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC" packet-mark=QUIC parent=DOWN priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER" packet-mark=OTHER parent=DOWN queue=redCustom
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=UP parent=pppoe-out1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS_" packet-mark=DNS parent=UP priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP_" packet-mark=VOIP parent=UP priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC_" packet-mark=QUIC parent=UP priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER_" packet-mark=OTHER parent=UP queue=redCustom
 
ivicask
Member
Member
Posts: 438
Joined: Tue Jul 07, 2015 2:40 pm
Location: Croatia, Zagreb

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jul 22, 2020 7:18 pm

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.
The reason why he marked connections instead of packets is to reduce load on the CPU.

I've encountered these anamolies you described as well. I've made a lot of modifications to the mangle rules since though.

One thing I'm not sure about is the priority of the mangle rules, like should ACK be marked before UDP/ICMP/QUIC or what? Should mark-packet http_big come first before mark-parket HTTP?
Maybe you know something about the mangle rules priority.


I also added established/related connection state when I discovered some connections would escape the mangel rules via connections tab. DoH traffic is also marked via address-list, DNS over UDP 443 is marked especially for Google Chrome as it uses that port for DNS queries sometimes, something I caught from the connections tab.

Here's mine:
add action=mark-connection chain=prerouting comment=DNS connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    src-address-list=""
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=DoH connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment="DoH (UDP 443)" connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=VOIP passthrough=yes port=\
    5060-5062,10000-15251,15253-20000,3478-3497,16384-16387,16393-16402,19302-19309 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no tcp-flags=""
add action=mark-connection chain=prerouting comment=QUIC connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=HTTP \
    passthrough=yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-300M connection-state=established,related,new dst-address-list=!not_in_internet \
    new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=OTHER packet-mark=no-mark \
    passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER packet-mark=no-mark passthrough=no

I've taken advantage of the 5 second bandwidth burst from my ISP that maxed out at 300M that I've seen so far. So I added burst for everything. I capped individual traffic types to 130M to prevent bufferbloat. That extra 300M for parent queue gives it that extra headroom.

The queue tree:
/queue tree
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=DOWN parent=bridge queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS" packet-mark=DNS parent=DOWN priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP" packet-mark=VOIP parent=DOWN priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC" packet-mark=QUIC parent=DOWN priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER" packet-mark=OTHER parent=DOWN queue=redCustom
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=UP parent=pppoe-out1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS_" packet-mark=DNS parent=UP priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP_" packet-mark=VOIP parent=UP priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC_" packet-mark=QUIC parent=UP priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER_" packet-mark=OTHER parent=UP queue=redCustom
I know reason is CPU load, but as i said this causes the entire connection (speedtest bandwidth for example) getting captured by ACK mangle rule and its queue tree and never reaching other rules which where supposed to actually capture it.

Im constantly changing and adapting my rules as i notice problems so not completely sure, but I prio ICMP and DNS above all, than ACK packets than rest..

About http packets, i mark big ones first than small ones, and have separate rule for SSL (443) and HTTP(80+8080) traffic (both small + big) while giving one step bigger priority to SSL traffic over HTTP in queue.

And yeah, if your ISP doesnt support big bursting same as i did in my above screenshot, you need to sacrifice some bandwidth, best is to limit high traffic consumers in one group and drop their total speed around 10-15% below total internet speed so they can never saturate it, while leaving other full bandwidth to other "bursty" queues so they have time to drop this big traffic consumer queues via their "limit-at" limits down if you get what i mean.

BTW there is one more trick im testing for some times, what do you think of some basic QOS in NAT it self, i use this on locations where there is very variable speed LTE links and i cant make proper QUEUE tree.
/ip firewall nat
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=icmp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=53 protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=443 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=80 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN
As nat is processed in order, this gives sort of packet priority which packets will be nated first in chain.
For example with this rules u wont notice such big slowdowns if someone is downloading torrents, or some other bandwidth hungry protocols, pages will still open snappy without any other mangle/queue rules.

Opinions?
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jul 22, 2020 7:35 pm

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.
The reason why he marked connections instead of packets is to reduce load on the CPU.

I've encountered these anamolies you described as well. I've made a lot of modifications to the mangle rules since though.

One thing I'm not sure about is the priority of the mangle rules, like should ACK be marked before UDP/ICMP/QUIC or what? Should mark-packet http_big come first before mark-parket HTTP?
Maybe you know something about the mangle rules priority.


I also added established/related connection state when I discovered some connections would escape the mangel rules via connections tab. DoH traffic is also marked via address-list, DNS over UDP 443 is marked especially for Google Chrome as it uses that port for DNS queries sometimes, something I caught from the connections tab.

Here's mine:
add action=mark-connection chain=prerouting comment=DNS connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    src-address-list=""
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=DoH connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment="DoH (UDP 443)" connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=VOIP passthrough=yes port=\
    5060-5062,10000-15251,15253-20000,3478-3497,16384-16387,16393-16402,19302-19309 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no tcp-flags=""
add action=mark-connection chain=prerouting comment=QUIC connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=HTTP \
    passthrough=yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-300M connection-state=established,related,new dst-address-list=!not_in_internet \
    new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=OTHER packet-mark=no-mark \
    passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER packet-mark=no-mark passthrough=no

I've taken advantage of the 5 second bandwidth burst from my ISP that maxed out at 300M that I've seen so far. So I added burst for everything. I capped individual traffic types to 130M to prevent bufferbloat. That extra 300M for parent queue gives it that extra headroom.

The queue tree:
/queue tree
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=DOWN parent=bridge queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS" packet-mark=DNS parent=DOWN priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP" packet-mark=VOIP parent=DOWN priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC" packet-mark=QUIC parent=DOWN priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER" packet-mark=OTHER parent=DOWN queue=redCustom
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=UP parent=pppoe-out1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS_" packet-mark=DNS parent=UP priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP_" packet-mark=VOIP parent=UP priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC_" packet-mark=QUIC parent=UP priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER_" packet-mark=OTHER parent=UP queue=redCustom
I know reason is CPU load, but as i said this causes the entire connection (speedtest bandwidth for example) getting captured by ACK mangle rule and its queue tree and never reaching other rules which where supposed to actually capture it.

Im constantly changing and adapting my rules as i notice problems so not completely sure, but I prio ICMP and DNS above all, than ACK packets than rest..

About http packets, i mark big ones first than small ones, and have separate rule for SSL (443) and HTTP(80+8080) traffic (both small + big) while giving one step bigger priority to SSL traffic over HTTP in queue.

And yeah, if your ISP doesnt support big bursting same as i did in my above screenshot, you need to sacrifice some bandwidth, best is to limit high traffic consumers in one group and drop their total speed around 10-15% below total internet speed so they can never saturate it, while leaving other full bandwidth to other "bursty" queues so they have time to drop this big consumer queues via their limit-at limits down if you get what i mean.

BTW there is one more trick im testing for some times, what do you think of some basic QOS in NAT it self, i use this on locations where there is very variable speed LTE links and i cant make proper QUEUE tree.
/ip firewall nat
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=icmp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=53 protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=443 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=80 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN
As nat is processed in order, this gives sort of packet priority which packets will be nated first in chain.
For example with this rules u wont notice such big slowdowns if someone is downloading torrents, or some other bandwidth hungry protocols, pages will still open snappy without any other mangle/queue rules.

Opinions?
Yeah, I am still confused about the order of the mangle rules.

I think pcunite needs to look into out input on the issues from over the past few months and give us his opinion on this things.

NAT QoS, is something I haven't thought of before, but it could be problematic if the ISP is using dynamic IPs.

Also I have IPv6 connectivity from my ISP, NAT QoS would limit it only for IPv4 traffic and therefore make it inefficient. In my opinion I suggest you stick with mangles/other mechanisms that would work for both IPv4/IPv6 smoothly.

QoS is such a pain...
 
ivicask
Member
Member
Posts: 438
Joined: Tue Jul 07, 2015 2:40 pm
Location: Croatia, Zagreb

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jul 22, 2020 7:45 pm

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.
The reason why he marked connections instead of packets is to reduce load on the CPU.

I've encountered these anamolies you described as well. I've made a lot of modifications to the mangle rules since though.

One thing I'm not sure about is the priority of the mangle rules, like should ACK be marked before UDP/ICMP/QUIC or what? Should mark-packet http_big come first before mark-parket HTTP?
Maybe you know something about the mangle rules priority.


I also added established/related connection state when I discovered some connections would escape the mangel rules via connections tab. DoH traffic is also marked via address-list, DNS over UDP 443 is marked especially for Google Chrome as it uses that port for DNS queries sometimes, something I caught from the connections tab.

Here's mine:
add action=mark-connection chain=prerouting comment=DNS connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    src-address-list=""
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=DoH connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment="DoH (UDP 443)" connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=VOIP passthrough=yes port=\
    5060-5062,10000-15251,15253-20000,3478-3497,16384-16387,16393-16402,19302-19309 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no tcp-flags=""
add action=mark-connection chain=prerouting comment=QUIC connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=HTTP \
    passthrough=yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-300M connection-state=established,related,new dst-address-list=!not_in_internet \
    new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=OTHER packet-mark=no-mark \
    passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER packet-mark=no-mark passthrough=no

I've taken advantage of the 5 second bandwidth burst from my ISP that maxed out at 300M that I've seen so far. So I added burst for everything. I capped individual traffic types to 130M to prevent bufferbloat. That extra 300M for parent queue gives it that extra headroom.

The queue tree:
/queue tree
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=DOWN parent=bridge queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS" packet-mark=DNS parent=DOWN priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP" packet-mark=VOIP parent=DOWN priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC" packet-mark=QUIC parent=DOWN priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER" packet-mark=OTHER parent=DOWN queue=redCustom
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=UP parent=pppoe-out1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS_" packet-mark=DNS parent=UP priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP_" packet-mark=VOIP parent=UP priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC_" packet-mark=QUIC parent=UP priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER_" packet-mark=OTHER parent=UP queue=redCustom
I know reason is CPU load, but as i said this causes the entire connection (speedtest bandwidth for example) getting captured by ACK mangle rule and its queue tree and never reaching other rules which where supposed to actually capture it.

Im constantly changing and adapting my rules as i notice problems so not completely sure, but I prio ICMP and DNS above all, than ACK packets than rest..

About http packets, i mark big ones first than small ones, and have separate rule for SSL (443) and HTTP(80+8080) traffic (both small + big) while giving one step bigger priority to SSL traffic over HTTP in queue.

And yeah, if your ISP doesnt support big bursting same as i did in my above screenshot, you need to sacrifice some bandwidth, best is to limit high traffic consumers in one group and drop their total speed around 10-15% below total internet speed so they can never saturate it, while leaving other full bandwidth to other "bursty" queues so they have time to drop this big consumer queues via their limit-at limits down if you get what i mean.

BTW there is one more trick im testing for some times, what do you think of some basic QOS in NAT it self, i use this on locations where there is very variable speed LTE links and i cant make proper QUEUE tree.
/ip firewall nat
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=icmp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=53 protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=443 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=80 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN
As nat is processed in order, this gives sort of packet priority which packets will be nated first in chain.
For example with this rules u wont notice such big slowdowns if someone is downloading torrents, or some other bandwidth hungry protocols, pages will still open snappy without any other mangle/queue rules.

Opinions?
Yeah, I am still confused about the order of the mangle rules.

I think pcunite needs to look into out input on the issues from over the past few months and give us his opinion on this things.

NAT QoS, is something I haven't thought of before, but it could be problematic if the ISP is using dynamic IPs.

Also I have IPv6 connectivity from my ISP, NAT QoS would limit it only for IPv4 traffic and therefore make it inefficient. In my opinion I suggest you stick with mangles/other mechanisms that would work for both IPv4/IPv6 smoothly.

QoS is such a pain...
No provider supports IPV6 in Croatia :)

You are using IPV6 in local network also ? There is no nat involved in router?If yes, than my rules would still work regardless as long your router is NAT-ing something at some point.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jul 23, 2020 12:20 am

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.
The reason why he marked connections instead of packets is to reduce load on the CPU.

I've encountered these anamolies you described as well. I've made a lot of modifications to the mangle rules since though.

One thing I'm not sure about is the priority of the mangle rules, like should ACK be marked before UDP/ICMP/QUIC or what? Should mark-packet http_big come first before mark-parket HTTP?
Maybe you know something about the mangle rules priority.


I also added established/related connection state when I discovered some connections would escape the mangel rules via connections tab. DoH traffic is also marked via address-list, DNS over UDP 443 is marked especially for Google Chrome as it uses that port for DNS queries sometimes, something I caught from the connections tab.

Here's mine:
add action=mark-connection chain=prerouting comment=DNS connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    tcp-flags=""
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=DNS passthrough=yes port=53 protocol=udp \
    src-address-list=""
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=DoH connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=53,443,853 protocol=tcp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment="DoH (UDP 443)" connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=established,related,new dst-address-list=DoH new-connection-mark=DNS passthrough=yes port=443 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=VOIP passthrough=yes port=\
    5060-5062,10000-15251,15253-20000,3478-3497,16384-16387,16393-16402,19302-19309 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no tcp-flags=""
add action=mark-connection chain=prerouting comment=QUIC connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new dst-address-list=!not_in_internet new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=HTTP \
    passthrough=yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-300M connection-state=established,related,new dst-address-list=!not_in_internet \
    new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=OTHER packet-mark=no-mark \
    passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER packet-mark=no-mark passthrough=no

I've taken advantage of the 5 second bandwidth burst from my ISP that maxed out at 300M that I've seen so far. So I added burst for everything. I capped individual traffic types to 130M to prevent bufferbloat. That extra 300M for parent queue gives it that extra headroom.

The queue tree:
/queue tree
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=DOWN parent=bridge queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS" packet-mark=DNS parent=DOWN priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP" packet-mark=VOIP parent=DOWN priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC" packet-mark=QUIC parent=DOWN priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER" packet-mark=OTHER parent=DOWN queue=redCustom
add bucket-size=0.01 burst-limit=300M burst-threshold=300M burst-time=1s max-limit=99M name=UP parent=pppoe-out1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="1. DNS_" packet-mark=DNS parent=UP priority=1 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=2M max-limit=99M name="2. VOIP_" packet-mark=VOIP parent=UP priority=2 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=18M max-limit=99M name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="7. QUIC_" packet-mark=QUIC parent=UP priority=6 queue=default
add burst-limit=130M burst-threshold=130M burst-time=1s limit-at=10M max-limit=99M name="8. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=7 queue=redCustom
add burst-limit=130M burst-threshold=130M burst-time=1s name="9. OTHER_" packet-mark=OTHER parent=UP queue=redCustom
I know reason is CPU load, but as i said this causes the entire connection (speedtest bandwidth for example) getting captured by ACK mangle rule and its queue tree and never reaching other rules which where supposed to actually capture it.

Im constantly changing and adapting my rules as i notice problems so not completely sure, but I prio ICMP and DNS above all, than ACK packets than rest..

About http packets, i mark big ones first than small ones, and have separate rule for SSL (443) and HTTP(80+8080) traffic (both small + big) while giving one step bigger priority to SSL traffic over HTTP in queue.

And yeah, if your ISP doesnt support big bursting same as i did in my above screenshot, you need to sacrifice some bandwidth, best is to limit high traffic consumers in one group and drop their total speed around 10-15% below total internet speed so they can never saturate it, while leaving other full bandwidth to other "bursty" queues so they have time to drop this big consumer queues via their limit-at limits down if you get what i mean.

BTW there is one more trick im testing for some times, what do you think of some basic QOS in NAT it self, i use this on locations where there is very variable speed LTE links and i cant make proper QUEUE tree.
/ip firewall nat
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=icmp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=53 protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=443 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN port=80 protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=tcp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN protocol=udp
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface-list=WAN
As nat is processed in order, this gives sort of packet priority which packets will be nated first in chain.
For example with this rules u wont notice such big slowdowns if someone is downloading torrents, or some other bandwidth hungry protocols, pages will still open snappy without any other mangle/queue rules.

Opinions?
Yeah, I am still confused about the order of the mangle rules.

I think pcunite needs to look into out input on the issues from over the past few months and give us his opinion on this things.

NAT QoS, is something I haven't thought of before, but it could be problematic if the ISP is using dynamic IPs.

Also I have IPv6 connectivity from my ISP, NAT QoS would limit it only for IPv4 traffic and therefore make it inefficient. In my opinion I suggest you stick with mangles/other mechanisms that would work for both IPv4/IPv6 smoothly.

QoS is such a pain...
No provider supports IPV6 in Croatia :)

You are using IPV6 in local network also ? There is no nat involved in router?If yes, than my rules would still work regardless as long your router is NAT-ing something at some point.
Your rules are only limited to IPv4 traffic. NAT is for IPv4. There's no NAT for IPv6 except something like NAT64 which is something I wished didn't exist to begin with. I'm anti-NAT anyway. The whole point of IPv6 was to bring back end-to-end principle.

I have native IPv6 connectivity from my ISP. My client devices each gets two publicly routable IPv6. I had to create separate mangle rules for v4 and v6.
 
srcrosset
just joined
Posts: 1
Joined: Mon Feb 03, 2020 3:47 am

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Jul 24, 2020 10:50 pm

@pcunite,
I followed your VLAN tutorial and set up router-on-a-stick which has been working great.
Now I am looking through your QoS tutorial and would like to know how this applied to the VLAN setup.
Would the LAN interface be the bridge?
Would this QoS setup only apply to traffic going to and from the WAN (ether1) port and not affect internal inter-LAN traffic
 
haj3s29a
newbie
Posts: 30
Joined: Sun Jul 05, 2020 5:02 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jul 26, 2020 12:27 am

/ip firewall mangle add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-300M connection-state=established,related,new dst-address-list=!not_in_internet \
    new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp

connection-bytes=5000000-0 doesn't work in RouterOS v7.1beta1 - it is possible to setup only 0-xxxxx.

Can someone confirm whether it works in stable version or not, please?

Sadly, documentation is outdated and I didn't find any further info.
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jul 26, 2020 6:49 pm

/ip firewall mangle add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-300M connection-state=established,related,new dst-address-list=!not_in_internet \
    new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp

connection-bytes=5000000-0 doesn't work in RouterOS v7.1beta1 - it is possible to setup only 0-xxxxx.

Can someone confirm whether it works in stable version or not, please?

Sadly, documentation is outdated and I didn't find any further info.
try this....connection-bytes=!0-5000000
 
haj3s29a
newbie
Posts: 30
Joined: Sun Jul 05, 2020 5:02 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jul 26, 2020 7:25 pm

try this....connection-bytes=!0-5000000
not possible to use "!" in connection-bytes. I have tried connection-bytes=5M-100M and it seems to work

UPDATE:

5M-0 seems to work over CLI but WebUI considers it as an error.
 
User avatar
osc86
Member Candidate
Member Candidate
Posts: 203
Joined: Wed Aug 09, 2017 1:15 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Jul 27, 2020 9:14 pm

Any idea how a mangle setup with policy routing (mark routing) and connection / packet marking for qos could look like?
Both need "stop processing" in prerouting chain.
I experimented a bit with my existing rules, but I didn't get both working at the same time. Maybe I have to use a new chain and some jump & return rules.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jul 29, 2020 3:31 pm

@pcunite,
I followed your VLAN tutorial and set up router-on-a-stick which has been working great.
Now I am looking through your QoS tutorial and would like to know how this applied to the VLAN setup.
Would the LAN interface be the bridge?
Would this QoS setup only apply to traffic going to and from the WAN (ether1) port and not affect internal inter-LAN traffic
The QoS model pcunite gave will mark LAN-to-LAN traffic as well and queue them. A solution I came up with is to use dst-address-list=!not_in_internet, where not_in_internet is a list of LAN subnets.

Something like this:
/ip firewall address-list
add address=0.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=172.16.0.0/12 comment=RFC6890 list=not_in_internet
add address=192.168.0.0/16 comment=RFC6890 list=not_in_internet
add address=10.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=169.254.0.0/16 comment=RFC6890 list=not_in_internet
add address=127.0.0.0/8 comment=RFC6890 list=not_in_internet
add address=224.0.0.0/4 comment=Multicast list=not_in_internet
add address=198.18.0.0/15 comment=RFC6890 list=not_in_internet
add address=192.0.0.0/24 comment=RFC6890 list=not_in_internet
add address=192.0.2.0/24 comment=RFC6890 list=not_in_internet
add address=198.51.100.0/24 comment=RFC6890 list=not_in_internet
add address=203.0.113.0/24 comment=RFC6890 list=not_in_internet
add address=100.64.0.0/10 comment=RFC6890 list=not_in_internet
add address=240.0.0.0/4 comment=RFC6890 list=not_in_internet
add address=192.88.99.0/24 comment="6to4 relay Anycast [RFC 3068]" list=not_in_internet
You apply it to all the "prerouting, mark connection" mangle rules
add action=mark-connection chain=prerouting comment=QUIC connection-state=established,related,new dst-address-list=!not_in_internet new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jul 29, 2020 3:34 pm

Any idea how a mangle setup with policy routing (mark routing) and connection / packet marking for qos could look like?
Both need "stop processing" in prerouting chain.
I experimented a bit with my existing rules, but I didn't get both working at the same time. Maybe I have to use a new chain and some jump & return rules.
I've been thinking about dual-WAN load balancing with this QoS model.

The mangle rules will mark all traffic regardless of the interface/direction by default, which simplifies the marking aspect.

But how do we apply separate queue trees to WAN1 (up/down) and WAN2 (up/down)?
 
TDJ211
just joined
Posts: 7
Joined: Mon Jul 01, 2019 2:30 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Aug 06, 2020 9:14 pm

I've been thinking about dual-WAN load balancing with this QoS model.

The mangle rules will mark all traffic regardless of the interface/direction by default, which simplifies the marking aspect.

But how do we apply separate queue trees to WAN1 (up/down) and WAN2 (up/down)?
Im a bit green so just throwing this out there to possibly spark some ideas

To answer your question, the only way ive found to QOS a dual wan setup is to queue only the UP of each WAN connection using simple queues.

At work we have a 50/50 dedicated fiber and a 200/20 DOCSIS connection. Im doing a dual WAN (no load balancing but there is failover) PBR routing. The dedicated fiber circuit is primarily used for live video streaming and VOIP, although I do have a little bit of regular traffic being routed through it (hence the need for QOS), and the DOCSIS WAN is just serving bulk data traffic. Since the mangles are being used to do the dual wan routing based on source IP/VLAN/subnets, I created simple queues for the fiber WAN connection based on source as well as thats the one that where we would really need QOS.

I havent needed/wanted to QOS the DOCSIS WAN as we have plenty bandwidth for our needs.

/queue simple
add max-limit=48M/48M name=ATT-queue queue=default/default target="eth5-m
    v33-VOIP,v44-LTN,v55-Radio-net,172.17.2.45/32,172.17.1.39/32,172.17.2.246/32,172.17.1.250/32"
add limit-at=16M/16M max-limit=48M/48M name=LTN parent=ATT-queue priority
    1/1 queue=default/default target=v44-LTN
add limit-at=3M/3M max-limit=48M/48M name=VOIP-only parent=ATT-queue \
    priority=2/2 queue=default/default target=\
    v33-VOIP,172.17.2.45/32,172.17.1.39/32    
add limit-at=2M/2M max-limit=48M/48M name=Radio-net parent=ATT-qu
    priority=3/3 queue=default/default target=v55-Radio-net
add limit-at=20M/20M max-limit=48M/48M name=VOIP-plus-data parent=ATT-que
    priority=4/4 queue=default/default target=\
    172.17.2.246/32,172.17.1.250/32
add max-limit=35M/35M name=other parent=ATT-queue priority=6/6 queue=\
    pcq-upload-default/pcq-download-default target=\
    eth5-mgmt,v88-test,eth3-pfSense
 
TDJ211
just joined
Posts: 7
Joined: Mon Jul 01, 2019 2:30 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Aug 11, 2020 10:41 am

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.

Yea, I hear people all the time stress that you have to conserve as much CPU as possible, but all my Mikrotiks in the field barely crack 2-3% CPU utilization at peak usage. So whats the point of saving all this CPU if youre not ever gonna use it? I get if you're actually approaching CPU limit, but if not, why let all that CPU power go to waste?

Ive heard a few others mention, like you describe, that sometimes packets escape the mangle and dont get marked properly. I believe the same thing happens with Fasttrack which is why its recommended to keep the same rule as the fasttrack right below it.

Anyways, on to my question. Can someone give me an example of marking all packets as opposed to just the first packet of the connection? After writing this out, I think I know the answer, but just want to confirm.

So instead of:

add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no


It would simply be:

add action=mark-packet chain=prerouting new-packet-mark=DNS passthrough=no port=53 protocol=udp 
add action=mark-packet chain=postrouting new-packet-mark=DNS passthrough=no port=53 protocol=udp

???
 
Acltmac
just joined
Posts: 7
Joined: Mon Jun 29, 2020 5:43 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Aug 11, 2020 11:47 am

Are you sure about marking DNS / ICMP or ACK connections than their packet marks, because what im seeing that when i run speedtest some or sometimes entire bandwidth goes thru this QUEUE(DNS/ICMP/ACK) and as they have top prio actually choke my net.

I changed it to only mark packets directly without prior connection marking and now works fine, only this packets pass thru.(rest of mangle rules still uses connection +packet marking)

I tried absolutely destroying my net with few torrents, http downloads, windows updates, some youtube, radio stream(doesnt break), dropbox upload, and ping is still stable without drops, and web pages opens super fast.

This is how my QOS looks now that im using for some time now.
Can you upload your scripts?

Sent from my SM-G965F using Tapatalk

 
haj3s29a
newbie
Posts: 30
Joined: Sun Jul 05, 2020 5:02 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Aug 11, 2020 12:35 pm

Change-dscp and mangle packets...30+ % CPU usage on Chateau LTE12.

It is unstable and unreliable though
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Aug 17, 2020 4:35 pm

I've been thinking about dual-WAN load balancing with this QoS model.

The mangle rules will mark all traffic regardless of the interface/direction by default, which simplifies the marking aspect.

But how do we apply separate queue trees to WAN1 (up/down) and WAN2 (up/down)?
Im a bit green so just throwing this out there to possibly spark some ideas

To answer your question, the only way ive found to QOS a dual wan setup is to queue only the UP of each WAN connection using simple queues.

At work we have a 50/50 dedicated fiber and a 200/20 DOCSIS connection. Im doing a dual WAN (no load balancing but there is failover) PBR routing. The dedicated fiber circuit is primarily used for live video streaming and VOIP, although I do have a little bit of regular traffic being routed through it (hence the need for QOS), and the DOCSIS WAN is just serving bulk data traffic. Since the mangles are being used to do the dual wan routing based on source IP/VLAN/subnets, I created simple queues for the fiber WAN connection based on source as well as thats the one that where we would really need QOS.

I havent needed/wanted to QOS the DOCSIS WAN as we have plenty bandwidth for our needs.

/queue simple
add max-limit=48M/48M name=ATT-queue queue=default/default target="eth5-m
    v33-VOIP,v44-LTN,v55-Radio-net,172.17.2.45/32,172.17.1.39/32,172.17.2.246/32,172.17.1.250/32"
add limit-at=16M/16M max-limit=48M/48M name=LTN parent=ATT-queue priority
    1/1 queue=default/default target=v44-LTN
add limit-at=3M/3M max-limit=48M/48M name=VOIP-only parent=ATT-queue \
    priority=2/2 queue=default/default target=\
    v33-VOIP,172.17.2.45/32,172.17.1.39/32    
add limit-at=2M/2M max-limit=48M/48M name=Radio-net parent=ATT-qu
    priority=3/3 queue=default/default target=v55-Radio-net
add limit-at=20M/20M max-limit=48M/48M name=VOIP-plus-data parent=ATT-que
    priority=4/4 queue=default/default target=\
    172.17.2.246/32,172.17.1.250/32
add max-limit=35M/35M name=other parent=ATT-queue priority=6/6 queue=\
    pcq-upload-default/pcq-download-default target=\
    eth5-mgmt,v88-test,eth3-pfSense
Yeah I dropped QoS entirely from my setup. I went with two ISPs.
ISP 1 = GPON, 100Mbps symmetrical bandwidth, burst up to 400Mbps for 2-5 seconds on each initial download/upload.
ISP 2 = EPON, 50Mbps symmetrical bandwidth, consistent bandwidth.

I use PCC load balancing and I am able to get aggregated bandwdith. HTTPS does not break, Banking works, gaming works, streaming works. Tested for nearly a month now.
 
oldunixguy
just joined
Posts: 9
Joined: Wed Sep 06, 2017 10:27 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Aug 18, 2020 8:45 am

I've seen some weird packet issues when using RED for the defaults. So, I've gone back to SFQ for default but use RED for the bulky flows. The behavior of RED as default causes the VoIP queue to drop a single packet here and there when it does not appear to be under pressure. Perhaps there is something else going on, but I never want a packet to drop there, so SFQ (or probably any other type) prevents that behavior.
This entry appears to be the last from pcunite... I have been wading thru this thread since the 2013 beginning looking for a "final" recommended way to provide the QoS to make a couple of VOIP phones to work. But that appears not to be. And the thread wanders a good bit. One person suggested the procedure we need be put on the wiki. I just looked that over again and saw nothing. I have had a RB450Gx4 since June 2018 and since that time keep coming back here to find a procedure recommend by the RouterOS experts. It seems to me that the VOIP phone QoS would be such a common need that the experts would put their heads together and give us a couple of well-thought, well-tested out templates. It seems pcunite was on track to do this but there is still no forum entry that sums up all the machinations. There also appears here conflicting input. I find conflicts with here on various other RouterOS experts elsewhere. I was hoping that the 2 years I have needed this VOIP QoS that there would be some "expert" convergence on what we should use. I want something reasonably basic to implement QoS for 2 VOIP phones that optimally uses the RouterOS features with typical other services such as DNS, icmp, http/s, etc. Give us the rules and arithmetic to use with this "expert's template" and we can tweak the config for our VOIP phone counts and line speeds. Thanks to all the contributors but we need convergence on this topic.
thanks
oldunixguy
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Aug 28, 2020 11:12 pm

I have been wading thru this thread since the 2013 beginning looking for a "final" recommended way to provide the QoS to make a couple of VOIP phones to work. But that appears not to be.

My configuration, as shown, is what you are looking for. It works. I use it on multiple networks. Don't worry about all the chatter. That's just me and other folks looking for the last bit of adjustment. They'll always be better ways of doing things for your exact network. But the main post at the top is what you should start with. If you have trouble, you can ask questions.
 
blurrybird
newbie
Posts: 38
Joined: Sun Jan 19, 2020 12:25 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Aug 29, 2020 10:38 am

@pcunite thanks for doing this. I've noticed a few things that I'd like your input on.

1. I'm trying to work out why you are detecting VoIP by just blanket accepting 10,000+ ports. Often these ports are also used for gaming etc. Is this intentional or by mistake? I'd like to see a variant made where you're making use of DSCP marking or some other method to detect this more accurately.

2. When I deploy this for an Aussie NBN connection (100/40) as-is, I get complaints about "the internet is slow" when I'm performing a large sequential download (Steam, direct, Speedtest, etc) and others are doing general browsing. Is there any changes that can be made to mitigate this effect? It does not seem to occur when all queues are disabled. This same thing happens when using the modified ones by bharrisau, and same experience across two different sites.

Cheers
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Aug 29, 2020 3:30 pm

@pcunite thanks for doing this. I've noticed a few things that I'd like your input on.

1. I'm trying to work out why you are detecting VoIP by just blanket accepting 10,000+ ports. Often these ports are also used for gaming etc. Is this intentional or by mistake? I'd like to see a variant made where you're making use of DSCP marking or some other method to detect this more accurately.

2. When I deploy this for an Aussie NBN connection (100/40) as-is, I get complaints about "the internet is slow" when I'm performing a large sequential download (Steam, direct, Speedtest, etc) and others are doing general browsing. Is there any changes that can be made to mitigate this effect? It does not seem to occur when all queues are disabled. This same thing happens when using the modified ones by bharrisau, and same experience across two different sites.

Cheers
The "slow down" of the "heavy" downloads is expected behaviour, that's the whole point of the QoS. To ensure each traffic type or groups of users get a fair share of the obviously limited bandwidth.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Aug 29, 2020 6:07 pm

@pcunite thanks for doing this. I've noticed a few things that I'd like your input on.

@blurrybird,

>>why you are detecting VoIP by just blanket accepting 10,000+ ports?
The original article was created a long time ago. The VoIP equipment I used at the time used those range of ports. I think it is still a standard practice with certain vendors. Yes, it should not be so wide. You only need two ports available for every active VoIP call you expect. So, based on the number of phones you have, on the PBX, you should make available the appropriate port count.

However, the article is not about how to identify all types of traffic. I don't ever want to get into that because port 123xyz means something different to each network. If you have a PBX server, and the phones are on a VLAN, you only care about the PBX traffic as it crosses the router boundary. So, to be clear, I do not care to write a massive article on how to identify traffic types. Therefore, I should clean up this section and mention that.

>> the internet feels slow to casual users when there is a large sequential download in progress. Can this be mitigated?
Yes. Before I explain. The downside to example configurations is all the side effects. You have to be an expert to use any of these examples. I had hoped experts would chime in, but they have not. Probably because information with MikroTik is actually an invoice line item. And that's fine, nothing wrong with that. I'm not in that market, so I gave my information away. Focus on the big picture of what QoS is doing and this type of event will be more clear. When bandwidth is given to one area, it no longer exists in another.

If you choose to QoS with MikroTik hardware, you must throw as much CPU power at it as you can. Don't expect much if not using an RB4011 or faster. The total number of active connections can overwhelm the context switching of the CPU. If every connection is given 50ms of time to do something, and you only have, say 4 cores doing this, 500 connections are going to overwhelm.

A mitigation suggestion:
Disclaimer: I don't know the specifics of your network, so this may not help. I'm not available to assist you further.
You must know how many active casual sessions there will be. You create a Queue type for them (or their traffic type). Then you use Limit-at to ensure they have that much set aside when contention starts. RouterOS provides some automation for this via PCQ. So, you must have carve outs for tier A, and then another for tier B, then one for tier C, and so on. You must identify large bandwidth users, put them in a queue of their own, and handle them appropriately. If you don't know who they are (or the traffic type) and amounts then that is your first problem. If you do and you don't think its workable, then that's your second challenge. When you overcome these two barriers, you will implement coping strategies. I'm assuming you can not buy more bandwidth. Someone in this thread mentioned a second ISP connection. That's more bandwidth. Bandwidth will fix everything. Otherwise somebody, somewhere is going to be limited. You decide who that is. Everyone is not going to get 100mbps (unless you have that much to give).

Study the examples. Read them slowly. The information is all there. After you cross reference with the MikroTik manual, it might take about three months to understand it.
 
biiiipy
just joined
Posts: 1
Joined: Thu Nov 12, 2020 9:14 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Tue Nov 24, 2020 10:17 pm

# UP
add name=UP max-limit=90M parent=WAN bucket-size=0.01 queue=default

add name="1. VOIP_"     packet-mark=VOIP     parent=UP priority=1 queue=default
add name="2. DNS_"      packet-mark=DNS      parent=UP priority=2 queue=default
add name="3. ACK_"      packet-mark=ACK      parent=UP priority=3 queue=default
add name="4. UDP_"      packet-mark=UDP      parent=UP priority=3 queue=default
add name="5. ICMP_"     packet-mark=ICMP     parent=UP priority=4 queue=default
add name="6. HTTP_"     packet-mark=HTTP     parent=UP priority=5 queue=default
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="8. QUIC_"     packet-mark=QUIC     parent=UP priority=7 queue=default
add name="9. OTHER_"    packet-mark=OTHER    parent=UP priority=8 queue=default

When I try to add upload queues to LTE1 interface, my router goes into crash loop. Do LTE routers need special config for queues?

add name=UP max-limit=90M parent=lte1 bucket-size=0.01 queue=default

Also, as somebody else asked, I assume there's no real QoS solution for LTE-like connections that have variable bandwidth?
Last edited by biiiipy on Wed Nov 25, 2020 8:18 pm, edited 2 times in total.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Dec 24, 2020 4:54 pm

What values do you guys use for default SFQ here?
Image
 
complex1
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Wed Jan 04, 2017 9:55 pm
Location: NL-NH

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Dec 24, 2020 5:33 pm

Same as yours:
5 sec and 1514 bytes
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Dec 24, 2020 5:46 pm

Same as yours:
5 sec and 1514 bytes
I wonder if those values need tweaking to give a potentially better outcome.
 
JJT211
Frequent Visitor
Frequent Visitor
Posts: 62
Joined: Sun Apr 28, 2019 9:01 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Feb 21, 2021 4:39 am

Same as yours:
5 sec and 1514 bytes
I wonder if those values need tweaking to give a potentially better outcome.
Yea, I was wondering the same. It seems the consensus in other places than this thread, is to use perturb=10 in order to mimic CAKE/FQ-CODEL as much as possible.
 
haris013
Frequent Visitor
Frequent Visitor
Posts: 82
Joined: Mon Jun 18, 2018 2:13 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jun 22, 2021 4:04 pm

Hello, nice guide, just a question as a newbie, why on down tree we use as parent the LAN (bridge) interface and on UP tree we use the WAN interface? If both down and up tree is WAN, what is the difference?
 
jy00022158
just joined
Posts: 2
Joined: Tue Jul 06, 2021 10:21 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jul 06, 2021 10:54 am

Do you have a good double Wan QoS script? It's better to send an email to share it. Thank you dukeng@qq.com
 
ahmedelbarbary
just joined
Posts: 19
Joined: Thu Dec 01, 2016 1:23 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jul 22, 2021 8:13 am

Nice script, i think there is a mistake in the order, ack should come after mark connection of TCP connections not before it, and remove ack postrouting rule
- in my opinion i think TCP must be like this way
-
/ip firewall mangle
add chain=prerouting action=mark-connection connection-mark=!heavy_traffic_conn \
new-connection-mark=all_conn connection-state=new
add chain=prerouting action=mark-connection connection-bytes=5M-0
connection-mark=all_conn connection-rate=2M-100M
new-connection-mark=heavy_traffic_conn protocol=tcp

add action=mark-packet chain=prerouting new-packet-mark="SYN" packet-size=0-666 passthrough=no protocol=tcp \
tcp-flags=syn
add action=mark-packet chain=prerouting new-packet-mark="ACK " packet-size=0-123 passthrough=no protocol=tcp \
tcp-flags=ack

add chain=prerouting action=mark-packet connection-mark=heavy_traffic_conn \
new-packet-mark=heavy_traffic passthrough=no
add chain=prerouting action=mark-packet connection-mark=all_conn \
new-packet-mark=light_traffic passthrough=no
-

I think in this way is better for TCP. ACK and SYN have the same priority, if i had a mistake just tell me and duscuss it .
Thanks
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Sep 09, 2021 8:27 am

I've done some testing with a closer server. Four cases considered: baseline, QOS2020, QOS2020-Optimised (my recommended changes), mikroCAKE (pseudo-CAKE implementation).

Graphs of the bandwidth vs latency, and a latency distribution plot. All tests were Perth to Sydney, 30 seconds of baseline either side of a 120 second test. Local connection is 100/40 Mbps fibre, remote connection is 1000/250 Mbps OVH dedicated server. I've used 90% of the rated line speed locally for the settings.

Configuration exports below

Baseline
/ip firewall mangle
/queue type
/queue tree
add max-limit=36M name=UP packet-mark=no-mark parent=WAN1 queue=default
add max-limit=90M name=DOWN packet-mark=no-mark parent=BR1 queue=default
QOS2020
/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
add action=mark-connection chain=prerouting comment=VOIP new-connection-mark=VOIP passthrough=yes port=5060-5062,10000-20000 protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP new-packet-mark=VOIP passthrough=no
add action=mark-connection chain=prerouting comment=QUIC connection-state=new new-connection-mark=QUIC passthrough=yes port=80,443 \
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no
add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new new-connection-mark=HTTP passthrough=\
    yes port=80,443 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=\
    HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-connection chain=prerouting comment=OTHER connection-state=new new-connection-mark=POP3 passthrough=yes port=995,465,587 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=POP3 new-packet-mark=OTHER passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no

/queue type
add kind=red name=customRED red-avg-packet=1514

/queue tree
add max-limit=36M name=UP parent=WAN1 queue=default
add max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=customRED
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=customRED
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=customRED
add name="9. OTHER" packet-mark=OTHER parent=DOWN queue=default
add name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=customRED
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=customRED
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=customRED
add name="9. OTHER_" packet-mark=OTHER parent=UP queue=default
QOS2020 - Optimised
# Mangle is as above
/queue type
set 0 kind=sfq
add kind=pcq name=bulkUp pcq-classifier=src-address,dst-address,src-port,dst-port pcq-limit=450KiB pcq-total-limit=4500KiB

/queue tree
add bucket-size=0.005 max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. VOIP" packet-mark=VOIP parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=default
add name="9. OTHER" packet-mark=OTHER parent=DOWN priority=8 queue=default
add bucket-size=0.005 max-limit=36M name=UP parent=WAN1 queue=default
add name="1. VOIP_" packet-mark=VOIP parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=bulkUp
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=bulkUp
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=bulkUp
add name="9. OTHER_" packet-mark=OTHER parent=UP priority=8 queue=bulkUp
mikroCAKE
/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=PRIO passthrough=yes port=53 protocol=udp
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=PRIO passthrough=yes port=53 protocol=udp
add action=mark-connection chain=prerouting comment=VOIP new-connection-mark=PRIO passthrough=yes port=5060-5062,10000-20000 protocol=udp
add action=mark-packet chain=postrouting connection-mark=PRIO new-packet-mark=PRIO passthrough=no
add action=mark-packet chain=prerouting connection-mark=PRIO new-packet-mark=PRIO passthrough=no
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=PRIO packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=PRIO packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting comment=Sparse connection-rate=0-1 dst-limit=5,2,addresses-and-dst-port/1m40s new-packet-mark=\
    SPARSE passthrough=no
add action=mark-packet chain=postrouting connection-rate=1-10k new-packet-mark=SPARSE passthrough=no
add action=mark-packet chain=postrouting comment=Bulk new-packet-mark=BULK passthrough=no

/queue type
set 0 kind=sfq
add kind=pcq name=bulkUp pcq-classifier=src-address,dst-address,src-port,dst-port pcq-limit=450KiB pcq-total-limit=4500KiB

/queue tree
add bucket-size=0.005 max-limit=90M name=DOWN parent=BR1 queue=default
add name="1. Priority" packet-mark=PRIO parent=DOWN priority=1 queue=default
add name="2. Sparse" packet-mark=SPARSE parent=DOWN priority=2 queue=default
add name="3. Bulk" packet-mark=BULK parent=DOWN priority=3 queue=default
add bucket-size=0.005 max-limit=36M name=UP parent=WAN1 queue=default
add name="1. Priority_" packet-mark=PRIO parent=UP priority=1 queue=default
add name="2. Sparse_" packet-mark=SPARSE parent=UP priority=2 queue=default
add name="3. Bulk_" packet-mark=BULK parent=UP priority=3 queue=bulkUp
why only apply to upload queues?
 
ultratoto14
just joined
Posts: 2
Joined: Mon May 28, 2018 3:35 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Sep 24, 2021 10:20 am

Did someone try to update this by using cake or codel available and not crashing since 7.1rc2 ?
Last edited by ultratoto14 on Fri Sep 24, 2021 10:21 am, edited 1 time in total.
 
josephelkhoury
just joined
Posts: 4
Joined: Thu Sep 16, 2021 10:51 am

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Nov 17, 2021 7:59 pm

Hello,

Thank you very much for this QoS guide.
It looks great.

I've been searching for a solution to apply this when I have 2-3 WANs with load balancing using PCC.
One of the solutions is to have a separate router for the QoS.
Is it possible to have both (load balancing and QoS) on the same router?
 
hendra
Frequent Visitor
Frequent Visitor
Posts: 93
Joined: Wed Nov 22, 2017 2:21 am

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Dec 03, 2021 3:23 pm

do you know how to block or making Iinternet download manager doesnt increase latency when i play game? i did QOS but still
 
blurrybird
newbie
Posts: 38
Joined: Sun Jan 19, 2020 12:25 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Dec 29, 2021 4:19 am

@pcunite

Do you have any plans to release an updated 2021 (or 2022) guide for QoS that focuses on RouterOS 7.1.1 and CAKE / FQ_Codel?

I have a 100/40 VDSL connection that could use a reduction in bufferbloat, if you need guinea pigs.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Dec 29, 2021 4:31 am

Do you have any plans to release an updated 2021 (or 2022) guide for QoS that focuses on RouterOS 7.1.1 and CAKE / FQ_Codel?

I wouldn't say I have plans. Have not really used v7 enough. I'll wait until it is more mainstream unless I find some pressing need for it. WireGuard is very interesting to me, however. Not sure I'm the right person to discuss QoS and bufferfloat. Was hoping a more knowledgeable person would assist with clear instructions.
 
User avatar
SecCon
Member Candidate
Member Candidate
Posts: 156
Joined: Mon May 03, 2021 10:52 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 11, 2022 12:15 pm

Thanks for this, trying to sort out what may be relevant for RoS 7.1.1.
This seems to be the most extensive about QoS I can find, so we'll see how it goes. I know I need it, since when games or systems get gigabytes of updates, the e-mail stalls. Typical scenario I guess.

Not really understanding this text from the help: https://help.mikrotik.com/docs/display/ ... witch+Chip - is it saying one should implement QoS on the Switch instead of the Router?
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 11, 2022 4:38 pm

Not really understanding this text from the help. Is it saying one should implement QoS on the Switch instead of the Router?

Theory, best practices, and what you need. It's all a balancing act.

As a rule, you should QoS as close to the end device as possible. Consider what would happen if you had clients across several Wi-Fi hops and implemented QoS only at the router. The back and forth latency would not be optimal. Terrible in fact as the packet leaves the device, hits the first Wifi AP, then another AP, then the Router. The Router drops a packet, and the ack's have to get all the way back to the end device before congestion control kicks in. The latency would be terrible.

However, if the Router and Switch are connected over fast links, and these links are faster than what the end device has, you'll be fine to only QoS at the router, assuming the Router can keep up with the CPU demands now made on it. You QoS at the switch so that less demands are made further up into the network, if you need to.

Whether you need to do this or not is up to you to discover. How much data are you moving across the switches, etc. Most networks are not saturating switches, or even the connection to the Router.

But if you have a 24 port switch, with 24 PC's connected to it, and every single PC is making a request that is 1GB in size, well, you probably need to QoS at the switch with a CRS3XX series switch. Use a switch with hardware that can actually handle QoS tasks. But that is a very busy scenario. Investigate how busy your links are.

So, in summary, if necessary, move QoS closer to the end devices so that the Router CPU is not overloaded and so that latency is reduced.
 
User avatar
SecCon
Member Candidate
Member Candidate
Posts: 156
Joined: Mon May 03, 2021 10:52 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 11, 2022 5:13 pm

@pcunite
Thanks. Yeah that make sense... I only have one router and one switch for now, will get an MT AP later, got an ASUS now, but still, it would be better to run it off the switch, I only have SwitchOS on that since I assumed I would do most "complex" operations on the Router, but that can of course be changed. Better to distribute the load I guess. That would also mean I should apply the same thinking for Wifi devices. Exclude the eth port I use for the AP from QoS on the Switch, and let the WiFi QoS be handled by the AP.
 
User avatar
pcunite
Forum Guru
Forum Guru
Topic Author
Posts: 1347
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 11, 2022 8:14 pm

That would also mean I should apply the same thinking for Wifi devices. Exclude the ethernet port I use for the AP from QoS on the Switch, and let the WiFi QoS be handled by the AP.

If the AP hardware can handle it. However, if the AP has a fast connection to the switch or router, then let the router do it. It is electrically or optically about the same speed. This way the CPU on the AP is kept busy doing Wifi related tasks. Two scenarios where you might consider doing it on the AP are:

One:
The AP is actually a CPE (customer premise equipment) and thus you have a lot AP's out in the field coming back into your data center or core switch. Some CPE's may even be over WAN, PTP Wifi, or other types of links that cause known latency. So, QoS there.

Two:
In an office LAN where an individual AP is causing ethernet port saturation or multiple AP's causing switch saturation. Normally, the Wifi link is slower than the ethernet port. A single wifi client cannot saturate the ethernet link so this means you are observing multiple extremely busy wifi clients or many (50) clients on that one AP. However, the hardware on the AP would need to be sufficient to handle this type of client behavior plus the new responsibilities of QoS. I'm not sure if any gains would be worth it if the Router can handle it. The speed (latency) of ethernet or fiber is so fast. Might be better to install another AP and reduce load. I don't have experience with this.

You want to be reducing latency and reducing load in a way that is manageable and observable.
 
TheDarKFake
just joined
Posts: 4
Joined: Thu Feb 11, 2021 10:12 am

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jan 12, 2022 12:22 am


@kswong
Hello friend, I have been working with your QoS and I have modified it, I would like you to review it and improve it before sharing it: D
You do not have the required permissions to view the files attached to this post.
 
User avatar
own3r1138
Forum Veteran
Forum Veteran
Posts: 727
Joined: Sun Feb 14, 2021 12:33 am
Location: Pleiades
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jan 12, 2022 1:02 pm

@ivicask
Hello
Can you share your QoS and mangle rules, please?
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jan 13, 2022 8:11 am


@kswong
Hello friend, I have been working with your QoS and I have modified it, I would like you to review it and improve it before sharing it: D

Mmm....sorry, not correct! no need to use JUMP at all. Below is my updated QOS code for ROS7.

Usually only "MANGLE FORWARD" is used. "MANGLE PREROUTING + POSTROUTING" will be used unless you are using the router's built-in services (e.g.: internal DNS proxy, internal VPN...etc).


The differences and explanations Chain Mikrotik Input, Output, Forward, Prerouting...

Forward:
– Used to process data packets through routers, connections that occur from the public to local
or
– The connection that occurs from local to public
with the provision that there is no process inside the router, it means that the connection is direct and only passing in the router does not happen a process inside the router.

Input:
– Used to process packets entering the router through one of the interfaces with the destination IP address which is one of the router addresses. Chain input is useful for limiting the configuration access to Mikrotik Router.
or
– The connection that occurs from local to router and ends in router
example: use of the internal proxy (internal proxy right in the router)

Output:
– Used to process data packets originating from the router and left through one of the interfaces, connections that occur from the router to the public.
or
– The connection from the router to the local.
means the connection is created because the router itself is requesting (the router needs it, not from local/public), for example, the result of internal proxy Mikrotik sent to the client.

Prerouting:
– Pre-routing means that the connection will enter the router (no matter from where / depends on the settings mangle in interface later)
this connection will be in the process inside the router, can the process of bending to the external proxy, can filtering port, can anything, anyway there is a process, prerouting it marks the connection before the process occurs.

Postrouting:
– Postrouting is a connection that will exit the router after a process occurs inside the router.
Last edited by kswong on Sun Jan 23, 2022 10:32 pm, edited 2 times in total.
 
TheDarKFake
just joined
Posts: 4
Joined: Thu Feb 11, 2021 10:12 am

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Jan 14, 2022 10:44 pm


@kswong
Hello friend, I have been working with your QoS and I have modified it, I would like you to review it and improve it before sharing it: D
I see thank you very much. I totally base myself on the code you showed above, the truth is I'm still not very good at this.
I have not migrated to ROS7 but these new rules can be applied to ROS6? I see that it adds a bucket size 0.5 and in the queue type fq_codel-default, it is no longer sfq? I would like to continue testing your way of marking packets and connections, thank you very much for your answer.
 
dtaht
Member Candidate
Member Candidate
Posts: 215
Joined: Sat Aug 03, 2013 5:46 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 18, 2022 3:48 am

I am a little concerned at the smallness of your bucket-size.

two other threads here worth sorting through:

viewtopic.php?t=179307

viewtopic.php?p=906135
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Jan 18, 2022 4:02 am

I am a little concerned at the smallness of your bucket-size.
In some cases a bucket size this small is necessary. Some vendors will police very strictly, and if an upstream provider does this, the "burst" that is allowed by smaller bucket sizes can unfortunately be dropped by the upstream, and in this case the customer will lose control over what is dropped. Smaller bucket sizes will react more quickly and be more eager to drop packets, and therefore by making the bucket size smaller, it can mean the difference between your device dropping the packets (which has at least some understanding of what is considered important) rather than the upstream ISP's policer.

I have used bucket sizes as small as 0.01 for this reason, but would not go below that. The default of 0.1 is fine unless the upstream has a very reactive and eager policer that kicks in quite rapidly to drop packets.
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Jan 23, 2022 11:00 pm

:-D not bad...
test1.jpg
test2.jpg
You do not have the required permissions to view the files attached to this post.
 
TheDarKFake
just joined
Posts: 4
Joined: Thu Feb 11, 2021 10:12 am

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Jan 24, 2022 10:52 pm

:-D not bad...
Great, I with ROS6 and the old QoS got this test on a RB with traffic limit. I hope to migrate to ROS7 and be able to update it soon.
TEST1.jpg
Could you send me your example to my email? for once I migrate use it as a template of my QoS? In addition I would also like to send my work to you.
Thanks a lot
You do not have the required permissions to view the files attached to this post.
 
User avatar
SecCon
Member Candidate
Member Candidate
Posts: 156
Joined: Mon May 03, 2021 10:52 am

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Jan 26, 2022 4:56 pm

sigh, I did this in Ubiquiti Edgemax SmartQueue functionality and it worked like 99% of the times.

It was literally one setting.


This https://help.mikrotik.com/docs/display/ ... ionexample seems simple enough and considering I only want to make sure none of my computers/devices is able to suck up 100% of my theoretical 100MBit/s B/W I guess I will take that path.

/queue simple
add limit-at=5M/5M max-limit=100M/100M name=queue1 target=192.168.1.0/24

for my Lan at 192.168.1.0/24 .

Opinions?


Result of buff with
/queue simple
add limit-at=5M/5M max-limit=100M/100M name=queuebw5 target=192.168.1.0/24
Image

Only way to know for sure is over time I guess, any other test may be applicable?
 
TheDarKFake
just joined
Posts: 4
Joined: Thu Feb 11, 2021 10:12 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jan 27, 2022 6:46 pm

You can use simple Queues with PCQ, for example, and make them dynamic, for day, afternoon and night for example.
/queue simple
add name=me queue=Ume/Dme target=192.168.100.10
add disabled=yes name=day queue=Uday/Dday target=ether2,bridge-lan
add name=afternoon queue=Uafternoon/Dafternoon target=ether2,bridge-lan
add disabled=yes name=Night queue=UNight/DNight target=ether2,bridge-lan

/queue-type
add kind=pcq name=Dme pcq-classifier=dst-address pcq-rate=60M
add kind=pcq name=Ume pcq-classifier=src-address pcq-rate=20M
add kind=pcq name=Dday pcq-classifier=dst-address pcq-rate=20M
add kind=pcq name=Uday pcq-classifier=src-address pcq-rate=10M
By adding this everything connected to those interfaces will have the limit set by the active rule or higher. You can also provide more connection to your PC or IP that requires it.
 
deveth0
just joined
Posts: 6
Joined: Thu Jan 20, 2022 6:19 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Feb 05, 2022 4:26 pm

Hey there....

i just replaced my unifi USG with a rb5009 and now closely followed this thread about implementation of a QoS. So far it works really great, all connections and packages are marked by the mangle rules and applied to my queues.
But there is also some strange stuff going on when running nightly backups from a remote system (using Proxmox Backup Server if that matters in any way...).
rb5009_mangle.PNG
You can see, that a single of the four cores is completely blocked and weird enought the throughput is limited to ~15mb/s download here (as in Megabyte / s). Given that this is a 1000mbit connections, I'd expect more.
As soon as I activate the fastpath rule in Firewall, QoS does not work anymore (as expected) but the throughput goes up to ~50-80mb/s and the cpu usage is distributed over multiple cores (right part of the diagram).

Interestingly the usual speedtest tools (speedtest.net etc) all show a rather okish performance (~800mbit/s) no matter if fastpath / QoS is active or not.

Honestly I have no clue, what's going on here and would appreciate ANY help from you, thank you soo much!
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Allow SNMP" dst-port=161 in-interface=Corporate_VLAN protocol=udp
add action=accept chain=input comment="Allow LAN DNS queries-UDP" dst-port=53 in-interface-list=VLAN protocol=udp
add action=accept chain=input comment="Allow LAN DNS queries-TCP" dst-port=53 in-interface-list=VLAN protocol=tcp
add action=accept chain=input comment="Allow LAN NTP queries" dst-port=123 in-interface-list=VLAN protocol=udp
add action=accept chain=input comment="defconf: allow RB5009 access from Mgmt VLAN" in-interface-list=MGMT
add action=drop chain=input comment="defconf: drop all else" log-prefix=drop

## Forward rules
add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="Allow Estab & Related" connection-state=established,related

add action=accept chain=forward comment="Corporate => WLAN" in-interface=Corporate_VLAN out-interface=WLAN_VLAN
add action=accept chain=forward comment="WLAN => Corporate" in-interface=WLAN_VLAN out-interface=Corporate_VLAN
add action=accept chain=forward comment="Corporate => IoT" in-interface=Corporate_VLAN out-interface=IoT_VLAN
add action=accept chain=forward comment="WLAN => IoT" in-interface=WLAN_VLAN out-interface=IoT_VLAN
add action=drop chain=forward in-interface=IoT_VLAN out-interface-list=WAN

add action=accept chain=forward comment="Allow Port Forwarding (dst-nat)" connection-nat-state=dstnat in-interface=e8-wan
add action=accept chain=forward comment="VLAN Internet Access" in-interface-list=VLAN out-interface-list=WAN
add action=drop chain=forward comment=Drop log-prefix=drop

/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=\
    udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no

add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no

add action=mark-connection chain=prerouting comment=QUIC connection-state=new new-connection-mark=QUIC passthrough=yes port=80,443 \      
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no

add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no

add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no

add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack      
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack

add action=mark-connection chain=prerouting comment=SSH connection-state=new new-connection-mark=SSH passthrough=yes port=22,8984 \       
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=SSH new-packet-mark=SSH passthrough=no

add action=mark-connection chain=prerouting comment=OpenVpn connection-state=new new-connection-mark=OPENVPN passthrough=yes port=54 \    
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=OPENVPN new-packet-mark=OPENVPN passthrough=yes

add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no

add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new new-connection-mark=HTTP \
    passthrough=yes port=80,443,8080 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-1G new-connection-mark=\   
    HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no

add action=mark-connection chain=prerouting comment=PBS connection-state=new new-connection-mark=PBS passthrough=yes port=8007 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=PBS new-packet-mark=PBS passthrough=no

add action=mark-connection chain=prerouting comment=OTHER connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no


/queue tree
add bucket-size=0.01 max-limit=1G name=DOWN parent=BR1 queue=default
add bucket-size=0.01 max-limit=50M name=UP parent=e8-wan queue=default

add name="1. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="2. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="3. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="4. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="5. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=default
add name="5. SSH" packet-mark=SSH parent=DOWN priority=5 queue=default
add name="5. QUIC" packet-mark=QUIC parent=DOWN priority=5 queue=default
add name="6. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="7. PBS" packet-mark=PBS parent=DOWN priority=7 queue=default
add name="8. OTHER" packet-mark=OTHER parent=DOWN queue=default

add name="1. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="2. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="3. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="4. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="5. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=default
add name="5. SSH_" packet-mark=SSH parent=UP priority=5 queue=default
add name="5. QUIC_" packet-mark=QUIC parent=UP priority=5 queue=default
add name="6. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="7. PBS_" packet-mark=PBS parent=UP priority=7 queue=default
add name="8. OTHER_" packet-mark=OTHER parent=UP queue=default

You do not have the required permissions to view the files attached to this post.
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Feb 11, 2022 6:16 pm

Hey there....

i just replaced my unifi USG with a rb5009 and now closely followed this thread about implementation of a QoS. So far it works really great, all connections and packages are marked by the mangle rules and applied to my queues.
But there is also some strange stuff going on when running nightly backups from a remote system (using Proxmox Backup Server if that matters in any way...).
rb5009_mangle.PNG

You can see, that a single of the four cores is completely blocked and weird enought the throughput is limited to ~15mb/s download here (as in Megabyte / s). Given that this is a 1000mbit connections, I'd expect more.
As soon as I activate the fastpath rule in Firewall, QoS does not work anymore (as expected) but the throughput goes up to ~50-80mb/s and the cpu usage is distributed over multiple cores (right part of the diagram).

Interestingly the usual speedtest tools (speedtest.net etc) all show a rather okish performance (~800mbit/s) no matter if fastpath / QoS is active or not.

Honestly I have no clue, what's going on here and would appreciate ANY help from you, thank you soo much!
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Allow SNMP" dst-port=161 in-interface=Corporate_VLAN protocol=udp
add action=accept chain=input comment="Allow LAN DNS queries-UDP" dst-port=53 in-interface-list=VLAN protocol=udp
add action=accept chain=input comment="Allow LAN DNS queries-TCP" dst-port=53 in-interface-list=VLAN protocol=tcp
add action=accept chain=input comment="Allow LAN NTP queries" dst-port=123 in-interface-list=VLAN protocol=udp
add action=accept chain=input comment="defconf: allow RB5009 access from Mgmt VLAN" in-interface-list=MGMT
add action=drop chain=input comment="defconf: drop all else" log-prefix=drop

## Forward rules
add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="Allow Estab & Related" connection-state=established,related

add action=accept chain=forward comment="Corporate => WLAN" in-interface=Corporate_VLAN out-interface=WLAN_VLAN
add action=accept chain=forward comment="WLAN => Corporate" in-interface=WLAN_VLAN out-interface=Corporate_VLAN
add action=accept chain=forward comment="Corporate => IoT" in-interface=Corporate_VLAN out-interface=IoT_VLAN
add action=accept chain=forward comment="WLAN => IoT" in-interface=WLAN_VLAN out-interface=IoT_VLAN
add action=drop chain=forward in-interface=IoT_VLAN out-interface-list=WAN

add action=accept chain=forward comment="Allow Port Forwarding (dst-nat)" connection-nat-state=dstnat in-interface=e8-wan
add action=accept chain=forward comment="VLAN Internet Access" in-interface-list=VLAN out-interface-list=WAN
add action=drop chain=forward comment=Drop log-prefix=drop

/ip firewall mangle
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=\
    udp
add action=mark-packet chain=prerouting connection-mark=DNS new-packet-mark=DNS passthrough=no

add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no

add action=mark-connection chain=prerouting comment=QUIC connection-state=new new-connection-mark=QUIC passthrough=yes port=80,443 \      
    protocol=udp
add action=mark-packet chain=prerouting connection-mark=QUIC new-packet-mark=QUIC passthrough=no

add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no

add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no

add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack      
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack

add action=mark-connection chain=prerouting comment=SSH connection-state=new new-connection-mark=SSH passthrough=yes port=22,8984 \       
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=SSH new-packet-mark=SSH passthrough=no

add action=mark-connection chain=prerouting comment=OpenVpn connection-state=new new-connection-mark=OPENVPN passthrough=yes port=54 \    
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=OPENVPN new-packet-mark=OPENVPN passthrough=yes

add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no

add action=mark-connection chain=prerouting comment=HTTP connection-mark=no-mark connection-state=new new-connection-mark=HTTP \
    passthrough=yes port=80,443,8080 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=5000000-0 connection-mark=HTTP connection-rate=2M-1G new-connection-mark=\   
    HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no

add action=mark-connection chain=prerouting comment=PBS connection-state=new new-connection-mark=PBS passthrough=yes port=8007 \
    protocol=tcp
add action=mark-packet chain=prerouting connection-mark=PBS new-packet-mark=PBS passthrough=no

add action=mark-connection chain=prerouting comment=OTHER connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no


/queue tree
add bucket-size=0.01 max-limit=1G name=DOWN parent=BR1 queue=default
add bucket-size=0.01 max-limit=50M name=UP parent=e8-wan queue=default

add name="1. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="2. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="3. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="4. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="5. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=default
add name="5. SSH" packet-mark=SSH parent=DOWN priority=5 queue=default
add name="5. QUIC" packet-mark=QUIC parent=DOWN priority=5 queue=default
add name="6. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="7. PBS" packet-mark=PBS parent=DOWN priority=7 queue=default
add name="8. OTHER" packet-mark=OTHER parent=DOWN queue=default

add name="1. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="2. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="3. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="4. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="5. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=default
add name="5. SSH_" packet-mark=SSH parent=UP priority=5 queue=default
add name="5. QUIC_" packet-mark=QUIC parent=UP priority=5 queue=default
add name="6. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="7. PBS_" packet-mark=PBS parent=UP priority=7 queue=default
add name="8. OTHER_" packet-mark=OTHER parent=UP queue=default

disabled this rule
add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes
 
deveth0
just joined
Posts: 6
Joined: Thu Jan 20, 2022 6:19 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Feb 11, 2022 7:11 pm

Not sure how this helps...
As I said, disabling fasttrack reduces the performance...
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Mar 15, 2022 6:29 am

Pfd.jpg

:? Starting in ROS V6, Queue on input and postrouting chain :?: why mark prerouting or forward chain

test3.jpg

:D I got good results using mangle chain input and postrouting

test2.jpg
You do not have the required permissions to view the files attached to this post.
 
User avatar
denisun
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Wed Jul 16, 2014 6:38 pm
Location: Greece

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Apr 06, 2022 9:14 pm

/queue simple
add limit-at=11M/110M max-limit=11M/110M name=pppoe-out1 target=bridge1
add limit-at=2500k/25M max-limit=11M/110M name=QoS_2 \
    packet-marks=QoS_2 parent=pppoe-out1 priority=2/2 queue=\
    default-sfq/default-sfq target=bridge1 total-queue=default-sfq
add limit-at=2500k/25M max-limit=11M/110M name=QoS_3 \
    packet-marks=QoS_3 parent=pppoe-out1 priority=3/3 queue=\
    default-sfq/default-sfq target=bridge1 total-queue=default-sfq

With the above confing all the mangles, priorities etc work correct but i have always C in bufferbloat test.
I disabled all the above rules (and mangles) and i made one simple queue with queue type cake and fq_codel, but i had again C in bufferbloat test.
(RB4011, ROS 7.2)
 
User avatar
SecCon
Member Candidate
Member Candidate
Posts: 156
Joined: Mon May 03, 2021 10:52 am

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Apr 07, 2022 10:57 am

/queue simple
add limit-at=11M/110M max-limit=11M/110M name=pppoe-out1 target=bridge1
add limit-at=2500k/25M max-limit=11M/110M name=QoS_2 \
    packet-marks=QoS_2 parent=pppoe-out1 priority=2/2 queue=\
    default-sfq/default-sfq target=bridge1 total-queue=default-sfq
add limit-at=2500k/25M max-limit=11M/110M name=QoS_3 \
    packet-marks=QoS_3 parent=pppoe-out1 priority=3/3 queue=\
    default-sfq/default-sfq target=bridge1 total-queue=default-sfq

With the above confing all the mangles, priorities etc work correct but i have always C in bufferbloat test.
I disabled all the above rules (and mangles) and i made one simple queue with queue type cake and fq_codel, but i had again C in bufferbloat test.
(RB4011, ROS 7.2)
I think bad bufferbloat could also be due to poor ISP, but I may be wrong.
Maybe reading a bit more about it will help you understand what happens and what you can do about it?

https://en.wikipedia.org/wiki/Bufferbloat
 
User avatar
denisun
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Wed Jul 16, 2014 6:38 pm
Location: Greece

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Apr 07, 2022 11:21 am

/queue simple
add limit-at=11M/110M max-limit=11M/110M name=pppoe-out1 target=bridge1
add limit-at=2500k/25M max-limit=11M/110M name=QoS_2 \
    packet-marks=QoS_2 parent=pppoe-out1 priority=2/2 queue=\
    default-sfq/default-sfq target=bridge1 total-queue=default-sfq
add limit-at=2500k/25M max-limit=11M/110M name=QoS_3 \
    packet-marks=QoS_3 parent=pppoe-out1 priority=3/3 queue=\
    default-sfq/default-sfq target=bridge1 total-queue=default-sfq

With the above confing all the mangles, priorities etc work correct but i have always C in bufferbloat test.
I disabled all the above rules (and mangles) and i made one simple queue with queue type cake and fq_codel, but i had again C in bufferbloat test.
(RB4011, ROS 7.2)
I think bad bufferbloat could also be due to poor ISP, but I may be wrong.
Maybe reading a bit more about it will help you understand what happens and what you can do about it?

https://en.wikipedia.org/wiki/Bufferbloat
Its very strange.
I change the queue types to fifo small and sfq and i have not any difference in bufferbloat test.
I try to change the queues from simple to tree but i have the same issue.
I change the bucket size from 0.1 to 0.01 in childes but nothing changed.
Always i get C in tests.
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Apr 07, 2022 7:10 pm



I think bad bufferbloat could also be due to poor ISP, but I may be wrong.
Maybe reading a bit more about it will help you understand what happens and what you can do about it?

https://en.wikipedia.org/wiki/Bufferbloat
Its very strange.
I change the queue types to fifo small and sfq and i have not any difference in bufferbloat test.
I try to change the queues from simple to tree but i have the same issue.
I change the bucket size from 0.1 to 0.01 in childes but nothing changed.
Always i get C in tests.
Try to change the queue types to fq_codel or cake
 
Felixhuynh271
just joined
Posts: 3
Joined: Sun Nov 28, 2021 11:33 am

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Apr 19, 2022 7:10 pm

Pfd.jpg


:? Starting in ROS V6, Queue on input and postrouting chain :?: why mark prerouting or forward chain


test3.jpg


:D I got good results using mangle chain input and postrouting


test2.jpg
Hi Kswong, could you please share the QOS script for reference. many thanks!
my email: felix.dinhhuynh@gmail.com
 
User avatar
SecCon
Member Candidate
Member Candidate
Posts: 156
Joined: Mon May 03, 2021 10:52 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Apr 23, 2022 9:15 am

2 months later, just wanted to get back on how my
 /queue simple
 add limit-at=10M/10M max-limit=100M/100M name=queuebw10 target=192.168.1.0/24

works so far and it does work very well indeed.

I got a solid A at https://www.waveform.com/tools/bufferbloat
 
anserk
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Wed Mar 02, 2022 5:08 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat May 21, 2022 12:48 am

I just recently started to play with QoS on MikroTik. There are a lot of discussions about fq_codel here, but unfortunately, not too many tested configurations from users. There is a thread about CAKE (viewtopic.php?t=179307) that has a lot of good examples. But I wanted to focus on fq_codel, especially given uncertain status of CAKE in RouterOS.

After reading information from various sources and the official documentation at https://www.bufferbloat.net/projects/co ... signed-for, I tried adjusting some settings and settled for these with one simple queue:
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue simple
add bucket-size=0.01/0.01 max-limit=118M/11M name=queue1 queue=fq-codel/fq-codel target=ether1 total-queue=fq-codel

My Internet connection is Xfinity cable 100M/10M, in reality it's 120M/12M. The default fq_codel settings are best suited for 10Gbps as indicated in the referenced article. And based on Waveform tests, those tweaks do make a difference. I get an A, but it's the objective figures that mean more: +7ms for download, +1ms for upload. Various test runs produced slightly different results, but consistently good. Cutting down the bandwidth even further gave me A+ but at present I don't see much value in it and would rather get more bandwidth.

I also tried to run more realistic test: downloading Ubuntu via BitTorrent from a PC that is routed to Internet through WireGuard VPN (using source-based routing rule). So it was a heavy download almost saturating the link AND WireGuard running on the router AND running pings and some browsing activity including 4K YouTube video. All via wired connections.

I'm very satisfied with the results, hAP ac2 is a powerful little device. The router CPU hardly went above 50%, although one core was getting over 90% occasionally. FQ_codel did its job really well. Ping times were very good. Only several pings were lost - and I think it coincided with one core getting above 90%. Other activities I tried during the test felt like nothing was going on. SSH session from my work laptop over corporate IPSec VPN was snappy as usual.

I suggest that people share their fq_codel configuration exports and use cases. It would be very helpful for new users who don't have time or desire to read through dozens of articles or forum posts to figure out how they can fix bufferbloat. If you search online about bufferbloat, OpenWRT and Ubiquiti come up a lot, and I think part of it is how easy it is to configure on those platforms. Well, with simple queue on MikroTik it's just as easy as long as you know what to do.

I know there is the wiki and the new doc site, but it would be nice to have a simple cookbook for common things. MikroTik's new videos seem to target new inexperienced users. Maybe MikroTik can make a short video "how to fix bufferbloat". This is a hot topic, I'm sure many people can benefit from it.
 
anserk
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Wed Mar 02, 2022 5:08 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat May 21, 2022 11:09 pm

I found an old thread (viewtopic.php?t=113308) that talks about using QoS with FastTrack enabled. That got me interested. After reading the thread and also reviewing packet flow documentation, I came to realize I can successfully use this approach with fq_codel.
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue tree
add bucket-size=0.01 max-limit=118M name=download packet-mark=no-mark parent=bridge1 queue=fq-codel
add bucket-size=0.01 max-limit=11M name=upload packet-mark=no-mark parent=ether1 queue=fq-codel

The key are interface queues that work along with FastTrack. Because interface-attached HTB works only for egress, I put my download queue on the bridge interface (BTW, I have two VLANs on top of it), so the queuing happens when Internet traffic leaves the router in LAN direction. The upload queue is on WAN ether1.

I then ran some tests and got the same good bufferbloat results as with a simple queue. Here is one sample: https://www.waveform.com/tools/bufferbl ... b4aa8dc505. And CPU usage was much lower. It wasn't too bad even before for my 120M/12M Internet connection, but now it got me thinking - how well would hAP ac2 handle a much faster WAN connection?

I pulled out a spare router, reset to all defaults, and ran some tests. I used iperf3 with 20 threads and 60 seconds run to simulate LAN to WAN heavy traffic. Client PC is on LAN side of hAP ac2, server PC is on WAN.
Same fq_codel configuration as above, only max-limit set to 1020M for both download and upload. Simulating a gigabit Internet service.
With FastTrack disabled, two CPU cores are maxed out at 100%, iperf reported ~560Mbps.
With FastTrack enabled, only one core reached 90-95%, overall utilization ~50%. Iperf speeds were 915-924Mbps. FastTrack makes a huge difference indeed.
I also ran continuous pings from client to server to see fq_codel working for the lack of a better test. The difference was also huge:
1ms unloaded.
With fq_codel average ping 3ms, 15ms max.
Without fq_codel (queues disabled) average ping 13ms, max 48ms. Fq_codel makes a difference even on such a fast connection.

It's a big discovery for me. MikroTik's clever solution - FastTrack - turns out to be even better than HW NAT in my case as every SOHO router with HW NAT I've seen can't do SQM with it enabled. If you search online for best routers with SQM for gigabit Internet service, you will find most recommendations are around x86 or some other beefy solutions. Unless my test was flawed somehow, you can actually get pretty close to a gigabit with fq_codel on an inexpensive hAP ac2. Granted, I don't have those kind of speeds, but this will certainly be on my list of recommendations for others.

It also means older low-performance MikroTik devices can do pretty well with fq_codel thanks to FastTrack. And even if you have a more powerful device like RB5009, the option to save some processing power is there. With container support coming up soon, there is always going to be a way to put that extra CPU to good use.

I want to point out that I don't use WiFi on hAP ac2, which would definitely consume a good chunk of CPU.
 
KristijanM
just joined
Posts: 1
Joined: Thu Feb 06, 2020 2:00 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat May 28, 2022 7:53 pm

@pcunite

This is very good tutorial and i using it for sure 2 years and im realy statisfied.

But i have question, how to make using this QoS and implement PCQ queue with equally queuing for all users?
 
felipefideli
just joined
Posts: 2
Joined: Thu Jun 16, 2022 4:45 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Jun 16, 2022 11:04 pm

I found an old thread (viewtopic.php?t=113308) that talks about using QoS with FastTrack enabled. That got me interested. After reading the thread and also reviewing packet flow documentation, I came to realize I can successfully use this approach with fq_codel.
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue tree
add bucket-size=0.01 max-limit=118M name=download packet-mark=no-mark parent=bridge1 queue=fq-codel
add bucket-size=0.01 max-limit=11M name=upload packet-mark=no-mark parent=ether1 queue=fq-codel

The key are interface queues that work along with FastTrack. Because interface-attached HTB works only for egress, I put my download queue on the bridge interface (BTW, I have two VLANs on top of it), so the queuing happens when Internet traffic leaves the router in LAN direction. The upload queue is on WAN ether1.

I then ran some tests and got the same good bufferbloat results as with a simple queue. Here is one sample: https://www.waveform.com/tools/bufferbl ... b4aa8dc505. And CPU usage was much lower. It wasn't too bad even before for my 120M/12M Internet connection, but now it got me thinking - how well would hAP ac2 handle a much faster WAN connection?

I pulled out a spare router, reset to all defaults, and ran some tests. I used iperf3 with 20 threads and 60 seconds run to simulate LAN to WAN heavy traffic. Client PC is on LAN side of hAP ac2, server PC is on WAN.
Same fq_codel configuration as above, only max-limit set to 1020M for both download and upload. Simulating a gigabit Internet service.
With FastTrack disabled, two CPU cores are maxed out at 100%, iperf reported ~560Mbps.
With FastTrack enabled, only one core reached 90-95%, overall utilization ~50%. Iperf speeds were 915-924Mbps. FastTrack makes a huge difference indeed.
I also ran continuous pings from client to server to see fq_codel working for the lack of a better test. The difference was also huge:
1ms unloaded.
With fq_codel average ping 3ms, 15ms max.
Without fq_codel (queues disabled) average ping 13ms, max 48ms. Fq_codel makes a difference even on such a fast connection.

It's a big discovery for me. MikroTik's clever solution - FastTrack - turns out to be even better than HW NAT in my case as every SOHO router with HW NAT I've seen can't do SQM with it enabled. If you search online for best routers with SQM for gigabit Internet service, you will find most recommendations are around x86 or some other beefy solutions. Unless my test was flawed somehow, you can actually get pretty close to a gigabit with fq_codel on an inexpensive hAP ac2. Granted, I don't have those kind of speeds, but this will certainly be on my list of recommendations for others.

It also means older low-performance MikroTik devices can do pretty well with fq_codel thanks to FastTrack. And even if you have a more powerful device like RB5009, the option to save some processing power is there. With container support coming up soon, there is always going to be a way to put that extra CPU to good use.

I want to point out that I don't use WiFi on hAP ac2, which would definitely consume a good chunk of CPU.


Thank you very much! It worked wonderfully well on my current setup
 
lebaran
just joined
Posts: 17
Joined: Mon Feb 21, 2022 6:24 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Sun Aug 28, 2022 5:18 pm

Implementing traffic prioritization (QoS) with RouterOS

To turn on the QoS capabilities of RouterOS, we implement two things: marking and then queuing.

The Marking Stage
..........
/ip firewall mangle
# Identify DNS on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes comment="DNS"
add chain=prerouting  action=mark-packet     connection-mark=DNS  new-packet-mark=DNS passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes
add chain=postrouting action=mark-packet     connection-mark=DNS  new-packet-mark=DNS passthrough=no

# Identify VoIP
add chain=prerouting  action=mark-connection new-connection-mark=VOIP port=5060-5062,10000-10050 protocol=udp passthrough=yes comment="VOIP"
add chain=prerouting  action=mark-packet     connection-mark=VOIP new-packet-mark=VOIP passthrough=no

# Identify HTTP/3 and Google's QUIC
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=QUIC port=80,443 protocol=udp passthrough=yes comment="QUIC"
add chain=prerouting  action=mark-packet     connection-mark=QUIC new-packet-mark=QUIC passthrough=no

# Identify UPD. Useful for further analysis. Should it be considered high priority or put in the catchall? You decide.
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=UDP protocol=udp passthrough=yes comment="UDP"
add chain=prerouting  action=mark-packet     connection-mark=UDP  new-packet-mark=UDP passthrough=no

# Identify PING on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes comment="ICMP"
add chain=prerouting  action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes
add chain=postrouting action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no

# Identify Acknowledgment packets
add chain=postrouting action=mark-packet     new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no comment="ACK"
add chain=prerouting  action=mark-packet     new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no

# Identify HTTP traffic but move it to a Streaming mark if necessary.
add chain=prerouting  action=mark-connection connection-mark=no-mark  connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp passthrough=yes comment="HTTP"
add chain=prerouting  action=mark-connection connection-bytes=5M-0    connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG protocol=tcp passthrough=yes
add chain=prerouting  action=mark-packet     connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add chain=prerouting  action=mark-packet     connection-mark=HTTP     new-packet-mark=HTTP passthrough=no

# Email goes to the catchall
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=POP3 port=995,465,587 protocol=tcp passthrough=yes comment="OTHER"
add chain=prerouting  action=mark-packet     connection-mark=POP3 new-packet-mark=OTHER passthrough=no

# Unknown goes to the catchall
add chain=prerouting  action=mark-connection connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add chain=prerouting  action=mark-packet     connection-mark=OTHER   new-packet-mark=OTHER passthrough=no
The Queuing Stage
......
I followed exactly as stated above, but DL/UL trafic from bufferbloat test on https://www.waveform.com/tools/bufferbloat is bypassed from Queue.
 
massinia
Member Candidate
Member Candidate
Posts: 184
Joined: Thu Jun 09, 2022 7:20 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Sep 13, 2022 10:42 pm

I found an old thread (viewtopic.php?t=113308) that talks about using QoS with FastTrack enabled. That got me interested. After reading the thread and also reviewing packet flow documentation, I came to realize I can successfully use this approach with fq_codel.
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue tree
add bucket-size=0.01 max-limit=118M name=download packet-mark=no-mark parent=bridge1 queue=fq-codel
add bucket-size=0.01 max-limit=11M name=upload packet-mark=no-mark parent=ether1 queue=fq-codel
Many thanks anserk, it worked wonderfully well also for me.
Now I have FastTrack active and Bufferbloat test with score A+
 
good2see
just joined
Posts: 5
Joined: Fri Nov 11, 2022 9:50 am

Re: Using RouterOS to prioritize (Qos) traffic for a Class C

Fri Jan 06, 2023 8:55 am

Implementing traffic prioritization (QoS) with RouterOS

To turn on the QoS capabilities of RouterOS, we implement two things: marking and then queuing.

The Marking Stage

How to Mark the Traffic Types:
RouterOS supplies the Mangle feature to mark packets. What you decide to mark is up to personal and business decisions. Here is a sample starting point. It can be appropriate to mark items, interesting to you, that will ultimately go into the same queue. This is useful for network monitoring purposes. The POP3 mark is an example of that.

Take time to get your marking correct. Test to ensure you are seeing the totals move as you expect. At this stage, we are only marking items. We will use another command to take actions on these marks.

/ip firewall mangle
# Identify DNS on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes comment="DNS"
add chain=prerouting  action=mark-packet     connection-mark=DNS  new-packet-mark=DNS passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes
add chain=postrouting action=mark-packet     connection-mark=DNS  new-packet-mark=DNS passthrough=no

# Identify VoIP
add chain=prerouting  action=mark-connection new-connection-mark=VOIP port=5060-5062,10000-10050 protocol=udp passthrough=yes comment="VOIP"
add chain=prerouting  action=mark-packet     connection-mark=VOIP new-packet-mark=VOIP passthrough=no

# Identify HTTP/3 and Google's QUIC
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=QUIC port=80,443 protocol=udp passthrough=yes comment="QUIC"
add chain=prerouting  action=mark-packet     connection-mark=QUIC new-packet-mark=QUIC passthrough=no

# Identify UPD. Useful for further analysis. Should it be considered high priority or put in the catchall? You decide.
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=UDP protocol=udp passthrough=yes comment="UDP"
add chain=prerouting  action=mark-packet     connection-mark=UDP  new-packet-mark=UDP passthrough=no

# Identify PING on the network or coming from the Router itself
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes comment="ICMP"
add chain=prerouting  action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes
add chain=postrouting action=mark-packet     connection-mark=ICMP new-packet-mark=ICMP passthrough=no

# Identify Acknowledgment packets
add chain=postrouting action=mark-packet     new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no comment="ACK"
add chain=prerouting  action=mark-packet     new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no

# Identify HTTP traffic but move it to a Streaming mark if necessary.
add chain=prerouting  action=mark-connection connection-mark=no-mark  connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp passthrough=yes comment="HTTP"
add chain=prerouting  action=mark-connection connection-bytes=5M-0    connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG protocol=tcp passthrough=yes
add chain=prerouting  action=mark-packet     connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add chain=prerouting  action=mark-packet     connection-mark=HTTP     new-packet-mark=HTTP passthrough=no

# Email goes to the catchall
add chain=prerouting  action=mark-connection connection-state=new new-connection-mark=POP3 port=995,465,587 protocol=tcp passthrough=yes comment="OTHER"
add chain=prerouting  action=mark-packet     connection-mark=POP3 new-packet-mark=OTHER passthrough=no

# Unknown goes to the catchall
add chain=prerouting  action=mark-connection connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add chain=prerouting  action=mark-packet     connection-mark=OTHER   new-packet-mark=OTHER passthrough=no


The Queuing Stage

How to act on Traffic Marks:
RouterOS supplies the Queue Tree structure that enable us to act on marks. This is how we truly classify the packet flows on the network. A whole book could be written on what is occurring here. There are many options one could use to dial in a very custom Queue Tree. The purpose of this article, however, is to present a simple yet very effective implementation. A few things do need to be understood.

Max-limit:
In order for queuing to occur in our equipment, and thus give us the control on packet flows, we have to set our interfaces to operate at 10% the rate of our ISP connection. This is only a starting number and is dependent upon your CPU speed and simultaneous connections. Apply to both the upload and download links. This way buffering always occurs inside of our equipment. The max-limit parameter is required for the algorithms to function and must not be 0. In our example, we have 100M service, so we have set it to 90M.

Limit-at:
This option is not something you will use commonly and is therefore recommended to leave it at 0 (disabled). However, there is a very special situation where you must enable it. Read the Protection with Limit-at section to learn more.

bucket-size:
During congestion, this value sets the amount of tokens to accrue before the chosen queue type takes effect. This is an equation and means that after Max-limit is reached (bucket-size * max-limit) worth of bytes will be engaged by the queue type. For our purposes, we only want a small amount of time addressing packets going over the limit, enough to smooth out any protocol windowing.


/queue tree

# DOWN
add name=DOWN max-limit=90M parent=LAN bucket-size=0.01 queue=default

add name="1. VOIP"     packet-mark=VOIP     parent=DOWN priority=1 queue=default
add name="2. DNS"      packet-mark=DNS      parent=DOWN priority=2 queue=default
add name="3. ACK"      packet-mark=ACK      parent=DOWN priority=3 queue=default
add name="4. UDP"      packet-mark=UDP      parent=DOWN priority=3 queue=default
add name="5. ICMP"     packet-mark=ICMP     parent=DOWN priority=4 queue=default
add name="6. HTTP"     packet-mark=HTTP     parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC"     packet-mark=QUIC     parent=DOWN priority=7 queue=default
add name="9. OTHER"    packet-mark=OTHER    parent=DOWN priority=8 queue=default


# UP
add name=UP max-limit=90M parent=WAN bucket-size=0.01 queue=default

add name="1. VOIP_"     packet-mark=VOIP     parent=UP priority=1 queue=default
add name="2. DNS_"      packet-mark=DNS      parent=UP priority=2 queue=default
add name="3. ACK_"      packet-mark=ACK      parent=UP priority=3 queue=default
add name="4. UDP_"      packet-mark=UDP      parent=UP priority=3 queue=default
add name="5. ICMP_"     packet-mark=ICMP     parent=UP priority=4 queue=default
add name="6. HTTP_"     packet-mark=HTTP     parent=UP priority=5 queue=default
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="8. QUIC_"     packet-mark=QUIC     parent=UP priority=7 queue=default
add name="9. OTHER_"    packet-mark=OTHER    parent=UP priority=8 queue=default
I am not a pro user but In my opinion this script doesn't catch the traffic(inpu-output) generated by router itself. I extended the script by adding input-output connection mark. Because my router checks some ips every 30 seconds by Netwatch. Without input-output connection mark, netwach traffic is unable to prioritized.
 
jy00022158
just joined
Posts: 2
Joined: Tue Jul 06, 2021 10:21 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Feb 11, 2023 6:41 pm

I now have a problem that the mangle tag does not work on ROS 7.7. Please help me check


# Identify HTTP/3 and Google's QUIC
add chain=prerouting action=mark-connection connection-state=new new-connection-mark=QUIC port=80,443 protocol=udp passthrough=yes comment="QUIC"
add chain=prerouting action=mark-packet connection-mark=QUIC new-packet-mark=QUIC passthrough=no


# Identify HTTP traffic but move it to a Streaming mark if necessary.
add chain=prerouting action=mark-connection connection-mark=no-mark connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp passthrough=yes comment="HTTP"
add chain=prerouting action=mark-connection connection-bytes=5M-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG protocol=tcp passthrough=yes
add chain=prerouting action=mark-packet connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add chain=prerouting action=mark-packet connection-mark=HTTP new-packet-mark=HTTP passthrough=no
 
kswong
just joined
Posts: 12
Joined: Mon May 11, 2020 6:21 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Feb 13, 2023 8:43 am

Please make sure fasttrack is disabled
I now have a problem that the mangle tag does not work on ROS 7.7. Please help me check


# Identify HTTP/3 and Google's QUIC
add chain=prerouting action=mark-connection connection-state=new new-connection-mark=QUIC port=80,443 protocol=udp passthrough=yes comment="QUIC"
add chain=prerouting action=mark-packet connection-mark=QUIC new-packet-mark=QUIC passthrough=no


# Identify HTTP traffic but move it to a Streaming mark if necessary.
add chain=prerouting action=mark-connection connection-mark=no-mark connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp passthrough=yes comment="HTTP"
add chain=prerouting action=mark-connection connection-bytes=5M-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG protocol=tcp passthrough=yes
add chain=prerouting action=mark-packet connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add chain=prerouting action=mark-packet connection-mark=HTTP new-packet-mark=HTTP passthrough=no
 
thomslorZQSD
just joined
Posts: 3
Joined: Fri Feb 10, 2023 11:58 am

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Feb 27, 2023 7:22 pm

Hello everyone !
I tried to adapt the example described at the start of thi topic to my usage : I want to prioritize gaming paquets in all circumstances. i don't want to drop a single one and i want the drop rate to be low for that type of packet. The others can be dropped or slow down.
Here is the configuration for QoS.
/ip firewall mangle
# Identify DNS on the network or coming from the Router itself
add chain=prerouting action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes comment="DNS"
add chain=prerouting action=mark-packet connection-mark=DNS new-packet-mark=DNS passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=DNS port=53 protocol=udp passthrough=yes
add chain=postrouting action=mark-packet connection-mark=DNS new-packet-mark=DNS passthrough=no

# Identify Games packets
add chain=prerouting action=mark-connection new-connection-mark=Games port=3478-3479,5000-5500,5060,5062,6112,3724,1119,7000-7500 protocol=udp passthrough=yes comment="Games-UDP"
add chain=prerouting action=mark-packet connection-mark=Games new-packet-mark=Games passthrough=no

# Identify HTTP/3 and Google's QUIC
add chain=prerouting action=mark-connection connection-state=new new-connection-mark=QUIC port=80,443 protocol=udp passthrough=yes comment="QUIC"
add chain=prerouting action=mark-packet connection-mark=QUIC new-packet-mark=QUIC passthrough=no

# Identify UPD. Useful for further analysis. Should it be considered high priority or put in the catchall? You decide.
add chain=prerouting action=mark-connection connection-state=new new-connection-mark=UDP protocol=udp passthrough=yes comment="UDP"
add chain=prerouting action=mark-packet connection-mark=UDP new-packet-mark=UDP passthrough=no

# Identify PING on the network or coming from the Router itself
add chain=prerouting action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes comment="ICMP"
add chain=prerouting action=mark-packet connection-mark=ICMP new-packet-mark=ICMP passthrough=no
add chain=postrouting action=mark-connection connection-state=new new-connection-mark=ICMP protocol=icmp passthrough=yes
add chain=postrouting action=mark-packet connection-mark=ICMP new-packet-mark=ICMP passthrough=no

# Identify Acknowledgment packets
add chain=postrouting action=mark-packet new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no comment="ACK"
add chain=prerouting action=mark-packet new-packet-mark=ACK packet-size=0-123 protocol=tcp tcp-flags=ack passthrough=no

# Identify HTTP traffic but move it to a Streaming mark if necessary.
add chain=prerouting action=mark-connection connection-mark=no-mark connection-state=new new-connection-mark=HTTP port=80,443 protocol=tcp passthrough=yes comment="HTTP"
add chain=prerouting action=mark-connection connection-bytes=5M-0 connection-mark=HTTP connection-rate=2M-100M new-connection-mark=HTTP_BIG protocol=tcp passthrough=yes
add chain=prerouting action=mark-packet connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
add chain=prerouting action=mark-packet connection-mark=HTTP new-packet-mark=HTTP passthrough=no

# Email goes to the catchall
add chain=prerouting action=mark-connection connection-state=new new-connection-mark=POP3 port=995,465,587 protocol=tcp passthrough=yes comment="OTHER"
add chain=prerouting action=mark-packet connection-mark=POP3 new-packet-mark=OTHER passthrough=no

# Unknown goes to the catchall
add chain=prerouting action=mark-connection connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add chain=prerouting action=mark-packet connection-mark=OTHER new-packet-mark=OTHER passthrough=no

/queue tree
# DOWN
add name=DOWN max-limit=190M parent=LAN bucket-size=0.01 queue=default

add name="1. GAMES" packet-mark=Games parent=DOWN priority=1 queue=default
add name="2. DNS" packet-mark=DNS parent=DOWN priority=2 queue=default
add name="3. ACK" packet-mark=ACK parent=DOWN priority=3 queue=default
add name="4. UDP" packet-mark=UDP parent=DOWN priority=3 queue=default
add name="5. ICMP" packet-mark=ICMP parent=DOWN priority=4 queue=default
add name="6. HTTP" packet-mark=HTTP parent=DOWN priority=5 queue=default
add name="7. HTTP_BIG" packet-mark=HTTP_BIG parent=DOWN priority=6 queue=default
add name="8. QUIC" packet-mark=QUIC parent=DOWN priority=7 queue=default
add name="9. OTHER" packet-mark=OTHER parent=DOWN priority=8 queue=default


# UP
add name=UP max-limit=190M parent=ether1-WAN bucket-size=0.01 queue=default

add name="1. GAMES_" packet-mark=GAMES parent=UP priority=1 queue=default
add name="2. DNS_" packet-mark=DNS parent=UP priority=2 queue=default
add name="3. ACK_" packet-mark=ACK parent=UP priority=3 queue=default
add name="4. UDP_" packet-mark=UDP parent=UP priority=3 queue=default
add name="5. ICMP_" packet-mark=ICMP parent=UP priority=4 queue=default
add name="6. HTTP_" packet-mark=HTTP parent=UP priority=5 queue=default
add name="7. HTTP_BIG_" packet-mark=HTTP_BIG parent=UP priority=6 queue=default
add name="8. QUIC_" packet-mark=QUIC parent=UP priority=7 queue=default
add name="9. OTHER_" packet-mark=OTHER parent=UP priority=8 queue=default
However, when testing packet loss on Valorant, I saw that having this config on or off isn't modifying anything and I've got the same results with or without QoS when playing with a download behind.
Is it possible to slow down the download ? What did I do wrong here ?
Thank you for all your examples and explanations, that helped me a lot when configuring my Mikrotik stuff.
 
good2see
just joined
Posts: 5
Joined: Fri Nov 11, 2022 9:50 am

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Mar 31, 2023 10:25 am

Hi, How can I catch and mark QUIC HEAVY traffic. Nowadays Quic traffic is greater than http browsing.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 3115
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Mar 31, 2023 6:38 pm

AFAIK QUIC traffic is on 443 UDP
 
moorezilla
just joined
Posts: 10
Joined: Fri Jan 20, 2023 4:29 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Fri Apr 07, 2023 5:35 pm

I found an old thread (viewtopic.php?t=113308) that talks about using QoS with FastTrack enabled. That got me interested. After reading the thread and also reviewing packet flow documentation, I came to realize I can successfully use this approach with fq_codel.
Code: Select all
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue tree
add bucket-size=0.01 max-limit=118M name=download packet-mark=no-mark parent=bridge1 queue=fq-codel
add bucket-size=0.01 max-limit=11M name=upload packet-mark=no-mark parent=ether1 queue=fq-codel
I just wanted to say, "thanks," as well, anserk. This seems to be the best current solution to smoothing traffic on the 5009 in a home environment with a gigabit internet connection, while maintaining as much speed as possible.
 
User avatar
spr41178
Member Candidate
Member Candidate
Posts: 114
Joined: Tue Apr 01, 2014 11:11 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Tue Aug 22, 2023 4:33 pm

I found an old thread (viewtopic.php?t=113308) that talks about using QoS with FastTrack enabled. That got me interested. After reading the thread and also reviewing packet flow documentation, I came to realize I can successfully use this approach with fq_codel.
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue tree
add bucket-size=0.01 max-limit=118M name=download packet-mark=no-mark parent=bridge1 queue=fq-codel
add bucket-size=0.01 max-limit=11M name=upload packet-mark=no-mark parent=ether1 queue=fq-codel

The key are interface queues that work along with FastTrack. Because interface-attached HTB works only for egress, I put my download queue on the bridge interface (BTW, I have two VLANs on top of it), so the queuing happens when Internet traffic leaves the router in LAN direction. The upload queue is on WAN ether1.

I then ran some tests and got the same good bufferbloat results as with a simple queue. Here is one sample: https://www.waveform.com/tools/bufferbl ... b4aa8dc505. And CPU usage was much lower. It wasn't too bad even before for my 120M/12M Internet connection, but now it got me thinking - how well would hAP ac2 handle a much faster WAN connection?

I pulled out a spare router, reset to all defaults, and ran some tests. I used iperf3 with 20 threads and 60 seconds run to simulate LAN to WAN heavy traffic. Client PC is on LAN side of hAP ac2, server PC is on WAN.
Same fq_codel configuration as above, only max-limit set to 1020M for both download and upload. Simulating a gigabit Internet service.
With FastTrack disabled, two CPU cores are maxed out at 100%, iperf reported ~560Mbps.
With FastTrack enabled, only one core reached 90-95%, overall utilization ~50%. Iperf speeds were 915-924Mbps. FastTrack makes a huge difference indeed.
I also ran continuous pings from client to server to see fq_codel working for the lack of a better test. The difference was also huge:
1ms unloaded.
With fq_codel average ping 3ms, 15ms max.
Without fq_codel (queues disabled) average ping 13ms, max 48ms. Fq_codel makes a difference even on such a fast connection.

It's a big discovery for me. MikroTik's clever solution - FastTrack - turns out to be even better than HW NAT in my case as every SOHO router with HW NAT I've seen can't do SQM with it enabled. If you search online for best routers with SQM for gigabit Internet service, you will find most recommendations are around x86 or some other beefy solutions. Unless my test was flawed somehow, you can actually get pretty close to a gigabit with fq_codel on an inexpensive hAP ac2. Granted, I don't have those kind of speeds, but this will certainly be on my list of recommendations for others.

It also means older low-performance MikroTik devices can do pretty well with fq_codel thanks to FastTrack. And even if you have a more powerful device like RB5009, the option to save some processing power is there. With container support coming up soon, there is always going to be a way to put that extra CPU to good use.

I want to point out that I don't use WiFi on hAP ac2, which would definitely consume a good chunk of CPU.
Good afternoon, hope you are well,

Are you using this post rules with fq-codel or the other topics rules?

Fq-Codel is implemented only in the WAN/BRIDGE parent or also on the child?

Thank you in advance
 
anserk
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Wed Mar 02, 2022 5:08 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Wed Aug 23, 2023 5:54 pm

I use it only with fq-codel on the bridge interface for download and ether1 (WAN) for upload.
 
User avatar
spr41178
Member Candidate
Member Candidate
Posts: 114
Joined: Tue Apr 01, 2014 11:11 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Thu Aug 24, 2023 9:16 am

I use it only with fq-codel on the bridge interface for download and ether1 (WAN) for upload.
Good morning, understood,

So two questions the rest of the childs you leave as queue=default?

And do you use it with pcunites solution?

Thank you very much and have a good day
 
anserk
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Wed Mar 02, 2022 5:08 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Aug 26, 2023 3:44 am

I haven't changed eth2-eth5 queues from the default, which is only-hardware-queue on my router.

I never tried pcunite's approach, sorry. I wanted a simple configuration and - the key point - ability to use while FastTrack is enabled. It works great without taxing CPU too much. I don't have to fiddle with it at all, it's "set and forget". The 3 configuration lines in my post above is all it takes.

Good luck with testing!
 
User avatar
spr41178
Member Candidate
Member Candidate
Posts: 114
Joined: Tue Apr 01, 2014 11:11 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sun Aug 27, 2023 10:36 am

I haven't changed eth2-eth5 queues from the default, which is only-hardware-queue on my router.

I never tried pcunite's approach, sorry. I wanted a simple configuration and - the key point - ability to use while FastTrack is enabled. It works great without taxing CPU too much. I don't have to fiddle with it at all, it's "set and forget". The 3 configuration lines in my post above is all it takes.

Good luck with testing!
Thank you very much, i am also fond of the set and forget idea but testing both out to see performance!

Have a nice day
 
MEDO11
just joined
Posts: 5
Joined: Wed Feb 23, 2022 10:49 am

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Sep 02, 2023 11:34 am

I use it only with fq-codel on the bridge interface for download and ether1 (WAN) for upload.
Just wanted to thank you for this easy qos setup, at the first glance seems to be doing a great job!
 
theprism
newbie
Posts: 29
Joined: Sun Sep 16, 2012 4:11 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Sat Sep 02, 2023 8:18 pm

I found an old thread (viewtopic.php?t=113308) that talks about using QoS with FastTrack enabled. That got me interested. After reading the thread and also reviewing packet flow documentation, I came to realize I can successfully use this approach with fq_codel.
/queue type
add fq-codel-limit=1000 fq-codel-quantum=300 fq-codel-target=12ms kind=fq-codel name=fq-codel
/queue tree
add bucket-size=0.01 max-limit=118M name=download packet-mark=no-mark parent=bridge1 queue=fq-codel
add bucket-size=0.01 max-limit=11M name=upload packet-mark=no-mark parent=ether1 queue=fq-codel

The key are interface queues that work along with FastTrack. Because interface-attached HTB works only for egress, I put my download queue on the bridge interface (BTW, I have two VLANs on top of it), so the queuing happens when Internet traffic leaves the router in LAN direction. The upload queue is on WAN ether1.

I then ran some tests and got the same good bufferbloat results as with a simple queue. Here is one sample: https://www.waveform.com/tools/bufferbl ... b4aa8dc505. And CPU usage was much lower. It wasn't too bad even before for my 120M/12M Internet connection, but now it got me thinking - how well would hAP ac2 handle a much faster WAN connection?

I pulled out a spare router, reset to all defaults, and ran some tests. I used iperf3 with 20 threads and 60 seconds run to simulate LAN to WAN heavy traffic. Client PC is on LAN side of hAP ac2, server PC is on WAN.
Same fq_codel configuration as above, only max-limit set to 1020M for both download and upload. Simulating a gigabit Internet service.
With FastTrack disabled, two CPU cores are maxed out at 100%, iperf reported ~560Mbps.
With FastTrack enabled, only one core reached 90-95%, overall utilization ~50%. Iperf speeds were 915-924Mbps. FastTrack makes a huge difference indeed.
I also ran continuous pings from client to server to see fq_codel working for the lack of a better test. The difference was also huge:
1ms unloaded.
With fq_codel average ping 3ms, 15ms max.
Without fq_codel (queues disabled) average ping 13ms, max 48ms. Fq_codel makes a difference even on such a fast connection.

It's a big discovery for me. MikroTik's clever solution - FastTrack - turns out to be even better than HW NAT in my case as every SOHO router with HW NAT I've seen can't do SQM with it enabled. If you search online for best routers with SQM for gigabit Internet service, you will find most recommendations are around x86 or some other beefy solutions. Unless my test was flawed somehow, you can actually get pretty close to a gigabit with fq_codel on an inexpensive hAP ac2. Granted, I don't have those kind of speeds, but this will certainly be on my list of recommendations for others.

It also means older low-performance MikroTik devices can do pretty well with fq_codel thanks to FastTrack. And even if you have a more powerful device like RB5009, the option to save some processing power is there. With container support coming up soon, there is always going to be a way to put that extra CPU to good use.

I want to point out that I don't use WiFi on hAP ac2, which would definitely consume a good chunk of CPU.
How this can be adjusted for 2 bridges - one for homelan and another one for guestlan?
Thanks
 
anserk
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Wed Mar 02, 2022 5:08 pm

Re: Using RouterOS to QoS your network - 2020 Edition

Mon Sep 04, 2023 5:00 pm

You should be able to apply the queue on both bridges. However, I'm not sure how well it would work for downloads in this case. Uploads would still be fine. Interface-attached HTB works only for egress, so the queue limits apply right before exiting the interface. This is why I put download limits on the LAN interface.

I also have a guest LAN, but I use just one bridge for both guest and private networks and use VLAN interfaces under bridge1. So the queue works correctly for both networks. With two bridges the queues would be independent from each other and will not work properly if you have simultaneous traffic on both. Unless you split the total Internet bandwidth between the two, in which case you would be wasting some of it.

I would suggest using VLAN approach under a single bridge if your scenario can work with it. If your guest network isn't used very much, you could probably just put the queue on the home LAN bridge and get the benefits most of the time when guest network isn't used.
 
User avatar
BrateloSlava
Member Candidate
Member Candidate
Posts: 201
Joined: Mon Aug 09, 2021 10:33 am
Location: Ukraine, Kharkiv

Re: Using RouterOS to QoS your network - 2020 Edition