Hi,
This is a very interesting topic.
I want to use it on a transparent mikrotik device.
So no router.
Is that also possible?
And than make difference in up and download
if i hawe spf as WAN and all lan+ wifi are in bridge
then
Ether-WAN= SPF
Ether-LAN=Bridge right ?
RouterOS v6.36.4 working properly, then upgrade to 6.37.3. Bug fix confirmed
Thanks freemannnn
Sorry for my long absence here.
Here’s an updated version of the script (from another device I have deployed, so I think the interface names might be different than my first script and the connection speed is also different–adjust as necessary) that’s been running for >1y without any issues.
I do see the report that connection-rate may be fixed in a newer release, but I have not re-incorporated it into my scripts. Perhaps I will someday. For now, here is my latest operating script:
Date: March 19, 2017
Version: 1.4, based on work by pcunite
Tested with RouterOS 6.38.5
Rename ether1-gateway and bridge-local to match your environment
###############################################################################
Mangle
Using prerouting/postrouting since we don’t have dst or src checks.
/ip firewall mangle
###############################################################################
DNS requests. Mark in two places because DNS is sent out by the router (itself) too.
add action=mark-connection chain=prerouting comment=DNS connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=DNS passthrough=yes port=53 protocol=udp
add action=mark-packet chain=postrouting connection-mark=DNS new-packet-mark=DNS passthrough=no
Mark all VoIP traffic based on DSCP marking 46 and 26.
add action=mark-connection chain=prerouting comment=VOIP dscp=46 new-connection-mark=VOIP-TCP passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting dscp=46 new-connection-mark=VOIP-UDP passthrough=yes protocol=udp
add action=mark-connection chain=prerouting new-connection-mark=VOIP-UDP passthrough=yes port=4500 protocol=udp
add action=mark-connection chain=prerouting dscp=26 new-connection-mark=VOIP-TCP passthrough=yes protocol=tcp
add action=mark-connection chain=prerouting dscp=26 new-connection-mark=VOIP-UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=VOIP-TCP new-packet-mark=VOIP-TCP passthrough=no
add action=mark-packet chain=prerouting connection-mark=VOIP-UDP new-packet-mark=VOIP-UDP passthrough=no
Mark all UDP traffic. Mark different UDP streams if you want more granularity.
add action=mark-connection chain=prerouting comment=UDP connection-state=new new-connection-mark=UDP passthrough=yes protocol=udp
add action=mark-packet chain=prerouting connection-mark=UDP new-packet-mark=UDP passthrough=no
Ping replies. Mark in two places because ICMP is sent out by the router (itself) too.
add action=mark-connection chain=prerouting comment=ICMP connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-connection chain=postrouting connection-state=new new-connection-mark=ICMP passthrough=yes protocol=icmp
add action=mark-packet chain=postrouting connection-mark=ICMP new-packet-mark=ICMP passthrough=no
ACK traffic. Based on viewtopic.php?f=2&t=67965
add action=mark-packet chain=postrouting comment=ACK new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=prerouting new-packet-mark=ACK packet-size=0-123 passthrough=no protocol=tcp tcp-flags=ack
Mark all new HTTP(s) connections with “HTTP” if they have not previously been marked as “HTTP_BIG”.
If the current mark of “HTTP” tranfers more than 5MB and at a rate of 200k+ then mark it as “HTTP_BIG” for the duration of the TCP session.
add action=mark-connection chain=prerouting comment=HTTP connection-state=new new-connection-mark=HTTP passthrough=yes port=80,443,8080 protocol=tcp
add action=mark-connection chain=prerouting connection-bytes=500000-0 connection-mark=HTTP new-connection-mark=HTTP_BIG passthrough=yes protocol=tcp
add action=mark-packet chain=prerouting connection-mark=HTTP new-packet-mark=HTTP passthrough=no
add action=mark-packet chain=prerouting connection-mark=HTTP_BIG new-packet-mark=HTTP_BIG passthrough=no
Mark everything else that has no mark applied.
add action=mark-connection chain=prerouting comment=OTHER connection-mark=no-mark new-connection-mark=OTHER passthrough=yes
add action=mark-packet chain=prerouting connection-mark=OTHER new-packet-mark=OTHER passthrough=no
###############################################################################
HTB Queue Tree a unidirectional queue
Based on 50M down / 5M up connection (measured at 54000k/54.0M down, 5600k/5.6M up).
Replace with your own values as needed and adjust downward if connection is unreliable.
Notes:
priority means ‘drop packets’ WHEN needed.
When limit-at=0 priority starts when max-limit is reached.
When limit-at=123 priority starts when limit-at is reached.
The priority option applies to children not parents. Parent is for setting
overall limits. Therefore use limit-at and max-limit on the children if
you want more granularity.
max-limit must always be set or priority will not happen.
Tips for TCP (not VoIP) SOHO network:
limit-at = Total bandwidth / max hosts
max-limit = Total bandwidth / min hosts
/queue tree
###############################################################################
The secret to ensuring VoIP quality (or any UDP traffic) is to put it into
a queue that will never be full and thus never prioritize (drop) packets.
add max-limit=5500k name=MASTER_UP parent=ether1-gateway queue=default
add max-limit=53M name=MASTER_DOWN parent=bridge-local queue=default
add limit-at=200k max-limit=5M name=VOIP-TCP_U packet-mark=VOIP-TCP parent=MASTER_UP priority=1 queue=default
add limit-at=200k max-limit=45M name=VOIP-TCP_D packet-mark=VOIP-TCP parent=MASTER_DOWN priority=1 queue=default
add limit-at=200k max-limit=5M name=ACK_U packet-mark=ACK parent=MASTER_UP priority=2 queue=default
add limit-at=4M max-limit=45M name=ACK_D packet-mark=ACK parent=MASTER_DOWN priority=2 queue=default
add limit-at=200k max-limit=5M name=DNS_U packet-mark=DNS parent=MASTER_UP priority=3 queue=default
add limit-at=4M max-limit=45M name=DNS_D packet-mark=DNS parent=MASTER_DOWN priority=3 queue=default
add limit-at=200k max-limit=5500k name=UDP_U packet-mark=UDP parent=MASTER_UP priority=4 queue=default
add name=UDP_D packet-mark=UDP parent=bridge-local priority=4 queue=default
add limit-at=200k max-limit=5M name=ICMP_U packet-mark=ICMP parent=MASTER_UP priority=5 queue=default
add limit-at=4M max-limit=45M name=ICMP_D packet-mark=ICMP parent=MASTER_DOWN priority=5 queue=default
add limit-at=200k max-limit=5500k name=HTTP_U packet-mark=HTTP parent=MASTER_UP priority=6 queue=default
add limit-at=4M max-limit=53M name=HTTP_D packet-mark=HTTP parent=MASTER_DOWN priority=6 queue=default
add limit-at=200k max-limit=5500k name=HTTP_BIG_U packet-mark=HTTP_BIG parent=MASTER_UP priority=7 queue=default
add limit-at=4M max-limit=53M name=HTTP_BIG_D packet-mark=HTTP_BIG parent=MASTER_DOWN priority=7 queue=default
add limit-at=200k max-limit=5500k name=OTHER_U packet-mark=OTHER parent=MASTER_UP queue=default
add limit-at=4M max-limit=53M name=OTHER_D packet-mark=OTHER parent=MASTER_DOWN queue=default
add name=VOIP-UDP_D packet-mark=VOIP-UDP parent=bridge-local priority=1 queue=default
add limit-at=200k max-limit=5M name=VOIP-UDP packet-mark=VOIP-UDP parent=MASTER_UP priority=1 queue=default
Note that the limit-at amounts could probably use some playing around with, and (as I mentioned above) connection-rate could probably be re-implemented. But this ruleset seems to be working pretty bulletproof for me–allowing me access to very, very near my max possible throughput while preventing any sort of connection issues on VoIP calls.
Can i implement PCQ on that script? i mean..for e.g.
add limit-at=200k max-limit=5M name=ACK_U packet-mark=ACK parent=MASTER_UP priority=2 queue=PCQ_U
And also i would like to specified any IP_Subnet for e.g. (address list) can i do this?
if it is posible will be exselent 
)
Thanks a lot
just I’m wondering it’ possible to do, on second part of QoS on this post, i wold like to do with Address List.Can anyone help me?
Thanks
@alaskanjackal your scripts are incredible.
How would I implement it with 2 WANs both with different speeds?
Thank you
I also do not understand how LEVEL_A_ gets priority over LEVEL_B_ and then over LEVEL_C_.
Would somebody be so kind explain this?
In my opinion, the only way to understand how the traffic prioritization works is to try to implement a simple one on your own and debug it, not to try to understand someone else’s complex example like the one above. Once you understand how it works using your simple case, reading the complex example becomes much easier.
So as I wrote in the other thread, you have to prioritize VoIP traffic among everything you send through the uplink. To do that, you have to set a DSCP mark (better to use the standard one, i.e. 46, for VoIP media and maybe also signalling) as DSCP is the only attribute of the packet which remains unchanged after IPsec encryption and encapsulation.
To answer your question - when talking about the classifier rules in the example above, the IPSec transport packets would definitely fall into the OTHER group, while the packets before encryption and encapsulation could fall into any of the groups depending on the criteria. So if you would use that script without modification of the classifier rules, you would prioritize VoIP traffic among everything that is going to be sent encrypted, but that would be useless because you would give the lowest (OTHER) priority to all the encrypted traffic.
So the whole implementation would have to include
- for the upload direction:
- mangle rules setting DSCP value for anything which is going to be sent via the IPsec tunnel (it is possible that the phones already do that, but you also need to eventually remove priority marking from other than VoIP packets should it eventually be set on them)
- mangle rules marking connections and packets going to be sent via the uplink, including the IPsec transport packets, based on DSCP and other criteria
- queue setup prioritizing the packets going to be sent via the uplink based on the marks assigned in the step above
- for the download direction:
- mangle rules marking connections and packets received based on protocol type and, possibly, DSCP marking (if it survives transit through internet as stated earlier)
- queue setup throttling the low priority packets in such a way that the necessary amount of download bandwidth is kept free for the priority packets.
The download margin you have to reserve for the priority packets will differ significantly depending on whether the DSCP markings survive the transit or not. If they do, you may keep the margin only as big as to accommodate the incoming VoIP traffic; if they don’t, you have to limit the IPsec transport bandwidth as a whole on the remote site upload and reserve enough space for all of it at local download because in such case, there is no way to freely distribute the “non-VoIP” download bandwidth between the tunnelled and non-tunnelled traffic.
I think I will forget about putting VoIP into VPN, this definitely sounds to complicated. And I will try to tune the example I have already implemented. Just after I understood how LEVEL_A_ gets priority over LEVEL_B_ and then over LEVEL_C_
Funny. To me the idea of setting the DSCP field doesn’t sound complicated at all as it is basically a single mangle rule. What may be complicated is to distinguish VoIP packets from non-VoIP ones but if you reserve an address range or subnet on each site for your VoIP devices, or if you can ask the VoIP devices to set the DSCP themselves, it is also not very complex. Softphones are a problem unless they can set DSCP themselves because they cannot be identified by IP address and the PCs they run at generate both kinds of traffic.
I mean, once you pushstart the prioritization as such, adding VoIP needs just a tiny bit compared to the overall effort.
What I still have to try is to practically deploy the queues, I didn’t have the need so far.
But I’ve had a look at the setup with LEVEL_X and it seems to me that LEVEL_A and LEVEL_B have no priority over each other, they simply each have their dedicated share of the total bandwidth as they have a common parent queue which doesn’t have any bandwidth limits set. So both LEVEL_A and LEVEL_B may reach their limits simultaneously, so the summary traffic may be 7800k up and 82M down, but none of them may “borrow” the other one’s bandwidth even if it is completely unused. So GROUP_A and GROUP_B would be more appropriate names.
Looks like my PBX boxes are adding DSCP = 40 already, I just afraid that by putting this traffic into VPN will add unnecessary overheat and I will end up with having bigger disaster that I have now.
So to make LEVEL_B and LEVEL_C always lower priority than LEVEL_A I will change parent of B and C to A?
Do you use VoIP phones or analog/ISDN phones and only the PBXes would talk VoIP to each other? If VoIP phones, do the PBXes tell the phones to send media to each other directly or do they force themselves into the media path? All that plays a role - if VoIP phones send media directly to each other, they must set DSCP or you must do it on their behalf when forwarding the packets to the remote site.
So to make LEVEL_B and LEVEL_C always lower priority than LEVEL_A I will change parent of B and C to A?
Yes but not only, you must also give the LEVEL_B queue the lowest priority among all the other child queues of LEVEL_A. And LEVEL_C must have the lowest priority among all child queues of LEVEL_B.
The queues play two roles - one is speed limitation and another one is prioritization. Speed limit doesn’t allow more data to be sent even if bandwidth is available. Priority determines the order of searching through the queues at each level for the next packet to be sent once the “line” gets free after the previous packet has been sent.
On the upload, it is basically enough for QoS to do the prioritization, speed limitation is only necessary if you want to guarantee some minimum bandwidth to individual users.
On the download, you cannot actually affect the priority with which the ISP is sending the packets down your connection, so you must make use of the TCP’s flow control - if you limit the speed with which received TCP packets are delivered from WAN to LAN, the remote senders limit their sending speed accordingly as they wait for ACKs from the recipients before sending another burst of packets. There is no point in throttling incoming UDP as it has no effect on the sender. So the trick is to limit the bandwidth for TCP in download direction and leave the rest for the peak amount of UDP you can expect (which is some 100k per audio RTP stream, but you have to inspect your traffic to see what else on top of RTP, DNS responses, NTP responses arrives to you as UDP). You’ll likely also see some icmp there.
I also have ipsec s2s not for the purpose of VoIP, to completely different site. It is mostly RDP.
Since IPsec s2s is also UDP (I think), how I can put it “to not disturb” my VoIP but not to go into LEVEL_C / OTHER?
That’s the role of DSCP as mentioned in the other (s2s) thread. The “plaintext” traffic on the WAN can be classified as “TCP” and “other protocols” directly; the IPsec transport packets will either be ESP ones or UDP ones (depending on existence of NAT in the path) with a particular source and destination; whether “TCP” or “other” is encrypted inside them can only be determined if the sending side sets the DSCP field and the transit through internet doesn’t destroy it completely.
On upload, you have to send the packets to the IPsec machinery without any throttling, only set their DSCP. The outgoing prioritization will take place on the plaintext and IPsec transport packets together, where the DSCP shows to which group or level the IPsec transport packets belong.
On download, there is a special situation - the IPsec transport packet and its decapsulated an decrypted payload are treated as two separate packets which came in via the same interface. As you do not want to queue the same actual packet twice, you can
- either inspect the remainders of DSCP (if any) on the IPsec transport packets and choose a queue for them on that basis, but in such case you have to exclude from the queueing the decapsulated an decrypted payload
- or exclude from queueing the IPsec transport packets, but in such case you have to inspect the protocol type and other attributes of the decapsulated and decrypted packets and choose the queue for them on that basis
I am afraid that “exclude from queueing” actually means to put them into a topmost queue with highest priority and no speed limits.
Classification of IPsec transport packets by DSCP in download saves CPU because the classification rules are simpler as the class has already been determined by the sending Mikrotik.
Should LEVEL_A as a parent of everything have
priority=1
?
Does priority is valid only among members of the same branch. So all children in sub branch will always step down to traffic if they parent priority is smaller than priority of an other branches on the same level. Does not mutter they have 1 but there is another purrent that have 7 ant they parrent has 8?
Regarding bandwidth, how to make C not to stal everything from B, and then B not to eat everything of A.
Do I need to change theirs max-limit to be smaller from each other?
and the example of proper “do not disturb my VoIP” should look like:
add name="LEVEL_A_UP" parent=ether-WAN queue=default max-limit=900k [b]priority=1
[/b]add name="LEVEL_A_DOWN" parent=bridge-LAN queue=default max-limit=4M [b]priority=1
[/b]add name="VOIP_U" parent="LEVEL_A_UP" packet-mark="VOIP" queue=default priority=1
add name="VOIP_D" parent="LEVEL_A_DOWN" packet-mark="VOIP" queue=default priority=1
add name="LEVEL_B_UP" parent=LEVEL_A_UP queue=default max-limit=[b]something smaler than parent A[/b] [b]priority=2
[/b]add name="LEVEL_B_DOWN" parent=LEVEL_A_DOWN queue=default max-limit=something smaler than parent A [b]priority=2
[/b]
add name="ACK_U" parent="LEVEL_B_UP" packet-mark="ACK" queue=default priority=1
add name="ACK_D" parent="LEVEL_B_DOWN" packet-mark="ACK" queue=default priority=1
add name="SOMETHING" parent="LEVEL_B_UP" packet-mark="SOMETHING" queue=default [b]priority=7
[/b]add name="SOMETHING" parent="LEVEL_B_DOWN" packet-mark="SOMETHING" queue=default [b]priority=7
[/b]
add name="LEVEL_C_UP" parent=[b]LEVEL_B_UP[/b] queue=default max-limit=[b]something smaler than parent B[/b] [b]priority=8
[/b]add name="LEVEL_C_DOWN" parent=[b]LEVEL_B_UP[/b] queue=default max-limit=[b]something smaler than parent B[/b] [b]priority=8
[/b]
add name="HTTP_U" parent="LEVEL_C_UP" packet-mark="HTTP" queue=default [b]priority=1
[/b]add name="HTTP_D" parent="LEVEL_C_DOWN" packet-mark="HTTP" queue=default [b]priority=1
[/b]
add name="OTHER_U" parent="LEVEL_C_UP" packet-mark="OTHER" queue=default priority=8
add name="OTHER_D" parent="LEVEL_C_DOWN" packet-mark="OTHER" queue=default priority=8
I don’t like to provide theoretical-only answers and as I’ve stated several times, I didn’t have a strong enough reason to test this practically so far. So already my previous response was wrong, as the manual says the following:
priority (1..8 ) : Prioritize one child queue over other child queue. Does not work on parent queues (if queue has at least one child). One is the highest, eight is the lowest priority. Child queue with higher priority will have chance to reach its max-limit before child with lower priority. Priority have nothing to do with bursts.
I have assumed that the priority works among all children of the same queue and recursively, but it seems not to be the case. So either you have to split the total available banwidth into subbands which do not affect each other, or you have to manage everything within the 8 priority levels of the common band. I just remind that in download direction there is basically no other way than throttling the traffic which can be throttled to leave a “fast lane” always free for the real-time traffic.
Did anyone figure out MPLS/VPLS QoS using DSCP/EXP bits?
This two rules are not the same… src/dst address are in use