I've been combing the Wiki and searching the forum for days. The only thing I can find is that the WiFi drivers will use the Priority marking to make WMM decisions. Do any of the Queue Types available in ROS respect the Priority mark in any way? I need to be able to demote the priority of a certain class of traffic as it passes through the queues. Right now I add Packet Marks and use little HTB trees for each customer. This works for static IP customers, but not for PPPoE customers because their queues are created dynamically. If one of the Queue Types respected Priority, then I could set the Priority and tell the PPPoE server to create Queues of that type.
The Linux default Queue, pfifo_fast, has 3 bands that are emptied in a strict priority fashion. Packets are queued to the bands based on a Priority Map. Although I would rather have something more intelligent than a FIFO queue, it's more important to squash the obnoxious traffic. So, just getting access to pfifo_fast would do the job.
Re: Do any queue types respect Priority markings?
Sorry but no, you have to tell it: http://forum.mikrotik.com/viewtopic.php ... 21#p580321
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
OK. So I've seen multiple mentions of creating a queue tree for priority, and then shaping in simple queues. As an ISP, I fail to see how this could work. I have identified an obnoxious service (MSDO) that will forcibly consume all of the bandwidth for a given customer when it fires up. But since MSDO is used for updates, I can't just ban it. If I demote it so all other traffic has priority, everything is fine.
I'm having trouble seeing how I can make two layers of queues work correctly. Lets say I have customer A and customer B. Both connect through PPPoE, so their bandwidth is handled by dynamically created simple queues. A new update comes out, so MSDO fires up. Customer A is streaming video, so MSDO should only get whatever portion of customer A's bandwidth the video stream doesn't need. Customer B is not using the internet at the moment, so MSDO should be allowed to consume all of customer B's bandwidth. How can a queue tree in HTB Global restrict the flow of MSDO packets into customer A's simple queue while letting customer B's MSDO packets flow freely?
I'm having trouble seeing how I can make two layers of queues work correctly. Lets say I have customer A and customer B. Both connect through PPPoE, so their bandwidth is handled by dynamically created simple queues. A new update comes out, so MSDO fires up. Customer A is streaming video, so MSDO should only get whatever portion of customer A's bandwidth the video stream doesn't need. Customer B is not using the internet at the moment, so MSDO should be allowed to consume all of customer B's bandwidth. How can a queue tree in HTB Global restrict the flow of MSDO packets into customer A's simple queue while letting customer B's MSDO packets flow freely?
Re: Do any queue types respect Priority markings?
Just curious, what does "MSDO" stand for?
As regards the other, just set the minimal queue-rate for A and the rest of the bandwith will be available for B. If nothing happens at A, then B will get it all. If the bandwidth requirement is changing during the day you probably need some scripting to alter the values..
Something like that?
As regards the other, just set the minimal queue-rate for A and the rest of the bandwith will be available for B. If nothing happens at A, then B will get it all. If the bandwidth requirement is changing during the day you probably need some scripting to alter the values..
Something like that?
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
MSDO - Microsoft Delivery Optimization
Customer A has a 3Mbps account and customer B has a 2Mbps account. Their bandwidth is not shared. They each get a simple queue created dynamically when their PPPoE Connection is authenticated through RADIUS. Because they are dynamic, I can't do more than pick what Queue Type will be used.
You can probably see my conundrum here. How can the HTB Global Queue Tree prioritize packets if it doesn't know the customer's speed? As far as I can tell, the only solution that doesn't require a major kludge of a headache is to find a Queue Type that respects the Priority marks (SKB->Priority).
Customer A has a 3Mbps account and customer B has a 2Mbps account. Their bandwidth is not shared. They each get a simple queue created dynamically when their PPPoE Connection is authenticated through RADIUS. Because they are dynamic, I can't do more than pick what Queue Type will be used.
You can probably see my conundrum here. How can the HTB Global Queue Tree prioritize packets if it doesn't know the customer's speed? As far as I can tell, the only solution that doesn't require a major kludge of a headache is to find a Queue Type that respects the Priority marks (SKB->Priority).
Re: Do any queue types respect Priority markings?
Well, you know that to have priority work it needs a reference, meaning queue1 has higher priority to queue2. That's why the queues need to be related as in queue tree. When they re not related, it is not possible to apply priority.
In case of pppoe users, do not set limits on the profile of users, instead put them on an address-list. By not setting limits, it will not create a simple queue. Then using address-list you can create mangle rules to use them in queue tree.
Or, in the profile of the pppoe user you have the option to set its parent queue. I don't know to tell radius about this, you should contact who built that radius and ask if such thing can be made from radius.
In case of pppoe users, do not set limits on the profile of users, instead put them on an address-list. By not setting limits, it will not create a simple queue. Then using address-list you can create mangle rules to use them in queue tree.
Or, in the profile of the pppoe user you have the option to set its parent queue. I don't know to tell radius about this, you should contact who built that radius and ask if such thing can be made from radius.
Re: Do any queue types respect Priority markings?
Good idea, worth checking out! And if you are lucky it might be possible to apply some kind of RADIUS custom attributes.I don't know to tell radius about this, you should contact who built that radius and ask if such thing can be made from radius.
Links to MikroTik Specific RADIUS Attributes
http://wiki.mikrotik.com/wiki/Manual:RADIUS_Client
http://wiki.mikrotik.com/wiki/Manual:RA ... dictionary
I assume you mean Windows Update Delivery Optimization. Then it might be worth examine if the customer is channeling all the updates through their own Windows Server Update Services (WSUS) and if it's properly configured to download just necessary updates. Windows 10 is much smarter nowadays but Window 7/8 can be pretty stupid to handle Windows updates thus I strongly recommend to use WSUS when the bandwidth is limited. With WSUS it's also easy to apply bandwidth throttling and limit active download hours. WSUS is included in Windows Server but it's quite easy to actually tweak it to be installed on a windows client.MSDO - Microsoft Delivery Optimization
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
I do know that Queue Priority is relative to other queues under a parent queue, and that is how I manage this for static IP customers. Identify and mark MSDO Connection - Apply packet marks to all packets in the marked connection - create 2 sub-queues of the customers simple queue, one for marked packets at queue priority 8 and one for all else at queue priority 6. But you can't have the Mikrotik dynamically create these little queue trees. I do not wish to have any kind of priority relationship between Customer A and Customer B. I only want a priority relationship between Customer A's MSDO packets and all of Customer A's other packets, completely independent of Customer B's packets.Well, you know that to have priority work it needs a reference, meaning queue1 has higher priority to queue2. That's why the queues need to be related as in queue tree. When they re not related, it is not possible to apply priority.
In case of pppoe users, do not set limits on the profile of users, instead put them on an address-list. By not setting limits, it will not create a simple queue. Then using address-list you can create mangle rules to use them in queue tree.
Or, in the profile of the pppoe user you have the option to set its parent queue. I don't know to tell radius about this, you should contact who built that radius and ask if such thing can be made from radius.
There is another Priority that is attached to each packet via the packets SKB structure. See IP/Firewall/Mangle action set-priority for how it's set. Linux, which ROS is based off of, respects this Packet Priority value in it's pfifo-fast classless queuing discipline. So if I could tell ROS to create dynamic simple queues using pfifo-fast for the queue type, then instead of marking the packets with Packet Marks, I could set the Packet Priority and get the demotion of the MSDO packets that I'm after. If PCQ queues used a Packet Priority aware internal queue, then I could potentially use your address-list suggestion as recommended in the http://mum.mikrotik.com/presentations/C ... _Megis.pdf. But even MicroTik doesn't seem to have a solution for my problem. In the QoS_Megis.pdf it just says "Create packet marks in the mangle chain 'Prerouting' for traffic prioritization in the global-in queue" without even giving a hint as to how to integrate it into the queuing structure they just recommended ISPs use.
So I do appreciate your trying to help, but I can't see daylight in any of your suggestions so far.
Re: Do any queue types respect Priority markings?
You mentioned that customers are dynamic and thus you can't apply the queue to your needs, so I offered an idea how to avoid dynamic queues and apply what you need. If you further need to prioritize MSDO connection you can identify it by mangle rules if it can be separated from other traffic.They each get a simple queue created dynamically when their PPPoE Connection is authenticated through RADIUS. Because they are dynamic, I can't do more than pick what Queue Type will be used.
I don't know how linux works, so if you are looking for exact feature as in linux I will leave it here since I am not knowledgeable at all on that.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
If I can figure out the queuing issue, those URLs could be valuable. Thanks.Good idea, worth checking out! And if you are lucky it might be possible to apply some kind of RADIUS custom attributes.
Links to MikroTik Specific RADIUS Attributes
http://wiki.mikrotik.com/wiki/Manual:RADIUS_Client
http://wiki.mikrotik.com/wiki/Manual:RA ... dictionary
Nope, it's Microsoft Delivery Optimization, check out the description of the MSDO service. It replaces the Background Intelligent Transfer Service(BITS) that was used by every version of windows since XP, until Windows 10. In the first release of Windows 10 they were still using BITS for updates and only used MSDO for the Windows Store. But in preparation for the Anniversary update they switched to MSDO for Windows Updates. On Windows 10 Pro and higher you can tell the computer to go back to BITS, but not on Home. The majority of our customers are residential, and there's hundreds of them, so WSUS is not a feasible solution.I assume you mean Windows Update Delivery Optimization. Then it might be worth examine if the customer is channeling all the updates through their own Windows Server Update Services (WSUS) and if it's properly configured to download just necessary updates. Windows 10 is much smarter nowadays but Window 7/8 can be pretty stupid to handle Windows updates thus I strongly recommend to use WSUS when the bandwidth is limited. With WSUS it's also easy to apply bandwidth throttling and limit active download hours. WSUS is included in Windows Server but it's quite easy to actually tweak it to be installed on a windows client.MSDO - Microsoft Delivery Optimization
Understand that MSDO is a P2P network client ala the old Napster. It can be told not to share or download from other clients, but even then it will still open dozens of connections to the Microsoft download servers. It can't be told to limit it's bandwidth or number of connections. And just like the old Napster, when it fires up it crowds out all the other connections on the customers network. I've even seen Windows 10 machines that were downloading updates claim that there is no Internet connectivity in the task bar at the same time that Performance in the Task Manager shows the computer is downloading at the customers full rate. MSDO has lost us customers, because it makes the customer think our Internet is unreliable. It has to be tamed. If Microsoft would tame it, then it would actually be a boon as it can share updates with other machines on the local LAN. But as it is, its firmly in the necessary evil category, like Blizzard choosing to use BitTorrent to distribute their updates. Only modern BitTorrent is a lot smarter than MSDO.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
Lets ignore Linux for now and talk about your suggestions. Perhaps you have discovered something I'm unaware of. I've only mentioned Customer A and Customer B, but there's actually hundreds of customers I need to fix this for. I have no problems identifying the traffic that needs to be demoted. It's how to actually demote it that's frustrating me. For our static IP based customers I have:You mentioned that customers are dynamic and thus you can't apply the queue to your needs, so I offered an idea how to avoid dynamic queues and apply what you need. If you further need to prioritize MSDO connection you can identify it by mangle rules if it can be separated from other traffic.They each get a simple queue created dynamically when their PPPoE Connection is authenticated through RADIUS. Because they are dynamic, I can't do more than pick what Queue Type will be used.
I don't know how linux works, so if you are looking for exact feature as in linux I will leave it here since I am not knowledgeable at all on that.
Code: Select all
add max-limit=0/10M name="Customer H" queue=default-small/synchronous-default target=172.29.236.40/32
add limit-at=0/256k max-limit=0/10M name="Customer H-Bulk" packet-marks=ToS-Bulk parent="Customer H" \
queue=default-small/ethernet-default target=172.29.236.40/32
add limit-at=0/768k max-limit=0/10M name="Customer H-Other" parent="Customer H" priority=6/6 \
queue=default-small/synchronous-default target=172.29.236.40/32
The problem I have is how to do the equivalent for customers who are on dynamic IP services like PPPoE. Each customer's bandwidth needs to be independent of the other customers. For each customer, all their other packets need to come before ToS-Bulk packets.
Re: Do any queue types respect Priority markings?
That is a lot of simple queues you would need to create for each customer. Mikrotik says that they have improved a lot the performance of simple queues, but I haven't tried it in real world since I am a lot more comfortable with queue tree. The good thing about queue tree is that all queues are treated at the same time, while with a simple queue the packet must check them all in their order until it matches the one which deals with it.
I think that there must be customers who are on the same bandwidth plan, say you got to have 10 customers who are on 10/10Mbps. If you have such groups, you can set the profile of pppoe with no limits and add them to address-list. Then use address-list to identify the MSDO coming-going from this address-list, and the other traffic. Once you have marked packets, you can apply them on queue-tree with one parent equivalent at the total of the group and then children queues with pcq to distribute it equally to the customers of the group. You can save a lot of queues this way, freeing up resources of the router.
I see you have no problem to identify the traffic, so can't see why not use address-list generated from the pppoe profile. Once a customers connects through pppoe, no matter what IP will be assigned, he will be added to the specified address-list, and with no limits on pppoe profile, no dynamic simple queue will be created, thus moving them on the queue tree.
I think that there must be customers who are on the same bandwidth plan, say you got to have 10 customers who are on 10/10Mbps. If you have such groups, you can set the profile of pppoe with no limits and add them to address-list. Then use address-list to identify the MSDO coming-going from this address-list, and the other traffic. Once you have marked packets, you can apply them on queue-tree with one parent equivalent at the total of the group and then children queues with pcq to distribute it equally to the customers of the group. You can save a lot of queues this way, freeing up resources of the router.
I see you have no problem to identify the traffic, so can't see why not use address-list generated from the pppoe profile. Once a customers connects through pppoe, no matter what IP will be assigned, he will be added to the specified address-list, and with no limits on pppoe profile, no dynamic simple queue will be created, thus moving them on the queue tree.
Re: Do any queue types respect Priority markings?
Ok, I was totally wrong there who assumed you meant a company that was hooked up but now I see your point. Yeah, that kind of p2p traffic might be a beast in that regards. Is it possible at all to identify any kind of patterns or port numbers that relates to MSDO so it might be down-prioritized? It seem that you gonna need a pretty complex kind of L7 content analysis in order to find out when MSDO kicks in..... The majority of our customers are residential, and there's hundreds of them, so WSUS is not a feasible solution...
Re: Do any queue types respect Priority markings?
Oh, you've already found a way how to identify MSDO traffic. Do you mind sharing how it's done?Notes: I mark the MSDO packets with ToS-Bulk...
Re: Do any queue types respect Priority markings?
a two seconds search in RavenWing71 posts would have told you that: http://forum.mikrotik.com/viewtopic.php ... 64#p577464Oh, you've already found a way how to identify MSDO traffic. Do you mind sharing how it's done?Notes: I mark the MSDO packets with ToS-Bulk...
Re: Do any queue types respect Priority markings?
To my knowledge, this is totally wrong. The current implementation if Simple Queues uses hash-table lookup, which is very fast and scales up really well.The good thing about queue tree is that all queues are treated at the same time, while with a simple queue the packet must check them all in their order until it matches the one which deals with it.
Re: Do any queue types respect Priority markings?
Well, as I said, I haven't used simple queues in very long time since I understood how queue tree works. But, for clarification, here is what Mikrotik wiki says (unless they haven't updated it):To my knowledge, this is totally wrong. The current implementation if Simple Queues uses hash-table lookup, which is very fast and scales up really well.The good thing about queue tree is that all queues are treated at the same time, while with a simple queue the packet must check them all in their order until it matches the one which deals with it.
http://wiki.mikrotik.com/wiki/Manual:Qu ... ple_QueuesSimple queues have a strict order - each packet must go through every queue until it reaches one queue which conditions fits packet parameters or until the end of queues list is reached. (In case of 1000 queues, packet for last queue will need to proceed through 999 queues before it will reach the destination)
Re: Do any queue types respect Priority markings?
Well, I'm not that well educated forum user but thank you anyhow!a two seconds search in RavenWing71 posts would have told you that: http://forum.mikrotik.com/viewtopic.php ... 64#p577464
Just a thought - will that also capture the actual p2p traffic that is initiated in a step further on in the update process?
Secondly, last time I checked it was ssl/tls that was utilized for most of the ms related services. This makes it hard to manage L7 filters and isn't a long term solution since it will become more or less impossible in the future when everything has been moved over to ssl. The update process will also continue to evolve with new application protocols.
It will probably be necessary for Microsoft to offer some way to cap service traffic in order to handle this type of obstacles. In any case, I would suggest that someone opens a support case at some of the microsoft forums to make them aware that not everyone is sitting on a high speed wan link...
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
My packet sniffing, crossed with internet research, many years of internet servicing, and dabbling in programming, gives me this rough framework for MSDO:Well, I'm not that well educated forum user but thank you anyhow!a two seconds search in RavenWing71 posts would have told you that: http://forum.mikrotik.com/viewtopic.php ... 64#p577464
Just a thought - will that also capture the actual p2p traffic that is initiated in a step further on in the update process?
Secondly, last time I checked it was ssl/tls that was utilized for most of the ms related services. This makes it hard to manage L7 filters and isn't a long term solution since it will become more or less impossible in the future when everything has been moved over to ssl. The update process will also continue to evolve with new application protocols.
It will probably be necessary for Microsoft to offer some way to cap service traffic in order to handle this type of obstacles. In any case, I would suggest that someone opens a support case at some of the microsoft forums to make them aware that not everyone is sitting on a high speed wan link...
It uses HTTPS for the command and control aspects and downloading the catalog with it's security checksums. The actual P2P downloading of the files is done through HTTP, allowing the L7 matcher to identify the P2P connections. Microsoft realized HTTPS incurs too much overhead, and since MSDO only downloads fragments of the file from any one source, someone sniffing in the middle would not get anything useful. The final assembled download gets checked against the checksum in the catalog to verify it's authenticity.
So with my L7 filter and queue structure I don't end up demoting the command and control, just the actual file downloads.
Last edited by RavenWing71 on Fri Feb 03, 2017 10:23 pm, edited 2 times in total.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
Yes it is a lot of queues. 3 per customer. I have nearly 900 simple queue entries and I haven't finished adding the bulk and other queues to all the customers yet.That is a lot of simple queues you would need to create for each customer. Mikrotik says that they have improved a lot the performance of simple queues, but I haven't tried it in real world since I am a lot more comfortable with queue tree. The good thing about queue tree is that all queues are treated at the same time, while with a simple queue the packet must check them all in their order until it matches the one which deals with it.
I'm glad to hear this. It settles my worry about all these queues.The current implementation if Simple Queues uses hash-table lookup, which is very fast and scales up really well.
This sounds like a variation on what is recommended by the QoS_Megis.pdf. There must be something I'm missing about how queue trees work. I could mark packets as 10/10 and 10/10-Bulk and then they would go into different branches of the tree, but if I make a PCQ for each mark how would it know that 172.29.236.40 only gets 10Mbps total and not 10Mbps for each mark since they are separate PCQs? Can I branch off of a PCQ, making two fifo queues beyond the PCQ, one for each mark? What do I really get if I do? Would it then make 2 fifo sub-queues for each dynamically generated queue within the PCQ? If it did, then this would work, otherwise it's just a mess. Perhaps you could mock up a queue tree that does what you are suggesting?I think that there must be customers who are on the same bandwidth plan, say you got to have 10 customers who are on 10/10Mbps. If you have such groups, you can set the profile of pppoe with no limits and add them to address-list. Then use address-list to identify the MSDO coming-going from this address-list, and the other traffic. Once you have marked packets, you can apply them on queue-tree with one parent equivalent at the total of the group and then children queues with pcq to distribute it equally to the customers of the group. You can save a lot of queues this way, freeing up resources of the router.
I see you have no problem to identify the traffic, so can't see why not use address-list generated from the pppoe profile. Once a customers connects through pppoe, no matter what IP will be assigned, he will be added to the specified address-list, and with no limits on pppoe profile, no dynamic simple queue will be created, thus moving them on the queue tree.
Re: Do any queue types respect Priority markings?
There are two way to configure PCQ using pcq-rate, one is to give full bandwidth to the child available from its parent, the other is giving no more than specified in pcq-rate even if parent might have more available. The parent will have a total of the group, say 100Mbps for 10 users with 10Mbps plan. The pcq will distribute it equally to the destination IPs. So if two of them are connected and if pcq-rate=0 it will give both of them 50. When pcq-rate=10Mbps and two of them are connected it will give each 10Mbps, which means you will be using 20Mbps from the total of parent queue.Yes it is a lot of queues. 3 per customer. I have nearly 900 simple queue entries and I haven't finished adding the bulk and other queues to all the customers yet.That is a lot of simple queues you would need to create for each customer. Mikrotik says that they have improved a lot the performance of simple queues, but I haven't tried it in real world since I am a lot more comfortable with queue tree. The good thing about queue tree is that all queues are treated at the same time, while with a simple queue the packet must check them all in their order until it matches the one which deals with it.I'm glad to hear this. It settles my worry about all these queues.The current implementation if Simple Queues uses hash-table lookup, which is very fast and scales up really well.This sounds like a variation on what is recommended by the QoS_Megis.pdf. There must be something I'm missing about how queue trees work. I could mark packets as 10/10 and 10/10-Bulk and then they would go into different branches of the tree, but if I make a PCQ for each mark how would it know that 172.29.236.40 only gets 10Mbps total and not 10Mbps for each mark since they are separate PCQs? Can I branch off of a PCQ, making two fifo queues beyond the PCQ, one for each mark? What do I really get if I do? Would it then make 2 fifo sub-queues for each dynamically generated queue within the PCQ? If it did, then this would work, otherwise it's just a mess. Perhaps you could mock up a queue tree that does what you are suggesting?I think that there must be customers who are on the same bandwidth plan, say you got to have 10 customers who are on 10/10Mbps. If you have such groups, you can set the profile of pppoe with no limits and add them to address-list. Then use address-list to identify the MSDO coming-going from this address-list, and the other traffic. Once you have marked packets, you can apply them on queue-tree with one parent equivalent at the total of the group and then children queues with pcq to distribute it equally to the customers of the group. You can save a lot of queues this way, freeing up resources of the router.
I see you have no problem to identify the traffic, so can't see why not use address-list generated from the pppoe profile. Once a customers connects through pppoe, no matter what IP will be assigned, he will be added to the specified address-list, and with no limits on pppoe profile, no dynamic simple queue will be created, thus moving them on the queue tree.
Now, when you have two children queues, which we are trying to do, I would put the two under another child queue:
Code: Select all
/queue tree
add name="download" parent=global packet-mark="" limit-at=0 queue=default priority=8 max-limit=100M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
add name="customer_download" parent=Download packet-mark=customer_down limit-at=0 queue=PcqDown priority=8 max-limit=100M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
add name="other_down" parent=customer_download packet-mark=other_down limit-at=2M queue=PcqDown priority=4 max-limit=100M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
add name="MSDO" parent=customer_download packet-mark=MSDO limit-at=1M queue=PcqDown priority=8 max-limit=95M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1But I believe that this will work, since the first child will separate the band for each IP, and then the second child can not get more from the what the first (his parent) delivers, which is 10, and within that 10 you will assign priority between MSDO and other traffic.
I have attached the main parent at the global interface assuming that in mangle you take care of marking traffic by specifying in-interface and out-interface among others, otherwise you can attach it to the WAN interface for download. I have also put a lower total-limit on the MSDO queue, since we consider it as heavy eating all the available bandwidth, so I leave a small window of 5Mbps in order to allow normal traffic to sneak in and then get the priority. It is as if you have a 100 people going through one single door, but the one with high priority is at the back, you would leave a small alley for him to avoid the queue and can go through as soon as he arrives.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
OK, so I implemented your suggested Queue Tree with little modifications to fit my setup and understanding. I have added a mangle rule that marks all packets not marked ToS-Bulk as ToS-Other. I choose 2M instead of 10M to make the downloads take longer, giving me more time to inspect results. I couldn't understand your third packet mark of Customer-Down. I can only apply one packet mark to any one packet, so I changed it to be both packet marks in the higher queue.
To test, I started downloading an app in the Microsoft Store. Once the Queue for ToS-Bulk was showing a rate of 2Mbps I started a test on Speedtest.net. Speedtest.net gave me a download result of approximately 2Mbps. But looking at the Ethernet speed in Task Manager, the port was receiving 4Mbps during the test, as you can see in this linked image. https://1drv.ms/i/s!AhME_4RqytbZjAyw052Q3eQTituS
So, what's happening? My guess based on my Linux knowledge of HTB, and the diagrams MikroTik has published, is this... When you create a tree structure with HTB, only the leaf nodes (Nodes that have no children) actually have a queue. In our test tree, the leaves are MSDO and other_down. All nodes that are a parent to some other node, have no actual queue, they only have a token bucket. In our example, customer_download and download are parent nodes. This means that customer_download never actually instantiates it's PCQ, so it doesn't limit the individual customer's speed.
Code: Select all
/queue tree
add max-limit=100M name=download parent=ether5 queue=default
add max-limit=100M name=customer_download packet-mark=ToS-Bulk,ToS-Other parent=download queue=2M-PCQ
add limit-at=400k max-limit=100M name=other_down packet-mark=ToS-Other parent=customer_download priority=4 \
queue=2M-PCQ
add limit-at=200k max-limit=95M name=MSDO packet-mark=ToS-Bulk parent=customer_download queue=2M-PCQ
So, what's happening? My guess based on my Linux knowledge of HTB, and the diagrams MikroTik has published, is this... When you create a tree structure with HTB, only the leaf nodes (Nodes that have no children) actually have a queue. In our test tree, the leaves are MSDO and other_down. All nodes that are a parent to some other node, have no actual queue, they only have a token bucket. In our example, customer_download and download are parent nodes. This means that customer_download never actually instantiates it's PCQ, so it doesn't limit the individual customer's speed.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
I do agree with this point of view in general. But when a pervasive company like Microsoft makes such a blunderous change in a critical service, just telling the customer that it's Microsoft's fault doesn't sit real well.Have to admit, I haven't tried this kind of configuration with children inside child because I never worry how customer uses his bandwidth, i just worry to deliver him what he requests. If he chooses to eat all bandwidth with torrent, so be it.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
Ok, so are there any other ideas as to how I would prioritize all other traffic ahead of MSDO for dynamic services like PPPoE on a per-customer basis?
Re: Do any queue types respect Priority markings?
What you might be able to do is use the PPPoE on-up and on-down script to locate the dynamic simple queue that was added and automatically add child queues to this to do what you want. There is a script here that makes dynamic simple queues, you might be able to do something similar to add child queues automatically to the parent queue for each customer: https://mum.mikrotik.com//presentations ... 686228.pdfOk, so are there any other ideas as to how I would prioritize all other traffic ahead of MSDO for dynamic services like PPPoE on a per-customer basis?
However, a warning you should keep in mind - in many countries, Net Neutrality laws make what you are wanting to do illegal (as in, it is actually illegal for the ISP to decide for the customer to de-prioritize MSDO over other traffic). Be very careful that your attempt to improve the customer's situation is not actually breaking any laws in your country. For instance, in the United States, de-prioritizing MSDO for the customer would be illegal.
One *potential* way around this is to provide the customers with routers that they can manage that are configured to de-prioritize MSDO by default at the customer end, but they have the ability and a documented way of changing the configuration. Now that MikroTik supports TR-069, providing customers with managed home routers is a possibility.
Another (perhaps simpler) way around this - if you use sfq as the default queue type for your PPPoE customers, it might try to somewhat equitably prioritize MSDO vs other traffic. This can help prevent MSDO from using all of the customer bandwidth without having to set up complicated child queues.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
SFQ might help some, but MSDO uses FAST TCP and multiple connections, as in I've caught it with 40 connections going. In situations where multiple FAST TCP connections are going through the same bottleneck, connections that start later have a tendency to hog more than their share of bandwidth because of how baseRTT is calculated and used. Since MSDO opens and closes connections at will over time, it's guaranteed to hog. And with 40 connections going, one more for the user gets a pretty small share even with SFQ.Another (perhaps simpler) way around this - if you use sfq as the default queue type for your PPPoE customers, it might try to somewhat equitably prioritize MSDO vs other traffic. This can help prevent MSDO from using all of the customer bandwidth without having to set up complicated child queues.
Microsoft-Delivery-Optimization is a harmful, yet necessary, protocol. Plain and simple.
-
-
RavenWing71
newbie
- Posts: 34
- Joined:
Re: Do any queue types respect Priority markings?
Thank you for this. I'll look into how it works. Hopefully I can make it do what I need.What you might be able to do is use the PPPoE on-up and on-down script to locate the dynamic simple queue that was added and automatically add child queues to this to do what you want. There is a script here that makes dynamic simple queues, you might be able to do something similar to add child queues automatically to the parent queue for each customer: https://mum.mikrotik.com//presentations ... 686228.pdfOk, so are there any other ideas as to how I would prioritize all other traffic ahead of MSDO for dynamic services like PPPoE on a per-customer basis?