That is a lot of simple queues you would need to create for each customer. Mikrotik says that they have improved a lot the performance of simple queues, but I haven't tried it in real world since I am a lot more comfortable with queue tree. The good thing about queue tree is that all queues are treated at the same time, while with a simple queue the packet must check them all in their order until it matches the one which deals with it.
Yes it is a lot of queues. 3 per customer. I have nearly 900 simple queue entries and I haven't finished adding the bulk and other queues to all the customers yet.
The current implementation if Simple Queues uses hash-table lookup, which is very fast and scales up really well.
I'm glad to hear this. It settles my worry about all these queues.
I think that there must be customers who are on the same bandwidth plan, say you got to have 10 customers who are on 10/10Mbps. If you have such groups, you can set the profile of pppoe with no limits and add them to address-list. Then use address-list to identify the MSDO coming-going from this address-list, and the other traffic. Once you have marked packets, you can apply them on queue-tree with one parent equivalent at the total of the group and then children queues with pcq to distribute it equally to the customers of the group. You can save a lot of queues this way, freeing up resources of the router.
I see you have no problem to identify the traffic, so can't see why not use address-list generated from the pppoe profile. Once a customers connects through pppoe, no matter what IP will be assigned, he will be added to the specified address-list, and with no limits on pppoe profile, no dynamic simple queue will be created, thus moving them on the queue tree.
This sounds like a variation on what is recommended by the QoS_Megis.pdf. There must be something I'm missing about how queue trees work. I could mark packets as 10/10 and 10/10-Bulk and then they would go into different branches of the tree,
but if I make a PCQ for each mark how would it know that 172.29.236.40 only gets 10Mbps total and not 10Mbps for each mark since they are separate PCQs? Can I branch off of a PCQ, making two fifo queues beyond the PCQ, one for each mark? What do I really get if I do? Would it then make 2 fifo sub-queues for each dynamically generated queue within the PCQ? If it did, then this would work, otherwise it's just a mess. Perhaps you could mock up a queue tree that does what you are suggesting?
There are two way to configure PCQ using pcq-rate, one is to give full bandwidth to the child available from its parent, the other is giving no more than specified in pcq-rate even if parent might have more available. The parent will have a total of the group, say 100Mbps for 10 users with 10Mbps plan. The pcq will distribute it equally to the destination IPs. So if two of them are connected and if pcq-rate=0 it will give both of them 50. When pcq-rate=10Mbps and two of them are connected it will give each 10Mbps, which means you will be using 20Mbps from the total of parent queue.
Now, when you have two children queues, which we are trying to do, I would put the two under another child queue:
/queue tree
add name="download" parent=global packet-mark="" limit-at=0 queue=default priority=8 max-limit=100M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
add name="customer_download" parent=Download packet-mark=customer_down limit-at=0 queue=PcqDown priority=8 max-limit=100M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
add name="other_down" parent=customer_download packet-mark=other_down limit-at=2M queue=PcqDown priority=4 max-limit=100M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
add name="MSDO" parent=customer_download packet-mark=MSDO limit-at=1M queue=PcqDown priority=8 max-limit=95M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
Have to admit, I haven't tried this kind of configuration with children inside child because I never worry how customer uses his bandwidth, i just worry to deliver him what he requests. If he chooses to eat all bandwidth with torrent, so be it.
But I believe that this will work, since the first child will separate the band for each IP, and then the second child can not get more from the what the first (his parent) delivers, which is 10, and within that 10 you will assign priority between MSDO and other traffic.
I have attached the main parent at the global interface assuming that in mangle you take care of marking traffic by specifying in-interface and out-interface among others, otherwise you can attach it to the WAN interface for download. I have also put a lower total-limit on the MSDO queue, since we consider it as heavy eating all the available bandwidth, so I leave a small window of 5Mbps in order to allow normal traffic to sneak in and then get the priority. It is as if you have a 100 people going through one single door, but the one with high priority is at the back, you would leave a small alley for him to avoid the queue and can go through as soon as he arrives.