Hi,

I have setup a kind of bandwidth manger, a RouterOS based PC in a bridge mode with setup like this:

/ip firewall mangle
add action=mark-packet chain=forward comment="" disabled=no dscp=26 new-packet-mark=voip-sip passthrough=no
add action=mark-packet chain=forward comment="" disabled=no dscp=46 new-packet-mark=voip-rtp passthrough=no
add action=mark-packet chain=forward comment="" disabled=no dst-port=443 new-packet-mark=ssl passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="" disabled=no dst-port=80 new-packet-mark=http passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=udp passthrough=no protocol=udp
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=icmp passthrough=no protocol=icmp
add action=mark-packet chain=forward comment="" disabled=no dst-port=110 new-packet-mark=pop3 passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="" disabled=no dst-port=25 new-packet-mark=smtp passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="" disabled=no dst-port=143 new-packet-mark=imap passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="" disabled=no dst-port=1863 new-packet-mark=msn-messenger passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=gre passthrough=no protocol=gre
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=ipsec-esp passthrough=no protocol=ipsec-esp
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=ipsec-ah passthrough=no protocol=ipsec-ah
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=ipencap passthrough=no protocol=ipencap
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=ipip passthrough=no protocol=ipip
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=p2p p2p=all-p2p passthrough=no
add action=mark-packet chain=forward comment="" disabled=no layer7-protocol=msnmessenger new-packet-mark=msnmessenger passthrough=no
add action=mark-packet chain=forward comment="" disabled=no dst-address=81.26.212.150 new-packet-mark=trionetcall passthrough=no
add action=mark-packet chain=forward comment="" disabled=no layer7-protocol=skypetoskype new-packet-mark=skypetoskype passthrough=no
add action=mark-packet chain=forward comment="" disabled=no layer7-protocol=skypeout new-packet-mark=skypeout passthrough=no
add action=mark-packet chain=forward comment="" connection-bytes=1-512000 disabled=no new-packet-mark=0bytes passthrough=no
add action=mark-packet chain=forward comment="" connection-bytes=512000-1000000 disabled=no new-packet-mark=1Mbyte passthrough=no
add action=mark-packet chain=forward comment="" connection-bytes=1000000-3000000 disabled=no new-packet-mark=3Mbyte passthrough=no
add action=mark-packet chain=forward comment="" connection-bytes=3000000-6000000 disabled=no new-packet-mark=6Mbyte passthrough=no
add action=mark-packet chain=forward comment="" connection-bytes=6000000-0 disabled=no new-packet-mark=Infinite-Bytes passthrough=no
add action=mark-packet chain=forward comment="" disabled=no new-packet-mark=other passthrough=yes

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=14000000 max-limit=14000000 name=OVERALL packet-mark="" parent=IN priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=HTTP packet-mark=http parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=SSL packet-mark=ssl parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=UDP packet-mark=udp parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=ICMP packet-mark=icmp parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=POP3 packet-mark=pop3 parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=SMTP packet-mark=smtp parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=IMAP packet-mark=imap parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=0-512 packet-mark=0bytes parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=1Mbyte packet-mark=1Mbyte parent=OVERALL priority=3 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=3Mbyte packet-mark=3Mbyte parent=OVERALL priority=6 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=6Mbyte packet-mark=6Mbyte parent=OVERALL priority=7 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=Infinite packet-mark=Infinite-Bytes parent=OVERALL priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=MSN-MESSENGER packet-mark=msn-messenger parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=GRE packet-mark=gre parent=OVERALL priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=IPSEC-ESP packet-mark=ipsec-esp parent=OVERALL priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=IPSEC-AH packet-mark=ipsec-ah parent=OVERALL priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=P2P packet-mark=p2p parent=OVERALL priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=IPENCAP packet-mark=ipencap parent=OVERALL priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=IPIP packet-mark=ipip parent=OVERALL priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=Msnmessenger packet-mark=msnmessenger parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=Skypeout packet-mark=skypeout parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=SkypetoSkype packet-mark=skypetoskype parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=TrionetCall packet-mark=trionetcall parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=VoIP-SIP packet-mark=voip-sip parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=VoIP-RTP packet-mark=voip-rtp parent=OVERALL priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=OTHER packet-mark=other parent=OVERALL priority=6 queue=default

The problem is that, even a priority one traffic is getting queued, is the issue that I have to put a limit for all queue tree rules instead of only a limit to parent queue.

Regards.

Faton

This is the example I liked before. But you had a problem with it. How did you solve it?

I didn't had any significant problem, just sometimes it was queuing also packets from the higher priority. I'm still using this setup but I had to tweak it a little bit, now it works fine and I'm happy with it. I also added some scripts to enable different rates for certain traffic in certain period of time. If you are interested I can give my setup or even write a wiki about it.

Regards

Faton

please write wiki!!! that would be great help..

Please start the wiki. You can start by pasting the config here in the forum, so we could help with suggestions for the WiKi article. IT will be great!

OK guys, I have just paisted my exported configuration at wiki, so lets make it look good I will give all the info that is necessary, here is the link:

http://wiki.mikrotik.com/wiki/Traffic_P ... lemetation

Regards

Faton

thanks for the wiki entry.

I have some parts that I dont understand. Could you explain briefly what the overall rules do ?

I just paisted my config, I will start to work on the wiki with all explanations and necessary information.

I just edited the wiki, there are more information about this setup, there are more to come as soon as I grab some time to work on it.

Regards.

Faton

Excellent work Fatonk. This is such an eye opener! However, I have the following questions:

Q1: I couldn't find out where you control the speed of individual customers at 1Mbit or 2Mbit, as you said at the beginning of your article. Or are you doing it somewhere else in your config that you didn't post? Can you please clarify?

Q2: You also put pcq-total-limit=2000 and pcq-limit=50. AFAIK it would mean only upto 40 simultaneous queues can be served, and anything above would be problematic. What do you think?

Q3: All the port-wise mangle rules use "dst-port". Isn't that marking only uploads? Don't you need to mark the downloads here?

Q4: Why do you put passthrough=yes for all the connection-bytes rules in the end? Is that necessary? Or just doesn't make a difference?

Q5: Also, because you are using connection-bytes, when a user uses a download manager, and stops the download, before starting it again, won't they get a higher priority?

For Youtube, the following blocks should cover all:
208.65.152.0/22 (https://ws.arin.net/whois/?queryinput=N%20.%20YOUTUBE)
64.15.112.0/20 (https://ws.arin.net/whois/?queryinput=N%20.%20YOUTUBE2)
208.117.224.0/19 (https://ws.arin.net/whois/?queryinput=N%20.%20YOUTUBE3)

I do have a question though, if I had 3 interfaces as follows:
eth_1 => upstream to internet (we pay for this)
eth_2 => upstream to local country exchange (this is almost free)
eth_3 => my user network

Objective: prioritize incoming data through eth_1, but do not prioritize incoming on eth_2
Ques #5: In this case, wouldn't it be better to mangle with chain=prerouting and in-interface=eth_1?

Sorry for the long post, but your example just got me going!

Ciao...

Excellent work Fatonk. This is such an eye opener! However, I have the following questions:

Q1: I couldn't find out where you control the speed of individual customers at 1Mbit or 2Mbit, as you said at the beginning of your article. Or are you doing it somewhere else in your config that you didn't post? Can you please clarify?

I think PCQ is taking care of that. Or dynamic simple queues. But he will answer more to the point.

Q2: You also put pcq-total-limit=2000 and pcq-limit=50. AFAIK it would mean only upto 40 simultaneous queues can be served, and anything above would be problematic. What do you think?

Have you tested such a scenario? I think this means simply more drops but PCQ behavior could be affected just a little, so maybe its working good enough

Q3: All the port-wise mangle rules use "dst-port". Isn't that marking only uploads? Don't you need to mark the downloads here?

He said he is prioritizing only uploads as download does not get so congested anyway.

Q4: Why do you put passthrough=yes for all the connection-bytes rules in the end? Is that necessary? Or just doesn't make a difference?

I would like to know the answer of this myself

Q5: Also, because you are using connection-bytes, when a user uses a download manager, and stops the download, before starting it again, won't they get a higher priority?

They probably will. But is there a way to deal with this? Maybe detect with L7 that is a resume of a big download...

For Youtube, the following blocks should cover all:
208.65.152.0/22 (https://ws.arin.net/whois/?queryinput=N%20.%20YOUTUBE)
64.15.112.0/20 (https://ws.arin.net/whois/?queryinput=N%20.%20YOUTUBE2)
208.117.224.0/19 (https://ws.arin.net/whois/?queryinput=N%20.%20YOUTUBE3)

I do have a question though, if I had 3 interfaces as follows:
eth_1 => upstream to internet (we pay for this)
eth_2 => upstream to local country exchange (this is almost free)
eth_3 => my user network

Objective: prioritize incoming data through eth_1, but do not prioritize incoming on eth_2
Ques #5: In this case, wouldn't it be better to mangle with chain=prerouting and in-interface=eth_1?

Yeah, sure, sounds good. But fatonk WiKi setup currently catches upload traffic. So in-interface is not the option here...

Sorry for the long post, but your example just got me going!

Ciao...

Thx for the quick answer NetworkPro. Here is my response to your remarks, I'm just putting in my answer, without quoting your reply to avoid making a long email:

Q1: I also think he is doing PCQ or something elsewhere which is not in this config.

Q2: Yes, I have tested this. Actually I was burnt very badly around 4 weeks ago, when I divided my NAT + BWMgr box into two MT Boxes, and forgot to migrate the pcq-total-limit values. We have around 3,000 customers in one particular category, where the users used to constantly complain during peak hours, that their BW would fluctuate between the 0 and max-assigned value. Took me 7 painful days, and a lot of re-configs in lots of places, before I noticed this small thing. And, since then, this has been singing like nothing. To repeat the behavior, bring up the (RX & TX)_Queued_Packets colums in simple queues, and will definitely see a problem if any category has more than the default 2000 packets in their queues. At peak time, this particular category had 7000+ queued packets!

Q3: I re-read the Wiki, and i says clearly in the first line, that he is trying to put QoS in Download Traffic. Or is this from the router's perspective, rather than his network's? I'm confused! Could you please correct me on this if I'm wrong?

Q4:

Q5: This is indeed a major headache. To me it appears more like a balancing act, between how long we make each connection-bytes bucket, and the PCQ which caps the maximum download. Will have to play around and see. But, need answer to Q4 first!

Ques 5 [should be 6 ]: While I'm not quite sure what fatonk is looking to do QoS on, I need to do QoS on downloads on Eth_1 only, as that is what we pay for by the Mbps. So, I'll play around a bit this week and find out!

Thank you once again NetworkPro for your quick answer. Looking forward to share more ideas!

Ciao

Hi,

I'm glad you like this wiki, I hope that all together here in this forum will get it to more perfection.

Let's go back to questions of Cartes:

First, the individual limit per user is applied in PPPoE Servers not here, Radius sends attributes for limiting the rate of users based on particular product speed.

I have put this bandwidth-manager between Main-Gateway and PPPoE-Servers, so all traffic comes from 6 PPPoE-Servers and means that only 6 source IP's (1 per PPPoE-Server) passes through the bandwidth-manager. In this case PCQ is configured with the parameters dst-port since there are different dst-ports for each connection, and dst-address since there are 6 different dst-addreses. Since NAT is applied in all PPPoE-Servers I found it necessary to configure PCQ in this way.

Only download traffic I intent to shape here, you can see it from the parent of the OVERALL Queue, the parent is the INTERNAL interface.

The port based mangle rules, are there in case somebody wants to QoS upload also, it can be used to identify some sensitive and crucial traffic.

Regarding the Youtube address-list, I just named it youtube but there are included also Metacaffe, Dailymotion, Redtube, Youporn etc. many of the web video sites and servers.

For any additional information we'll keep in touch here.

Regards.

Faton

The connection bytes rules has passthrough=yes by default, there are no issues with it.

hi, congrat for your post it very good, just some remarks, to mark youtube, metacafe and others i use the mark content, i use content=flv, and it catch all the sites with videos, youtube, youporn redtube, googlevideo, dailymotion, etc... also i've read that the piority just work win t limitation, it was said by janisk, how did you test your config? it's possible prioritize without set a limit-at for the leaf queues? if so it would be much better. thanks..

As far as I have played around, no limit-at set gives you a round robin of the sub-queues, which is almost equality in priority.

How exaclty you use content=flv ? Share example rules please.

Config testing may be done in a lab with a lab setup with special software. Testing live networks is a bit harder and trickier.

about the limit-at i've tested and woks better if you set limit-at and max-lmit in parent queue, then set a max-limit in a leaf queue but not a limit-at with that config it works as you said, something like a round robin,

network can you help me with something? i dont find a definitive way to mark the ACK packets to prioitize it.

/ip firewall mangle
add action=mark-connection chain=postrouting comment=".flv extensions" content=flv disabled=no new-connection-mark=flv-conn passthrough=yes
add action=mark-packet chain=postrouting comment=".flv extensions" connection-mark=flv-conn disabled=no new-packet-mark=flv passthrough=no

it works perfect.. tested in a real life..

Hello
Tried to read the wiki but did not ghrab it well.

can u redefine and explain more on the settings both for the uplink and downlink so that we can understand it. I know it will not be easy but for learning sake.

Thanks