Is there any way to detect and QoS download managers, such as “Free Download Manager”?
They can easily open tons of connections with pretty impressive download rate, and hog as much bandwidth as they can.
Is it possible to detect and queue this sort of traffic?
Have you checked the layer7 protocols? Maybe you can find sth there
No, nothing in there.
I was under impression that L7 is not a good idea for smaller routerboards (mine is 450G), as it will hog CPU and affect the overall performance - am I wrong here?
I think you are wrong here 
Of course it will use more CPU, but that doesn’t mean it will hog it.
is this “free download manager” using a given port?
If yes, you can capture it’s traffic using the port, other wise I will
suggest to have a look at the “conection rate”
You can also use connection-limit to set a limit on how many connections each user can have.
There are quite a few such tools, but this one is one of more popular.
http://www.freedownloadmanager.org
I suppose it does regular HTTP downloads, but it opens many connections on the same file loading different parts simultaneously. And it usually works with few files at the same time - total is pretty impressive.
I don’t really want to limit it permanently, but rather to slow it down dynamically when necessary.
Than, the example of wiki about the connection-rate is definitly for this case
I don’t understand how is download manager different from any other web activity? Either limit your customer to certain total speed, certain max download limit, or certain connection number.
In my opinion there is a difference between normal web browsing and web downloading.
Web browsing is short in time and not that big, that means it’s connection are short,
from time to time, with small quantities of packets.
Web downloading instead, lasts as long as it needs to download the file, at it will grab the
whole bandwidth available.
Now, in a situation when you give priorities, high priority to web browsing and lower to
download (using connection-rate) you would expect that the webpage will open as soon as
you click on it, even if in the meantime there is a download going on. I have seen this is not true.
This is a situation like when you try to pass through a crowded door. When there are a lot of people
trying to pass through, even if the doorkeeper is a “friend” of yours, he still needs to see you
so that he can pass you before the others.
Let me explain with to different configuration of queue tree:
the first one
1 name=“download” parent=ether1 limit-at=0 priority=8 max-limit=1M
burst-limit=0 burst-threshold=0 burst-time=0s
2 name=“other_download” parent=download packet-mark=other_traffic
limit-at=768k queue=PcqDown priority=1 max-limit=1M burst-limit=0
burst-threshold=0 burst-time=0s
3 name=“heavy_download” parent=download packet-mark=heavy_traffic
limit-at=128k queue=PcqDown priority=8 max-limit=1M burst-limit=0
burst-threshold=0 burst-time=0s
the second one
1 name=“download” parent=ether1 limit-at=0 priority=8 max-limit=1M
burst-limit=0 burst-threshold=0 burst-time=0s
2 name=“other_download” parent=download packet-mark=other_traffic
limit-at=768k queue=PcqDown priority=1 max-limit=1M burst-limit=0
burst-threshold=0 burst-time=0s
3 name=“heavy_download” parent=download packet-mark=heavy_traffic
limit-at=128k queue=PcqDown priority=8 max-limit=896k burst-limit=0
burst-threshold=0 burst-time=0s
As you can see, in the second configuration, I leave always a little room for packets
to pass, although there could be a heavy download going on. Untill now this is working better.
@normis
Download managers use different strategy comparing to browsing or “manual” download - their very purpose is to make max use of all available bandwidth to minimize the download time. Take into account a typical use case: some fifty 200kB files queued, 6-8 files at a time, each of then accessed by 6-10 connections at max speed possible. I did some tests and saw 30-35mbps on WAN interface just from that, not even sure it is the maximum - on our 50mbps link. And usually the person has no idea what he’s doing to other users.
@Caci99
Let me see if I understand it right - in the second example you decreased the queue buffer size, thus decreasing the time to free the way for higher priority traffic - is it correct?
I would really appreciate if you can type a couple of words on limit-at and max-limit - what they really affect. Is there a downside of minimizing the buffer?
Couldn’t you use Bursts to jump start http pages?
edmidor, if you limit this guy at, say, 2mbps. what is the difference if he uses one or 10 connections?
That probably would do, if I can detect his activity, slow him down to 2mbps, and after he finished ‘abusing’ the connection remove that limit - that what I’m looking for. The only question is “how”.
What I posted was just a snip of the “Connection rate” configuration.
This configuration is very good in situation where heavy download disturbs
normal internet operation. I strongly suggest you to read the wiki about connection rate.
With this configuration, you will be able to divide the heavy download, whatever it is, P2P,
download manager, youtube etc., from the “normal” web browsing, Voip etc.
In what I have posted, you will see that there is a queue that acts as parent with limit to 1MB,
and there are two other child queues with the same limit, but with different priorities.
Later, I changed a little bit this configuration, living always a space of 128kbps to the child queue
with the highest priority.
If you read the wiki you will understand me better.
http://mum.mikrotik.com/presentations/CZ09/QoS_Megis.pdf
Just put the user in a class of “abusers”
I read wiki. Now, given my connection numbers 50mbps down and 1 up - is it effective applied on download at all?
Should I create queues for upload only, or for both?
At my worth tests I never seen download traffic reaching 50mbps, but I think 1mbps upstream is easily saturated.
Your 1Mbps upload related to the 30Mbps real download is terribly low.
This internet line is so asymmetric! But this is not the issue we are discussing right now.
I think you should really implement that configuration, so that no one could grab anymore the
whole bandwidth available as long as you are sharing it for the all users.
The other way is to specify simple queues for each user. You choose.
My advice:
Always backup current configuration before any change.
/ip firewall mangle
add action=mark-connection chain=forward comment=“Connection rate”
connection-mark=!heavy_traffic disabled=no new-connection-mark=all_con
passthrough=yes protocol=tcp
add action=mark-connection chain=forward comment=“” connection-bytes=500000-0
connection-mark=all_con connection-rate=50k-100M disabled=no
new-connection-mark=heavy_traffic passthrough=yes protocol=tcp
add action=mark-connection chain=forward comment=“” connection-bytes=500000-0
connection-mark=all_con connection-rate=80k-100M disabled=no
new-connection-mark=heavy_traffic passthrough=yes protocol=udp
add action=mark-packet chain=forward comment=“” connection-mark=heavy_traffic
disabled=no new-packet-mark=heavy_traffic passthrough=no
add action=mark-packet chain=forward comment=“” connection-mark=
!heavy_traffic disabled=no new-packet-mark=other_traffic passthrough=no
/queue type
add name=“PcqDown” kind=pcq pcq-rate=0 pcq-limit=50 pcq-classifier=dst-address
pcq-total-limit=2000
add name=“PcqUp” kind=pcq pcq-rate=0 pcq-limit=50 pcq-classifier=src-address
pcq-total-limit=2000
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0
max-limit=1M name=upload parent=wlan1 priority=6
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=512k
max-limit=1M name=other_upload packet-mark=other_traffic parent=upload
priority=1 queue=PcqUp
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=512k
max-limit=1M name=heavy_upload packet-mark=heavy_traffic parent=upload
priority=6 queue=PcqUp
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0
max-limit=50M name=download parent=ether1 priority=8
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=30M
max-limit=50M name=other_download packet-mark=other_traffic parent=
download priority=1 queue=PcqDown
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=20M
max-limit=28M name=heavy_download packet-mark=heavy_traffic parent=
download priority=8 queue=PcqDown
Note: wlan1 is the out interface and ether1 is the LAN interface. I have left default values
por the pcq queues in /queue type. You should correct them as your network requests it.
You should also be sure if your download is 30M or 50M and change the values in the
queue limits as needed.
Something seems not good on wireless when some programs (i.e. torrent) start downloading files through many connections that transport a very small part of it with a lot of packets. That obviously make wireless more busy (more overhead than data throughput) than any other common connection like web surfing. There is a game server I know that keep about 40 packets @ 100 bytes each per second per user connected, quiet troublesome on wireless Ive seen.
Still didn’t know a better way to handle this on wireless than moving such kind of user from 2.4g to 5.8g. Then web surfing users and gamers stop complaining were both of them was together on 2.4g… Any other method we tried just caused one or another to get its connectivity and performance bad. Also, didn’t ever experience any of these problems on a 10/100 ethernet medium user.
Thanks Toni!
How can I make sure outbound VPN connection (Cisco VPN client) won’t fall under ‘heavy traffic’?
I’m not sure about the rate used by VPN, but I suppose it can be fast, and it lasts long…
Unfortunately I have not worked that much with VPN, so I am not sure about their behaviour,
But i guess you can put a mangle rule at the top of the others with action “accept” sth like:
/ip firewall mangle
add chain=prerouting src-addres=VPN-IP action=accept
this way it’s traffic will not be processed any more.
Running with your config for few days - VoIP works even with heavy download - thanks!
Question - how do you decide on limit-at value? Any general rules?