[SOLVED]An Interesting Challenge... ~or~ Choking Youtube...

RouterOS General
1

So I’ve got this client with a seemingly innocent request. He doesn’t want to block Youtube outright, he just wants to queue traffic down so that the video streams don’t kill his 3Mbps connection. Simple enough, right? Here’s the solution I had come up with:

  1. We can’t throttle Youtube by name, but we can throttle the IP. First, we set up a script to resolve hostnames and put them in an address list:
# define variables
:local list
:local hosts
:local newip

# Loop through each entry in the address list.
:foreach i in=[/ip firewall address-list find] do={

# Get the first four characters of list name
  :set list [:pick [/ip firewall address-list get $i list] 0 4]

# Condition: if the value of $list is "dns_" process it.
  :if ($list = "dns_") do={

# Of the items being processed, store their "comment" fields as the variable "hosts"
    :set hosts [/ip firewall address-list get $i comment]

# Resolve it and set the address list entry accordingly.
    :set newip [:resolve $hosts]
    /ip firewall address-list set $i address=$newip
    }
  }

*Credit goes to Paul Gu for this idea: http://wiki.mikrotik.com/wiki/Use_host_names_in_firewall_rules

  1. Now we make the list and put an entry in it for Youtube. The way that script works is that you just put the FQDN in the comment field of the list entry, and script does dns lookups based on that.
/ip firewall address-list add address=0.0.0.0 comment=youtube.com list=dns_choke

Now we invoke the script from before, it will resolve and update the IP for Youtube.

  1. The next step is to make some firewall rules to mark all packets with .flv or .mp4 content coming from the IP we have in the address list:
/ip firewall mangle add chain=forward protocol=tcp src-address-list=dns_choke content=.flv action=mark-packet new-packet-mark=choked_video comment="Mark .flv content from address list for queuing"

/ip firewall mangle add chain=forward protocol=tcp src-address-list=dns_choke content=.mp4 action=mark-packet new-packet-mark=choked_video comment="Mark .mp4 content from address list for queuing"
  1. Finally, we create the queue:
/queue simple add name="Youtube Video Content" max-limit=64k/64k packet-marks=choked_video

I was psyched about this solution, but it does not work. The mangle rule hasn’t processed any packets whatsoever. I noticed that when I ran the dns lookup script multiple times in a row, it kept resolving a different last octet for Youtube.

…of course. Youtube uses load distribution through DNS (such as the round-robin technique.) Since the IP could be any of a number of servers, the computers resolving on the LAN side aren’t likely to hit the same IP that’s in the address list, so the packets aren’t being processed.

Anyone know a way to accomplish what I’m attempting here?

2

Find packets with content-type: video by using layer7 processing. Then mark connections as “video” and shape them :wink:

3

This is working for me:

Ip Firewall Mangle: to add Youtube server to address list for 5 minutes.

;;; Youtube Address List
     chain=prerouting action=add-dst-to-address-list protocol=tcp address-list=Youtube 
     address-list-timeout=5m in-interface=!(PUBLIC) dst-port=80 content=youtube.com

Ip Firewall Mangle: To mark youtube packets (web-proxy enabled):

 ;;; Youtube
     chain=prerouting action=mark-connection new-connection-mark=youtubeconn passthrough=yes dst-address-list=Youtube 

     chain=output action=mark-packet new-packet-mark=youtube_pct passthrough=yes connection-mark=youtubeconn

Queue Types (to limit each user to 512K stream, 240p needs only 384k):

name="Youtube" kind=pcq pcq-rate=512k pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=2000 pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-src-address-mask=32 pcq-dst-address-mask=32 pcq-src-address6-mask=128 pcq-dst-address6-mask=128

Queue Tree (use 512K youtube PCQ & set priority = 8 )

name="Youtube" parent=global-out packet-mark=youtube_pct limit-at=0 queue=Youtube priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s

Regards;

4

Thank you!

This did pretty much what I wanted. How would you configure it just to add .flv and .mp4 content to the list, though? With this, if someone googles “youtube.com” Google will be throttled for 5 minutes.

5

Nice to hear that helped you.

Abt flv & mp4 … well, I guess it’s necessary then to use layer 7. Too much for my knowledge :smiley:

And yes, if someone uses any search tool to find http://www.youtube.com the search tool itself will be added to address list (google, bing, yahoo etc).

But … 512K is not enough to make a search tool smooth? :smiley:

Another aproach … if you left more bandwidht for youtube … at least this user will release the necessary bw earlier, lefting it avialable for other users. Sometimes is better to store content at user PC asap than keep it using 512K for 10 minutes. Need some statistical analysis.

Regards;

6

Thanks for the reply. In the implementation I’m planning for this solution has only 3M down from their ISP so 512k is still going to be too generous. They’ve got many hosts on the LAN side. I think I may try adding another rule to filter content matching .flv or .mp4 first, then have the rules you defined above use my list as a source. Search for youtube.com from a smaller subset of rules. It’s sorta sloppy, but I don’t know regular expressions, either so L7 is going to be a little much. On a related note, http://xkcd.com/208/ <= this.

Anyway, I’ll mark the thread as solved. Thanks again!

7

this is a wonderful solution…

dont know how people wont start abusing it because you maybe giving priority to lets say facebook.com and all of a sudden they are feeding crap from that.

it would also be nice to unify or lessen the rules?