Community discussions

 
0ldman
Forum Guru
Forum Guru
Topic Author
Posts: 1445
Joined: Thu Jul 27, 2006 5:01 am

L7 and youtube

Thu Mar 12, 2015 3:48 am

I've got a Youtube layer 7 regxp of
get /videoplayback[\x09-\x0d -~]* http/[01]\.[019]
Works fine, grabs Youtube and I set it to 3Mbps limit, but higher priority than generic downloads. I'm trying to guarantee the quality of streaming.

The problem I'm having is that this L7 code grabs speed tests as well.

Anyone have any advice on that?

I can go in a set a list of servers for the Youtube mangle entry to ignore, but that doesn't address the problem of the L7 grabbing the wrong data.

Anyone have a better L7 for Youtube?

Another issue is that if someone logs on to Youtube everything is encrypted, so that can't be shaped without marking all port 443... if anyone has any better ideas on that I'd love to hear it.
 
kivimart
newbie
Posts: 40
Joined: Thu Oct 10, 2013 3:06 pm

Re: L7 and youtube

Thu Mar 12, 2015 1:08 pm

This is my Layer7-protocol:

/ip firewall layer7-protocol
add name=Bittorrent regexp="^(\\x13bittorrent protocol|azver\\x01\$|get /scrape\
\\\?info_hash=get /announce\\\?info_hash=|get /client/bitcomet/|GET /data\\\
\?fid=)|d1:ad2:id20:|\\x08'7P\\)[RP]"
add name=streaming regexp=videoplayback|video
add name=streaming-audio regexp=audioplayback|audio

These seems to work wery well (even with youtube) and speedtest from speedtest.net dosent get any hits, in sweden we got a test site called http://www.bredbandskollen.se and that one gets hit by the video l7.

Dont know if this is the best solution but but it work. if enyone have better solution please post it.

i have a RB1100ahx2 as home router with 200/30 mbit down/up.

Martin
 
Abbasmcse
just joined
Posts: 16
Joined: Sat Oct 31, 2015 12:11 pm

Re: L7 and youtube

Thu Dec 31, 2015 1:47 am

MT = RouterBoard 750UP
Ether1: 200.200.201.100/29 (WAN)
Ether2: 192.168.1.100 (LAN network)

I want PC1- 192.168.1.150/24 get only 1Mbps traffic while he requesting for browsing internet, but PC1 get 10M speed while it's request for youtube streaming.


I am using this script to achieve this task.

/ip firewall mangle add action=add-dst-to-address-list address-list=Youtube address-list-timeout=10m chain=prerouting comment=youtube content=youtube.com dst-port=80,443 protocol=tcp
/ip firewall mangle add action=mark-packet chain=forward comment=youtube new-packet-mark=Youtube passthrough=no src-address-list=Youtube

but i don't know how to create separates parent queue to match.

Thanks
Abbas
 
User avatar
dunga
Member Candidate
Member Candidate
Posts: 254
Joined: Fri Jan 23, 2009 9:51 am
Location: Nigeria

Re: L7 and youtube

Tue Mar 29, 2016 6:04 pm

This is my Layer7-protocol:

/ip firewall layer7-protocol
add name=Bittorrent regexp="^(\\x13bittorrent protocol|azver\\x01\$|get /scrape\
\\\?info_hash=get /announce\\\?info_hash=|get /client/bitcomet/|GET /data\\\
\?fid=)|d1:ad2:id20:|\\x08'7P\\)[RP]"
add name=streaming regexp=videoplayback|video
add name=streaming-audio regexp=audioplayback|audio

These seems to work wery well (even with youtube) and speedtest from speedtest.net dosent get any hits, in sweden we got a test site called http://www.bredbandskollen.se and that one gets hit by the video l7.


Martin
What of the other scripts like mangle and queue you used to set the limit or other commands.
I've got a Youtube layer 7 regxp of
get /videoplayback[\x09-\x0d -~]* http/[01]\.[019]
Works fine, grabs Youtube and I set it to 3Mbps limit, but higher priority than generic downloads. I'm trying to guarantee the quality of streaming.


Anyone have a better L7 for Youtube?

Another issue is that if someone logs on to Youtube everything is encrypted, so that can't be shaped without marking all port 443... if anyone has any better ideas on that I'd love to hear it.
something seems to miss in your export, try post other commands that follows it. so that i can test and see if it works for me.
 
0ldman
Forum Guru
Forum Guru
Topic Author
Posts: 1445
Joined: Thu Jul 27, 2006 5:01 am

Re: L7 and youtube

Tue Mar 29, 2016 6:35 pm

That is exactly as it is set in my L7 protocol. Create a new protocol, name it Youtube, paste that into the box.

At one time MT had a list of L7 protocols set as a script of sorts to add to ROS. I think that is where I got mine.
I'm not sure if I can post that here. It is very, very long.

See if this helps.
/ip firewall layer7-protocol
add name=netflix regexp="^.*(host|HOST).+(netflix).*\$"

add name=youtube regexp="get /videoplayback[\\x09-\\x0d -~]* http/[01]\\.[019]"

add name="netflix2" regexp="^.*(get|GET) /s.+(wma|wmv|token|random|p=).*\$"

add name=hulu regexp=^\\\\x03.+\\\\x14.+\\\\x02.+\\\\x07.(connect)\\\?.+(app)\\\?
These may not be applicable anymore. They've all changed their compression and delivery since then.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5521
Joined: Mon Jun 08, 2015 12:09 pm

Re: L7 and youtube

Tue Mar 29, 2016 7:25 pm

These may not be applicable anymore. They've all changed their compression and delivery since then.
And even more important: they have all changed to https making it impossible to do this today.
The proponents of https like what they have accomplished: impossible for men-in-the-middle to influence the
handling of their traffic. Of course it is shortsighted, because this now causes operational problems in the
networks that required this traffic classification to be able to operate.
 
cicserver
Member
Member
Posts: 300
Joined: Sun Jul 24, 2011 12:04 pm

Re: L7 and youtube

Sun Apr 03, 2016 12:23 pm

I have two WAN links.
the L7 works fine, the Queue works fine for these marked packets.
but the problem is that when i try to route these marked packets to another WAN link, the VIDEO doesn't play. Other websites routing is working fine. its just YT videos not play when route packets.

Is there any method I can route YT video traffic to another WAN link ?
 
valent
newbie
Posts: 43
Joined: Wed Dec 07, 2005 8:47 pm

Re: L7 and youtube

Mon Apr 04, 2016 11:34 am

Is L7 necessary for youtube? I'm wondering could it be handled with hostname or ip adress lists for youtube servers?
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24042
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: L7 and youtube

Mon Apr 04, 2016 11:51 am

When user is logged in, youtube will use HTTPS, meaning that L7 will not be able to match this traffic. Only unencrypted HTTP can be matched.
No answer to your question? How to write posts
 
cicserver
Member
Member
Posts: 300
Joined: Sun Jul 24, 2011 12:04 pm

Re: L7 and youtube

Mon Apr 11, 2016 11:23 am

OK I pulled all IP from ASN 15169 , now YT routing to second WAN link, Primary objective achieved.
But now issue is that its dragging many other services like Gmail / Dropbox, apps etc. is there any way I can route only YOUTUBE ? how can i found just YOUTUBE IP Address list only ?
 
nigslaysa
just joined
Posts: 15
Joined: Wed Apr 06, 2016 11:08 am

Re: L7 and youtube

Fri Nov 25, 2016 10:41 am

Try to filter UDP 443 problem is it catches even non youtube traffic
 
MayestroPW
newbie
Posts: 47
Joined: Wed Oct 26, 2016 3:28 pm

Re: L7 and youtube

Mon Feb 20, 2017 10:11 am

YouTube uses QUIC, which uses UDP not TCP protocol.
For me this regex works fine:
videoplayback|video
works fine for youtube and every other online video stream even HTTPS, besides live streams ofc.
 
sebus
newbie
Posts: 38
Joined: Sun Mar 12, 2017 6:29 pm

Re: L7 and youtube

Sun May 14, 2017 9:28 pm

Drastic, but does work. Could not get anything else working for https YT, although I would prefer something more specific to DNS entries only (ie youtube.* etc)
 
pe1chl
Forum Guru
Forum Guru
Posts: 5521
Joined: Mon Jun 08, 2015 12:09 pm

Re: L7 and youtube

Sun May 14, 2017 9:55 pm

For me this regex works fine:
videoplayback|video
works fine for youtube and every other online video stream even HTTPS, besides live streams ofc.
It will work "fine" only when you do not count the massive false positives.
It will hit on any packet with the word video in it.
 
sebus
newbie
Posts: 38
Joined: Sun Mar 12, 2017 6:29 pm

Re: L7 and youtube

Sun May 14, 2017 10:05 pm

Hence it is drastic.
But the only one that works.
There are plenty of Guides on Internet for "How To Block HTTPS websites on Mikrotik Router"
and using L7, none of which I could make work (with any regex for YT)

sebus
 
MayestroPW
newbie
Posts: 47
Joined: Wed Oct 26, 2016 3:28 pm

Re: L7 and youtube

Mon May 15, 2017 11:09 am

For me this regex works fine:
videoplayback|video
works fine for youtube and every other online video stream even HTTPS, besides live streams ofc.
It will work "fine" only when you do not count the massive false positives.
It will hit on any packet with the word video in it.
False positives? When a packet has word video, in most cases there is a video. So, for my Queuing it's OK because I want the video to be streamed with priority.
So there are no false positives at all. Well, at least for me, and my config.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5521
Joined: Mon Jun 08, 2015 12:09 pm

Re: L7 and youtube

Mon May 15, 2017 11:29 am

You think that this discussion has a video and needs to be blocked?
It has the word video all over the place. That would be no false positive for you?

Of course when you access a site over https there is no match with the actual word video in the content,
on the other hand it could trigger on any other content that results in the "word" video in the encrypted data.

Of course it has always been a bad idea to scan entire datastreams like this, it would be better to scan
only certain types of traffic like DNS lookups and the outgoing request on a HTTP(S) connection. That will
reduce the load and the risk of false positives a little, but it would still make your regexp fail on every
domain that has the word video in it, e.g. http://www.montevideo.gub.uy/
 
MayestroPW
newbie
Posts: 47
Joined: Wed Oct 26, 2016 3:28 pm

Re: L7 and youtube

Mon May 15, 2017 11:52 am

You think that this discussion has a video and needs to be blocked?
It has the word video all over the place. That would be no false positive for you?

Of course when you access a site over https there is no match with the actual word video in the content,
on the other hand it could trigger on any other content that results in the "word" video in the encrypted data.

Of course it has always been a bad idea to scan entire datastreams like this, it would be better to scan
only certain types of traffic like DNS lookups and the outgoing request on a HTTP(S) connection. That will
reduce the load and the risk of false positives a little, but it would still make your regexp fail on every
domain that has the word video in it, e.g. http://www.montevideo.gub.uy/
Even when there is a false positive, and it will mark it as a stream, it will just "stream" only few KB. So it's nothing compared to hundreds of MBs videos.
Doing it only in DNS, it would be stupid because not only youtube streams video, and it would mean the only youtube would have priority. What about millions other websites that stream videos?

I have RB750Gr3, 323 filter rules, 124 nat rules, 243 mangle rules, 79 simple queues, and 17 trees, 3 scripts launched in 15s intervals and two 50mbs/10mbs WANs in PCC. In worst case scenario it's 50-70% usage.
You do not have the required permissions to view the files attached to this post.

Who is online

Users browsing this forum: No registered users and 87 guests