Community discussions

MUM Europe 2020
 
User avatar
GeneralMarmite
just joined
Topic Author
Posts: 6
Joined: Sun Nov 22, 2015 3:03 pm

Bandwidth limiting Facebook, YouTube, etc. by AS and CIDR IP Address

Thu Jun 02, 2016 9:09 pm

If you Google or search these forums you'll find lots of people desperate to limit or block Facebook, YouTube, and other bandwidth-hungry web sites. Many of them suggest using Layer7 content matching. I've implemented that with a regexp of (youtube|dailymotion|metacafe|vimeo|facebook|fbcdn|1e100), and surprisingly it seems to work. I say "surprisingly" because these days most web sites use HTTPS. That means the router cannot look into the Layer7 content and see anything at all. It's encrypted. Yay for security, but makes identifying traffic a bit harder. Here's what I've done to use IP addresses instead. This seems to work well but it also has some obvious limitations.

Before You Start
I've already implemented everything in this post http://agratitudesign.blogspot.co.uk/20 ... reams.html. That means I've already set up a content matcher that does packet marking. I've already implemented a bandwidth-limiting queue based on the marker. All I need to do now is put in some IP addresses for Facebook, YouTube, and so on to make sure they're actually marked.

Getting the IP addresses
First I look up the ASNs for YouTube. I went to https://www.ultratools.com/tools/asnInfo and searched "YouTube." That gives me the ASN for YouTube (AS43515). I don't know what networks are associated with that ASN, but it's a start. Then I went to http://www.cidr-report.org/ and I entered AS43515. That gets me the list of networks that are being announced associated with YouTube. It's a bunch. About 7 (see below).

It turns out that those IP addresses aren't enough. A lot of IP addresses from the US also seem to be used, even though I'm in the UK. I did a reverse lookup on one of those IP addresses and discovered that it comes out of Google's AS15169. A quick report here gives you the list of networks associated with that AS, and it's pretty substantial. http://www.cidr-report.org/cgi-bin/as-r ... 9&view=2.0. Note that this is a problematic approach for a few reasons.¹

Make Some Address Lists

Given those addresses, I can make some address lists. Here's what I've gotten from those two ASNs:
/ip firewall address-list add list=YouTube address=64.15.112.0/20    
/ip firewall address-list add list=YouTube address=104.237.160.0/19  
/ip firewall address-list add list=YouTube address=208.65.152.0/22   
/ip firewall address-list add list=YouTube address=208.117.224.0/19  
/ip firewall address-list add list=YouTube address=208.117.234.0/24  
/ip firewall address-list add list=YouTube address=208.117.236.0/24  
/ip firewall address-list add list=YouTube address=208.117.239.0/24  
/ip firewall address-list add list=YouTube address=208.117.242.0/24  
/ip firewall address-list add list=YouTube address=216.239.60.0/24  
/ip firewall address-list add list=Google address=8.34.208.0/20    
/ip firewall address-list add list=Google address=8.35.192.0/20    
/ip firewall address-list add list=Google address=8.8.4.0/24                
/ip firewall address-list add list=Google address=23.236.48.0/20            
/ip firewall address-list add list=Google address=23.251.128.0/19           
/ip firewall address-list add list=Google address=64.233.160.0/19           
/ip firewall address-list add list=Google address=66.102.0.0/20             
/ip firewall address-list add list=Google address=66.249.64.0/19            
/ip firewall address-list add list=Google address=70.32.128.0/19            
/ip firewall address-list add list=Google address=72.14.192.0/18            
/ip firewall address-list add list=Google address=74.114.24.0/21            
/ip firewall address-list add list=Google address=74.125.0.0/16             
/ip firewall address-list add list=Google address=104.132.0.0/23            
/ip firewall address-list add list=Google address=104.133.0.0/24            
/ip firewall address-list add list=Google address=104.154.0.0/15            
/ip firewall address-list add list=Google address=104.196.0.0/14            
/ip firewall address-list add list=Google address=107.167.160.0/19          
/ip firewall address-list add list=Google address=107.178.192.0/18          
/ip firewall address-list add list=Google address=108.170.192.0/18          
/ip firewall address-list add list=Google address=108.177.0.0/17            
/ip firewall address-list add list=Google address=108.59.80.0/20            
/ip firewall address-list add list=Google address=130.211.0.0/16            
/ip firewall address-list add list=Google address=142.250.0.0/15            
/ip firewall address-list add list=Google address=146.148.0.0/17            
/ip firewall address-list add list=Google address=162.216.148.0/22          
/ip firewall address-list add list=Google address=162.222.176.0/21          
/ip firewall address-list add list=Google address=172.102.8.0/21            
/ip firewall address-list add list=Google address=172.110.32.0/21           
/ip firewall address-list add list=Google address=172.217.0.0/16            
/ip firewall address-list add list=Google address=172.253.0.0/16            
/ip firewall address-list add list=Google address=173.194.0.0/16            
/ip firewall address-list add list=Google address=173.255.112.0/20          
/ip firewall address-list add list=Google address=185.150.148.0/22          
/ip firewall address-list add list=Google address=185.25.28.0/23            
/ip firewall address-list add list=Google address=192.104.160.0/23          
/ip firewall address-list add list=Google address=192.158.28.0/22           
/ip firewall address-list add list=Google address=192.178.0.0/15            
/ip firewall address-list add list=Google address=193.200.222.0/24          
/ip firewall address-list add list=Google address=199.192.112.0/22          
/ip firewall address-list add list=Google address=199.223.232.0/21          
/ip firewall address-list add list=Google address=207.223.160.0/20          
/ip firewall address-list add list=Google address=209.107.176.0/20          
/ip firewall address-list add list=Google address=209.85.128.0/17           
/ip firewall address-list add list=Google address=216.239.32.0/19           
/ip firewall address-list add list=Google address=216.252.220.0/22          
/ip firewall address-list add list=Google address=216.58.192.0/19
/ip firewall address-list add list=Facebook address=31.13.24.0/21      
/ip firewall address-list add list=Facebook address=45.64.40.0/22      
/ip firewall address-list add list=Facebook address=66.220.144.0/20    
/ip firewall address-list add list=Facebook address=69.171.224.0/19    
/ip firewall address-list add list=Facebook address=69.63.176.0/20     
/ip firewall address-list add list=Facebook address=74.119.76.0/22     
/ip firewall address-list add list=Facebook address=103.4.96.0/22      
/ip firewall address-list add list=Facebook address=173.252.64.0/18  
/ip firewall address-list add list=Facebook address=173.252.96.0/19
/ip firewall address-list add list=Facebook address=179.60.192.0/22    
/ip firewall address-list add list=Facebook address=185.60.216.0/22    
/ip firewall address-list add list=Facebook address=204.15.20.0/22            
Mark Connections to Those IP Addresses
/ip firewall mangle
add action=mark-packet chain=forward new-packet-mark=streaming passthrough=no \
    src-address-list=YouTube
add action=mark-packet chain=forward new-packet-mark=streaming passthrough=no \
    dst-address-list=YouTube 
add action=mark-packet chain=forward new-packet-mark=streaming passthrough=no \
    src-address-list=Google
add action=mark-packet chain=forward new-packet-mark=streaming passthrough=no \
    dst-address-list=Google
add action=mark-packet chain=forward new-packet-mark=streaming passthrough=no \
    src-address-list=Facebook
add action=mark-packet chain=forward new-packet-mark=streaming passthrough=no \
    dst-address-list=Facebook
Put a Bandwidth Limit on Them
I'm on a residential DSL line and the main reason I'm doing this is to leave some bandwidth left over for the rest of us if someone starts streaming a lot of YouTube. Hence the 2M limits.
/queue simple
add limit-at=2M/2M max-limit=2M/2M name=StreamQ packet-marks=streaming parent=dslq \
    total-limit-at=2M total-max-limit=2M
¹ Limitations
  1. This stuff changes. It doesn't change often and a lot, but it does change. Maybe not daily, but it isn't stable for years on end. So this list of IP addresses will be a royal pain in the butt to maintain. Since I'm only limiting bandwidth, there's no chance I end up blocking a whole network that I didn't mean to block. But there's a decent chance that one of these sites gets some new IP addresses that I don't know about, and so I'm not limiting transmission to them. If you use a technique like this to completely block IP addresses, you can end up blocking the wrong IP addresses.
  2. The list of IP addresses is culled from BGP announcements. It might be accurate, it might not be. It could change tomorrow and I wouldn't know.
  3. Only the biggest network services have dedicated IP space like this. This can't work for torrent downloads or downloads off a random web site. If a service is based on Amazon Web Services (AWS), Azure, or some other hosting provider/service, then it probably doesn't have its own dedicated set of IP addresses that you can do this with.
  4. This can be extended to limit bandwidth used by services like Spotify, Apple Music, and numerous other online properties. The more services you try to do this way, the more fragile your list becomes and the more work you have to put into maintaining it.
 
User avatar
davidnvega
just joined
Posts: 20
Joined: Wed Aug 28, 2013 11:11 pm
Location: San Juan, Argentina
Contact:

Re: Bandwidth limiting Facebook, YouTube, etc. by AS and CIDR IP Address

Fri Jul 22, 2016 2:32 am

Thanks, i will test it. 
Sorry for my English, I speak Spanish.
 
loveman
Member
Member
Posts: 327
Joined: Tue Mar 10, 2015 9:32 pm

Re: Bandwidth limiting Facebook, YouTube, etc. by AS and CIDR IP Address

Fri Jul 22, 2016 12:12 pm

Thanks
If you can drop vpn program like (psiphon vpn) by mikrotik. Please help me

Who is online

Users browsing this forum: fedus, Guntis and 22 guests