Community discussions

 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Fight against rapidshare

Thu Aug 09, 2007 11:22 pm

Hi all,
as a WISP, we are fightng our battle against P2P every day... :?
P2P is identified with mangle for port different than well known ports (>1024) and redirected through dedicated connection (MT default gateway).
A big problem is represented by P2P on port 80, like Rapidshare. This traffic is not identified as P2P, but is considered as normal web traffic... a disaster !!! :shock:
Rapidshare has many servers around the Internet and newer are added avery day, so it is almost impossible have an updated list of its servers IP address to limits traffic to/from these IP addresses.
With MT DNS client & cache http://www.mikrotik.com/testdocs/ros/2. ... scache.php Rapidshare IP addresses will be stored (in cache), every time they are requested by customers.
If it might be possible to add these IP address to a MT address list dinamically (with a script), traffic to/from this address list will be mangled as P2P and problem will be solved.
Could it be possible to write a script to do this? Any advice?

Rgds
 
changeip
Forum Guru
Forum Guru
Posts: 3803
Joined: Fri May 28, 2004 5:22 pm

Re: Fight against P2P on port 80

Fri Aug 10, 2007 12:12 am

if they end up in the cache then you can write a script to make a address-list from them probably. send me an example of them and I'll help you:

/ip dns cache print detail

Sam
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Fri Aug 10, 2007 11:15 am

Thank you Sam,
please find here below cache content:

Flags: S - static
# NAME ADDRESS TTL
0 za.akadns.org 195.219.3.169 11h13m58s
1 zb.akadns.org 206.132.100.105 11h13m58s
2 zc.akadns.org 61.200.81.111 11h13m58s
......
......
56 http://www.rapidshare.com 195.122.131.250 14m22s
57 images.rapidshare.com 195.122.131.251 2m31s
58 rs181l3.rapidshare.com 195.122.131.182 14m24s
59 g.msn.com 207.68.179.219 12m40s

I think that writing a script is quite difficult, because these information are not stored in a file, but simply written in cache...

Rgds,
Carlo
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23993
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Fri Aug 10, 2007 3:45 pm

rapidshare is not p2p, it's just a regular file hosting site.
 
titius
Member
Member
Posts: 338
Joined: Mon Oct 17, 2005 11:43 am
Location: Titel Serbia

Re: Fight against P2P on port 80

Fri Aug 10, 2007 4:00 pm

@normis

Cpresto is aware of that, but many clients use rapidshare, and downloadind all day long. So it is like p2p bandwith is 100% used.

@cpresto

Cmon man, what do you want? buy more bandwith.

Do you expect from your customers to read&send mail. Yeah right. :roll:
 
User avatar
sergejs
MikroTik Support
MikroTik Support
Posts: 6615
Joined: Thu Mar 31, 2005 3:33 pm
Location: Riga, Latvia
Contact:

Re: Fight against P2P on port 80

Fri Aug 10, 2007 4:20 pm

Very interesting option is 'connection-bytes' in firewall/mangle, it allows you to filter/mark/limit connections that are exceeding specific limit (somehow determine which request is to open web-page and which is download file).
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Fri Aug 10, 2007 4:50 pm

Yes sergejs,
this might be a solution, but to mangle a connection and establish limits for this (if port 80 is used), I do have to know its IP address to differentiate it from all the others, so DNS it's back again... :?
I don't want to limit users that download software from web site (antivirus trials, updates, etc...) but I don't want to give them the opportunity to download P2P files all day long from hosting sites like Rapidshare. Since know a small number of them have discovered this "Eldorado", but as this voice goes around our Internet connection will be saturated by this type of downloads. Unfortunately for WISPs bandwidth is a very precious resource and not be wasted...

I suppose that a script will be easier be written if DNS cache information are available in a file (.txt o .csv), could this be possible?

Rgds,
Carlo
 
abab_rafiq
Member Candidate
Member Candidate
Posts: 120
Joined: Thu Aug 24, 2006 12:47 pm
Location: Dhaka

Re: Fight against P2P on port 80

Sat Aug 11, 2007 1:54 pm

Use proxy to drop or down priority of downloading from rapidshare.com

Actually for stop or make priority for file transferring on port 80 it is the way to use layer 7 classifier.
Regular expression, good knowledge in C and some other types of packet marking is needed to do the better.

Rafiq...
 
User avatar
tgrand
Long time Member
Long time Member
Posts: 671
Joined: Mon Aug 21, 2006 2:57 am
Location: Winnipeg, Manitoba, Canada

Re: Fight against P2P on port 80

Sat Aug 11, 2007 3:47 pm

As opposed to trying to identify P2P would it not be better to identify known valid traffic.
pop3, snmp, http, https, sip, irc, ftp, etc.
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Mon Aug 13, 2007 5:57 pm

Hi tgrand,
this is already done (port < 1024) as described in my post.
The problem is that P2P on port 80 (file sharing hosting as Rapidshare) is identified (based on its port traffic) as normal http traffic instead of P2P...
 
User avatar
tgrand
Long time Member
Long time Member
Posts: 671
Joined: Mon Aug 21, 2006 2:57 am
Location: Winnipeg, Manitoba, Canada

Re: Fight against P2P on port 80

Mon Aug 13, 2007 9:22 pm

Yes but http is very easy to identify if you do deep packet inspection
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Mon Aug 13, 2007 9:49 pm

Please note that Rapidshare traffic is normal http traffic on port 80, the same "normal browsing" traffic.
The only difference is that on RapidShare servers are stored files to be shared.
Definetly, it is not P2P traffic but simply "P2P content".
 
User avatar
tgrand
Long time Member
Long time Member
Posts: 671
Joined: Mon Aug 21, 2006 2:57 am
Location: Winnipeg, Manitoba, Canada

Re: Fight against P2P on port 80

Tue Aug 14, 2007 6:23 am

Then mark and queue this range: 195.0.0.0/8
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23993
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Tue Aug 14, 2007 9:34 am

it doesn't seem that cpresto understands what p2p is.

rapidshare is recognized as http precisely for the reason that it is http. p2p content? what's that? I can put my work documents on rapidshare. it doesn't mean that everything on rapidshare is pirated programs.

rapidshare is regular file download from www page over http. the same as you download netinstall from http://www.mikrotik.com !
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Tue Aug 14, 2007 11:27 am

Probably I've not been clear enough... :?
From my post it should be clear that I perfectly understood that Rapidshare is normal http traffic, and this is exactly the problem !!! :(

I do refer to Rapidshare as P2P because it is used like this: with P2P you leave your PC on and download files 24h a day. This is ok if a "normal" P2P program is used, because trafficic on these ports (> 1024) can be mangled and putted into queues to be filtered.
With Rapidshare (and similars) this is not possible, because it is not P2P traffic, but it is simple http traffic... 24h a day at maximum rate allowed for the customer that is using it: a disaster!!! :shock:
The only way (in my understanding) to identify this type of traffic is to identify Rapidshare servers, using DNS request from customers: if a customer looks for "rs181l3.rapidshare.com" (please look @ my second reply to this post), MT DNS proxy replies "195.122.131.182" and store this entry into its cache.
Now I simply check DNS entry cache every two/three days, and add Rapidshare address to MT address list named "Rapidshare": traffic to/from this address list is queued together with P2P traffic. I'm trying to understand if this process might be automated using a script.

Unfortunately it is not possible to identify Rapidshare traffic with the whole 195.0.0.0/8 network, 2^24-2 are too many servers also for Rapidshare and for sure in this huge range other normal web sites are hosted.

Rgds
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23993
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Tue Aug 14, 2007 11:37 am

what do you mean by 24h a day for rapidshare? rapidshare hosts files one by one, you can't download all day from it.

maybe you simply need some burst limitations (user can download fast for some time, then slows down), or use webproxy and block rapidshare entirely?
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Tue Aug 14, 2007 11:58 am

Yes,
you can download files one by one from Rapidshare but, due to their nature (software, porno video,...) customers spend ours a day downloading files.
Burst limitations might be a solution, but IP server identification is necessary as well, otherwise this will be applied also for other web downloads. Blocking it entirelly is not a (polite) solution.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23993
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Tue Aug 14, 2007 1:19 pm

OK, let's just concentrate on "slowing down rapidshare, and not affecting other http". your subject is very misleading.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Tue Aug 14, 2007 4:18 pm

you can create address list that will hold all the rapidshare ip addresses, and for that list create queue that will limit speed available.
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Tue Aug 14, 2007 6:29 pm

Thank janisk,
but, as you can read in my posts... this is exactly what I'm doing :D
The objective is to let MT do this automatically...
 
User avatar
sergejs
MikroTik Support
MikroTik Support
Posts: 6615
Joined: Thu Mar 31, 2005 3:33 pm
Location: Riga, Latvia
Contact:

Re: Fight against P2P on port 80

Wed Aug 15, 2007 8:40 am

How do you imagine this automatically ?
MikroTik should create rules for specific resource rapidshare ? In my opinion not all the users will find this option useful (as resources might be different), unless you have the opportunity to create rules with queues and address-list.
Do you have any problems with address-lists and queues configuration (indeed address-list is already automation tool, as you do not need to create multiple mangle/firewall rules to mark all rapidshare data, but just put one rule to mangle and use address-list with rapidshare addresses).
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Wed Aug 15, 2007 10:56 am

create script, tool that recovers rapdishare ip addresses, and then add them to address-list thats all the automation you need
 
kev23m
just joined
Posts: 12
Joined: Tue Jul 17, 2007 2:23 pm

Re: Fight against P2P on port 80

Wed Aug 15, 2007 11:13 am

I just tried this by collecting the ip addresses of the mirrors by d'loading a file.
Here are the ips i got.

195.122.131.88
212.162.63.88
62.67.57.88
207.138.168.88
80.239.151.88
62.67.46.88
64.215.245.88
195.219.1.88
82.129.39.88
80.129.35.88
80.239.236.88
82.129.36.18
80.239.159.18

Now i added this in mangle n marked the conn as rapidshare and all packets as rapidpackets.
Made a simple queue and tried to limit the speeds but it is not working, even tried dropping these packets but its not catching the d'load at all as i dont see any packets increasing in the mangle rule.
What could be the problem?

Kev
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Wed Aug 15, 2007 11:21 am

read here about mangle:
http://www.mikrotik.com/testdocs/ros/2.9/ip/mangle.php

about queues
http://www.mikrotik.com/testdocs/ros/2.9/root/queue.php

and do not forget that src address list should be used, set protocol tcp port 80
 
kev23m
just joined
Posts: 12
Joined: Tue Jul 17, 2007 2:23 pm

Re: Fight against P2P on port 80

Wed Aug 15, 2007 11:29 am

I am using Mikrotik in bridge mode.

Did try setting tcp dest port to 80 , also tried giving the ip pool assigned , no luck.
It just goes thru as before.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Fight against P2P on port 80

Wed Aug 15, 2007 12:04 pm

address-list is under /ip firewall address-list not /ip pool these are 2 different things

and check wiki, there was example how to limit that if you use bridge
 
Henrik
just joined
Posts: 16
Joined: Tue Jun 20, 2006 11:12 pm

Re: Fight against P2P on port 80

Wed Aug 15, 2007 5:23 pm

Hi

It should is possible to mark http connections exceeding a certain amount of B and classify it with a low priority in ques. And mark http traffic with low amount of Bytes, as surfing, and put it in front of the ques with a high priority.

Then surfing would be higher priority than http Download. And would work on any server, and any http download, even from “legal” sites. We use it, and it works fine.

Sergejs mentioned it at the top of this tread.

Isn’t this the solution or did I miss anything.?
:shock:
Best regards
Henrik
 
User avatar
GWISA-Kroonstad
Member Candidate
Member Candidate
Posts: 111
Joined: Fri Nov 10, 2006 3:34 pm

Re: Fight against P2P on port 80

Wed Aug 15, 2007 8:09 pm

rapidshare is not p2p, it's just a regular file hosting site.
Agree with you Normunds. Users are confusing multiple mirror sites with P2P. Yet, for all those who use FileTopia... Have you seen the extensive P2P-like downloads on HTTPS? Several times realized the download goes straight between the clients. That is definitely P2P! And on HTTPS! What now, limit HTTPs? LOL
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Thu Aug 16, 2007 11:09 am

Thank you Sergej/Janisk,
but what you suggest is what I've written at the beginning of this post (if I correctly understood): I'm using MT DNS cache to identify Rapidshare IP addresses and put them into a dedicated access list, let's call it Rapid_list. Http traffic to/from Rapid_list will be considered as P2P, so P2P queues are applied to this (known process).
At the moment I copy manually these IPs into Rapid_list, what I would like to have is a script that's able to do this for me, this should be the argument of this post.
 
User avatar
sergejs
MikroTik Support
MikroTik Support
Posts: 6615
Joined: Thu Mar 31, 2005 3:33 pm
Location: Riga, Latvia
Contact:

Re: Fight against P2P on port 80

Thu Aug 16, 2007 11:50 am

No, this traffic will not be considered as P2P, it will be considered as rapidshare traffic, that is being marked with mangle by 'address-list' option.

Your argument is clear, the best way to do this, find out all addresses used by rapidshare, create mangle+queues, and create export for this configuration, then copy to all routers.
There is no automated option as far as I know, there is not automatic configuration for this, because other user might want to block/limit another resource.
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Thu Aug 16, 2007 1:23 pm

Thank you sergejs,
yes, it is not P2P, but I "shape" this traffic as it is, in order to limit its download.
This is because I already have a queue for P2P traffic but, once identified, another queue might be created and used specifically for Rapidshare. The problem remains, on how to do this automatically...
I think that the problem is due to the fact that MT does not store DNS cache entries in a file, but simply into its memory. Probably using an external DNS server that stores its entries in a file (better in a mySQL table database) will give better results, because search operation could be done on this file and actions (insert into MT address list) performed by external scheduler (linux chron). Unfortunately I do no have these sw knowledge so I have to look for someone that can do this for me...
 
cmit
Forum Guru
Forum Guru
Posts: 1552
Joined: Fri May 28, 2004 12:49 pm
Location: Germany

Re: Fight against P2P on port 80

Thu Aug 16, 2007 4:15 pm

cpresto,

here is your script:
:foreach i in=[/ip dns cache find] do={
  :if ([:find [/ip dns cache get $i name] "rapidshare"] > 0) do={
    :log info ("rapidshare: " . [/ip dns cache get $i name] . " (ip address " . [/ip dns cache get $i address] . ")")
    /ip firewall address-list add address=[/ip dns cache get $i address] list=rapidshare disabled=no
  }
}
Run this regularly using the scheduler, and it will scan the DNS cache of your MikroTik, and add all addresses the have the phrase "rapidshare" in the DNS name to an address-list named "rapidshare".

As I read your original post, you know how to apply your traffic shaping to addresses in that list, right?

This script will only ADD to the address-list, and as the scripting language does not allow to set a life-time for address-list entries (like you can do from a firewall rule), the addresses will stay there forever (or until manually deleted).
If you want to have a smaller/cleaner/more recent address list, you could add the line
/ip firewall address-list remove [/ip firewall address-list find list=rapidshare]
to the beginning of the script, to clear the address list every time the script is run. Then only hosts which have recently been used (i.e. are in the DNS cache) will be put on the address list.

Does that help?

Best regards,
Christian Meis
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Thu Aug 16, 2007 4:35 pm

Great cmit,
I'll try and let you know

Thank you,
Carlo
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Thu Aug 16, 2007 8:14 pm

Hi cmit,
script works almost fine: IPs whose name is "xxx.rapidshare.com" are added to the list, while IPs that correspond to "rapidshare.com" are not inserted into the list. Please have a look at addresses here below.
Probably something have to be changed into script search criteria...

[admin@AdiesselleP2K] > ip dns cache print
Flags: S - static
# NAME ADDRESS TTL
...........
...........
12 http://www.rapidshare.com 195.122.131.250 8m55s
13 images.rapidshare.com 195.122.131.251 7m7s
14 rs144l3.rapidshare.com 195.122.131.145 8m56s
15 rs178cg.rapidshare.com 82.129.39.179 9m45s

16 rapidshare.com 195.122.131.250 6m44s
17 rapidshare.com 195.122.131.2 6m44s
18 rapidshare.com 195.122.131.3 6m44s
19 rapidshare.com 195.122.131.4 6m44s
20 rapidshare.com 195.122.131.5 6m44s
21 rapidshare.com 195.122.131.6 6m44s
22 rapidshare.com 195.122.131.7 6m43s
23 rapidshare.com 195.122.131.8 6m43s
24 rapidshare.com 195.122.131.9 6m43s
25 rapidshare.com 195.122.131.10 6m43s
26 rapidshare.com 195.122.131.11 6m43s
27 rapidshare.com 195.122.131.12 6m43s
28 rapidshare.com 195.122.131.13 6m43s
29 rapidshare.com 195.122.131.14 6m43s
30 rapidshare.com 195.122.131.15 6m43s
31 rapidshare.com 195.122.131.16 6m43s
32 rapidshare.com 195.122.131.17 6m43s
33 rs148cg.rapidshare.com 82.129.39.149 14m22s

[admin@AdiesselleP2K] ip firewall> address-list
<s-list> print
Flags: X - disabled, D - dynamic
# LIST ADDRESS
......
......
53 rapidshare 195.122.131.250
54 rapidshare 195.122.131.251
55 rapidshare 195.122.131.145
56 rapidshare 82.129.39.179
57 rapidshare 82.129.39.149

[admin@AdiesselleP2K] ip firewall>

Rgds
 
cmit
Forum Guru
Forum Guru
Posts: 1552
Joined: Fri May 28, 2004 12:49 pm
Location: Germany

Re: Fight against P2P on port 80

Thu Aug 16, 2007 10:13 pm

make that ">0" a ">= 0" in the second line and try again...

Best regards,
Christian Meis
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Fri Aug 17, 2007 1:35 pm

Almost done Christian...
it stops when an already inserted entry is found into DNS cache address list, please have a look here below:

[admin@AdiesselleP2K] > ip dns cache print
Flags: S - static
# NAME ADDRESS TTL

.........
55 ns1.rapidshare.com 195.122.131.250 1d19h59m
56 ns2.rapidshare.com 80.237.244.50 1d19h59m
81 rapidshare.com 195.122.131.2 13m10s
82 rapidshare.com 195.122.131.3 13m10s
83 rapidshare.com 195.122.131.4 13m10s
84 rapidshare.com 195.122.131.5 13m10s
85 rapidshare.com 195.122.131.6 13m10s
86 rapidshare.com 195.122.131.7 13m10s
87 rapidshare.com 195.122.131.8 13m10s
88 rapidshare.com 195.122.131.9 13m10s
89 rapidshare.com 195.122.131.10 13m10s
90 rapidshare.com 195.122.131.11 13m10s
91 rapidshare.com 195.122.131.12 13m10s
92 rapidshare.com 195.122.131.13 13m10s
93 rapidshare.com 195.122.131.14 13m10s
94 rapidshare.com 195.122.131.15 13m8s
95 rapidshare.com 195.122.131.250 13m8s :!:
96 rs144cg.rapidshare.com 82.129.39.145 13m8s
97 rs26cg.rapidshare.com 82.129.39.27 13m18s
98 rs91cg.rapidshare.com 82.129.39.92 13m29s
99 rs67cg.rapidshare.com 82.129.39.68 13m50s
100 rs140cg.rapidshare.com 82.129.39.141 13m58s
......

[admin@AdiesselleP2K] > ip firewall address-list print
Flags: X - disabled, D - dynamic
# LIST ADDRESS
...........
53 rapidshare 195.122.131.250
54 rapidshare 80.237.244.50
55 rapidshare 195.122.131.2
56 rapidshare 195.122.131.3
57 rapidshare 195.122.131.4
58 rapidshare 195.122.131.5
59 rapidshare 195.122.131.6
60 rapidshare 195.122.131.7
61 rapidshare 195.122.131.8
62 rapidshare 195.122.131.9
63 rapidshare 195.122.131.10
64 rapidshare 195.122.131.11
65 rapidshare 195.122.131.12
66 rapidshare 195.122.131.13
67 rapidshare 195.122.131.14
68 rapidshare 195.122.131.15 :!:
 
Closed_1
just joined
Posts: 7
Joined: Fri Dec 08, 2006 11:40 am

Re: Fight against P2P on port 80

Sat Aug 18, 2007 11:18 pm

maybe we can add rule checking if the address list contained the same ip from dns cache it will bypass adding ip to addresslist, so next ip will added.
:foreach i in=[/ip dns cache find] do={
  :if ([:find [/ip dns cache get $i name] "rapidshare"] > 0) do={
    :log info ("rapidshare: " . [/ip dns cache get $i name] . " (ip address " . [/ip dns cache get $i address] . ")")
(......add ip rule checking in addresslist here: w/ foreach and if)
    /ip firewall address-list add address=[/ip dns cache get $i address] list=rapidshare disabled=no
(......end of add ip rule checking in addressllist here: w/ foreach and if)
  }
}
thats my suggestion, CMIIW
 
radocicala
Member Candidate
Member Candidate
Posts: 136
Joined: Fri Aug 10, 2007 6:56 pm

Re: Fight against P2P on port 80

Sun Aug 19, 2007 5:59 pm

that new script doesn´t work, could you repair it, it is good idea to check if ip of searched word isn´t allready in address list and if it is there it won´t be written
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Mon Aug 20, 2007 11:30 am

Yes,
it should be something similar, new part in red. It doesn't work, I'm not good enough with scripts :? ...anyone may have a look and correct it?

:foreach i in=[/ip dns cache find] do={
:if ([:find [/ip dns cache get $i name] "rapidshare"] != 0) do={
:log info ("rapidshare: " . [/ip dns cache get $i name] . " (ip
address " . [/ip dns cache get $i address] . ")")
# a new for cicle is need to search for already inserted IP address
# into rapidshare address list
# only new IPs will be added to address list
:foreach f in=[/ip firewall address-list find] do={
:if ([:find [/ip firewall address-list get $f address]] != [:find [/ip
dns cache get $i address]]) do={

/ip firewall address-list add address=[/ip dns cache get $i
address] list=rapidshare disabled=no
}
}
}
}
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5886
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Fight against P2P on port 80

Mon Aug 20, 2007 12:57 pm

replace your red code with this
:foreach f in=[/ip firewall address-list find] do={
   :if ([/ip firewall address-list get $f address ] != [/ip dns cache get $i address] ) do={ 
...
 
radocicala
Member Candidate
Member Candidate
Posts: 136
Joined: Fri Aug 10, 2007 6:56 pm

Re: Fight against P2P on port 80

Mon Aug 20, 2007 2:48 pm

Still doesn´t work, could you make it right? and write it complet, not in parts
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5886
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Fight against P2P on port 80

Mon Aug 20, 2007 4:02 pm

so this is complete WORKING script:
I added comments so that guys, who don't know what each line do, could learn.
# check every dns entry
:foreach i in=[/ip dns cache find] do={
    :local bNew "true";
#  check if dns name contains rapidshare
    :if ([:find [/ip dns cache get $i name] "rapidshare"] != 0) do={
        :local tmpAddress [/ip dns cache get $i address] ;
#---- if address list is empty do not check ( add address directly )
        :if ( [/ip firewall address-list find ] = "") do={
            /ip firewall address-list add address=$tmpAddress list=rapidshare disabled=no;
        } else={
#------- check every address list entry
            :foreach j in=[/ip firewall address-list find ] do={
#---------- set bNew variable to false if address exists in address list
                :if ( [/ip firewall address-list get $j address] = $tmpAddress ) do={
                    :set bNew "false";
                }
            }
#------- if address is new then add to address list
            :if ( $bNew = "true" ) do={
                /ip firewall address-list add address=$tmpAddress list=rapidshare disabled=no
            }
        }
    }
}

# [ THE END ]
 
radocicala
Member Candidate
Member Candidate
Posts: 136
Joined: Fri Aug 10, 2007 6:56 pm

Re: Fight against P2P on port 80

Mon Aug 20, 2007 7:24 pm

Still doesn´t work like it is supposed. It writes every ip saved in cache in DNS. But it is finding only rapidshare when you change
!= 0
for
>= 0
.
But there is still some problem: if there is subnet´s ip in address list(237.138.168.0/24) it is also writes the ip´s that belong that subnet(207.138.168.61). Could it be possible to edit it not to write ips of subnet if the subnet is allready in address list?
 
User avatar
cpresto
Member Candidate
Member Candidate
Topic Author
Posts: 170
Joined: Tue Jul 18, 2006 3:12 pm

Re: Fight against P2P on port 80

Mon Aug 20, 2007 9:24 pm

If I use
>=0, scripts stops if an already addeded IP address is found into address list,
With !=0 it seems ok...
 
radocicala
Member Candidate
Member Candidate
Posts: 136
Joined: Fri Aug 10, 2007 6:56 pm

Re: Fight against P2P on port 80

Mon Aug 20, 2007 11:44 pm

no if you use != 0, all ip written in dns cache are written to address list, check some ip from address list to put it in you browser, if it is rapidshare, the official rapidshare site will open if not no rapidshare site will be shown. Or check dns cache you will see that all ips from there are written to ip address. I am 100% sure, I confirmed it clearing DNS cache, running the script with !=0, no rapidshare was not in DNS cache and it wrote me all ips from DNS cache.(NO RAPIDSHARE).
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5886
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Fight against P2P on port 80

Tue Aug 21, 2007 10:00 am

Still doesn´t work like it is supposed. It writes every ip saved in cache in DNS.
It's not possible ( TESTED, WORKING ). Make sure you typed code correctly.
But there is still some problem: if there is subnet´s ip in address list(237.138.168.0/24) it is also writes the ip´s that belong that subnet(207.138.168.61). Could it be possible to edit it not to write ips of subnet if the subnet is allready in address list?
Yes it is possible. Calculate subnets address range and check if IP is in that range. It's quite easy, but you will have to do it for yourself. If you want to use scripts then learn how to script.
 
sgsmc
just joined
Posts: 9
Joined: Wed Jul 25, 2007 5:01 pm

Re: Fight against P2P on port 80

Thu Aug 23, 2007 2:13 pm

Calculate subnets address range and check if IP is in that range. It's quite easy, but you will have to do it for yourself. If you want to use scripts then learn how to script.

mrz i am newbie

tried to learn script and tried to do it but could naot mange
now need someone. can you give it out how to do
 
User avatar
jdejansb
Frequent Visitor
Frequent Visitor
Posts: 65
Joined: Thu Jul 13, 2006 1:35 pm
Location: Srbija
Contact:

Re: Fight against P2P on port 80

Fri Aug 31, 2007 11:38 am

what do you mean by 24h a day for rapidshare? rapidshare hosts files one by one, you can't download all day from it.

maybe you simply need some burst limitations (user can download fast for some time, then slows down), or use webproxy and block rapidshare entirely?
Hi all, long time since my last post, but here is something that is interesting - using BURSTS. While I had users in MT's internal "base" I used profiles with bursts, everyting worked just fine. Ppl had great speeds for surf and something less for long (large) downloads. And then I switched to a Radius server and sql database and couldn't make bursts to work ......

Is it possible to use BURST when authorize thru RADIUS server? I "see" the INTEGER value for tx/rx speeds, where (if I'd like to use bursts) should be a TEXT string for bursts :( (512k/256k ccc/ddd etc...). Any help on this??

Dejan
 
sgsmc
just joined
Posts: 9
Joined: Wed Jul 25, 2007 5:01 pm

Re: Fight against P2P on port 80

Wed Oct 24, 2007 10:17 pm

Yes it is possible. Calculate subnets address range and check if IP is in that range. It's quite easy, but you will have to do it for yourself. If you want to use scripts then learn how to script.
mrz please!!!!

I am newbie and tried quite a lot but there is some small error and it is not working.
 
wildbill442
Forum Guru
Forum Guru
Posts: 1044
Joined: Wed Dec 08, 2004 7:29 am
Location: Sacramento, CA

Re: Fight against P2P on port 80

Thu Oct 25, 2007 1:20 am

Use OpenDNS as your forwarding servers and just block access to Rapidshare.com :)

I didn't feel like reading through all the replies.... but I agree with one of the other users who mentioned purchasing more bandwidth ;)

either that or setup a queue to limit bandwidth going to the rapid share webservers..

do an nslookup on their content servers domain name and it will give you all the IP's associated with that domain.

ex:
C:\>nslookup
*** Can't find server name for address 192.168.0.2: Non-existent domain
*** Default servers are not available
Default Server:  UnKnown
Address:  192.168.0.2

> rapidshare.com
Server:  UnKnown
Address:  192.168.0.2

Non-authoritative answer:
Name:    rapidshare.com
Addresses:  195.122.131.14, 195.122.131.15, 195.122.131.250, 195.122.131.2
          195.122.131.3, 195.122.131.4, 195.122.131.5, 195.122.131.6, 195.122.13
1.7
          195.122.131.8, 195.122.131.9, 195.122.131.10, 195.122.131.11, 195.122.
131.12
          195.122.131.13

>
William Burnett
Network Engineer

Who is online

Users browsing this forum: Bing [Bot] and 71 guests