Community discussions

MUM Europe 2020
 
Testingpepe
newbie
Topic Author
Posts: 42
Joined: Tue Apr 19, 2005 6:25 pm

P2P the battle for control

Fri Aug 04, 2006 8:47 pm

Ok, let's start this new topic and concentrate all our knowledge (or only your knowledge) here. I spend 6 hours reading about the control (limiting and blocking) of P2P connections but nothing seems to work well.......so, all i want is all of your experiences and what you do to manage this big problem like what is the best rule to apply in any situation. I test everything and allways something manage to evade the rules. When I say "I try everything" I mean EVERYTHING. So, let's start this battle and hope for a solution for all of us.


Regards
 
UniKyrn
Member Candidate
Member Candidate
Posts: 245
Joined: Fri Dec 24, 2004 9:27 pm
Location: Spokane, WA

Fri Aug 04, 2006 10:11 pm

If you seriously want to ban P2P traffic on your network, then the best tool you've got is your own eyes and brain. Graph the traffic for each of your customers and then do periodic spot checks of those graphs. P2P traffic leaves patterns you'll learn to recognize and if you've got connection tracking turned on, you'll quickly learn to spot the mess it leaves behind even after it's been turned off. When you spot it, that customer is now an ex-customer, or if you're feeling generous, give them one warning before they loose their account and make sure you talk to the parents.

When I was fighting this kind of thing, I found that most cases were kids having fun with parents not knowing what they were up to. When the parent would call in and ask why their account had been suspended, I'd explain why and suggest they talk to their kids. More often then not they called back and confirmed their darling children had downloaded this "really neat program" they'd heard about at school and turned it loose.

This assumes you've actually got the authority to terminate customers and that your AUP warns them that P2P traffic isn't allowed. If you don't have that authority, don't waste your time chasing them, it's a sure way to stress yourself out for no gain.
 
believewireless
Member Candidate
Member Candidate
Posts: 231
Joined: Wed Jul 06, 2005 6:30 pm

Fri Aug 04, 2006 10:24 pm

I have sent in multiple supouts on this, so no default "Send in a supout!" It appears encrypted P2P bandwidth is invisible to Mikrotik queues. Take a look at the interfaces and we'll see 3 Mbps of traffic going through but look at simple queues and only 20kbps is happening. Running torch shows the IPs repsonsible for the bandwidth and further limiting them has zero effect.

So not only can P2P bandwidth not be throttled, it can't even be shaped! Any solutions for this?
 
Testingpepe
newbie
Topic Author
Posts: 42
Joined: Tue Apr 19, 2005 6:25 pm

Fri Aug 04, 2006 10:38 pm

UniKym: you dont want your customers think you are a dictator, im a ISP and have the power to terminate customers but i dont want to be like Fidel Castro, just want to control them, giving them the feeling that they can do whatever they want but controling them so they do what they whant but in my way. So the Customer is happy and we can manage our network in our way. I think this is what every ISP want.

Control is best than block
Sorry for my english i whrote the best i can.
 
UniKyrn
Member Candidate
Member Candidate
Posts: 245
Joined: Fri Dec 24, 2004 9:27 pm
Location: Spokane, WA

Fri Aug 04, 2006 11:26 pm

Have you ever thought about the resources it takes for control attempts?

Given we're in the MikriTik Forum, it's reasonable to believe we're talking about P2P traffic over wireless. Ok, think about what that traffic looks like compared to normal residential traffic. Your normal user is going to establish a connection, transfer data, then close it down. Their browser might even have several connections open at a time. Because it's initiated by the user, there is a throttleing that takes place, they don't saturate your AP, especially when they're not in front of the computer.

Now look at what P2P traffic does to you. To even try to shape or drop it, you've got to turn on connection tracking in the Router O/S and that burns resources. Their P2P application forces them to act as a server, so you've now got a flood of incoming connections trying to get to that customer, possibly a violation of your AUP if you don't allow customers to run servers. These connections attempts are going to keep the radio busy during every poll cycle and the more popular the material they're sharing the worse it gets. That one customer begins to monoplize your AP because such a high percentage of the traffic is for them. This is not going to make your other customers happy, I took a fair number of calls about AP's being run into the ground by one or two P2P users to the exclusion of the other 60 paying users.

If you want to try to control it instead of block it, now you've got the resource usage of the queues as well, and the fact that you're in an arms race that you're usually going to be behind in. Just to try to stay even you'll have to install each new O/S version that comes out and pray that you don't get bit by some bug that drops your AP offline.

Oh yeah, as you burn more resources for blocking/controlling, decrease the number of customers you can support through that AP as well. If they're running a CPE112 with limited memory as their client, or some CPE with similar issues, you'll also get more support calls complaining about their CPE going dead or they can't browse the web, all because the CPE's connection table is full of P2P connections.

P2P traffic effects all areas of your network and company, sometimes in ways you didn't realize. How much is your time worth? Enough to get into an arms race with the P2P application writers? Enough to have to upgrade hardware or add more AP's than you expected to need?

It's an ugly mess and it's only going to get worse as the legit uses of P2P become more and more common.
 
Testingpepe
newbie
Topic Author
Posts: 42
Joined: Tue Apr 19, 2005 6:25 pm

Fri Aug 04, 2006 11:45 pm

I dont share your thought but is a valid way to resolve the problem, dont worry about resources thats not the problem. I only ask for another way of control, not for 100% of control but the best who any as used.
 
changeip
Forum Guru
Forum Guru
Posts: 3804
Joined: Fri May 28, 2004 5:22 pm

Sat Aug 05, 2006 2:56 am

You dont really need connection tracking on the CPE do you? Turn it off and save yourself tons of resources at least there. If your blocking p2p at the CPE directly then maybe ...

Sam
 
UniKyrn
Member Candidate
Member Candidate
Posts: 245
Joined: Fri Dec 24, 2004 9:27 pm
Location: Spokane, WA

Sat Aug 05, 2006 7:55 am

Idealy you'd want blocking at both ends of your network. Blocking at your border router would prevent all the incoming connection attempts from messing with your network and that machine probably has more horsepower than your AP's. Blocking at the AP would prevent the user from hogging the AP.

And for blocking, you need the connection table enabled.
 
Nuke
newbie
Posts: 42
Joined: Mon Jul 31, 2006 7:35 pm
Location: South Africa
Contact:

Sat Aug 05, 2006 12:23 pm

We also have problems with P2P, we use a M$ ISA behind our Mikrotik to try to block it. And even now some people get torrents out here and there. Good thing that our ISP shapes our traffic(I live in south africa) So the chances of P2P killing a line is pretty small. You general speed is 10KB/s on a 1Mb line.
 
Ozelo
Member
Member
Posts: 338
Joined: Fri Jun 02, 2006 3:56 am

Sat Aug 05, 2006 4:18 pm

Since speed limiting on some p2p is impossible, I thought that limiting the number of p2p connections per user would fits better. Is that even possible? :|
MTCRE - 1104RE006
MTCINE - 1104INE001
 
Krunoslav
just joined
Posts: 11
Joined: Wed Mar 08, 2006 11:53 pm

Sat Aug 05, 2006 4:40 pm

I know this is offtopic, but best results I've got is with another linux box, and iptables patched with layer7 filter. Yes, it's not as easy as mikrotik, and setup is not easy at all, but I've got rid of almost all p2p traffic.
 
believewireless
Member Candidate
Member Candidate
Posts: 231
Joined: Wed Jul 06, 2005 6:30 pm

Sat Aug 05, 2006 6:37 pm

Care to share what you did to block the traffic? Are you able to shape it as well?
 
User avatar
samsoft08
Long time Member
Long time Member
Posts: 617
Joined: Sat Nov 26, 2005 10:52 pm

Sat Aug 05, 2006 10:36 pm

Block it dont waste your time controling it ..
 
trtmrt
Frequent Visitor
Frequent Visitor
Posts: 62
Joined: Fri Aug 04, 2006 3:44 pm

Sat Aug 05, 2006 11:48 pm

you know that you can gooogllleeee it a little bit and find all the port's used by clients on torrent, dc++, emule etc...

then tag all the connection (connection mark) w/ some mark ... P2P_CONNECTION and then shape it :) ...

or ban all port and let only smpt, pop3, imap and http (https) and other by request of user (MSN, ICQ, ... ) :)
 
Krunoslav
just joined
Posts: 11
Joined: Wed Mar 08, 2006 11:53 pm

Sun Aug 06, 2006 1:46 am

Care to share what you did to block the traffic? Are you able to shape it as well?
Yes, I'm able to shape the traffic, and to block almost any P2P:

here is complete howto: http://gentoo-wiki.com/HOWTO_Packet_Shaping

If you want to shape the traffic, you will have to patch kernel source tree with IMQ patch. After that, you will have to set your own rules for QOS. If you want to be able to mark P2P traffic, you'll have to patch iptables with L7 filter. This way you can mark P2P traffic, pass it to IMQ, and "slow it down", or simply drop it. Anyway, all is written in this FAQ.
If you simply want to DROP all p2p traffic, IMQ is not necessary.

Finally, all of this above is a lot of work to do and to fix it correctly. If you REALLY don't need it, don't do it, you will waste alot of time and effort.

You have this in mikrotik, but i think mikrotik doesn't "recognise" packets so well as does iptables+L7
 
User avatar
samsoft08
Long time Member
Long time Member
Posts: 617
Joined: Sat Nov 26, 2005 10:52 pm

Sun Aug 06, 2006 1:21 pm

very good link.... thanx alot
 
abc123
newbie
Posts: 34
Joined: Fri Mar 31, 2006 6:13 pm

Mon Aug 07, 2006 11:40 am

> It appears encrypted P2P bandwidth is invisible to Mikrotik queues.

correct, partially. Encrypted bittorrent is seen by MT, but as with some other p2p protocols and their detection, you can just drop it and not bandwidth limit p2p connections. Reason is simple : you DON'T have way how to mark packets "these are encrypted bittorrent packets" so you can't shape them afterwards.


> Take a look at the interfaces and we'll see 3 Mbps of traffic going
> through but look at simple queues and only 20kbps is happening.

this could be caused by some other things - like direct communication between associated clients. Have you done default-forwarding=no ? If not, your radio clients can communicate direct between them without your server knowing this.


> Running torch shows the IPs repsonsible for the bandwidth and further
> limiting them has zero effect.
> So not only can P2P bandwidth not be throttled, it can't even be shaped!

these statements are not correct at all. You can limit / shape whatever traffic, p2p or not p2p, encrypted or not. If shaping does not work for you, you are definitely doing something wrong. Two notes :

a) if you try to shape some traffic to very low bandwidth, like 30kbps, shaper is not able to shape this very precisely and exactly.

b) you are not able to shape encrypted p2p protocols. That's whats encrypting is for - hiding real content from you.

c) shaping p2p to very low numbers does not work precisely and may appear to not work at all.


p2p is definitely the biggest hassle in last years and we fight it day and night. There are some partial solutions:

- limit each customer to bandwidth they bought. If you sold him 512kbit/512kbit, limit his line to these numbers and let him do whatever he wants to do. Does he want to download p0rn? Ok, let him be. Does he want to use p2p? Fine, you have to be prepared for that. Of course, this brings another thing in: you have to have reasonable oversell ratio. If you sold 100x megabit connection to your customers and you have only 10Mbit internet connection, you deserve nothing good.

- account traffic each customer is doing with /ip accounting (very very painful to collect, download from server, process and make some solution on this. On the other hand, you will see many things here) or /ip traffic-flow. Export traffic-flow numbers to some Netflow collecting device and analyze what you've grabbed. You can then make reports - daily usage, weekly usage per client, etc, etc.

huh, long post. sorry.
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Mon Aug 07, 2006 12:28 pm

Your an ISP, its not your place to block traffic. P2P is not illegal.

If someone is abusing your TOS cut them off, it they have gone over a datacap limit the entire connection.

Its that simple
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1729
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Mon Aug 07, 2006 1:42 pm

OMG!!! :shock: :shock: I really got tired reading this topic! It is too long for such problem!

So THE PROBLEM:

p2p=all-p2p option have troubles with some encrypted p2p traffic (it is impossible to find some kind of "pattern" in this traffic, so it is impossible to detect them as p2p!)

MY SOLUTION:

1) I divided traffic for each client into 3 groups:
(a) standard_services (HTTP,FTP,DNS,mails,ICMP,VPN,Telnet,
ssh,HTTPS,SFTP)
(b) standard_p2p (without encryption)
(c) other traffic (skype, VOIP, encrypted p2p, and other)

2) For group (c) i created a SFQ queue (SOLUTION ITSELF :) ) and put limitation on it!


EXPLANATION:
Some of clients run into problem with VOIP communication, because encrypted traffic generated too much connections for group (c) queue (SFQ divided available traffic into too many equally small pieces).

I got some calls about this, and my answer was simple - "Please, disable encryption on your p2p and your voip, Skype, on-line game server will work fine!"
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
Testingpepe
newbie
Topic Author
Posts: 42
Joined: Tue Apr 19, 2005 6:25 pm

Mon Aug 07, 2006 5:34 pm

MacGiver, give us some more detailed explanation about your way of control. Seems like is a good solution in the mean time(mean time - mientras tanto o algo asi :P ) mikrotik resolve how to shape this traffic. This Topic was 80% about politics of services but that is not the purpose of the topic....so.....lets change it and post some solutions(technical solutions)


Regards
 
User avatar
sten
Forum Veteran
Forum Veteran
Posts: 920
Joined: Tue Jun 01, 2004 12:10 pm

Tue Aug 08, 2006 1:34 pm

This Topic was 80% about politics of services but that is not the purpose of the topic....so.....lets change it and post some solutions(technical solutions)
Alright, since you started the thread then it's only natural that you post first.
Give us a technical solution. Please spare no detail.
Move along. Nothing to see here.
 
Testingpepe
newbie
Topic Author
Posts: 42
Joined: Tue Apr 19, 2005 6:25 pm

Tue Aug 08, 2006 5:58 pm

I dont find the final solution, that's why I start this topic.

This is the one Im using now:
Mangle
0 chain=prerouting protocol=tcp p2p=all-p2p action=mark-packet
new-packet-mark=p2p passthrough=yes

1 chain=prerouting protocol=udp p2p=all-p2p action=mark-packet
new-packet-mark=p2p passthrough=yes
Filter
2 chain=forward packet-mark=p2p p2p=all-p2p action=drop

Some of the traffic dont get drop just go to the queue (like 10kbps) and others get free pass.
But is the best i can find testing a lot of rules
 
User avatar
sten
Forum Veteran
Forum Veteran
Posts: 920
Joined: Tue Jun 01, 2004 12:10 pm

Re: P2P the battle for control

Tue Aug 08, 2006 7:10 pm

That doesn't sound like what you are saying here:
Ok, let's start this new topic and concentrate all our knowledge (or only your knowledge) here. I spend 6 hours reading about the control (limiting and blocking) of P2P connections but nothing seems to work well.......so, all i want is all of your experiences and what you do to manage this big problem like what is the best rule to apply in any situation. I test everything and allways something manage to evade the rules. When I say "I try everything" I mean EVERYTHING. So, let's start this battle and hope for a solution for all of us.

What made you think it was a battle in the first place?

the solution is fairly easy. consider the following;

when in a sequence of tcp packets will you know the tcp session is initiated by a program doing p2p type transfers?

the first packet?
the second packet?
.... any packet after the third?

and then consider; when you identify wether it's a p2p related tcp session then what do you do about it?

forget about it?
remember it for future reference of that tcp session?
Move along. Nothing to see here.
 
Testingpepe
newbie
Topic Author
Posts: 42
Joined: Tue Apr 19, 2005 6:25 pm

Tue Aug 08, 2006 7:42 pm

I use that solution because trying to control them (so far) is not possible. So I block it, but is not working like a full block, then is like a way of control. I think. For the other things you say I dont understand what you want to expose.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1729
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Wed Aug 09, 2006 3:09 pm

Lets take a look at the problem in more detail:

1) Each p2p traffic consist of two type connections - "search" and "data" connections

2) Only torrents can be encrypted

3) Encrypted p2p = means "data" connection is encrypted and it is (and will be) impossible to determine that this traffic is p2p

4) "search" connections always will be without encryption

5) if you drop the "search" connection it is impossible to create the "data" connection (even encrypted one)

There are no point to drop all-p2p, you can drop only torrent traffic if you would like to drop encrypted traffic

You should really create a per-user limitation, and then user can do whatever he likes with his connection speed!
Last edited by macgaiver on Wed Aug 09, 2006 3:16 pm, edited 1 time in total.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1729
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Wed Aug 09, 2006 3:14 pm

I use that solution because trying to control them (so far) is not possible. So I block it, but is not working like a full block, then is like a way of control. I think. For the other things you say I dont understand what you want to expose.

OMG, why mangle??? Just use this one rule

/ip firewall filter add action=drop chain=forward p2p=all-p2p

or take a look at my previus reply

/ip firewall filter add action=drop chain=forward p2p=bit-torrent
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
n5ltc
Frequent Visitor
Frequent Visitor
Posts: 55
Joined: Sun Jun 13, 2004 7:01 am
Location: Texas

Wed Sep 20, 2006 6:46 pm

I know this topic has probably been beat to death. However, the problem we see with P2P is not with the bandwidth. We have each user set to the speed they signed up for and I don't really care what they do with it. It becomes a problem when the connections per second get very high and cause problems with the AP and other customers on the AP. We have set a limit of 100 connections and a few make and exceed that limit. It is those that we turn off until they clean up. It's either a very agressive P2P, a virus, spyware or someone who mangled their winXP SP2 connection limiting to get more connections and faster P2P downloads.

This is the rule I use for that:

add chain=connections protocol=tcp tcp-flags=syn connection-limit=100,32 connection-state=new action=drop \
comment="" disabled=no
 
hci
Long time Member
Long time Member
Posts: 604
Joined: Fri May 28, 2004 5:10 pm

Wed Sep 20, 2006 7:31 pm

We use Mikrotik to throttle p2p to a degree but it does not get it all clearly. We also track each users bandwidth consumption and have a quotta we really do not enforce. What I was thinking is that in the future we could throttle all the bandwidth hogs back at peak times but let them go during off peak.

Matt
 
User avatar
jdejansb
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Thu Jul 13, 2006 1:35 pm
Location: Srbija
Contact:

Thu Sep 21, 2006 12:41 am

...So THE PROBLEM:

p2p=all-p2p option have troubles with some encrypted p2p traffic (it is impossible to find some kind of "pattern" in this traffic, so it is impossible to detect them as p2p!)

MY SOLUTION:

1) I divided traffic for each client into 3 groups:
(a) standard_services (HTTP,FTP,DNS,mails,ICMP,VPN,Telnet,
ssh,HTTPS,SFTP)
(b) standard_p2p (without encryption)
(c) other traffic (skype, VOIP, encrypted p2p, and other)

2) For group (c) i created a SFQ queue (SOLUTION ITSELF :) ) and put limitation on it!


EXPLANATION:
Some of clients run into problem with VOIP communication, because encrypted traffic generated too much connections for group (c) queue (SFQ divided available traffic into too many equally small pieces).

I got some calls about this, and my answer was simple - "Please, disable encryption on your p2p and your voip, Skype, on-line game server will work fine!"
Ok, old topic, but very usefull :) only, the question is: Could the stuff you sugest be done with PPPoE users ??? And if it could - how?

Dejan :?
 
firebat
Member
Member
Posts: 390
Joined: Mon Apr 11, 2005 8:38 am

Wed Sep 27, 2006 7:23 am

Macgiver:

Do you have multiple queues per user? Could you give us some more details on your config?

Who is online

Users browsing this forum: Bing [Bot], eworm, jebz, jindranix and 81 guests