I’m trying to program a simple MikroTik CPE to pass P2P traffic at a low bandwidth while reserving the bulk of the customer bandwidth for other traffic. I used the instructions anyone can find in multiple places on mikrotik.com:
Then, as a test, I start up a simple torrent – an unencrypted download of version 5.8 from MikroTik’s own website. (I figure if this code is going to work on anything, it should work on that, right?)
Instead, what I see in /ip firewall connections is an absolute boatload of connections, only about 10-20% of which are actually marked with the desired connection mark. All of the other ones are default blank, and my CPE is running at full customer speed servicing them. I know they’re all torrent connections because I’m not running anything but Bit Torrent and Winbox; also because great swathes of them all share the same torrent-ish port numbers to a constellation of different hosts.
This leads me to suspect that “all-p2p” isn’t anywhere near as comprehensive as its name suggests.
It is impossible to recognize peer-to-peer traffic from the first packet. Only already established connections can be matched.
If this is true, how can mark-connection for P2P ever work at all?
Is there some way to re-mark a connection if I discover P2P content in subsequent packets? Does this mean I have to examine all the packets, not just the connection as shown in the sample code?
Certainly someone out there has been more than 20% successful in identifying P2P traffic. How do you REALLY do it?
This leads me to suspect that “all-p2p” isn’t anywhere near as comprehensive as its name suggests.
It’s not. It’s a very old, built in matcher. The p2p protocols since then have evolved a lot, so it doesn’t match a lot. It’ll probably catch eMule traffic and whatnot…it’s mostly useless today.
Is there some way to re-mark a connection if I discover P2P content in subsequent packets? Does this mean I have to examine all the packets, not just the connection as shown in the sample code?
Of course. You can re-mark a connection any time you want, and that connection mark will then be available for all future packets in that connection.
The first packet of a TCP connection is a TCP/SYN. That’s not a lot to go by - all you know at that point are source and destination ports. p2p uses ephemeral ports, so you can’t judge by that. So you have to wait until some packets further down the line can be inspected for actual content outside of just a bog standard three way handshake.
Certainly someone out there has been more than 20% successful in identifying P2P traffic. How do you REALLY do it?
Sorry to say it, but search the forums. This has been discussed in threads with hundreds of posts. The summary - in my opinion - is still this: you can’t detect p2p reliably. Your best bet is to whitelist all traffic you CAN reliably detect (HTTP, HTTPS, SMTP, POP3, IMAP, RDP, SSH, IPsec, etc.) and give it preference, and then to give low priority to everything else. Other people have had luck with fairly complex DNS matchers - current p2p protocols mostly evade application layer inspection.
Then I ran the RouterOS 5.8 torrent again. Rule #2 got exercised 11,000 times. Rule #3 got exercised 38,000 times. Rule #4 got exercised only 5 times, when I expected it to get exercised at least as often as #3. What am I doing wrong?
Your best bet is to whitelist all traffic you CAN reliably detect (HTTP, HTTPS, SMTP, POP3, IMAP, RDP, SSH, IPsec, etc.) and give it preference, and then to give low priority to everything else.
Ouch. Well, I guess I’ll search the forum and see if I can nick somebody else’s comprehensive whitelist. It wouldn’t do to go slowing something down at a customer’s just because I wasn’t specifically aware of it.
I got maybe 20 hits by searching “p2p whitelist” and unfortunately none of them had specifics.
Sure, I can easily enumerate rules for HTTP, HTTPS, POP3, IMAP, SMTP, and SSH, but then things get complicated. What about the slave ports opened by FTP? What about automatic UPNP connections (it’s a CPE router, after all)? What ports are used by Skype, Netflix, Flash, iChat video/audio, iTunes radio, FaceTime, xBox, VOIP, Sorenson videophones for the deaf, etc.? I have customers who use all of these things, and I don’t want to slow them down, especially the video and audio applications.
I feel like I’ve just been drafted into a never-ending task, like collecting Pokemon cards. Can somebody post a set of rules in CLI that I can use as a basis to do this instead of having to reinvent the wheel?
Can somebody post a set of rules in CLI that I can use as a basis to do this instead of having to reinvent the wheel?