Feature Request: real connection counter (defeating p2p)

RouterOS General
1

Hi all,
I need a system that will enable me to count both TCP and UDP connections created by a customer’s router, because I need to identify clearly who is doing p2p traffic and who is doing another unknown traffic (like videogames, they use few connection).
For me this is the only way to resolve definitively the longstanding problem of p2p in my network (about 1000 customers).

I will not block p2p, but I want to give priority to videogames than p2p.
A customer makes a lot of unknown connections → he are using p2p, and videogames (and other little unknown traffic) not working.
A customer don’t use a lot of unknown connections → videogames (and other little unknown traffic) ok.

Today it is impossible for me do this: connection-limit, limit, destination-limit, packet size, connection-rate, connection-bytes…the combination of these… working at 60%, not more.
Dst-port and address-list (for videogame server) are not manageable in a large network.

Dear Mikrotik staff : is it possible see it in the future? Or I must search in the other vendor solutions?

Thank’s a lot for the response and sorry for my bad english

2

You can add simple firewall rule which will count new udp and tcp connections created by user.

You can also open connection tracking, filter out entries of specific user and see all connections that are open by the user.

3

Thanks for the answer,
but I need to set this system to work automatically in our central QOS.

First of all, I detect all of the well kwon connection (HTTP, FTP, POP, IMAP, …) in our main border router ,
then I want to divide the rest of traffics: “small and fast service” and “p2p traffics”. The first have priority on the second ones.
The only way to discern videogames traffic from p2p traffic, is to count the number of the connections generated by the router in that moment or in a determined moment.

I want to collect the customers who are using a lot of unknown connections in a special address-list automatically. These customers, in our QOS, have e low priority when using unknown connections.
The other costumers, with low connections, have high priority when using unknown services (and bit-rate <400kbit).
I’d like to have a simple rule to match customer ‘s ip addresses, when using a lot of connection: something similar to “connection-limit” - but please note that “connection-limit” works with TCP only, and not with UDP.
I hope I have been clear.

Thanks.

4

MT, again: why doesn’t ‘connection-limit’ count UDP connections? are TCP and UDP use different conntrack tables?.. but they are in one tab in Firewall =)

5

I’m trying to do a script to count the connections for costumer IP address, with:

ip firewall connection print count-only where dst-address=x.x.x.x

It always gives me error:

expected value of port

but if I put the port, how can I count the connections for ip address?
In winbox, in the filter options, there are: “dst-address” and “dst-address/port”. In new terminal?

6

try

ip firewall connection print count-only where dst-address~"^x.x.x.x:"
7

:smiley:
Great!

Karma +1

8

Can you xplain this code? Thanks!

9

~ means to use regex instead of string comparison.

^ is regex for start of string.

10

In this moment I haven’t the time to try this script in my network, but last night I wrote these codes in my home router. These drafts I think will help you.
I remember to you the script for detect p2p must work with the unknown connections, so you must mark these with “other”, and the second must be loaded every 30-60 seconds.

If someone tries in a large network these script please report problems (CPU?) and solutions :slight_smile:. Thanks.

# Draft Script for populated address-list.
#Start Address
:local Address 192.168.1.1;
#End address
:local EndAddress 192.168.1.254;

:while ($Address <= $EndAddress) do={
        ip firewall address-list add address=$Address list=P2PUser disabled=yes;
        :set Address ($Address + 1);
}




#Draft Script for detect P2P users
:local Address 192.168.1.1;
:local EndAddress 192.168.1.254;

:while ($Address <= $EndAddress) do={
:if ([ip firewall connection print count-only where dst-address~"$Address:" connection-mark=other] > 15 ) do={ 
 	ip firewall address-list enable numbers=[/ip firewall address-list find address=$Address] ;
} else={       
      	ip firewall address-list disable numbers=[/ip firewall address-list find address=$Address];     
        }
:set Address ($Address + 1);
}
11

I’m back to the issue this week.

Summarizing:
I need to know who doing p2p in my network. The only way I know is to count customer’s (unknow) connections.


But:
If I put this code on CPE:

/ip firewall connection print count-only

I receive, for example, “100” while customer doing p2p.

If I put this code on any routers where traffic passes:

ip firewall connection print count-only where src-address~"IP_address:"

I receive < 4 connections, while customer doing p2p.
I think it is too difficult count udp connections in the routers.

Now the question is:
with mikrotik, I have any chance to see this feature (connection count tcp+udp) in the future?

12

There’s a way to do it, I think, as long as you don’t use connection mark for anything else:

  1. mangle all traffic with connection-mark=no-mark

after some time (when all connections have no-mark):
2) every nth 100 packet, if packet has no connection mark and is incoming jump to:
a) dst-limit based on dst address and port. dst-limit parameters determine the threshold of connections action: mark connection=counted
b) if connection=no-mark action: add dst address to list (dst-limit threshold reached)
b) if no-mark action: mark-connection=counted
c) return

With a script you’ll have to turn on/off rules of step 1 and 2. The result would be a list of addresses that have more connections than limit set in dst address.

(I added every nth 100 packet because I’m not sure if there’s some latency before all packets start having connection-mark after one is marked.)

13

Are you sure about udp?

[admin@MikroTik] /ip firewall mangle add action=add-src-to-address-list address-list=maxed-conn address-list-timeout=10m chain=postrouting 
comment="" connection-limit=!3,32 disabled=no out-interface=ether2-WAN protocol=udp

results in:

failure: connection limit works only with tcp
[admin@MikroTik]
14

Thanks rmichael,
sorry, I don’t understand your rules.
Dst-limit counts pps and not connections per second, right?

15

Well, I am trying a different approach. (I have a wireless network. A fixed cabled might be treated a bit different.)
First of all you have to establish what is most used and what is important to your customers.

  1. Nowadays Skype or Voip or other live communications is popular. It needs moderate bandwidth but high priority on the network

  2. Then we have browsing, mail etc.

  3. Then we have simple (big)file downloads, ftp, http, usenet etc. Movies, files, music etc.

  4. Then we have long lasting downloads, streaming video, webcam.

  5. Gaming

  6. P2P.

  7. Is relative identifiable, I separate Skype traffic by L7-protocol or better, people using Skype phones give their Skype phone separate IP and ´tunnel´ over the network. Voip is good identifiable and can easy be mangled.
    This traffic gets highest priority.

  8. Gaming. Most popular games use ports. I filter for these and give them high priority.

  9. Simple (big) file downloads get moderate priority, and when hit ¨heavy traffic¨ flter limit for long lasting downloads it gets lowest priority.

  10. Browsing etc. get moderate priority while any long lasting port 80 traffic will finally hit the ´heavy traffic´ filter (see 3.)

  11. Streaming video, webcam. Get moderate priority and ´heavy traffic´ filter of “3.” is set such that most streaming video is not ´touched´ by this filter.

  12. P2P is basically filtered out by L7 filter and P2P filter and gets low-moderate priority.

  13. All other traffic not ´seen´ by filters get low priority anyway.

In mange I also have set a “forward” chain ´udp´ filter. It filters udp streams in another “heavy traffic” filter so these downloads are also further down graded to lowest priority. At the same time are all srce addresses that are not coming from my network in this chain put in a address list with 24hrs timer. Now in firewall filter I block all access of these ´foreign´ IP address in the forward chain to my network. Meaning that remote peers (´leechers’) that were put in the list because ´heavy udp traffic´ was flowing to, or coming from, these machines are blocked. This means that these remote peers cannot download from users of my network any more for 24 hours. (Users of my network are ´seeders`= upload for them.) This stops a lot of upload P2P traffic on my network which because of its nature (wireless) profits a lot from not too much upload. It also makes P2P machines on my network less and less interesting for the ´rest of the world´. The rest of the world thinks seeders on my network are slow. The moment they give speed they get blocked for foreign leecher.

Download P2P is basically not specially processed but that doesn’t matter. I don’t care if user downloads 3Mb with P2P program, ftp, video stream or http download. A download is a download and by the normal ´heavy traffic´ filter these long running downloads all get low priority (ex. video). At the same time the normal P2P filters already filter most P2P traffic to low priority traffic anyway. Also because all ¨unknown´ traffic gets in last mangle rule lowers priority anyway.

Users are in their total down and upload limited to certain bandwidths. If they want to hammer their own connection with P2P I don’t care. As long as my network doesn’t suffer from it.
Gamers almost always get good result, as long as their game ports are known and listed and my ´pipes´ are not full.
Voip and Skype is sort of the last traffic that ´feels’ congestion on my network.

udp can also not be filtered for the amount of connections. This is due the nature of udp traffic. These are just send as packages to a destination and technically are no “connections”.
Try to stop certain dst-P2P udp-ports and immediately the P2P programs start using other ports to reach the leecher and send the packages to other ports. The firewall will only learn about it if the package is already passed. I tried to block these dst-ports once and withing an hour had several thousands udp ¨connections¨ coming from my network in connection tracker. I contacted a user and he didn’t notice any decrease in his P2P program.

It is also not possible to limit based on PCC with srce-IP/dst-port or dst-Ip/dst-prt combination.All you do is dropping a certain combination but the P2P programs just opens a ´new´ combination and the P2P traffic just keeps flowing… After all, because the router only knows for a short period about a udp connection (data packet that passed) it can try to limit it but all combined, new or existing, udp ´steams can still occupy the full bandwidth belonging to that srce address.
Since mangle also can´t filter on these connections you have to make a straight “packet marker” that give each udp package a ´label´ to down priority that package and this is eating away the cpu time of the router since the P2P programs each time send new IP-port combination. It also means ALL udp traffic through this mangle filter. Even the VOIP and Skype traffic. To do this in the border router gives it a hell lot of work. For the CPE this can also be too much since it is less powerfull.

So, in stead of really trying to limit or stop P2P traffic I sort of ´discourage´ external ´leechers´ which in return in most programs also sees a reflection in that my users are limited in the amount of download from his P2P network. (“You need to give some to get other…”)
In the end P2P users find these kind of programs less usufull then ordinary http downloads etc and that works out just fine for me…

Since I put this policy in place on my 200+ (average 3M/356k client subscription) clients, P2P traffic, special the uploads, are almost completely gone. Only slow running (low bandwidth) uploads are still around which is just my aim!

But OK, it only runs some weeks now and maybe the complaints still have to start coming…!

Any comments are appreciated. I don’t own the truth and have to work also based on the ´feed back´ of my customers which is not always very helpful. Maybe my system is not that good as I believe it is… and may be there are some side effect I am not aware off…

Oh, sorry for the long text… :confused: