Community discussions

 
p3rad0x
Long time Member
Long time Member
Topic Author
Posts: 603
Joined: Fri Sep 18, 2015 5:42 pm
Location: South Africa
Contact:

Firewall rules only using one CPU

Thu May 11, 2017 2:27 pm

Good day,

Recently our upstream provider has been threatening to terminate our service because they have started to receive a metric Sh!t ton of abuse mails from because internet users on the network is downloading illegal torrents, with 5000 customers that's no surprise.

So i started to setup a method that only logs udp ports from users that is busy downloading torrents.

I used 3 rules, first one is to add all users to a address list timeout 30min
 chain=forward action=add-src-to-address-list layer7-protocol=L7-Torrent 
      src-address=10.0.0.0/8 address-list=Local_Torrent_User 
      address-list-timeout=30m log=no log-prefix=""
second one is to add all the dst torrent traffic to a different address list.
 chain=forward action=add-dst-to-address-list layer7-protocol=L7-Torrent 
      src-address=10.0.0.0/8 address-list=Remote_Torrent_user 
      address-list-timeout=30m log=no log-prefix="" 
and finally is setup a log rule that matches udp connections from src addr list with the dst-addr list and sending that away to a remote logging server.
chain=forward action=log protocol=udp src-address-list=Local_Torrent_User 
      dst-address-list=Remote_Torrent_user log=no log-prefix="torrenttraffic" 
Now i noticed our ccr1072 that was normally running @ about 25% cpu with 2Gbps data is now doing about 45%.

When inspecting tool profile it shows that the L7 matcher is using multiple cpu cores.

When checking resources and cpu there is a core running @ 100%.

Disabling the newly created rules all cores is operating normal again.

Is there any way I can make the rules more efficient?
There you go then you touched something ;-) : it only takes a change in wind direction to screw with your nat :-)
 
ivicask
Member Candidate
Member Candidate
Posts: 238
Joined: Tue Jul 07, 2015 2:40 pm
Location: Croatia, Zagreb

Re: Firewall rules only using one CPU

Thu May 11, 2017 3:16 pm

Good day,

Recently our upstream provider has been threatening to terminate our service because they have started to receive a metric Sh!t ton of abuse mails from because internet users on the network is downloading illegal torrents, with 5000 customers that's no surprise.

So i started to setup a method that only logs udp ports from users that is busy downloading torrents.

I used 3 rules, first one is to add all users to a address list timeout 30min
 chain=forward action=add-src-to-address-list layer7-protocol=L7-Torrent 
      src-address=10.0.0.0/8 address-list=Local_Torrent_User 
      address-list-timeout=30m log=no log-prefix=""
second one is to add all the dst torrent traffic to a different address list.
 chain=forward action=add-dst-to-address-list layer7-protocol=L7-Torrent 
      src-address=10.0.0.0/8 address-list=Remote_Torrent_user 
      address-list-timeout=30m log=no log-prefix=""
and finally is setup a log rule that matches udp connections from src addr list with the dst-addr list and sending that away to a remote logging server.
chain=forward action=log protocol=udp src-address-list=Local_Torrent_User 
      dst-address-list=Remote_Torrent_user log=no log-prefix="torrenttraffic"
Now i noticed our ccr1072 that was normally running @ about 25% cpu with 2Gbps data is now doing about 45%.

When inspecting tool profile it shows that the L7 matcher is using multiple cpu cores.

When checking resources and cpu there is a core running @ 100%.

Disabling the newly created rules all cores is operating normal again.

Is there any way I can make the rules more efficient?
Well im having the same issue with new HEX3, even putting single simple queue kills performance and single core locks to 100%, Mikrotik support told be i should try with multiple TCP streams like torrents, but makes no difference to me, still one core get locked, others CPU cores dont do much and performance suffers..and they suggested much much more powerful / expensive router..

I dont understand how multi cores work on this routers, but why cant use all cores for everything transparently like its single core?
 
freemannnn
Long time Member
Long time Member
Posts: 669
Joined: Sun Oct 13, 2013 7:29 pm

Re: Firewall rules only using one CPU

Thu May 11, 2017 3:20 pm

"So i started to setup a method that only logs udp ports from users that is busy downloading torrents."

"p3rad0x" can u post here your full rules for catching torrrenting. i like the idea of "log udp ports" and i would like to test them. thanx
 
p3rad0x
Long time Member
Long time Member
Topic Author
Posts: 603
Joined: Fri Sep 18, 2015 5:42 pm
Location: South Africa
Contact:

Re: Firewall rules only using one CPU

Thu May 11, 2017 4:14 pm

I use the following Regexp

"^(\x13bittorrent protocol|azver\x01$|get /scrape\?info_hash=get /announce\?info_hash=|get /client/bitcomet/|GET /data\?fid=)|d1:ad2:id20:|\x08'7P\)[RP]"

It seems to be catching most (not all) of the traffic.

I had that rule up for about 2 hours and there was over 10 million packets logged.
There you go then you touched something ;-) : it only takes a change in wind direction to screw with your nat :-)
 
savage
Forum Guru
Forum Guru
Posts: 1213
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:

Re: Firewall rules only using one CPU

Thu May 11, 2017 6:58 pm

Good day,

Recently our upstream provider has been threatening to terminate our service because they have started to receive a metric Sh!t ton of abuse mails from because internet users on the network is downloading illegal torrents, with 5000 customers that's no surprise.
Out of pure curiosity... How much BW, and who's the upstream? With 5K customers, I presume you have your own ASN and IP Space? Why would they be complaining to your upstream?
Regards,
Chris
 
User avatar
nickshore
Member
Member
Posts: 473
Joined: Thu Mar 03, 2005 4:14 pm
Location: Suffolk, UK.
Contact:

Re: Firewall rules only using one CPU

Thu May 11, 2017 7:05 pm

If they are already in your address list then don't match them again on the L7 matcher !
Nick Shore MTCNA MTCWE MTCRE MTCINE MTCTCE
LinITX.com - MultiThread Consultants
Get your MikroTik RBs and Training: http://linitx.com/brand/mikrotik
Official UK MikroTik Distributor
IRC chan: #routerboard on irc.z.je (IPv4 and IPv6)
 
p3rad0x
Long time Member
Long time Member
Topic Author
Posts: 603
Joined: Fri Sep 18, 2015 5:42 pm
Location: South Africa
Contact:

Re: Firewall rules only using one CPU

Fri May 12, 2017 12:42 am

Good day,

Recently our upstream provider has been threatening to terminate our service because they have started to receive a metric Sh!t ton of abuse mails from because internet users on the network is downloading illegal torrents, with 5000 customers that's no surprise.
Out of pure curiosity... How much BW, and who's the upstream? With 5K customers, I presume you have your own ASN and IP Space? Why would they be complaining to your upstream?
Using 2Gbps from neotel atm, we are still in the planning phase so start peering at teraco and get an ASN
There you go then you touched something ;-) : it only takes a change in wind direction to screw with your nat :-)
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1743
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Firewall rules only using one CPU

Fri May 12, 2017 1:44 am

 
p3rad0x
Long time Member
Long time Member
Topic Author
Posts: 603
Joined: Fri Sep 18, 2015 5:42 pm
Location: South Africa
Contact:

Re: Firewall rules only using one CPU

Fri May 12, 2017 12:16 pm

So last night I router most of the torrent users over a different service provider.

And guess what.

The new provider also forwarded the abuse mail from IP-Echelon :(
There you go then you touched something ;-) : it only takes a change in wind direction to screw with your nat :-)

Who is online

Users browsing this forum: No registered users and 103 guests