Community discussions

MUM Europe 2020
 
Zebble
newbie
Topic Author
Posts: 45
Joined: Mon Oct 17, 2011 4:07 am

Public interest in blacklist service w/ MikroTik script?

Thu Jan 12, 2012 5:11 pm

We have a central server that creates a MikroTik .rsc script every night that creates a blacklist address-list using lists from OpenBL, DShield and SpamHaus. A script on the MikroTik then does a "fetch" of this script on a nightly basis and runs it. Corresponding firewall rules then block connections from this address-list. This has been a big help for all the networks sitting behind MikroTik's that we manage.

If there is public interest in this setup, we may consider opening up our central server, and releasing the corresponding simple MikroTik script for public use.

Any interest?
 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2411
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: Public interest in blacklist service w/ MikroTik script?

Thu Jan 12, 2012 7:44 pm

Could you possibly publish just the IP list (without the MT commands)? Perhaps in addition to the version that is a script itself?

I'd imagine some people might have a problem allowing a 3rd party to potentially execute commands, but a "plain list" that they can verify and construct commands from would surely be of help.
PEAR2_Net_RouterOS(1.0.0b6) - My API client in PHP
(Rate my posts? If you want... no pressure...)
 
changeip
Forum Guru
Forum Guru
Posts: 3804
Joined: Fri May 28, 2004 5:22 pm

Re: Public interest in blacklist service w/ MikroTik script?

Thu Jan 12, 2012 7:54 pm

or turn it into a bgp feed and then you can use my script to turn that back into an address-list.
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com
 
Zebble
newbie
Topic Author
Posts: 45
Joined: Mon Oct 17, 2011 4:07 am

Re: Public interest in blacklist service w/ MikroTik script?

Wed Jan 25, 2012 4:50 pm

ChangeIp: That method is very interesting to me! I'm a bit of a BGP newb, so any advice/howto you can offer would be appreciated.

In the meantime, here's the relevant portion of the linux-side script that produces the Mikrotik code to update a "blacklist" address-list, in case anyone wants to use it. This script is run nightly on one of our linux servers, and we then have a script running "fetch" on our MikroTiks that grabs this script over HTML and runs it.

#!/bin/bash
echo "/ip firewall address-list" > /tmp/blacklist.rsc
echo "remove [/ip firewall address-list find list=blacklist]" >> /tmp/blacklist.rsc
wget -q -O - http://feeds.dshield.org/block.txt | awk --posix '/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.0\t/ { print "add list=blacklist address=" $1 "/24 comment=DShield";}' >> /tmp/blacklist.rsc
wget -q -O - http://www.spamhaus.org/drop/drop.lasso | awk --posix '/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\// { print "add list=blacklist address=" $1 " comment=SpamHaus";}' >> /tmp/blacklist.rsc
wget -q -O - http://www.openbl.org/lists/base.txt | awk --posix '/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[9-9]{1,3}/ { print "add list=blacklist address=" $1 " comment=OpenBL";}' >> /tmp/blacklist.rsc

The resultant /tmp/blacklist.rsc is then a script that will remove all entries in the current blacklist, and then add blacklist entries from DShield, SpamHaus and OpenBL.

Hope someone can use this.
 
ditonet
Forum Veteran
Forum Veteran
Posts: 841
Joined: Mon Oct 19, 2009 12:52 am
Location: Europe/Poland/Konstancin-Jeziorna
Contact:

Re: Public interest in blacklist service w/ MikroTik script?

Fri Feb 03, 2012 11:57 am

@Zebble
First of all it's great idea to keep some block-list on main router.
I have one question: openbl.org list contains over 23000 unique addresses,
how this affects router performance?
Even with only one firewall rule which drops connections from this address-list.

TIA,
Grzegorz | MTCNA, MTCRE, MTCSE | konsultacje MikroTik Warszawa
It is a book about a Spanish guy called Manual. You should read it. - Dilbert
 
Zebble
newbie
Topic Author
Posts: 45
Joined: Mon Oct 17, 2011 4:07 am

Re: Public interest in blacklist service w/ MikroTik script?

Fri Feb 03, 2012 6:46 pm

Good point ditonet, which lead me to digging...

I found that our script was only pulling in just over 1000 entries instead of the full ~23K due to a bug! It was only pulling in OpenBL entries that started with a 9 in the last octet! Whoops...

I've loaded the full list on a reasonably loaded RB1200, and CPU didn't change at all, neither did memory for the most part (maybe a meg or two). I did change the firewall jump to only check for "new" connection state packets, so that may have also helped with CPU usage.

I'm going to run the full list for a day on this single router to see if there are any issues, but will likely flip to the much smaller OpenBL 60-day list instead. I've fixed the script, and updated it to include the 60-day list as below:

wget -q -O - http://feeds.dshield.org/block.txt | awk --posix '/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.0\t/ { print "add list=blacklist address=" $1 "/24 comment=DShield";}' >> /tmp/blacklist.rsc
wget -q -O - http://www.spamhaus.org/drop/drop.lasso | awk --posix '/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\// { print "add list=blacklist address=" $1 " comment=SpamHaus";}' >> /tmp/blacklist.rsc
wget -q -O - http://www.openbl.org/lists/base_60days.txt | awk --posix '/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/ { print "add list=blacklist address=" $1 " comment=OpenBL";}' >> /tmp/blacklist.rsc

-zeb
 
User avatar
Squidblacklist
Frequent Visitor
Frequent Visitor
Posts: 54
Joined: Wed Jun 26, 2013 11:06 am

Re: Public interest in blacklist service w/ MikroTik script?

Fri Sep 20, 2013 6:02 am

We are pleased to announce that Squidblacklist.org is the worlds first commercial supplier of domain blacklists
for Mikrotik RouterOS Web Proxy

See press release for more information.

http://www.squidblacklist.org/press_rel ... ilter.html

Who is online

Users browsing this forum: SiB and 24 guests