Community discussions

MikroTik App
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sat Jul 15, 2017 5:14 pm

in my testing, the 64M units are struggling with anything other than the small list. I'm seeing about 60% of the 64M units pull the medium list 10+ times in a row. That is telling me that the 64M units are having kernel panics and rebooting.

At this time, the server is now forcing the small list on 32M and 64M, medium for 128M and 256M, and large for 512M and up.
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Sun Jul 16, 2017 11:41 am

Request to be able to allow or disallow the automatic blacklist update script in the addresslist rsc file from the config file.
:global blScriptVersion;
if ($blScriptVersion != "2.0.1") do={
:local sourceServer "https://mikrotikfilters.com/";
:local scriptName "blInstaller.rsc";
.
.
:do { /ip firewall address-list remove [find where list=dynamicBlacklist] } on-error={}
/system script run blacklistUpdate
} else={ :put "script is current" }
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Sun Jul 16, 2017 11:47 am

in my testing, the 64M units are struggling with anything other than the small list. I'm seeing about 60% of the 64M units pull the medium list 10+ times in a row. That is telling me that the 64M units are having kernel panics and rebooting.

At this time, the server is now forcing the small list on 32M and 64M, medium for 128M and 256M, and large for 512M and up.
So I can't even test with the huge...really huge file how the filter scoring is? ......it is really huge! With the medium list in....not that huge...I have 187MB free RAM.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2865
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Blacklist Filter update script

Sun Jul 16, 2017 12:21 pm

Plenty of free RAM in 951Ui-2HnD ... 60% free.
IBL.PNG
You do not have the required permissions to view the files attached to this post.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Mon Jul 17, 2017 2:46 am

I've shut down the old service (pre 2.0 script).

I found that several users were leaching the large list and rebranding it as their own. They were also trying to probe the server side for exploits.

Again, I offer my list as a free service to the MikroTik community. If people continue to abuse it, I will shut it down completely.
(I've also added one of the offending IP's to the blacklist... I'm sure that will get some attention)
 
xlighting
just joined
Posts: 6
Joined: Wed Apr 02, 2014 6:08 pm

Re: Blacklist Filter update script

Mon Jul 17, 2017 5:55 am

Hi Dave:
I'm not able to see any log output when using scheduler, even if the blDebug is set to 1 in .conf file...
however if I manually run "blacklistUpdate", the logging is shown... is there anything I can do to show the log?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Mon Jul 17, 2017 6:24 am

You should be getting the same logging both ways. If not, check the schedule policy and the script policy and make sure all of the boxes are checked.
 
jo2jo
Forum Guru
Forum Guru
Posts: 1003
Joined: Fri May 26, 2006 1:25 am

Re: Blacklist Filter update script

Mon Jul 17, 2017 9:54 pm

thank you so much for putting all this work into this FREE project, that you offer for us. Its really great!

Today i started getting the alarm on my rb3011 (great idea to singal updates btw!) , so i checked the log and went and found the required script update.

So im now running the latest from OP of this thread (i updated it as of today, my script has :global blVersion "2017.07.05a"; in the script im running)

however, when running this updated script, im still getting the audio beep alarm, and log msg "Script is outdated"
any ideas?
tks


EDIT: Looks like the fix was to NOT replace the old script with the NEW code, but rather to remove OLD script and run the new Auto Install / Updater script from scratch. (im assuming that the initial way i did it was failing maybe bc i didnt have a blacklist.conf file as thats a new feature) -- either way all is good and its updating again!


EDIT: Feature Request: maybe at some point you could add a line in blacklist.conf for "Additional, user specified, Logging Rules to TEMP disable during bl updates" , so that users can optionally specify ADDITIONAL /sys logging rule numbers to have TEMP disabled (or have !firewall temp appended) during updates... obviously you have the 0 "memory" rule taken care of , but i often have Firewall action=remote rules as well (pointing to offsite syslogd servers) , and it would be nice to disable these as well (i had done this myself on the old script, but i assume any mods i make now to the actual script will be overwritten by the auto-update feature, thus blacklist.conf would be the new, proper place to define such a feature)

thanks again for this GREAT script/feature.
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Mon Jul 17, 2017 11:09 pm

I've shut down the old service (pre 2.0 script).

I found that several users were leaching the large list and rebranding it as their own. They were also trying to probe the server side for exploits.

Again, I offer my list as a free service to the MikroTik community. If people continue to abuse it, I will shut it down completely.
(I've also added one of the offending IP's to the blacklist... I'm sure that will get some attention)
Hi Dave, I support you block anyone that tries to collect your list and misuse it, however 'poisoning' your list in a way is not good practice.

I hope not that abuse continues because what you setup, with some help of considerable amount of others as I can see in the thread, is used and recognized by many as an valuable addition. Even so valued that they steal the list.

In the new setup you have much more control over the usage of the list and see how that works out now.

Thanks for all the work and effort you put in this all and we keep in putting in complai...suggestions to improve it even more. ;-)
 
User avatar
Rhoos
just joined
Posts: 22
Joined: Sun Dec 20, 2015 3:48 pm
Location: Costa Rica
Contact:

Re: Blacklist Filter update script

Tue Jul 18, 2017 1:28 am

Greetings Dave, my thanks for such an important job and sharing it with us. My support in making decisions to block abusive users. Thank you!
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Tue Jul 18, 2017 2:11 am

So two things... Some users are simply blocked at my firewall, and now two users have been added to the list itself. I don't see this as "poisoning" as they are the ones that were actively trying to find security holes. (They have been trying SQL injections) Given that they are active attacks, I see them as no different than the botnets and spammers that the list is intended to block.

I find it VERY sad that MikroTik users on this forum would stoop this level. And frankly, if the USA passes this current bill that will allow sys-admins to "hack back" then the next time they pull my list, it will include a command to clear their config. until that time - they will remain blacklisted.
 
eddieb
Member
Member
Posts: 324
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: Blacklist Filter update script

Tue Jul 18, 2017 10:35 am

Hi Dave,

first my compliments for your work and all the effort you put into this !!!

I have updated a couple of different devices and most went fine without a glitch.
Now I am struggling with a RB2011UiAS-2HnD and it does not work ...
I installed the blinstall script, run it and the 2 scripts and 2 schedulers are installed.
The counters on both scripts are updated so something happens.
BUT : no download of the tmp file, no new list with ips to block AND no ENV variables visible ... AND nothing in the log, even on general debug.

Any suggestions ?

Regards,

Eddie
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Tue Jul 18, 2017 7:12 pm

Check for the Scheduler and Script Policies. Make sure that all of the boxes are marked.
You do not have the required permissions to view the files attached to this post.
 
eddieb
Member
Member
Posts: 324
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: Blacklist Filter update script

Tue Jul 18, 2017 7:41 pm

Hi Dave,
tnx for the hint, that did it. For some reason the scripts where created with only read, write, test ...

It works now

Eddie
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jul 19, 2017 12:54 am

Just released 2.0.2 with minor bug fixes. Run the auto-update/install script to update.
 
planetcoop
Member Candidate
Member Candidate
Posts: 140
Joined: Thu May 15, 2014 2:32 pm
Location: Sacramento, CA

Re: Blacklist Filter update script

Wed Jul 19, 2017 1:37 am

Just released 2.0.2 with minor bug fixes. Run the auto-update/install script to update.
Im first on 2.0.2.1 :) Dave if you don't mind, please reach out to me: ------@planetcoop.com I am in the general forum running a btest server with Tom and I am seeing real benefits to this list on spam and attackers of the btest.
Last edited by planetcoop on Wed Jul 19, 2017 3:55 am, edited 1 time in total.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jul 19, 2017 2:11 am

Just pushed out 2.0.2.2 :)
new auto-script-update script is included. It pulls the current version from the server and updates if needed.
 
foxxiu7
just joined
Posts: 5
Joined: Sun Aug 25, 2013 3:30 am

Re: Blacklist Filter update script

Wed Jul 19, 2017 5:57 am

Hi Dave,

First of all thanks for an amazing job and all effort you're putting into this. It's working just fantastic on my hAP-ac router.

A small idea to consider: how about extending firewall filter rules with autoblock functionality for intruders trying to get to a router or network? A dynamic list with banned IP's trying to do excessive pings, scanning ports, attempting DoS attacks, etc?
I'm using your rules from post #2 and having this autoban functionality will just make them more complete and make the network more secure, I think.
 
eddieb
Member
Member
Posts: 324
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: Blacklist Filter update script

Wed Jul 19, 2017 8:36 am

Hi Dave,
tnx for the updates, it seems to work fine on my RB2011 and my RB1100.

But, I have a very strange problem on my test VM CHR ...
Until 2.0.1 it worked fine, with the latest it wipes my blacklistUpdate and blacklistScriptUpdater ... they are EMPTY ...
Screen Shot 2017-07-19 at 07.29.28.png
in winbox they show up red because they are empty ...
nothing in the logs

even when I copy the content back into the scripts from my RB1100 and run again, the scripts are empty after the run

Eddie
You do not have the required permissions to view the files attached to this post.
 
xlighting
just joined
Posts: 6
Joined: Wed Apr 02, 2014 6:08 pm

Re: Blacklist Filter update script

Wed Jul 19, 2017 10:22 am

eddie: give your install script ALL policies(check all boxes, as Dave's picture show above), then you will get the correct content :)
Hi Dave,
tnx for the updates, it seems to work fine on my RB2011 and my RB1100.

But, I have a very strange problem on my test VM CHR ...
Until 2.0.1 it worked fine, with the latest it wipes my blacklistUpdate and blacklistScriptUpdater ... they are EMPTY ...

Screen Shot 2017-07-19 at 07.29.28.png
in winbox they show up red because they are empty ...
nothing in the logs

even when I copy the content back into the scripts from my RB1100 and run again, the scripts are empty after the run

Eddie
 
xlighting
just joined
Posts: 6
Joined: Wed Apr 02, 2014 6:08 pm

Re: Blacklist Filter update script

Wed Jul 19, 2017 10:37 am

Hi Dave:
the update scripts only require read+write+test+policy to run properly(tested with un-check those policies one-by-one), but I see your install script is trying to import update scripts with all policies.. so if the install script does not have all policies checked, it will ends up with EMPTY import files (that is what Eddie was showing)....

I assume most people would prefer only grant a minimum policy set, so would you mind change your install file, to import scripts with only read+write+test+policy?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jul 19, 2017 4:33 pm

Unfortunately, taking away the permissions ends with empty scripts. Taking away ANY of them causes issues - I do not know why. You *SHOULD NOT* need "password" or "sensitive", but removing them causes the failure.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jul 19, 2017 4:36 pm

Hi Dave,

First of all thanks for an amazing job and all effort you're putting into this. It's working just fantastic on my hAP-ac router.

A small idea to consider: how about extending firewall filter rules with autoblock functionality for intruders trying to get to a router or network? A dynamic list with banned IP's trying to do excessive pings, scanning ports, attempting DoS attacks, etc?
I'm using your rules from post #2 and having this autoban functionality will just make them more complete and make the network more secure, I think.
It's a bit beyond the scope of the blacklist. But I do agree.
Right now, I don't use any auto-ban because a current bug in RouterOS. The dynamic address-lists are not expiring as expected, which will cause a lot of false positives.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2865
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Blacklist Filter update script

Wed Jul 19, 2017 6:01 pm

Please do not mix sugar and salt in one script :-)

List is a list. Period.
Rules should be not installed automatically as they could have influence on all other rules.
Where to install them? At the beggining, before all others? At the end?
Big NO, NO, NO for mixing things in this script.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jul 19, 2017 6:51 pm

No worries, I have no intention of including rules beyond the basic examples provided in the initial posts.
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Wed Jul 19, 2017 8:00 pm

So two things... Some users are simply blocked at my firewall, and now two users have been added to the list itself. I don't see this as "poisoning" as they are the ones that were actively trying to find security holes. (They have been trying SQL injections) Given that they are active attacks, I see them as no different than the botnets and spammers that the list is intended to block.

I find it VERY sad that MikroTik users on this forum would stoop this level. And frankly, if the USA passes this current bill that will allow sys-admins to "hack back" then the next time they pull my list, it will include a command to clear their config. until that time - they will remain blacklisted.
I have an ethical standpoint in this and what laws enable is not always sane. If you attack the attacker than you find yourself both back on the same level.

I assume that you have excellent means to defend yourself in this all. Also the IP address you are thinking to can easily be an VPN or use an other mean for hide their real address and if not then they are not that smart.
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Wed Jul 19, 2017 8:15 pm

Hi Dave,

First of all thanks for an amazing job and all effort you're putting into this. It's working just fantastic on my hAP-ac router.

A small idea to consider: how about extending firewall filter rules with autoblock functionality for intruders trying to get to a router or network? A dynamic list with banned IP's trying to do excessive pings, scanning ports, attempting DoS attacks, etc?
I'm using your rules from post #2 and having this autoban functionality will just make them more complete and make the network more secure, I think.
I have been there and you are building a list of your own however the chance to have a secondary hit on that address is small. There are many devices in the net that are trying yo have a response.
Better is to only allow the services that you offer and protect those very well. All the rest of the traffic you drop. I have the blacklist running which filters 2048 tries and after that I have a filter port 22 and 23 which result in another 2163 hits and then I have the service filter that then filters an other 1280 tries which makes over 5000 tries in a little more than four days.

The blacklist is an valued addition in the whole concept but like a virusscanner living in the past not the now and here.
 
User avatar
Rhoos
just joined
Posts: 22
Joined: Sun Dec 20, 2015 3:48 pm
Location: Costa Rica
Contact:

Re: Blacklist Filter update script

Wed Jul 19, 2017 8:32 pm

"I have the blacklist running which filters 2048 tries and after that I have a filter port 22 and 23 which result in another 2163 hits and then I have the service filter that then filters an other 1280 tries which makes over 5000 tries in a little more than four days."

Greetings msatter, can you share those filter rules for a beginner? Thank you !
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Wed Jul 19, 2017 8:46 pm

"I have the blacklist running which filters 2048 tries and after that I have a filter port 22 and 23 which result in another 2163 hits and then I have the service filter that then filters an other 1280 tries which makes over 5000 tries in a little more than four days."

Greetings msatter, can you share those filter rules for a beginner? Thank you !
Only use the first one TCP and thee has as avaiable services mail and website.

viewtopic.php?f=9&t=98804&p=607503&hilit=Raw#p607503

Because I don't access my router from outside on port 23 and 22 I drop those a simple rule also again with my WAN as entry point pppoe-out1 and your wan could have a different name.
 
User avatar
Rhoos
just joined
Posts: 22
Joined: Sun Dec 20, 2015 3:48 pm
Location: Costa Rica
Contact:

Re: Blacklist Filter update script

Thu Jul 20, 2017 4:37 am

Thank you msatter !
 
eddieb
Member
Member
Posts: 324
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: Blacklist Filter update script

Thu Jul 20, 2017 9:16 am

Morning,

tnx for explaining the script rights issue, to bad we are struggling with that, for now it works here.

@Dave
I noticed the script got updated to 2.0.3 in the past 12 hours, it would be nice to see some kind of changelog if possible ?

Keep up the good work !

Eddie
 
nico599
just joined
Posts: 2
Joined: Mon Jun 12, 2017 11:42 am

Re: Blacklist Filter update script

Thu Jul 20, 2017 12:43 pm

Hi all
i'm running in my ccr-1009-8G-1S-1S+
log is show notthing
but
Script List show this messeage https://goo.gl/yYE2do Image
messeage is "
LOG 【;(eval (eval /putmessage=$t) (eval /log warningmessage=$t))】
urlEncode【;(eval (eval /localname=$temp) (eval /forcounter=$i;do=;(eval (eval /localname=$char;value=(eval (eval /pickbegin=$i;counter=$t))) (eval /ifcondition=(= $char );do=;(eval (eval /setname=$char;value=%20) /)) (eval /ifcondition=(= $char -);do=;(eval (eval /setname=$char;value=%2D) /)) (eval /ifcondition=(= $char /);do=;(eval (eval /setname=$char;value=%2D) /)) (eval /ifcondition=(= $char &);do=;(eval (eval /setname=$char;value=%26) /)) (eval /ifcondition=(= $char =);do=;(eval (eval /setname=$char;value=%3D) /)) (eval /setname=$temp;value=( . $temp $char)) /);from=0;to=(- (eval (eval /lenvalue=$t)) 1)) (eval /returnvalue=$temp) /)】

how can i do?
 
User avatar
inteq
Member
Member
Posts: 406
Joined: Wed Feb 25, 2015 8:15 pm
Location: Romania

Re: Blacklist Filter update script

Thu Jul 20, 2017 2:27 pm

Thank you for the script, but I have to say that, as least in my limited testing, I stumbled upon too many blocked gmail servers.
I couldn't even send an email from my gmail account to my corporate address.
The worst part is that gmail somehow didn't even alert me that the message did not go through. Even after one day.
So I have to pass on this one.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Jul 20, 2017 6:40 pm

Hi all
i'm running in my ccr-1009-8G-1S-1S+
log is show notthing
but
Script List show this messeage https://goo.gl/yYE2do Image
messeage is "
LOG 【;(eval (eval /putmessage=$t) (eval /log warningmessage=$t))】
urlEncode【;(eval (eval /localname=$temp) (eval /forcounter=$i;do=;(eval (eval /localname=$char;value=(eval (eval /pickbegin=$i;counter=$t))) (eval /ifcondition=(= $char );do=;(eval (eval /setname=$char;value=%20) /)) (eval /ifcondition=(= $char -);do=;(eval (eval /setname=$char;value=%2D) /)) (eval /ifcondition=(= $char /);do=;(eval (eval /setname=$char;value=%2D) /)) (eval /ifcondition=(= $char &);do=;(eval (eval /setname=$char;value=%26) /)) (eval /ifcondition=(= $char =);do=;(eval (eval /setname=$char;value=%3D) /)) (eval /setname=$temp;value=( . $temp $char)) /);from=0;to=(- (eval (eval /lenvalue=$t)) 1)) (eval /returnvalue=$temp) /)】

how can i do?
set the blDebug in the config to 1

the code you pasted is the environment, not script list. those are the functions used for url encoding and displaying the log, when enabled.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Jul 20, 2017 6:40 pm

Morning,

tnx for explaining the script rights issue, to bad we are struggling with that, for now it works here.

@Dave
I noticed the script got updated to 2.0.3 in the past 12 hours, it would be nice to see some kind of changelog if possible ?

Keep up the good work !

Eddie
release notes are in the first post. 2.0.3 is included there.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Jul 20, 2017 6:43 pm

Thank you for the script, but I have to say that, as least in my limited testing, I stumbled upon too many blocked gmail servers.
I couldn't even send an email from my gmail account to my corporate address.
The worst part is that gmail somehow didn't even alert me that the message did not go through. Even after one day.
So I have to pass on this one.
Yes, unfortunately, Google is now allowing spammers to use their servers for a price. You are welcome to create a whitelist of servers that you do not want blocked. Unfortunately Google is using their size to try and force admins to stop using block lists. They make money on spam. For this reason, I do not use or support google.
 
jgro
newbie
Posts: 49
Joined: Sat Jun 10, 2017 7:33 am

Re: Blacklist Filter update script

Thu Jul 20, 2017 11:03 pm

I go away for a week and everything has changed. :shock:

@IntrusDave, thank you again for all your work on this blacklist.

Unfortunately for me, the automated scripting is now too intrusive and is itself a serious security risk, so I'm out. If in the future you resume publishing a blacklist of addresses/networks that I can import using my own scripting I will probably use that. Meanwhile, I will just use the service from squidblacklist.org that repackages a few public lists and has not caused me any false positive problems.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Jul 20, 2017 11:37 pm

I go away for a week and everything has changed. :shock:

@IntrusDave, thank you again for all your work on this blacklist.

Unfortunately for me, the automated scripting is now too intrusive and is itself a serious security risk, so I'm out. If in the future you resume publishing a blacklist of addresses/networks that I can import using my own scripting I will probably use that. Meanwhile, I will just use the service from squidblacklist.org that repackages a few public lists and has not caused me any false positive problems.
Add this to the config file. Auto-update is not disabled by default, and can be enabled by setting this to "yes"
:global blScriptUpdate "no";
 
ilivlad
just joined
Posts: 14
Joined: Tue Mar 12, 2013 2:02 pm

Re: Blacklist Filter update script

Fri Jul 21, 2017 1:19 am

Thanks Dave for the great work!

One thing I would like to see is maybe to add an entry to config for update interval,
once an hour is a bit excessive for me (or its just me :) )
Originally it was once a day and it was ok for me.

Thanks again!
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Fri Jul 21, 2017 2:10 am

The script is called once an hour, however that only means that you will make a single DNS lookup to see if the filters have changed. If there is no change, then no update is downloaded. If the DNS returns a newer serial number than the current installed list, then the new list is downloaded.

The list is regenerated several times though the day. A regeneration is triggered when more than 100 addresses change, or if more than 10 full subnets change.

So, polling each hour isn't much bandwidth at all (just a DNS lookup).
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Fri Jul 21, 2017 2:27 am

The frequency can be determined by yourself by setting that in the scheduler to a time that suits you. And as Dave wrote the inquiry itself is not big.

@Dave, if you only have additions to the list and no removals then you could send only those additions, without a removal of all addresses. And thinking even further on that, you can do each 6 or 12 hour total removal of the list and in the hours in between only the additions.

You keep maximum protection and limited time false positives but reduce the traffic considerable on both ends.
 
ilivlad
just joined
Posts: 14
Joined: Tue Mar 12, 2013 2:02 pm

Re: Blacklist Filter update script

Fri Jul 21, 2017 10:18 am

Thanks for your reply guys!
I did set the scheduler time for 24 hours but that gets overwritten with the script update.

My concern is towards writes to the storage device and possible bad blocks.
For example, mine rb2011 is serving me for some 20 months.
Im using Dave's script for a month now and I got 1.5 million total sector writes for this time period.
And I had 2 million total before.
Im not saying this is too much but perhaps guys from Mikrotik can advise where we should keep NAND wear and tear.
Btw msatter idea of updating only the difference sounds good, add the new ones, remove whitelisted, if
thats less expensive Mbyte wise, if Dave didnt already take that into account.
Thanks guys for the good work again!
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Fri Jul 21, 2017 10:53 am

You can now disable the auto updates and Dave write earlier about that for the 2.04 update.

The list are dynamic addresses and are flushed on reboot and have a look at your available RAM after loading the list. It should be in RAM and not in Flash.

When you add the removals to a whitelist it would require a extra rule in firewall beside the already static-whitelist I got. The main fear of Dave is that thing would get out of sync.

However this could work and dynamic and static can be in the same addresslist as for the white and the black list and Dave only removes the dynamic ones on update and less frequent all dynamic address white or black.

sequence: White containing static+dynamic
Black static+dynamic

Problem could be when adding to a list and the IP already exist then you get an error and the script stops. However on smaller updates "on-error" can be used due to the limited nummber of addresses added.

*****update***** You can selectively remove only the dynamic addresses by using this line:
:do { /ip firewall address-list remove [find where dynamic && list=intrusBL] } on-error={}
Last edited by msatter on Sat Jul 22, 2017 9:57 am, edited 2 times in total.
 
rllavona13
just joined
Posts: 7
Joined: Mon Nov 28, 2016 12:41 am

Re: Blacklist Filter update script

Fri Jul 21, 2017 5:39 pm

Sorry the Script is broken? We used it for months now, today we notice that the script is deleting itself and creates 3 empty scripts.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Fri Jul 21, 2017 5:55 pm

The previous version has been disabled because of abuse. Please remove all the blacklist scripts, and run the installer from the first post.
It provides you with a much more stable and flexible platform. Once installed, read over the .conf file and make changes to suit your needs.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Fri Jul 21, 2017 5:57 pm

I've updated the server side to prevent units with 64M or less from pulling list 3. It's simply too big and causes the units to panic with an out of memory error. I watched one unit download the list and reboot more than 30 times last night, until I forced it to grab list two on the server side.
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter update script

Fri Jul 21, 2017 6:20 pm

Thank you for the work and offering the service to us other MikroTik users.
I found that several users were leaching the large list and rebranding it as their own. They were also trying to probe the server side for exploits.

Again, I offer my list as a free service to the MikroTik community. If people continue to abuse it, I will shut it down completely.
That would be a pity but understandable.
Or in the script a key query. The key is only available by e-mail request, with forum user name. You only send the key to forum users who are members of n-years or who have x-post's.
But that would be a lot of work :(
So two things... Some users are simply blocked at my firewall, and now two users have been added to the list itself. I don't see this as "poisoning" as they are the ones that were actively trying to find security holes. (They have been trying SQL injections) Given that they are active attacks, I see them as no different than the botnets and spammers that the list is intended to block.

I find it VERY sad that MikroTik users on this forum would stoop this level.
+1
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Fri Jul 21, 2017 6:35 pm

I've updated the server side to prevent units with 64M or less from pulling list 3. It's simply too big and causes the units to panic with an out of memory error. I watched one unit download the list and reboot more than 30 times last night, until I forced it to grab list two on the server side.
The huge list is taking a lot of memory and 64 is not enough certainly if you do a remove and read in. I am not trying for the first time the huge list and it took 7 minutes to push it into the addresslist.

It took extra 40MB in RAM than the medium list so not for 64MB routers as you experienced for yourself. On a reboot I am left with 112MB of free RAM and that will go down when the first refresh is a fact.

Update: with the huge list loaded and after a refresh I had 82MB left so about 100MB used extra compared to the medium list. The 7 minutes reload is a big factor to me and I am sticking to the medium list for the moment.
Last edited by msatter on Sat Jul 22, 2017 9:46 am, edited 1 time in total.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Fri Jul 21, 2017 6:48 pm

for those interested, the DNS now holds the list sizes.
{
  :local list1 [ :resolve server=mikrotikfilters.com server-port=6502 domain-name=127.0.0.4 ];
  :local list2 [ :resolve server=mikrotikfilters.com server-port=6502 domain-name=127.0.0.5 ];
  :local list3 [ :resolve server=mikrotikfilters.com server-port=6502 domain-name=127.0.0.6 ];
  :put "List 1 Entries: $list1\n\rList 2 Entries: $list2\n\rList 3 Entries: $list3";
  :log warning "List 1 Entries: $list1\n\rList 2 Entries: $list2\n\rList 3 Entries: $list3";
}
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Sat Jul 22, 2017 10:34 am

OK writing this I see a problem with the blacklist containing static and dynamic addresses because you can get collisions on the complete import and it will stop it and leave you with a incomplete blacklist loaded. So for now the request to have already the dynamic added to the removal to have it in place in case a solution is found for the collisions in the blacklist. I don't see Mikrotik enabling on-error standard when :do is put in a variable and so an easy solution for the collisions in only the blacklist is unsure

Here starts my original request:

Request to be able to have static addresses in the blacklist and on complete refresh only the dynamic addresses are removed?
:do { /ip firewall address-list remove [find where dynamic && list=intrusBL] } on-error={}
[/i]

This can also be used for a whitelist setup (intrusWL) for freed IP addresses that not longer have to be blacklisted. Saving time on reloading the huge list to often and eventually the medium list.
:do { /ip firewall address-list remove [find where dynamic && list=intrusWL] } on-error={}
In the filtering/RAW firewall the whitelist is above the blacklist and have an accept as action. The blacklist still has an drop and both are able to contain dynamic and static addresses at the same time.

On a complete reload both lists are cleared with the only dynamic settings and the blacklist is populated again with the new addresses to block. This can happen once a day due to the 1d+1hour timeout.
Every hour the client contacts the server for an up update and the server decides if it will give a update or a complete list. The update list does not have a remove in it and only add addresses. Because the number of additions is not that big the lines can be writen out completely and you can use on-error if someone has an static address in the list that matches the dynamic address to be added.
Adding addresses is done in the blacklist but also in the whitelist and the whitelist will contain besides static addresses also dynamic addresses of those addresses that don't have to block any more. Those addresses will stay in the whitelist until the next complete remove is done and they are not needed any more because the complete blacklist does not contain them any more...or thay must have been naughty again in the meantime.

Thanks to Jo2jo for the whitelist idea.
 
RFCOM
just joined
Posts: 3
Joined: Mon Jul 24, 2017 7:31 pm

Re: Blacklist Filter update script

Mon Jul 24, 2017 8:12 pm

Hello,

I need your help, I had the previous script working and I could see the whole list, after this change I am copying the new script but it does not work, I get the following error.


[mkadmin@MikroTik] > :global blScriptVersion;
[mkadmin@MikroTik] > if ($blScriptVersion != "2.0.1") do={
{... :local sourceServer "https://mikrotikfilters.com/";
{... :local scriptName "blInstaller.rsc";
{... .
{... .
{... :do { /ip firewall address-list remove [find where list=dynamicBlacklist] } on-error={}
{... /system script run blacklistUpdate
{... } else={ :put "script is current" }
script is current
[mkadmin@MikroTik] > :do {
{... :local currentScriptVersion [ :resolve server=mikrotikfilters.com server-port=6502 domain-name=127.0.0.2 ]
{... :put "Installing blacklistUpdate script version: $currentScriptVersion";
{... :local sourceServer "https://mikrotikfilters.com/";
{... :local scriptName "blInstaller.rsc";
{... :put "Downloading update script...";
{... :do {
{{... /tool fetch url="$sourceServer$scriptName" mode=https dst-path="/$scriptName";
{{... } on-error={
{{... :put "Error. Download failed";
{{... }
{... :put "Importing update script...";
{... :do {
{{... /import "$scriptName";
{{... } on-error={
{{... :put "import failed. unknown error.";
{{... }
{... :put "Removing update script...";
{... :do {
{{... /file remove "$scriptName";
{{... } on-error={}
{... :put "Update Complete.";
{... }
failure: dns server failure

Thanks

BR

Note: I'm not an expert, I'm sorry
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Mon Jul 24, 2017 10:00 pm

Either your firewall is blocking DNS to my server, or your IP is blocked by the list already.

What is your public IP?
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Wed Jul 26, 2017 3:27 pm

A update on the score and RC contains a fix for the not timing out of the address list entries.

In 6.41RC3 has the following fix:
firewall - properly remove "address-list" entry after timeout ends;
I keep away for a while from this because I get less speed by dropping master-slave and going to bridge-'slave'

My score after a new start and now 4 days and 14 hours I have the following score. The score is in the position in the filtering list:

drop port 22,222,2222,2323 2776 hits
drop IntrusBL (blacklist) 1970 hits
drop unknown services 1281 hits

static whitelist in/out 180/231 hits
static blacklist 0 hits

I don't use external ports 22,222,2222,2323 so those are always blocked.

I am going now try switching unknown services and the blacklist and reset the counters. The blacklist blocks a lot of traffic that looks legit but is not and so it is not reaching the second line of defence at service level. I had some false positives but those are easy added to the static whitelist that I have now.
 
User avatar
sri2007
Member Candidate
Member Candidate
Posts: 205
Joined: Wed May 20, 2015 10:14 pm
Location: Lake Grove, NY

Re: Blacklist Filter update script

Wed Jul 26, 2017 6:21 pm

Hi Dave, can you help me to check if my public IP is banned on your list?? The script was working fine, but now i've trying several things to make it work again with your updates, but it allways shows me a dns error. If I post my public IP here, it makes me an easy target :) so, i'm wondering if i can send you that info via any message, I also need more information about your DNS, because in our Core Router I blocked DNS requests from any server that is not allowed, so i need an IP of your DNS server to check if that can help me.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jul 26, 2017 10:01 pm

The DNS address is "mikrotikfilters.com" port 6502. The IP changes, based on current load - so if you add it to an address-list, just put the domain name and let it resolve. The port shouldn't be blocked, unless you are doing level 7 DNS filtering.

you can post the last 3 octets of your IP, and still remain fairly anonymous.
 
User avatar
sri2007
Member Candidate
Member Candidate
Posts: 205
Joined: Wed May 20, 2015 10:14 pm
Location: Lake Grove, NY

Re: Blacklist Filter update script

Thu Jul 27, 2017 3:58 am

Great thanks for your help!! I added a new address-list associated with your domain and it works, thanks for your help! It seems that i'm not banned.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Jul 27, 2017 8:05 pm

I've updated to 2.0.5. This update moves the server to tcp port 6501 and udp port 6502. These ports are excluded from being blocked by the list on my end, and should allow users on "bad" subnets to pull the list.

make sure to give positive rating if you are using and like this service
 
hhgttg42
just joined
Posts: 8
Joined: Wed Oct 12, 2016 4:48 am

Re: Blacklist Filter update script

Sat Jul 29, 2017 4:48 pm

Hi Dave,

Have you or anyone else noticed that their routerboard emits a chime (alternating high/low like a siren 2-3 times) around when the logging is turned back on and the script completes?? I can't find anything in your scripts or in my setup that would be causing this...it's beeping at least at 9:33am/pm EDT daily like clockwork...

I am running your latest script without modification on a RB951G-2HnD.

Thank you.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sat Jul 29, 2017 9:33 pm

Yes. If you notice in your log, it is telling you that the script is out of date. The server inserts an alarm into the script when your local script version is out of sync with the server. You can run the code in the first post to keep your script up to date with the latest big fixes.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Mon Jul 31, 2017 5:43 am

Due to a bug in RouterOS, versions below 6.36 are now blocked on the server side.
It appears they they are not able to compare the local blacklist serial with the server side serial.
This is causing the 6.35 and earlier routers to update constantly.

So, to save the NAND as well as bandwidth, I've chosen to block them and attempt to alert the owner.
 
Taylor
newbie
Posts: 33
Joined: Wed Aug 13, 2014 12:27 am

Re: Blacklist Filter update script

Thu Aug 03, 2017 4:06 am

Did you make a typo? Its telling me versions below 6.38 are blocked. im on 6.37.5
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Fri Aug 04, 2017 11:18 am

My score after a new start and now 8 days and 19 hours I have the following score. The score written in the position in the RAW filtering list:

drop port 22,222,2222,2323 = 5555 hits
drop unknown services = 5409 hits
drop IntrusBL (blacklist) = 2820 hits

static whitelist in/out = 407/591 hits
static blacklist = 0 hits

So the Blacklist adds, from the already filtered traffic, an other 20% and in my case the most on port 25 (mail) which is a really good score.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2865
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Blacklist Filter update script

Fri Aug 04, 2017 11:39 am

My busiest mail filtering router router. Stats for 27 days of uptime.
52 000 all SMTP/IMAP/POP access tries.
8600 unique sources of traffic
1200 blocked by my own list
26 000 blocked by Dave's list.
No client's complains for missing mails
smtpmm.PNG
You do not have the required permissions to view the files attached to this post.
 
RFCOM
just joined
Posts: 3
Joined: Mon Jul 24, 2017 7:31 pm

Re: Blacklist Filter update script

Fri Aug 04, 2017 8:37 pm

Either your firewall is blocking DNS to my server, or your IP is blocked by the list already.

What is your public IP?
181.225.100.117
190.253.66.37
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Fri Aug 04, 2017 9:12 pm

Either your firewall is blocking DNS to my server, or your IP is blocked by the list already.

What is your public IP?
181.225.100.117
190.253.66.37
Your ISP /AS262186 is UCEPROTECT-Level3 listed for hosting a total of 462 abusers.
Your ISP COLOMBIA TELECOMUNICACIONES S.A. ESP/AS3816 is UCEPROTECT-Level3 listed for hosting a total of 5478 abusers.
Unfortunately, both of your IP's fall into ASN's that are blocked.
If you are using script version 2.0.5, then you should be able to pull the current blacklist, as the DNS and HTTPS servers are on unfiltered ports.
 
RFCOM
just joined
Posts: 3
Joined: Mon Jul 24, 2017 7:31 pm

Re: Blacklist Filter update script

Fri Aug 04, 2017 9:57 pm

Either your firewall is blocking DNS to my server, or your IP is blocked by the list already.

What is your public IP?
181.225.100.117
190.253.66.37
Your ISP /AS262186 is UCEPROTECT-Level3 listed for hosting a total of 462 abusers.
Your ISP COLOMBIA TELECOMUNICACIONES S.A. ESP/AS3816 is UCEPROTECT-Level3 listed for hosting a total of 5478 abusers.
Unfortunately, both of your IP's fall into ASN's that are blocked.
If you are using script version 2.0.5, then you should be able to pull the current blacklist, as the DNS and HTTPS servers are on unfiltered ports.
Thanks, Put the version 2.0.5 and OK. is posible that you share ip rules and filters again or update?

Thanks again
 
hhgttg42
just joined
Posts: 8
Joined: Wed Oct 12, 2016 4:48 am

Re: Blacklist Filter update script

Sat Aug 05, 2017 6:13 pm

Yes. If you notice in your log, it is telling you that the script is out of date. The server inserts an alarm into the script when your local script version is out of sync with the server. You can run the code in the first post to keep your script up to date with the latest big fixes.
Thank you for the insight Dave. I'll keep an eye on that.
 
drzen
Frequent Visitor
Frequent Visitor
Posts: 82
Joined: Thu Aug 09, 2007 3:59 pm
Location: Pordenone, Italy
Contact:

Re: Blacklist Filter update script

Wed Aug 09, 2017 5:22 pm

Great script and hard works. Thanks.
A question about security: all go rights but what if your server is violated? For example a malicious code add "/system reset-configuration" or others dangerous commands in front of a downloaded lists?

This is the last doubt before adopting your solution.

thanks in advance.
v.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Aug 09, 2017 5:40 pm

That would truly be bad.
My solution is to isolate the server that generates the scripts from the rest. The generation server is not accessible from the internet. For me to access it, I must connect to my firewall via VPN, then SSH to the database server, then ssh to the blacklist server. The second layer of protection is a 3rd server that parses the lists to check for any commands other than the expected before it is encrypted and passed to the web server for distribution. Last step is a custom nginx module that decrypts the list on the fly to send it out to the firewall.

I've been working on other solutions to push out the list, but have yet to find a good process that is simple and available to all users / firewalls.
 
drzen
Frequent Visitor
Frequent Visitor
Posts: 82
Joined: Thu Aug 09, 2007 3:59 pm
Location: Pordenone, Italy
Contact:

Re: Blacklist Filter update script

Wed Aug 09, 2017 5:45 pm

That would truly be bad.
I've been working on other solutions to push out the list, but have yet to find a good process that is simple and available to all users / firewalls.
THE solution is to output only raw ip addresses' list. But this solution collides with the length of variables in routeros. :(

Thanks for your efforts.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Aug 09, 2017 5:48 pm

THE solution is to output only raw ip addresses' list. But this solution collides with the length of variables in routeros. :(

Thanks for your efforts.
Unfortunately, it's only a solution if it's possible. The amount of 4kb files needed to be downloaded and processes would cause so much wear on the NAND and take up so many filesystem blocks, it would kill most of the routers pulling the list.
 
foxxiu7
just joined
Posts: 5
Joined: Sun Aug 25, 2013 3:30 am

Re: Blacklist Filter update script

Thu Aug 10, 2017 5:58 am

Unfortunately, it's only a solution if it's possible. The amount of 4kb files needed to be downloaded and processes would cause so much wear on the NAND and take up so many filesystem blocks, it would kill most of the routers pulling the list.
Is it an option to change script to download file to USB drive and use external flash drive instead of internal NAND?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Aug 10, 2017 6:21 am

Is it an option to change script to download file to USB drive and use external flash drive instead of internal NAND?
Yes, the path is set in the config file.
 
foxxiu7
just joined
Posts: 5
Joined: Sun Aug 25, 2013 3:30 am

Re: Blacklist Filter update script

Thu Aug 10, 2017 6:42 am

Is it an option to change script to download file to USB drive and use external flash drive instead of internal NAND?
Yes, the path is set in the config file.
Awesome! Have to check the config file then.
 
SPKA16
newbie
Posts: 29
Joined: Fri Aug 05, 2016 8:41 pm

Re: Blacklist Filter update script

Wed Aug 16, 2017 8:51 pm

First of all, thanks for the list! Been using it for a while and started to use it on costumer routers as well! Appreciate all the hard work!

Today at work we noticed a (strong) increase in upload from the router and noticed that Winbox was constantly keeps loading all records in the firewall/adress-list view when its opened.
See attachment/screenshot of my homerouter where the example it on a LAN interface. Whenever you switch tab on the firewall tab or close it the load on the interface goes away. It is constantly around 5Mbps+ when this view is opened.

Is this a known problem, by design, worth a report to Mikrotik? Don't know if this is since the start or just when the list changed to dynamic entries.
You do not have the required permissions to view the files attached to this post.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Aug 16, 2017 9:23 pm

This is expected. Because the address list is dynamic, the timeout is constantly changing. This causes WinBox to reload the entire address list output every second. I'll see if I can find a fix, but I don't think there is much you can do about it, other than not leaving the Address List tab open.
 
Moblaw
just joined
Posts: 7
Joined: Sun Aug 27, 2017 7:56 pm

Re: Blacklist Filter update script

Tue Sep 05, 2017 7:15 pm

I'm trying to get it to work, just can't get the address-list (intrusBL) or (dynamicBlacklist) for that matter, to be filled out.

When I execute the "# Import Intrus Managed Filter Lists
# © 2017 David Joyce, Intrus Technologies" script via /system script run updateBlacklist, which contains the script from the first page.

I get:
invalid value for argument server:
invalid value for argument ipv6-address
failure: bad name
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Tue Sep 05, 2017 7:21 pm

At this time the script is IPv4 only. The servers are able to deal with IPv6, but the client script is not.
 
Moblaw
just joined
Posts: 7
Joined: Sun Aug 27, 2017 7:56 pm

Re: Blacklist Filter update script

Tue Sep 05, 2017 7:30 pm

Now getting /system script run blacklistUpdate
expected command name (line 16 column 1)

I wasn't using IPV6 before, I've disabled the interface now, but cant get the script to properly script as executeable.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Tue Sep 05, 2017 7:32 pm

have you tried remove the scripts and schedules and reinstalling?
There is not much I can do to help, as I have no access to your router.
 
Moblaw
just joined
Posts: 7
Joined: Sun Aug 27, 2017 7:56 pm

Re: Blacklist Filter update script

Tue Sep 05, 2017 9:17 pm

have you tried remove the scripts and schedules and reinstalling?
There is not much I can do to help, as I have no access to your router.
Agree.

I got it working. Fault was this firewall rule "Disallow anything from anywhere on any interface" on input. Disabled it, and the script loaded fine

Its kinda obivious, that, that rule blocks the script from executing. tbh, I hard a hard time point out, in which senario, that rule would be beneficial.

Thanks man. You have my full and sincere gratitude, for your Work done scripting and "posting".

I had to adjust the update-interval, 10min is to often and 3 hours is fine for me.

I'm using my router front end, and pfsense af IDS backend. So what the "intrusBL list" doesn't catch, pfsense pfBlocker & snort will, maybe.

Would it be realistic for me to use front end, without an "ids" or pfsense? Since you mention using it at hospitals etc.
 
mhyll
just joined
Posts: 12
Joined: Wed Feb 01, 2017 3:50 pm

Re: Blacklist Filter update script

Wed Oct 11, 2017 6:57 pm

When can I see the source blacklists? From where you are taking IP addresses? Can I exclude some blacklist sources?

Thanks!

PS: Are you accepting some donations? This stuff is bloody hell good. :)
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Oct 11, 2017 7:07 pm

The source isn't open. It's a collection of 140 routers and servers running as honeypots.
You can exclude addresses by creating a whitelist address-list, and a rule to accept those addresses before the blacklist drop rules.

I don't feel that donations are needed, but thank you for the offer.
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3292
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Blacklist Filter update script

Wed Nov 08, 2017 7:57 pm

@IntrusDave
In post one, can you add information on what this script does, who it is for etc?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Nov 09, 2017 3:30 am

The scripts are for maintaining an address list that is intended to help filter out as much of the bad crap on the internet as possible. This includes spam, viruses, hackers, etc. the script contacts my servers and pulls the latest list of known bad IPs and subnets. The list comes in three sizes. The smallest is meant for home users. It just filters botnets and such. The medium list adds spam hosts and is intended for small to medium businesses. The large filters everything that we can, over 200,000 entries and is only intended for the larger CCR routers protecting servers.
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3292
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Blacklist Filter update script

Thu Nov 09, 2017 9:01 am

I did found this out after reading the thread, but edit post #1 and add this info there.
People like me, that opens a thread to see what is this about, should not need to read long down the thread to see what its about.
For you and other posters here its obvious, but not for my mother....
 
User avatar
hilton
Long time Member
Long time Member
Posts: 634
Joined: Thu Sep 07, 2006 5:12 pm
Location: Jozi (aka Johannesburg), South Africa

Re: Blacklist Filter update script

Thu Nov 09, 2017 9:17 am

The list comes in three sizes. The smallest is meant for home users. It just filters botnets and such. The medium list adds spam hosts and is intended for small to medium businesses. The large filters everything that we can, over 200,000 entries and is only intended for the larger CCR routers protecting servers.
Dave, which list do we get?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Nov 09, 2017 6:40 pm

For you and other posters here its obvious, but not for my mother....
It's not really intended for your mother. :)
Intended for network admins that don't really need help or explanation.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Nov 09, 2017 6:41 pm

Dave, which list do we get?
That's your choice. Select the list that fits your needs, and set it in the config file.
 
amity2kare
newbie
Posts: 35
Joined: Tue Feb 13, 2007 4:24 pm
Location: INDIA

Re: Blacklist Filter update script

Fri Dec 15, 2017 12:58 pm

@IntrusDave

Brilliant script. Worked on my CCR1072 from the word go. Karma from my side.

Regards
 
arkbyte
just joined
Posts: 2
Joined: Thu Dec 21, 2017 6:47 pm

Re: Blacklist Filter update script

Thu Dec 21, 2017 6:50 pm

This blacklist is blocking, among other things, Github. It has been for a while.
It's a great idea, but clearly is not curated or monitored. I would recommend not using it.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Dec 21, 2017 10:56 pm

This blacklist is blocking, among other things, Github. It has been for a while.
It's a great idea, but clearly is not curated or monitored. I would recommend not using it.
Welcome to the board. Not sure why your first post would be to trash someone's work instead of asking a question about it, but okay..

I'd recommend that you read a bit more before posting next time. After reading the thread and notes, you would notice that the list is designed to block *incoming* connections. If it is preventing you from accessing a website, then you have implemented your firewall rules wrong. Note that the list DOES NOT provide rules, only an address list. you supply the rules on how you would like to use it.

That said, I suppose that I need to point out, the list isn't blocking your access to websites, you are blocking access.

If you insist on using the list for outgoing connections, as many do, you will want to also create a whitelist of addresses that you do not want blocked. (as pointed out several times in this topic)

As for GitHub being blocked, if it's being blocked, then it's for good reason. Most of the time its because a server is hosting ads with malicious content, or the site's mail servers are being used for spam.

Anyway. Please fix your rules to block incoming connections, not outgoing. The list is intended to prevent incoming connections from IP's that have been proven to have malicious intent.
 
arkbyte
just joined
Posts: 2
Joined: Thu Dec 21, 2017 6:47 pm

Re: Blacklist Filter update script

Fri Dec 29, 2017 7:14 pm

I'm sorry you feel that comment 'trashed your work'. I did not ask a question because I don't have one. I pointed out that your list contains erroneous entries. Based on your response, I'd say you don't have much tolerance for criticism.

The IP 192.30.255.112 hosts three (and only three) domains: githubcom, www.github.com, and shop.github.com and none of those are attacking your honeypots. It is not listed on a spam bl https://mxtoolbox.com/SuperTool.aspx?ac ... n=toolpage, and is not serving malicious ads (as far as anyone knows).
More than anything, you should take one look at the fact your list includes github and say, "Oh, well that's obviously not right," instead of deflecting with nonsense about one of the top 100 websites https://www.alexa.com/siteinfo/github.com trying to infect you with malware.

Additionally, please don't insult my intelligence. I block incoming AND outgoing communications to known hostile servers for what I believe most enterprise admins would agree is a pretty obvious reason: users in the organization need to be protected from these places. You claim that the list blocks servers which spread malware through ads; well guess what, if you allow outgoing connections to it, your users are still going to pick up that malware, even if the host can't hammer your ssh. So you adding malware hosts is literally pointless in that use case. Jesus Christ, how is that not obvious?

Lastly, yes. We could manually whitelist mistakes from your blocklist. But the list should be managed at the source rather than the endpoints. Especially considering you're using this list with your clients.

Though I'm not using it, I greatly I appreciate your work on this. I think it's fantastic that you made this script and set up these servers, and allow people to access it completely free of charge. I have no problem with that. Only with the way you responded to a legitimate issue.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Fri Dec 29, 2017 7:59 pm

As I’m on vacation, I’ll keep it short. The IP you listed has been serving a malformed PDF with a known Microsoft Edge exploit.
 
User avatar
acortesguasch
just joined
Posts: 7
Joined: Tue Dec 19, 2017 6:04 pm

Re: Blacklist Filter update script

Tue Jan 09, 2018 1:28 pm

Beginning to tinker with Mikrotik and I found this topic.
Just a big thanks for all the work put into it.
 
w0lt
Long time Member
Long time Member
Posts: 537
Joined: Wed Apr 02, 2008 2:12 pm
Location: Minnesota USA

Re: Blacklist Filter update script

Wed Jan 17, 2018 12:52 am

Has anyone had difficulty getting a "Blacklist" update today?

Thanks,

-tp
 
Ryo
just joined
Posts: 5
Joined: Thu Jan 11, 2018 8:00 am

Re: Blacklist Filter update script

Wed Jan 17, 2018 1:36 am

Has anyone had difficulty getting a "Blacklist" update today?

Thanks,

-tp
yup, it show dns server failure
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: Blacklist Filter update script

Wed Jan 17, 2018 2:57 am

Has anyone had difficulty getting a "Blacklist" update today?

Thanks,

-tp
Yes it appears to be failing today.
 
User avatar
Rhoos
just joined
Posts: 22
Joined: Sun Dec 20, 2015 3:48 pm
Location: Costa Rica
Contact:

Re: Blacklist Filter update script

Wed Jan 17, 2018 5:49 am

Has anyone had difficulty getting a "Blacklist" update today?

Thanks,

-tp
Yes, today does not work well!
 
servaris
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Tue May 20, 2014 4:30 pm
Location: Planet Earth
Contact:

Re: Blacklist Filter update script

Wed Jan 17, 2018 5:53 pm

The script seems to not be working.

Ran blacklistUpdate script in terminal.
Log displays:
10:46:56 script,warning Checking server for current blacklist serial number.
10:46:56 script,warning Blacklist is already up to date. Nothing to do.
10:46:56 system,info log rule changed by admin
10:46:57 script,error Download failed. Received bytes.
The firewall list intrusBL is empty.
intrusBL.png
Do you have a fix for this?
Thanks.
You do not have the required permissions to view the files attached to this post.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jan 17, 2018 7:20 pm

Very sorry about that guys. I had to upgrade some server hardware, so I migrated the VM's to a different server. The new server didn't import the DNS vm. The old server if back online now and the VM's returned to their home. All should be good now.
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: Blacklist Filter update script

Wed Jan 17, 2018 8:12 pm

Very sorry about that guys. I had to upgrade some server hardware, so I migrated the VM's to a different server. The new server didn't import the DNS vm. The old server if back online now and the VM's returned to their home. All should be good now.
I had to manually lower the serial number to get the blacklist back, it thought it had the latest blacklist but it was actually empty. (under Scripts in the Environment tab)
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jan 17, 2018 8:23 pm

I had actually started looking into moving the service to a distributed cloud to prevent this, but It looks like I may be shutting down my business and taking over running a nonprofit.
 
servaris
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Tue May 20, 2014 4:30 pm
Location: Planet Earth
Contact:

[SOLVED] Re: Blacklist Filter update script

Wed Jan 17, 2018 9:47 pm

Thanks to mducharme for pointing out the fix to the failed updates.

Go to system -> scripts
Click on Environment

Negatively increment your blSerial by 1. To be clear, the blSerial WAS 1516197642 and now its 1516197641 as shown below.
script-environment.png
You do not have the required permissions to view the files attached to this post.
 
amity2kare
newbie
Posts: 35
Joined: Tue Feb 13, 2007 4:24 pm
Location: INDIA

Re: Blacklist Filter update script

Thu Jan 18, 2018 10:25 pm

Script works fine at my end. However the address list entries (IntrusBL) disappear in a couple of hours. I have been noticing this behavior since i installed this script and have tried upgrading my routeros version as well but to no avail. My current config is routeros 6.39.3 on CCR1072-1G-8S+.
 
Jacka
Member Candidate
Member Candidate
Posts: 125
Joined: Thu Jan 13, 2011 11:34 am

Re: Blacklist Filter update script

Fri Jan 19, 2018 4:23 pm

The script is still operational ? It's not working in my case. The is no code inside the script...

Image
 
User avatar
frank333
Member
Member
Posts: 330
Joined: Mon Dec 18, 2017 12:17 pm
Location: S.Marino Router model: RB3011UiAS-RM+RBM11G

Re: Blacklist Filter update script

Mon Jan 22, 2018 12:02 am

The script works very well! Thanks IntrusDave, you are a Wizard Master! :)
 
Jacka
Member Candidate
Member Candidate
Posts: 125
Joined: Thu Jan 13, 2011 11:34 am

Re: Blacklist Filter update script

Mon Jan 22, 2018 9:59 am

The script works very well! Thanks IntrusDave, you are a Wizard Master! :)
Then, what I'm doing wrong? Can smb help me out. Thank you.
 
User avatar
frank333
Member
Member
Posts: 330
Joined: Mon Dec 18, 2017 12:17 pm
Location: S.Marino Router model: RB3011UiAS-RM+RBM11G

Re: Blacklist Filter update script

Mon Jan 22, 2018 11:33 am

Then, what I'm doing wrong? Can smb help me out. Thank you.
Jacka,
  • eliminates old variables in the environment
1.png
  • eliminates all blacklists scripts and scheduler
  • reboot router
  • rewrites the launch script
3.png
  • starts the blacklistUpdate script --->run script
  • Must work, it controls ip-->firewall-->address list list
PS:What router and what ros version do you have?
You do not have the required permissions to view the files attached to this post.
Last edited by frank333 on Mon Jan 29, 2018 3:30 am, edited 1 time in total.
 
amity2kare
newbie
Posts: 35
Joined: Tue Feb 13, 2007 4:24 pm
Location: INDIA

Re: Blacklist Filter update script

Mon Jan 22, 2018 2:35 pm

Script works fine at my end. However the address list entries (IntrusBL) disappear in a couple of hours. I have been noticing this behavior since i installed this script and have tried upgrading my routeros version as well but to no avail. My current config is routeros 6.39.3 on CCR1072-1G-8S+.
Any help IntrusDave?
 
kakaxa
just joined
Posts: 15
Joined: Thu Feb 01, 2018 5:46 am

Re: Blacklist Filter update script

Thu Feb 01, 2018 12:00 pm

Thanks @IntrusDave, great work!
 
kakaxa
just joined
Posts: 15
Joined: Thu Feb 01, 2018 5:46 am

Re: Blacklist Filter update script

Fri Feb 02, 2018 8:21 pm

Help me please.
The script works fine, but after adding records to the dynamic sheet, after 15-20 minutes, these records are deleted
Help me understand what this is happening? in the log or what is not

Sorry for my English

-------
Помогите пожалуйста.
Скрипт работает отлично, но после добавления записей в динамический лист, спустя 15-20 минут, эти записи удаляются
Помогите понять от чего такое происходит? в логах ни чего нет
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sat Feb 03, 2018 4:51 am

Make sure you blDataPath does not start with a /
i.e. it should read "disk1/blTemp.rsc" NOT "/disk1/blTemp.rsc"
 
User avatar
nichky
Forum Guru
Forum Guru
Posts: 1275
Joined: Tue Jun 23, 2015 2:35 pm

Re: Blacklist Filter update script

Sat Feb 03, 2018 6:14 am

Thanks :)
 
kakaxa
just joined
Posts: 15
Joined: Thu Feb 01, 2018 5:46 am

Re: Blacklist Filter update script

Sat Feb 03, 2018 9:59 am

Make sure you blDataPath does not start with a /
i.e. it should read "disk1/blTemp.rsc" NOT "/disk1/blTemp.rsc"
I'm use the default, path is not change.
On 6.41 work fine, or 6.39.3 the list is cleared after 15min :(

UPD: update mikrotik to 6.41 - script works, list nod cleared
 
dehghanimeysam
just joined
Posts: 1
Joined: Sat Feb 10, 2018 4:41 am

Re: Blacklist Filter update script

Sat Feb 10, 2018 4:56 am

Thank you for providing this
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11442
Joined: Thu Mar 03, 2016 10:23 pm

Re: Blacklist Filter update script

Sun Mar 11, 2018 11:16 pm

A huge thank you from me!

And a question: does anybody know of a similar list, covering IPv6 address filtering?
 
specky
just joined
Posts: 1
Joined: Sun Aug 24, 2014 5:43 pm

Re: Blacklist Filter update script

Mon Mar 19, 2018 11:47 pm

Great script

But its caught a few false positives in the mix from a service we use called Everycloud.
Thanks
 
Grolski
just joined
Posts: 1
Joined: Mon Mar 19, 2018 8:05 pm

Re: Blacklist Filter update script

Thu Mar 22, 2018 1:22 pm

Hello,

I am very impressed with the effectiveness with the IP blacklist by IntrusDave and the scripts he has written. It is the most effective from the ones (the usual suspects) I have used so far.

I have two issue/questions (as none expert) that someone may easily solve.

1. I Drop all Input into my router with some exceptions to allow management. This is effective but also blocks the script to contact the servers and download the latest ip blacklist. What is the best solution to overcome this issue. Remove the Drop all input towards my router or enter an exception rule to allow the script to work. The attempts that I have tried (Layer 7 protocol with URL) were not successful. :(
2. Is there a easy way to extend the Timeout of Dave's list so one could run the script every two days or so? Maybe it is unwise to do so and that could also be an answer.

Regards
Wilko
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sat Mar 24, 2018 5:32 pm

It's been a nice run. Almost 3 years, and over 2200 active users. But I am shutting down the this service. With the new US tax laws and this new US "sex trafficking law" (which isn't really about sex trafficking) I simply can't afford to keep the service running. Bandwidth and rack space is just too expensive now, and I'm making $0.

Thank you all for the support.
 
RyperX
Frequent Visitor
Frequent Visitor
Posts: 72
Joined: Thu May 21, 2015 11:14 am

Re: Blacklist Filter update script

Sat Mar 24, 2018 6:12 pm

Thank you for providing this service for so long, worked really well!

Maybe you could create a guide how it would be possible to create such lists by his own or where you fetch all the information?
 
Taylor
newbie
Posts: 33
Joined: Wed Aug 13, 2014 12:27 am

Re: Blacklist Filter update script

Sat Mar 24, 2018 6:23 pm

It's been a nice run. Almost 3 years, and over 2200 active users. But I am shutting down the this service. With the new US tax laws and this new US "sex trafficking law" (which isn't really about sex trafficking) I simply can't afford to keep the service running. Bandwidth and rack space is just too expensive now, and I'm making $0.

Thank you all for the support.
bw is cheap i can get a server for 8 bucks a month with 12TB of bw. i feel its more than cost as to why you're stopping it.

Need any help?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sat Mar 24, 2018 7:40 pm

bw is cheap i can get a server for 8 bucks a month with 12TB of bw. i feel its more than cost as to why you're stopping it.

Need any help?
If you think $8/month is the cost of a real server, a real firewall, real rack space, real bandwidth, real maintenance, real electricity... Then you are either delusional or have never owned/operated a true network. My servers are not shared VPS servers at some bulk hosting company. My firewalls are not software firewalls. For the last 17 years I have maintained a 48U rack, with 10gbps redundant fiber, a diesel generator with enough fuel for 7 days, a double-conversion UPS with 8 hours of backup time. The rack holds 13 servers, the smallest is an 8 core xeon with 8TB storage and 64GB ram. The largest being dual 12 core xeon, 384GB ram and 64TB. I have 3 CCR1016's and 2 CCR1072's.

So no, $8 doesn't cover it.

I'm also betting that 95% of the 2200 users of the list would not accept pulling a script from a shared server and then running it on their routers. The amount of security and isolation required is far beyond what you can do with a bulk host. Each step of the process to collect the data, build the list, and then push out the list is isolated from the net, not even on the same network that is connected to the internet.

Not to mention the amount of posts, PMs, and emails I get from users demanding that I make changes or run things a different way. It's just not worth it anymore.
 
Taylor
newbie
Posts: 33
Joined: Wed Aug 13, 2014 12:27 am

Re: Blacklist Filter update script

Sat Mar 24, 2018 10:01 pm

I would be fine with keeping the service alive. Having a 48u rack for such a piss easy and small script is a bit outrageous. I think you're the delusional one. I"m sure you used that rack for way more than this script.

I'm simply saying you can keep the script stuff online for way cheaper if you wanna still help the community.
 
User avatar
Rhoos
just joined
Posts: 22
Joined: Sun Dec 20, 2015 3:48 pm
Location: Costa Rica
Contact:

Re: Blacklist Filter update script

Sun Mar 25, 2018 12:31 am

Thank you Dave for such a great service provided all these years; would it be possible for you to put at the disposal of "premium" users of this forum your IP blacklist system so that someone else can continue to provide the service?

Richard
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sun Mar 25, 2018 2:00 am

I would be fine with keeping the service alive. Having a 48u rack for such a piss easy and small script is a bit outrageous. I think you're the delusional one. I"m sure you used that rack for way more than this script.

I'm simply saying you can keep the script stuff online for way cheaper if you wanna still help the community.
Never said my company was dedicated to the blacklist. It's BS like this that helped me decide to shut it down.
 
chippers
newbie
Posts: 26
Joined: Tue Apr 02, 2013 7:45 am

Re: Blacklist Filter update script

Sun Mar 25, 2018 2:29 am

Hey Dave, really sorry to hear you are shutting down but completely understand. Just like to thank you for the script and for me personally, I used it on multiple devices with multiple customers and it must have saved me more than a few times from bad things happening. I used to monitor the firewall rule connected to the script and see the bytes get blocked. It was amazing how high it got at times.

Echoing others here, I would encourage you to throw everything on GitHub or similar, if for nothing else, for us all to learn from your scripting and infra experience, its how the community grows. Once again, thanks for the excellent script and your (free) support of this product. I wish you well in your new role.
 
User avatar
Deantwo
Member
Member
Posts: 331
Joined: Tue Sep 30, 2014 4:07 pm

Re: Blacklist Filter update script

Sun Mar 25, 2018 2:31 am

So this is why have gotten notification emails from this thread all day.
Sorry to hear you are shutting down Dave, but with what you explain it is understandable.

I ended up making my own blacklist script last year, so I never actually used your service much more than as a test. But I at least want to thank you for the help and inspection you gave me. ^^

I'd glady share a few snippets from my scripts if people are interested. Mostly I made a C# program that create a dynamic blacklist file every 4 or so hours from different public blacklists and format them into a single script files for my routers to download.
 
Taylor
newbie
Posts: 33
Joined: Wed Aug 13, 2014 12:27 am

Re: Blacklist Filter update script

Sun Mar 25, 2018 4:01 am

I would be fine with keeping the service alive. Having a 48u rack for such a piss easy and small script is a bit outrageous. I think you're the delusional one. I"m sure you used that rack for way more than this script.

I'm simply saying you can keep the script stuff online for way cheaper if you wanna still help the community.
Never said my company was dedicated to the blacklist. It's BS like this that helped me decide to shut it down.
You did, otherwise why would we care if you had to pay for a 48u rack? Why even mention that? That has nothing to do with this script.

I'd advise you to publish your scripts and collections on github and maybe someone will pick it up and use an $8/m VDS for us ;)

But i'm fairly certain you wont.
 
jgro
newbie
Posts: 49
Joined: Sat Jun 10, 2017 7:33 am

Re: Blacklist Filter update script

Sun Mar 25, 2018 4:47 am

It's been a nice run. Almost 3 years, and over 2200 active users. But I am shutting down the this service.
Thank you, Dave, for a valiant effort.

For everyone who was using Dave's Blacklist, let me recommend the Malicious IP blacklist from SquidBlackList.org, available for download from https://www.squidblacklist.org/download ... licous.rsc . I've been using it for a while and have not run into any problems because of it.

You can download and import it with a simple script:
# File path for squid blacklist. Change to use attached storage if available (e.g. "disk1/bl/drop.malicious.rsc")
:local sblPath "flash/bl/drop.malicious.rsc"
 

:log warning "Downloading squidblacklist malicious BL to $sblPath"

/tool fetch address=www.squidblacklist.org host=www.squidblacklist.org mode=http src-path=/downloads/drop.malicious.rsc dst-path=$sblPath  ;


:log warning "Importing squidblacklist malicious BL from $sblPath"

/import  $sblPath  ;
The downloaded blacklist does not actually block anything, it just creates 3 address lists you can do what you want with (1 for each of the 3 sources they use to compose the final list). I have:
/ip firewall raw
add action=drop chain=prerouting comment="Attack from sbl dshield" log=yes log-prefix="BL dshield" src-address-list="sbl dshield"
add action=drop chain=prerouting comment="Attack from sbl blocklist.de" log=yes log-prefix="BL blocklist.de" src-address-list="sbl blocklist.de"
add action=drop chain=prerouting comment="Attack from sbl spamhaus" src-address-list="sbl spamhaus"
 
Ryo
just joined
Posts: 5
Joined: Thu Jan 11, 2018 8:00 am

Re: Blacklist Filter update script

Sun Mar 25, 2018 7:22 pm

Hi Dave, thanks for ur work.

but i think mod should unpin this thread because this script/service is no longer working.
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Sun Mar 25, 2018 9:14 pm

Thanks Dave for the free ride and it worked very good. Learned a lot with which I can help other...even with my limiting knowledge of scripting.

Gonna miss the flawless working of it!
 
amity2kare
newbie
Posts: 35
Joined: Tue Feb 13, 2007 4:24 pm
Location: INDIA

Re: Blacklist Filter update script

Sun Mar 25, 2018 9:24 pm

Thanks Dave,

It's people like you who keep the community alive with their contributions. Best of luck with your new role.

Regards
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2865
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Blacklist Filter update script

Sun Mar 25, 2018 11:49 pm

Thank you Dave.
Could you be so kind and shere your valuable technology? Could you publish all scripts?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sun Mar 25, 2018 11:57 pm

Thank you Dave.
Could you be so kind and shere your valuable technology? Could you publish all scripts?
Unfortunately no. The server side (contrary to what a few hear think) isn't just a "script" it's a network of over 300 honeypots and some very advanced AI code to analyze threats. That system is proprietary and is still in use for the paying clients that I have left. Even if it wasn't in use, it's not just a simple script that I can post. Nor do I want to give away thousands of hours of code.

there is a chance that I will bring it back, but on a low cost subscription basis. though it's doubtful.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2865
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Blacklist Filter update script

Mon Mar 26, 2018 12:04 am

I understand your decision about server side. What about just client side code?
 
eddieb
Member
Member
Posts: 324
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: Blacklist Filter update script

Mon Mar 26, 2018 11:46 am

Thanks for all the effort you put into this Dave.
I was using your service for over a year and it helped me to keep my network save.
I surely would consider a small subscription fee to keep it

Eddie
 
User avatar
amt
Long time Member
Long time Member
Posts: 529
Joined: Fri Jan 16, 2015 2:05 pm

Re: Blacklist Filter update script

Mon Mar 26, 2018 12:08 pm

Thank you for providing this great service .....
 
User avatar
amt
Long time Member
Long time Member
Posts: 529
Joined: Fri Jan 16, 2015 2:05 pm

Re: Blacklist Filter update script

Mon Mar 26, 2018 12:12 pm

It's been a nice run. Almost 3 years, and over 2200 active users. But I am shutting down the this service.
Thank you, Dave, for a valiant effort.

For everyone who was using Dave's Blacklist, let me recommend the Malicious IP blacklist from SquidBlackList.org, available for download from https://www.squidblacklist.org/download ... licous.rsc . I've been using it for a while and have not run into any problems because of it.

You can download and import it with a simple script:
# File path for squid blacklist. Change to use attached storage if available (e.g. "disk1/bl/drop.malicious.rsc")
:local sblPath "flash/bl/drop.malicious.rsc"
 

:log warning "Downloading squidblacklist malicious BL to $sblPath"

/tool fetch address=www.squidblacklist.org host=www.squidblacklist.org mode=http src-path=/downloads/drop.malicious.rsc dst-path=$sblPath  ;


:log warning "Importing squidblacklist malicious BL from $sblPath"

/import  $sblPath  ;
The downloaded blacklist does not actually block anything, it just creates 3 address lists you can do what you want with (1 for each of the 3 sources they use to compose the final list). I have:
/ip firewall raw
add action=drop chain=prerouting comment="Attack from sbl dshield" log=yes log-prefix="BL dshield" src-address-list="sbl dshield"
add action=drop chain=prerouting comment="Attack from sbl blocklist.de" log=yes log-prefix="BL blocklist.de" src-address-list="sbl blocklist.de"
add action=drop chain=prerouting comment="Attack from sbl spamhaus" src-address-list="sbl spamhaus"
what should i use for storage ? I have 72 core and there is no any attached storage. should i add for this job ? now its using flash. is make any problem for using flash for this job ?

thanks.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2865
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Blacklist Filter update script

Mon Mar 26, 2018 12:25 pm

It is temporary locations to download ... it does not matter where it is ... after importing lists script could be removed form flash, disk etc.
 
User avatar
acortesguasch
just joined
Posts: 7
Joined: Tue Dec 19, 2017 6:04 pm

Re: Blacklist Filter update script

Mon Mar 26, 2018 3:54 pm

DAve, thank you fo your insight and the servce you provided. It will be missed.
I just started in the MikroTik world and your script was one of the first things I studied in order to improve my RouterOS knowledge.

Thank you very much :)
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1120
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Blacklist Filter update script

Mon Mar 26, 2018 5:49 pm

I'm totally gutted that I've now only seen this thread at the end of it's life. I was looking for something exactly like this and was under the impression nothing was about.

@OP You really need to market this as a subscription based list, judging by the amount of people who have said thanks I'm sure you'd make something out of it, whether that offsets your costs only you can determine though.
 
idoch
just joined
Posts: 6
Joined: Mon Mar 26, 2018 6:54 pm

Re: Blacklist Filter update script

Mon Mar 26, 2018 7:06 pm

First off: a Hearty THANK YOU to Dave for putting this together and sharing with the community for all this time. The usefulness of this list is undeniable and it's sad to see it go. Many networks and their users are less safe today because of the end of this list. :(

On another note, Dave - I hope that you DO begin a subscription service for this list. While the list has been demonstrably useful; I have always been reluctant to use this list in a production / business capacity BECAUSE it's completely free and BECAUSE there is no agreement, or "consideration" for using the list. Many of us are happy to pay for the service, I (for one) didn't even know that there was an option to pay for the list!.
 
User avatar
jspool
Member
Member
Posts: 468
Joined: Sun Oct 04, 2009 4:06 am
Location: Oregon

Re: Blacklist Filter update script

Mon Mar 26, 2018 8:50 pm

Thanks Dave! Its always a battle protecting ones network. The people that use free antivirus would likely be the ones that expect such a service free and the people that understand that you get what you pay for would support a paid service. Personally I prefer a BGP based solution. Since there isn't anything worth while in that arena I decided to build my own BGP blocklist system. So far beta testing has been good. On average its blocking 55,000 prefixes. As Dave mentioned it takes a considerable amount of time coding and deduplication across all of the available sources. I am integrating my own honeypot collectors now since I always seem to have a considerable number of attackers that are not present on existing lists.
 
andcz
just joined
Posts: 5
Joined: Tue Mar 20, 2018 10:36 am

Re: Blacklist Filter update script

Tue Mar 27, 2018 9:39 am

How about MikroTik company will pick up this effort, and provide the service to all the MikroTik owners ?

That would be great (and I will be even totaly willing to pay extra, like a per-year subscription or such),
and
most importantly,
this will provide a specific chain of trust - on getting the correct IP black-list from the manufacturer, that could be actually trusted.

The active black-list is a must-have for anyone running any network.

Also, there are many free, respectable services, that do publish blacklists coming from honeypots.
Example: https://project.turris.cz/en/greylist
So there should be not so much issue on getting the inputs for the official service.

I do definitelly vote for this. Anyone else ?
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1120
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Blacklist Filter update script

Tue Mar 27, 2018 10:46 am

How about MikroTik company will pick up this effort, and provide the service to all the MikroTik owners ?

That would be great (and I will be even totaly willing to pay extra, like a per-year subscription or such),
and
most importantly,
this will provide a specific chain of trust - on getting the correct IP black-list from the manufacturer, that could be actually trusted.

The active black-list is a must-have for anyone running any network.

Also, there are many free, respectable services, that do publish blacklists coming from honeypots.
Example: https://project.turris.cz/en/greylist
So there should be not so much issue on getting the inputs for the official service.

I do definitelly vote for this. Anyone else ?
I highly doubt MikroTik are going to take on a project like this. Maybe it could be something we do as a community?
 
andcz
just joined
Posts: 5
Joined: Tue Mar 20, 2018 10:36 am

Re: Blacklist Filter update script

Tue Mar 27, 2018 11:19 am

I'm in a need of such a service, thus my reaction. No offense given, I hope :)

In today's world, this shall be a native service provided with the platform.
Whichever manufacturer of routing platforms is going to provide this as a basic service, is then going to win the market.

Sure, it might be simple money to just build the hw platform with a software stack,
and let the users to build and run add ons;
yet given the actual (and future) world-wide situation, everyone shall
not only be looking for a market differentiator (which such a service could and will provide), but also to ensure the core cyber security features.
Having a official and long-term ip-black-list as a service, would be a great leap forward.
This should come from a trusted source (as the platform manufacturer would be, to a certain degree) and be a part of a standard installation (configurable, of course).

Running this from a community source is always a temporary and non-audited solution, as this thread has shown.
Sure, this original source was a side result (if I read the topic correctly) of a cyber security outfit, using all the side ways on how to remove attacks to their customer's infrastructure,
which required pushing as many RB owners to adapt the distributed list; so it in fact paid for itself (this way around).
And why not. In the end, it helped everybody. But once it stopped to be useful to the OP, the service ceased.
This is why I have this rather pushy approach to ask the manufacturer to provide this service on their own.

Just my 5 cents. Hope no one would get any bad feeling from this, not my intention.
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1120
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Blacklist Filter update script

Tue Mar 27, 2018 11:36 am

I agree with you, I would love to have a service like this available, I can think of multiple places where it would be employed.

It's not just a bunch of hardware though is it? What happens about false positives and somewhere legitimate is not reachable? Who will administer that, the manufacturer is then employing bodies to vet the lists and handle inbound queries on domains and what if something isn't picked up and it causes an issue, as a client you would hold the vendor responsible for X amount of loss of kit/earnings etc. It's a vast ocean to dive into because of a users project no longer being maintained.

Also what about when the worlds internet gears finally turn and all this investment into a platform becomes null overnight with a move to IPv6? Complete change/upgrade.

You could wait for a long time for a manufacturer to do this, which they won't as they're making money from selling their kit, or you can go to a real RBL provider and pay for a list or you hook onto community versions such as this was.
 
i4jordan
Frequent Visitor
Frequent Visitor
Posts: 77
Joined: Mon Sep 02, 2013 1:42 am

Re: Blacklist Filter update script

Wed Mar 28, 2018 11:12 pm

I also would pay for such a service, no problem.
Maybe you can make something with a pay per device/year option?

In any way thank you for the intrusBL service!
 
kakaxa
just joined
Posts: 15
Joined: Thu Feb 01, 2018 5:46 am

Re: Blacklist Filter update script

Thu Mar 29, 2018 6:51 pm

at whom the record set remained(blacklist) share please.
Dave - you many thanks that shared such remarkable set
=================
что ни у кого не сохранился лист?
поделитесь остатками пожалуйста
 
blakhawk23
just joined
Posts: 2
Joined: Thu Aug 10, 2017 11:10 am

Re: Blacklist Filter update script

Sat Apr 14, 2018 10:02 am

Thank you Dave for a great service. It was nice. And yes... i would pay for this extra layer of security....
 
User avatar
frank333
Member
Member
Posts: 330
Joined: Mon Dec 18, 2017 12:17 pm
Location: S.Marino Router model: RB3011UiAS-RM+RBM11G

Re: Blacklist Filter update script

Sat Apr 21, 2018 5:20 pm

dear dave, sorry you want to close the server, I ask you just this, sets a monthly price fair, and post it qua payment via paypal or bitcoin, if the initiative is not successful then you will decide whether to close or not, what do you say?
Your script I need a lot is a shame to lose it!!
frank.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sun Apr 22, 2018 11:33 pm

So, I've thought about this a lot. I was quite surprised by the support generated. So I think I will restart the service as a paid service to pay for the server and backspace. In total, the servers currently cost me about $1750 a month, but I'm downsizing everything and going to try and get it down to a more manageable $250~500/month.

I'm going to rebuild this from the ground up, no I want be doing BGP just because of the amount of administrative overhead.
I would really like to wait on the script side, as I'm hoping to see if I can use the function library that Mikrotik has mentioned may be coming soon.

If their are features you would want to see, let me know and I'll put them on the list of possibles.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sun Apr 22, 2018 11:39 pm

One feature for a paid version I would like to implement is a management portal.
* Manage your subscription options (Number of devices permitted to use your sub / payment method)
* Manage your device serial numbers associated with your subscription
* Manage a whitelist of IPs/Subnets that you need excluded from your list
* Manage a blacklist of IPs/Subnets that you need included in your list
 
User avatar
acortesguasch
just joined
Posts: 7
Joined: Tue Dec 19, 2017 6:04 pm

Re: Blacklist Filter update script

Mon Apr 23, 2018 12:55 pm

Please, when you have it all worked up and ready to accept suscriptions notify it here.
We'll be eager to use it.
 
amity2kare
newbie
Posts: 35
Joined: Tue Feb 13, 2007 4:24 pm
Location: INDIA

Re: Blacklist Filter update script

Mon Apr 23, 2018 9:58 pm

@IntrusDave you can count me in for the paid service. The feature list sounds good. Do let us know on this thread and we'll be most happy to do beta testing for you.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Tue Apr 24, 2018 7:22 pm

I'd like some input on what this is worth to people in $USD.
My thought was $5/month for 5 devices, $10/month for 15 devices.
 
aboiles
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Sat Nov 07, 2015 6:52 pm

Re: Blacklist Filter update script

Tue Apr 24, 2018 10:21 pm

Dave that sounds reasonable to me, could also be yearly subscriptions.
Say $50 yr 5 devices, $150 yr 15 devices.
 
dmercer
just joined
Posts: 3
Joined: Sat Jul 08, 2017 3:53 am

Re: Blacklist Filter update script

Wed Apr 25, 2018 12:38 am

Thanks for all your work. I too would be happy to pay. 50/year for 5 devices would be great. I would sign up for two blocks
 
nwa
newbie
Posts: 29
Joined: Sun Aug 17, 2014 3:02 pm

Re: Blacklist Filter update script

Mon Apr 30, 2018 6:03 pm

privat... 10 for 1 device in a year... i dont need more devices
 
jausovec
just joined
Posts: 17
Joined: Fri Mar 06, 2015 9:55 am

Re: Blacklist Filter update script

Wed May 02, 2018 1:23 pm

10€/device/year. Private use.
 
User avatar
kometchtech
Member Candidate
Member Candidate
Posts: 194
Joined: Sat Jun 15, 2013 4:25 am
Location: Japan
Contact:

Re: Blacklist Filter update script

Sun May 06, 2018 3:53 pm

  • I think that I am pleased if I can pay annually and discounted.
  • I am glad that payment can be made with paypal.
 
foxxiu7
just joined
Posts: 5
Joined: Sun Aug 25, 2013 3:30 am

Re: Blacklist Filter update script

Mon May 07, 2018 2:15 am

single device, $10/year
 
hhgttg42
just joined
Posts: 8
Joined: Wed Oct 12, 2016 4:48 am

Re: Blacklist Filter update script

Sun May 13, 2018 3:53 am

Dave! First off I will GLADLY pay 5$/month for up to 5 devices!

Secondly, thank you for this awesome service you have provided to the community. I had been using it since sometime last year and it helped me sleep better at night knowing a good chunk of the negative aspects of the internet will stay away from my setups.

Please keep us posted and godspeed!
 
User avatar
acortesguasch
just joined
Posts: 7
Joined: Tue Dec 19, 2017 6:04 pm

Re: Blacklist Filter update script

Mon May 14, 2018 1:22 am

Private Use: 10$/year/device
Commercial Use: 5$/month up to 5 devices

Let's hope you can make it work
 
dadaniel
Member Candidate
Member Candidate
Posts: 220
Joined: Fri May 14, 2010 11:51 pm

Re: Blacklist Filter update script

Thu May 17, 2018 5:34 pm

It is temporary locations to download ... it does not matter where it is ... after importing lists script could be removed form flash, disk etc.
But it is imported as static entries because of missing timeout parameter in the script, so they are written to NAND on every change. They should change it in the script
 
zeane
just joined
Posts: 12
Joined: Fri Apr 11, 2008 5:14 am
Location: Cape Town - SOUTH AFRICA

Re: Blacklist Filter update script

Thu Jun 21, 2018 5:55 pm

Please do keep us posted as to your development.
Please also consider those of us in places like South Africa, where US$1 works out at almost 15 times that price in ZAR.

- LESS IS MORE -
If you look at statistical data of the most successful PAID apps in the Google Play Store, the 99c apps do far better in sales than anything priced over that amount. (You do the math >> Google charges mere cents [or a fraction thereof] for many of its ad services << Doesn't that tell you something about charging less for more, and still being financially successful!!)

- REMEMBER -
In todays online environment, you have customers not only from a single country, but now the entire globe as your market place....

- ANOTHERWORDS -
Please make your products more accessible to more customers, and you'll be able to make way more money, which in turn will facilitate your ability to have better resources to maintain and deliver your product.

The weather and politicans are already killing us -
FOR THE MOST PART - here in Cape Town. So, your kind consideration [when determining a fee], is GREATLY APPRECIATED!

Thank you and keep up the excellent work...

ANTHONY.capetown
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter update script

Thu Jun 21, 2018 8:25 pm

Antony, I understand your suggestion and you can't ask for a lower price for parts of the world. It would be possible if Dave could also his daily shoppings in SA.

Maybe Bitcoin or other digital money have beter rates or as you sugest Playstore in a app with a subscription. You still have yo buy that 'currency'.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Blacklist Filter update script

Mon Jun 25, 2018 11:23 am

you can't ask for a lower price for parts of the world
You cannot just because everyone is doing like this? :) Google Play Music, Battlenet Shop - they all have different prices for Russia, for example.
 
jausovec
just joined
Posts: 17
Joined: Fri Mar 06, 2015 9:55 am

Re: Blacklist Filter update script

Thu Jul 05, 2018 1:38 pm

Hi. Do we maybe have some new info? Do we maybe know, if the service will be started again (in exchange for money)? As wrote I am willing to pay 10€/year/device.
Thank you, Uros
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Mon Jul 09, 2018 2:53 am

Not really an update, just information.

I'm still trying to figure out what to do with my life. At 43, I started not liking what I was doing for work. I've tried a few different things, including running a large non-profit for a while. Now I'm closing in on 45 and I still don't know. Fact is, I'm best at tech stuff and car stuff.

Anyway. I've shutdown all of my hardware servers and pulled them from the datacenter. It was just costing too much.

I'm currently experimenting with Google Cloud Compute platform to see if it will be a viable home for a new blacklist service. I hope to have new servers completed this week, and hope to start building a new Blacklist from scratch maybe next week. I still haven't found any decent subscription management systems. Anyone have suggestions on something Open Source?
 
blakhawk23
just joined
Posts: 2
Joined: Thu Aug 10, 2017 11:10 am

Re: Blacklist Filter update script

Mon Jul 09, 2018 4:04 pm

Hi Dave,
I think your black-ist vas a great extra security layer. I think 10$ / device /year sounds great and from me and the company where i work you will have 3 subscriptions.
Again, nice work!
Keep it up.
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1120
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Blacklist Filter update script

Mon Jul 09, 2018 5:07 pm

Not really an update, just information.

I'm still trying to figure out what to do with my life. At 43, I started not liking what I was doing for work. I've tried a few different things, including running a large non-profit for a while. Now I'm closing in on 45 and I still don't know. Fact is, I'm best at tech stuff and car stuff.

Anyway. I've shutdown all of my hardware servers and pulled them from the datacenter. It was just costing too much.

I'm currently experimenting with Google Cloud Compute platform to see if it will be a viable home for a new blacklist service. I hope to have new servers completed this week, and hope to start building a new Blacklist from scratch maybe next week. I still haven't found any decent subscription management systems. Anyone have suggestions on something Open Source?
You just need to do what makes you happy. It's fine not to know what to do with your life as long as you enjoy not knowing what to do.
If you love tech and cars, do something that involves putting tech in cars and innovate in that area. 8 years ago I was made redundant from my then dream job of being a car audio specialist, I quickly fell into my second hobby which was PC gaming and computers. Now I would love to apply my knowledge of tech, small SoC systems and mobile networking into the automotive sector however I am happy(ish) with what I am doing and this thread is about your amazing service!
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2865
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Blacklist Filter update script

Mon Jul 09, 2018 5:58 pm

...
I'm still trying to figure out what to do with my life. At 43, I started not liking what I was doing for work. I've tried a few different things, including running a large non-profit for a while. Now I'm closing in on 45 and I still don't know.
You need "grown-up gap year" :D :D
 
hhgttg42
just joined
Posts: 8
Joined: Wed Oct 12, 2016 4:48 am

Re: Blacklist Filter update script

Mon Jul 09, 2018 6:27 pm

[...]

I'm still trying to figure out what to do with my life. At 43, I started not liking what I was doing for work. I've tried a few different things, including running a large non-profit for a while. Now I'm closing in on 45 and I still don't know. Fact is, I'm best at tech stuff and car stuff.

[...]

I'm currently experimenting with Google Cloud Compute platform to see if it will be a viable home for a new blacklist service. I hope to have new servers completed this week, and hope to start building a new Blacklist from scratch maybe next week. I still haven't found any decent subscription management systems. Anyone have suggestions on something Open Source?
To the first point, you gotta do what you love. Often the things we're good at aren't what we love though...so that's a tricky one.

As for a subscription system, check out 'Servicebot' and 'Kill Bill'. I don't have experience with them, but they're open-source and look like they'll get the job done. There's always PayPal (*shudders*) or setting up something directly e.g. with Stripe as an alternative.

Best of luck!
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Wed Jul 11, 2018 3:15 am

New topic for development of the new service

viewtopic.php?f=9&t=136666
 
marcrisse
just joined
Posts: 24
Joined: Tue Feb 16, 2016 9:16 pm
Location: Germany

Re: Blacklist Filter update script

Thu Aug 02, 2018 9:19 pm

Do you need some servers in europe? Perhaps I can help you?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Aug 02, 2018 9:51 pm

Servers shouldn’t be an issue, I’ve moved my physical servers to google cloud and will be running on google’s CDN
 
jo2jo
Forum Guru
Forum Guru
Posts: 1003
Joined: Fri May 26, 2006 1:25 am

Re: Blacklist Filter update script

Fri Aug 03, 2018 8:19 am

Thanks for running the service all these years and doing such a great job with the script and updates! ive really appreciated it. I too would pay 5 (maybe 10$) a month for this same service to continue.

Is there anywhere we can signup for an email/alert when your paid service goes live? (other than sub'ing to this thread which will have lots of other "noise").

(also you might want to put a date under your red txt about the service going down, in your first post. its a bit hard to figure out if the service is already down, or if you are planning to take it down). thanks
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Fri Aug 03, 2018 8:28 am

Thanks! Check out the development topic. I posted an early pre-alpha script to try and start stress testing the database.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Sat Aug 04, 2018 11:20 pm

Here is a form to fill out if you want to be notified when the new service goes live:

https://goo.gl/forms/UQMYqKJ54E0iV35l2
 
willianmp
just joined
Posts: 1
Joined: Tue Jul 21, 2015 7:39 am

Re: Blacklist Filter update script

Tue Sep 11, 2018 6:31 pm

failure: closing connection: <404 Not Found> 35.236.78.203:443 (4)
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1286
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter update script

Thu Sep 13, 2018 8:09 pm

Who is online

Users browsing this forum: own3r1138 and 24 guests