:global blScriptVersion;
if ($blScriptVersion != "2.0.1") do={
:local sourceServer "https://mikrotikfilters.com/";
:local scriptName "blInstaller.rsc";
.
.
:do { /ip firewall address-list remove [find where list=dynamicBlacklist] } on-error={}
/system script run blacklistUpdate
} else={ :put "script is current" }
So I can't even test with the huge...really huge file how the filter scoring is? ......it is really huge! With the medium list in....not that huge...I have 187MB free RAM.in my testing, the 64M units are struggling with anything other than the small list. I'm seeing about 60% of the 64M units pull the medium list 10+ times in a row. That is telling me that the 64M units are having kernel panics and rebooting.
At this time, the server is now forcing the small list on 32M and 64M, medium for 128M and 256M, and large for 512M and up.
Hi Dave, I support you block anyone that tries to collect your list and misuse it, however 'poisoning' your list in a way is not good practice.I've shut down the old service (pre 2.0 script).
I found that several users were leaching the large list and rebranding it as their own. They were also trying to probe the server side for exploits.
Again, I offer my list as a free service to the MikroTik community. If people continue to abuse it, I will shut it down completely.
(I've also added one of the offending IP's to the blacklist... I'm sure that will get some attention)
Im first on 2.0.2.1 Dave if you don't mind, please reach out to me: ------@planetcoop.com I am in the general forum running a btest server with Tom and I am seeing real benefits to this list on spam and attackers of the btest.Just released 2.0.2 with minor bug fixes. Run the auto-update/install script to update.
Hi Dave,
tnx for the updates, it seems to work fine on my RB2011 and my RB1100.
But, I have a very strange problem on my test VM CHR ...
Until 2.0.1 it worked fine, with the latest it wipes my blacklistUpdate and blacklistScriptUpdater ... they are EMPTY ...
Screen Shot 2017-07-19 at 07.29.28.png
in winbox they show up red because they are empty ...
nothing in the logs
even when I copy the content back into the scripts from my RB1100 and run again, the scripts are empty after the run
Eddie
It's a bit beyond the scope of the blacklist. But I do agree.Hi Dave,
First of all thanks for an amazing job and all effort you're putting into this. It's working just fantastic on my hAP-ac router.
A small idea to consider: how about extending firewall filter rules with autoblock functionality for intruders trying to get to a router or network? A dynamic list with banned IP's trying to do excessive pings, scanning ports, attempting DoS attacks, etc?
I'm using your rules from post #2 and having this autoban functionality will just make them more complete and make the network more secure, I think.
I have an ethical standpoint in this and what laws enable is not always sane. If you attack the attacker than you find yourself both back on the same level.So two things... Some users are simply blocked at my firewall, and now two users have been added to the list itself. I don't see this as "poisoning" as they are the ones that were actively trying to find security holes. (They have been trying SQL injections) Given that they are active attacks, I see them as no different than the botnets and spammers that the list is intended to block.
I find it VERY sad that MikroTik users on this forum would stoop this level. And frankly, if the USA passes this current bill that will allow sys-admins to "hack back" then the next time they pull my list, it will include a command to clear their config. until that time - they will remain blacklisted.
I have been there and you are building a list of your own however the chance to have a secondary hit on that address is small. There are many devices in the net that are trying yo have a response.Hi Dave,
First of all thanks for an amazing job and all effort you're putting into this. It's working just fantastic on my hAP-ac router.
A small idea to consider: how about extending firewall filter rules with autoblock functionality for intruders trying to get to a router or network? A dynamic list with banned IP's trying to do excessive pings, scanning ports, attempting DoS attacks, etc?
I'm using your rules from post #2 and having this autoban functionality will just make them more complete and make the network more secure, I think.
Only use the first one TCP and thee has as avaiable services mail and website."I have the blacklist running which filters 2048 tries and after that I have a filter port 22 and 23 which result in another 2163 hits and then I have the service filter that then filters an other 1280 tries which makes over 5000 tries in a little more than four days."
Greetings msatter, can you share those filter rules for a beginner? Thank you !
set the blDebug in the config to 1Hi all
i'm running in my ccr-1009-8G-1S-1S+
log is show notthing
but
Script List show this messeage https://goo.gl/yYE2do
messeage is "
LOG 【;(eval (eval /putmessage=$t) (eval /log warningmessage=$t))】
urlEncode【;(eval (eval /localname=$temp) (eval /forcounter=$i;do=;(eval (eval /localname=$char;value=(eval (eval /pickbegin=$i;counter=$t))) (eval /ifcondition=(= $char );do=;(eval (eval /setname=$char;value=%20) /)) (eval /ifcondition=(= $char -);do=;(eval (eval /setname=$char;value=%2D) /)) (eval /ifcondition=(= $char /);do=;(eval (eval /setname=$char;value=%2D) /)) (eval /ifcondition=(= $char &);do=;(eval (eval /setname=$char;value=%26) /)) (eval /ifcondition=(= $char =);do=;(eval (eval /setname=$char;value=%3D) /)) (eval /setname=$temp;value=( . $temp $char)) /);from=0;to=(- (eval (eval /lenvalue=$t)) 1)) (eval /returnvalue=$temp) /)】
how can i do?
release notes are in the first post. 2.0.3 is included there.Morning,
tnx for explaining the script rights issue, to bad we are struggling with that, for now it works here.
@Dave
I noticed the script got updated to 2.0.3 in the past 12 hours, it would be nice to see some kind of changelog if possible ?
Keep up the good work !
Eddie
Yes, unfortunately, Google is now allowing spammers to use their servers for a price. You are welcome to create a whitelist of servers that you do not want blocked. Unfortunately Google is using their size to try and force admins to stop using block lists. They make money on spam. For this reason, I do not use or support google.Thank you for the script, but I have to say that, as least in my limited testing, I stumbled upon too many blocked gmail servers.
I couldn't even send an email from my gmail account to my corporate address.
The worst part is that gmail somehow didn't even alert me that the message did not go through. Even after one day.
So I have to pass on this one.
Add this to the config file. Auto-update is not disabled by default, and can be enabled by setting this to "yes"I go away for a week and everything has changed.
@IntrusDave, thank you again for all your work on this blacklist.
Unfortunately for me, the automated scripting is now too intrusive and is itself a serious security risk, so I'm out. If in the future you resume publishing a blacklist of addresses/networks that I can import using my own scripting I will probably use that. Meanwhile, I will just use the service from squidblacklist.org that repackages a few public lists and has not caused me any false positive problems.
:global blScriptUpdate "no";
:do { /ip firewall address-list remove [find where dynamic && list=intrusBL] } on-error={}
That would be a pity but understandable.I found that several users were leaching the large list and rebranding it as their own. They were also trying to probe the server side for exploits.
Again, I offer my list as a free service to the MikroTik community. If people continue to abuse it, I will shut it down completely.
+1So two things... Some users are simply blocked at my firewall, and now two users have been added to the list itself. I don't see this as "poisoning" as they are the ones that were actively trying to find security holes. (They have been trying SQL injections) Given that they are active attacks, I see them as no different than the botnets and spammers that the list is intended to block.
I find it VERY sad that MikroTik users on this forum would stoop this level.
The huge list is taking a lot of memory and 64 is not enough certainly if you do a remove and read in. I am not trying for the first time the huge list and it took 7 minutes to push it into the addresslist.I've updated the server side to prevent units with 64M or less from pulling list 3. It's simply too big and causes the units to panic with an out of memory error. I watched one unit download the list and reboot more than 30 times last night, until I forced it to grab list two on the server side.
{
:local list1 [ :resolve server=mikrotikfilters.com server-port=6502 domain-name=127.0.0.4 ];
:local list2 [ :resolve server=mikrotikfilters.com server-port=6502 domain-name=127.0.0.5 ];
:local list3 [ :resolve server=mikrotikfilters.com server-port=6502 domain-name=127.0.0.6 ];
:put "List 1 Entries: $list1\n\rList 2 Entries: $list2\n\rList 3 Entries: $list3";
:log warning "List 1 Entries: $list1\n\rList 2 Entries: $list2\n\rList 3 Entries: $list3";
}
:do { /ip firewall address-list remove [find where dynamic && list=intrusBL] } on-error={}
:do { /ip firewall address-list remove [find where dynamic && list=intrusWL] } on-error={}
181.225.100.117Either your firewall is blocking DNS to my server, or your IP is blocked by the list already.
What is your public IP?
181.225.100.117Either your firewall is blocking DNS to my server, or your IP is blocked by the list already.
What is your public IP?
190.253.66.37
Unfortunately, both of your IP's fall into ASN's that are blocked.Your ISP /AS262186 is UCEPROTECT-Level3 listed for hosting a total of 462 abusers.
Your ISP COLOMBIA TELECOMUNICACIONES S.A. ESP/AS3816 is UCEPROTECT-Level3 listed for hosting a total of 5478 abusers.
Thanks, Put the version 2.0.5 and OK. is posible that you share ip rules and filters again or update?181.225.100.117Either your firewall is blocking DNS to my server, or your IP is blocked by the list already.
What is your public IP?
190.253.66.37Unfortunately, both of your IP's fall into ASN's that are blocked.Your ISP /AS262186 is UCEPROTECT-Level3 listed for hosting a total of 462 abusers.
Your ISP COLOMBIA TELECOMUNICACIONES S.A. ESP/AS3816 is UCEPROTECT-Level3 listed for hosting a total of 5478 abusers.
If you are using script version 2.0.5, then you should be able to pull the current blacklist, as the DNS and HTTPS servers are on unfiltered ports.
Thank you for the insight Dave. I'll keep an eye on that.Yes. If you notice in your log, it is telling you that the script is out of date. The server inserts an alarm into the script when your local script version is out of sync with the server. You can run the code in the first post to keep your script up to date with the latest big fixes.
THE solution is to output only raw ip addresses' list. But this solution collides with the length of variables in routeros.That would truly be bad.
I've been working on other solutions to push out the list, but have yet to find a good process that is simple and available to all users / firewalls.
Unfortunately, it's only a solution if it's possible. The amount of 4kb files needed to be downloaded and processes would cause so much wear on the NAND and take up so many filesystem blocks, it would kill most of the routers pulling the list.THE solution is to output only raw ip addresses' list. But this solution collides with the length of variables in routeros.
Thanks for your efforts.
Is it an option to change script to download file to USB drive and use external flash drive instead of internal NAND?Unfortunately, it's only a solution if it's possible. The amount of 4kb files needed to be downloaded and processes would cause so much wear on the NAND and take up so many filesystem blocks, it would kill most of the routers pulling the list.
Yes, the path is set in the config file.Is it an option to change script to download file to USB drive and use external flash drive instead of internal NAND?
Awesome! Have to check the config file then.Yes, the path is set in the config file.Is it an option to change script to download file to USB drive and use external flash drive instead of internal NAND?
Agree.have you tried remove the scripts and schedules and reinstalling?
There is not much I can do to help, as I have no access to your router.
Dave, which list do we get?The list comes in three sizes. The smallest is meant for home users. It just filters botnets and such. The medium list adds spam hosts and is intended for small to medium businesses. The large filters everything that we can, over 200,000 entries and is only intended for the larger CCR routers protecting servers.
It's not really intended for your mother.For you and other posters here its obvious, but not for my mother....
That's your choice. Select the list that fits your needs, and set it in the config file.Dave, which list do we get?
Welcome to the board. Not sure why your first post would be to trash someone's work instead of asking a question about it, but okay..This blacklist is blocking, among other things, Github. It has been for a while.
It's a great idea, but clearly is not curated or monitored. I would recommend not using it.
yup, it show dns server failureHas anyone had difficulty getting a "Blacklist" update today?
Thanks,
-tp
Yes it appears to be failing today.Has anyone had difficulty getting a "Blacklist" update today?
Thanks,
-tp
Yes, today does not work well!Has anyone had difficulty getting a "Blacklist" update today?
Thanks,
-tp
The firewall list intrusBL is empty.10:46:56 script,warning Checking server for current blacklist serial number.
10:46:56 script,warning Blacklist is already up to date. Nothing to do.
10:46:56 system,info log rule changed by admin
10:46:57 script,error Download failed. Received bytes.
I had to manually lower the serial number to get the blacklist back, it thought it had the latest blacklist but it was actually empty. (under Scripts in the Environment tab)Very sorry about that guys. I had to upgrade some server hardware, so I migrated the VM's to a different server. The new server didn't import the DNS vm. The old server if back online now and the VM's returned to their home. All should be good now.
Then, what I'm doing wrong? Can smb help me out. Thank you.The script works very well! Thanks IntrusDave, you are a Wizard Master!
Jacka,Then, what I'm doing wrong? Can smb help me out. Thank you.
Any help IntrusDave?Script works fine at my end. However the address list entries (IntrusBL) disappear in a couple of hours. I have been noticing this behavior since i installed this script and have tried upgrading my routeros version as well but to no avail. My current config is routeros 6.39.3 on CCR1072-1G-8S+.
I'm use the default, path is not change.Make sure you blDataPath does not start with a /
i.e. it should read "disk1/blTemp.rsc" NOT "/disk1/blTemp.rsc"
bw is cheap i can get a server for 8 bucks a month with 12TB of bw. i feel its more than cost as to why you're stopping it.It's been a nice run. Almost 3 years, and over 2200 active users. But I am shutting down the this service. With the new US tax laws and this new US "sex trafficking law" (which isn't really about sex trafficking) I simply can't afford to keep the service running. Bandwidth and rack space is just too expensive now, and I'm making $0.
Thank you all for the support.
If you think $8/month is the cost of a real server, a real firewall, real rack space, real bandwidth, real maintenance, real electricity... Then you are either delusional or have never owned/operated a true network. My servers are not shared VPS servers at some bulk hosting company. My firewalls are not software firewalls. For the last 17 years I have maintained a 48U rack, with 10gbps redundant fiber, a diesel generator with enough fuel for 7 days, a double-conversion UPS with 8 hours of backup time. The rack holds 13 servers, the smallest is an 8 core xeon with 8TB storage and 64GB ram. The largest being dual 12 core xeon, 384GB ram and 64TB. I have 3 CCR1016's and 2 CCR1072's.bw is cheap i can get a server for 8 bucks a month with 12TB of bw. i feel its more than cost as to why you're stopping it.
Need any help?
Never said my company was dedicated to the blacklist. It's BS like this that helped me decide to shut it down.I would be fine with keeping the service alive. Having a 48u rack for such a piss easy and small script is a bit outrageous. I think you're the delusional one. I"m sure you used that rack for way more than this script.
I'm simply saying you can keep the script stuff online for way cheaper if you wanna still help the community.
You did, otherwise why would we care if you had to pay for a 48u rack? Why even mention that? That has nothing to do with this script.Never said my company was dedicated to the blacklist. It's BS like this that helped me decide to shut it down.I would be fine with keeping the service alive. Having a 48u rack for such a piss easy and small script is a bit outrageous. I think you're the delusional one. I"m sure you used that rack for way more than this script.
I'm simply saying you can keep the script stuff online for way cheaper if you wanna still help the community.
Thank you, Dave, for a valiant effort.It's been a nice run. Almost 3 years, and over 2200 active users. But I am shutting down the this service.
# File path for squid blacklist. Change to use attached storage if available (e.g. "disk1/bl/drop.malicious.rsc")
:local sblPath "flash/bl/drop.malicious.rsc"
:log warning "Downloading squidblacklist malicious BL to $sblPath"
/tool fetch address=www.squidblacklist.org host=www.squidblacklist.org mode=http src-path=/downloads/drop.malicious.rsc dst-path=$sblPath ;
:log warning "Importing squidblacklist malicious BL from $sblPath"
/import $sblPath ;
/ip firewall raw
add action=drop chain=prerouting comment="Attack from sbl dshield" log=yes log-prefix="BL dshield" src-address-list="sbl dshield"
add action=drop chain=prerouting comment="Attack from sbl blocklist.de" log=yes log-prefix="BL blocklist.de" src-address-list="sbl blocklist.de"
add action=drop chain=prerouting comment="Attack from sbl spamhaus" src-address-list="sbl spamhaus"
Unfortunately no. The server side (contrary to what a few hear think) isn't just a "script" it's a network of over 300 honeypots and some very advanced AI code to analyze threats. That system is proprietary and is still in use for the paying clients that I have left. Even if it wasn't in use, it's not just a simple script that I can post. Nor do I want to give away thousands of hours of code.Thank you Dave.
Could you be so kind and shere your valuable technology? Could you publish all scripts?
what should i use for storage ? I have 72 core and there is no any attached storage. should i add for this job ? now its using flash. is make any problem for using flash for this job ?Thank you, Dave, for a valiant effort.It's been a nice run. Almost 3 years, and over 2200 active users. But I am shutting down the this service.
For everyone who was using Dave's Blacklist, let me recommend the Malicious IP blacklist from SquidBlackList.org, available for download from https://www.squidblacklist.org/download ... licous.rsc . I've been using it for a while and have not run into any problems because of it.
You can download and import it with a simple script:The downloaded blacklist does not actually block anything, it just creates 3 address lists you can do what you want with (1 for each of the 3 sources they use to compose the final list). I have:Code: Select all# File path for squid blacklist. Change to use attached storage if available (e.g. "disk1/bl/drop.malicious.rsc") :local sblPath "flash/bl/drop.malicious.rsc" :log warning "Downloading squidblacklist malicious BL to $sblPath" /tool fetch address=www.squidblacklist.org host=www.squidblacklist.org mode=http src-path=/downloads/drop.malicious.rsc dst-path=$sblPath ; :log warning "Importing squidblacklist malicious BL from $sblPath" /import $sblPath ;
Code: Select all/ip firewall raw add action=drop chain=prerouting comment="Attack from sbl dshield" log=yes log-prefix="BL dshield" src-address-list="sbl dshield" add action=drop chain=prerouting comment="Attack from sbl blocklist.de" log=yes log-prefix="BL blocklist.de" src-address-list="sbl blocklist.de" add action=drop chain=prerouting comment="Attack from sbl spamhaus" src-address-list="sbl spamhaus"
I highly doubt MikroTik are going to take on a project like this. Maybe it could be something we do as a community?How about MikroTik company will pick up this effort, and provide the service to all the MikroTik owners ?
That would be great (and I will be even totaly willing to pay extra, like a per-year subscription or such),
and
most importantly,
this will provide a specific chain of trust - on getting the correct IP black-list from the manufacturer, that could be actually trusted.
The active black-list is a must-have for anyone running any network.
Also, there are many free, respectable services, that do publish blacklists coming from honeypots.
Example: https://project.turris.cz/en/greylist
So there should be not so much issue on getting the inputs for the official service.
I do definitelly vote for this. Anyone else ?
But it is imported as static entries because of missing timeout parameter in the script, so they are written to NAND on every change. They should change it in the scriptIt is temporary locations to download ... it does not matter where it is ... after importing lists script could be removed form flash, disk etc.
You cannot just because everyone is doing like this? Google Play Music, Battlenet Shop - they all have different prices for Russia, for example.you can't ask for a lower price for parts of the world
You just need to do what makes you happy. It's fine not to know what to do with your life as long as you enjoy not knowing what to do.Not really an update, just information.
I'm still trying to figure out what to do with my life. At 43, I started not liking what I was doing for work. I've tried a few different things, including running a large non-profit for a while. Now I'm closing in on 45 and I still don't know. Fact is, I'm best at tech stuff and car stuff.
Anyway. I've shutdown all of my hardware servers and pulled them from the datacenter. It was just costing too much.
I'm currently experimenting with Google Cloud Compute platform to see if it will be a viable home for a new blacklist service. I hope to have new servers completed this week, and hope to start building a new Blacklist from scratch maybe next week. I still haven't found any decent subscription management systems. Anyone have suggestions on something Open Source?
You need "grown-up gap year"...
I'm still trying to figure out what to do with my life. At 43, I started not liking what I was doing for work. I've tried a few different things, including running a large non-profit for a while. Now I'm closing in on 45 and I still don't know.
To the first point, you gotta do what you love. Often the things we're good at aren't what we love though...so that's a tricky one.[...]
I'm still trying to figure out what to do with my life. At 43, I started not liking what I was doing for work. I've tried a few different things, including running a large non-profit for a while. Now I'm closing in on 45 and I still don't know. Fact is, I'm best at tech stuff and car stuff.
[...]
I'm currently experimenting with Google Cloud Compute platform to see if it will be a viable home for a new blacklist service. I hope to have new servers completed this week, and hope to start building a new Blacklist from scratch maybe next week. I still haven't found any decent subscription management systems. Anyone have suggestions on something Open Source?