Community discussions

 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Sat Sep 01, 2018 1:53 pm

Unfortunately, I don't have IPv6 yet. The system is designed for it, but I have no routers in IPv6 networks that I can test with. My home internet supports it, but it's so unstable, I don't bother with it.
If it helps and the IPv4 sevice is done, I can provide an IPv6 router as honeypot.
I get a ::/48 prefix length and could then put a router¹ behind the Mikrotik. @Dave: You can have full admin access on it.
I get a new dynamic prefix from my provider every 36-48 hours. I can get a static IP but I have to pay extra for it. IPv6 has been stable for years, and I've had it since the pilotphase. (Year 2013 / Provider: NetCologne.de)

¹On a UBNT (ER-8) router, a honeypot package can be loaded from the Debian reposity.
Of course, the Mikrotik can serve as honeypot directly, if someone has finished scripts for it.
╰_╯ Ciao Marco!
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1092
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Blacklist Filter (Development Topic)

Mon Sep 10, 2018 2:50 pm

Have just noticed 6.43 has moved into the current branch so have updated accordingly. Can't seem to find IP>Cloud though?? Looking forward to using the IntrusBL again.

**It's not in Winbox but is there in the terminal.
Steve "Steveocee" Carter
PC Gamer, Airsofter, MikroTik Nerd
My Website - My MikroTik Tutorials
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Mon Sep 10, 2018 8:16 pm

ip Cloud terminal-only when running CHR
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Mon Sep 10, 2018 8:21 pm

I'm in a holding pattern while my lawyer researches the EU "GDPR" laws.
It's looking like I will not be able to use 3rd party honeypots, as the GDPR requires companies to allow users to delete any data collected from there.
That means that anyone with a honeypot running on their router will be able to delete any IP's in the database that their router may have submitted.
While this may not sound like a big deal, it's ENTIRELY possible for a user to fake an update and delete the whole database.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
User avatar
43north
Member Candidate
Member Candidate
Posts: 197
Joined: Fri Nov 14, 2014 7:06 am

Re: Blacklist Filter (Development Topic)

Mon Sep 10, 2018 9:25 pm

I'm in a holding pattern while my lawyer researches the EU "GDPR" laws.
It's looking like I will not be able to use 3rd party honeypots, as the GDPR requires companies to allow users to delete any data collected from there.
That means that anyone with a honeypot running on their router will be able to delete any IP's in the database that their router may have submitted.
While this may not sound like a big deal, it's ENTIRELY possible for a user to fake an update and delete the whole database.
Thanks for the update Dave.
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Tue Sep 11, 2018 11:23 am

I also thank you for the update.
 
idoch
just joined
Posts: 3
Joined: Mon Mar 26, 2018 6:54 pm

Re: Blacklist Filter (Development Topic)

Fri Sep 21, 2018 5:24 pm

I'm in a holding pattern while my lawyer researches the EU "GDPR" laws.
It's looking like I will not be able to use 3rd party honeypots, as the GDPR requires companies to allow users to delete any data collected from there.
That means that anyone with a honeypot running on their router will be able to delete any IP's in the database that their router may have submitted.
While this may not sound like a big deal, it's ENTIRELY possible for a user to fake an update and delete the whole database.
Hey Dave,
I am looking forward to the new service. Obligatory - I am not a lawyer - but... You're over thinking this. An IP address itself is not "personally identifiable" -- until it is associated with other specific data that could be personally identifiable to a natural person (cookies, MAC, RFID, etc.). Classification of an IP as the source of infection, malicious behavior, etc. (or any of the behavior a honeypot would flag) is not at all identifiable to a natural person. Check into Recital 26 in full https://gdpr-info.eu/recitals/no-26/ Here's a piece:

"The principles of data protection should therefore not apply to anonymous information, namely information which does not relate to an identified or identifiable natural person or to personal data rendered anonymous in such a manner that the data subject is not or no longer identifiable. This Regulation does not therefore concern the processing of such anonymous information, including for statistical or research purposes."

If you get Honeypot information secondhand under general classifications of "port scanner" or "wordpress prober" or "SSH Brute Force" -- I can't think of much that could be less personally identifiable to a natural person. You don't know any personally identifiable information about their interactions with the honeypot - all you have is an IP address and a general label. They can't force you to pretend that IP doesn't exist or that you couldn't find their ISP and report them to abuse@ or that you can't make decisions based on anonymized data based on "reputation" of a network -- which (by itself) is not identifiable to a natural person.

Absolute worst case: If your lists were created with an automatic timeout or expiration that was less than the 24 hour requested removal; wouldn't that be compliant? The IP would have been removed from the list after 24 hours (requested or not) -- if they get added back for ongoing bad behavior that's their problem. The list doesn't "track" them at all; each list is an independent serial number.
 
idoch
just joined
Posts: 3
Joined: Mon Mar 26, 2018 6:54 pm

Re: Blacklist Filter (Development Topic)

Fri Sep 21, 2018 10:30 pm

Showing you further that you (and your subscribers) are in the clear...
https://gdpr-info.eu/recitals/no-49/
 
szir
just joined
Posts: 1
Joined: Tue Oct 16, 2018 6:14 pm

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 12:14 pm

I'm new to MikroTik and I just found this thread.
I like the work that you do.
Security is important to me, so I would like to use your list. I put together a couple of rules for brute force prevention, (also reported some on abuseipdb) but blocking IPs with malicious activity that others found would be nice.

I read someone suggested DNS for updating the block list (instead of downloading a script). I would also like that. One problem I see is that as far as I know you cannot put an IP range into a DNS A record, which would make blocking whole subnets harder.
msatter suggested DNS as a means to ease the traffic generated by distributing the list.
I would like it for a different reason.

Security is important for me so I don't like the idea of downloading a script form an external source and running it on a schedule. I'm sure you are a nice and trustworthy guy, but I don't know you and don't know what security you put in place that prevents (and will prevent at any point in the future) others from hijacking your update script file. I don't want my router to become part of a botnet because it "accidentally" downloaded the wrong script...

Using DNS to update the list would eliminate the need to download and run unknown scripts, the worst that I can imagine happen in case someone takes over the DNS is that they could block innocent IPs/censor the Internet. (There might be some other way that doesn't require auto-running a downloaded script.)
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 5:02 pm

If using DNS is a requirement for you, I suggest you look elsewhere for the service.
I have no plans to use DNS for this service. It's not a viable distribution method.
DNS is not able to send a response of 200,000+ IP addresses.
BGP is also not going to happen, as it require a large amount of labor on both ends to configure.

I've gotten the server side stable enough to move forward. Though I may be changing the pricing.
My current thought is keeping the small list free for all..
Medium list will be accessible via donations.
Full list and custom configuration will be accessible via monthly subscription.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1092
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 6:42 pm

I've watched list "2" slowly grow over time, I think it was "only" around 14,000 entries when you first started this thread off and now it is up to 23,500+ entries. Seriously amazing stuff Dave.
Steve "Steveocee" Carter
PC Gamer, Airsofter, MikroTik Nerd
My Website - My MikroTik Tutorials
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 9:00 pm

Dave since when will be donated for a medium list?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 9:40 pm

I’m not sure I understand the question
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 9:57 pm

I ask since when we pay for the product.
I'm sorry but I'm not good at English.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 10:52 pm

The pay service will begin on the 1st of the year
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Fri Oct 19, 2018 11:22 pm

OK, I'm waiting for information on how to transfer money.
 
jausovec
just joined
Posts: 3
Joined: Fri Mar 06, 2015 9:55 am

Re: Blacklist Filter (Development Topic)

Tue Oct 30, 2018 12:24 pm

Hi.

Can someone sum up the latest script/instructions on how to install the new service (and uninstall the old one :) )?
Or are we not so far yet, that we could start using it in home environment?
I am also avaiting info about how/where to we can make the payment for the service.

Thank you
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Fri Nov 02, 2018 2:02 pm

It's looking like I will not be able to use 3rd party honeypots, as the GDPR requires companies to allow users to delete any data collected from there.
This fucking GDPR :evil:
This law harms citizens more than it helps. A shot in the knee.
Some good forums has closed and from Germany you can not order anything from some shops in Switzerland anymore.

@Dave
Do you have IP blacklists from squidblacklist.org in priority 1 or 2?

@all
Because here is increasingly asked for payment:
Just fill out the form by Dave, if you haven't yet. Then you will receive an e-mail in time.
https://goo.gl/forms/UQMYqKJ54E0iV35l2

@jausovec
Disable or delete the old Blacklist script(s) and scheduler and fetch the new.
Post Nr.9 in this topic:
viewtopic.php?f=9&t=136666#p677573
Adjust 'destPath' and 'priority'.
Fix new schedulers or adjust the old ones.
╰_╯ Ciao Marco!
 
HZsolt
just joined
Posts: 24
Joined: Tue Apr 24, 2018 7:31 pm

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 12:35 pm

For IntusDave:
Do you have any problem or do you update? I run your script but the script didn't download nothing.

I thank you for your help!
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1092
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 1:27 pm

For IntusDave:
Do you have any problem or do you update? I run your script but the script didn't download nothing.

I thank you for your help!
Are you running IP > Cloud ? Would be the "easiest" thing to check at this point as it is a prerequisite.
Steve "Steveocee" Carter
PC Gamer, Airsofter, MikroTik Nerd
My Website - My MikroTik Tutorials
 
HZsolt
just joined
Posts: 24
Joined: Tue Apr 24, 2018 7:31 pm

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 1:43 pm

For IntusDave:
Do you have any problem or do you update? I run your script but the script didn't download nothing.

I thank you for your help!
Are you running IP > Cloud ? Would be the "easiest" thing to check at this point as it is a prerequisite.
Yes, IP -> Cloud is running. DDNS Enabled and updated.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 5:47 pm

The server is hosted on the google cloud platform. It appears that Google has oversold the zone that my servers are in, and my servers have been taking offline to allow others to run. I'll be moving the server to a different zone ASAP.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 7:07 pm

@Dave
Do you have IP blacklists from squidblacklist.org in priority 1 or 2?
squidblacklist.org is not included, as it is a pay service. The IP's they have on the free lists are pretty much duplicates of my list 2.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 8:10 pm

I bought a new network "level 3" honeypots online. These are my first located in "hostile" countries. Bringing the current Level 3 list to over 189,000 entries.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
HZsolt
just joined
Posts: 24
Joined: Tue Apr 24, 2018 7:31 pm

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 8:15 pm

The server is hosted on the google cloud platform. It appears that Google has oversold the zone that my servers are in, and my servers have been taking offline to allow others to run. I'll be moving the server to a different zone ASAP.
Thanks!

Your blacklist works well again!

Thanks!
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 8:17 pm

No problem, and I'm sorry about the downtime.
I've changed the policy on my servers so that Google can not longer preempt mine to make room for higher paying customers. Not happy that it's adding another $50/month onto my bill, but I can't have them dropping me because someone bigger wants my cpu or memory.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 8:56 pm

What are everyone's thoughts on using Patreon for the subscription service?
I've started setting up a page here: https://www.patreon.com/IntrusTechnologies
Once I have everything linked, the existing scripts will stop working and I will post the current scripts on the Patreon page.
The new script will not require any modification, as the server will select the list based on your router's serial number and IP address.
The system will also disable accounts that are using forged serial numbers or IP addresses.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
HZsolt
just joined
Posts: 24
Joined: Tue Apr 24, 2018 7:31 pm

Re: Blacklist Filter (Development Topic)

Sat Nov 03, 2018 9:30 pm

What are everyone's thoughts on using Patreon for the subscription service?
I've started setting up a page here: https://www.patreon.com/IntrusTechnologies
Once I have everything linked, the existing scripts will stop working and I will post the current scripts on the Patreon page.
The new script will not require any modification, as the server will select the list based on your router's serial number and IP address.
The system will also disable accounts that are using forged serial numbers or IP addresses.
How much memory use on routers with these Blacklist Filters?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 9:20 am

It's hard to give accurate numbers, but it looks like the List 1 uses about 768k, List 2 uses 3M, and List 3 uses 54M to load and 50M once the load is done. The other two load too fast on my RB110AHx4 to see the memory load update.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
HZsolt
just joined
Posts: 24
Joined: Tue Apr 24, 2018 7:31 pm

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 10:55 am

It's hard to give accurate numbers, but it looks like the List 1 uses about 768k, List 2 uses 3M, and List 3 uses 54M to load and 50M once the load is done. The other two load too fast on my RB110AHx4 to see the memory load update.
What do you think about this service?: viewtopic.php?t=137632
 
User avatar
acortesguasch
just joined
Posts: 7
Joined: Tue Dec 19, 2017 6:04 pm

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 12:54 pm

What are everyone's thoughts on using Patreon for the subscription service?
I've started setting up a page here: https://www.patreon.com/IntrusTechnologies
Once I have everything linked, the existing scripts will stop working and I will post the current scripts on the Patreon page.
The new script will not require any modification, as the server will select the list based on your router's serial number and IP address.
The system will also disable accounts that are using forged serial numbers or IP addresses.
Any estimate regarding the end-of-service date as it is? I have to convince my boss of the benefits of being a paying member of the community and I do not want to be caught offguard.

Keep the fantastic work!
To Be Continued...
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 5:53 pm

It's hard to give accurate numbers, but it looks like the List 1 uses about 768k, List 2 uses 3M, and List 3 uses 54M to load and 50M once the load is done. The other two load too fast on my RB110AHx4 to see the memory load update.
What do you think about this service?: viewtopic.php?t=137632
I think it's a ripoff of my project by someone that hasn't been a part of the community as long as I have. I think I am far more transparent in the development process.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 6:05 pm


squidblacklist.org is not included, as it is a pay service. The IP's they have on the free lists are pretty much duplicates of my list 2.
OK, now I'll be clear here ;-) Thanks.
Will test how much RAM a RB2011 needed. Only with priority 2
or priority 1 + drop.malicious.rsc

PS:
SBL Malicious IP Blacklist from: https://www.squidblacklist.org is free of charge.
╰_╯ Ciao Marco!
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 6:05 pm

What are everyone's thoughts on using Patreon for the subscription service?
I've started setting up a page here: https://www.patreon.com/IntrusTechnologies
Once I have everything linked, the existing scripts will stop working and I will post the current scripts on the Patreon page.
The new script will not require any modification, as the server will select the list based on your router's serial number and IP address.
The system will also disable accounts that are using forged serial numbers or IP addresses.
Any estimate regarding the end-of-service date as it is? I have to convince my boss of the benefits of being a paying member of the community and I do not want to be caught offguard.

Keep the fantastic work!
The goal is NO end-of-service date. This started out as a project out of a personal need, I shared it with the community because I thought others could use it too. Last year, the original service hit just over 17,000 active devices. I realized that several LARGE businesses were using the service and SELLING IT to their customers. At that point I felt that it was "fair" for me to be paying several hundred per month for servers and honeypots out of pocket, while others were making money off it it.

My vision is to have a service that is simple to implement, stable, secure, fast, and self-sustaining. Once the income is able to cover the expenses, and it is able to keep running without me, then I can focus on bringing new features and adding support for new platforms.

I don't want or plan to get rich from this. I want to provide a valuable service at a price that anyone can afford.

As for yearly payments - At this time, I am going to keep it monthly. I don't want to accept a payment for a year of service before I know the service will be able to sustain itself.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 6:07 pm


squidblacklist.org is not included, as it is a pay service. The IP's they have on the free lists are pretty much duplicates of my list 2.
OK, now I'll be clear here ;-) Thanks.
Will test how much RAM a RB2011 needed. Only with priority 2
or priority 1 + drop.malicious.rsc

PS:
SBL Malicious IP Blacklist from: https://www.squidblacklist.org is free of charge.
I compared mine with that one - All of the IP's in that are also in mine. The key difference is the delivery method. My .rsc is much smaller and processed faster.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 6:31 pm

The new script will not require any modification, as the server will select the list based on your router's serial number and IP address.

As for yearly payments - At this time, I am going to keep it monthly. I don't want to accept a payment for a year of service before I know the service will be able to sustain itself.
Would be nice to be able to select priority 1, although you have paid for priority 2.

Uh, monthly. I hope the monthly payment can be automated.
╰_╯ Ciao Marco!
 
User avatar
acortesguasch
just joined
Posts: 7
Joined: Tue Dec 19, 2017 6:04 pm

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 6:38 pm

Any estimate regarding the end-of-service date as it is? I have to convince my boss of the benefits of being a paying member of the community and I do not want to be caught offguard.

Keep the fantastic work!
The goal is NO end-of-service date. This started out as a project out of a personal need, I shared it with the community because I thought others could use it too. Last year, the original service hit just over 17,000 active devices. I realized that several LARGE businesses were using the service and SELLING IT to their customers. At that point I felt that it was "fair" for me to be paying several hundred per month for servers and honeypots out of pocket, while others were making money off it it.
When I asked about the end-of-service as it is I was referring when the current scripts will be deactivated and only working via Patreon.

I think I understood since the beginning the kind of project you are running, for you and for the Community, and I cannot see any flaw in you reasoning. You are giving a lot to the Community and it is only fair to try to cover expenses.
To Be Continued...
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 7:49 pm

My goal is January first.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
HZsolt
just joined
Posts: 24
Joined: Tue Apr 24, 2018 7:31 pm

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 8:18 pm

My goal is January first.
After January first the current your script will not work?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 9:07 pm

The new script will not require any modification, as the server will select the list based on your router's serial number and IP address.

As for yearly payments - At this time, I am going to keep it monthly. I don't want to accept a payment for a year of service before I know the service will be able to sustain itself.
Would be nice to be able to select priority 1, although you have paid for priority 2.

Uh, monthly. I hope the monthly payment can be automated.
Payment is automated via the Patreon page. Each Tier includes the Tier below it. I've updated the tires on the page to better explain what you get.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 10:00 pm

How to make a payment from Poland?
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 10:01 pm

How to make a payment from Poland?
https://www.patreon.com/IntrusTechnologies

You can go to this Patreon page to sign up.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Sun Nov 04, 2018 10:02 pm

OK thanks.
 
hhgttg42
just joined
Posts: 8
Joined: Wed Oct 12, 2016 4:48 am

Re: Blacklist Filter (Development Topic)

Mon Nov 05, 2018 5:05 pm

What are everyone's thoughts on using Patreon for the subscription service?
I've started setting up a page here: https://www.patreon.com/IntrusTechnologies
Once I have everything linked, the existing scripts will stop working and I will post the current scripts on the Patreon page.
The new script will not require any modification, as the server will select the list based on your router's serial number and IP address.
The system will also disable accounts that are using forged serial numbers or IP addresses.
I'm already signed up! Thanks again for the great service Dave.
 
tippenring
Member Candidate
Member Candidate
Posts: 179
Joined: Thu Oct 02, 2014 8:54 pm
Location: St Louis MO
Contact:

Re: Blacklist Filter (Development Topic)

Mon Nov 05, 2018 5:53 pm


OK, now I'll be clear here ;-) Thanks.
Will test how much RAM a RB2011 needed. Only with priority 2
or priority 1 + drop.malicious.rsc
I'm using the priority 2 list on an RB2011. Memory is fine. I currently have free 74MB of 128MB with ~30k blacklist entries. The RB2011 is more CPU starved when it's updating the list or if you view the address list.
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Mon Nov 05, 2018 7:21 pm

Payment is automated via the Patreon page. Each Tier includes the Tier below it. I've updated the tires on the page to better explain what you get.
Perfect thank you.

Mikrotik forum logged me off last night again and again. :(

I'm using the priority 2 list on an RB2011. Memory is fine. I currently have free 74MB of 128MB with ~30k blacklist entries. The RB2011 is more CPU starved when it's updating the list or if you view the address list.
I've deleted most of the drop.malicious.rsc address-list entrys (~30k blocklist.de) and switched to priority 2 (also about ~30k). I have 7 MB more Ram free. (Free Memory now 55MB) :D
╰_╯ Ciao Marco!
 
anav
Forum Guru
Forum Guru
Posts: 2936
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada

Re: Blacklist Filter (Development Topic)

Mon Nov 05, 2018 9:45 pm

It's hard to give accurate numbers, but it looks like the List 1 uses about 768k, List 2 uses 3M, and List 3 uses 54M to load and 50M once the load is done. The other two load too fast on my RB110AHx4 to see the memory load update.
What do you think about this service?: viewtopic.php?t=137632
I think it's a ripoff of my project by someone that hasn't been a part of the community as long as I have. I think I am far more transparent in the development process.
Sounds like an emotional response based on little fact. You would be better served by recognizing and supporting a like minded fellow provider who decided to share his work for his customers/clients with the community at large. The development cycle taken matches up with what I would have done, first being exposed to Josh Haven and the various sources, such as the Firehol lists, magically available whether you were on this less and less green earth or not. On his own he matured his script skills to be able to create the database and program with the flexibility and 'scalability' of his clients in mind (hex to larger units) which is critical to many of us with lesser units (not companies with fat wallets). Its stable, it works and its phukking affordable. He obviously has put much time and effort into the program, considering the servers required and the fail over and many other detailed minutia it takes to run a credible service. As to transparency, what are you referring too? He opened up his development to anybody that was interested for testing purpose. He clearly outlines his sources which do not contain 'secret' honeypots.

Don't get me wrong, I support what you are attempting to provide as much as the next person. I just hate to see unwarranted antagonism. If I had the skills I would be tempted to do the same and provide such a service. Good luck on progress in the next months!!
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Mon Nov 05, 2018 9:51 pm

If you want to support him, please support him in his topics.
If you would like to be involved in the me-vs-him debate, please at least do a little background research on the history.
My project has been the subject of "IP Theft" several times, and I do my best to keep my server side tech hidden now because of that.
I shut down my project once before, one of the many factors was other projects taking my lists and pushing them out as their own.

People are free to choose what they want to use, but if you want to talk about his stuff, please do so elsewhere.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Mon Nov 05, 2018 10:03 pm

Tech Note: 6.44beta20 causes some issues, make sure you update to the current beta, if you are running the betas.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Tue Nov 06, 2018 12:33 am

I'm already signed up payment from January?
 
RackKing
Member Candidate
Member Candidate
Posts: 257
Joined: Wed Oct 09, 2013 1:59 pm

Re: Blacklist Filter (Development Topic)

Tue Nov 06, 2018 6:39 pm

So maybe a dumb question... I did have a look a the Patreon page. What level would you recommend to an integrator like who would offer this to his customers as part of a annual service offering? I would bill them directly and purchase your service. I suppose I could buy a tier and then upgrade as I cross that threshold? Will it be easy to see how many "routers" I have left to use?

I think this is a great project and thank you for all your efforts.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Tue Nov 06, 2018 10:12 pm

Their are no dumb questions!

I will for sure have a UI for you to manage your routers. My goal is to have the UI finished by mid December. Though I just missed a pretty big deadline, so I may end up having to hire another developer to work on the front end while I continue on the back end. The UI is expected to use the Patreon login, then provide you with the total number of routers you can enroll, as well as the currently enrolled and management of the serial numbers. Authentication of the routers will be handled by the email address you subscribed with, along with the serial number of the router. Each router will have one unique address entry (randomly generated) that will allow me to find users that are misusing or claiming my lists as their own.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Cooperdale
just joined
Posts: 5
Joined: Fri Feb 17, 2017 8:35 pm

Re: Blacklist Filter (Development Topic)

Sun Dec 30, 2018 6:16 pm

Hello, are there any news on this? I can't wait for this service to come alive.
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Sun Dec 30, 2018 8:44 pm

The service works well, the first payment has been sent.
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Mon Dec 31, 2018 11:03 pm

The service works well, the first payment has been sent.
My Also, ;-)
but where to submit my router's Cloud DNS name?
╰_╯ Ciao Marco!
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Mon Dec 31, 2018 11:36 pm

let's wait for a message from Dave.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Tue Jan 01, 2019 10:36 am

Hey guys, sorry for the radio silence, it’s been a pretty tough year but I’m trying to survive it.

I’ve been trying to get the automated registration process done but have ran into a few pretty nasty issues that I didn’t foresee. I’m going to put it on hold and process things manually until I can get some cash to pay another coder to work with me.

Going to spend the rest of the week with my kids, and then start fresh Monday morning.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
User avatar
sjoram
Member Candidate
Member Candidate
Posts: 116
Joined: Sun Feb 10, 2013 8:47 pm
Location: Essex, UK

Re: Blacklist Filter (Development Topic)

Sun Jan 27, 2019 2:42 pm

I found your Patreon.

I looked at the different 'tiers' - $10 currently works out about £7.50 a month...I'd be more than happy to support your work.

However, I do have a couple of questions (others with knowledge of your project may also have views) - sorry if this is not the best place to ask, but it seemed most likely to get the right visibility:

- I'm a home user, with ROS on two RouterBOARDs. Just about to swap both out with new RB750Gr3. I believe these have 256MB RAM, can these handle the largest tier?
- I found out about your work as I was one of the silly people that when a novice to ROS didn't include a PPPoE WAN interface on the default drop rules AND had some ROS services open to the wild, without port knocking. Also lax with updates, so Winbox vulnerability got exploited on one box. Yes, I know... Now fixed.
- One box particularly has a plethora of open dst-nat rules - mail server and the likes. I'm thinking that despite now having the default drop rules correct such that ROS itself is less vulnerable, your project may still be a benefit in protecting against unknown/unpatched exploits from known rogue addresses, not just for ROS but also the services behind those dst-nat rules... Would you agree?
Home user, working in IT. Home network is my lab.
ISP: Uno Communications
Hardware:
RB750 - Draytek Vigor 120v2 ADSL2+ Annex M
RB750Gr3 - Draytek Vigor 130 FTTC (VDSL) & RBD52G-5HacD2HnD
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Tue Feb 12, 2019 3:21 am

Hey everyone. I'm sorry for being out of touch. Here is an update.

So, life has been pretty rough over the last 12-14 months. I've lost most of my clients, the new government tax code is killing me, and server costs keep going up. On the personal side, I've been dealing some some pretty serious health issues, and I'm not entirely sure how this is going to end for me.

That said, I can't in good conscience bill people for something that I can't guarantee that I can keep working on. I do have my notebook with me all the time, but some of the drugs are simply preventing me from thinking clear enough to do any coding at all.

So, I'm going to leave it open to all, and hope that at least a few of you will donate to keep the servers up and running.
I will do my very best to continue to improve the lists and the script.

I'd like to leave this PayPal Donation button here. Use if you like, don't if you don't like.

Again, as long as the service can support itself (about $200/month USD) I'll leave it running.
I have also taught my 14 year old daughter how to keep the servers running (she's already studding quantum physics) so that if the worst case happens, she can keep it running for you all.

I can certainly promise that NONE of the donations will go to my medical bills. I don't want anyone thinking that I'm using donations for anything other than what they expect.

Thank you, and I hope to keep posting. :)
Image
https://www.paypal.com/cgi-bin/webscr?c ... source=url



BTW - this is the code that I use, and I prefer anyone to use for the script... make note of the path.
:local destPath "disk1/filterImport.rsc";
:local priority "2";

:local sn [:pick [/ip cloud get dns-name] 0 [:find [/ip cloud get dns-name] "."]];
/tool fetch mode=https url="https://bl.mikrotikfilters.com/fetch.php?priority=$priority" http-method=post http-data="$sn" dst-path="$destPath" output=file; /import file-name=$destPath;  /file remove $destPath;
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
User avatar
sjoram
Member Candidate
Member Candidate
Posts: 116
Joined: Sun Feb 10, 2013 8:47 pm
Location: Essex, UK

Re: Blacklist Filter (Development Topic)

Tue Feb 12, 2019 10:27 am

Hi Dave,

Very sorry to hear of the challenges that life has thrown at you of late. I sincerely wish you and your family all the very best.

Thank you for your work on this, you know yourself how much demand your servers have seen, so I am sure this is benefiting and making life easier for a lot of people!
Home user, working in IT. Home network is my lab.
ISP: Uno Communications
Hardware:
RB750 - Draytek Vigor 120v2 ADSL2+ Annex M
RB750Gr3 - Draytek Vigor 130 FTTC (VDSL) & RBD52G-5HacD2HnD
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Tue Feb 12, 2019 9:14 pm

Dave donates funds to patreon and here he will also donate
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Fri Feb 15, 2019 12:59 am

Humans are truly awful.
While at my treatment yesterday, I dozed off. (I'm there for 6 hours every 3 days)
While sleeping, someone stole my backpack with my 6 month old notebook.

I'll still be doing some coding at home, but it's hard to sit at my desktop. I'll keep you all posted.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
ihave
just joined
Posts: 5
Joined: Wed Feb 01, 2017 4:38 pm

Re: Blacklist Filter (Development Topic)

Fri Feb 15, 2019 2:45 am

That is truly awful indeed!
Sorry to hear that.

Can someone sum up the latest script/instructions on how to install the new service?
The old install script with blInstaller.rsc doesn't show anything in the log.

Filterimport.rsc is running fine and updating the address list. Am I only missing the scheduler or did the BlInstaller.rsc install additional things I am still missing?

Thank you
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Fri Feb 15, 2019 8:10 pm

Can someone sum up the latest script/instructions on how to install the new service?
Script is 4 post higher. :lol: :mrgreen:
╰_╯ Ciao Marco!
 
ihave
just joined
Posts: 5
Joined: Wed Feb 01, 2017 4:38 pm

Re: Blacklist Filter (Development Topic)

Fri Feb 15, 2019 9:18 pm

Script is 4 post higher. :lol: :mrgreen:
That script is running fine. I am just wondering about the frequency of the scheduler and maybe other things that were installed with the old installer.
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Sun Feb 17, 2019 3:07 pm

That script is running fine. I am just wondering about the frequency of the scheduler and maybe other things that were installed with the old installer.

OK. Mmmmh, frequency of the scheduler:
That makes everyone different. I update every 12 hours. I think Dave updates the blacklists every hour.

I summarize everything for the people who are new here:

- Enable ddns. I have configured the SNTP client and do not need time here.
/ip cloud set ddns-enabled=yes update-time=no

- The script (destPath and priority may need to be adjusted):
/system script add dont-require-permissions=no name=blacklistScript owner=admin policy=read,write,policy,test source=\
\n"# Intrus Technologies blacklist installer/updater\
\n# \A92017 David Joyce, Intrus Technologies\
\n\
\n:local destPath \"filterImport.rsc\";\
\n:local priority \"2\";\
\n\
\n#If you do not want to delete the script directly after importing, remove the comment and paste one in the last line\
\n#:do { /file remove \$destPath } on-error={};\
\n\
\n:local sn [:pick [/ip cloud get dns-name] 0 [:find [/ip cloud get dns-name] \".\"]];\
\n/tool fetch mode=https url=\"https://bl.mikrotikfilters.com/fetch.php\?priority=\$priority\" http-method=post http-data=\"\$sn\" dst-path=\"\$destPath\" output=file;\
\n/import file-name=\$destPath;\
\n/file remove \$destPath;\
\n"

- The sheduler (After every reboot and then every 12 hours at 6:00 and 18:00) Adjust times as you want:
/system scheduler add interval=12h name=blacklistScriptUpdate on-event="/system script run blacklistScript" policy=read,write,policy,test start-date=jan/01/1970 start-time=06:00:00
/system scheduler add name=blacklistScriptUpdateOnBoot on-event=":delay 30;system script run blacklistScript" policy=read,write,policy,test start-time=startup

- The firewall rule:
Others also filter outgoing traffic (dst-address-list) and/or other interfaces. I block all port scanners, bruteforcer and blacklists in the beginning in the RAW chain. (Connection tracking would be a waste of time ;-) I prefer to use the saved computing time for tarpit rules. I tarpit all WAN connections to unused TCP ports of the router.
/ip firewall raw add action=drop chain=prerouting comment="Drop WAN connections from 'intrusBL' blacklisted hosts <- Src. Address List: intrusBL" in-interface-list=WAN src-address-list=intrusBL

Hope that helps :wink:
Last edited by boldsuck on Sun Feb 17, 2019 5:00 pm, edited 1 time in total.
╰_╯ Ciao Marco!
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Sun Feb 17, 2019 4:15 pm

@Dave
:D 8) Wow, 14 year old woman is interested in coding! Very nice.

Crap. Stolen laptop is one of the worst cases.
I do not know exactly "New US government tax" but I think that's what Mr. Trump introduced. (He needs money for his wall. In Germany, we are glad that the wall is gone!)

Should we still run the reportStatus script? (That from #Post 37)
╰_╯ Ciao Marco!
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Fri Mar 01, 2019 3:16 am

The ReportStatus is 100% optional. It's mostly for stats and being able to ty and tailor the lists to the bulk of the routers. Surprisingly, their are far more CloudCore routers running the blacklist now than the smaller units.

I wanted to thank those of you whom have reached out with support. Every little bit helps! It's been a few weeks and I'm starting to grow hair again... Just when I was getting used to being bald. It won't be long though, treatments start again in 6 weeks.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Tue Apr 02, 2019 6:05 pm

Good Morning and happy April.

A few updates for you this morning.
  • Server is running stable with no issues other than the random LetsEncrypt cert error.
  • Once the list's income manages to cover it's costs, I'll be switching to Digicert for ssl certs.
  • I've begun work on adding IPv6 to the service, it's pretty easy on the router side, changes things quite a bit on the backend
  • For those following, the police found my notebook, though it's completely wrecked. they are keeping it as evidence. They didn't find any of my tools.
  • as for the service pricing, I'm still keeping it as a donation only service. Again, I don't want to bill for a service that I may not be around to keep running.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Tue Apr 02, 2019 9:12 pm

It can be paid by Patreon?
Last edited by Rico40 on Wed Apr 03, 2019 12:52 am, edited 1 time in total.
 
msatter
Forum Guru
Forum Guru
Posts: 1200
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter (Development Topic)

Tue Apr 02, 2019 11:18 pm

Humans can be truly awfull but using you undergoing your treatment to steal from you then there are no words to describe my feelings about that.

I am sorry to read that you are ill and that the outcome is uncertain. I wish all the strength to overcome this horrible time in your life.
Two RB760iGS (hEX S) in series. One does PPPoE and both do IKEv2.
Running:
RouterOS 6.46Beta / Winbox 3.19 / MikroTik APP 1.3.2
Having an Android device, use https://github.com/M66B/NetGuard/releases (no root required)
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Wed Apr 03, 2019 1:10 am

It can be paid by Patreon?
Yes, somewhere up above, there is a Patreon link, as well as PayPal links.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Wed Apr 03, 2019 1:11 am

Humans can be truly awfull but using you undergoing your treatment to steal from you then there are no words to describe my feelings about that.

I am sorry to read that you are ill and that the outcome is uncertain. I wish all the strength to overcome this horrible time in your life.
Thank you. No worries, I’ve survived everything else life has tossed my way. Even an earthquake that brought my apartment down on me.

If I could just keep my fingers and toes warm, I would be happy.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Fri Apr 26, 2019 12:23 am

Just an update... I'm still here, and I have hair again. :) Well, I have hair for a few weeks at least.

I'm still working on the IPv6 version of the list. My home ISP has finally managed to provide a stable v6 connection, so I'll be able to start testing.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Sun May 05, 2019 4:59 pm

I'm still working on the IPv6 version of the list. My home ISP has finally managed to provide a stable v6 connection, so I'll be able to start testing.

IPv6 Yeah! That's very good news.
Have meanwhile a few servers on the run. (Debian, Static IP & IPv6)
If you want to test something on it ...
can I give you access. I only need a pub ssh-key from you.
╰_╯ Ciao Marco!
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Tue May 28, 2019 5:43 am

I regret to inform you all that I have shut down the servers.
With over 35,000 routers hitting the server every few hours, and only 10 supporters (totally $50/month), the expenses are not even remotely being covered. I'm putting out almost $500/month now just in bandwidth costs.

I'm moving the code to my in home server where it will just be supporting my personal units now.

it was a good run. I tried to keep it going, I thought that more people would be willing to help, but sadly not.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
wanos
just joined
Posts: 3
Joined: Thu Aug 16, 2018 12:43 pm

Re: Blacklist Filter (Development Topic)

Tue May 28, 2019 2:02 pm

We truly appreciated all your efforts Dave. Well, too few of us apparently did. So many want something for nothing. Sucks.

Thank-you for that and hope all goes well for you and your family.
 
hhgttg42
just joined
Posts: 8
Joined: Wed Oct 12, 2016 4:48 am

Re: Blacklist Filter (Development Topic)

Tue May 28, 2019 4:41 pm

Sorry to hear that Dave. Please let us know if you ever change your mind!

Best of luck.
 
HZsolt
just joined
Posts: 24
Joined: Tue Apr 24, 2018 7:31 pm

Re: Blacklist Filter (Development Topic)

Tue May 28, 2019 8:08 pm

[DELETED]
Last edited by HZsolt on Tue May 28, 2019 8:28 pm, edited 1 time in total.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Tue May 28, 2019 8:25 pm

Dude, really? My lively-hood is going down the drain, and you pop up to help push the knife in. Nice.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Wed May 29, 2019 8:11 am

I resigned from patreon in favor of donate paypal.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8305
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Blacklist Filter (Development Topic)

Sat Jun 08, 2019 11:47 am

Dave, any chance to get the code open-sourced? I mean, maybe someone would like to donate servers/bandwidth instead of money...
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5715
Joined: Mon Jun 08, 2015 12:09 pm

Re: Blacklist Filter (Development Topic)

Sat Jun 08, 2019 12:43 pm

I regret to inform you all that I have shut down the servers.
With over 35,000 routers hitting the server every few hours, and only 10 supporters (totally $50/month), the expenses are not even remotely being covered. I'm putting out almost $500/month now just in bandwidth costs.
I'm curious how much bandwidth your service consumes... there is another poster complaining about $500/mon bandwidth cost for his service, but frankly I cannot understand how this can happen.
For that $50/mon I would get about 10TB/mon of bandwidth at local cloud hosting companies, and with 35000 routers that would be 285MB per router per month.
When that isn't enough by a factor of 10 I would seriously consider revising the method for distributing the data... e.g. some form of incremental updating.
 
msatter
Forum Guru
Forum Guru
Posts: 1200
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter (Development Topic)

Sat Jun 08, 2019 1:41 pm

I helped with an earlier version and it is should be incremental and your get the changes you missed since the last sucessful update you had.

The sheer number of routers connecting still can give a heavy bandwith usage.

Dave is doing a great job despite his personal set backs.

viewtopic.php?t=98804
Two RB760iGS (hEX S) in series. One does PPPoE and both do IKEv2.
Running:
RouterOS 6.46Beta / Winbox 3.19 / MikroTik APP 1.3.2
Having an Android device, use https://github.com/M66B/NetGuard/releases (no root required)
 
Rico40
just joined
Posts: 16
Joined: Sun Aug 19, 2018 8:53 pm
Location: Poland

Re: Blacklist Filter (Development Topic)

Sat Jun 08, 2019 8:19 pm

it's more about reducing the cost of the service.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5715
Joined: Mon Jun 08, 2015 12:09 pm

Re: Blacklist Filter (Development Topic)

Sat Jun 08, 2019 9:25 pm

Please show a calculation of how much bandwidth per month you need per router using the service, and how much the cost for different amounts of bandwidth per month is.
 
IntrusDave
Forum Guru
Forum Guru
Topic Author
Posts: 1284
Joined: Fri May 09, 2014 4:36 am
Location: Rancho Cucamonga, CA

Re: Blacklist Filter (Development Topic)

Thu Jun 13, 2019 9:00 pm

Sorry, but I feel no need to disclose my stats and financial needs for a service that is free.
I can tell that you 4 servers, 120 honeypots, a CDN, storage and the bandwidth needed for all of it is quite a lot.
I wont be open sourcing the code either. it's 100% written by my with no use of any open source code.
It has a use to me still and I will be keeping it for myself.
David Joyce
Network & Security Engineer
Intrus Technologies, LLC.
Rancho Cucamonga, CA, USA
 
suszi
just joined
Posts: 6
Joined: Mon Apr 10, 2017 2:08 pm

Re: Blacklist Filter (Development Topic)

Thu Jun 13, 2019 9:28 pm

the problem is, with RB, that ip firewall raw action=drop doent work with src-address-list=intrusBL
according to documentation:
address-list (string; Default: ) Name of the address list to be used. Applicable if action is add-dst-to-address-list or add-src-to-address-list
https://wiki.mikrotik.com/wiki/Manual:IP/Firewall/Raw
/ip firewall raw
add action=drop chain=prerouting comment="DROP intrusBL" src-address-list=intrusBL
add action=drop chain=prerouting comment="DROP intrusBL" dst-address-list=intrusBL
any way to use RAW? or just regular firewall rule?
 
suszi
just joined
Posts: 6
Joined: Mon Apr 10, 2017 2:08 pm

Re: Blacklist Filter (Development Topic)

Thu Jun 13, 2019 9:29 pm

I'm updating my second portscanners (TCP SYN) list manually - where can I send it, to be included ?
maybe there is a better way ?
 
msatter
Forum Guru
Forum Guru
Posts: 1200
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Blacklist Filter (Development Topic)

Thu Jun 13, 2019 11:05 pm

The ADD in the rules is there to add the line to the RAW section in the firewall. After thst it not used anymore.

Dropping unwanted traffic is most efficient in RAW and so it won't reach connection tracking.
Two RB760iGS (hEX S) in series. One does PPPoE and both do IKEv2.
Running:
RouterOS 6.46Beta / Winbox 3.19 / MikroTik APP 1.3.2
Having an Android device, use https://github.com/M66B/NetGuard/releases (no root required)
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8305
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Blacklist Filter (Development Topic)

Fri Jun 14, 2019 12:38 am

the problem is, with RB, that ip firewall raw action=drop doent work with src-address-list=intrusBL
according to documentation:
address-list (string; Default: ) Name of the address list to be used. Applicable if action is add-dst-to-address-list or add-src-to-address-list
https://wiki.mikrotik.com/wiki/Manual:IP/Firewall/Raw
/ip firewall raw
add action=drop chain=prerouting comment="DROP intrusBL" src-address-list=intrusBL
add action=drop chain=prerouting comment="DROP intrusBL" dst-address-list=intrusBL
any way to use RAW? or just regular firewall rule?
Address-list, src-address-list and dst-address-list are three different parameters. You're talking about one and look at the description of another.
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
boldsuck
Frequent Visitor
Frequent Visitor
Posts: 58
Joined: Sun Sep 01, 2013 1:07 am
Location: Germany

Re: Blacklist Filter (Development Topic)

Thu Jun 20, 2019 10:42 pm

I thought that more people would be willing to help, but sadly not.
me too.

I have now unsubscribed at Patreon. ;-(

Thanks for everything Dave. You and this thread inspired me to refine my own raw bruteforce & portscan rules.

I'm trying to do something similar. With fail2ban (0.10 has ipv6 support) My servers run as fail2ban reporting service anyway. For months there is already a UBNT ER-Pro 8 around here. On EdgeOS, packages from the debian archives can be installed. Fail2ban should then be easy to install on the router.
╰_╯ Ciao Marco!

Who is online

Users browsing this forum: No registered users and 4 guests