MikroTik Dynamic IP Blocklist Generator – Update (v0.9.2)

[rextended]

Here we are starting over again to see if we can find a better wheel than the round one...
...or rather, no knows of its existence...

[rextended]

Finally something logical.

[abyss]

At this stage, this is not a technical competition.
The goal of this post is simply to gauge community interest in the project and collect constructive feedback to see if there’s a need and to publish it freely on github…

[rextended]

@abyss

constructive feedback

You've certainly had some from me and others.

Anything that prevents the end user from being scammed is always constructive.

[rextended]

Hmm... Looking ahead, I see MikroTik's move to avoid these botnets...
A new function like in device-mode:

[rex@tended] > import file=myscript.rsc
import: please confirm by pressing reset or mode button in 45s

[BartoszP]

Just my 0.03$ on that topic - dates to 2014:

6.16 import stops when there is a duplicate entry

A duplicate entry in the import will cause RouterOS to stop and not process further down the list (v6.16) ie (dshield.rsc below) /ip firewall address-list add list=DShield address=122.226.73.131/32 timeout=2h add list=DShield address=85.13.160.11/32 timeout=2h add list=DShield address=50.23.113.146/32 timeout=2h add list=DShield address=50.23.113.146/32 timeout=2h add list=DShield address=122.225.109.212/32 timeout=2h add list=DShield address=176.31.251.200/32 timeout=2h add list=DShiel…

6.16 import stops when there is a duplicate entry

Could you prepare the script that way do { /ip firewall address-list add address=111.251.111.129 list=blackmail timeout=3h } on-error={} instead of /ip firewall address-list add address=111.251.111.129 list=blackmail timeout=3h I have no 6.16 so it is only my guess that such solution works.

6.16 import stops when there is a duplicate entry

works like a charm, as a side effect log is not spammed by address list entry messages Thank you very much!

[jaclaz]

I think we are back to the advice Mikhail Kalashnikov once gave:

A lot of Russian Army soldiers ask me how one can become a constructor, and how new weaponry is designed. These are very difficult questions. Each designer seems to have his own paths, his own successes and failures. But one thing is clear: before attempting to create something new, it is vital to have a good appreciation of everything that already exists in this field. I myself have had many experiences confirming this to be so.

[BartoszP]

Jaclaz: you kill the crativity at the very begining.

You know the sentence: no one said it's impossible, so we did it.

I have saved a link that appeared on that forum: https://sbwi.edu/blog/learning-by-doing-vs-learning-by-watching-or-reading#:~:text=Hands-On%20Learning%20and%20the,in%20better%20understanding%20and%20retention.

You have to practice on a simple, repetable well known examples to master some skills. IT/CS studends implement 10 lines long sorting algorithms again and again. What for?
I remember the student workshops of my colleagues who were triangulating in the square in front of their construction department. It was the best triangulated square in the world. What for? Teachers could recite data about every paving slab by heart, even those that had been replaced once or even twice.

[jaclaz]

@BartoszP

you kill the crativity at the very begining.

Not at all, I am only trying to make sure that it lays on good foundations.

Anecdotally, I had a friend which taught in a land surveyor high school, students had to team in groups of three and make a levelling survey of the same square near the school, so my friend got roughly ten surveys per year, and he could swear that he never had two identical surveys in the twenty + years he was at that school.

[BartoszP]

@jaclaz

he could swear that he never had two identical surveys

It's called "students' CREATIVITY". Did he concider rock mass movements? Earthquakes? Different temperatures? Problems with meserument tools' precision? I know .... drifting too far

[abyss]

Hi everyone,

After the recent clean-up of the discussion atmosphere on the forum, I finally feel confident enough to share something I’ve been working on for a while.

I’m happy to present my MikroTik Blocklist Generator, now publicly available and open-source:

https://github.com/reptechwifx/mikrotik-blocklist
Docker Hub: reptechwifx/mikrotik-blocklist

It aggregates several IPv4 threat sources, applies automatic subnet aggregation, handles a YAML-based whitelist, and generates ready-to-import RouterOS scripts (/ip firewall address-list).
All inputs remain fully transparent, and absolutely nothing is hidden or obfuscated.

Since the atmosphere on the forum has recently been re-extended in a more constructive direction, I’ll be happy to continue contributing and improving the tool based on your feedback.

I’ve taken the comments above into account for this initial release.
The default address-list name is now configurable via environment variables, and the import script has been kept as simple and transparent as possible.

I’m also considering optimizing the import script further using {on-error} to see if there is any real performance benefit, but I’ll only add that if it remains readable and safe for normal users.

For reference, on a small device (RB750UP) it currently takes about 15 seconds to import a list of roughly 9,000 entries, which I find acceptable for a scheduled update.

Feel free to test it, challenge it, break it, suggest new features

I’m still pretty new to GitHub and Docker Hub, so any feedback or suggestions to make the project easier to use are very welcome!

Thanks,
Julien

[abyss]

Changelog

v0.9.1

• Kept /ip firewall address-list at the beginning of the .rsc to prevent a large file increase with
  :do { add … } on-error={}

Added

• Added environment variable MIKROTIK_LIST_NAME to define the default MikroTik address-list name. - Added safe MikroTik import format using:

• Added safe MikroTik import format using:

:do { add … } on-error={}

This prevents script failures due to duplicates.

• Unified output format for /custom.rsc, /all.rsc, and /mikrotik.rsc (all now use the safe do-wrapper logic).

• Added a single /ip firewall address-list header to reduce script size.

• HTML UI now uses MIKROTIK_LIST_NAME as the placeholder for the address-list input field.

Changed

• Removed all remove [find] calls to avoid clearing existing lists on each update.

• Improved /24 aggregation logic for large feeds.

• Improved handling of per-source comments.

• Whitelist (yaml + on-URL) is now evaluated uniformly across all output types.

Fixed

• Resolved placeholder text mismatch in the HTML generator.

• Corrected inconsistencies between the behavior of /all.rsc and /custom.rsc.

• Stability improvements when processing very large blocklists.

[BartoszP]

Tested.

Still produces:

[abyss]

Yes, this is currently an internal service tied to our internal git setup. I still need to update a few things internally, because it’s already connected to some routers.

The idea is to let users run their own instance, so they can protect their routers and avoid ending up in someone’s botnet.

Version 0.9.1 is published on Docker Hub:

https://hub.docker.com/repository/docker/reptechwifx/mikrotik-blocklist/general

and can be tested here:

https://blocklist.wifx.net

[abyss]

updated the demo… https://blocklist.wifx.net

you will also find the source code of our list generator http://speedtest.wifx.net:8888 here:

GitHub - reptechwifx/banlist: Easy Service in Python to monitor auth.log and...

Thanks

[safelogj]

fix to http speedtest link

[BartoszP]

@abyss

Please stop copying links again and again to make web crawlers index it better.
We are aware that the proper links are in the first post. Everyone interested would find it.

[abyss]

Apologies if it looked like unnecessary repetition or promotion.
I just wanted to update the first post once and avoid cluttering the thread with follow-ups for every change.