Hi there,

https://blocklister.gefoo.org

Blocklister is a simple API for generating MikroTik RouterOS compatible addresslists. The ipaddresses included in these lists are regularly updated using source sites such as iblocklist.com, dshield.org etc. Importing the resulting scripts into your routers is done by using a simple script which can later be scheduled to run every x days for example. There is no need to clean you addresslists manually as the resulting scripts includes a cleanup based on the comment of the addresslists, just be careful that you don't change the comments on the go. Name and comment of the resulting addresslists scripts can be customized using query parameters.

This solution has been heavily inspired by http://joshaven.com/resources/tricks/mi ... ress-list/.

Please note that each source ip address is limited to 10 requests per day!


Usage Example

Following a simplified example output. Each response's content-type is in plain/text. In this example I customize the addresslist name to "foo" and the comment to "bar".
Result (please note that the result has been striped of here and the ips have been replaced by bogus ips) As you can see the resulting script also takes care of cleaning your addresslists first by removing everything in your addresslist which had the comment "bar" in it.

RouterOS Script Example

To automatically download and import the addresslists in your router you could use a script similar to this one. You will find the same script in the documentation link of the API https://blocklister.gefoo.org .

API Links

• Basic documentation for the API and a list of available blocklists - https://blocklister.gefoo.org
• Ads - https://blocklister.gefoo.org/ads
• Spyware - https://blocklister.gefoo.org/spyware
• Level1 - https://blocklister.gefoo.org/level1
• Level2 - https://blocklister.gefoo.org/level2
• Level3 - https://blocklister.gefoo.org/level3
• Edu - https://blocklister.gefoo.org/edu
• Proxy - https://blocklister.gefoo.org/proxy
• Badpeers - https://blocklister.gefoo.org/badpeers
• Microsoft - https://blocklister.gefoo.org/microsoft
• Spider - https://blocklister.gefoo.org/spider
• Hijacked - https://blocklister.gefoo.org/hijacked
• Dshield - https://blocklister.gefoo.org/dshield
• Malwaredomainlist - https://blocklister.gefoo.org/malwaredomainlist
• Openbl - https://blocklister.gefoo.org/openbl
• Openbl_180 - https://blocklister.gefoo.org/openbl_180
• Openbl_360 - https://blocklister.gefoo.org/openbl_360
• Spamhausdrop - https://blocklister.gefoo.org/spamhausdrop
• Spamhausedrop - https://blocklister.gefoo.org/spamhausedrop

Query parameters

With these query parameters you can customized the name and comment which will be used to create the address-list on your router.

• listname - The name of the resulting addresslist
• comment - Use this query parameter to adjust the comment which will be used for this addresslist (Please make sure that this comment is unique among your addresslists)

WARNING

Please use this with care as you could possibly delete all of your addresslists entries.
Be careful with lists such as Level1, Badpeers they are extremely big and could crash your router.

Suggestions

Any suggestions are welcome. If you have some other ip lists that you would like to have added please let me know.

Hi,

would it be possible to split the level1 list? The other lists are working fine.
I tried to apply the list but the router crashes. It rans out of Memory.
I have an rb2011

Regards

This is great thanks.

Hi,

would it be possible to split the level1 list? The other lists are working fine.
I tried to apply the list but the router crashes. It rans out of Memory.
I have an rb2011

Regards

This could be done and idea that pops into mind is to introduce a pagination of some sort. So the idea would be that you would have to define the page_nr and page_size in your request. So in final you would say http://blocklister.domain.com/level1?pa ... ge_size=10 this would return you the 10 first firewall entries of the level1 list.

What do you think of this idea?

Hi,

this would help me and is an fine solution.

I dont know its a problem of your script but when i update badpeers i get the error in the log
script error: expected end of command (line 47713 column68)
And this error i got one time:
sricpt error: missing values(s) of arguments list

I will need to control this.

I want to point out a general update on this, I am quite ready to push this project to github now, so you could open up an issue for this.

Hi guys. I am happy to announce that blocklister is available on https://github.com/flazzarini/blocklister, now. On top of that I've released and pushed a new version of blocklister version 1.3.0. No major changes only to the code structure and dependencies.

If you have any requests please feel free to post them over at https://github.com/flazzarini/blocklister.


If I find the time I will prepare as well a Virtual Machine having blocklister already installed, so you could deploy blocklister conveniently at home.

Based on experience, you may want to use blackhole routes instead of an Address List. For larger lists, blackhole routes take up less CPU and RAM so they tend to perform much better and on lesser hardware. The only thing you lose is the ability to count the number of "hits" to the blacklists through firewall rules, as blackhole routes are processed far earlier in the packet flow than firewall rules (part of the reason they're so efficient).

Example blackhole route:

/ip route add dst-address=x.x.x.x/x type=blackhole comment="blacklist"

Example to clear out the blacklist before importing a new one:

/ip route remove [find comment="blacklist"]


This could also all be done with BGP routes, which would allow you to dynamically add/remove individual entries without having to clear out and re-import the entire list every time.

Just my $0.02!

-wade

Based on experience, you may want to use blackhole routes instead of an Address List. For larger lists, blackhole routes take up less CPU and RAM so they tend to perform much better and on lesser hardware. The only thing you lose is the ability to count the number of "hits" to the blacklists through firewall rules, as blackhole routes are processed far earlier in the packet flow than firewall rules (part of the reason they're so efficient).

Thanks @Zebble for pointing that out. We could introduce a new feature to blocklister where you would have the choice between addresslist and blackholing. Not sure yet the route on the api would look like but it could be as simple as.

http://domain.com/ads?render_as=blackhole


Let me know what you think. Might also be worth it to open up an issue over at https://github.com/flazzarini/blocklister/issues

Hi there,

happy to announce that version 1.6.0 of blocklister is now available at https://blocklister.gefoo.org and the code has been published at https://github.com/flazzarini/blocklister. For the changelog please visit https://blocklister.gefoo.org/changelog, major new feature is to return ip ranges as CIDR notation, by passing a query parameter for instance https://blocklister.gefoo.org/ads?cidr=True

Hope this helps