Hello, are you having problem adding url list of StevenBlack ? When I add it to Adlist it doesn’t recognize any results and matches. I think it’s because the sheet starts with this:
# Title: StevenBlack/hosts
#
# This hosts file is a merged collection of hosts from reputable sources,
# with a dash of crowd sourcing via GitHub
#
# Date: 11 July 2024 19:19:26 (UTC)
# Number of unique domains: 155,096
#
# Fetch the latest version of this file: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
# Project home page: https://github.com/StevenBlack/hosts
# Project releases: https://github.com/StevenBlack/hosts/releases
#
# ================================================ ==============
127.0.0.1 localhost
127.0.0.1 localhost.localdomain
127.0.0.1 local
255.255.255.255 broadcasthost
::1 localhost
::1 ip6-localhost
::1 ip6-loopback
fe80::1%lo0 localhost
ff00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
0.0.0.0 0.0.0.0
# Custom host records are listed here.
# End of custom host records.
# Start StevenBlack
#=====================================
# Title: Hosts contributed by Steven Black
# http://stevenblack.com
0.0.0.0 ck.getcookiestxt.com
0.0.0.0 eu1.clevertap-prod.com
0.0.0.0 wizhumpgyros.com
0.0.0.0 coccyxwickimp.com
0.0.0.0 webmail-who-int.000webhostapp.com
...........................
If I make my own file and copy only the useful information from its list things work.
0.0.0.0 XXXXXXXXXXXX
Because in the video tutorial things work out, but I don’t know then if the leaf looked like this .
You’re right, I tried it on the RB750Gr3, but I also have a HAP AX³ and it worked, got the list without problems. Well, I will use it for DNS. Thank you very much for the help.
quote: "MOAB has two tracks, one for MikroTik Routers like the hEX, hAP ax 2, hAP ax 3, the Audience - tracking between 5K and 16K ipset entries - MikroTik Router models like the CHR, RB3011, RB4011, RB5009, RB1100 and all CCR - tracking between 35K and 60K ipset entries. Once we know which MikroTik Router model you have we will decide if your model qualifies and which track to put you on. Both tracks cover over 600 million IP addresses of known perpetrators.
For MikroTik Routers like the CHR, RB3011, RB4011, RB5009, RB1100 and all CCR models the maximum download file size is 1 MB or less - 3 times daily.
For MikroTik Routers like the hEX, hAP ax 2, hAP ax 3, the Audience - the maximum download file size is 0.5 MB or less - 3 times daily."
You have to set your DHCP server or the hosts themselves manually, the DNS server to be the IP address of the Routerboard on which you use Adlist, then you will start to have reporting. The interesting thing is that for me it gives me 154749 on hAP AX3, and on my virtual machine I installed RouterOS with x86 architecture and there it recognizes them as 17266 entries, and for you it gives them 100908. It’s different for everyone, I can’t explain it.
hAP AX3
Flags: X - disabled
0 url="https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts"
ssl-verify=no match-count=2368 name-count=154749
Virtual RouterOS x86
Flags: X - disabled
0 url="https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts"
ssl-verify=no match-count=0 name-count=17266
It’s not correct because it removes the lines with hash tags even if the hash tag come after correct value.
Like this lines.
0.0.0.0 iesnare.com # See http://www.codingthewheel.com/archives/online-gambling-privacy-iesnare
0.0.0.0 www.iesnare.com # See http://www.codingthewheel.com/archives/online-gambling-privacy-iesnare
So these lists have to be massaged a little bit before going into the mikrotik device.
@mkx Thx,
I am not the best regex script kiddies
But if Mikrotik device count is false, it’s also removes all the lines with hash tags.
But i not getting it with this low value 17266. Maybe it’s not getting the hole file.
I’ve never used adlist feature, but I’d expect it to emit some kind of diagnostic messages upon importing the list. At least, say, number of items successfully imported in info channel and any crucial problem in error channel (e.g. if import breaks in the mid of file due to lack of memory or some such). Without it it’s impossible to tell why some hosts have lower number of entries than others (and I’d assume that file parser acts the same on all MT platforms so the difference should then be tied to individual device state differences).
BTW, I don’t expect comments to be the problem, at least config file parser is well able to ignore comments on the code lines.
The problem is poor/stupid programmer or design decision to download adlist files first to device storage instead of memory… Since the HexS only has 16 MB of storage, and at best only a few MB free with ROS 7, you see the problem.
I use this script I made, which downloads the target file into memory and not the permanent storage:
:delay 10s
# Check cache size and adjust if not large enough
:if ([ip/dns/get cache-size]!=32768) do={/ip/dns/set cache-size=32768}
# Remove all current adlists
/ip/dns/adlist remove [find]
# Fetch new adlist source file(s)
/tool/fetch url=https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts output=file dst-path=sblack-unified-adlist
# Short delay to ensure file is created in RAM
:delay 1s
# Create new adlist from file
/ip/dns/adlist/add file=sblack-unified-adlist
# Remove temp file from RAM
/file remove sblack-unified-adlist
And it works just fine. That list on my HexS shows:
@codelogic
Thank you!
Finally I can also use it with hAP AC2, manually updating the host list every time was a nightmare…
I created a scheduler with your script and it works perfectly.
