I saw today that one of the biggest adlist providers - hagezi will stop providing the "hosts" format which mikrotik adlist feature uses. The main concern is they want only to optimize their lists for size and efficiency.
As i looked into AdGuard format converting blindly will not work, because this format blocks domain and all subdomains listed in each record. For example:
I guess if they want to block only specific subdomains they will do the below:
||subdomain.example.org^
I compared two versions of the hagezi pro lists - one with mikrotik (hosts) format and one with adguard(wildcard) format. Normal one was 300k and adguard one was 170k records.
Will it be possible mikrotik to support this format or the format that for example is most popular with pihole, so we can use most of the lists.
I think the ||example.com^ format is the most widely used format nowadays.
I’ve tried adding https://small.oisd.nl/ as blacklist and it seem it is at least working so far, that the listed domains are blocked, but without regex matching. So ||example.com^ does not block www.example.com. Probably the special characters are just stripped.
I guess regex matching could be quite hard on CPU time and memory usage. But why not just add a switch “block subdomains”? This should be enough for the majority of list entries.
These wildcard formats are not "real" regexes. They match in the usual dns wildcard way. (Well, almost.) So the matching, even if it's implemented in the plainest possible way via progressive hash table lookups, only leads to a 2-3x load on the cpu...
I never really understood why this wasn't originally included in the feature.