Recommend way to block Ads with Mikrotik

As you already wrote, ad-blocking using DNS is limited and will not always work.
The best ad-blocking is still in the browser, as it can operate on full URL instead of only hostname.
Also it can in theory (unfortunately ABP cannot do that) load some ad data and not display it, e.g. in cases where a website will not work when you do not load the ads.

@stuartkoh

Thanks for the write-up.

Nice find. Will give them a try.

Yeah I’ve been using them for several days now, with no obvious issues.

Will probably use them as secure upstream dns servers from a pihole too, seeing as they reckon they don’t log queries…

“AdGuard DNS takes several measures to ensure your privacy safety. We support DNSCrypt, DNS-over-HTTPS (DoH) and DNS-over-TLS (DoT) protocols.”

Sent from my Mi 9T using Tapatalk


\

there are still some unofficial cloudflared builds for pi. But I’ve configured ‘unbound’ DNS resolver on pi zero w with forwarders to cloudflare and quad9 over DoT (dns over TLS). Config posted in the same reddit thread. this works better than cloudflared daemon. My new lookups are 0.2 seconds , which is a bit slow imo, but not too bad. also Unbound refreshes cache on its own for the repeating queries to reduce the lookup time (ie the longer it runs the more it “learns” how to pre-populate the cache).

there seems to be an initiative by chrome and Firefox to enable a DoH resolver built in to the browsers. this would bypass pihole. I would stay away from this setting if you are using dns based ad blocking. sorry if this is obvious but wanted to mention anyway..:slight_smile:

I am using 3 ways to block unwanted content (ads are a part of the unwanted content)

  1. uBlock on any browser on our computers I use Vivaldi (Chromium based) and push uBlock with an GPO on Windows
  2. Pi-Hole running on a virtual linux computer
  3. IP blocking in Mikrotik
    With uBlock you are also blocking unwanted scripts in the pages.

For the IP’s to block I’d like to refer to:

  1. http://www.squidblacklist.org/downloads.html which serves various IP list you can autmaticaly import with a script into your MKT device.
  2. https://mikrotikconfig.com/firewall/, with this site you can create a IP block list for a lot of countries

Hey.
Squidblacklist.org is dead (as it’s owner), sorry.
Some time ago i was showing ads-blocker made on squidblacklist blacklists… (https://www.youtube.com/watch?v=DhheFeUPo2g)

However squidblacklist is dead, the pi-hole project is so vital now…
Now presenting: the pi-hole to mikrotik converter.
Check it out:
The converter: https://pastebin.com/rQVYcdi3

##############
#/bin/sh
mtuser=admin
sshpass=“$(cat mtpass)”
mtip=192.168.2.1

echo $sshpass

#expand aliases
shopt -s expand_aliases
source ~/.bashrc
listy=("> https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/hostfile.txt> “)
listy+=(”> https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/adguard_host.txt> “)
listy+=(”> https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/Ad_filter_list_by_Disconnect.txt> “)
listy+=(”> https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/adguard_mobile_host.txt> “)
listy+=(”> https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/ads-and-tracking-extended.txt> “)
listy+=(”> https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/adservers.txt> “)
listy+=(”> https://raw.githubusercontent.com/MajkiIT/polish-ads-filter/master/polish-pihole-filters/android-tracking-pihole.txt> ")

echo ${listy[@]}

#nazwa pliku wynikowego
_script_file=blacklista.rsc
rm -f $_script_file 2>/dev/null 1>/dev/null
rm -f “${_script_file}.tmp” 2>/dev/null 1>/dev/null
rm -f “${_script_file}.tmp.tmp” 2>/dev/null 1>/dev/null


for i in “${listy[*]}”
do
curl -sL $i >> ${_script_file}.tmp
done;

Convert to unix format - wymaga doinstalowania

dos2unix ${_script_file}.tmp 2>/dev/null 1>/dev/null

LC_ALL=C


######################

Help aliases

alias incl_ipaddr=“egrep -e ‘([0-9]|[0-9][0-9]|[0-9][0-9][0-9]).([0-9]|[0-9][0-9]|[0-9][0-9][0-9]).([0-9]|[0-9][0-9]|[0-9][0-9][0-9]).([0-9]|[0-9][0-9]|[0-9][0-9][0-9]).*’”

alias strip_comment=“sed ‘s/#.//'"
alias remove_empty_lines="sed -r '/^\s
$/d’”

alias excl_regex_rules=“grep -v ‘\$’”
alias incl_regex_rules=“grep ‘\$’”

real cmd rule: sed -e ‘s/\/\\/g’ -e ‘s/$//g’

alias conv_expr_2_rsc_fmt=“sed -e ‘s/\/\\\/g’ -e ‘s/\$//g’”

alias conv_2_alphanumeric=“tr -cd ‘[:alnum:].-\n-'"
alias conv_2_alphanumexpr="tr -cd '[:alnum:].+*[]
-\(|^$)\n-’”

\

Help func

prnt_col_2()
{
awk ‘{print $2}’
}

Convert plain name rules

cat ${_script_file}.tmp | remove_empty_lines | incl_ipaddr | prnt_col_2 | strip_comment | excl_regex_rules | conv_2_alphanumeric | sed -e ‘s/^//ip dns static add address=127.0.0.1 name=/i’ | sed -e ‘s/$/ ttl=600w/i’ >> “${_script_file}.tmp.tmp”

Convert regexp name rules

cat ${_script_file}.tmp | remove_empty_lines | incl_ipaddr | prnt_col_2 | strip_comment | incl_regex_rules | conv_2_alphanumexpr | conv_expr_2_rsc_fmt | sed -e ‘s/^//ip dns static add address=127.0.0.1 regexp="/i’ | sed -e ‘s/$/" ttl=600w/i’ >> “${_script_file}.tmp.tmp”

#####################

sync

Sort and remove doubles - keep unique lines

sort -u “${_script_file}.tmp.tmp” > “${_script_file}”

rm -f ${_script_file}.tmp.tmp 2>/dev/null 1>/dev/null
rm -f ${_script_file}.tmp 2>/dev/null 1>/dev/null

#wymaga doinstalowania sshpass
sshpass -p $sshpass scp “${_script_file}” $mtuser@$mtip:/
##############

It’s mostly based on someone else’s idea; (https://forum.qnap.net.pl/threads/pi-hole-–-blokowanie-reklam-ale-bez-pi-hole-za-to-w-samym-mikrotiku.27017/)

Now - just put the converter to cron (on pc or eg. home NAS) and add simple script oo mikrotik to refresh dns blacklist list

/ip dns static remove [find where ttl=600w]
:delay 50
/import file-name=blacklista.rsc;:log warning “import regulek PiHole”;
:delay 5
:log warning “Removed old AdsBlockList records and imported new list”;

May it serve you well!

.
If the Force might be with us :open_mouth:
.

@scob What device are you running this on? I get several errors running it from a couple of Debian based devices. Maybe it’s due to them being ARM based?

I wanted to add an update to my original post about exactly this.

I found that the cloudflared daemon just wasn’t very reliable, no matter what version I used. I had used unbound in the past at various workplaces, so I installed it and set it up to do DoT and removed cloudflared. I’m happy with the results. Like you I do see some initial queries are a bit slow, but not bad. A lot of queries are then answered out of cache, so overall there’s no performance problem.

A friend of mine uses Plume devices for his home network (he said that their Superpod uses the same IPQ-4019 and QCA9984 that the Mikrotik Audience uses, but the Wave2 features work) and he’s not happy with their ad-blocking functionality. He’s planning to add a Pi Hole device too. He isn’t concerned about DoH or DoT, so I told him a Raspberry Pi Zero W should be fine.

(I’ve been wondering whether it’s really worth doing DoH or DoT myself. I might be better off just having unbound do queries to the authoritative nameservers rather than forwarding to another, outside server like Quad9 or Cloudflare.)

it works. Excellent !@#$%

Thanks for linking to this. I’m trying to find the ideal pi-hole setup and this post & subreddit will definitely be helpful!

Convert Pihole address lists to static Mikrotik DNS records and enjoy. Manual update.

[quote=Karimzi007 post_id=859142 time=1621982011 user_id=187219]
You can block ads by mikrotik in several way. You can be use static dns, create firewall rules or ad block dns doh.

In this link you can find some appropriate methods



https://karimzi.blogspot.com/search/label/mikrotik?m=1



Follow this blog for advanced level of mikrotik security.
[/quote]


I'm currently using NextDNS via DoH from my Mikrotik router. It's working really well and I'm pleased with the results. I was never completely happy with pihole and had periodic issues with it, plus it was another thing and another system that I had to maintain. NextDNS is as easy to use as Quad9 was (which is what I was using with pihole). Performance is really good too.

download the file to desktop;
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

open notepad++ replace with regex;

find: ^#[^\n]*
replace: “” (empty)

find: ^![^\n]*
replace: “” (empty)

find: [1]
replace: “” (empty)

find: #[^\n]*
replace: “” (empty)

find: ^(0.[^ ]) ([^\n])
replace: add address=$1 name=$2

set mode to normal (not regex)
find: "=0.0.0.0 "
replace: "=127.0.0.1 "

upload the file to routerOS;

/ip dns
/import StevenBlack_hosts_20210808.rsc

TURKISH

download the file to desktop;
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

open notepad++ replace with regex;

find: ^(127.0.0.1) ([^\n]*)
replace: add address=$1 name=$2

remove res.cloudinary.com entry as it breaks www.dev.to

paste on terminal. because many lines conflict with generic list.

note: uses lots of memory and takes about 10 minutes on RB4011.

note: static entries are always in the cache. so I needed to set cache size to 307200 (300 mb)

note: I update static list monthly and when I receive complaint.


  1. ^0 ↩︎

It would be easier if MikroTik officially supported importing hosts file.

for example: IP->DNS->Filter Subscriptions

you enter URL(s] to download hosts files… and interval (for example: daily, weekly, monthly)

after that, they could add whitelisting feature.

but whatever. it is just a dream unless they want to gain market-share from Pi-Hole :wink:

I temporarily use it on mikrotik. Because I plan to buy a Raspberry Pi 400.

This script adds a big list of IP to block ads in a MikroTik router:
http://forum.mikrotik.com/t/modify-a-3rd-part-script/147186/1

Works OK but resource intensive.

You are wrong at some points.

Memory used 25% to 48% used, still have lots of memory free.
CPU does not see any difference. Lookup a name on an internal tabell may be faster and use less CPU compare to lookup at external DNS server.
Disk use: if you have space, it does not mater if its 80% or 90% full. OK at my RB750G r3

You should use a tool to monitor your ruter to see if anything run full.