Steps to setup Mikrotik Router for content filtering. I should mention this is on a quad core pc with 4GB of Ram and 500GB of hard drive space running routeros v4.11 x86 Level 6 license as a central router.

I did many searches on this topic and found no clear example of how to do this except people suggesting using squid proxy or other hardware. This still a work in progess, and I welcome thoughts and opinions on how to improve it.

Problem : Have client that want access to porn and some that want it filtered.

Step 1.) Setup router to let everyone on the internet, including all QoS, firewall filtering, and Userman Test as in other tutorials.

Step 2.) In Userman Test, under profiles, limitations, under constraints set address list to "kid_friendly" on the profile used by client wanting content filtered. You may have to setup two different profiles and limitations. One profile and limitation with rate limits and address list set to something like "normal" and another profile and limitation with the address llist set to "kid_friendly". Userman Test is very picky and all profiles might have to be removed, reboot router, then added back for the list to populate correctly. Address list associations are somehow cached and do not take affect until router is rebooted.

Step 3.) Add these rules to the firewall filter:
;;; block adult www
chain=forward action=add-dst-to-address-list dst-address=!10.0.0.0/8 address-list=adult_block
address-list-timeout=0s layer7-protocol=kid_friendly-www
;;; block adult www
chain=forward action=drop dst-address=!10.0.0.0/8 src-address-list=kid_friendly layer7-protocol=kid_friendly-www

Step 4.)Now add this rule to the firewall NAT:
;;; Kid Friendly Web Proxy - BAD Sites
chain=dstnat action=redirect to-ports=8080 protocol=tcp src-address-list=kid_friendly dst-address-list=adult_block
dst-port=80

Step 5.) Now add words to the Layer 7 - Protocol to catch and filter:
4 kid_friendly - WWW ^.*(get|GET).+(choice|words|to|filter).*$

Step 6.) Enable and configure web proxy as suggested in other tutorials. Then add under access add these two rules.
;;; Not for kids URL (Enter word with vertical bar)
Dst. Host : (choice|words|to|filter) deny
;;; Block All
Dst. Host : deny



This may be the hard way of doing it but so far it is working. There are some false positives, in which the dst host address can manually be added and placed before the web proxy rules denying access. In this setup, only clients wanting to be filtered will be filtered, and then only hit the web proxy if a webpage is requested from the adult_blocked list. It also allows other clients that does not wish to be filtered to help populate the block list. I hope some of the experts will chime in with some advice. Hope this helps someone else.

Still fine tuning results but here is the manual address list to help get other people started:

/ip proxy
set always-from-cache=no cache-administrator=admin@multi-path.net \
cache-hit-dscp=4 cache-on-disk=yes enabled=yes max-cache-size=83900000KiB \
max-client-connections=600 max-fresh-time=3d max-server-connections=600 \
parent-proxy=0.0.0.0 parent-proxy-port=0 port=8080 serialize-connections=\
no src-address=0.0.0.0
/ip proxy access
add action=allow comment="Allow Google" disabled=no dst-host=*.google.com \
redirect-to="www.google.com/&safe=strict"
add action=allow comment="Allow Google pic" disabled=no dst-host=\
*.gstatic.com
add action=allow comment="Allow Yahoo" disabled=no dst-host=*.yahoo.com
add action=allow comment="Allow Yahoo +" disabled=no dst-host=*.yimg.com
add action=allow comment="Allow Dogpile" disabled=no dst-host=*.dogpile.com
add action=allow comment="Allow Dogpile" disabled=no dst-host=*.edgesuite.net
add action=allow comment="Allow Bing" disabled=no dst-host=*.bing.com
add action=allow comment="Allow Ask" disabled=no dst-host=*.ask.com
add action=allow comment="Allow Ask Formatting" disabled=no dst-host=\
*.2mdn.net
add action=allow comment="Allow Ebay" disabled=no dst-host=*.ebay.*
add action=allow comment="Allow Ebay pics" disabled=no dst-host=\
*.ebaystatic.*
add action=allow comment="Allow Wikipedia" disabled=no dst-host=*.wikipedia.*
add action=allow comment="Allow wikimedia" disabled=no dst-host=*.wikimedia.*
add action=allow comment="Allow Mikrotik" disabled=no dst-host=*.mikroik.*
add action=allow comment="Additional Mikrotik" disabled=no dst-host=\
*.routerboard.*
add action=allow comment="Allow NBC13 pics" disabled=no dst-host=\
*.247realmedia.com
add action=allow comment="Allow NBC13 pics 2" disabled=no dst-host=\
*.dukecms.com
add action=allow comment="Allow NBC13" disabled=no dst-host=*.nbc13.com
add action=allow comment="Allow ABC3340" disabled=no dst-host=*.abc3340.*
add action=allow comment="Allow Sears" disabled=no dst-host=*.sears.*
add action=allow comment="Allow Lowes" disabled=no dst-host=*.lowes.*
add action=allow comment="Allow Newegg" disabled=no dst-host=*.newegg.*
add action=allow comment="Allow howtoforge" disabled=no dst-host=\
*.howtoforge.*
add action=allow comment="Allow Facebook Pics" disabled=no dst-host=\
*.fbcdn.net path=""
add action=allow comment="Allow Facebook" disabled=no dst-host=*.facebook.com \
path=""
add action=allow comment="Allow MySpace" disabled=no dst-host=*.myspace.com
add action=allow comment="Allow MySpace Formatting" disabled=no dst-host=\
*.myspacecdn.com
add action=allow comment="Allow MySpace Formatting" disabled=no dst-host=\
*.fimservecdn.com
add action=allow comment="Allow MySpace Formatting" disabled=no dst-host=\
*.unicast.com
add action=allow comment="Allow usatoday" disabled=no dst-host=*.usatoday.*
add action=allow comment="Allow Hulu" disabled=no dst-host=*.hulu.com
add action=allow comment="Allow Hulu formatting" disabled=no dst-host=\
*.huluim.com
add action=allow comment="Allow YouTube " disabled=no dst-host=*.youtube.com
add action=allow comment="Allow YouTube formatting" disabled=no dst-host=\
*.ytimg.com
add action=allow comment="Allow Microsoft" disabled=no dst-host=\
*.microsoft.com
add action=allow comment="Allow Amazon" disabled=no dst-host=*.amazon.com
add action=allow comment="Allow Amazon formatting" disabled=no dst-host=\
*.images-amazon.com
add action=allow comment="Allow *.adobe.com" disabled=no dst-host=*.adobe.com
add action=allow comment="Allow AOL" disabled=no dst-host=*.aol.com
add action=allow comment="Allow AOL formatting" disabled=no dst-host=\
*.aolcdn.com
add action=allow comment="Allow Live" disabled=no dst-host=*.live.com
add action=allow comment="Allow PBS" disabled=no dst-host=*.pbs.*
add action=allow comment="Allow PBS Kids" disabled=no dst-host=pbskids.org
add action=allow comment="Allow PBS Kids" disabled=no dst-host=\
www-tc.pbskids.org
add action=allow comment="Allow PBS Kids" disabled=no dst-host=*.pbskids.org
add action=allow comment="Allow PBS Kids formatting" disabled=no dst-host=\
*.pbskidsplay.*
add action=allow comment="Allow Nickelodeon" disabled=no dst-host=*.nick.*
add action=allow comment="Allow Nickelodeon formatting" disabled=no dst-host=\
*.mtvnimages.com
add action=allow comment="Allow Nick Jr" disabled=no dst-host=*.nickjr.com
add action=allow comment="Allow Nick Jr formatting" disabled=no dst-host=\
*.atdmt.com
add action=allow comment="Allow Nick At Nite" disabled=no dst-host=\
*.nickatnite.com
add action=allow comment="Allow teennick" disabled=no dst-host=*.teennick.com
add action=allow comment="Allow icarly" disabled=no dst-host=*.icarly.com
add action=allow comment="Allow Spongebob" disabled=no dst-host=\
*.spongebob.com
add action=allow comment="Allow Neopets" disabled=no dst-host=*.neopets.com
add action=allow comment="Allow petpetpark" disabled=no dst-host=\
*.petpetpark.com
add action=allow comment="Allow Parentsconnect" disabled=no dst-host=\
*.parentsconnect.com
add action=allow comment="Allow Parentsconnect formatting" disabled=no \
dst-host=*.mtvnservices.com
add action=allow comment="Allow theslap" disabled=no dst-host=*.theslap.com
add action=allow comment="Allow Upickdaily" disabled=no dst-host=\
*.upickdaily.com
add action=allow comment="Allow Shockwave" disabled=no dst-host=\
*.shockwave.com
add action=allow comment="Allow troopgrid" disabled=no dst-host=\
*.troopgrid.com
add action=allow comment="Allow disney" disabled=no dst-host=*.go.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.doubleclick.net
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.dolimg.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.llnwd.net
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.starwave.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.disney.starwave.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
adsatt.disney.starwave.com
add action=allow comment="Allow Cartoon Network" disabled=no dst-host=\
*.cartoonnetwork.com
add action=allow comment="Allow Turner" disabled=no dst-host=*.turner.com

All of these have to be added before any denied rules. I could not post the denied rules. If I did I would have been banned, I am filtering on some pretty nasty words. As you can see some websites have multiple entries to keep the formatting. I am pretty amazed at how many websites have the same ip address. Guess everyone but the small websites use datacenters for hosting. Feel free to fine tune these rules to suit your needs.

Just discovered a flaw in the Userman Test Database. All was working fine for a day. I had two scripts scheduled one to reboot the router every 24 hours and one script thirty minutes after reboot to rebuild the Userman Database. Reboot of the router seems to not have affected the address list, but a rebuild of the database corrupts the address list associations making every profile have the address list kid friendly even though the userman web gui shows otherwise. I had to assign a new address list kid_ok to every profile with the old kid_friendly address list and reboot the router. This seems to reassociate the address lists back correctly. For now going to disable the script rebuilding the database until a better solution is found.

On side note, search engines and news websites seem to give the false positives. Working on enforcing VSS (very strict searching) with google, yahoo, youtube, bing, and ask.

Ended up creating three differen levels of filtering. Adult, teen, and kid. Adult - no filtering, teen - filtering with OpenDNS, and Kid - filtering almost everything and redirecting Search Engines to kid friendly search engines.

And

Ip proxy cache ????

What should I write to better work?

Excuse my English, I am Brazilian.

Still fine tuning results but here is the manual address list to help get other people started:

/ip proxy
set always-from-cache=no cache-administrator=admin@multi-path.net \
cache-hit-dscp=4 cache-on-disk=yes enabled=yes max-cache-size=83900000KiB \
max-client-connections=600 max-fresh-time=3d max-server-connections=600 \
parent-proxy=0.0.0.0 parent-proxy-port=0 port=8080 serialize-connections=\
no src-address=0.0.0.0
/ip proxy access
add action=allow comment="Allow Google" disabled=no dst-host=*.google.com \
redirect-to="www.google.com/&safe=strict"
add action=allow comment="Allow Google pic" disabled=no dst-host=\
*.gstatic.com
add action=allow comment="Allow Yahoo" disabled=no dst-host=*.yahoo.com
add action=allow comment="Allow Yahoo +" disabled=no dst-host=*.yimg.com
add action=allow comment="Allow Dogpile" disabled=no dst-host=*.dogpile.com
add action=allow comment="Allow Dogpile" disabled=no dst-host=*.edgesuite.net
add action=allow comment="Allow Bing" disabled=no dst-host=*.bing.com
add action=allow comment="Allow Ask" disabled=no dst-host=*.ask.com
add action=allow comment="Allow Ask Formatting" disabled=no dst-host=\
*.2mdn.net
add action=allow comment="Allow Ebay" disabled=no dst-host=*.ebay.*
add action=allow comment="Allow Ebay pics" disabled=no dst-host=\
*.ebaystatic.*
add action=allow comment="Allow Wikipedia" disabled=no dst-host=*.wikipedia.*
add action=allow comment="Allow wikimedia" disabled=no dst-host=*.wikimedia.*
add action=allow comment="Allow Mikrotik" disabled=no dst-host=*.mikroik.*
add action=allow comment="Additional Mikrotik" disabled=no dst-host=\
*.routerboard.*
add action=allow comment="Allow NBC13 pics" disabled=no dst-host=\
*.247realmedia.com
add action=allow comment="Allow NBC13 pics 2" disabled=no dst-host=\
*.dukecms.com
add action=allow comment="Allow NBC13" disabled=no dst-host=*.nbc13.com
add action=allow comment="Allow ABC3340" disabled=no dst-host=*.abc3340.*
add action=allow comment="Allow Sears" disabled=no dst-host=*.sears.*
add action=allow comment="Allow Lowes" disabled=no dst-host=*.lowes.*
add action=allow comment="Allow Newegg" disabled=no dst-host=*.newegg.*
add action=allow comment="Allow howtoforge" disabled=no dst-host=\
*.howtoforge.*
add action=allow comment="Allow Facebook Pics" disabled=no dst-host=\
*.fbcdn.net path=""
add action=allow comment="Allow Facebook" disabled=no dst-host=*.facebook.com \
path=""
add action=allow comment="Allow MySpace" disabled=no dst-host=*.myspace.com
add action=allow comment="Allow MySpace Formatting" disabled=no dst-host=\
*.myspacecdn.com
add action=allow comment="Allow MySpace Formatting" disabled=no dst-host=\
*.fimservecdn.com
add action=allow comment="Allow MySpace Formatting" disabled=no dst-host=\
*.unicast.com
add action=allow comment="Allow usatoday" disabled=no dst-host=*.usatoday.*
add action=allow comment="Allow Hulu" disabled=no dst-host=*.hulu.com
add action=allow comment="Allow Hulu formatting" disabled=no dst-host=\
*.huluim.com
add action=allow comment="Allow YouTube " disabled=no dst-host=*.youtube.com
add action=allow comment="Allow YouTube formatting" disabled=no dst-host=\
*.ytimg.com
add action=allow comment="Allow Microsoft" disabled=no dst-host=\
*.microsoft.com
add action=allow comment="Allow Amazon" disabled=no dst-host=*.amazon.com
add action=allow comment="Allow Amazon formatting" disabled=no dst-host=\
*.images-amazon.com
add action=allow comment="Allow *.adobe.com" disabled=no dst-host=*.adobe.com
add action=allow comment="Allow AOL" disabled=no dst-host=*.aol.com
add action=allow comment="Allow AOL formatting" disabled=no dst-host=\
*.aolcdn.com
add action=allow comment="Allow Live" disabled=no dst-host=*.live.com
add action=allow comment="Allow PBS" disabled=no dst-host=*.pbs.*
add action=allow comment="Allow PBS Kids" disabled=no dst-host=pbskids.org
add action=allow comment="Allow PBS Kids" disabled=no dst-host=\
www-tc.pbskids.org
add action=allow comment="Allow PBS Kids" disabled=no dst-host=*.pbskids.org
add action=allow comment="Allow PBS Kids formatting" disabled=no dst-host=\
*.pbskidsplay.*
add action=allow comment="Allow Nickelodeon" disabled=no dst-host=*.nick.*
add action=allow comment="Allow Nickelodeon formatting" disabled=no dst-host=\
*.mtvnimages.com
add action=allow comment="Allow Nick Jr" disabled=no dst-host=*.nickjr.com
add action=allow comment="Allow Nick Jr formatting" disabled=no dst-host=\
*.atdmt.com
add action=allow comment="Allow Nick At Nite" disabled=no dst-host=\
*.nickatnite.com
add action=allow comment="Allow teennick" disabled=no dst-host=*.teennick.com
add action=allow comment="Allow icarly" disabled=no dst-host=*.icarly.com
add action=allow comment="Allow Spongebob" disabled=no dst-host=\
*.spongebob.com
add action=allow comment="Allow Neopets" disabled=no dst-host=*.neopets.com
add action=allow comment="Allow petpetpark" disabled=no dst-host=\
*.petpetpark.com
add action=allow comment="Allow Parentsconnect" disabled=no dst-host=\
*.parentsconnect.com
add action=allow comment="Allow Parentsconnect formatting" disabled=no \
dst-host=*.mtvnservices.com
add action=allow comment="Allow theslap" disabled=no dst-host=*.theslap.com
add action=allow comment="Allow Upickdaily" disabled=no dst-host=\
*.upickdaily.com
add action=allow comment="Allow Shockwave" disabled=no dst-host=\
*.shockwave.com
add action=allow comment="Allow troopgrid" disabled=no dst-host=\
*.troopgrid.com
add action=allow comment="Allow disney" disabled=no dst-host=*.go.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.doubleclick.net
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.dolimg.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.llnwd.net
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.starwave.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
*.disney.starwave.com
add action=allow comment="Allow disney formatting" disabled=no dst-host=\
adsatt.disney.starwave.com
add action=allow comment="Allow Cartoon Network" disabled=no dst-host=\
*.cartoonnetwork.com
add action=allow comment="Allow Turner" disabled=no dst-host=*.turner.com

All of these have to be added before any denied rules. I could not post the denied rules. If I did I would have been banned, I am filtering on some pretty nasty words. As you can see some websites have multiple entries to keep the formatting. I am pretty amazed at how many websites have the same ip address. Guess everyone but the small websites use datacenters for hosting. Feel free to fine tune these rules to suit your needs.

In winbox, under web Proxy - > Cache tab...I just left it blank as the web proxy is not really a web proxy for caching websites in this setup. It is a filter web proxy, basically helping to filter out bad web sites.

Woow that`s great topic, what I`m searching for. I`m searching content and URL filter with "AdminBlockedPage". I tried using by ip firewall filter/ content filtering function. It was working nice but my customer couldn`t know, that page is blocked or it is problem of network connection.

Now I`m trying with proxy option and still can`t solve my problem. For example adult requests redirect to proxy and the proxy access rule DENY is catching those connections (hit count increases) but doesn`t block and no error page, just customers successful connect to adult pages. That`s very very strange... I have tried to change address-list-timeout, but still not blocking. Some times it works, maybe blocking 1 connection of 15 requests.
Multipath, what can you advise me in this case? Ofcourse now I can copy your settings and use it. But I think if use temporary address-list, it doesn`t give the false positives. Sorry for my bad english. Please Multi advise me.

you can also simply use http://www.opendns.com/ as your DNS server, and turn on adult-filtering in there. They have made pretty good lists that you won't have to maintain. and RouterOS configuration is simple - just add them as your DNS servers, and make DST-NAT redirect to intercept all DNS requests.

Thank you Normis.

my company does this with several customers of ours throughout the state. the web url proxy list works great most of the time. however i am running into issues where the list will have a url (*.taxwise.com) for example set to be allowed and yet it still gets blocked and does not show a count as an attempt to get there nor does the blocked list count go up either. my deny rule is to deny all websites except the ones in the list that are set to allow. if anyone would like to see my rules let me know and i will be happy to post them.

you can also simply use http://www.opendns.com/ as your DNS server, and turn on adult-filtering in there. They have made pretty good lists that you won't have to maintain. and RouterOS configuration is simple - just add them as your DNS servers, and make DST-NAT redirect to intercept all DNS requests.

Thanks! I was looking for such solution for a long time! Awesome!

The problem with Open DNS is that there's no google image filtering. You can see anything and everything with Open DNS active. It drives me nuts that they claim to be able to filter porn on their site, it's absurd.
Only way to block undesirable images is to block google entirely.

To block google images, you can force Safesearch by adding a static DNS entry for Google - whilst not 100% foolproof , it does block pretty much everything required.

I cant quite recall the IP address at the moment, but its something like :
Obviously you would need to add the same for each Google domain you want to intercept.

Hello, I'm new to the forum, friends. First of all I apologize for my English.
I am having difficulties related to this topic.
Would there be the possibility of having "domains.txt" files linked to the access lists?
Example: Is there correct syntax?

you can also simply use http://www.opendns.com/ as your DNS server, and turn on adult-filtering in there. They have made pretty good lists that you won't have to maintain. and RouterOS configuration is simple - just add them as your DNS servers, and make DST-NAT redirect to intercept all DNS requests.

Good evs sir,

I am very nood in mikrotik, please share your video on how to do this. thank you

Is there a way to block all websites except whatsapp and facebook? Any help would be greatly appreciated!