Community discussions

 
User avatar
vasilevkirill
Trainer
Trainer
Topic Author
Posts: 48
Joined: Tue May 22, 2012 7:38 am
Location: Russian, Saint-Petersburg
Contact:

[Feature request] Address List extension

Sat Apr 06, 2019 1:37 am

Hi,
I would be interested to see such functionality in the address lists.

Code: Select all

/ip firewall address-list add list=hosts address=192.168.0.0/16 wildcard=255.255.10-20.255
Only networks will be third octet is in the range of 10-20

Code: Select all

/ip firewall address-list add list=hosts address=192.168.0.0/16 wildcard=255.255.0.10
third octet 0-255 and fourth octet 10

Code: Select all

/ip firewall address-list add list=hosts address=192.168.0.0/16 wildcard=255.255.255.10,15,20,30-35
fourth octet 10 or 15 or 20 or in range 30-35

Code: Select all

/ip firewall address-list add list=GOOGLE address=AS15169
Dynamic address list, address from AS number

Code: Select all

[admin@MT-AP-KIRILL-ROOM] /ip firewall address-list> pr
Flags: X - disabled, D - dynamic
# LIST ADDRESS CREATION-TIME TIMEOUT
0 GOOGLE AS15169 apr/06/2019 01:07:35
1 D ;;; AS15169
GOOGLE 103.21.184.0/22 apr/06/2019 01:27:57
2 D ;;; AS15169
GOOGLE 103.227.68.0/22 apr/06/2019 01:27:57
3 D ;;; AS15169
GOOGLE 103.240.192.0/22 apr/06/2019 01:27:57
.......

Code: Select all

/ip firewall address-list add list=mysite.com address=*.mysite.com
wildcard or asterisk in all sub domain.

I know that you cannot get a list of sub domains, but if the router is a DNS server, it can find all such domains that are in the cache.
Vasilev Kirill
( MTCNA, MTCRE, MTCWE, MTCTCE, MTCUME, MTCINE, MTCIPv6E, MTCSE )
MikroTik Certified Trainer & Consultant
Trainer Сertificate Number TR0417
https://www.mikrotik.me
Cell:+7 (905) 207-35-78
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 1309
Joined: Sat Dec 24, 2016 11:17 am
Location: jo.overland at gmail.com

Re: [Feature request] Address List extension

Sat Apr 06, 2019 7:23 pm

I would like a command that list all the address list and if possible the number of entries in them.
 
How to use Splunk to monitor your MikroTik Router

MikroTik->Splunk
 
 
neutronlaser
Member Candidate
Member Candidate
Posts: 212
Joined: Thu Jan 18, 2018 5:18 pm

Re: [Feature request] Address List extension

Sun Apr 07, 2019 3:18 am

How could we use this:

whois -h whois.radb.net -- '-i origin AS15169' | grep ^route

Which gets every IP address range Google uses

Into a Mikrotik address list?
 
nostromog
Member Candidate
Member Candidate
Posts: 161
Joined: Wed Jul 18, 2018 3:39 pm

Re: [Feature request] Address List extension

Sun Apr 07, 2019 10:57 am

EDIT: Change sort to sort -u so that no full duplicates remain.
How could we use this:

whois -h whois.radb.net -- '-i origin AS15169' | grep ^route

Which gets every IP address range Google uses

Into a Mikrotik address list?
Those two give separate raw prefix lists, one for IPv4 and another for IPv6:
whois -h whois.radb.net -- '-i origin AS15169' | grep ^route: | awk '{print $2}'
whois -h whois.radb.net -- '-i origin AS15169' | grep ^route6: | awk '{print $2}'
Splitting into ipv4 and ipv6 is needed due to the strange way RouterOS deals with IPv6.

Now this will turn them into .rsc files ready to import:
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS15169' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=goog address=$prefix timeout=1d"; done; echo "}") > goog4.rsc
(echo "/ipv6 firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS15169' | grep ^route6: | awk '{print $2}'| sort -u); do echo "add list=goog address=$prefix timeout=1d"; done; echo "}") >goog6.rsc
$ ls -l goog*
-rw-r--r-- 1 user user  345227 apr  7 08:25 goog4.rsc
-rw-r--r-- 1 user user   11439 apr  7 08:24 goog6.rsc
$ wc -l goog*.rsc
  7146 goog4.rsc
   225 goog6.rsc
  7371 total
#...
$ scp goog6.rsc 192.168.88.1:
$ ssh 192.168.88.1 "/import goog6.rsc"
$ ssh 192.168.88.1 "/ipv6 firewall address-list print where list=goog"
Flags: X - disabled, D - dynamic 
 #   LIST                 ADDRESS                                              
 0 D goo                  2001:1900:2292::/48                                  
 1 D goo                  2001:4860::/32                                       
 2 D goo                  2401:fa00::/32                                       
 3 D goo                  2401:fa00::/42                                       
 4 D goo                  2401:fa00:4::/48                                     
 5 D goo                  2404:6800::/32                                       
 6 D goo                  2404:6800:4001::/48        
 (...)                          
Note that I only tested the ipv6 one, it is way smaller. Also, I put a timeout so that the lists will go on RAM.

Note also that the IPv6 list will need deduplication. It contains separate sub-prefixes of a given prefix as can be seen in my example. For instance it contains 2401:fa00::/32, 2401:fa00::/42 and 2401:fa00:4::/48, all included in the first.
Last edited by nostromog on Sun Apr 07, 2019 10:38 pm, edited 1 time in total.
 
neutronlaser
Member Candidate
Member Candidate
Posts: 212
Joined: Thu Jan 18, 2018 5:18 pm

Re: [Feature request] Address List extension

Sun Apr 07, 2019 7:10 pm

That's awesome. It is a good start to making a script that could for example let Google or Facebook in a Walled Garden list or perhaps QoS rule or blocking. I wish I knew how to deduplicate it.
It would be great as an online script generator. I tested it and it seemed an effective way to block Facebook in my firewall.
 
nostromog
Member Candidate
Member Candidate
Posts: 161
Joined: Wed Jul 18, 2018 3:39 pm

Re: [Feature request] Address List extension

Mon Apr 08, 2019 12:27 pm

I wish I knew how to deduplicate it.
When I tried ipv4 it was failing due to a duplicate, but changing sort -> sort -u makes it load. I edited the post. Removing entries that fall "inside"other entries, though, is a non-trivial programming problem.
 
neutronlaser
Member Candidate
Member Candidate
Posts: 212
Joined: Thu Jan 18, 2018 5:18 pm

Re: [Feature request] Address List extension

Mon Apr 08, 2019 2:34 pm

That's awesome. The extra efficiency of deduplication of overlapping ranges isn't important enough in my case.

Here is how to export Facebook to a single address list:
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS63293' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=facebook address=$prefix timeout=1d"; done; echo "}") > facebook.rsc
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS54115' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=facebook address=$prefix timeout=1d"; done; echo "}") > facebook.rsc
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS32934' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=facebook address=$prefix timeout=1d"; done; echo "}") > facebook.rsc
 
msatter
Forum Guru
Forum Guru
Posts: 1287
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: [Feature request] Address List extension

Mon Apr 08, 2019 2:43 pm

That's awesome. It is a good start to making a script that could for example let Google or Facebook in a Walled Garden list or perhaps QoS rule or blocking. I wish I knew how to deduplicate it.
It would be great as an online script generator. I tested it and it seemed an effective way to block Facebook in my firewall.
If you look on bgp.he.net then you see for Facebook huge IPv6 ranges that they own and so easily block-able:

2a03:2880::/32
2a03:2887:ff34::/48
2401:db00::/32
2620:0:1c00::/40
2803:6080::/32

https://bgp.he.net/search?search%5Bsear ... mit=Search
Two RB760iGS (hEX S) in series. One does PPPoE and both do IKEv2.
Running:
RouterOS 6.46Beta59 / Winbox 3.20 / MikroTik APP 1.3.7
Having an Android device, use https://github.com/M66B/NetGuard/releases (no root required)
 
neutronlaser
Member Candidate
Member Candidate
Posts: 212
Joined: Thu Jan 18, 2018 5:18 pm

Re: [Feature request] Address List extension

Mon Apr 08, 2019 2:54 pm

I don't use IPv6 but great information for those who do.
 
Simono
Frequent Visitor
Frequent Visitor
Posts: 50
Joined: Tue Mar 20, 2018 9:41 am

Re: [Feature request] Address List extension

Tue Apr 09, 2019 1:48 pm

Now I don't have Linux (HDD failure). How do this on router or Windows PC?

Sent from my phone by Tapatalk


 
neutronlaser
Member Candidate
Member Candidate
Posts: 212
Joined: Thu Jan 18, 2018 5:18 pm

Re: [Feature request] Address List extension

Tue Apr 09, 2019 2:03 pm

Cygwin with package for whois (not sure which one it is in, maybe bindutils)
 
Simono
Frequent Visitor
Frequent Visitor
Posts: 50
Joined: Tue Mar 20, 2018 9:41 am

Re: [Feature request] Address List extension

Tue Apr 09, 2019 2:17 pm

Thanks, I will try

Sent from my phone by Tapatalk

Who is online

Users browsing this forum: No registered users and 95 guests