Page 1 of 1
[Feature request] Address List extension
Posted: Sat Apr 06, 2019 1:37 am
by vasilevkirill
Hi,
I would be interested to see such functionality in the address lists.
Code: Select all
/ip firewall address-list add list=hosts address=192.168.0.0/16 wildcard=255.255.10-20.255
Only networks will be third octet is in the range of 10-20
Code: Select all
/ip firewall address-list add list=hosts address=192.168.0.0/16 wildcard=255.255.0.10
third octet 0-255 and fourth octet 10
Code: Select all
/ip firewall address-list add list=hosts address=192.168.0.0/16 wildcard=255.255.255.10,15,20,30-35
fourth octet 10 or 15 or 20 or in range 30-35
Code: Select all
/ip firewall address-list add list=GOOGLE address=AS15169
Dynamic address list, address from AS number
Code: Select all
[admin@MT-AP-KIRILL-ROOM] /ip firewall address-list> pr
Flags: X - disabled, D - dynamic
# LIST ADDRESS CREATION-TIME TIMEOUT
0 GOOGLE AS15169 apr/06/2019 01:07:35
1 D ;;; AS15169
GOOGLE 103.21.184.0/22 apr/06/2019 01:27:57
2 D ;;; AS15169
GOOGLE 103.227.68.0/22 apr/06/2019 01:27:57
3 D ;;; AS15169
GOOGLE 103.240.192.0/22 apr/06/2019 01:27:57
.......
Code: Select all
/ip firewall address-list add list=mysite.com address=*.mysite.com
wildcard or asterisk in all sub domain.
I know that you cannot get a list of sub domains, but if the router is a DNS server, it can find all such domains that are in the cache.
Re: [Feature request] Address List extension
Posted: Sat Apr 06, 2019 7:23 pm
by Jotne
I would like a command that list all the address list and if possible the number of entries in them.
Re: [Feature request] Address List extension
Posted: Sun Apr 07, 2019 3:18 am
by neutronlaser
How could we use this:
whois -h whois.radb.net -- '-i origin AS15169' | grep ^route
Which gets every IP address range Google uses
Into a Mikrotik address list?
Re: [Feature request] Address List extension
Posted: Sun Apr 07, 2019 10:57 am
by nostromog
EDIT: Change sort to sort -u so that no full duplicates remain.
How could we use this:
whois -h whois.radb.net -- '-i origin AS15169' | grep ^route
Which gets every IP address range Google uses
Into a Mikrotik address list?
Those two give separate raw prefix lists, one for IPv4 and another for IPv6:
whois -h whois.radb.net -- '-i origin AS15169' | grep ^route: | awk '{print $2}'
whois -h whois.radb.net -- '-i origin AS15169' | grep ^route6: | awk '{print $2}'
Splitting into ipv4 and ipv6 is needed due to the strange way RouterOS deals with IPv6.
Now this will turn them into .rsc files ready to import:
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS15169' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=goog address=$prefix timeout=1d"; done; echo "}") > goog4.rsc
(echo "/ipv6 firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS15169' | grep ^route6: | awk '{print $2}'| sort -u); do echo "add list=goog address=$prefix timeout=1d"; done; echo "}") >goog6.rsc
$ ls -l goog*
-rw-r--r-- 1 user user 345227 apr 7 08:25 goog4.rsc
-rw-r--r-- 1 user user 11439 apr 7 08:24 goog6.rsc
$ wc -l goog*.rsc
7146 goog4.rsc
225 goog6.rsc
7371 total
#...
$ scp goog6.rsc 192.168.88.1:
$ ssh 192.168.88.1 "/import goog6.rsc"
$ ssh 192.168.88.1 "/ipv6 firewall address-list print where list=goog"
Flags: X - disabled, D - dynamic
# LIST ADDRESS
0 D goo 2001:1900:2292::/48
1 D goo 2001:4860::/32
2 D goo 2401:fa00::/32
3 D goo 2401:fa00::/42
4 D goo 2401:fa00:4::/48
5 D goo 2404:6800::/32
6 D goo 2404:6800:4001::/48
(...)
Note that I only tested the ipv6 one, it is way smaller. Also, I put a timeout so that the lists will go on RAM.
Note also that the IPv6 list will need deduplication. It contains separate sub-prefixes of a given prefix as can be seen in my example. For instance it contains 2401:fa00::/32, 2401:fa00::/42 and 2401:fa00:4::/48, all included in the first.
Re: [Feature request] Address List extension
Posted: Sun Apr 07, 2019 7:10 pm
by neutronlaser
That's awesome. It is a good start to making a script that could for example let Google or Facebook in a Walled Garden list or perhaps QoS rule or blocking. I wish I knew how to deduplicate it.
It would be great as an online script generator. I tested it and it seemed an effective way to block Facebook in my firewall.
Re: [Feature request] Address List extension
Posted: Mon Apr 08, 2019 12:27 pm
by nostromog
I wish I knew how to deduplicate it.
When I tried ipv4 it was failing due to a duplicate, but changing sort -> sort -u makes it load. I edited the post. Removing entries that fall "inside"other entries, though, is a non-trivial programming problem.
Re: [Feature request] Address List extension
Posted: Mon Apr 08, 2019 2:34 pm
by neutronlaser
That's awesome. The extra efficiency of deduplication of overlapping ranges isn't important enough in my case.
Here is how to export Facebook to a single address list:
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS63293' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=facebook address=$prefix timeout=1d"; done; echo "}") > facebook.rsc
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS54115' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=facebook address=$prefix timeout=1d"; done; echo "}") > facebook.rsc
(echo "/ip firewall address-list {"; for prefix in $(whois -h whois.radb.net -- '-i origin AS32934' | grep ^route: | awk '{print $2}'| sort -u); do echo "add list=facebook address=$prefix timeout=1d"; done; echo "}") > facebook.rsc
Re: [Feature request] Address List extension
Posted: Mon Apr 08, 2019 2:43 pm
by msatter
That's awesome. It is a good start to making a script that could for example let Google or Facebook in a Walled Garden list or perhaps QoS rule or blocking. I wish I knew how to deduplicate it.
It would be great as an online script generator. I tested it and it seemed an effective way to block Facebook in my firewall.
If you look on bgp.he.net then you see for Facebook huge IPv6 ranges that they own and so easily block-able:
2a03:2880::/32
2a03:2887:ff34::/48
2401:db00::/32
2620:0:1c00::/40
2803:6080::/32
https://bgp.he.net/search?search%5Bsear ... mit=Search
Re: [Feature request] Address List extension
Posted: Mon Apr 08, 2019 2:54 pm
by neutronlaser
I don't use IPv6 but great information for those who do.
Re: [Feature request] Address List extension
Posted: Tue Apr 09, 2019 1:48 pm
by Simono
Now I don't have Linux (HDD failure). How do this on router or Windows PC?
Sent from my phone by Tapatalk
Re: [Feature request] Address List extension
Posted: Tue Apr 09, 2019 2:03 pm
by neutronlaser
Cygwin with package for whois (not sure which one it is in, maybe bindutils)
Re: [Feature request] Address List extension
Posted: Tue Apr 09, 2019 2:17 pm
by Simono
Thanks, I will try
Sent from my phone by Tapatalk