Address lists downloader (DShield, Spamhaus DROP/EDROP, etc)

[msatter]

Do not insert lists that are bigger than 63KiB, those would only will be loaded incomplete.

# Written by Shumkov
# Adapted by blacklister
# 20191108

/ip firewall address-list
:local update do={
 :do {
 :local result [/tool fetch url=$url as-value output=user]; :if ($result->"downloaded" != "63") do={ :local data ($result->"data")
  :do { remove [find list=$blacklist] } on-error={}
   :while ([:len $data]!=0) do={
      :if (([:pick $data 0 [:find $data "\n"]]~"^[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}") do={
      :do {add list=$blacklist address=([:pick $data 0 [:find $data $delimiter]].$cidr) timeout=7d} on-error={}
      }
   :set data [:pick $data ([:find $data "\n"]+1) [:len $data]]
   } ;  :log warning "Imported address list < $blacklist> from file: $url"
   } else={:log warning "Address list: <$blacklist>, downloaded file to big: $url" }
 } on-error={:log warning "Address list <$blacklist> update failed"}
}

$update url=https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset blacklist="firehole-1" delimiter=("\n") 
$update url=https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level2.netset blacklist="firehole-2" delimiter=("\n")

The first is loaded and the second is not because of the size being over 63KiB

I use separate blacklists and not one blacklist with different comments.

I gave this a shot - but it did not run. No message in the log and no address list.

Remove one of the “(” in the line beginning with

:if (([:pick