Feature or Bugfix Request: ip firewall addess-list import shall not abort when dupe seen

Observed in beta8:
Currently when importing addresses into “/ip firewall address-list” the import aborts with an error message when it sees a dupe address that already is in the list.
This behavior of aborting the importing process is unnecessary, IMO even incorrect.
It rather shall simply skip the dupe, print a warning and continue importing the rest of the import-file.
This fix would be enormously helpful in maintaining big address lists.

:do {…} on-error={};

# The local variable "i" is a function and is called by $i. $1 inside the function contains the IP-address from that line.
:local i do={ :do {/ip firewall address-list add list=Probe-China timeout=35w3d13:13:56 address=$1} on-error={:log warning "IP already exists: $1"} }

# First clean out the old list. You also could just overwrite and have many many log entries and the old addresses are still present. So a total clean is the best.
:do { /ip firewall address-list remove [find where list=Probe-China] } on-error={}

#"17621","CNCGROUP-SH China Unicom Shanghai network, CN"
$i 220.196.0.0/17
$i 220.248.64.0/18
$i 112.64.0.0/17
$i 112.64.0.0/16
$i 112.65.0.0/17
$i 116.128.128.0/18
$i 58.247.0.0/16
$i 211.95.0.0/17
.
.
.

$i $i $i $i $i…señor.

Señor @msatter, yes, a very good solution. ¡muchas gracias! :slight_smile:

You are welcome and I is not only my work but work of many others to distill this to the most lean solution.

Mikrotik could take this and make it possible to enter the name of the list, the timeout and if you want to clean the old list first so that you can add a list to a already existing list. To make this adding less stating/logging of double addresses that could be disabled and maybe state then the total addresses in a list and how many are just added and how many were double.

double = total in file - (new total - old total)
added = new total - old total

Chupaka gave us the solution and BartoszP the on-error.
https://forum.mikrotik.com/viewtopic.php?f=9&t=98804&start=300#p606911

Used it day in day out now and it never failed me.

I made a adapted version to read an address list that was exported and contains disabled entries.

As you can see you only need to leave the “yes” and remove the labels put in in an export. The file size goes from exported plus 1MB to under 500KB.

Update: adapting to if a comment is provided (Now tested and working)

# Belgium / Netherlands / United Kingdom static allow list
# Format: IP-address comment disable=(yes/no (no is default))
# Adapts to any of the extra fields being omitted.
:local i   do={/
:if ($2 = "yes")  do={:set $3 $2; :set $2};/
:do {/ip firewall address-list add list=CountryIPBlockstest comment=$2 disable=$3 address=$1} on-error={:log warning "IP already exists: $1"}}
:do { /ip firewall address-list remove [find where list=CountryIPBlockstest] } on-error={}


$i 46.101.0.0/16 "NETHERLANDS - Digital Ocean" yes
$i 94.102.48.0/20 "NETHERLANDS - cloud.census.shodan.io" yes
$i 150.129.8.0/22 "NETHERLANDS - Liteserver Holding B.v 20200716" yes
$i 145.220.0.0/16 "NETHERLANDS - Surfnet" yes
$i 178.79.128.0/18 "UNITED KINGDOM - Linode" yes
$i 170.205.64.0/18 yes
$i 192.69.64.0/24 yes
$i 199.103.123.0/24
$i 199.103.124.0/22
$i 204.79.170.0/24 BELGIUM
$i 204.79.204.0/24 BELGIUM
$i 204.231.231.0/24 BELGIUM

A problem was that the boolean value “yes” was send to the function ($i) as a string and so I had to adopt the “if do”.