How to Read line by line from a file using a script?

RouterOS Scripting
21

Thanks ! It solves the problem of “deserialization”. :slight_smile:

22

I can’t get it to run. Will try another approach.

23

Hi,

a little of self-promotion in 2026 :}, check this Blacklist Filter update script - #385 by BartoszP

1 Like
24

The example you posted worked for me for putting out the IPs (haven't tried the firewall command). There could be something with the text file, maybe some extra character?

25

I think it is similar to the approach I was trying to take, am I right ? I want to make ever IP line of the blacklist a command to append the firewall list.

26

I am not quite sure. Nothing was inserted in the table. It might be a bit technical for me but I suppose that the addresses at the junction of two chunks must be parted. May it cause the rejection of the entire script ? I don’t know.

27

When each line is separate command with proper on-error handling then only a particular command fails and the import go on.
As your list is quite long, then the whole import script would be quite huge. Please read more of pointed topic starting there: Blacklist Filter update script - #397 by BartoszP how to make it smaller.

1 Like
28

I’d like your point of view about this list, before I take it into first production.

/ip/firewall/address-list/remove [ find where list=Blacklist ];
:global BlacklistAddIP;
:set BlacklistAddIP do={
    :do { /ip/firewall/address-list/add list=Blacklist address=$1 timeout=$2; } on-error={ }
}
BlacklistAddIP "100.0.71.9" "25h"
BlacklistAddIP "100.10.72.114" "25h"

… (184124 lines)

BlacklistAddIP "99.92.60.133" "25h"
BlacklistAddIP "99.93.57.133" "25h"
BlacklistAddIP "99.99.12.59" "25h"
29

I would change the "25h" direct value to a variable to have one line to change instead of thousends, if you decide to change it.

1 Like
30 
/ip/firewall/address-list/remove [ find where list=Blacklist ];
:global BlacklistAddIP;
:set BlacklistAddIP do={
    :do { /ip/firewall/address-list/add list=Blacklist address=$1 timeout="25h"; } on-error={ }
}
BlacklistAddIP "100.0.71.9"
BlacklistAddIP "100.10.72.114"
31

Nice, but ... :slight_smile:

  • shorten the script using shorter name for BlacklistAddIP as you never know how much free space is on the device, so saving space could prevent troubles. It also makes interpretation faster as shorten name has to be parsed
  • change the name of a list to something else as BGP generates biuld-in Blacklist named list, so it would be easier to distinguish them
1 Like
32

List integrated. Thanks a lot ! I’ll publish the shell script when I’m done.

33

So, as promised, here is my script. It is a shell script as you notice. I am more comfortable with shell. So in order to have it run you need a Linux device.

There are some prerequisites

  • an existing blacklist (here blacklist.txt), made of individual IPs or IP ranges, and that already excludes all internal addresses in order to avoid locking yourself out of your own network (I can publish a script to build such a list if required)
  • the list is built in one column of IP addresses or ranges and nothing else (separation \n),
  • a password-free (based on keys) write-access from the Linux server to the MikroTik (that might not be the best security-wise but it makes things much easier).
#!/bin/sh

info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; }

# initiate logfile
LOG_FILE="~/firewall/blocklists-router.log"
exec 1>"$LOG_FILE" 2>&1
echo $(date)

# source file
SRC_FILE="~/firewall/blacklist.txt"

# destination file
DEST_FILE="~/firewall/blacklist-router.rsc"


info "Initialisation du script"

echo '/ip/firewall/address-list/remove [ find where list=Black ];\n:global Add;\n:set Add do={\n    :do { /ip/firewall/address-list/add list=Black address=$1 timeout="25h"; } on-error={ }\n}' > "$DEST_FILE"

info "Insertion des adresses IP"

cat "$SRC_FILE" | while read IP ; do
	echo "\$Add \"$IP\"" >> "$DEST_FILE"
done

info "Insertion terminée"

info "Dépôt sur le routeur"

scp -P 2222 $DEST_FILE admin@192.168.1.1:/

info "Dépôt effectué"

info "Lancement de l'intégration de la liste noire"

ssh -p 2222 admin@192.168.1.1 "/import file-name=blacklist-router.rsc"

info "Intégration de la liste noire terminée"

The logging is in French so you can discover that magnificent language :wink:

34

Beau travail
It is said that French is a language of love, love to Mikrotik :rofl: :rofl: :rofl:

1 Like
35

Does File length (aka number of lines) matter? Because I have 2 files, one with few lines, and one with many lines. The files are the same except for length. The long file fails with JSON error, the short file has no issues.