Address list import script - bug?

RouterOS General
1

I have some dynamic address lists used in a firewall drop rule across 3 different routers.
Seems that one of the lists picked up a 64.0.0.0/2 entry in error yesterday.

I have a powershell script running on a Windows server which downloads the lists hourly and formats into a RouterOS script having stripped out the RFC1918/multicast nets.

Example

$script= "C:\inetpub\wwwroot\<snipped>\firehol1.rsc"
"/ip firewall address-list remove [/ip firewall address-list find list=firehol_L1]" | Add-Content $script
$blacklist= Get-Content "C:\inetpub\wwwroot\<snipped>\firehol_L1.txt" | Where-Object {$_ -notmatch "^#|0.0.0.0|10.0.0.0|172.16.0.0/12|192.168.0.0|224.0.0.0"}
ForEach ($address in $blacklist){
"/ip firewall address-list add addres=$address list=firehol_L1 timeout=2d" | Add-Content $script
}

I then have a task which imports the script on the router every 3 hours. Seemingly yesterday, the 64.0.0.0/2 entry appeared, but this did not clear when the lists were updated automatically.

/tool fetch address=<snipped> host=<snipped> mode=https src-path=/<snipped>/firehol1.rsc keep-result=yes;
/import firehol1.rsc;

Additionally, one of the routers impacted was rebooted this morning prior to the entry being manually removed and it persisted after reboot.
As you can see, I add a timeout value to each entry, my understanding being that this should cause the entries to be held in memory rather than written to disk. As such, I’d expect these to be lost on reboot, but the 64.0.0.0/2 entry was not. I also manually checked the source file downloaded directly from the internet and this did not appear to contain the 64.0.0.0/2 entry.

Could this be a bug of some sort in RouterOS?

2

I’m not saying that ROS doesn’t contain a bug which add static entry to address list … but I find it highly improbable. It also seems improbable that ROS would ignore timeout property of certain netry (out of thousands).

However, I would find much more probable the possibility that the list you’re fetching and parsing contained some error which made your powershell script to create erroneous command file … and that the error was corrected at source soon after so you are not able to repeat the steps.

And then there are other possibilities (such as some HW error on your devices, but the fact that more than one misbehaved at the same time speaks against this idea; or some hacking activity in your networks; or …).

3

Most likely scenario is that for an entry added by someone/something, the subnet mask was specified as /2 instead of e.g. /24
RouterOS will automatically match the aaa.bbb.ccc.ddd/2 address to leave only the first two bits, which can be 64, instead of throwing an error when an address is specified with 1-bits outside the subnetmask (as it should do).
So that at least explains why you would get such a “strange” address.

4

I would generally agree, but all list entries including the problematic one are added with timeout=2d which should mean it’s kept in memory and therefore disappear on reboot, which this one didn’t.
Also, the original source list when checked an hour after initial issue arose no longer contained the problematic entry and script re-runs every 3 hours with first action to delete all existing entries from the address list, so can’t see why this router still had the entry this morning.

5

I’d agree that an error in the netmask seems likely, but also a bit odd that the network address was a neat 64.0.0.0. Definitely likely an issue with source data that initiated the problem, but odd that it persisted once cleared at source.

6

No, that is what I tried to explain to you, that is not odd.
When you add e.g. 80.81.82.83/2 to a RouterOS address list, it will add 64.0.0.0/2 instead.
So when a.g. an address ending in /24 (or another /2x) is mistakenly truncated, this is what will be the result.

7

Ah yes, understand now. :smiley:

8

I would bet that you have a bug in the powershell script. Parse firehol1.rsc and look for a syntax error in the ip formation.

If you say that after restarting the ip still appears I also have doubts that you are running well the /ip firewall address-list remove “/ip firewall address-list find list=firehol_L1]”.
I would go by parts, open firehol1.rsc and delete all lines except the first/first (/ip firewall address-list remove [/ip firewall address-list find list=firehol_L1]) and try to do an /import. See if you really see that it is completely deleting the list.

I have a similar script in bash, but I run the remove list directly in the mikrotik script (above the import), I also usually put a :delay between functions.

9

I think the original error was in the 3rd party address list I’m importing, per earlier replies.

The initial delete is definitely working, as I’ve watched the contents of the address list whilst the script is running.

10

Well, you could do an extra check before accepting an entry.
When the entry has bits in the subnet part of the address, it is bad and should be discarded.
E.g. when the entry is 80.81.82.83/24 it is bad. When it is 80.81.82.0/24 it is good. But when it is 80.81.82.0/2 it is bad.
It would also be worthwile to check the number after / and e.g. accept only values of 10 and above.
You rarely want to block an entire /8 or even /2.

11

Yes that’s what I’m planning to do when I get some time.

12

As already wroted on old post:

  1. for security accept only from /12 to /32 prefix. /11 or less on IPv4 is too much big for be true…

Some procedures for check the subnet mask:
http://forum.mikrotik.com/t/rextended-fragments-of-snippets/151033/1
{
:local source 10.31.42.56/16
:if ([:len [:find $source “/”]] = 0) do={:set source “$source/32”}
:local ip [:toip [:pick $source 0 [:find $source “/”]]]
:local prefix [:tonum [:pick $source ([:find $source “/”] + 1) [:len $source]]]
:local submask (255.255.255.255<<(32 - $prefix))
:local network ($ip & $submask)
:put “$source = $network/$prefix”
}

{
:local source 10.31.42.56
:if ([:len [:find $source “/”]] = 0) do={:set source “$source/32”}
:local ip [:toip [:pick $source 0 [:find $source “/”]]]
:local prefix [:tonum [:pick $source ([:find $source “/”] + 1) [:len $source]]]
:local submask (255.255.255.255<<(32 - $prefix))
:local network ($ip & $submask)
:put “$source = $network/$prefix”
}