no such item (4) - FQDN address-list find where comment=X

I have script that is doing following thing:

:local addrlst [/ip firewall address-list find where comment=$d disabled=no]
# some more irrelevant code here
:local rrlst [/ip firewall address-list find where address=$d dynamic=no disabled=no]

And my mailbox get spammed with notifications about no such item (4) script error randomly. Every few minutes (during that time those instructions are called several hundreds or maybe even thousands times). I verified that script always crashes on one of those two lines.

I know this error is due to fact that ROS is “forgetting” some dynamiic entries during script execution but thiis is basically single, atomic instruction. How can I do some workaround? My goal is to get list of IPs that ROS resolved fqdn based address-lists to (script is basically flushing DNS cache and removing all dynamic entries forcing ROS to resolve address-lists again whenever there’s some synchronization issue between. Synchronization is checked by performing manual DNS resolution against specified server and comparison every 2 seconds)

This can be how ROS handles subroutines. Without showing the whole script its not easy to see what is wrong.

Add the whole script in { } and cut and past it to command line to run i directly.
Using :put everywhere to see output as script goes along.

Eks

{
local test "1234"
:put $test
}

I did that. I mean not exactly that but I put :log debug message=N where N was some number and I put it every line, then I browsed through syslog logs to see after which log script stops and throws “no such item (4)” and like I said it’s always one of those 2 lines.

Here’s full script (just without those debug logs to not spam):

[lapsio@ACSWAG] /system script> print without-paging  where name=dnsreload
Flags: I - invalid 
 0   name="dnsreload" owner="lapsio" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon 
     dont-require-permissions=no run-count=0 source=
       :log debug "script-enter: dnsreload"
       
       :local dynuniq [:toarray ""]
       :local baddns [:toarray ""]                                                                                  
       :local baddnsn [:toarray ""]                                                                                 
       :local sysfailure false
       
       :if ([:len $dynuniq] > 0 || [:len $baddns] > 0 || [:len $baddnsn] > 0) do={
           :set sysfailure true
           /log error message="SYSTEM MALFUNCTION DETECTED - ABORTING"
       }                                                                                                            
       
       :if ([:len [/system script job find where script=dnsreload]] = 1 && $sysfailure = false) do={
           :local dyn [/ip firewall address-list find where dynamic=yes disabled=no]                                
       
           :foreach counter=d in=$dyn do={
               :local lst [/ip firewall address-list get $d comment]                                                
               :local dup false
               :foreach dd in=$dynuniq do={
                   :if ($dd = $lst) do={                                                                            
                       :set dup true
                   }
               }                                                                                                    
               :if ($dup = false) do={                                                                              
                   :local dynlen [:len $dynuniq]                                                                    
                   set ($dynuniq->$dynlen) $lst
               }
           }                                                                                                        
           :local dnssrv [/ip dns get servers]                                                                      
           :do {                                                                                                    
               :put ("DNS condition test... ".$dnssrv." :: ".[:resolve google.com server=$dnssrv])
               :put "Passed"
               :foreach counter=d in=$dynuniq do={
                   :do {
                       :local caddr [:resolve $d server=$dnssrv]
                       :local addrlst [/ip firewall address-list find where comment=$d disabled=no]
                       :local found false
                       :foreach counter=dd in=$addrlst do={
                           :if ([/ip firewall address-list get $dd address] = $caddr) do={
                               :set found true
       #                        :put ("Match : ".$d." ".$caddr)
                           }
                       }
                       :local rrlst [/ip firewall address-list find where address=$d list=rrDNS dynamic=no disabled=no]
                       :if ([:len $rrlst] > 0) do={
                           :set found true
                           :put ("Ignoring mismatch on rrDNS domain: ".$d)
                           /log info message=("Ignoring mismatch on rrDNS domain: ".$d)
                       }
                       :if ($found = false && [:len $addrlst] > 0) do={
                           :put ("DNS mismatch : ".$d." ".$caddr)
                           /ip firewall address-list print where comment=$d
                           /log info message=("DNS mismatch on domain ".$d)
                           :local cbaddns [/ip firewall address-list find where address=$d dynamic=no disabled=no]
                           :foreach counter=cbd in=$cbaddns do={
                               :local badlen [:len $baddns]
                               :set ($baddns->$badlen) $cbd
                           }
                           :local badnlen [:len $baddnsn]
                           :set ($baddnsn->$badnlen) $d
                       } else={
                           :if ([:len $addrlst] = 0) do={
                               :put ("Cache miss on domain ".$d)
                               /log info message=("DNS cache not populated yet. Could not verify ".$d)
                           }
                       }
                   } on-error={
                       :if ($d != "") do={
                           :put ("Cannot resolve domain : ".$d)
                           :do {
                               :put ("Ensuring DNS condition... ".$dnssrv." :: ".[:resolve google.com server=$dnssrv])
                               :put "DNS functional. Domain still unreachable"
                               /log warning message=("DNS domain unreachable: '".$d."'")
                           } on-error={
                               :put "DNS unavailable"
                               /log error message="DNS server became unavailable"
                           }
                       }
                   }
               }
               :if ([:len $baddns] > 0) do={
                   :put ("Mismatched: ".$baddnsn)
                   /log info message=("Mismatched domains found: ".$baddnsn)
       #            /tool e-mail send to="lapsio3+log@gmail.com" subject="DNS CACHE STALE - FORCE FLUSH" body=("Mism tch on domain: '".$baddnsn."'.  Forcing cache flush...")
                   /ip firewall address-list disable $baddns
                   /ip dns cache flush
                   /ip firewall address-list enable $baddns
               } else={
                   :put "Clean exit"
                   /log info message="DNS reload clean exit"
               }
           } on-error={
               :put ("DNS check failed - aborting")
               /log error message="DNS server unreachable. Integrity check aborted"
           }
       }
       
       :log debug "script-exit: dnsreload"

It’s worth to note that on-error doesn’t catch “no such iitem (4)” error. I’ve noticed it multiple times because I started using try-catch for whole scripts and still I receive from time to time “no such item (4)” errors from some scripts.

Its som hard to debug such a big script.

If you wrap the script in {} and past it to the terminal, its easier to see where things go wrong.
Same with put instead of log. All on screen when things are running.

Also split the scripts up in parts and cut/past part by part to terminal.

What version is your RouterOS? It may be a bug and other version works better.
I do see other have had similar problems: http://forum.mikrotik.com/t/no-such-item-4-while-counting-connections/123561/1 http://forum.mikrotik.com/t/such-item-4/104084/1

It may be that the table are to large and some is change in beweeen when script are running so things get lost and gives error.
I do see that they suggest to use foreach

Your issue probably is that some of your DNS-derived address list items have really short TTL (like when you try to get addresses of CDN servers in an address list), and there is a reasonable chance that your find retrieves a list of items where some of them expire some milliseconds later and the next usage indicates that they are missing.
One would indeed expect that for a single command line this “cannot happen”, but on the other hand it is good that the router uses multitasking and address list expiry can happen simultaneously with script execution. You would not want to block the expiry during any execution of a script.

I agree with you that there should be some way to catch those errors, preferably just using on-error= or otherwise by setting some mode where such errors are just ignored for the script.

Yeah, I don’t want to just skip sending alerts about script errors because while this script is not really all that mission critical there are other script on router that monitor various aspects of network and they’re more stateful so error in them could leave script in “undefined behavior” state and may require manual cleanup. It doesn’t happen often but once every few weeks. Now this one script is spamming my mailbox with “no such item (4)” errors and I don’t know which errors are from this one and which are from more important scripts :confused:

I already filed “bug request” to mikrotik if they could maybe at least add in log indication which script crashed, and even better in which line - this way I could just set up filter in mailbox to ignore crashes of this one particular script that is not really all that critical.

I do agree that RouterOS is no good to debug. IF some stops, it should log a line telling where and what failed.
At least on-error should pick this up so that the sctipt does not stop.
I do see that this problem has been around for 10 years+ http://forum.mikrotik.com/t/error-no-such-item-4-when-remove-active-user-hotspot/23925/1

You need to make a script that test if some is there before trying to change/delete it.

With “I agree with you that there should be some way to catch those errors, preferably just using on-error= or otherwise by setting some mode where such errors are just ignored for the script.” of course I mean that this ignore action is only for the current script or current function in a script, not globally.
The on-error= clause is already local. It is unfortunate that this does not catch the “no such item” error, that is imho a bug that should be fixed.
There are other topics on the forum where people encounter unpredictable “no such item” errors (due to race conditions) and cannot handle them in a reasonable way.

W.r.t. printing the line number of an error: the issue is that scripts are “compiled” from the text that you type into a stack-based intermediate language which is then executed by the script engine. The engine sees only the compiled code, not the original lines. So when an error occurs, it is only known in what instruction it happens but not which line this came from.
Under certain conditions you can see this intermediate code in the environment variables, e.g. when you define a function and then print the environment. You will see that there is no line-number information in this intermediate code.