[SCRIPTING] - Anomaly in using global "keyed" arrays

RouterOS Scripting
1

While writing a small script to be scheduled at regular interval I discovered an anomaly in using global “keyed” (i.e. associative) arrays: according to the available documentation, entire array variables can be assigned with a single set command:

:global MyArray
:set MyArray {a=1; b=2; c=3}

Well, such kind of assignment sometime fails without apparent reason.

Here is a simple script that shows the problem:

:global Y

:if ([:typeof $Y] != "array") do={
  :set Y {a=0; b=0; c=0}
  :log info ("TEST-SCRIPT: c=$($Y->"c"); global data initialized as $[:typeof $Y]")
} else={
  :local count (($Y->"c") + 1)
  :set ($Y->"c") $count

  :if ($count = 1) do={
    :set ($Y->"a") 10
    :set ($Y->"b") 11
    :log info "TEST-SCRIPT: c=$count; global array assigned"
  } else={
    :if ($count >= 3) do={
      :set Y {a=0; b=0; c=0}
      :if (($Y->"c") = 0) do={
        :log info "TEST-SCRIPT: c=$count; global array reset OK"
      } else={
        :log info "TEST-SCRIPT: c=$count; global array reset FAILED"
      }
    } else={
      :log info "TEST-SCRIPT: c=$count; nothing to do"
    }
  }
}

The above code simulates a script scheduled at regular intervals which uses a global array to store status informations to be passed between different runs. It works as follows:

  1. each time the script is started it declares the global variable Y
  2. on the first time the variable was just declared but has no type/value yet, so it is initialized as a “keyed array” with three elements a,b,c (all set to 0); elements “a” and “b” are dummy and are just used to build a multi-element array, while the “c” element will be used as counter and incremented on each subsequent run (1,2,3…)
  3. on second run (c=1) the elements “a” and “b” are assigned some dummy values (don’t care)
  4. on third run (c=2) nothing happens (only counter element incremented)
  5. on fourth and, if the problem arises, subsequent runs (c>=3) the array is reassigned to initial value (all three elements set to 0 in a single command), so on the next run the sequence should restart incrementing c from 0 to 1 (i.e. as in second run above)

Each specific action done by the script is logged, so to be analyzed by looking at the log file.

The problem is that sometimes reassigning the array in step 5 above (i.e. when c=3) fails, so the sequence does not reset and continues with c>3. Here is a partial logfile showing the malfunction:

12:59:21 script,info TEST-SCRIPT: c=0; global data initialized as array 
12:59:38 script,info TEST-SCRIPT: c=1; global array assigned 
12:59:46 script,info TEST-SCRIPT: c=2; nothing to do 
13:00:01 script,info TEST-SCRIPT: c=3; global array reset OK 
13:00:08 script,info TEST-SCRIPT: c=1; global array assigned 
13:00:09 script,info TEST-SCRIPT: c=2; nothing to do 
13:00:11 script,info TEST-SCRIPT: c=3; global array reset FAILED 
13:00:22 script,info TEST-SCRIPT: c=4; global array reset FAILED 
13:00:23 script,info TEST-SCRIPT: c=5; global array reset FAILED

As you can see the first sequence is correctly restarted but the subsequent one is not.

Additional note: global “keyed array” variables are NOT listed correctly in WinBox (System/Scripts->Environment): value is not shown and not editable. Printing the environment variables from the CLI prompt (/environment print) works as expected.

Reference HW/SW: RB433AH, RouterOS 6.11

Best Regards,
rock

2

Have you tried adding a delay before you reference the new array? I’m thinking maybe RouterOS doesn’t make the new value available instantly, as it otherwise should, and when further attempts to reset it fail, the actual modification is delayed further and further.

you know, f.e.

      :set Y {a=0; b=0; c=0}
      :delay 2
      :if (($Y->"c") = 0) do={

(P.S. Yes, this is a big bug; Yes, MikroTik should fix it; I’m just curious about the cause…)

3

Hi again, boen_robot.

My first thought when reading your reply was that timing issues could NOT be the origin of the problem, for the following reasons:

  1. the strict write/read-back sequence is present only in the posted “sample” script I wrote to reproduce the problem; it is NOT present in the actual “real” script which originally exhibited the bug.
  2. the scripts (both the real and the sample) are intended to be scheduled at regular interval of al least 5 seconds (hence a global memory is used to store status info between different runs): in the sample script, where the strict write/read-back sequence is performed, any delayed-write issue would appear only the first time the array is reassigned (i.e when c=3) but by the time the script is rescheduled (i.e. few seconds after the failed read-back) the global array would have been updated (that’s why the sample script try to reset the array also if c>3). Please note that in subsequent runs the “c” element of the array is read (during the increment involving the local “count” variable) BEFORE being written again (with the incremented value) so the delayed further and further issue does not apply. Moreover, as you can see from the posted logfile, the FIRST attempt to reset the array is successful (and this involves the strict write/read-back sequence) while the subsequent ones are failing: this is exactly the opposite behaviour to the one expected in case of timing issues.
  3. from a general point of view, in ANY programming environment (from assembly-level code, compiled code and up to interpreted scripts) the semantics of a “memory variable” is that any write access MUST be (or, at the very least, must appear) synchronous (i.e. reading a variable MUST return the value last written to it, independent of the time elapsed between the two operations): this is true for any kind of “variable” (i.e. “physical/permanent” and/or “virtual/swapped” ones as in MMU-based devices); any timing issue MUST be managed transparently by the hardware (i.e. by inserting wait-states, bus-holds, etc.) or by the O.S. (i.e. virtual memory manager layer insuring page-coherency, etc.): should this not hold true, most programs would become non-deterministic and would bring wrong results. The simplest example which comes to my mind is a “loop” construct (“for”, “while”, etc.) where the variable used as loop-counter is used (just after being incremented/decremented) as part of the loop’s operations (e.g. used as index into an array).

I do know that timing issues are possible in ROS scripting when accessing some kind of resources (typical example: writing to a disk file then instantly reading back its content) but, as said above, this is absolutely NOT acceptable for “variables” (well, also for file-access this should NOT happen and should be regarded as a “bug”).

I tried anyway to follow your hint and added a delay between the write/read accesses: the result is that, as expected, the script failed exactly the same.

Yes, there is definitely something wrong with the management of environment variables. During my tests I also noticed something very strange: when issuing a :global declaration without value assignment, if was previously existing and DELETED than the new declaration sometimes “revives” the value had before the deletion, instead of resulting in a “fresh” variable of “undefined” type/value. It happened several times: as soon as I will have a concise example reproducing the error I will post it here.

Side note: the type of a variable declared but not assigned (as reported by the “:typeof” command) is nothing and NOT nil as incorrectly documented.

[admin@TESTBED] > :local X ; :put [:typeof $X]
nothing
[admin@TESTBED] >

Best Regards,
rock

4

I just ran into a similar problem on router OS 6.13. If I previously ran a script that set a global variable for an array, when I run the script again, it appears to append to the existing global var rather than overwriting or resetting the var when you declare it.