Test if array is associative, multi-dimensional, or simple list

mike548141 · Mon Feb 12, 2024 9:40 am

I'm hoping someone will tell me I'm missing something obvious.

I'm trying to find a way for a RouterOS script to test if an array is a simple list of values, an associative array (key,value pairs), or a multi-dimensional array (a list of array's within a array). [Descriptions in brackets in case I'm using the wrong terms]
For use in this script (its not 100% yet) https://gist.github.com/mike548141/ba12 ... cc2887f6d6

:typeof does not distinguish the type of array, is there another command or a way to test?

Thanks,

MC

optio · Mon Feb 12, 2024 12:27 pm

Problem is that you can combine all in single array, you can have single element, k/v element and inner array element in single array. See "Operations with Arrays" in documentation. You can iterate array and check its elements type to determine this or just use 1st element as reference, but how you will handle mix?

Amm0 · Mon Feb 12, 2024 2:24 pm

What you can check is that ([:typeof ($var->0)] = "nothing"). A key-value "map" cannot be accessed using a num index (only ->"key"), while a simple "list" would have ->0... so if $var is array and has $var->0 then it's a list type...

e.g.

:global isList do={:if (([:typeof $1] = "array") && ([:typeof ($1->0)] != "nothing")) do={:return true} else={:return false}}
:global isMap do={:if (([:typeof $1] = "array") && ([:typeof ($1->0)] = "nothing")) do={:return true} else={:return false}}

:global mymap {a="aval";b="bval"}
:global mylist ("aval", "bval")
:put "mymap isList: $[$isList $mymap] "
:put "mymap isMap: $[$isMap $mymap]"
:put "mylist isList: $[$isList $mylist] "
:put "mylist isMap: $[$isMap $mylist]"
mymap isList: false
mymap isMap: true
mylist isList: true
mylist isMap: false

But since a list can contain a map, or a map entry can contain a list... @optio is correct: you'd only know if "mixed" or multi-dimensional if you recursively iterated any array... as each element can have different type.

rextended · Mon Feb 12, 2024 2:50 pm

One array can contain all types of data, can only be used "foreach" for search if a sub-array exist.
if ->0 is undefined ("nothing"), but the lenght is > 0, the array have only key pairs,
if last ->n is defined (can also be "nil"), and is the same length (- 1) of the array, the array have only numeric values.

:global x [:toarray ""]
:set ($x->9) 9
:put [:len $x]
:put [:typeof ($x->0)]
:put [:typeof ($x->10)]
:put $x

:global x [:toarray ""]
set ($x->0) 1
set ($x->"test") "test"
set ($x->"atest") [:toarray ""]
:put $x
:put [:typeof ($x->"test")]
:put [:typeof ($x->"atest")]

no matter the order, everytime numeric index be first, and undeclared numbers automatically created for fill the gap,
keyed values are at the end, and everytime sorted (I use this trick to sort items)

:global x [:toarray ""]
:set ($x->"t2") "v2"
:set ($x->2) 2
:set ($x->4) 4
:set ($x->"t1") "v1"
:set ($x->9) 9
:put [:len $x]
:put [:typeof ($x->0)]
:put [:typeof ($x->([:len $x] - 1))]
:put $x

Test if one array contains also keyed values and if one index value are one sub-array, and if one keyed value is also one sub-array.

example code

{
:local x [:toarray ""]
:set ($x->9) test

:put $x
:local multiarray    false
:local multikeyarray false
:local numericonly   true
:for counter from=0 to=([:len $x] - 1) do={
    :if ([:typeof ($x->$counter)] = "array")   do={:set multiarray   true}
    :if ([:typeof ($x->$counter)] = "nothing") do={:set numericonly false}
}
:foreach v,w in=$x do={
    :if (([:typeof $v] != "num") and ([:typeof $w] = "array")) do={:set multikeyarray true}
}
:put "multiarray on numeric $multiarray"
:put "multiarray on key value $multikeyarray"
:put "numericonly $numericonly"


:put "\r\n***\r\n"


:set ($x->"test") "test"
:put $x
:local multiarray    false
:local multikeyarray false
:local numericonly   true
:for counter from=0 to=([:len $x] - 1) do={
    :if ([:typeof ($x->$counter)] = "array")   do={:set multiarray   true}
    :if ([:typeof ($x->$counter)] = "nothing") do={:set numericonly false}
}
:foreach v,w in=$x do={
    :if (([:typeof $v] != "num") and ([:typeof $w] = "array")) do={:set multikeyarray true}
}
:put "multiarray on numeric $multiarray"
:put "multiarray on key value $multikeyarray"
:put "numericonly $numericonly"


:put "\r\n***\r\n"


:set ($x->4) [:toarray ""]
:put $x
:local multiarray    false
:local multikeyarray false
:local numericonly   true
:for counter from=0 to=([:len $x] - 1) do={
    :if ([:typeof ($x->$counter)] = "array")   do={:set multiarray   true}
    :if ([:typeof ($x->$counter)] = "nothing") do={:set numericonly false}
}
:foreach v,w in=$x do={
    :if (([:typeof $v] != "num") and ([:typeof $w] = "array")) do={:set multikeyarray true}
}
:put "multiarray on numeric $multiarray"
:put "multiarray on key value $multikeyarray"
:put "numericonly $numericonly"


:put "\r\n***\r\n"


:set ($x->"something") [:toarray ""]
:put $x
:local multiarray    false
:local multikeyarray false
:local numericonly   true
:for counter from=0 to=([:len $x] - 1) do={
    :if ([:typeof ($x->$counter)] = "array")   do={:set multiarray   true}
    :if ([:typeof ($x->$counter)] = "nothing") do={:set numericonly false}
}
:foreach v,w in=$x do={
    :if (([:typeof $v] != "num") and ([:typeof $w] = "array")) do={:set multikeyarray true}
}
:put "multiarray on numeric $multiarray"
:put "multiarray on key value $multikeyarray"
:put "numericonly $numericonly"
}

and for the end of test, if [:typeof ($x->0)] is "nothing" but the length is > 0, the array do not have numeric values, but only keyed pairs.

Amm0 · Mon Feb 12, 2024 3:35 pm

Code: Select all
:global x [:toarray ""]
:set ($x->9) 9
:put [:len $x]
:put [:typeof ($x->0)]
:put [:typeof ($x->10)]
:put $x
[...]

no matter the order, everytime numeric index be first, and undeclared numbers automatically created for fill the gap,
keyed values are at the end, and everytime sorted (I use this trick to sort items)

LOL. I guess scripting supports a "sparse array" type too (e.g. nil elements). https://en.wikipedia.org/wiki/Sparse_matrix

rextended · Mon Feb 12, 2024 3:48 pm

[some concepts I already know you know, but the explanation is for others who don't know]

I do not understand what you mean

:global x [:toarray ""]
create the array

:set ($x->9) 9
set the index 9 (zero based) to 9

:put [:len $x]
the lenght is 10 (from 0 to 9)
because previous instruction also create index 0, 1, 2, 3, 4, 5, 6, 7 and 8 with "nil" value.

:put [:typeof ($x->0)]
this is "nil" because 0 is not defined

:put [:typeof ($x->10)]
this is "nothing" because the 11th element (is 0 based, so 0 i s the first) do not exist

optio · Mon Feb 12, 2024 5:59 pm

:put [:typeof ($x->10)]
this is "nothing" because the 11th element (is 0 based, so 0 i s the first) do not exist

and in most other languages this is index out of bounds error/exception...

Amm0 · Mon Feb 12, 2024 6:07 pm

:put [:typeof ($x->10)]
this is "nothing" because the 11th element (is 0 based, so 0 i s the first) do not exist
and in most other languages this is index out of bounds error/exception...

Most other language don't have to marshall their data types via environment variables & fit language into 16MB with an OS. e.g. everything is a simple list that's parsed.

Lucky I think we've clarified @mike548141 question

I'm hoping someone will tell me I'm missing something obvious.

Amm0 · Tue Feb 13, 2024 6:00 am

For use in this script (its not 100% yet) https://gist.github.com/mike548141/ba12 ... cc2887f6d6

FWIW, while no "pretty print"... v7.13 has JSON support as built-in function

:put [:serialize {a=123} to=json]
{"a":123}
:put [:deserialize "{ \"a\": 123 }" from=json ]                 
a=123

mike548141 · Tue Feb 13, 2024 9:03 am

FWIW, while no "pretty print"... v7.13 has JSON support as built-in function

Yip I'm aware thanks, and I expect I will swap to that eventually as I've no interest in reinventing the wheel - I've been using that script [well an older version] for the last few years on 6.x and early 7.x thus a script as a workaround. I have 7.13.4 on test gear but not rolled out everywhere yet.

mike548141 · Tue Feb 13, 2024 9:10 am

You can iterate array and check its elements type to determine this or just use 1st element as reference, but how you will handle mix?

Yeah that script already walks the array elements to convert RouterOS data types to JSON data types. If its a mix or a key/value then it will map to a JSON object output, otherwise its a JSON array of values or objects output.

mike548141 · Tue Feb 13, 2024 9:12 am

if ->0 is undefined ("nothing"), but the lenght is > 0, the array have only key pairs,
if last ->n is defined (can also be "nil"), and is the same length (- 1) of the array, the array have only numeric values.

Thanks for all the example code to test the array types, I will give that a whirl

Amm0 · Tue Feb 13, 2024 6:15 pm

FWIW, while no "pretty print"... v7.13 has JSON support as built-in function
Yip I'm aware thanks, and I expect I will swap to that eventually as I've no interest in reinventing the wheel - I've been using that script for the last few years on 6.x and early 7.x thus a script as a workaround. I have 7.13.4 on test gear but not rolled out everywhere yet.

I'm in the same boat.

I notice your comments about YAML. I have an example of array to YAML here: viewtopic.php?t=199581&hilit=yaml – more complex since can "tee" outputs to a file ($YAML file=out.yaml $myarray), but there is a regex inside to know if strings need to be escaped in YAML that might be useful if your were going to add YAML to your script.

I actually mainly use my $YAML function to "debug" arrays at the terminal... since ":put $myarray" output the ";element;;;" stuff make it hard to know WTF structure it has.

Amm0 · Tue Feb 13, 2024 6:39 pm

I forgot what I did for this in my $YAML...

So you can also check the array "subtype" in two-value foreach like:

:foreach k,v in=$ar do={
        :if ([:typeof $v]="array") do={
            :if ([:typeof "$k"]="num") do={:put "list"} else={:put "map"}
         }
}

But you can't avoid loops or recursion to know if mixed. But the :typeof of the key in foreach will be num if a "simple list", or string if associative/"map

rextended · Tue Feb 13, 2024 7:12 pm

All possible cases inside one function.
II skipped the recursive check to see how many sub-arrays there are within the sub-arrays and so on, it seemed excessive to me.

testArray code

:global testArray do={
    :local isNumeric     false
    :local isMapped      false
    :local arrayNumeric  false
    :local mappedNumeric false
    :foreach tmpTAx,tmpTAy in=$1 do={
        :if ([:typeof $tmpTAx] = "num") do={
           :set isNumeric true
           :if ([:typeof $tmpTAy] = "array") do={:set arrayNumeric  true}
        } else={
           :set isMapped true
           :if ([:typeof $tmpTAy] = "array") do={:set mappedNumeric true}
        }
    }
    :local isEmpty (!($isNumeric and $isMapped))
    :return {"isEmpty"=$isEmpty;"isNumeric"=$isNumeric;"isMapped"=$isMapped;"arrayNumeric"=$arrayNumeric;"mappedNumeric"=$mappedNumeric}
}

test use code

{
:local x [:toarray ""]
:put ">$[:tostr $x]<"
:put [$testArray $x]
:put "\r\n"

:local x [:toarray ""]
:set ($x->0) "zero"
:put ">$[:tostr $x]<"
:put [$testArray $x]
:put "\r\n"

:local x [:toarray ""]
:set ($x->"test") "value"
:put ">$[:tostr $x]<"
:put [$testArray $x]
:put "\r\n"

:local x [:toarray ""]
:set ($x->0) "zero"
:set ($x->"test") "value"
:put ">$[:tostr $x]<"
:put [$testArray $x]
:put "\r\n"

:local x [:toarray ""]
:set ($x->0) [:toarray "array,inside,numeric"]
:put ">$[:tostr $x]<"
:put [$testArray $x]
:put "\r\n"

:local x [:toarray ""]
:set ($x->"test")  [:toarray "array,inside,mapped"]
:put ">$[:tostr $x]<"
:put [$testArray $x]
:put "\r\n"

:local x [:toarray ""]
:set ($x->0) [:toarray "array,inside,numeric"]
:set ($x->"test")  [:toarray "array,inside,mapped"]
:put ">$[:tostr $x]<"
:put [$testArray $x]
:put "\r\n"


# can also be get single values:
:put "isEmpty is $([$testArray $x]->"isEmpty")"
:put "isNumeric is $([$testArray $x]->"isNumeric")"
:put "isMapped is $([$testArray $x]->"isMapped")"
:put "arrayNumeric is $([$testArray $x]->"arrayNumeric")"
:put "mappedNumeric is $([$testArray $x]->"mappedNumeric")"
}

mike548141 · Wed Feb 14, 2024 12:06 pm

there is a regex inside to know if strings need to be escaped in YAML that might be useful if your were going to add YAML to your script.
...
So you can also check the array "subtype" in two-value foreach like:

Thanks - those both look good!

I actually mainly use my $YAML function to "debug" arrays at the terminal...

I wrote my original JSON script for the same reason

Cheers