iterate over all elements of an array of unknown dimension

I need to iterate over all elements of an array of unknown dimension (like a Cartesian identity)
The problem must be solved by recursively calling a function to traverse the entire tree with an unknown number and size of “branches” in advance
At the same time, there is no need to reinvent the wheel, there is practically a similar function for Mikrotik - here it is (the author of the great Chupaka,
https://github.com/Winand/mikrotik-json-parser)

# ------------------------------- fJParsePrint ----------------------------------------------------------------
:global fJParsePrint
:if (!any $fJParsePrint) do={ :global fJParsePrint do={
  :global JParseOut
  :local TempPath
  :global fJParsePrint

  :if ([:len $1] = 0) do={
    :set $1 "\$JParseOut"
    :set $2 $JParseOut
   }
   
  :foreach k,v in=$2 do={
    :if ([:typeof $k] = "str") do={
      :set k "\"$k\""
    }
    :set TempPath ($1. "->" . $k)
    :if ([:typeof $v] = "array") do={
      :if ([:len $v] > 0) do={
        $fJParsePrint $TempPath $v
      } else={
        :put "$TempPath = [] ($[:typeof $v])"
      }
    } else={
        :put "$TempPath = $v ($[:typeof $v])"
    }
  }
}}

But I don’t need to print the paths to subarrays and element values, but iterate over their values ​​and replace them with others.

How to do it ? Who can help ? Maybe our Dear Guru Rextended?

Hi,

I can help you, but I must first have to understand correctly what is need.

Provide one array for example and what do you want to do with that array.
No matter if is correct syntax, you can use pseudoinstructions.

Example bi-dimensional x array of y array:

(y)
4 Y X W V U
3 P O N M T
2 I H G L S
1 D C F K R
0 A B E J Q
0 1 2 3 4 (x)
:global test {{"A";"D";"I";"P";"Y"};{"B";"C";"H";"O";"X"};{"E";"F";"G";"N";"W"};{"J";"K";"L";"M";"V"};{"Q";"R";"S";"T";"U"}}
:put ($test->0->0)
:put ($test->0->1)
:put ($test->0->2)
:put ($test->0->3)
:put ($test->0->4)
:put ($test->1->0)
:put ($test->1->1)
:put ($test->1->2)
:put ($test->1->3)
:put ($test->1->4)
:put ($test->2->0)
:put ($test->2->1)
:put ($test->2->2)
:put ($test->2->3)
:put ($test->2->4)
:put ($test->3->0)
:put ($test->3->1)
:put ($test->3->2)
:put ($test->3->3)
:put ($test->3->4)
:put ($test->4->0)
:put ($test->4->1)
:put ($test->4->2)
:put ($test->4->3)
:put ($test->4->4)

EDIT: Fix formatting in new forum

It’s hard for me to give an example, but I’ll try.
Imagine that there is an array containing as elements not only values, but also arrays. You need to go through all the values ​​of the elements of this tree and assign them other values ​​(for example, process them with your $myFunc function).

Let the input tree array be in $ArrayIN
I’m assuming it should look something like this, but there are probably bugs in my code:

:global fJParseArray do={
  :local TempPath
  :global fJParseArray
   
  :if ([:len $1] = 0) do={
    :set $1 "\$ArrayIn"
    :set $2 $ArrayIn
   }

  :foreach k,v in=$2 do={
    :if ([:typeof $k] = "str") do={
      :set k "\"$k\""
    }
    :set TempPath ($1. "->" . $k)
    :if ([:typeof $v] = "array") do={
      :if ([:len $v] > 0) do={
        $fJParseArray $TempPath $v
      } else={
        :set ($TempPath) []
      }
    } else={
        :set ($TempPath) [$MyFunc $v]
    }
  }
}

For an example of what ArrayIn contains (paths to values):

$ArrayIn->“ok” = true (bool)
$ArrayIn->“result”->“has_custom_certificate” = false (bool)
$ArrayIn->“result”->“last_error_date” = 1524483204 (num)
$ArrayIn->“result”->“last_error_message” = Connection timed out (str)
$ArrayIn->“result”->“max_connections” = 40 (num)
$ArrayIn->“result”->“pending_update_count” = 0 (num)
$ArrayIn->“result”->“url” = https://*****.ru:8443 (str)

Example bi-dimensional x array of y array, It’s too easy. We initially have an array with an unknown data nesting level

I must go away to the office I came later and I reply to you, on meantime:

:global test {{"A";"D";"I";"P";"Y"};{"B";"C";"H";"O";"X"};{"E";"F";"G";"N";"W"};{"J";"K";"L";"M";"V"};{"Q";"R";"S";"T";"U"}}

:foreach x in=$test do={
    :foreach y in=$x do={
        :put "$[:find $test $x],$[:find $x $y] = $y"
    }
}

Do you think that in this way we can process all elements of arrays, including key associative arrays?

Here is an example of an array to be traversed. Note that it contains other arrays as elements.

alertList=;audioSettings=DoP=title=DoP playback;type=boolean;value=false;autoPlay=title=AutoPlay after boot;type=boolean;value=false;soundCard=data=id=0;name=Default;title=Sound Card;type=spinner;value=0;soundType=data=id=0;name=Mono Differential;id=1;name=Stereo;title=Sound Type;type=spinner;value=1;title=Audio settings;broadcastModeSettings=broadcastModeOptions=downloadURL=;streamURL=;broadcastType=data=id=0;name=Default;id=1;name=Stream URL;id=2;name=Download URL;title=Broadcast Type;type=spinner;value=0;enabled=false;title=Broadcast mode settings;deviceInfo=autoUpgrade=true;autoUpgradeInstall=true;cpuTemp=48°C;version=2.16.37;multiroomSettings=masterMode=false;slaveList=;networkSettings=connections=LAN=addresses=192.168.0.101/24;dns=192.168.0.1;gateway=192.168.0.1;method=auto;state=connected;WLAN=state=disconnected;interfaces=LAN=title=Ethernet;type=boolean;value=true;WLAN=title=Wi-Fi;type=boolean;value=false;title=Network settings;wifiList=

And here are the paths to the elements of each array from my example above, indicating the value of the element and its type:

ArrayIN->“alertList” = (array)
ArrayIN->“audioSettings”->“DoP”->“title” = DoP playback (str)
ArrayIN->“audioSettings”->“DoP”->“type” = boolean (str)
ArrayIN->“audioSettings”->“DoP”->“value” = false (bool)
ArrayIN->“audioSettings”->“autoPlay”->“title” = AutoPlay after boot (str)
ArrayIN->“audioSettings”->“autoPlay”->“type” = boolean (str)
ArrayIN->“audioSettings”->“autoPlay”->“value” = false (bool)
ArrayIN->“audioSettings”->“soundCard”->“data”->0->“id” = 0 (num)
ArrayIN->“audioSettings”->“soundCard”->“data”->0->“name” = Default (str)
ArrayIN->“audioSettings”->“soundCard”->“title” = Sound Card (str)
ArrayIN->“audioSettings”->“soundCard”->“type” = spinner (str)
ArrayIN->“audioSettings”->“soundCard”->“value” = 0 (num)
ArrayIN->“audioSettings”->“soundType”->“data”->0->“id” = 0 (num)
ArrayIN->“audioSettings”->“soundType”->“data”->0->“name” = Mono Differential (str)
ArrayIN->“audioSettings”->“soundType”->“data”->1->“id” = 1 (num)
ArrayIN->“audioSettings”->“soundType”->“data”->1->“name” = Stereo (str)
ArrayIN->“audioSettings”->“soundType”->“title” = Sound Type (str)
ArrayIN->“audioSettings”->“soundType”->“type” = spinner (str)
ArrayIN->“audioSettings”->“soundType”->“value” = 1 (num)
ArrayIN->“audioSettings”->“title” = Audio settings (str)
ArrayIN->“broadcastModeSettings”->“broadcastModeOptions”->“downloadURL” = (str)
ArrayIN->“broadcastModeSettings”->“broadcastModeOptions”->“streamURL” = (str)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“data”->0->“id” = 0 (num)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“data”->0->“name” = Default (str)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“data”->1->“id” = 1 (num)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“data”->1->“name” = Stream URL (str)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“data”->2->“id” = 2 (num)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“data”->2->“name” = Download URL (str)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“title” = Broadcast Type (str)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“type” = spinner (str)
ArrayIN->“broadcastModeSettings”->“broadcastType”->“value” = 0 (num)
ArrayIN->“broadcastModeSettings”->“enabled” = false (bool)
ArrayIN->“broadcastModeSettings”->“title” = Broadcast mode settings (str)
ArrayIN->“deviceInfo”->“autoUpgrade” = true (bool)
ArrayIN->“deviceInfo”->“autoUpgradeInstall” = true (bool)
ArrayIN->“deviceInfo”->“cpuTemp” = 48 C (str)
ArrayIN->“deviceInfo”->“version” = 2.16.37 (str)
ArrayIN->“multiroomSettings”->“masterMode” = false (bool)
ArrayIN->“multiroomSettings”->“slaveList” = (array)
ArrayIN->“networkSettings”->“connections”->“LAN”->“addresses” = 192.168.0.101/24 (str)
ArrayIN->“networkSettings”->“connections”->“LAN”->“dns” = 192.168.0.1 (str)
ArrayIN->“networkSettings”->“connections”->“LAN”->“gateway” = 192.168.0.1 (str)
ArrayIN->“networkSettings”->“connections”->“LAN”->“method” = auto (str)
ArrayIN->“networkSettings”->“connections”->“LAN”->“state” = connected (str)
ArrayIN->“networkSettings”->“connections”->“WLAN”->“state” = disconnected (str)
ArrayIN->“networkSettings”->“interfaces”->“LAN”->“title” = Ethernet (str)
ArrayIN->“networkSettings”->“interfaces”->“LAN”->“type” = boolean (str)
ArrayIN->“networkSettings”->“interfaces”->“LAN”->“value” = true (bool)
ArrayIN->“networkSettings”->“interfaces”->“WLAN”->“title” = Wi-Fi (str)
ArrayIN->“networkSettings”->“interfaces”->“WLAN”->“type” = boolean (str)
ArrayIN->“networkSettings”->“interfaces”->“WLAN”->“value” = false (bool)
ArrayIN->“networkSettings”->“title” = Network settings (str)
ArrayIN->“networkSettings”->“wifiList” = (array)

I’d listen to @rextended here.

But this may give you another example of using recursive functions to iterate over an array. The trick is you have check if it’s “array” type to know to call it recursively. Basically instead of the :put’s in the code, you could use a :set for your use case. See:
http://forum.mikrotik.com/t/need-some-scripting-help-bounty-available/162737/1

Thank you very match, Amm !

But :set no working …

I tri :

:global ParseArray do={
  :global ArrayIn
  :local TempPath
  :global ParseArray
   
  :if ([:len $1] = 0) do={
    :set $1 "\$ArrayIn"
    :set $2 $ArrayIn
   }

  :foreach k,v in=$2 do={
    :if ([:typeof $k] = "str") do={
        :set k "\"$k\""
    }
    :set TempPath ($1. "->" . $k)
    :parse [[":set (\$$TempPath) hello"]]
    :if ([:typeof $v] = "array") do={
      :if ([:len $v] > 0) do={
        $ParseArray $TempPath $v
      } else={
        :set ($TempPath) []
      }
    } else={

#       [[:parse ":set ($TempPath) hello"]]
    }
  }
}

It’s hard to answer. It’s less code to change values in the same “associative” array (e.g. like map() in other programming languages). While you can creating some other structure than the original (in a reduce() in CS terms), this is more code and becomes dependent on the specifics of what you’d want to do.

:global FLATTEN do={
  :global FLATTEN
  :local memo [:toarray $2]
  :local FNPUT true
  :foreach i,k in=$1 do={
    :if ([:typeof $k]="array") do={      
        :if ($FNPUT) do={:put "$[:tostr $i]=(array)"}
        :set memo [$FLATTEN $k $memo]
    } else={
      :if ($FNPUT) do={:put "$[:tostr $i]=$[:tostr $k] ;$[:typeof $k]"}
      :set memo {$memo; [:toarray "$i , $k"]}
    }
  }
  :return $memo
 }

In this example, you call the function with your original array, and it will return a new array with all the values as string. e.g.
:global ParseArray [$FLATTEN $ArrayIn “”]

What you’re code doing is assuming using the original array inside the function – this can work – but does make the code more complex than keeping the map/reduce() function “generic” by accepting the original array as the 1st param, and the output array is the 2nd (and this 2nd parameter is what’s returned when all recursion is done, containing an “indexed” array with strings of all values from $ArrayIn).

edit: forgot }

This is your array example created with RouterOS syntax, for work over that array.

The only difference with what you wrote: I replace empty array alertList, slaveList and wifiList with {“arrayvalue1”,“arrayvalue2”} for testing purpose.
:global ArrayIN { “alertList”=“arrayvalue1”,“arrayvalue2”;
“audioSettings”={ “DoP”={ “title”=“DoP playback”;
“type”=“boolean”;
“value”=false
};
“autoPlay”={ “title”=“AutoPlay after boot”;
“type”=“boolean”;
“value”=false
};
“soundCard”={ “data”={ {“id”=0;“name”=“Default”}
};
“title”=“Sound Card”;
“type”=“spinner”;
“value”=0
};
“soundType”={ “data”={ {“id”=0;“name”=“Mono Differential”};
{“id”=1;“name”=“Stereo”}
};
“title”=“Sound Type”;
“type”=“spinner”;
“value”=1
};
“title”=“Audio settings”
};
“broadcastModeSettings”={ “broadcastModeOptions”={ “downloadURL”=“no URL provided”;
“streamURL”=“no URL provided”
};
“broadcastType”={ “data”={ {“id”=0;“name”=“Default”};
{“id”=1;“name”=“Stream URL”};
{“id”=2;“name”=“Download URL”}
};
“title”=“Broadcast Type”;
“type”=“spinner”;
“value”=0
};
“enabled”=false;
“title”=“Broadcast mode settings”
};
“deviceInfo”={ “autoUpgrade”=true;
“autoUpgradeInstall”=true;
“cpuTemp”=“48 C”;
“version”=“2.16.37”
};
“multiroomSettings”={ “masterMode”=false;
“slaveList”=“arrayvalue1”,“arrayvalue2”
};
“networkSettings”={ “connections”={ “LAN”={ “addresses”=“192.168.0.101/24”;
“dns”=“192.168.0.1”;
“gateway”=“192.168.0.1”;
“method”=“auto”;
“state”=“connected”
};
“WLAN”={ “state”=“disconnected” }
};
“interfaces”={ “LAN”={ “title”=“Ethernet”;
“type”=“boolean”;
“value”=true
};
“WLAN”={ “title”=“Wi-Fi”;
“type”=“boolean”;
“value”=false
}
};
“title”=“Network settings”;
“wifiList”=“arrayvalue1”,“arrayvalue2”
};
}

Function updated, see:
http://forum.mikrotik.com/t/iterate-over-all-elements-of-an-array-of-unknown-dimension/163033/1

Example:
Replace inside ArrayIN on “networkSettings” / “connections” / “LAN” / “addresses” from “192.168.0.101/24” to “192.168.0.102/24”

Obviously if already we know all path, is useless, because just set
:set ($ArrayIN->“networkSettings”->“connections”->“LAN”->“addresses”) “192.168.0.102/24”

But if the path is unknow and containing only one “addresses” (if not, is replaced only the first, for check for more, the script must be modified)

:global searchpath do={ :global searchpath
                            :local path "$4"
                            /system script environment
                            :foreach j in=[find] do={
                                :if ([get $j value] = $1) do={:set path "\$$[get $j name]"}
                            }
                            :foreach x,y in=$1 do={
                                :local lpath $path
                                :if ([:typeof $x] = "str") do={:set lpath "$path->\"$x\""} else={:set lpath "$path->$x"}
                                :if (($x = $2) and ($y = $3)) do={
                                    :return "$lpath"
                                } else={
                                    :if ([:typeof $y] = "array") do={
                                        :local ret [$searchpath $y $2 $3 $lpath]
                                        :if ($ret != "KO") do={:return $ret}
                                    }
                                }
                            }
                            :return "KO"
                      }

{
# show previous value
:put ($ArrayIN->"networkSettings"->"connections"->"LAN"->"addresses")
# show current path
:put [$searchpath $ArrayIN "addresses" "192.168.0.101/24"]
# execute the substitution of value
[[:parse (":global ArrayIN; :set ($[$searchpath $ArrayIN "addresses" "192.168.0.101/24"]) \"192.168.0.102/24\"")]]
# show new value
:put ($ArrayIN->"networkSettings"->"connections"->"LAN"->"addresses")
}

This is what you need?

Thank you, very match, Rextended and Amm0 !

:global fJParsePrint
:if (!any $fJParsePrint) do={ :global fJParsePrint do={
  :global JParseOut
  :local TempPath
  :global fJParsePrint

  :if ([:len $1] = 0) do={
    :set $1 "\$JParseOut"
    :set $2 $JParseOut
   }
   
  :foreach k,v in=$2 do={
    :if ([:typeof $k] = "str") do={
      :set k "\"$k\""
    }
    :set TempPath ($1. "->" . $k)
    :if ([:typeof $v] = "array") do={
      :if ([:len $v] > 0) do={
        $fJParsePrint $TempPath $v
      } else={
        :put "$TempPath = [] ($[:typeof $v])"
      }
    } else={
        :put "$TempPath = $v ($[:typeof $v])"
    }
  }
}}

I will definitely think about and test your solutions. But I would like to ask you to make edits to the code that Chupaka proposed, since it seems to me shorter, clearer and more perfect.
In theory, the Chupaka code is absolutely universal, since it provides a search of all elements of the array, regardless of the composition of its values and depth, by forming a path to the value through recursion. I would like to use it and it would be useful to everyone.
Note that it generates the full path to the value in the string

:set TempPath ($1. "->" . $k).

You just need to add :set to it in order to replace the values of the array when iterating through it where necessary.
I think I need to form :set by

 [[:parse ":set (\$$TempPath) $newvalue"]]

, but somewhere I have an error in it, because set does not work like that. Please look at this code with your “experienced eye”.

I think that to iterate through the entire tree in order to replace the final values, you should end up with something like this code:

:global ParseArray do={
  :global ArrayIn
  :local TempPath
  :global ParseArray
   
  :if ([:len $1] = 0) do={
    :set $1 "\$ArrayIn"
    :set $2 $ArrayIn
   }

  :foreach k,v in=$2 do={
    :if ([:typeof $k] = "str") do={
        :set k "\"$k\""
    }
    :set TempPath ($1. "->" . $k)
    :if ([:typeof $v] = "array") do={
      :if ([:len $v] > 0) do={
        $ParseArray $TempPath $v
      } else={
         [[:parse ":set (\$$TempPath) []"]]
      }
    } else={
       [[:parse ":set (\$$TempPath) $newvalue"]]
    }
  }
}

Here $newvalue is the new value. I think that I have a mistake in the formation of the construction :parse - [[:parse “:set ($$TempPath) $newvalue”], therefore :set does not work.

It seems to me the opposite, not that Chupaka hasn’t done a good job, but I find it hard to follow it, as it is written, since I didn’t write it.


But didn’t you realize that the same description can be applied to my code on previous post?
http://forum.mikrotik.com/t/iterate-over-all-elements-of-an-array-of-unknown-dimension/163033/1


My code already work, see the example included (and read the note)…


Read first reply o this topic…


Differencies on the two functions, based on your description:

Your modified Chupaka function, not writed directly from Chupaka, iterate on all array,
and during that iteraction replace values…
But you replace values on TEMPORARY array used to construct the path, not directly to the source array…

My function simply return KO if no occurrency found, or the full path to the value on the array.
On 2nd time, with that info, you can update the true array, not one temporary copy.

But you replace values on TEMPORARY array used to construct the path, not directly to the source array…

[[:parse “:set ($$TempPath) $newvalue”]]

I can agree with everything, but with this… I don 't understand why ?

For example:

:set ($ArrayIN->“NetworkSettings”->“address”) newaddress

After all, a specific path to the value is formed in this line and set to this value is executed. Does :parse in this case not work with a specific value of a specific array ? Why temporary ?

By the way, looking at these lines now, I think I understood the reason for my failures - $ArrayIN was formed in quotation marks, but it should be without them!