Using two arrays, process the text and create a third array

I have text. It repeats a phrase, which consists of an alphabetic part and a digital part. The letter part does not change, but the digital part changes and the numbers in it are separated by commas. Something like this: “bla-bla 10,2,125” or “bla-bla 2,1,80” or “bla-bla 7,0,8080”. There is one array that contains the indexes of the occurrence of the searched phrase. There is a second array with the length of the phrase. Tell me how to use these arrays to process the text in such a way that a new multilevel array is obtained, in which there is the first number before the comma from the phrase and two other digits from this phrase are attached to it. Unfortunately, I don’t understand at all how multilevel arrays are written in Mikrotik.
It is clear that you can cut off the phrase using

:local numbers [:pick "text" <start>[<count>]]

, where “start” is the data from the first array plus the number of letters in the phrase (bla-bla = 7 characters), and “count” is the data from the second array minus the number of letters in the phrase, i.e. 7.
The cut phrase is also easy to put in an array like this

:local newNumbers [:toarray $numbers]

But how to create a new array using this? And the array should turn out something like this (unfortunately I don’t know how to write a multilevel array correctly)

[
[10[2,125]]
[2[1,80]]
[7[0,8080]]
]

That is, so that the first numbers are of the index type

Perhaps that will be clearer. Here is a piece of text in which you need to do a search

bla-bla 10,2,,125
43800200437043C043E04360435043C043E002107918350000014F0600ED0D63739500030C07070438043C04300439043C043E0441044F00210020042004300437043E043C0020043C06C7EBBCB0008221120513141803C0
bla-bla 2,1,,80
BBCB0008221120513141808C0500030C070104140440044304370456002C0020044707918350000014F0400ED0D637396C7E04350440043504370020043F043E043204420443E0440043E0433043000200432002004340435044F043A0438044500200440043504330456043E043D0430044500200423043A044004300457043D043800200432045604340441044304423E0440043D04560020043004420430043A0438002004320
bla-bla 7,0,,8080
120513141808C0500030C070207918350000014F0400ED0D637396C7EBBCB0008221043D045400200435043B0435043A004470430043D043D044F002E0020041C043E0436044304420440043E043F043E044104420430442044C002004320438043D0438043A043004420438002004423E0432045600200441043A043B04300434043D043E044904560020043704560020043704322019044F0437043A0438043C04470430044104

Here are two arrays obtained from this segment.

:local start [0,197,552]
:local len [18,17,19]

It’s not clear at all, you don’t give its purpose or usefulness, so I don’t care.

I’ve already answered this:
http://forum.mikrotik.com/t/subtraction-of-elements-in-the-mikrotik-array-does-not-count-the-first-element-correctly/166914/3
but I still don’t see the usefulness for everyone, so if you need such a particular script with a purpose unknown to us in the forum,
I suggest you hire a consultant, if you really need it…

For your example:

# init main array
:local arr {"10"={2;125}; "2"={1;80}; "7"={0;8080}}
# extracts inner array for index 10 (actually is key)
:local arr10 ($arr->"10")
# adds 1345 into inner array
:set arr10 ($arr10, 1345)
# sets updated inner array to main array
:set ($arr->"10") $arr10

If you need new inner array at some index, ex. 44:

# init new inner array
:local arr44 {77;0}
# sets new inner array to main array at index 44
:set ($arr->"44") $arr44

And you can go in depth as like you want…:

:local arr {"10"={"1"={2;125}; "2"={3;34}}};
:put ($arr->"10"->"1")
:put ($arr->"10"->"2")

etc…

This is example of little “nested” ( :laughing: ) array of arays of arrays and other non array inside other arrays…
http://forum.mikrotik.com/t/iterate-over-all-elements-of-an-array-of-unknown-dimension/163033/47

for convert this
[
[10[2,125]]
[2[1,80]]
[7[0,8080]]
]
to
new array with on index 2 another array of values 1 and 80, on index 7 another array of values 0 and 8080, on index 10 another array of values 2 and 125
there are some ways to convert that on one array of arrays:
{
:local newarr {2,125;1,80;0,8080}
:put ($newarr->0->0)
:put ($newarr->0->1)
:put ($newarr->1->0)
:put ($newarr->1->1)
:put ($newarr->2->0)
:put ($newarr->2->1)
}

{
:local newarr [:toarray “”]
:set ($newarr->10) {2;125}
:set ($newarr->2) {1;80}
:set ($newarr->7) {0;8080}
:put ($newarr->10->0)
:put ($newarr->10->1)
:put ($newarr->2->0)
:put ($newarr->2->1)
:put ($newarr->7->0)
:put ($newarr->7->1)
}

{
:local newarr {“10”=2,125;“2”=1,80;“7”=0,8080}
:put ($newarr->“10”->0)
:put ($newarr->“10”->1)
:put ($newarr->“2”->0)
:put ($newarr->“2”->1)
:put ($newarr->“7”->0)
:put ($newarr->“7”->1)
}

Your answer helped a lot to create an array that contains the length of the phrase. That is, the array len
The general meaning of the future script is to learn how to parse SMS. In a long-standing situation, this will be the SMS number, its status, length. Based on it, I hope to learn how to parse the rest of the SMS. I know that there are topics on the forum in which SMS is sent, for example, to telegrams or mail. But I don’t need it. I just want to get some information about sms.

Yeah, I was editing my post with additional examples, you were faster… Arrays and maps can be created and manipulated in ros script in various ways.

Good fortune…
http://forum.mikrotik.com/t/script-for-sending-incoming-sms-to-mail-with-full-parsing/140125/1

I just done this two parts:
Function to convert Hex GSM7 to CP1252 string (for decode the PDU message part on GSM7)
http://forum.mikrotik.com/t/rextended-fragments-of-snippets/151033/1
and
Function to convert one CP1252 (or simply ASCII-7-bit) string to one Hex GSM7 string (for create the PDU message part on GSM7)
http://forum.mikrotik.com/t/rextended-fragments-of-snippets/151033/1

and also this two for UCS-2
Function to convert UCS-2 to UTF-8 (for decode the PDU message part on UCS-2)
http://forum.mikrotik.com/t/convert-any-text-to-unicode/164329/18
Function to convert UTF-8 to UCS-2 (for create the PDU message part on UCS-2)
http://forum.mikrotik.com/t/convert-any-text-to-unicode/164329/26

But only the author on the first link have patience to decode the rest of the PDU message…

I guess I need to update my sms2email script with this. :slight_smile: Maybe also checking UDH header for joining splitted messages will be nice :slight_smile:

Thank you. I have seen this thread. It contains a lot of redundant information. And there is no one that interests me. But I already said that I don’t need someone else’s function. I want to figure it out myself. Maybe later I will want to forward the SMS somewhere, but for now it’s enough for me to read the SMS. And in my opinion it would be much more correct to put the read SMS in an array. So it would be more convenient for further work with him. The author of that project does not do this. The structure is more rigid than I would like. Maybe I’m wrong and I don’t understand all the code in that thread. In any case, I would like to learn how to read SMS myself.
Maybe I’ll use your code snippets later for transcoding, for now I just want to split the sms and put it in an array.

Thanks for your examples. I’m starting to understand a little. Of course, I’m still far from fully understanding, but these examples are beginning to help.

In ROS script you have one dimensional array in format

{val1; val2; val3; ...}

and key=value maps in format

{"key1"=val1; "key2"=val2; "key3"=val3; ...}

, multidimensional arrays can be achieved with maps if key is string of dimension index and value is array of that dimension or map with inner dimension, not optimal and simple as in common programing languages but possible to go in depth like that.

I more or less understand this, but how can I access the internal data index in key 10 directly?
If I do like this:

foreach a,k in=$arr do={
  :put "$a=$k";
  }

then I get the following:

10=2;10=125
2=1;2=80
7=0;7=8080

If I do like this:

:local key10 ($arr->"10")
:put $key10

then I get the following:

2;125

But how do I get only 2 or only 125?

At the moment, I have more or less figured out how to access a specific index of a nested array. There seem to be two ways, but I’m not sure which one is correct.

# init 2 main multy array
:local multy {"10"={k1={2};k2={125}}; "2"={k1={1};k2={80}}; "7"={k1={0};k2={8080}}}
:put $multy

:local multy1 {"10"={k1=2;k2=125}; "2"={k1=1;k2=80}; "7"={k1=0;k2=8080}}
:put $multy1

Both options will produce the same result:

10=k1=2;k2=125;2=k1=1;k2=80;7=k1=0;k2=8080
10=k1=2;k2=125;2=k1=1;k2=80;7=k1=0;k2=8080

We can refer to any of them like this:

:local mu ($multy->"10"->"k1")
:put $mu
:local mu1 ($multy1->"10"->"k1")
:put $mu1

Both options will produce the same result:

2
2

At the moment, I can’t figure out how to iterate over such an array, for example, using the foreach loop. The following options do nothing

foreach aa,kk in=$multy do={
  :put "$aa=$kk";
  }

foreach aaa,kkk in=$multy1 do={
  :put "$aaa=$kkk";
  }

You have to use the 3rd example here.
Why lost time to call first value (index 0) k1 and second value (index 1) k2?
Simply call the value…
http://forum.mikrotik.com/t/using-two-arrays-process-the-text-and-create-a-third-array/166923/1

@DyadyaGenya
I’m wondering why is this all needed for processing sms inbox.
If you need some model map per sms message to process messages outside :foreach sms in=[/tool/sms/inbox find] do={…} loop, you can just create array of maps with properties of sms messages stored in that loop, which will look like:

:local smsList {{"phone"="+123456787"; "message"="sms message 1"; "timestamp"="May/25/2023 18:08:09 GMT -0"; "type"="class-0"}; {"phone"="+4356546456"; "message"="sms message 2"; "timestamp"="May/25/2023 18:08:09 GMT -0"; "type"="class-0"}}

#to get single sms at index 0
:local sms ($smsList->0)
#to get sms phone property
:put ($sms->"phone")
#to get sms message property
:put ($sms->"message")
# etc...

#or to process whole list:
:foreach sms in=$smsList do={
  :put ($sms->"phone")
  :put ($sms->"message")
}

You can achieve that with inner array instead of using properties map and get certain property by index of inner array, but for me this approach has less cognitive complexity code.

Maybe you’re right. I haven’t decided yet how it will be. In this tutorial, I’m trying to understand what arrays can do in Mikrotik scripts. One of the goals is to display a table like this:

--------------------------------------------------------------------------------|
№ |	SCA	|   timestamp  |status|			Body			|
--------------------------------------------------------------------------------|

And I understand that your foreach loop can theoretically accomplish this.
But I also want to have access to any part of the SMS from anywhere and using any index, to understand which SMS it belongs to. Perhaps your solution will be sufficient. Am I understanding correctly that it is a two-dimensional array?

Unfortunately, I haven’t had time to take a closer look at your example yet. Hope to do it tonight.

then for each column you can have map property key per sms message like in my example, either you are using sms tool inbox or extracting PDU directly from modem.


also in example: ($smsList->0), index can be variable also, ($smsList->$index)


Depends how you look at maps :slight_smile: https://www.geeksforgeeks.org/difference-between-array-and-map/. In this case its one-dimensional array of maps, but maps are linked structures in memory with unique key (which is a kind of array) and its elements are commonly accessed by keys, not by index (you can by index also with :pick in ROS) to have better cognitive readable code or unique value per key. Also maps are not ordered (in ROS), ROS is ordering map elements by key name as I see when printing, which means order can change depending which key name you add to map not sequentially when is added, that’s why accessing by index doesn’t make much sense.
For your table example, row is array index and column is map key at that index (except for №, which can be value of index+1).

I can’t say for certain which arrays I will use, but I wanted to try something very basic, specifically, obtaining the message number, its status, and length. To do this, I need to find all phrases of the “+CMGL: 15,1,155” type and trim the alphabetic part. It seems like I’ve gone through all the necessary steps: found all occurrences of such phrases, created an array of these occurrences, found the length of each phrase, and created an array with their values. However, when I try to trim them, only the first phrase gets trimmed for some reason, even though there’s an array of occurrence positions created next to it and some other unclear chunk. Could you please take a look and let me know where my mistake might be?

#The variable where all the unprocessed SMS messages go
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
#Four variables for the counter.
:local matchPosition -1;
:local matchCount 0;
:local previous 0
:local continue true;
#The array where all occurrences of the found line endings are collected.
:local whereEnd ([:toarray ""]);
#The array where all the values of the phrase length, such as "+CMGL: 15,1,,155", are collected
:local arrayLen ({})
#An array where all positions of the occurrence of the phrase go
:local arrayHead ({})
#An array where all trimmed data from a phrase like "15,1,,155" should go
:local arrayStat ({})

#The loop for finding all the identified line ending characters.
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
#        :put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
        :set $whereEnd ($whereEnd, $matchPosition)
    } else={ :set continue false; };
};

#The loop for creating an array where all the calculated lengths of phrases like "+CMGL: 15,1,,155" are stored.
:foreach key,num in=$whereEnd do={
    if (($key % 2) = 0) do={
        :put "$($num - $previous)"
        :set $arrayLen ($arrayLen, $num - $previous)
    } else={
        :set previous $num
        #:put "Find position #$key at:$num"
    }
}

#Checking the result
:put $whereEnd
:put $arrayLen



#The loop for searching and creating an array from the results of trimming the phrase "+CMGL: 15,1,,155".
:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $searchHead $matchPosition ];
   :put $arrayLen
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
        :put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
        :set $arrayHead ($arrayHead, $matchPosition)
        :set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) ($arrayLen->$matchCount) ] )
    } else={ :set continue false; };
};

#Checking the result
:put $arrayHead
:put $arrayStat

Perhaps I have chosen not the most optimal approach, but I am just starting to learn scripts, and I am trying to go from simple to complex in order to develop a solid understanding.

I want to add that all the screen outputs are done for result verification purposes, to make it easier to understand where and at which stage the data exists.