As already written on another post, never use ({}) the result is unexpected.
The only way to create empty array is [:toarray “”] and without ( ).
Okay, I'll fix this. I have inserted a few lines for clarification in the code. I forgot to add them when I did the translation
+CMGL: 0,1,“”,48\r\n07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72\r\n
+CMGL: 1,1,“”,40\r\n07912180958739F1040B917120069876F000009140503223438A17D4F29C9E769F4141EA6AD83C3241C377BB1D769341\r\n
+CMGL: 2,1,“”,38\r\n07912180958739F1040B917120069876F000009140503233048A15D272980C0AB3D9A069730A6A97E7F3F0B93C07\r\n
OK
Do not split at \n (or more precisely to \r\n), split at +CMGL:
Array obtained afer filtering out the last two characters (\r\n) and the end OK at the same time:
0,1,“”,48\r\n07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
1,1,“”,40\r\n07912180958739F1040B917120069876F000009140503223438A17D4F29C9E769F4141EA6AD83C3241C377BB1D769341
2,1,“”,38\r\n07912180958739F1040B917120069876F000009140503233048A15D272980C0AB3D9A069730A6A97E7F3F0B93C07
Using :toarray on each line can be possible have one
array of arrays: on each array the 1st value (0) is the index number, the 2nd (1) the message status, the 3rd (2) the used memory, and on the 4th (3) the PDU message.
and now is possible to create a function that replace the PDU with one array with converted PDU contents.
the PDU start with the DECIMAL number total PDU length, the \r\n and the hex code of the PDU. The length of the hex code is the double of the decimal number that represent each hex pair.
example of one SMS array inside the main array
48\r\n07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
==>>
+12085978931 ; SMS-DELIVER ; +17026089670 ; GSM-7 ; 19-04-05 23:32:12 TZ GMT-7 ; 21 ; “Test message to AT Command Tester”
07 91 2180958739F1 (sms center, length [7 x 2 bytes] and international format, F is a filler without value)
PDU HEADER: 04 (SMS-DELIVER)
TP-OA: 0B 91 7120069876F0 (sender number length [B = 11 characters] and international format, F is a filler without value)
TP-PID: 00 (message class 0)
TP-DCS: 00 (message text on GSM-7 format)
TP-SCTS 9140503223218A (date and time) (19-04-05 23:32:12 TZ GMT-7 [negative timezone and 28 “times” 15 minutes = 420 minutes = 7 hours ] )
TP-UDL: 21 (message lenght on septept, because is on GSM-7 format. Are octects if the mesage is on UCS-2)
TP-UD: D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
(message encoded on octect, but internally is on septept for GSM-7, is on octect if is ASCII-8 or UCS-2)
Decode the GSM-7 or eventually the UCS-2 part is easy with my already existent functions.
Function to convert Hex GSM7 to CP1252 string (for decode the PDU message part on GSM7)
http://forum.mikrotik.com/t/rextended-fragments-of-snippets/151033/1
Function to convert UCS-2 to UTF-8 (for decode the PDU message part on UCS-2)
http://forum.mikrotik.com/t/convert-any-text-to-unicode/164329/18
Unfortunately, I did not have time to notice that there is a carriage return symbol here. This makes the job easier to some extent.
As far as I understand, in MikroTik there is no built-in way to split by a specific character. Maybe I haven't found it, but it seems that it only splits by comma.
Currently, I haven't been able to split the overall SMS string into a multidimensional array using anything other than commas. However, in this case, all the results are essentially obtained in a single column. That's the approach I have used so far.
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local matches [:toarray $modemOutput]
:foreach chast in=$matches do={
:put $chast;
}
That's why I chose the method of searching for the length of the phrase.
I actually know what parts a SMS consists of. I even know that the phone number needs to be rearranged in pairs of digits. At the moment, my goal is to learn how to split the SMS into parts. However, I didn't understand two things: 1. How you split the modem response into a multidimensional array, and 2. Why there is an error in my code, as the other arrays are being filled.
I have realized my mistake. It turns out I was translating and applying the “:pick” command incorrectly. The second parameter of the command is not the length of the segment to be cut from the beginning (from the first parameter), but rather the length from the beginning of the string. Therefore, I was performing unnecessary calculations for the segment length. It is possible to directly replace the string with the message number 52
:set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) ([ :find $modemOutput "\n" $matchPosition ] -1 ) ] )
Thus, the entire code will now look like this:
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]
:local continue true;
while ( $continue ) do={
:set matchPosition [ :find $modemOutput $search1 $matchPosition ];
:if ( [ :typeof $matchPosition ] != "nil" ) do={
:set matchCount ( $matchCount + 1 );
:put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
:set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) ([ :find $modemOutput $search2 $matchPosition ] ) ] )
} else={ :set continue false; };
};
:put $arrayStat
Let’s consider that I have figured out the first step
Replaced with:
https://forum.mikrotik.com/viewtopic.php?p=1005348#p1005348
I'm still thinking and deciding what would be more convenient. But your option looks if not exactly how I want it, then very close to what I want
I have a couple of clarifications regarding your code, although they are not directly related to it.
- How did you determine that the excerpt '\r\n+CMGL: ' at the beginning of the string contains carriage return and line feed characters, and that the excerpt '\r\nOK' contains some 'OK' and what does that 'OK' mean? I haven't seen such Constant Escape Sequences.
- I used to see the 'alpha' field before. It was always empty. At least I haven't seen it non-empty anywhere. And I haven't come across any explanations about what it is. In the screen output, you wrote 'phonebook name=$($item->"alpha")'. So, 'alpha' seems to be some kind of phonebook name. Can you suggest where I can read about this phonebook name and what it means?
Nothing to determine, is the GSM standard.
Just read the “concrete example code” on previous post "\r\n+CMGL: ", included the space, is what eactly separate fields (go to new line are just for forum formatting).
at the start is :local start -2 because the first record start with "+CMGL: " that have 2 less characters: \r\n
“\r\nOK” is the end of AT output, so must be removed at the end.
You do not have noticed the “OK” at the end of AT+CMGL on your tests?
this field is for the SIM that have in the internal SIM phonebook the number, also return the phonebook name… not all modem support that.
I did not have time to check, but does it find the first occurrence in this form?
If I output an AT command, then of course there is, but here in the script I did not notice it. This "OK" does not seem to be present when displaying the contents of the SMS.
That is, if I save a number on the SIM card, for example, John Carter, and receive SMS from this number, will this field be filled in as John Carter?
I have a request. Below I'm showing the code. It seems to be working and generates two arrays. The first array is the one I made yesterday, which contains variations of the CMGL phrase. And the second array is the body of the SMS. At the end, I print both arrays for verification. There are no spaces/gaps between elements of the first array. The next element starts right after the semicolon. But when I print the second array, there is a huge space between elements, and I don't understand where it comes from and if it's normal.
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]
:local smsArray [:toarray ""]
:local continue true;
while ( $continue ) do={
:set matchPosition [ :find $modemOutput $search1 $matchPosition ];
:if ( [ :typeof $matchPosition ] != "nil" ) do={
:set matchCount ( $matchCount + 1 );
:set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] )
:set matchPosition [ :find $modemOutput $search2 $matchPosition ];
:set smsArray ( $smsArray , [ :pick $modemOutput ( $matchPosition + 1 ) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
} else={ :set continue false; };
};
:put $arrayStat
:put $smsArray
If I do such a check, then these gaps are also clearly visible.
foreach key,sms in=$smsArray do={
:put "$key=$sms";
}
Even if I replace I will do this replacement:
:set smsArray ( $smsArray , [ :pick $modemOutput ( $matchPosition+1) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
to
:set smsArray ( $smsArray , [ :pick $modemOutput $matchPosition ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
I don't understand what you mean, in the first line only "+CMGL: " must be removed, because there is no \r\n at the start, so skip "\r\n+CMGL: " length - 2
Try with
:put ([ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" )
and you see the "OK" at the end.
It seems to me that I have already written everything clearly.
This is what I was talking about. In such a situation, I see "OK". But if I do this
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search "\r\nOK";
:local matchPosition -1;
:local matchCount 0;
:local continue true;
while ( $continue ) do={
:set matchPosition [ :find $modemOutput $search $matchPosition ];
:if ( [ :typeof $matchPosition ] != "nil" ) do={
:set matchCount ( $matchCount + 1 );
:put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
} else={ :set continue false; };
};
The search doesn't find anything.
The thing is, I'm not sure if Google translate is correct.
Later I will try to check and formulate my question in a different way to be more confident in the google translation.
Now I can say that I found an error in my version and now there are no extra characters in the array elements. The error is that I forgot that now two characters "\r\n" are participating in the search instead of one "\n", so the offset should be two characters. It turned out like this:
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]
:local smsArray [:toarray ""]
:local continue true;
while ( $continue ) do={
:set matchPosition [ :find $modemOutput $search1 $matchPosition ];
:if ( [ :typeof $matchPosition ] != "nil" ) do={
:set matchCount ( $matchCount + 1 );
:put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
:set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] )
:set matchPosition [ :find $modemOutput $search2 $matchPosition ];
:set smsArray ( $smsArray , [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
} else={ :set continue false; };
};
:put $arrayStat
:put "-----------";
:put $smsArray
And when I try to find only “OK”, it is found only once at the very end
@DyadyaGenya
If you still have troubles with parsing PDU, here is mine script if you find it useful:
sms-read.rsc (33.7 KB)
and sms2email script which is using it:
sms2email.rsc (1.31 KB)
sms-read script is mostly created from scripts at http://forum.mikrotik.com/t/script-for-sending-incoming-sms-to-mail-with-full-parsing/140125/1 with some mine modifications and code formatting.
Thanks, I’ll take a look anyway. But now I have a problem with arrays. I can’t figure out how to work with them.
Of course your code works. But I'm learning and trying to do the same and use my experience. Below is my attempt to do something similar without taking into account phonebook data. How to substitute them I understand. But I can't put the SMS text into the array. Although I can either place it in a separate array or print it.
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]
:local continue true;
while ( $continue ) do={
:set matchPosition [ :find $modemOutput $search1 $matchPosition ];
:if ( [ :typeof $matchPosition ] != "nil" ) do={
:set matchCount ( $matchCount + 1 );
:local tmpStat [:toarray [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] ]
:local subarray [:toarray ""]
:set ($subarray->"index" ) [:tonum ($tmpStat->0)]
:set ($subarray->"status") [:tonum ($tmpStat->1)]
:set ($subarray->"length") [:tonum ($tmpStat->2)]
:set matchPosition [ :find $modemOutput $search2 $matchPosition ];
#The line below is just for checking
:put [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ];
:local tmpStat2 [:toarray [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] ];
:set ($subarray->"sms" ) [:tonum ($tmpStat2->0)]
:set ($arrayStat->$matchCount) $subarray
} else={ :set continue false; };
};
:put $arrayStat
You want to do too many things at once without knowing how to code.
Even I who “get by” don’t do it this way (although I don’t post the intermediate steps).
The final goal must be divided into subproblems and only when the subproblem is solved, do we move on to the next one.
Problem 1)
Understand what is the final scope: “Have one array with mixed arrays and values that represent all the info possible on SMS inside the SIM”
*** done ***
Problem 2)
How to read the SMS (on unaltered default modem config) and put in one variable?
new problem 2a) after tests, the result can’t be directly one array
:put ([/interface lte at-chat lte1 wait=yes input=“AT+CMGL=4” as-value ]->“output”)
new problem 2b) and if I want to read only what I do not have already readed, etc.?
I can add one array that represent all possible parameters:
{
:local smsstatus [:toarray “REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL”]
:put ([/interface lte at-chat lte1 wait=yes input=“AT+CMGL=$[:find $smsstatus “ALL” -1]” as-value ]->“output”)
}
Problem 3)
Is not array, ok, now identifying the data sctructure.
The data sctructure usually is:
+CMGL: 0,1,48\r\n07[…]72\r\n+CMGL: 1,1,TOBY WORK,40\r\n07[…]41\r\n+CMGL: 2,1,38\r\n07[…]07\r\nOK
So, if contain at least one record, contain +CMGL
Everytime the data end with \r\nOK
on RouterOS the space is _
Each SMS start with “+CMGL_” and end with \r\n, so the separator between the SMS are “\r\n+CMGL_”,
except the first that start with two character less: “+CMGL_” so the length of “\r\n+CMGL_” -2
and the last that end with “\r\nOK”
Thw “wile” must be executed until “\r\n+CMGL_” or “\r\nOK” are present [NOT until I set inside one variable to something…]
With all this info just create a cycle that split raw SMS reply:
{
:local smsstatus [:toarray “REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL”]
:local txtSMSpdu ([/interface lte at-chat lte1 wait=yes input=“AT+CMGL=$[:find $smsstatus “ALL” -1]” as-value ]->“output”)
:local endsrch1 "\r\n+CMGL: " ; :local endsrch2 “\r\nOK”
:local start -2; :local tmppos 0 ; :local index 0
:while (([:typeof [:find $txtSMSpdu $endsrch1 $start]] = “num”) or ([:typeof [:find $txtSMSpdu $endsrch2 $start]] = “num”)) do={
:if ([:typeof [:find $txtSMSpdu $endsrch1 $start]] = “num”) do={
:set tmppos [:find $txtSMSpdu $endsrch1 $start]
} else={
:set tmppos [:find $txtSMSpdu $endsrch2 $start]
}
:put [:pick $txtSMSpdu ($start + [:len endsrch1] + 1) $tmppos]
:set index ($index + 1)
:set start $tmppos
}
}
Problem 4)
split each value (already separated with ,) inside one array of values
*** to be tested later ***: if the phonebook entry already containing “,” parser do wrong result
for example splitting “0,1,“CARLO”,48\r\n07[…]72” on one array:
{0 ; 1 ; “CARLO” ; 48\r\n07[…]72 }
BUT, if the array do not containing inside the ponebook name, RouterOS ignore the doube “,”
{0 ; 1 ; 48\r\n07[…]72 }
{
:local smsstatus [:toarray “REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL”]
:local txtSMSpdu ([/interface lte at-chat lte1 wait=yes input=“AT+CMGL=$[:find $smsstatus “ALL” -1]” as-value ]->“output”)
:local endsrch1 "\r\n+CMGL: " ; :local endsrch2 “\r\nOK”
:local start -2; :local tmppos 0 ; :local index 0
:while (([:typeof [:find $txtSMSpdu $endsrch1 $start]] = “num”) or ([:typeof [:find $txtSMSpdu $endsrch2 $start]] = “num”)) do={
:if ([:typeof [:find $txtSMSpdu $endsrch1 $start]] = “num”) do={
:set tmppos [:find $txtSMSpdu $endsrch1 $start]
} else={
:set tmppos [:find $txtSMSpdu $endsrch2 $start]
}
:local tmparray [:toarray [:pick $txtSMSpdu ($start + [:len endsrch1] + 1) $tmppos]]
:put $tmparray
:set index ($index + 1)
:set start $tmppos
}
}
Problem 5a)
Manage the different length of the returned array if phonebook entry is not present.
So, if the length of obained array is 3, the name is missing, if is 4 is present.
If length is 3, add a 4th empty value o array.
Create one array that map the values to the right position.
5b) Also the length and PDU part are on the same variable. What divide the length and the PDU is just “\r\n”, so just split at it.
5c) Add a print cycle at the end for display the values.
Full code here:
https://forum.mikrotik.com/viewtopic.php?p=1004891#p1004540
The previous problems are solved, except test the script on one SIM with one received SMS from one name phonebook containing “,” in name.
You are at the
Problem 6)
create one SEPARATE function $PDU2arr to be putted here:
:set ($subarray->“PDU” ) [$PDU2arr [:tostr [:pick ($tmparray->$idx3) ([:find ($tmparray->$idx3) “\r\n” -1] + 2) [:len ($tmparray->$idx3)]]] ]
for add one array of PDU values.
Problem 6) Subproblem A)
Create one function $PDU2arr that split each SMS part and return it as array.
example:
[$PDU2arr “07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72”]
return one array containing
{
“smsc”=“07912180958739F1”;
“type”=“04”; [this determines what other fields there will be after, for type=“04” they are:]
“sender”=“0B917120069876F0”;
“extra-data”=“00”;
“aplhabet-class-and-others”=“00”;
“date”=“9140503223218A”;
“timezone”=“8A”;
“length”=“21”;
“message”=“D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72”
}
Problem 6) Subproblem B)
decode the values for return instead one aray of decoded vallues:
{
“smsc”=“+12085978931”;
“type”=“SMS-DELIVER”; [this determines what other fields there will be after, for type=“SMS-DELIVER” they are:]
“sender”=“+17026089670”;
“class”=“standard”; [from aplhabet-class-and-others]
“aplhabet”=“GSM-7”; [from aplhabet-class-and-others]
…
… (some other decoded values from aplhabet-class-and-others and extra-data are possible, but I do not studied the full list)
… for example “SMS ID” and “SMS 1 of 2” for SMS with lenght > 160 characters.
…
“date”=“19-04-05 23:32:12”
“timezone”=-28"; (is GMT-7, so 28 times -15 minutes)
“length”=21;
“message”=“Test message to AT Command Tester”
}
Apparently all the same Google translator translates incorrectly. You misunderstood me. I'm just trying to do it step by step. And I built the steps for myself in my head approximately the same as you described. In my last post, I complained that I was missing one step. It was approximately at your step 3. Now I made it and went through all the steps that you wrote for step 6.
Here is my version of the code, how do I get to step 6
:local smsStatus [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]
:local continue true;
while ( $continue ) do={
:set matchPosition [ :find $modemOutput $search1 $matchPosition ];
:if ( [ :typeof $matchPosition ] != "nil" ) do={
:local tmpStat [:toarray [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] ]
:local subarray [:toarray ""]
:if ( [:len $tmpStat]=3 ) do={
:set ($subarray->"index" ) [:tonum ($tmpStat->0)]
:set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
:set ($subarray->"length") [:tonum ($tmpStat->2)]
:set ($subarray->"alpha") ("")
} else={
:set ($subarray->"index" ) [:tonum ($tmpStat->0)]
:set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
:set ($subarray->"length") [:tonum ($tmpStat->3)]
:set ($subarray->"alpha") [:tonum ($tmpStat->2)]
}
:set matchPosition [ :find $modemOutput $search2 $matchPosition ];
# :put [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ];
:set ($subarray->"sms" ) ( [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] )
:set ($arrayStat->$matchCount) $subarray
:set matchCount ( $matchCount + 1 );
} else={ :set continue false; };
};
:put $arrayStat
:put "-----------";
Now I can split the body of the SMS into parts, select ASC, TP-MTI & Co, TP-UDHI and other parts. And already further to do the transcoding of the SMS text.
Please see my version.
You need to decode PDU line from sms key in $arrayStat values map from offsets described at post #23 or see here: https://www.gsmfavorites.com/documents/sms/pdutext/. A lot of work is a front of you if you want to create all from scratch (there is a 1077 lines in sms-read script!). In that script this is handled by several functions, $extracted array contains PDU line for each sms (like from your array under value sms key), examine rest of a code from that point (
:local extracted [$smsReadExtractPDUs]
) and you will get a picture what all you need to implement decoding…