PDU Fields for incoming SMS

This is just a starting point for one complete function, on this function you have all needed example for decode everything inside.

Updated Code here:
https://forum.mikrotik.com/viewtopic.php?p=1006044#p1006044

I know people who write programs / applications perfectly, but do not understand bits and bytes. A program is just an algorithm and syntax, and it doesn't care what information to process, even a command to brew coffee, even count bytes. Programmers don't know how to do barista work, but they can make a program to make a coffee machine work. Or a more complex example with drones. A man does not fly, but he can write a program for a drone. However, he does not need to count bytes. Only if they set such a task.

That is, if the length is 16, then it will be 08, and if the length is 12, then it will be 06?

And if I understand your example correctly, then "10" and "91" will be 02. Right?

You never said what 48 is on a line

:put [$PDUdecode 07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72 48]
}

From the previous messages I did not understand where it came from. I understand that it has something to do with the length of the message.
And maybe I somehow use your function in a wrong way, but none of my messages are read. Basically writes about the length error from the first condition: "ERROR: Provided Length does not match SMS PDU part"

Also interested in this line:

    :local length [:tonum "0$2"]

Do I understand correctly that this is the second argument in the function and is somehow related to the number 48?
Can you elaborate on the syntax of this argument?

How many time same question?
http://forum.mikrotik.com/t/cant-turn-code-into-a-function/167087/1



Again: The length at the end is optional, if specified the validity of the PDU is calculated.
http://forum.mikrotik.com/t/cant-turn-code-into-a-function/167087/1

if the parameter is provided, length is, on the example, “048” and converted to one number is 48
if the parameter is no provided, length is “0” on this way and converted to one number is 0
if the “0” is not added on front of $2, the conversion from empty string “” to one number do not produce 0, but nothing.
If you do not know waht $1, $2, etc, are inide one function, at least deign to read what already exists on scripting.

Then, you can't produce anything useful for the PDU, because it contains parts in hexadecimal, parts in mixed and parts in binary...

If programmers don't know the subject of what they are programming, maybe they will eventually do something that works, but they will never come up with a program worth calling it that name...

Sorry, I don't have time to read all the replies in different threads. A few hours ago I read there and answered. Now it's time to read this thread.

Sometimes I am amazed at the quality of Google translation. He translates so poorly that you, a smart person, did not see my clarifications about the $2 argument in my question. I made the assumption that it was him. But I did not understand its syntax, and what it means. But thanks for your clarification. Now I will know that you can merge a line not only with a dot, but also in this way.

Someone doesn't understand someone. Again I will complain about Google translation. I clearly said that a programmer should know what he is programming, but not know everything from the very beginning, including bytes. Here is my quote:

Thus, if a programmer needs to write a program for a coffee machine, he studies the coffee machine. If he needs to write a program for a machine that milks cows, he studies the process of milking cows. If he is told to write a bit-transcoding program or some other deciphering technique, he learns that technique. But the programmer is not required to initially know all this. That's why I'm trying to find people who, in a nutshell, as an ordinary person from the street, will explain to me what bytes are and how they are read.
Perhaps in the future, based on your explanations, I will turn to other people, but at the moment I am looking for basic knowledge.
Thus, I study the subject that I intend to program. This is exactly what you said.

I have tried reading this post several times. But there is too much information for those who know what bytes are and how they are recalculated. But there is no clear explanation for the common man. I tried looking at your sample code you showed in another thread. I see that you use an offset to determine the PDU type, but I do not understand what is the start of the offset and why such an offset. You could explain in simple terms how to determine the type of PDU. At the moment, the only thing I seem to understand is that “04” is SMS-DELIVER. Maybe there is some set of ready-made values?

In addition to the article you mentioned, I have two more schemes, but I do not understand what they mean and how to work with them.
SMS_type 2.jpg
SMS_type.jpg

@DyadyaGenya: for what it’s worth, ChatGPT is way better than Google Translate at understanding important nuances of the language which is also the real strength of a language modell like GPT (as opposed to facts that shouldn’t be trusted at all).

Okay, I’ll try to register in the GPT chat today

Just don’t post ChatGPT generated code here, rextended will not love you after that :slight_smile:

I doubt that I will quickly figure out how this chatGPT works. Moreover, I heard that it generates code poorly, especially if the programming language is rare.
But I have a request for you, which I voiced above. Tell me how to work with those tables that I attached above. I also found this document. Perhaps you are already familiar with it, although there is more information, but it is written even more intricately.

https://www.etsi.org/deliver/etsi_gts/03/0340/05.03.00_60/gsmts_0340v050300p.pdf

For example, I know for sure that this snippet ‘4404810102’ contains the SMS type, the length of the sender’s number, the number type, and the actual number. I know for sure that the sender’s number is ‘1020’ and the number type is ‘81’. But how is its length encrypted? It seems like it has 4 characters, so it should be ‘04’. However, according to the logic used for the SCA number, it should include 14 characters, including the number type indicator. But here, it seems that only the length of the number is specified without considering the number type characters.

Or, for example, I suspect that in this snippet ‘400DD04669B1487C3601’, the sender’s number is encrypted in alphanumeric form. The ‘40’ could represent the SMS type, and ‘0D’ the length of the SMS. But how many characters does it correspond to?

44 04 81 0102

0x44 is the SMS type

length “0x04” = 4
because 0x04 are 4 also on decimal and 4 mean four characters

0x81 is the type already explained here http://forum.mikrotik.com/t/pdu-fields-for-incoming-sms/167111/1 and is “unknown - national numbering plan”
because 1020 is one internal number of the operator.

and if you count one for each digit 1, 0, 2 and 0 are four digit, and is the same amount of digit of that 0x04 indicate…


\


about 40 0D D0 4669B1487C3601


0x40 type
0x0D on decimal is… 13 because for store seven 7-bit GSM-7 characters are needed 13 nibble (half byte), [for store 7 bit, each 8 bytes, one less byte is required]
and because the nibble can’t be one odd number, the last nibble is unused, but is still required 13+1 nibble = 7 bytes and 46/69/B1/48/7C/36/01 is 7 bytes long…
0xD0 is the coding: binary 1101 - 0000 = PDU encoded name with GSM-7 alphabet
4669B1487C3601 = 0x64961B84C76310 = binary 0110010010010110000110111000010011000111011000110001**0000** (notice the unused last nibble / 4 bits)
and the 7-bit 0x64 0x96 0x1B 0x84 0xC7 0x63 0x10 converted to 8-bit GSM-7 is 0x46 0x52 0x45 0x45 0x44 0x4F 0x4D,
and 8-bit GSM-7 0x46 0x52 0x45 0x45 0x44 0x4F 0x4D converted to readable CP 1252 is…
FREEDOM

I have already answered you in a precise way, it would be enough for you to read,
but if you don’t know the terms “bytes” I understand that you don’t know what it means…
http://forum.mikrotik.com/t/pdu-fields-for-incoming-sms/167111/1

Now let’s talk about the sender’s address field (TP-OA). Although it may resemble the SCA number at first glance, there are certain aspects that are not clear:

  1. Why does the same length of the SCA number get encoded as “07,” while for the sender’s address it is “0B”?

On the smsc the first “07” is the needed bytes, on the senders are the length of the number, not the bytes…

Refreshed, added some data controls and simplified.
{
:local PDUdecode do={
:local pdu [:tostr $1]
:local length [:tonum “0$2”]
:local return [:toarray “”]

:if (!($pdu~"^[0-9A-F]*\$")) do={:error "ERROR: Only 0-9 and A-F are valid hexadecimal PDU digits."}

:if (([:len $pdu] % 2) != 0) do={:error "ERROR: PDU length must be even."}

:local skip (2 + ([:tonum "0x$[:pick $pdu 0 2]"] * 2))
:if ($length > 0) do={
    :local PDUlength (([:len $pdu] - $skip) / 2)
    :if ($length != $PDUlength) do={:return "ERROR: Provided Length does not match SMS PDU part."}
    :set ($return->"PDUlength") $PDUlength
}

:local invertnibble do={
    :local string [:tostr $1]
    :local positn 0
    :local output ""
    :while (([:len $string] + 1) > $positn) do={
        :set output "$output$[:pick $string ($positn + 1) ($positn + 2)]$[:pick $string $positn ($positn + 1)]"
        :set positn ($positn + 2)
    }
    :return $output
}

# BEGIN SMSC section
:local smsc    "<SIM>"
:local type    "<SIM>"
:local smscLen [:tonum "0x$[:pick $pdu 0 2]"]
:if ($smscLen != 0) do={
    :set type [:tonum "0x$[:pick $pdu 2 4]"]
    # 0x91 = 145
    :if ($type = 145) do={
        :set smsc "+$[$invertnibble [:pick $pdu 4 $skip]]"
        :if ($smsc~"F\$") do={:set smsc [:pick $smsc 0 ([:len $smsc] - 1)]}
    } else={
        :return "ERROR: Unexpected SMSC number type (0x[:pick $pdu 2 4])"
    }
}
:set ($return->"smsc"    ) $smsc
:set ($return->"smscType") $type
# END SMSC section

# BEGIN SMS Type section
:local type [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
# 0x04 = 4
:if ($type = 4) do={
    :set ($return->"SMStype") $type
} else={
    :return "ERROR: Unexpected SMSC number type (0x$[:pick $pdu $skip ($skip + 2)])"
}
:set skip ($skip + 2)
# END SMS Type section

# BEGIN SENDER section
:local sender    ""
:local senderLen [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
:if (($senderLen % 2) = 1) do={:set senderLen ($senderLen + 1)}
:local type [:tonum "0x$[:pick $pdu ($skip + 2) ($skip + 4)]"]
# 0x91 = 145
:if ($type = 145) do={
    :set sender "+$[$invertnibble [:pick $pdu ($skip + 4) ($skip + 4 + $senderLen)]]"
    :if ($sender~"F\$") do={:set sender [:pick $sender 0 ([:len $sender] - 1)]}
} else={
    :return "ERROR: Unexpected SENDER number type (0x$[:pick $pdu ($skip + 2) ($skip + 4)])"
}
:set ($return->"sender"    ) $sender
:set ($return->"senderType") $type
:set skip ($skip + 4 + $senderLen)
# END SENDER section

# BEGIN PROTOCOL section
:local protocol ""
:local type [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
# 0x00 = 0
:if ($type = 0) do={
    :set protocol $type
} else={
    :return "ERROR: Unexpected PROTOCOL number type (0x$[:pick $pdu $skip ($skip + 2)])"
}
:set ($return->"protocol") $protocol
:set skip ($skip + 2)
# END PROTOCOL section

# BEGIN CLASS section
:local class ""
:local type [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
# 0x00 = 0
:if ($type = 0) do={
    :set class $type
} else={
    :return "ERROR: Unexpected CLASS number type (0x$[:pick $pdu $skip ($skip + 2)])"
}
:set ($return->"class") $class
:set skip ($skip + 2)
# END CLASS section

# BEGIN DATETIME section
:local datetime [$invertnibble [:pick $pdu $skip ($skip + 12)]]
:if ([:tonum "$[:pick $datetime 0 2]"] > 39) do={:set datetime "19$datetime"} else={:set datetime "20$datetime"}
:local separators {"";"-";"-";" ";":";":"}
:local positn -1
:local output ""
:while (13 > $positn) do={
    :set output "$output$[:pick $datetime ($positn + 1) ($positn + 2)]"
    :if (($positn % 2) = 0) do={:set output "$output$($separators->($positn / 2))"}
    :set positn ($positn + 1)
}
:set ($return->"datetime") $output
:set skip ($skip + 12)
# END DATETIME section

# BEGIN TIMEZONE section
:local tza [:tonum "0x$[:pick $pdu ($skip + 1) ($skip + 2)]"]
:local tzb [:tonum "0x$[:pick $pdu $skip       ($skip + 1)]"]
:local tzs "+" ; :if (($tza & 0x8) = 0x8) do={:set tzs "-"}
:set tza ($tza & 0x7)
:local tzmin ((($tza * 10) + $tzb) * 15)
:set ($return->"tzminutes") [:tonum "$tzs$tzmin"]
:local tzhou ($tzmin / 60) ; :if ($tzhou < 10) do={:set tzhou "0$tzhou"}
:set   tzmin ($tzmin % 60) ; :if ($tzmin < 10) do={:set tzmin "0$tzmin"}
:set ($return->"timezone") "GMT$tzs$tzhou:$tzmin"
:set skip ($skip + 2)
# END TIMEZONE section

# BEGIN SMSlength section
:local SMSlength [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
:set ($return->"SMSlength") $SMSlength
:set skip ($skip + 2)
# END SMSlength section

# BEGIN SMSPDU section
:local smspdu [:pick $pdu $skip [:len $pdu]]
:set ($return->"SMSPDU") $smspdu
# END SMSPDU section

:return $return

}

:put [$PDUdecode 07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72 48]
}
The length at the end is optional, if specified, the validity of the PDU is calculated.

Fortunately, I found an easy way to convert bits into regular numbers. It turns out that in MikroTik, there is a function for that:

:put [ :tonum ( "0x" . "bits" ) ]

It seems that I would like to have a more comprehensive understanding of the date field. On the surface, it should consist of 14 characters representing the year, month, day, hour, minute, second, and time zone offset. However, I have come across an example where the same operator used different sequences and time zone offsets within a one-year interval. Interestingly, the PDUspy program correctly identifies the date but shows different time zone offsets. For example, two dates are provided: 22112051314180 and 32501031613521. Ideally, the first date should be interpreted as 22/11/02 15:13:41 with an 08 time zone offset, and the second date as 23/05/01 13:16:53 with a 12 time zone offset. However, according to the PDUspy program, the first date is displayed as 02.11.22 15:13 GMT+2.00, and the second date as 01.05.23 13:16 GMT+3.00. Both dates are considered correct by the program.
How do you actually convert the time zone?

There is another aspect that I don’t fully understand. It’s the SMS text header, which contains information about the encoding of the SMS, the number of parts it consists of, and which part is included in the current segment.

They don’t make anything easy. Section 9.2.3.11 of the Rel 16 SMS spec:

9.2.3.11 TP-Service-Centre-Time-Stamp (TP-SCTS)
The TP-Service-Centre-Time-Stamp field is given in semi-octet representation, and represents the local time in the following way:
Year, Month, Day, Hour, Minute, Second, Time Zone (2 semi-octets each)
The Time Zone indicates the difference, > expressed in quarters of an hour, between the local time and GMT> . In the first of the two semi-octets, the first bit (bit 3 of the seventh octet of the TP-Service-Centre-Time-Stamp field) represents the algebraic sign of this difference (0: positive, 1: negative).

See https://www.etsi.org/deliver/etsi_ts/123000_123099/123040/16.00.00_60/ts_123040v160000p.pdf

Honestly, I would prefer not to deal with these octets. I would prefer a simpler explanation.

PDU are all octet, you want to take a bath without touch the water…

Create one array with all 256 possible values with corresponding real value…

To create, we need to understand why.

Found a pretty good article on the subject
http://www.doubleblak.com/m/blogPosts.php?id=6