Community discussions

MikroTik App
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

PDU Fields for incoming SMS

Thu Jun 01, 2023 7:19 pm

Hi all. I'm trying to figure out the PDU fields of the incoming SMS. On the Internet, there is a lot of any disordered information and it is scattered. It seems to have found several more or less informative schemes. But there is no description for them that is understandable to an ordinary person.
In the topic header (first message), I attach some of the most successful schemes in my opinion, which show the division of SMS into fields.
To make the messages easier to read, I will place the information for each field in a separate message. And if someone wants to add or comment on some field value, it will be more convenient to do this. And probably the ability to add information to a specific message will remain.
So the next post will be about TP-SCA.
You do not have the required permissions to view the files attached to this post.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 7:22 pm

If I understand correctly, the TP-SCA field consists of three parts: the length of the number, the type of number, and the actual number. There are a few questions that I don't understand:

1. If I understand correctly, TP-SCA can be absent and taken from the SIM card itself. In that case, it seems the first pair of digits will be "00". However, it's unclear how the entire number is encrypted and where it is displayed, if at all.
2. I read that the SCA number type can have several types, but I only found descriptions for 91 and 81. I also read that there are other types like
000 - unknown;
001 - international;
010 - national;
011 - network-specific;
100 - network subscriber type;
101 - alphanumeric;
110 - abbreviated;
111 - reserved.
However, I don't know how to translate bits, and it doesn't seem necessary to write a function for decoding such a small piece of information. I would like to understand how these encrypted data will be represented in human-readable form
Later, from this template, it would be possible to create an array and retrieve values from it.
3. I'm also unsure about the possible length of the SCA number. So far, I've encountered numbers with 11-12 characters. But perhaps someone knows other possibilities and how the first two digits, which are usually '07,' would appear in that case.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 7:36 pm

Now let's talk about the TP-MTI & Co. field. First of all, it's not entirely clear how to determine the SMS type. There seems to be an explanation: SMS-DELIVER, SMS-STATUS REPORT, SMS-SUBMIT REPORT, RESERVED. There's even a translation from bits to regular numbers. But I don't understand it yet. If someone can clarify, please explain which numbers will be used in this context.
In addition, there are TP-MMS, TP-SRI, TP-UDHI, TP-RP fields. Where are their data located? How do they appear in the digital representation for an ordinary person?
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 7:46 pm

Now let's talk about the sender's address field (TP-OA). Although it may resemble the SCA number at first glance, there are certain aspects that are not clear:
1. Why does the same length of the SCA number get encoded as "07," while for the sender's address it is "0B"?
2. What types of addresses can be used in this field?
3. How can we distinguish between a numeric address and an alphanumeric address?
4. And how can we determine the length of an alphanumeric address in terms of the number of characters allocated for it, rather than the number of bytes?
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 7:49 pm

Is the TP-PID field ever different from "00"? What other values can it have, how are they encoded, and how many characters do they occupy?
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 7:52 pm

TP-DCS field.
1. It seems that GSM-7 encoding is "00" and UCS2 encoding is "08". However, I have also seen "10" and "18". What are these encodings?
2. Are there any other types of encoding?
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 7:55 pm

Regarding the TP-SCTS field, I think I understand it. It is encoded similarly to phone numbers. The only clarification I need is whether there is another time format, and if so, how it is encoded and indicated as a different time format.
 
optio
Long time Member
Long time Member
Posts: 655
Joined: Mon Dec 26, 2022 2:57 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 7:56 pm

In addition, there are TP-MMS, TP-SRI, TP-UDHI, TP-RP fields. Where are their data located? How do they appear in the digital representation for an ordinary person?
https://www.activexperts.com/serial-por ... technical/ see how PDU Type is encoded.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 8:04 pm

The TP-UDL field.
1. Length of the user data, including the User Data Header if present.
2. If the SMS is encoded with a 7-bit encoding, this field indicates the number of characters in the message.
3. If the encoding is UCS2, then the field indicates the number of bytes in the message. Here, it is unclear how to count the characters. What exactly needs to be counted?
4. Did I understand correctly that to determine the length of the message and whether it is multipart, I need to refer to the values in the TP-MTI field?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 8:09 pm

1. If I understand correctly, TP-SCA can be absent and taken from the SIM card itself.
Only for sended messages, not for received

it's unclear how the entire number is encrypted and where it is displayed,
you have already your reply, and is not encrypted anything at all
viewtopic.php?t=196637#p1005497

2. I read that the SCA number type can have several types, but I only found descriptions for 91 and 81. I also read that there are other types like....
0x91 and 0x81 do not exist (everytime use hex when the number are hex...)
are BIT 10010001 and 10000001 and must be checked what each bit is.

However, I don't know how to translate bits, and it doesn't seem necessary to write a function for decoding such a small piece of information.
I would like to understand how these encrypted data will be represented in human-readable form
Is not encrypted, and you are forced to check BIT by BIT or you need one "if" function for each 256 possibilities....
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 8:11 pm

If you notice, one of my screenshots is taken from there. Unfortunately, I do not understand all the information in this article. And first of all, how bits are translated. That's why I asked to show what bit options are when they are arranged as ordinary numbers in the PDU text
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 8:16 pm

Now let's talk about the sender's address field (TP-OA). Although it may resemble the SCA number at first glance, there are certain aspects that are not clear:
1. Why does the same length of the SCA number get encoded as "07," while for the sender's address it is "0B"?
On the smsc the first "07" is the needed bytes, on the senders are the length of the number, not the bytes...

There are some technical reasons behind these choices, like to "shuffle the number" or instead of converting the number +12356789 to BC8CB5 hexadecimal which also takes up less space, they chose that way.

But it is based on the computing power of the networks and terminals of the time, but the basic standard of GSM, over time, has remained the same...
So some things now trivial seem incomprehensible...

4. And how can we determine the length of an alphanumeric address in terms of the number of characters allocated for it, rather than the number of bytes?
It's obvious... 0x07 is 7 bytes and 0x0B are 11 characters, because 0x0B = 11...
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 8:19 pm

0x91 and 0x81 do not exist (everytime use hex when the number are hex...)
are BIT 10010001 and 10000001 and must be checked what each bit is.
How does it not exist? It's even mentioned in the link above. Here is a screenshot from the article.
Is not encrypted, and you are forced to check BIT by BIT or you need one "if" function for each 256 possibilities....
Why do this if there are only a few such values? Perhaps even one value. In addition to "07" I have not yet met.
You do not have the required permissions to view the files attached to this post.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 8:28 pm

0x91 and 0x81 do not exist (everytime use hex when the number are hex...)
are BIT 10010001 and 10000001 and must be checked what each bit is.
I was referring to the fact that 81 and 91 don't exist because they are hexadecimal numbers, not decimals.
As I have written several times, put something before or after the number to understand the difference... 0x81 / 81H / hex 81, etc.
How does it not exist? It's even mentioned in the link above. Here is a screenshot from the article.
1) The article is incomplete
2) I'm referring that

Is not encrypted, and you are forced to check BIT by BIT or you need one "if" function for each 256 possibilities....
Why do this if there are only a few such values? Perhaps even one value. In addition to "07" I have not yet met.
1) The article is incomplete, for example missing the alphanumeric value, you notice some SMS that coming from one alphanumeric string?
2) 07 are from 11 to 12 numbers, and the are usual number size for SMSCenter, but if the SMSCenter have 9 or 10 numbers, is 06, and so on.


You can't draw the conclusion that there aren't others, just because an article (which among other things, if you haven't noticed, call the complete rules)
doesn't mention them or because you don't know them.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 8:45 pm

10.5.4.7 (not 6, the article is also wrong)

Possible plausible values used only on SMS (read the full guide to understand what are)...
10010001 0x91 international format - ISDN / standard telephony
10100001 0xA1 national format - ISDN / standard telephony
10110001 0xB1 operator specific number - ISDN / standard telephony
11000001 0xC1 dedicated access short number - ISDN / standard telephony
10011000 0x98 international format - national only number
10101000 0xA8 national format - national only number
10111000 0xB8 operator specific number - national only number
11001000 0xC8 dedicated access short number - national only number
10011001 0x99 international format - mobile operator only number
10101001 0xA9 national format - mobile operator only number
10111001 0xB9 operator specific number - mobile operator only number
11001001 0xC9 dedicated access short number - mobile operator only number
..........................
0x81 is 10000001 = unknown - national numbering plan


The possible total not-reserved values combination for that field are ~48. Approx, not counted exactly.
So, must be checked every BIT, or create one array of 48 or more values are one nonsense...
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 9:54 pm

you have already your reply, and is not encrypted anything at all
viewtopic.php?t=196637#p1005497
I haven't looked at this thread yet. After your post, I did it. But apparently again my google translator is to blame. It was about a number consisting of letters, ie like "InternetSMS" or "Freedom".

0x91 and 0x81 do not exist (everytime use hex when the number are hex...)
are BIT 10010001 and 10000001 and must be checked what each bit is.
Unfortunately I don't understand when hex is used
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 10:01 pm

It's obvious... 0x07 is 7 bytes and 0x0B are 11 characters, because 0x0B = 11...
Unfortunately, it's not obvious to me. And first of all, it is not clear when bytes are used, when something else is used. The length of the number in both cases is 11 characters. At the 12th character, we add "F" to an even number in both cases. But at the same time, for SCA, the length will be displayed as "07", and for the sender's number "0B". Perhaps because I do not understand bytes, but perhaps it is Google that translates in such a way that it cannot convey the meaning of what was said.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 10:47 pm

10.5.4.7 (not 6, the article is also wrong)
I don't understand what article you are talking about. Perhaps there is described work with text numbers of the sender.
The possible total not-reserved values combination for that field are ~48. Approx, not counted exactly.
So, must be checked every BIT, or create one array of 48 or more values are one nonsense...
In any case, even if we count every bit, and at the same time we want to somehow use this information, we will need an array of values ​​in order to understand what to do with this information next. If there is no value in the array, we will have to do some default action so that the code execution does not interrupt. If we assume that we will use a maximum of two options: 91 or 81, then we also do not need to count the bits, but only make two conditions. So I don't see the point in counting bits when executing a script anyway. It is better to write down/remember the values ​​once and work with them further.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 01, 2023 11:46 pm

It's obvious... 0x07 is 7 bytes and 0x0B are 11 characters, because 0x0B = 11...
That is, "07" is not the length of characters, but bytes? Does "0B" count characters, not bytes?
If so, am I right in thinking that 7 bytes means 7 pairs of characters? That is, one pair of characters - the type of number, and 6 pairs - the length of the number? Ie 11-12 characters.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 3:12 am

It's obvious... 0x07 is 7 bytes and 0x0B are 11 characters, because 0x0B = 11...
That is, "07" is not the length of characters, but bytes? Does "0B" count characters, not bytes?
If so, am I right in thinking that 7 bytes means 7 pairs of characters? That is, one pair of characters - the type of number, and 6 pairs - the length of the number? Ie 11-12 characters.
I understand not knowing the programming language, but at least what is a Byte you should know...

A Byte consists of 8 bits ranging from 00000000 to 11111111, because bits can only have 2 values, either 0 or 1,
this so that there are 256 possible combinations,
whose value ranges from 0 to 255, in decimal,
and in hexadecimal it is represented by the combinations of symbols, in increasing order of value, 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F
(conceptually it doesn't matter if uppercase or lowercase)

0x is written in front to avoid ambiguity when using only symbols that could be mistaken for decimals (or even binary), for example how much is "10"? or "91"?
therefore the possible values of a Byte, in hexadecimal, range from 00 to FF,
and that's why 0x07 represents 7 Bytes, which obviously are represented by 14 characters because every single Bytes is represented with two.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 5:45 am

This is just a starting point for one complete function, on this function you have all needed example for decode everything inside.

PDU decode code

..............
Updated Code here:
viewtopic.php?p=1006044#p1006044
Last edited by rextended on Fri Jun 02, 2023 11:35 pm, edited 3 times in total.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 12:04 pm

I understand not knowing the programming language, but at least what is a Byte you should know...
I know people who write programs / applications perfectly, but do not understand bits and bytes. A program is just an algorithm and syntax, and it doesn't care what information to process, even a command to brew coffee, even count bytes. Programmers don't know how to do barista work, but they can make a program to make a coffee machine work. Or a more complex example with drones. A man does not fly, but he can write a program for a drone. However, he does not need to count bytes. Only if they set such a task.
0x is written in front to avoid ambiguity when using only symbols that could be mistaken for decimals (or even binary), for example how much is "10"? or "91"?
therefore the possible values of a Byte, in hexadecimal, range from 00 to FF,
and that's why 0x07 represents 7 Bytes, which obviously are represented by 14 characters because every single Bytes is represented with two.
That is, if the length is 16, then it will be 08, and if the length is 12, then it will be 06?

And if I understand your example correctly, then "10" and "91" will be 02. Right?
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 12:12 pm

This is just a starting point for one complete function, on this function you have all needed example for decode everything inside.
You never said what 48 is on a line
:put [$PDUdecode 07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72 48]
}
From the previous messages I did not understand where it came from. I understand that it has something to do with the length of the message.
And maybe I somehow use your function in a wrong way, but none of my messages are read. Basically writes about the length error from the first condition: "ERROR: Provided Length does not match SMS PDU part"

Also interested in this line:
    :local length [:tonum "0$2"]
Do I understand correctly that this is the second argument in the function and is somehow related to the number 48?
Can you elaborate on the syntax of this argument?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 2:17 pm

You never said what 48 is on a line
:put [$PDUdecode 07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72 48]
}
From the previous messages I did not understand where it came from. I understand that it has something to do with the length of the message.
How many time same question?
viewtopic.php?t=196637#p1005859


Also interested in this line:
    :local length [:tonum "0$2"]
Do I understand correctly that this is the second argument in the function and is somehow related to the number 48?
Can you elaborate on the syntax of this argument?
Again: The length at the end is optional, if specified the validity of the PDU is calculated.
viewtopic.php?t=196637#p1005497

if the parameter is provided, length is, on the example, "048" and converted to one number is 48
if the parameter is no provided, length is "0" on this way and converted to one number is 0
if the "0" is not added on front of $2, the conversion from empty string "" to one number do not produce 0, but nothing.
If you do not know waht $1, $2, etc, are inide one function, at least deign to read what already exists on scripting.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 2:23 pm

I understand not knowing the programming language, but at least what is a Byte you should know...
I know people who write programs / applications perfectly, but do not understand bits and bytes. A program is just an algorithm and syntax, and it doesn't care what information to process, even a command to brew coffee, even count bytes. Programmers don't know how to do barista work, but they can make a program to make a coffee machine work. Or a more complex example with drones. A man does not fly, but he can write a program for a drone. However, he does not need to count bytes. Only if they set such a task.
Then, you can't produce anything useful for the PDU, because it contains parts in hexadecimal, parts in mixed and parts in binary...

If programmers don't know the subject of what they are programming, maybe they will eventually do something that works, but they will never come up with a program worth calling it that name...
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 4:27 pm

How many time same question?
Sorry, I don't have time to read all the replies in different threads. A few hours ago I read there and answered. Now it's time to read this thread.
If you do not know waht $1, $2, etc, are inide one function, at least deign to read what already exists on scripting.
Sometimes I am amazed at the quality of Google translation. He translates so poorly that you, a smart person, did not see my clarifications about the $2 argument in my question. I made the assumption that it was him. But I did not understand its syntax, and what it means. But thanks for your clarification. Now I will know that you can merge a line not only with a dot, but also in this way.
If programmers don't know the subject of what they are programming, maybe they will eventually do something that works, but they will never come up with a program worth calling it that name...
Someone doesn't understand someone. Again I will complain about Google translation. I clearly said that a programmer should know what he is programming, but not know everything from the very beginning, including bytes. Here is my quote:
Only if they set such a task.
Thus, if a programmer needs to write a program for a coffee machine, he studies the coffee machine. If he needs to write a program for a machine that milks cows, he studies the process of milking cows. If he is told to write a bit-transcoding program or some other deciphering technique, he learns that technique. But the programmer is not required to initially know all this. That's why I'm trying to find people who, in a nutshell, as an ordinary person from the street, will explain to me what bytes are and how they are read.
Perhaps in the future, based on your explanations, I will turn to other people, but at the moment I am looking for basic knowledge.
Thus, I study the subject that I intend to program. This is exactly what you said.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 5:07 pm

In addition, there are TP-MMS, TP-SRI, TP-UDHI, TP-RP fields. Where are their data located? How do they appear in the digital representation for an ordinary person?
https://www.activexperts.com/serial-por ... technical/ see how PDU Type is encoded.
I have tried reading this post several times. But there is too much information for those who know what bytes are and how they are recalculated. But there is no clear explanation for the common man. I tried looking at your sample code you showed in another thread. I see that you use an offset to determine the PDU type, but I do not understand what is the start of the offset and why such an offset. You could explain in simple terms how to determine the type of PDU. At the moment, the only thing I seem to understand is that "04" is SMS-DELIVER. Maybe there is some set of ready-made values?

In addition to the article you mentioned, I have two more schemes, but I do not understand what they mean and how to work with them.
You do not have the required permissions to view the files attached to this post.
Last edited by DyadyaGenya on Fri Jun 02, 2023 5:19 pm, edited 1 time in total.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1025
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 5:16 pm

@DyadyaGenya: for what it's worth, ChatGPT is way better than Google Translate at understanding important nuances of the language which is also the real strength of a language modell like GPT (as opposed to facts that shouldn't be trusted at all).
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 5:22 pm

@DyadyaGenya: for what it's worth, ChatGPT is way better than Google Translate at understanding important nuances of the language which is also the real strength of a language modell like GPT (as opposed to facts that shouldn't be trusted at all).
Okay, I'll try to register in the GPT chat today
 
optio
Long time Member
Long time Member
Posts: 655
Joined: Mon Dec 26, 2022 2:57 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 6:29 pm

Okay, I'll try to register in the GPT chat today
Just don't post ChatGPT generated code here, rextended will not love you after that :)
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 7:00 pm

Just don't post ChatGPT generated code here, rextended will not love you after that :)
I doubt that I will quickly figure out how this chatGPT works. Moreover, I heard that it generates code poorly, especially if the programming language is rare.
But I have a request for you, which I voiced above. Tell me how to work with those tables that I attached above. I also found this document. Perhaps you are already familiar with it, although there is more information, but it is written even more intricately.

https://www.etsi.org/deliver/etsi_gts/0 ... 50300p.pdf
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 8:14 pm

For example, I know for sure that this snippet '4404810102' contains the SMS type, the length of the sender's number, the number type, and the actual number. I know for sure that the sender's number is '1020' and the number type is '81'. But how is its length encrypted? It seems like it has 4 characters, so it should be '04'. However, according to the logic used for the SCA number, it should include 14 characters, including the number type indicator. But here, it seems that only the length of the number is specified without considering the number type characters.

Or, for example, I suspect that in this snippet '400DD04669B1487C3601', the sender's number is encrypted in alphanumeric form. The '40' could represent the SMS type, and '0D' the length of the SMS. But how many characters does it correspond to?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 8:51 pm

44 04 81 0102

0x44 is the SMS type

length "0x04" = 4
because 0x04 are 4 also on decimal and 4 mean four characters

0x81 is the type already explained here viewtopic.php?t=196747#p1005808 and is "unknown - national numbering plan"
because 1020 is one internal number of the operator.

and if you count one for each digit 1, 0, 2 and 0 are four digit, and is the same amount of digit of that 0x04 indicate...



************************
about 40 0D D0 4669B1487C3601
************************
0x40 type
0x0D on decimal is... 13 because for store seven 7-bit GSM-7 characters are needed 13 nibble (half byte), [for store 7 bit, each 8 bytes, one less byte is required]
and because the nibble can't be one odd number, the last nibble is unused, but is still required 13+1 nibble = 7 bytes and 46/69/B1/48/7C/36/01 is 7 bytes long...
0xD0 is the coding: binary 1101 - 0000 = PDU encoded name with GSM-7 alphabet
4669B1487C3601 = 0x64961B84C76310 = binary 01100100100101100001101110000100110001110110001100010000 (notice the unused last nibble / 4 bits)
and the 7-bit 0x64 0x96 0x1B 0x84 0xC7 0x63 0x10 converted to 8-bit GSM-7 is 0x46 0x52 0x45 0x45 0x44 0x4F 0x4D,
and 8-bit GSM-7 0x46 0x52 0x45 0x45 0x44 0x4F 0x4D converted to readable CP 1252 is....
FREEDOM

I have already answered you in a precise way, it would be enough for you to read,
but if you don't know the terms "bytes" I understand that you don't know what it means...
viewtopic.php?t=196747#p1005798
Now let's talk about the sender's address field (TP-OA). Although it may resemble the SCA number at first glance, there are certain aspects that are not clear:
1. Why does the same length of the SCA number get encoded as "07," while for the sender's address it is "0B"?

On the smsc the first "07" is the needed bytes, on the senders are the length of the number, not the bytes...
Last edited by rextended on Sat Jun 10, 2023 7:14 am, edited 1 time in total.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Fri Jun 02, 2023 11:33 pm

Refreshed, added some data controls and simplified.

PDU decode code

{
:local PDUdecode do={
    :local pdu    [:tostr $1]
    :local length [:tonum "0$2"]
    :local return [:toarray ""]

    :if (!($pdu~"^[0-9A-F]*\$")) do={:error "ERROR: Only 0-9 and A-F are valid hexadecimal PDU digits."}

    :if (([:len $pdu] % 2) != 0) do={:error "ERROR: PDU length must be even."}

    :local skip (2 + ([:tonum "0x$[:pick $pdu 0 2]"] * 2))
    :if ($length > 0) do={
        :local PDUlength (([:len $pdu] - $skip) / 2)
        :if ($length != $PDUlength) do={:return "ERROR: Provided Length does not match SMS PDU part."}
        :set ($return->"PDUlength") $PDUlength
    }

    :local invertnibble do={
        :local string [:tostr $1]
        :local positn 0
        :local output ""
        :while (([:len $string] + 1) > $positn) do={
            :set output "$output$[:pick $string ($positn + 1) ($positn + 2)]$[:pick $string $positn ($positn + 1)]"
            :set positn ($positn + 2)
        }
        :return $output
    }

    # BEGIN SMSC section
    :local smsc    "<SIM>"
    :local type    "<SIM>"
    :local smscLen [:tonum "0x$[:pick $pdu 0 2]"]
    :if ($smscLen != 0) do={
        :set type [:tonum "0x$[:pick $pdu 2 4]"]
        # 0x91 = 145
        :if ($type = 145) do={
            :set smsc "+$[$invertnibble [:pick $pdu 4 $skip]]"
            :if ($smsc~"F\$") do={:set smsc [:pick $smsc 0 ([:len $smsc] - 1)]}
        } else={
            :return "ERROR: Unexpected SMSC number type (0x[:pick $pdu 2 4])"
        }
    }
    :set ($return->"smsc"    ) $smsc
    :set ($return->"smscType") $type
    # END SMSC section

    # BEGIN SMS Type section
    :local type [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
    # 0x04 = 4
    :if ($type = 4) do={
        :set ($return->"SMStype") $type
    } else={
        :return "ERROR: Unexpected SMSC number type (0x$[:pick $pdu $skip ($skip + 2)])"
    }
    :set skip ($skip + 2)
    # END SMS Type section

    # BEGIN SENDER section
    :local sender    ""
    :local senderLen [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
    :if (($senderLen % 2) = 1) do={:set senderLen ($senderLen + 1)}
    :local type [:tonum "0x$[:pick $pdu ($skip + 2) ($skip + 4)]"]
    # 0x91 = 145
    :if ($type = 145) do={
        :set sender "+$[$invertnibble [:pick $pdu ($skip + 4) ($skip + 4 + $senderLen)]]"
        :if ($sender~"F\$") do={:set sender [:pick $sender 0 ([:len $sender] - 1)]}
    } else={
        :return "ERROR: Unexpected SENDER number type (0x$[:pick $pdu ($skip + 2) ($skip + 4)])"
    }
    :set ($return->"sender"    ) $sender
    :set ($return->"senderType") $type
    :set skip ($skip + 4 + $senderLen)
    # END SENDER section

    # BEGIN PROTOCOL section
    :local protocol ""
    :local type [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
    # 0x00 = 0
    :if ($type = 0) do={
        :set protocol $type
    } else={
        :return "ERROR: Unexpected PROTOCOL number type (0x$[:pick $pdu $skip ($skip + 2)])"
    }
    :set ($return->"protocol") $protocol
    :set skip ($skip + 2)
    # END PROTOCOL section

    # BEGIN CLASS section
    :local class ""
    :local type [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
    # 0x00 = 0
    :if ($type = 0) do={
        :set class $type
    } else={
        :return "ERROR: Unexpected CLASS number type (0x$[:pick $pdu $skip ($skip + 2)])"
    }
    :set ($return->"class") $class
    :set skip ($skip + 2)
    # END CLASS section

    # BEGIN DATETIME section
    :local datetime [$invertnibble [:pick $pdu $skip ($skip + 12)]]
    :if ([:tonum "$[:pick $datetime 0 2]"] > 39) do={:set datetime "19$datetime"} else={:set datetime "20$datetime"}
    :local separators {"";"-";"-";" ";":";":"}
    :local positn -1
    :local output ""
    :while (13 > $positn) do={
        :set output "$output$[:pick $datetime ($positn + 1) ($positn + 2)]"
        :if (($positn % 2) = 0) do={:set output "$output$($separators->($positn / 2))"}
        :set positn ($positn + 1)
    }
    :set ($return->"datetime") $output
    :set skip ($skip + 12)
    # END DATETIME section

    # BEGIN TIMEZONE section
    :local tza [:tonum "0x$[:pick $pdu ($skip + 1) ($skip + 2)]"]
    :local tzb [:tonum "0x$[:pick $pdu $skip       ($skip + 1)]"]
    :local tzs "+" ; :if (($tza & 0x8) = 0x8) do={:set tzs "-"}
    :set tza ($tza & 0x7)
    :local tzmin ((($tza * 10) + $tzb) * 15)
    :set ($return->"tzminutes") [:tonum "$tzs$tzmin"]
    :local tzhou ($tzmin / 60) ; :if ($tzhou < 10) do={:set tzhou "0$tzhou"}
    :set   tzmin ($tzmin % 60) ; :if ($tzmin < 10) do={:set tzmin "0$tzmin"}
    :set ($return->"timezone") "GMT$tzs$tzhou:$tzmin"
    :set skip ($skip + 2)
    # END TIMEZONE section

    # BEGIN SMSlength section
    :local SMSlength [:tonum "0x$[:pick $pdu $skip ($skip + 2)]"]
    :set ($return->"SMSlength") $SMSlength
    :set skip ($skip + 2)
    # END SMSlength section

    # BEGIN SMSPDU section
    :local smspdu [:pick $pdu $skip [:len $pdu]]
    :set ($return->"SMSPDU") $smspdu
    # END SMSPDU section

    :return $return
}

:put [$PDUdecode 07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72 48]
}
The length at the end is optional, if specified, the validity of the PDU is calculated.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Sat Jun 03, 2023 1:25 pm

I understand not knowing the programming language, but at least what is a Byte you should know...

A Byte consists of 8 bits ranging from 00000000 to 11111111, because bits can only have 2 values, either 0 or 1,
this so that there are 256 possible combinations,
Fortunately, I found an easy way to convert bits into regular numbers. It turns out that in MikroTik, there is a function for that:
:put [ :tonum ( "0x" . "bits" ) ]
It seems that I would like to have a more comprehensive understanding of the date field. On the surface, it should consist of 14 characters representing the year, month, day, hour, minute, second, and time zone offset. However, I have come across an example where the same operator used different sequences and time zone offsets within a one-year interval. Interestingly, the PDUspy program correctly identifies the date but shows different time zone offsets. For example, two dates are provided: 22112051314180 and 32501031613521. Ideally, the first date should be interpreted as 22/11/02 15:13:41 with an 08 time zone offset, and the second date as 23/05/01 13:16:53 with a 12 time zone offset. However, according to the PDUspy program, the first date is displayed as 02.11.22 15:13 GMT+2.00, and the second date as 01.05.23 13:16 GMT+3.00. Both dates are considered correct by the program.
How do you actually convert the time zone?
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Sat Jun 03, 2023 1:28 pm

There is another aspect that I don't fully understand. It's the SMS text header, which contains information about the encoding of the SMS, the number of parts it consists of, and which part is included in the current segment.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: PDU Fields for incoming SMS

Sat Jun 03, 2023 7:10 pm

For example, two dates are provided: 22112051314180 and 32501031613521. Ideally, the first date should be interpreted as 22/11/02 15:13:41 with an 08 time zone offset, and the second date as 23/05/01 13:16:53 with a 12 time zone offset. However, according to the PDUspy program, the first date is displayed as 02.11.22 15:13 GMT+2.00, and the second date as 01.05.23 13:16 GMT+3.00. Both dates are considered correct by the program.
How do you actually convert the time zone?
They don't make anything easy. Section 9.2.3.11 of the Rel 16 SMS spec:
9.2.3.11 TP-Service-Centre-Time-Stamp (TP-SCTS)
The TP-Service-Centre-Time-Stamp field is given in semi-octet representation, and represents the local time in the following way:
Year, Month, Day, Hour, Minute, Second, Time Zone (2 semi-octets each)
The Time Zone indicates the difference, expressed in quarters of an hour, between the local time and GMT. In the first of the two semi-octets, the first bit (bit 3 of the seventh octet of the TP-Service-Centre-Time-Stamp field) represents the algebraic sign of this difference (0: positive, 1: negative).
See https://www.etsi.org/deliver/etsi_ts/12 ... 60000p.pdf
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Sat Jun 03, 2023 10:55 pm

9.2.3.11 TP-Service-Centre-Time-Stamp (TP-SCTS)
...(bit 3 of the seventh octet of the TP-Service-Centre-Time-Stamp field) represents the algebraic sign of this difference (0: positive, 1: negative).
See https://www.etsi.org/deliver/etsi_ts/12 ... 60000p.pdf
Honestly, I would prefer not to deal with these octets. I would prefer a simpler explanation.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Sun Jun 04, 2023 12:05 am

Honestly, I would prefer not to deal with these octets. I would prefer a simpler explanation.
PDU are all octet, you want to take a bath without touch the water...

Create one array with all 256 possible values with corresponding real value...
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Sun Jun 04, 2023 1:55 am

Create one array with all 256 possible values with corresponding real value...
To create, we need to understand why.

Found a pretty good article on the subject
http://www.doubleblak.com/m/blogPosts.php?id=6
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Wed Jun 07, 2023 2:58 pm

After reading many explanations on the topic of the time field in SMS, yet the best explanation turned out to be on this forum in these posts:

viewtopic.php?p=1006390#p1006390

viewtopic.php?p=1006452#p1006452

viewtopic.php?p=1006561#p1006561
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Wed Jun 07, 2023 3:10 pm

In fact, it remains to understand how to work with the TP-UDL field. I understand that it indicates both the length of the message and the presence of message parts. But it is not clear which parameter determines this. Already in the field itself, it is more or less clear that if this is a composite SMS, then both the message number and the number of the message part and other information about the message are indicated there. But where is the parameter that shows the presence of these UDHL fields
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 08, 2023 1:17 am

I wanted to clarify about the TP-MTI field. Do I understand correctly that two digits of this field need to be converted from hex to a bit representation
and in them to consider the value of each bit in accordance with the table? That is, "04" to do as "00000100", "40" as "01000000", "44" as "01000100", "60" as "01100000"
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Thu Jun 08, 2023 1:03 pm

And following the clarifications, if I understood correctly, it should look something like this:
+----+--------+----------------------------------------+
|04  | Value  | Explanation                            |
+----+--------+----------------------------------------+
|    |        |                                        |
| 1. | = 0    | TP-MTI: SMS-DELIVER (since 0)           |
| 2. | = 0    | TP-MMS: Messages exist (since 0)        |
| 3. | = 0    |                                        |
| 4. | = 0    |                                        |
| 5. | = 0    |                                        |
| 6. | = 1    | TP-SRI: Report will be sent (since 1)    |
| 7. | = 0    | TP-UDHI: User Data block contains only   |
|    |        |         message (since 0)               |
| 8. | = 0    | TP-RP: No reply path requested (since 0) |
+----+--------+----------------------------------------+

+----+--------+----------------------------------------+
| 40  | Value  | Explanation                            |
+----+--------+----------------------------------------+
|    |        |                                        |
| 1. | = 0    | TP-MTI: SMS-DELIVER (since 0)           |
| 2. | = 1    | TP-MMS: No message (since 1)             |
| 3. | = 0    |                                        |
| 4. | = 0    |                                        |
| 5. | = 0    |                                        |
| 6. | = 0    | TP-SRI: No report requested (since 0)    |
| 7. | = 0    | TP-UDHI: User Data block contains only   |
|    |        |         message (since 0)               |
| 8. | = 0    | TP-RP: No reply path requested (since 0) |
+----+--------+----------------------------------------+

+----+--------+----------------------------------------+
| 44  | Value  | Explanation                            |
+----+--------+----------------------------------------+
|    |        |                                        |
| 1. | = 0    | TP-MTI: SMS-DELIVER (since 0)           |
| 2. | = 1    | TP-MMS: No message (since 1)             |
| 3. | = 0    |                                        |
| 4. | = 0    |                                        |
| 5. | = 0    |                                        |
| 6. | = 1    | TP-SRI: Report will be sent (since 1)    |
| 7. | = 0    | TP-UDHI: User Data block contains only   |
|    |        |         message (since 0)               |
| 8. | = 0    | TP-RP: No reply path requested (since 0) |
+----+--------+----------------------------------------+

+----+--------+----------------------------------------+
| 60  | Value  | Explanation                            |
+----+--------+----------------------------------------+
|    |        |                                        |
| 1. | = 0    | TP-MTI: SMS-DELIVER (since 0)           |
| 2. | = 1    | TP-MMS: No message (since 1)             |
| 3. | = 1    |                                        |
| 4. | = 0    |                                        |
| 5. | = 0    |                                        |
| 6. | = 0    | TP-SRI: No report requested (since 0)    |
| 7. | = 0    | TP-UDHI: User Data block contains only   |
|    |        |         message (since 0)               |
| 8. | = 0    | TP-RP: No reply path requested (since 0) |
+----+--------+----------------------------------------+
But I am certain that where there are 40, 44, and 60, there are additional header fields because it is a concatenated SMS. This means that the SMS is divided into parts and sent in segments, and then the modem/phone reassembles them. And there are headers for collecting the SMS parts.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 09, 2023 12:18 am

I have dealt with the previous issue. We need to take it in reverse order.
But I would like to be specific about the type of sms. Do I need to read two bits or is it one bit? The fact is that in the table, if only one bit is taken into account, then the bits of the SMS-DELIVER and SMS-SUBMIT REPORT pair seem to match. As well as a pair of SMS-STATUS REPORT and RESERVED. But if we take two bits, then everything becomes clear.
 
optio
Long time Member
Long time Member
Posts: 655
Joined: Mon Dec 26, 2022 2:57 pm

Re: PDU Fields for incoming SMS

Fri Jun 09, 2023 12:27 am

But if we take two bits, then everything becomes clear.
TPDU Fields: https://en.wikipedia.org/wiki/GSM_03.40 - TP-MTI M M M M M M 2 bits Message Type Indicator
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 09, 2023 1:13 pm

10.5.4.7 (not 6, the article is also wrong)

Possible plausible values used only on SMS (read the full guide to understand what are)...
Only now I'm starting to understand what it's all about.
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Fri Jun 09, 2023 8:29 pm

I wanted to clarify about the TP-UD field, that is, the text of the SMS itself. I can't figure out how to get the message length. It is clear that if the encoding is UCS2, then the length of the TP-UDL received in octets is multiplied by two. And if the encoding is 7-bit, and the length is obtained in septets, then how to calculate how many characters the SMS text will be? (I do not take into account the length of the user header).

For example, how many characters will take 12 septets, or 33 septets, or 160?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: PDU Fields for incoming SMS

Sat Jun 10, 2023 7:08 am

instead of constantly asking the same questions in a circle, it would be enough for you to read what has already been written...
On poost #33, 16 post ago...
viewtopic.php?t=196747#p1006022
[…]
************************
about 40 0D D0 4669B1487C3601
************************
0x40 type
0x0D on decimal is... 13 because for store seven 7-bit GSM-7 characters are needed 13 nibble (half byte), [for store 7 bit, each 8 bytes, one less byte is required]
and because the nibble can't be one odd number, the last nibble is unused, but is still required 13+1 nibble = 7 bytes and 46/69/B1/48/7C/36/01 is 7 bytes long...
0xD0 is the coding: binary 1101 - 0000 = PDU encoded name with GSM-7 alphabet
4669B1487C3601 = 0x64961B84C76310 = binary 01100100100101100001101110000100110001110110001100010000 (notice the unused last nibble / 4 bits)
and the 7-bit 0x64 0x96 0x1B 0x84 0xC7 0x63 0x10 converted to 8-bit GSM-7 is 0x46 0x52 0x45 0x45 0x44 0x4F 0x4D,
and 8-bit GSM-7 0x46 0x52 0x45 0x45 0x44 0x4F 0x4D converted to readable CP 1252 is....
FREEDOM
[…]
 
DyadyaGenya
Member Candidate
Member Candidate
Topic Author
Posts: 215
Joined: Mon May 08, 2023 10:34 pm

Re: PDU Fields for incoming SMS

Sat Jun 10, 2023 8:29 pm

instead of constantly asking the same questions in a circle, it would be enough for you to read what has already been written...
The situation with the explanation of the time zone field is repeated. I had to ask a lot of questions to get a description for the average person. Although I have already begun to understand bits and bytes a little, questions still remain. And your explanation does not shed any light on them.

For example, the passage "C8329BFD065DDF72363904" is 12 septets long, but 22 characters (perhaps we need to add 2 characters from the title, then it will be 24), and the passage "D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72" this is 33 septets long, but 58 characters plus 2 characters from the title, i.e. 60 characters. Or, for example, "DCE532B94C06CDDF6F37" it seems to be 18 septets, but there are 20 characters plus 10 (this is a composite SMS) from the header and 2 from the length header, that is, 32 characters (050003590303DCE532B94C06CDDF6F37)

Who is online

Users browsing this forum: No registered users and 20 guests