Sending SMS with GSM7 characters

RouterOS Scripting
1

Hi!

I’m trying to send a SMS with an RBSXTR (RouterOS v7.1.3) via the API (using using the gopkg.in/routeros.v2 Go library).

I’ve got most parts working, however, sending the text “Hellö!” fails, I’m receiving just “Hell!” (I’ve masked the phone number for obvious reasons):

// omitting error handling for brevity
client, _ := routeros.Dial("192.168.0.123", "api", "apipassword")
client.Run("/tool/sms/send", "=port=lte1", "=phone-number=+4917611111111", "=message=Hellö!")

In tcpdump, I can see the API request contains the ö (0xC3B6 in UTF-8 encoding):

In the logs, I find these entries:

12:34:48 lte,async lte1: sent AT+CMGS=17\r
12:34:48 lte,async lte1: sent 0021000D91947116111111F1001105C8329B1D02\0x1a 
12:34:49 lte,async,event lte1: +CMGS: 65 
12:34:49 lte,async lte1: rcvd +CMGS: 65 
12:34:49 lte,async lte1: rcvd OK 
12:34:50 lte,async,event lte1: +CDS: 27\r 
12:34:50 lte,async,event 079194710600400706410d91947116111111f122201221433540222012214345400000

The CMGS PDU 0021000D91947116111111F1001105C8329B1D02 already only contains the message “Hell!”. According to the GSM spec (3GPP TS 23.038, “Alphabets and language-specific information”, § 6.2.1 GSM 7 bit Default Alphabet, p. 20), ö should be encoded as 0x7C (before packing and semi-octet encoding).

This post from 2012 suggests transforming the message string Windows-1252, but that didn’t work either (ö = 0xF6):

To me, it looks like the /tool/sms/send command performs input sanitization and drops that character. Is there any way to circumvent this?

I also tried to directly sent an AT+CMGS low-level command using the /interface/lte/at-chat command, but I couldn’t get the router to accept that command (I could not figure out how to pass the LTE port as argument).

2

I’ve raised a support issue with MikroTik and got the following response:

The issue here is that RouterOS terminal only accepts ASCII characters, so everything else than those will not work.

Arguing that I’m using the API, not the terminal, they clarified:

It is the same for API, it also supports only the ASCII characters, the same as the console.

In conclusion, the RouterOS needs proper support for input encoding before anyone can send text messages with non-ASCII characters.

3

If I recall, while true they support only ASCII, in someplaces RouterOS don’t care if you can encode UTF-8 (or whatever) yourself. Basically into ROS’s string type you insert single 8-bit chars, using the escape "" followed by a 8-bit hex number, like “\20\20\20” (three spaces). I’d see this approach for adding emoji’s in SSIDs, see https://r-1.ch/mikrotik-unicode-ssid-generator.php - which converts unicode into Mikrotik string with these “escaped single-byte chars”, which includes the UTF-8 escape as one of the bytes. The string :len is the byte count, not char count, since ROS is unaware and single-byte everywhere.

Now SMS little different. And your Go code with API/REST, add more complexity. I suspect it the API that’s striping the unicode before the SMS code see it – so might want to try the string literals in the shell with /tool/sms first & if that worked worth giving it a Go :wink:. But perhaps if Mikrotik got “\0A” as a 3 char string literal over the wire from the Go API, if worked in the shell.

Just an idea – typically they do let you add any single byte char to a “ASCII” string – which theoretically allows any encoding you want. Now where/when RouterOS sanitize stuff hard to know.

4

Basically into ROS’s string type you insert single 8-bit chars, using the escape "" followed by a 8-bit hex number, like “\20\20\20” (three spaces).

Thanks for the tip, I’ll try this should I get the chance again to send SMS through the SXT.

For now, I’ve changed my setup to an RPi with a cheap Huawei LTE/USB dongle. I don’t expect this to survive 24/7 operation, but I only need to send at most 50 non-time critical text messages per day. On the other hand, I get a plain old serial console and I can “just” send plain old AT commands.

5

LOL, “just”…

I guess if you “just” open up the AT channel on the SXT via System>Port>Remote Access, AND find some script on Linux to send sent AT commands for SMS, AND wire up a TCP port as some /dev to remote TCP port “serial port”. I guess the RPi could connect with some Linux SMS AT script to the SXT via TCP-to-serial AT.

Something like this opens up the AT port (you need to tweak channel, port, tcp-port, etc), but this is the “just” the easy part:

/port remote-access add port=usb1 protocol=raw channel=1 tcp-port=9999

The remote TCP socket to an AT serial channel won’t do any encoding, you have that in your favor. It’s “just” Mikrotik’s at-chat does that will “sanitize” things. But yeah finding/writing a Linux script with AT commands for SMS doesn’t sound fun.

6

I’ve never said it was easy :slight_smile:

I can externalize most of the complexity into library code (I’ve found and extended https://github.com/xlab/at in the past month), and I need a proxy in front of it anyway for authorization, message queueing and delivery notification callbacks. From there, the cost in terms of complexity/effort to talk to a serial interface vs. the RouterOS API is approximately the same.

I didn’t know this was even possible.

7

Quite the rabbit hole. But why I mentioned that “remote-access” – basically could run the Go code on the RPi, but have the xlab/at library use a TCP port that exposes that Mikrotik’s AT channel using the command above. Don’t know Go or that library, but should be able to convert TCP socket to an the os.File used by xlab/at. When Mikrotik supports containers, you code could theoretically run as a Docker on the Mikrotik (still likely have to use the remote-access TCP port even then, see http://forum.mikrotik.com/t/v7-1rc3-adds-container-support/151712/251)

On the Mikrotik side, you’d need to make sure there was an AT channel to even expose via /port remote-access. So there is also

/system/serial-terminal port=usb1 channel=1

you can use to test that AT commands even work. Again play with port/channel &

 /port print

will show the number of channels for the modem in the SXT. Some are AT, other may be NMEA (GPS) or “Qualcomm Debug” - all depending on RouterOS version/modem type/mode. And the port names sometimes are usb2 etc…

And, if you were only using the SXT for SMS/AT stuff, then you might be able to use

/interface/lte/settings/set mode=serial

which likely get you more ports (but you’d lose the LTE interface on the Mikrotik). I for sure can’t see it work, but might be able to essentially “proxy” the LTE modem module via serial-to-TCP. How fast, I dunno.

8

I’ll keep this in mind, thank you for your insights.

9

@dmke, I saw a recent thread in this forum, not quite same problem, but thought of your SMS :smiley: issues. So apparently someone did write PDU and i18n encoding parsing logic in RouterOS script, quite the doozy: http://forum.mikrotik.com/t/script-for-sending-incoming-sms-to-mail-with-full-parsing/140125/1 :open_mouth:
– you case is actually the reverse, but perhaps one of the encoding helpers there could be used to generate the right sequence for “at-chat”, somehow – but I don’t know Go, that still seems easier