Fetch returns "failure: Unexpected payload received"

RouterOS General
1

I am trying to using Netwatch to call a script that will turn a Shelly relay off and then back on when internet connectivity is lost. This will power-cycle a cable modem.

The Shelly 1 uses an http-post with an ON or an OFF:

http://192.168.0.144/relay/0?turn=off

When issued from a web browser, it returns:

{“ison”:true,“has_timer”:false,“timer_started_at”:0,“timer_duration”:0,“timer_remaining”:0,“source”:“http”}

The script that Netwatch calls uses this command:

:tool fetch http-method=post http-header-field="Content-Type: application/json" output=none url="http://192.168.0.144/relay/0?turn=off"

But, that command, whether issued within a script or in a Terminal, results in:

failure: Unexpected payload received

Does anyone know what “Unexpected payload received” means and how to correct it?

Is there a better way in ROS to send an http-post?

Thank you.

2

I don’t think you should set Content-Type to json in the request, as there is no content at all in the request, the command for Shelly is in the url - only the result of the command comes as a json payload in the 200.

But since it is plaintext http, sniffing should show you what the Shelly actually responds.

3

I actually tried it without content-type and the result was the same.

Not sure what I should be looking for in the Shelly’s response.

As for plain text, I tried using http://user:pass@ and got a 401 error

4

What I had in mind was to capture (sniff) the communication with Shelly when the request is sent from the browser and when it is sent from the Tik and use Wireshark to “find 10 differences” between the two cases. “Plaintext” means that the communication is not encrypted (you use http://, not https://); whether you use a username and password or not has no effect on whether the communication is encrypted or not, you can have encrypted communication without credentials and plaintext communication with credentials.

5

First: Thank you so much for the suggestions and the explanations of plaintext vs encrypted.

I ran Wireshark and immediately found the problem: I was using POST but the browser (which works) was using GET.

I switched the MT script to GET and now it works.

FROM BROWSER to SHELLY:

GET /relay/0?turn=on HTTP/1.1
Host: 192.168.0.144
Connection: keep-alive
Cache-Control: max-age=0
Authorization: Digest username=“admin”, realm=“shellyplus1-b8d61a886a74”, nonce=“1724228391”, uri=“/relay/0?turn=on”, algorithm=SHA-256, response=“bc13677cd1efc4bb5520c547eef394e05e7ac4e0479b4f252c3162fa3697f06e”, qop=auth, nc=00000007, cnonce=“a5729ce3fcc217f2”
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9


FROM SHELLY TO BROWSER:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 123
Pragma: no-cache
Server: ShellyHTTP/1.0.0
Connection: close
{“ison”: true, “has_timer”:false, “timer_started_at”: 0, “timer_duration”: 0.00, “timer_remaining”: 0.00, “source”: “http”}

But, I tried to set a password on the Shelly and am unable to get it to work on either the browser or the MT with:

http://:@192.168.0.144…

On the browser, a dialog box opens requesting username/password. On the MT, it fails with an “unauthorized access” error.

6

The user:password actually works from a browser.

This is the capture:

GET /relay/0?turn=on HTTP/1.1
Host: 192.168.0.144
Connection: keep-alive
Cache-Control: max-age=0
Authorization: Digest username=“admin”, realm=“shellyplus1-b8d61a886a74”, nonce=“1724241354”, uri=“/relay/0?turn=on”, algorithm=SHA-256, response=“f4355a1cdc4f7ab000e68dfb5e88d4c85944c7cafc78f4eea4ec6e2969f8ca47”, qop=auth, nc=00000001, cnonce=“94f4cfda18523d87”
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9

7

Very likely because the browser itself manages to encrypt/encode/hash/whatever the username/password before sending it to the Shelly.

https://reqbin.com/req/c-fkj7kdqi/curl-request-with-credentials

Try with curl:
curl http://192.168.0.144 --user “user:pwd”

If it works, then maybe you need to use something like:
https://github.com/mqnoy/Mikrotik-HTTP-Auth
(it is for 6.x, may need changes to run in 7.x)

8

Where would I issue that command? I don’t see a curl facility in ROS.

9

Try it on a PC (with curl installed/running), then - if it works - you need a script for “basic auth”, see my previous edited post.

10

In a Windows command prompt, curl does not work with the Shelly authentication enabled.

It works fine with the Shelly authentication disabled.

11

Post the exact CURL command you are using.
No error provided when it fails?
Under Windows, try running.
curl -v -u “user” http://192.168.0.144/..
it should prompt for your password and show more details.
Try also the -anyauth option:
https://curl.se/docs/manpage.html#--anyauth
maybe it provides some more info.

12

With authentication enabled on the Shelly, the following curl command does not work:

curl -u admin: > http://192.168.0.144/relay/0?turn=off

The following command:

C:\Users\jrsko>curl --anyauth -u admin: > http://192.168.0.144/relay/0?turn=on >

returns:

curl: (94) An authentication function returned an error

13

Try with the -v (verbose) switch, possibly there will be more details.

Checking the capture of the browser, it mentions “algorithm=SHA-256,” the Shelly1 and 1plus, from what I read should be using basic auth, later devices do actually use a different method of authentication:
https://shelly-api-docs.shelly.cloud/gen2/General/Authentication/

There have been issues with sha256 with curl on windows:
https://github.com/curl/curl/issues/6302
connected to SSPI, cannot say if they are now solved or if there is a known workaround.

If your Shelly uses digest authentication with SHA256, even if it can be found how to use curl for it, (probably using --digest instead of–user) I doubt that there is a Mikrotik script for it, but cannot say.

14

I figured it out: “http-auth-scheme=digest” made it work.

The following command works:

:tool fetch http-method=get user="admin" password="<my_password>" url="http://192.168.0.144/relay/0?turn=on" http-header-field="Content-Type: application/x-www-form-urlencoded" output=none http-auth-scheme=digest

Thank you everyone!

15

So that Shelly uses digest and fetch supports it directly, good to know.

It remains strange that curl with -anyauth didn’t work/didn’t provide this info, but all is well that ends well :slight_smile:.