Community discussions

MikroTik App
 
vanikcz
newbie
Topic Author
Posts: 31
Joined: Wed Oct 14, 2015 11:06 pm

Feature request: /tool fetch HTTP-POST can send a file

Wed Jan 05, 2022 9:40 am

Dear Mikrotik forums,
it would be great if command
/tool/fetch http-method=post url="https://blabla.server/fileservice" mode=https 
can specify the file on mikrotik that will be send as data of POST request.

Better without that annoying 4kB limitation!

May I believe in that it will be implemented?
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Mon Jan 24, 2022 10:16 pm

I also hope this feature will be implemented.
So, I can write a script which will upload my backup files and /export to remote https server...
 
msatter
Forum Guru
Forum Guru
Posts: 2663
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Feature request: /tool fetch HTTP-POST can send a file

Mon Jan 24, 2022 10:25 pm

Better without that annoying 4kB 63KB limitation!

Also:
:set $data "up to 63KB..."
/tool fetch http-method=post http-content-type="application/json" http-data=$data url="https://testserver.lv/index.php"

https://help.mikrotik.com/docs/display/ROS/Fetch

Then, for downloading "chunking" can be used when the server supports it. I don't know if "chunking" is also supported for posting.

viewtopic.php?f=9&t=177530
Loving my freedom and so, no Twitter, no Meta/Facebook/Instagram/WhatsApp, no Apple and no Alphabet/Google, no Amazon/Cloudfront/AWS. 12% inflation but still giving money to Italy.

Running:
RouterOS 7.2RC6 and 7.21 / Winbox 3.35 64bits
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Tue Jan 25, 2022 1:22 am

Can you please help how to read file content (.backup and .rsc) from local mikrotik storage to be uploaded using $data to https remote server?
I do not see how can you use chunks for upload / http PUT method.
Can you please reference documentation stating that $data is limited do 63 KB in RouterOS 7 ?
 
msatter
Forum Guru
Forum Guru
Posts: 2663
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Feature request: /tool fetch HTTP-POST can send a file

Tue Jan 25, 2022 6:58 pm

Mikrotik did not implement a way to read files larger than 63KB. Writing bigger files is possible with the execute command.

I don't think it is documented but the 63KB for variables is reality in ROS 6+7.

A few months ago no one knew that chunking was possible in combination with variable in ROS and now it used widely.
Loving my freedom and so, no Twitter, no Meta/Facebook/Instagram/WhatsApp, no Apple and no Alphabet/Google, no Amazon/Cloudfront/AWS. 12% inflation but still giving money to Italy.

Running:
RouterOS 7.2RC6 and 7.21 / Winbox 3.35 64bits
 
vanikcz
newbie
Topic Author
Posts: 31
Joined: Wed Oct 14, 2015 11:06 pm

Re: Feature request: /tool fetch HTTP-POST can send a file

Mon Jan 31, 2022 12:49 am

Dear msatter,
please canyou specify how can I load a 60kB file in variable? I cant find any example of that...
A few months ago no one knew that chunking was possible in combination with variable in ROS and now it used widely.
Thank you
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Thu Feb 10, 2022 8:19 am

@msatter I am also interested and did not find a documented way for reading the file into variable, on RouterOS 7 when file is bigger then 10 KB but smaller then 60 KB.
 
heavenlyangel
newbie
Posts: 35
Joined: Fri Apr 16, 2021 5:48 pm

Re: Feature request: /tool fetch HTTP-POST can send a file

Sat Feb 19, 2022 1:34 am

Trash limit
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Fri Feb 25, 2022 10:19 am

I have made a test:

:global $data "test";
:global $url "https://prod-51.westeurope.logic.azure.com:443/workflows/blabla/triggers/manual/paths/invoke....";
/tool fetch mode=https http-method=put http-data=$data url=$url
and this is recieved as, which seems kind of fine
{
    "headers": {
        "User-Agent": "Mikrotik/7.x,Fetch",
        "Content-Type": "application/x-www-form-urlencoded"
    },
    "body": {
        "$content-type": "application/x-www-form-urlencoded",
        "$content": "dGVzdA==",
        "$formdata": [
            {
                "key": "test",
                "value": ""
            }
        ]
    }
}
However, if I try to load file content into variable (file is about 7 KB)
/export file=export.rsc
:global data [/file get [/file find name=export.rsc] contents];
and after http fetch , there is no body data received on http server.

How to send a file content? also, for export and for backup files using HTTP ?
 
msatter
Forum Guru
Forum Guru
Posts: 2663
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Feature request: /tool fetch HTTP-POST can send a file

Fri Feb 25, 2022 2:04 pm

It is the first code example in this tread: viewtopic.php?f=9&t=177530
Loving my freedom and so, no Twitter, no Meta/Facebook/Instagram/WhatsApp, no Apple and no Alphabet/Google, no Amazon/Cloudfront/AWS. 12% inflation but still giving money to Italy.

Running:
RouterOS 7.2RC6 and 7.21 / Winbox 3.35 64bits
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Fri Feb 25, 2022 2:32 pm

It is the first code example in this tread: viewtopic.php?f=9&t=177530
this is example for "file download" and using chunks.
My request is to "upload file", which is not bigger then 64 KB.
 
User avatar
Amm0
Long time Member
Long time Member
Posts: 611
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: Feature request: /tool fetch HTTP-POST can send a file

Thu Mar 10, 2022 7:22 pm

The key detail is what on the other end accepts. I'll presume your referring to `curl` that supports a "-F" for forms and supports a "@<filename>" to include a file. This depends on HTML5's <... input=file> within some <form enctype=...>, and enctype can be "multipart/form-data" or "x-www-urlencoded" ( curl defaults to one, depending on version and content-type). So those are TWO types of file upload, and other ways too.

But unlike `curl`,
/tool/fetch
does nothing to help with the data= part. AFAIK there is not a script I've seen that build the needed MIME nor encodes a /file. And, if it's binary file, even a script may not be able to access the contents of the file to send nor encode it without more scripting. For text files, it might be possible to build the needed data= for a file upload, but not easy.

Basically RouterOS doesn't have any primitives to build or parse any of the HTTP content-types. It will happily send a content-type and whatever data you build into a script "str" type. But ain't going to help with the data at all today. Currently, you can't even encode or decode JSON to RouterOS array, which is even bigger limitation IMO. This is why you need scripts for JSON, or rextended's "chunk download", to workaround to the lack of support dealing with the HTTP data.

Maybe someone else know of a workaround to form data and encoding... but you may want to look at other ways of transfer files than "HTTP upload". You could turn this around and have the REST API or SFTP on the webserver and pull the file based on an HTTP request.
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Fri Mar 11, 2022 9:44 am

Doing SFTP or other things is not possible due to other reasons in my case. I am aware of this options.
Please, lets focus here on how to anyhow send any file from mikrotik to remote HTTP server. (not vice versa, etc.)

The other end is HTTP web server, working with standard HTTP stuff. I do have control on that side and I can program whatever is needed. For now I have setup HTTP REST APi endpoint accepting anything / anyhow, just to get somehow the file from mikrotik.
Regarding content type, it can be setup manually via /tool fetch as I can see, but I can also fix that on server side.

I do not need manipulation with JSON on routeros side anyhow.

So, what exactly is RouterOS limitation for /tool fetch working with http post for sending file? I think is just documentation lacking but, could be worse / not implemented features, which exactly ones?
 
User avatar
Amm0
Long time Member
Long time Member
Posts: 611
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: Feature request: /tool fetch HTTP-POST can send a file

Fri Mar 11, 2022 2:10 pm

Please, lets focus here on how to anyhow send any file from mikrotik to remote HTTP server. (not vice versa, etc.)

You can't.

Mikrotik will need to add support for it.

Since they have a backlog of bugs and other features, you may be waiting a while. Essentially /tool/fetch only lets you send a string (but the string can be what ever you want). RouterOS doesn't have any helpers to encode a form submission (of which a file is a type).

And, RouterOS script can only read files of 4K or less into a string, so this prevent more cleaver solution in scripting. Basically, this would be need to work, as a first step of many, to be able to send any data from a file:
:global filedata [/file get myfile contents]
:put $filedata
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Wed Apr 06, 2022 11:08 am

@Amm0
https://help.mikrotik.com/docs/display/ ... -Variables
> Note: Variable value size is limited to 4096bytes

https://help.mikrotik.com/docs/display/ROS/Fetch

/export file=export.rsc
:global data [/file get [/file find name=export.rsc] contents];
:global $url "https://prod-51.westeurope.logic.azure. ... ths/invoke....";
/tool fetch mode=https http-method=put http-data=$data url=$url
 
User avatar
Amm0
Long time Member
Long time Member
Posts: 611
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: Feature request: /tool fetch HTTP-POST can send a file

Wed Apr 06, 2022 4:54 pm

@Amm0
https://help.mikrotik.com/docs/display/ ... -Variables
> Note: Variable value size is limited to 4096bytes

https://help.mikrotik.com/docs/display/ROS/Fetch

/export file=export.rsc
:global data [/file get [/file find name=export.rsc] contents];
:global $url "https://prod-51.westeurope.logic.azure. ... ths/invoke....";
/tool fetch mode=https http-method=put http-data=$data url=$url
Look I ain't arguing you ROS shouldn't support some "file upload". All I was saying "file upload" can happen in a variety of ways (form-data, WebDAV, etc.) – so it's not as simple as a missing "files=" min fetch.

What you're asking "There has to be some workaround!"... Here the problem, as you note, is variable sizes are limited, so you can't encode anything beyond 4K in ROS script. So yes you might have more workarounds possible if variables WERE bigger than >4K. But this one is likely tied to ROS variable aren't much different than environment variable and the Linux kernel has some limit to size of them. Since scripts have to run the same on a CHR as a mAP, the limit can't be TOO high since it break scripts on low memory devices. And limitless variables are attach surfaces for buffer overflows.

Containers (e.g. Docker) will likely be here first, so you'd be able to spin up an container to run whatever `curl` or other method is needed.

Should /tool/fetch has more of of `curl` "encoding" options? You'd have my vote.
 
hkusulja
Frequent Visitor
Frequent Visitor
Posts: 66
Joined: Fri Apr 13, 2012 1:14 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Wed Apr 06, 2022 5:34 pm

I understand your point about option to send file is not always the same.
In my case, i test with some test.txt about 1 KB, and is not working, received data does not contain data from file anyhow
 
User avatar
vittore
just joined
Posts: 18
Joined: Tue Jan 26, 2021 8:33 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Wed Apr 20, 2022 12:15 pm

Dear Mikrotik forums,
it would be great if command
/tool/fetch http-method=post url="https://blabla.server/fileservice" mode=https 
can specify the file on mikrotik that will be send as data of POST request.

Better without that annoying 4kB limitation!

May I believe in that it will be implemented?


+1 for this feature
 
Deeleave
just joined
Posts: 3
Joined: Mon Dec 16, 2019 4:21 pm

Re: Feature request: /tool fetch HTTP-POST can send a file

Thu Apr 21, 2022 8:08 am

Dear Mikrotik forums,
it would be great if command
/tool/fetch http-method=post url="https://blabla.server/fileservice" mode=https 
can specify the file on mikrotik that will be send as data of POST request.

Better without that annoying 4kB limitation!

May I believe in that it will be implemented?
+1 for this feature
 
Shikiko
just joined
Posts: 2
Joined: Thu Apr 21, 2022 11:25 am

Re: Feature request: /tool fetch HTTP-POST can send a file

Thu Apr 21, 2022 11:27 am

Dear Mikrotik forums,
it would be great if command
/tool/fetch http-method=post url="https://blabla.server/fileservice" mode=https 
can specify the file on mikrotik that will be send as data of POST request.

Better without that annoying 4kB limitation!

May I believe in that it will be implemented?
+1 for this feature

Who is online

Users browsing this forum: nishadul and 6 guests