Dear Mikrotik forums,
it would be great if command
/tool/fetch http-method=post url="https://blabla.server/fileservice" mode=https
can specify the file on mikrotik that will be send as data of POST request.
Better without that annoying 4kB limitation!
May I believe in that it will be implemented?
I also hope this feature will be implemented.
So, I can write a script which will upload my backup files and /export to remote https server…
Better without that annoying 4kB 63KB limitation!
Also:
:set $data "up to 63KB..."
/tool fetch http-method=post http-content-type="application/json" http-data=$data url="https://testserver.lv/index.php"
https://help.mikrotik.com/docs/display/ROS/Fetch
Then, for downloading “chunking” can be used when the server supports it. I don’t know if “chunking” is also supported for posting.
Can you please help how to read file content (.backup and .rsc) from local mikrotik storage to be uploaded using $data to https remote server?
I do not see how can you use chunks for upload / http PUT method.
Can you please reference documentation stating that $data is limited do 63 KB in RouterOS 7 ?
Mikrotik did not implement a way to read files larger than 63KB. Writing bigger files is possible with the execute command.
I don’t think it is documented but the 63KB for variables is reality in ROS 6+7.
A few months ago no one knew that chunking was possible in combination with variable in ROS and now it used widely.
Dear msatter,
please canyou specify how can I load a 60kB file in variable? I cant find any example of that…
Thank you
@msatter I am also interested and did not find a documented way for reading the file into variable, on RouterOS 7 when file is bigger then 10 KB but smaller then 60 KB.
I have made a test:
:global $data "test";
:global $url "https://prod-51.westeurope.logic.azure.com:443/workflows/blabla/triggers/manual/paths/invoke....";
/tool fetch mode=https http-method=put http-data=$data url=$url
and this is recieved as, which seems kind of fine
{
"headers": {
"User-Agent": "Mikrotik/7.x,Fetch",
"Content-Type": "application/x-www-form-urlencoded"
},
"body": {
"$content-type": "application/x-www-form-urlencoded",
"$content": "dGVzdA==",
"$formdata": [
{
"key": "test",
"value": ""
}
]
}
}
However, if I try to load file content into variable (file is about 7 KB)
/export file=export.rsc
:global data [/file get [/file find name=export.rsc] contents];
and after http fetch , there is no body data received on http server.
How to send a file content? also, for export and for backup files using HTTP ?
It is the first code example in this tread: http://forum.mikrotik.com/t/how-to-download-only-one-piece-of-file-at-a-time-with-tool-fetch-and-put-it-inside-a-variable/151020/1
this is example for “file download” and using chunks.
My request is to “upload file”, which is not bigger then 64 KB.
The key detail is what on the other end accepts. I’ll presume your referring to curl that supports a “-F” for forms and supports a “@” to include a file. This depends on HTML5’s <… input=file> within some , and enctype can be “multipart/form-data” or “x-www-urlencoded” ( curl defaults to one, depending on version and content-type). So those are TWO types of file upload, and other ways too.
But unlike curl,
/tool/fetch
does nothing to help with the data= part. AFAIK there is not a script I’ve seen that build the needed MIME nor encodes a /file. And, if it’s binary file, even a script may not be able to access the contents of the file to send nor encode it without more scripting. For text files, it might be possible to build the needed data= for a file upload, but not easy.
Basically RouterOS doesn’t have any primitives to build or parse any of the HTTP content-types. It will happily send a content-type and whatever data you build into a script “str” type. But ain’t going to help with the data at all today. Currently, you can’t even encode or decode JSON to RouterOS array, which is even bigger limitation IMO. This is why you need scripts for JSON, or rextended’s “chunk download”, to workaround to the lack of support dealing with the HTTP data.
Maybe someone else know of a workaround to form data and encoding… but you may want to look at other ways of transfer files than “HTTP upload”. You could turn this around and have the REST API or SFTP on the webserver and pull the file based on an HTTP request.
Doing SFTP or other things is not possible due to other reasons in my case. I am aware of this options.
Please, lets focus here on how to anyhow send any file from mikrotik to remote HTTP server. (not vice versa, etc.)
The other end is HTTP web server, working with standard HTTP stuff. I do have control on that side and I can program whatever is needed. For now I have setup HTTP REST APi endpoint accepting anything / anyhow, just to get somehow the file from mikrotik.
Regarding content type, it can be setup manually via /tool fetch as I can see, but I can also fix that on server side.
I do not need manipulation with JSON on routeros side anyhow.
So, what exactly is RouterOS limitation for /tool fetch working with http post for sending file? I think is just documentation lacking but, could be worse / not implemented features, which exactly ones?
You can’t.
Mikrotik will need to add support for it.
Since they have a backlog of bugs and other features, you may be waiting a while. Essentially /tool/fetch only lets you send a string (but the string can be what ever you want). RouterOS doesn’t have any helpers to encode a form submission (of which a file is a type).
And, RouterOS script can only read files of 4K or less into a string, so this prevent more cleaver solution in scripting. Basically, this would be need to work, as a first step of many, to be able to send any data from a file:
:global filedata [/file get myfile contents]
:put $filedata
@Amm0
https://help.mikrotik.com/docs/display/ROS/Scripting#Scripting-Variables
Note: Variable value size is limited to 4096bytes
https://help.mikrotik.com/docs/display/ROS/Fetch
/export file=export.rsc
:global data [/file get [/file find name=export.rsc] contents];
:global $url “https://prod-51.westeurope.logic.azure.com:443/workflows/blabla/triggers/manual/paths/invoke…”;
/tool fetch mode=https http-method=put http-data=$data url=$url
Look I ain’t arguing you ROS shouldn’t support some “file upload”. All I was saying “file upload” can happen in a variety of ways (form-data, WebDAV, etc.) – so it’s not as simple as a missing “files=” min fetch.
What you’re asking “There has to be some workaround!”… Here the problem, as you note, is variable sizes are limited, so you can’t encode anything beyond 4K in ROS script. So yes you might have more workarounds possible if variables WERE bigger than >4K. But this one is likely tied to ROS variable aren’t much different than environment variable and the Linux kernel has some limit to size of them. Since scripts have to run the same on a CHR as a mAP, the limit can’t be TOO high since it break scripts on low memory devices. And limitless variables are attach surfaces for buffer overflows.
Containers (e.g. Docker) will likely be here first, so you’d be able to spin up an container to run whatever curl or other method is needed.
Should /tool/fetch has more of of curl “encoding” options? You’d have my vote.
I understand your point about option to send file is not always the same.
In my case, i test with some test.txt about 1 KB, and is not working, received data does not contain data from file anyhow
+1 for this feature
+1 for this feature
+1 for this feature
*) fetch - added “head” option for “http-method”;
*) fetch - added “patch” option for “http-method”;
*) fetch - allow specifying link-local address in FTP mode;
*) fetch - allow to use certificate and check-certificate parameters only in HTTPS mode;
*) fetch - do not require “content-length” for HTTP (introduced in v7.13);
*) fetch - fixed DNS resolving when domain has only AAAA entries (introduced in v7.13);
*) fetch - fixed fetch execution when unexpected data is received in HTTP payload;
*) fetch - fixed fetch when using “src-path” with HTTP/HTTPS modes (introduced in v7.13);
*) fetch - fixed fetch when using “src-path” with SFTP mode (introduced in v7.13);
*) fetch - fixed incorrect “src-path” error message when “upload=yes”;
*) fetch - fixed IPv4 address logging (introduced in v7.13);
*) fetch - improved fetch stability in SFTP mode;
*) fetch - improved file download stability with HTTP/HTTPS modes;
*) fetch - less verbose logging;
*) fetch - print all “Set-Cookies” headers in response;
*) fetch - treat any 2xx HTTP return code as success (introduced in v7.13);
What about multipart/form-data ? 