fetch capable of following redirects

Dear community,

To my surprise, the standard fetch tool in routerOS has no option to deal with following redirect.

So I decided to enhance it by scripting a function that would loop x times the fetch, each loop with the url pointing towards the redirect provided by the previous iteration.

My issue is that I don’t think the scripting language enables me to do such basic task: it seems that the fetch command throws an error when facing a 301 (or 302).

So I can’t consume what if the 302 pointing towards in order to loop again towards that new target.

• if I run fetch without error handling, the script stops at the fetch command
• if I run fetch with error handing, then the script resumes, but I get no output from the fetch action, and consequently can’t assess where to redirect to

Would you have any advise how to overcome this?

Sample script below that shows a redirect on mikrotik.com website.
I would be keen to get a script POC that manages to extract where the redirect points towards

:do {:local output [/tool fetch url="http://mikrotik.com/" as-value output="user" mode="http"];} on-error={
	:put "error handling";
	:put $output; # shows nothing when error is triggered
}

I have looked around,

The core issue is that when inside the “on-error” error catching, then there is no way to get the error message to parse it.

This seems like a recurring issue raised by several forum threads already:
How to retrieve error details inside the on-error clause: http://forum.mikrotik.com/t/how-to-retrieve-error-details-inside-the-on-error-clause/138639/1
Any way to get the error-string in on-error=: http://forum.mikrotik.com/t/any-way-to-get-the-error-string-in-on-error/98204/1
Error handling?: http://forum.mikrotik.com/t/error-handling/102310/1

Any hints on how I can get around this?

I’m able to catch the error.

Thanks,

can you please post some sample code to display the error message and or store that error message to a variable?

Thanks

From what I see till now, 99% of sript programmers are not programmers.
All must be programmed.
Just the handle of external tools like fetch can justify the use of the “on-error”

1. This is the only true question
2. This is an erratic way to do something, not a question of error or not
3. The problem is the script code: is not done well, is not that way than the thing must done, is why the script give errors…
   
   
   One example over all from 3):

# wrong coding
:do {/interface bridge add name=loopback; } on-error={:put "loopback exists"}
# translating: create loopback, if you got an error because already exist, ignore it
# on short: create it, hoping not already exist

# right coding
/interface bridge
:if ([:len [:find where name="loopback"]] = 0) do={add name=loopback}
# translating: if the result of the count of interface named loopback are zero, create the interface
# on short: if not exist, create it

Thanks for the contribution.

Not having access to the error message that triggered the on-error only gets you so far.

But on our topic at hand: the error message contains the redirect url.
One cannot just guess it, or am I missing the bigger picture?
How can one get that redirect url?

What would be the “right coding” to do it?
I’m a bit out of option.

search tag # rextended manage fetch errors

I write one post a time, leave me the time!!!

Is not for suspance, I need some time to write and test…

This is the code:

/file remove [find where name="testfetch.txt"]
{
    :local jobid [:execute file=testfetch.txt script="/tool fetch url=http://mikrotik.com"]
    :put "Waiting the end of process for file testfetch.txt to be ready, max 20 seconds..."
    :global Gltesec 0
    :while (([:len [/sys script job find where .id=$jobid]] = 1) && ($Gltesec < 20)) do={
        :set Gltesec ($Gltesec + 1)
        :delay 1s
        :put "waiting... $Gltesec"
    }
    :put "Done. Elapsed Seconds: $Gltesec\r\n"
    :if ([:len [/file find where name="testfetch.txt"]] = 1) do={
        :local filecontent [/file get [/file find where name="testfetch.txt"] contents]
        :put "Result of Fetch:\r\n****************************\r\n$filecontent\r\n****************************"
    } else={
        :put "File not created."
    }
}

This is the result:

Waiting the end of process for file testfetch.txt to be ready, max 20 seconds...
waiting... 1
waiting... 2
Done. Elapsed Seconds: 2

Result of Fetch:
****************************
  status: connecting

  status: failed

failure: closing connection: <302 Found "https://mikrotik.com/"> 159.148.147.196:80 (4)
                                                                                       
****************************

the result is a multiline file between the ****************************

with :find and :pick inside the string closing connection: <302 Found "https://mikrotik.com/"> 159.148.147.196:80 (4) can be isolated

the part containing the redirect “https://mikrotik.com/”

you can recall again the test code and you discover another redirect to “https://www.mikrotik.com/”

End.

Thanks

It’s very a interesting approach to isolate the code execution into a separate job, and monitor that job progress!

I’ll get my head around this and do some testing: I understand that within that job, the code is outputing the fetch to a file.
I was keen to limit such disk access to preserve the SSD life.
So I’ll try to get that value in a variable (such as a global variable).

But it’s a very cool approach, many thanks!

Do not worry about disk writes, on flash the write go already n ramdisk, not wearing at all,
about NAND, are years (from 2009) than I use user-manager on nand, and no problem,
and user-manager write continuosly on nand…

I respect your faith in SSD … but a small voice in my head will push me to attempt to get this running without disk access

After a bit of (unsuccessful) testing, there doesn’t seem to be possible to get fetch error message without using disk read/write.

rextended your solution is still the best workaround, and I’ll mark this as the solution.
But it still remain damn painful to not have access on a simple variable of the error message, like most programming language offer by standard.