Community discussions

MikroTik App
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

`/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 12:22 pm

Hi everybody,

I have run across a weird problem. I am trying to download a file via HTTPS on all of my MikroTik devices. This works perfectly on most devices, but on three of them, I just run into a timeout. All those three devices happen to be CRS112-8P-4S.

So this is what happens:
[admin@Switch] > /tool fetch url="https://192.168.100.246/file"
  status: failed

failure: connection timeout
The irritating thing is, if I use FTP instead, it works perfectly. Other devices with identical network setups (except for the actual IP address of the device, of course) have no problem whatsoever, so I tend to think there is not network problem.

I did a `tcpdump` on the web server, and from the perspective of that server, everything is fine: Network packets from the switch arrive and are answered properly.

Anyone have a clue where to look?

THX & Cheers,
Toby.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 26322
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 2:29 pm

Can this same device ping this server?
Does this file actually exist?
Does this specific switch have access to this server? Maybe the other routers are connected differently?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 2:35 pm

I have dozen of this model, no problem.

RouterOS version?

try full url (remove the spaces):

https : //download.mikrotik.com/routeros/6.47.9/routeros-mipsbe-6.47.9.npk

ptrobably there are some firewall rules somewhere
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 3:23 pm

Hi,

sorry, I just realized I should have been more verbose.

So the CRS112-8P-4S is running RouterOS 6.47.9.
Can this same device ping this server?
Does this file actually exist?
Does this specific switch have access to this server? Maybe the other routers are connected differently?
- The device can ping the server perfectly. I can transfer the file via FTP and HTTP just fine.
- The file does exist (as a side note, I would hope that the resulting error code would be different for a 404 than for a timeout).
- The switch can access the server via all other means I have tried. In fact, some of the other MT devices that *can* download via HTTPS are connected through the switch.
- No local firewall rules on the switch whatsoever.
- Downloading, say, `/graphs/cpu/daily.gif` from the switch to the web server works perfectly (even when I disable the web server and use port 443 as source port), so I really don't think there is a firewalling or network problem.
- I *can* download via HTTPS from *other* servers just fine.

Maybe some cipher issue? Keylength? Supported TLS cipher suites? The thing that irritates me is that just these three devices seem to have that problem. All other MT devices I run don't seem to have the problem even though they are running the same ROS version.

I'm a bit stymied.

Cheers,
Toby.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 4:35 pm

Ecliptic Key instead of RSA?
MikroTik supports only TLS 1.2
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 4:36 pm

Ecliptic Key instead of RSA?
MikroTik supports only TLS 1.2
Nope. No EC cert. RSA with 4096 bits.

Besides, all my _other_ MikroTik devices can download this very same file just fine.

Cheers,
Toby.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 4:54 pm

... all my _other_ MikroTik devices can download this very same file just fine...
ops...

full export without omit anything except username and password of working and non-working?

probably hidden firewall rules between devices, also on web server?
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:05 pm

... all my _other_ MikroTik devices can download this very same file just fine...
ops...

probably hidden firewall rules between devices, also on web server?
Non-working (switch):
[admin@Switch] > /ip firewall export
# apr/29/2021 15:57:23 by RouterOS 6.47.9
# software id = ###
#
# model = CRS112-8P-4S
# serial number = ###
[admin@Switch] > /ip firewall export
Working (AP behind the switch):
[admin@AP] > /ip firewall export
# apr/29/2021 15:58:48 by RouterOS 6.47.9
# software id = ###
#
# model = RouterBOARD wsAP 5Hac2nD
# serial number = ###
[admin@AP] > 
No firewall rules whatsover. Same on the server.

Besides, I *can* access the server. If I turn off HTTPS on port 443 and turn on HTTP on port 443, I can download just fine:
[admin@Switch] > /tool fetch url="http://192.168.100.246:443/FILE"
      status: finished
  downloaded: 3KiBC-z pause]
       total: 3KiB
    duration: 1s

[admin@Switch] >
If I turn TLS back on, the timeout strikes again:
[admin@Switch] > /tool fetch url="https://192.168.100.246/FILE"
  status: failed

failure: connection timeout
[admin@Switch] >
Cheers,
Toby.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:12 pm

...for full export I intend full export of devices, not only firewall filter section...


/certificate settings
set crl-download=no crl-store=ram crl-use=no

/tool fetch url="https://192.168.100.246//graphs/cpu/daily.gif" check-certificate=no
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:19 pm

...for full export I intend full export of devices, not only firewall filter section...
No particular settings on either the (non-working) switch or the (working) AP:
[admin@Switch] /certificate settings> print
  crl-download: no
       crl-use: no
     crl-store: ram
[admin@Switch] /certificate settings> 
and:
[admin@AP] /certificate settings> print
  crl-download: no
       crl-use: no
     crl-store: ram
[admin@AP] /certificate settings> 
Cheers,
Toby.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:37 pm

Have try?
/tool fetch url="https://192.168.100.246/graphs/cpu/daily.gif" check-certificate=no

dhcp and security package are active on "8P"?

I disable dhcp, security and other unused packages until 6.46.8, after that I leave that packages active...
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:41 pm

Have try?
/tool fetch url="https://192.168.100.246/graphs/cpu/daily.gif" check-certificate=no
No, I have not tried, because that IP is the IP of a web server that does not server this particular file. However, nothing has changed. I just posted the configuration for `/certificate` (which is boring because it is the same unchanged default values on all systems, whether they run into the timeout or not).

I do not see where you are going with this, to be honest?

Cheers,
Toby.
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:43 pm

dhcp and security package are active on "8P"?

I disable dhcp, security and other unused packages until 6.46.8, after that I leave that packages active...
Both DHCP and Security are active on both the (non-working) switch and the (working) AP behind it.

Cheers,
Toby.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:52 pm

Have try?
/tool fetch url="https://192.168.100.246/graphs/cpu/daily.gif" check-certificate=no
...No, I have not tried, because that IP is the IP of a web server that does not server this particular file...

mmm...
Have try the option "check-certificate=no" on the right url?

I do not see where you are going with this, to be honest?
Sorry, you're absolutely right, I'm just wasting time trying to help you.

Have a nice day.
 
KayBur
just joined
Posts: 16
Joined: Thu Apr 29, 2021 3:33 pm
Location: Springfield

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:56 pm

Have try?
/tool fetch url="https://192.168.100.246/graphs/cpu/daily.gif" check-certificate=no
No, I have not tried, because that IP is the IP of a web server that does not server this particular file. However, nothing has changed. I just posted the configuration for `/certificate` (which is boring because it is the same unchanged default values on all systems, whether they run into the timeout or not).

I do not see where you are going with this, to be honest?

Cheers,
Toby.
What's the problem with trying what they recommend to you? You asked for help and immediately say yourself that this is not right and wrong, and that there is no point in trying. Then there is probably no point in asking for help, since you do not need it. Am I wrong?
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 5:56 pm

Hi,
mmm...
Have try the option "check-certificate=no" on the right url?
Yes, I have tried that. No difference.
Sorry, you're absolutely right, I'm just wasting time trying to help you.
Have a nice day.
Just to make sure you don't misunderstand: I am very grateful for your effort. I just don't understand why you have asked the questions you asked.

Cheers,
Toby.
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Thu Apr 29, 2021 6:18 pm

Hi,
/tool fetch url="https://192.168.100.246/graphs/cpu/daily.gif" check-certificate=no
No, I have not tried, because that IP is the IP of a web server that does not server this particular file. However, nothing has changed. I just posted the configuration for `/certificate` (which is boring because it is the same unchanged default values on all systems, whether they run into the timeout or not).

I do not see where you are going with this, to be honest?
What's the problem with trying what they recommend to you? You asked for help and immediately say yourself that this is not right and wrong, and that there is no point in trying. Then there is probably no point in asking for help, since you do not need it. Am I wrong?
OK, I appear to have pushed some wrong buttons inadvertently. I am sorry, I did not mean to imply I am not grateful for help.

What I meant to say was: I have not re-tried downloading the file because nothing has changed. I have been asked whether the CRL handling configuration was identical on both systems, and I have answered that. I have not retried downloading the file again. I did not understand the reason for retrying to download that URL (particularly since it was not the one I had been talking about). That is all I meant to say. If I stepped on somebody's toes, I apologize, that was not my intention.

I am, indeed, very grateful for any hints, and have tried very much, as you can see, to answer any question faithfully and correctly.

Cheers,
Toby.
 
KayBur
just joined
Posts: 16
Joined: Thu Apr 29, 2021 3:33 pm
Location: Springfield

Re: `/tool fetch` via HTTPS times out on CRS112

Fri Apr 30, 2021 3:53 pm

Hi,
/tool fetch url="https://192.168.100.246/graphs/cpu/daily.gif" check-certificate=no
No, I have not tried, because that IP is the IP of a web server that does not server this particular file. However, nothing has changed. I just posted the configuration for `/certificate` (which is boring because it is the same unchanged default values on all systems, whether they run into the timeout or not).

I do not see where you are going with this, to be honest?
What's the problem with trying what they recommend to you? You asked for help and immediately say yourself that this is not right and wrong, and that there is no point in trying. Then there is probably no point in asking for help, since you do not need it. Am I wrong?
OK, I appear to have pushed some wrong buttons inadvertently. I am sorry, I did not mean to imply I am not grateful for help.

What I meant to say was: I have not re-tried downloading the file because nothing has changed. I have been asked whether the CRL handling configuration was identical on both systems, and I have answered that. I have not retried downloading the file again. I did not understand the reason for retrying to download that URL (particularly since it was not the one I had been talking about). That is all I meant to say. If I stepped on somebody's toes, I apologize, that was not my intention.

I am, indeed, very grateful for any hints, and have tried very much, as you can see, to answer any question faithfully and correctly.

Cheers,
Toby.
Everything is ok, just try experimenting and follow all the recommendations. It happens that a seemingly meaningless action is a real solution to a problem.
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Fri Apr 30, 2021 5:24 pm

Hi again,

I have had the chance to do some more testing with some more devices. It still looks very much like the CRS112-8P-4S has some sort of weird issue; a friend of mine happens to run one as well, and he confirms the very same behavior for his CRS112, while all his other MikroTik devices were able to download the file just fine.

So, as far as my observations go, I have seen four different CRS112s (three of which run ROS 6.47.9 on firmware 6.47.9, one running ROS 6.48.1 on firmware 6.46.1) that all fail to download the file but run into a timeout, while a number of other MikroTik devices were able to download the file without any issues (in particular, at least one each of CRS326-24G-2S+, Groove A-52HPn r2, RB760iGS, RB941-2nD, RB960PGS, RBD52G-5HacD2HnD, RB952Ui-5ac2nD, cAP Gi-5acD2nD, cAP L-2nD, mAP 2nD, mAP L-2nD, wAP 2nD r2, wAP G-HacT2HnD, wsAP 5Hac2nD, all running ROS 6.47.9/firmware 6.47.9).

I am certain that there is no network or firewall issue because if I change the protocol from HTTPS to HTTP (while staying on port 443), the CRS112 can _also_ download the file without a problem. The issue must be with the HTTPS stack somewhere. Certificate validation seems to be no factor, because all of my devices are configure identically in this respect, and adding `check-certificate=no` to the `fetch` command does not make a difference.

Here was a concrete example of a five-byte file that the CRS112s cannot download: hXXps://www.XXXXX.de/.acme.sh/Test
(I have removed the file because I have implemented a workaround.)

I am completely out of ideas as to what is different on the CRS112s. Doesn't seem to be the architecture (the wsAP, for example, is also `mipsbe`-based, albeit with a different processor version and speed), shouldn't be the available RAM (the wsAP with 64 MB and 32 MB free works, while the CRS112s with 128 MB and 98 MB free do not), shouldn't be the HDD size (again, wsAP with 16 MB/2 MB free works, CRS112s with 16 MB/2.5 MB free do not). About the only consistent observation I can make is that the CRS112s are the only devices with a MIPS 24Kc V8.5 CPU.

Any hints are greatly appreciated.

Cheers,
Toby.
Last edited by tdussa on Fri Apr 30, 2021 8:49 pm, edited 7 times in total.
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Fri Apr 30, 2021 5:30 pm

Hi,
What I meant to say was: I have not re-tried downloading the file because nothing has changed. I have been asked whether the CRL handling configuration was identical on both systems, and I have answered that. I have not retried downloading the file again. I did not understand the reason for retrying to download that URL (particularly since it was not the one I had been talking about). That is all I meant to say. If I stepped on somebody's toes, I apologize, that was not my intention.

I am, indeed, very grateful for any hints, and have tried very much, as you can see, to answer any question faithfully and correctly.
Everything is ok, just try experimenting and follow all the recommendations. It happens that a seemingly meaningless action is a real solution to a problem.
You are undoubtedly correct, but I am sure you agree that printing the current configuration is definitely not an example of a meaningless-but-actually-effective solution here? ;-)

Cheers,
Toby.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Fri Apr 30, 2021 5:53 pm

I asky you yesterday the export
... full export without omit anything except username and password of working and non-working? ...
and you post only firewall filter section

I ask again
... for full export I intend full export of devices, not only firewall filter section ...
and you reply
... No particular settings on either the (non-working) switch or the (working) AP:..

If the situation remains "I ask for help, but I think what you ask me is useless so I know how to get by on my own"
go ahead alone and don't complain if at some point people don't care about your problems.

I have about twenty CRS112-8P-4S, and I spent my time to test on all, and no one gives me problems on https.
The only thing I can't do is test on your server, which is probably the cause of all the problems.
... I *can* download via HTTPS from *other* servers just fine...
Not probably... CERTAINLY
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Fri Apr 30, 2021 7:37 pm

Hi there,

first of all, let me -- again -- explicitly say that I do appreciate your effort to help. Thank you for your time and effort. Again, I apologize if I have offended you; I did not intend to. I am still looking for hints as to how to troubleshoot my problem and am grateful for any pointers.

Having said that, I do have some remarks to your comments.

I asky you yesterday the export
... full export without omit anything except username and password of working and non-working? ...
and you post only firewall filter section

I ask again
... for full export I intend full export of devices, not only firewall filter section ...
and you reply
... No particular settings on either the (non-working) switch or the (working) AP:..
Indeed. Instead, I have asked you where you were headed (which you chose not to explain), because as you well know, cleaning and reading a potentially large complete config export is tedious work that is completely unnecessary and a waste of everybody's time (yours, mine, and of everyone else reading this thread). If you know what you are looking for, fine, then say so, that was *exactly* what I was originally asking for: A clue where to look. If you don't feel like telling me what you think the problem is, fine as well, then don't.

If the situation remains "I ask for help, but I think what you ask me is useless so I know how to get by on my own"
The situation indeed remains "I am asking for pointers where to look." So, I wonder, if you know exactly what you want to look for in my "complete export", then why don't you tell me? Something like "sounds like your problem could be caused by CRL handling, have you tried adding `check-certificate=no` and verified that the CRL handling options are set correctly in `/certificate settings`" would have been very appreciated indeed. Being rude and talking down to me because I didn't do exactly as you asked is not appreciated. I suppose that if you asked a question about whatever and I answered "try `/system reset-configuration`", you would certainly not just run that command without thinking about it first and trying to understand why it would help?

go ahead alone and don't complain if at some point people don't care about your problems.
I don't whine, you do. And I do think I am allowed to ask for help *and* do some thinking of my own.

I have about twenty CRS112-8P-4S, and I spent my time to test on all, and no one gives me problems on https.
The only thing I can't do is test on your server, which is probably the cause of all the problems.
That is indeed very unfortunate, because it would actually be interesting and potentially helpful to see whether your CRS112s also show the same problem.

And, yes, *of* *course* the problem is somewhere in the combination of my server and the CRS112s. That is obvious, and I never said otherwise. What I am trying to figure out is what exactly the problem is.

... I *can* download via HTTPS from *other* servers just fine...
Not probably... CERTAINLY
Obviously the problem is triggered by some property of my server, yes. Question is, what is the trigger and why is it a problem exactly for those four devices, but for no other clients at all (neither any other MikroTik devices that I was able to try nor any other browser on any other device/computer/mobile whatsoever)? And, to repeat my original question, I am still very grateful for any hints of where to look for the reason.

Cheers,
Toby.
 
tdussa
newbie
Topic Author
Posts: 37
Joined: Tue Mar 06, 2018 2:19 pm

Re: `/tool fetch` via HTTPS times out on CRS112

Fri Apr 30, 2021 8:35 pm

Hi,

I was able to do some more testing. Looked at the ciphers used by my server. It turns out that the problem occurs when my web server and my CRS112 negotiate to use TLS cipher 0x0088 (DHE-RSA-CAMELLIA256-SHA). My other MikroTik devices _also_ agree with the web server to use 0x0088, but for some reason, this is no problem on the other devices, as mentioned before. So I think the root cause of the problem is in the handling of that cipher suite on the CRS112, but I don't see how to look into this further, to be honest. But at least I have figured out a workaround now.

@MikroTik: Are you interested in troubleshooting what goes wrong here?

Cheers,
Toby.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11982
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: `/tool fetch` via HTTPS times out on CRS112

Fri Apr 30, 2021 8:52 pm

... I apologize if I have offended you; I did not intend to. I am still looking for hints as to how to troubleshoot my problem and am grateful for any pointers.
NOOOOOOOOO, you DO NOT have offended me, I simple do not like your way of being presumptuous,
but probably as my way of presumptuousness I beat you ...
if you competed to see who is more (censored), I would win for sure.

Again one last bit of hint,
try this, do not ask a question, I do not reply.
Just, please, if work write it on forum.
/ip ssh regenerate-host-key
(and obviously reply "y"es)

and reboot.

Who is online

Users browsing this forum: Ahrefs [Bot], GoogleOther [Bot], Luanscps and 21 guests