so, I first allow 80 for WWW and disable DSTNAT on the external server. I also delete the old certificate first because I remember there was an error before some verion routeros they didn't update the existing certificate, but added a new one, and over time, there were many copies. I also saw adding a new certificate in the on-error section so that the script wouldn't terminate if an error occurred, otherwise the last part of the rules wouldn't apply.
But it doesn't work? What's wrong? What broke the new version in 7.20.7?
What part doesn't work? Does the log has the red "Failed to renew certificate!" message? You may check whether www is enabled under IP -> Services, and whether this following setting is active:
using https://sslip.io/ the free one for the test, then everything works, [admin@MikroTik] > /system/script/run script2 progress: [success] ssl certificate updated [admin@MikroTik] >
I understand nothing..
for the test, I also installed debian and certbot and successfully obtained a certificate for 12345678.sn.mynetname.net why mikrotik himself can't get it, I do not know
Ah, sorry, that dialog above is only available from 7.21+.
If you want to get the certificate for the IP cloud domain (xxxxxxx.sn.mynetname.net) then don't specify the parameter dns-name with the enable-ssl-certificate command (omit it).
It's a bit subtle in docs, but automatic renew without opening port 80 works only if /ip/cloud name default is used:
IF the used DNS name is not the default http://example.sn.mynetname.net, port TCP/80 must be available from the WAN.
The certificate is automatically renewed when 80% of its validity period had passed.
But I still think you need port 80 open when using dns-names=. The v7.21 opition for "ACME challenge" is just an additional layer, but the port has to be opened to get the /ip/service/webserver setting.
I can't, because the address for the cloud is set by my other mikrotik on the local network,
cloud is turned off on the central mikrotik, which manages all WAN connections.
I just did some tests, and here are some findings:
Using enable-ssl-certificate with multiple other subdomains (non IP Cloud) pointing to the same WAN IP address -> Port TCP 80 is contacted by multiple remote hosts during the request. Certificates could successfully be acquired and process took a few tens of seconds each time.
Using enable-ssl-certificate with dns-name using the IP Cloud subdomain of another router located in my LAN, that also points to the same WAN IP address -> progress: [error] failed to update ssl certificate and failed very quickly, process only lasted a few seconds. During the whole time, there was no connection attempt to port TCP 80 on my WAN IP!
Using enable-ssl-certificate with dns-name using a domain pointing to another location that I control -> progress: [error] failed to update ssl certificate too, however process lasted much longer (a few tens of seconds) and the other location did see incoming connection to port TCP 80.
Using enable-ssl-certificate with dns-name using the IP Cloud subdomain of another router located in another location (not the same WAN IP) -> progress: [error] failed to update ssl certificate too but process was very quick (a few seconds) and the remote location saw no connection attempts to port 80.
Conclusion: I think if you use the xxxxxx.sn.mynetname.net, even if specified manually with dns-name, then the router will try to use the DNS-01 challenge (with help from MikroTik's server), and no connection to port 80 will be made. And the MikroTik server probably refuses when xxxxxxx does not match the current serial number of the device, without even trying to contact Let's Encrypt, hence why the response was so fast.
Either you'll have to use the router with enabled IP cloud to request the certificate if you really need the xxxxxx.sn.mynetname.net subdomain. Otherwise it might be better if you use another (sub)domain, pointing a CNAME record to xxxxxx.sn.mynetname.net, and request the certificate with that (sub)domain. You can purchase a domain for some $ yearly, or maybe use a free subdomain from one of the Dyn DNS services.
Interesting idea in trying xxxx.sn.mynetname.net, but I guess the results are expected that dns-name= always use HTTP-01 auth.
And, while 7.22 adds the ability to you to run enable-ssl-certificatemultiple times, it does not add "automatic renewal" to any of them. Given only HTTP-01 auth is support for dns-name, means opening port 80 to enable-ssl-certificate dns-name=.
And DNS-01 still does not really solve auto-renewal since the DNS needs to be update each renewal, I don't see MikroTik adding the dozens of DNS providers APIs that a "real" ACME client use to handle DNS-01 required TXT RR updates.
Obviously the core problem in HTTP-01 auth is that port has to be opened to internet, and given it's unsecure that may not be desirable. So you still need to some script to deal with renewal when using dns-name, and that script get tricky since it's not just the enable-ssl-certificate, it's also enabling port 80 in firewall from internet, disabling same firewall after cert or failure, and ideally some retry logic, logging, email alert, etc) .
@CGGXANNX points out the new /ip/service/webserver controls, enable-ssl-certificate IMO. So that helps to limit what's exposed on port 80, like enabling acme-client=yes but none of the *-plain=no when port 80 is open. But that does not help that much with the complexity to maintain custom domain certs..
But nothing is really "automatic" with certificates and custom domains. So "best" approach is often using CNAME in your custom domain that points /ip/cloud DDNS name, but you'd still get certificate errors since the DNS name won't match when using CNAME indirection.
Hm, but when I did the tests above, specifying dns-name=xxxxxx.sn.mynetname.net, with xxxxxx.sn.mynetname.net pointing to my WAN IP, and the main router on that IP has a DSTNAT rule to the test router, with logging, no incoming connection to that DSTNAT rule is registered. And enable-ssl-certificate fails after a few seconds.
Specifying dns-name with other subdomains I own, also pointing to the same WAN IP, then the DSTNAT rules registers incoming connections from multiple IP addresses, and enable-ssl-certificate succeeds for those subdomain.
Which means HTTP-01 was not used when using dns-name with IP Cloud domain not owned by the router. That could explain the problem in the OP.
Automatic renewal could work with the idea I put out in this post V7.22beta [development] is released! - #56 by CGGXANNX and no APIs of DNS providers are needed. During the request or renewal it's MikroTik that updates the TXT record of long_hash_string.acme.mynetname.net with the content required by Let's Encrypt while _acme-challenge.real.sub.domain is simply a CNAME to that. This will allows DNS-01 challenge for real.sub.domain to work.
The user as owner of real.sub.domain only has to add the CNAME record once.
I've posted a support ticket with that idea and got this answer to the ticket:
We are aware of the current limitation, perhaps something from the proposed will be added in the future.
And they have not yet closed the ticket, so maybe something similar might come in the future.
Likely. But at some older version you could use dns-names= with the "mynetname.net".
In fact I used to regularly do this (use DDNS name, but also our FQDNs)... until the automatic renewal support was added. Since any dns-name= broke the auto-renewal and wanted avoid dealing with port 80... I went the CNAME to /ip/cloud road to allow autorenewal to work. But I didn't know DDNS name in dns-name= just didn't work now...
I'm still not 100% a CNAME can point to TXT record in DNS rules in that approach. Perhaps it's okay, and also I don't know who'd enforce redirection rules from ACME POV, so IDK.
Now I'm less picky on the implementation particulars, and get it not an easy problem...
But better support for auto-renew with custom domains be great!
Which is a working free helper service. They allow you to update the TXT record of the subdomain with the hash if you know the secret that produces the hash. You send a simple web request with the secret and the challenge content from Let's Encrypt.
So automating HTTP-01 so port 80 is only open during operations and only for acme-client be equally acceptable to me. And avoid a depencency on /ip/cloud for certs.
Like I said any solution for automatic cert with custom FQDN be welcomed. I already have a depencency on /ip/cloud today, so your approach work just as well.
That method is currently not easily doable with only RouterOS and scripting.
You still need to some run tools (for example certbot) in container. Because the DNS-01 challenge requires something that contacts the ACME server, initiates the request, gets the challenge string, updates the TXT record in DNS with the string (this is the step that challenges.addr.tools can assist), notifies the ACME server, gets the issued certificate.
Of course instead of the tools you can write scripts that issue fetch requests and analyze response manually, but that would be like reimplementing certbot in RouterOS script
Then you'll also need something that updates the RouterOS certificate store with the acqired certificate and updates www-ssl setting.
That's why it would be much better if all the steps above can be done by RouterOS itself, with assistance from MikroTik's server for the role of challenges.addr.tools.
I support all useful features and I'm not trying to stifle progress, but ultimately I think that getting certificates issued, especially via dns challanges, is best left to certbot. It has a properly full feature set, and either via official extensions or purpose-built scripts, it supports the most common dns providers already.
What I do think would be very useful is a fairly standard and reasonably version-independent utility for shipping certs to RouterOS, perhaps via the REST API. Now that I would be interested in.
This could also run in a container on the router itself along with certbot, if so desired...
I think that creating a built-in version of certbot, cloudflare dns updater, acme.challenge client (and then why not route53?) an unending path to bloat. It's also clear that the vendor supplied scripts and helpers can't run on RouterOS natively, but have to be re-implemented, which is a maintenance nightmare.
I also agree that bloat should be avoided as much as possible. However, there are quite a few features/services of RouterOS (that I actively use) that greatly benefit from the ability to issue valid TLS certificates for subdomains of choice, on the devices that run the features/services, without the subdomains having to point to some public IP addresses, or having to temporarily open port 80, examples are:
Hotspot with HTTPS
User Manager, that acts as RADIUS server for WPA Enterprise and dot1x.
SSTP server
The newly introduced reverse proxy features, which will be much more useful with wildcard certs.
For those RouterOS services, having the certificates being renewed on the device itself with DNS-01, without having to move certificates and private keys across devices, without having to forward ports and run www, and with the domains pointing to fully local or even non-existing IP addresses, without the need to setup containers and external tools, is a very big plus.
And if you read the proposal linked above, there is no need to implement individual compatibility layers for Cloudflare DNS, Route53, & Co. No need for 3rd party API integration apart from DNS-01 itself. MikroTik only have to implement a small service on their server, that answers DNS queries for short-live dynamic TXT records (you can see the source code of the backend of challenges.addr.tools on GitHub, it's very simple, but unfortunately due to the License used, MikroTik should not look at that source code when they build their version). But I think MikroTik already have something like that on their server because right now when you use the RouterOS to get LE certificate for the IP Cloud subdomain, DNS-01 is already being used.
Maybe I misunderstood. As far as I can tell, the current automatic certificate system does roughly what acme.challange does. You could easily cname an _acme-challenge to MT's dns. The thing it does not do is allow you to submit other dns names for verification. If that's your feature request, then that's pretty reasonable.
What doesn't sit well for me is that (in general) whoever controls the way to update the _acme-challenge txt record, either directly or via a cname redirect, controls the issuing of certificates, and is that sense has the "keys to the kingdom." A router that's otherwise forwarding traffic is probably one of the worst places to have this kept.
I also happen to think that anyone who is serious enough about their business to have a domain and automate certificates for it already has some method of doing it. Many are available.
I think your ask to be able to relatively freely parameterize the Mikrotik acme client is reasonable, but I really wouldn't suggest anyone who is serious about what they're doing to rely on it too extensively.
My described process is meant for custom domains (including wildcard domains / subdomains), such as, let's say, the hotspot that has the URL guest-wifi.my-restaurant-name.com, and I own the domain my-restaurant-name.com. And let's say the restaurant only has one RouterOS device that also manages the hotspot. Currently with the built-in RouterOS tool we have to have guest-wifi.my-restaurant-name.com A or AAAA record pointing to the IP address, or guest-wifi.my-restaurant-name.com CNAME to the IP cloud or similar DDNS subdomain. Then during the renewal, currently we need to keep www running and port 80 open (and/or DSTNATed to the RouterOS device) if not using IP Cloud. But using IP Cloud still doesn't support wildcard domains, and the domain still has to point to the public side IP address (through CNAME) even if not needed. The alternative is to request the certificate with external tools/services, and transferring it to the RouterOS device.
It would be better if the RouterOS device supported the proposed DNS-01 challenge workflow. The restaurant owner would only need to create a _acme-challenge.guest-wifi.my-restaurant-name.com CNAME record (once) pointing to 6a58297848d3864fa2dd706a2875f75ebb9b1c0a9306bfb93982d3d8.acme.mynetname.net (an example hash generated from the secret set in the entry on the router). No DNS record even needs to exit on the internet for guest-wifi.my-restaurant-name.com (because that subdomain is only used inside the hotspot network). RouterOS will take care of the rest when generating / renewing the certificate. It will perform the DNS-01 challenge, with the step where the TXT record needs to be updated is done through RouterOS sending the challenge text and the secret to MikroTik server so that the temporary TXT record for 6a58297848d3864fa2dd706a2875f75ebb9b1c0a9306bfb93982d3d8.acme.mynetname.net can be present for a couple of minutes. No need for www running or inbound port 80 access.
RouterOS would have a table where entries for multiple custom domains/subdomains can be managed. Each entry only requires an additional field (beside the name of the subdomain) for the secret string, and maybe a "Is Wildcard" checkbox, and can display the calculated matching xxxxxx.acme.mynetname.net subdomain (so that the user can create the required CNAME record). Renewal for the whole table can be performed automatically by RouterOS. The domains can point to arbitrary IP address (including local) or don't need any record (when the TLS certificate is only used for RADIUS for example).
Mikrotik's cloud already does the part around publishing the verification codes and has the ability to (somehow) authenticate the router. All that's needed is the ability to request a dns01 verification for non-sn-mynetname names, including wildcards. That's not a really big request.
The part that I'm uncomfortable with is that this makes the router able to generate these certs for itself (okay, with MT Cloud's and Let's Encrypt's help). I don't think that this should be generally encouraged. Again, I see the feature as useful, I just wouldn't use it in a scenario like the one you refer to.
You can already do the same today with a ssl-certificate dns-names=x.example.com,y.example.com,..., you just need to make sure port 80 is open. But the crux of issue is you need to repeat that manually (perhaps via a custom script) to renew the cert every 90 days.
Now I had similar concerns orginally with @CGGXANNX approach. But given the https://challenges.addr.tools example, it seems the approach is allowed and works. And then just one step, once, to add a CNAME to your custom DNS to enable it.
Although still think this is also solvable using more dynamic rules to allow port 80, and do same certificate renewal request at 80% of lifetime. Or perhaps using TLS-ALPN-01 as alternative to HTTP-01, which avoid port 80 needing to be open.
But leveraging the existing logic of /ip/cloud feels cleaner than the complex dance that's be needed to open port automatically and safely.
