Let's face it. Intercepting https requests to redirect users to the login page, is a bad idea. And as we move forward and more sites start using HSTS, it gets worse. it also undermines the seriousness of certificate error messages by the browsers as users could easily get used to that and click continue as soon as they see the message anywhere.
Using https for the login page in a LAN environment however, is a good idea(as long as you bother getting a valid cert) as it would allow securely transferring users credentials. The TLS layer, would ensure that to some very good extend.

The default behavior of Mikrotik hotspot when using https authentication method however, is to also intercept all 'un-catched'(as in there is no Walled Garden IP rule for it) and unauthenticated https traffic and redirect them to the login page. this of course results in an ugly certificate warning message in the users browsers.

Luckily, we are able to disable this interception by a NAT rule in pre-hotspot chain, accepting dest-port 443 if the traffic is coming from an unauthenticated client and is not destined to the device itself. This however, also disables the neat feature of Walled Garden(No, i do not mean Walled Garden IP), which is capable of dynamically allowing access to https hosts for unauthorized users by comparing the destinations IP address to the local DNS cache(at least i think that's how it works). I really like this feature. it works reliably, does not add any additional IP based rules in filter/nat tables(unlike Walled Garden IP), and can perfectly handle the most complex domains with hundreds of IPs and very low DNS TTL as long as clients use Mikrotik DNS server exclusively (which is the case in hotspot enviroment).

Unfortunately, it does not seem easy to reliably differentiate between the encrypted traffic for when the router is trying to intercept an external host, from it trying to display it's login page directly (both use the same local port: 64875 ). This means we effectively loose Wallden Garden functionality for port 443 as well.

Probably the best way to deal with this, is by adding a new option in hotspot server settings to disable interception of port 443 for external hosts without sacrificing walled garden functionality.

I am not in this business so I am not aware of the most recent developments, but I wonder if there is no widely
accepted method of handling logins in a hotspot environment without having to resort to snooping internet
traffic. Something distributed via the WiFi layer, the DHCP server, the DNS proxy, whatever, so that the client
knows it first has to go to some URL before trying to connect to the outside world?

@pe1chl
As far as i know, there is no standard method for this. probably the best solution so far, comes from chrome/chromium ( https://docs.google.com/document/d/1k-g ... oW9WRaEmfM ). it is a good one, aside from the fact that it's been disabled in windows 8+. sure, newer versions of windows are capable of recognizing captive portals by using similar methods and open up web browser automatically, but they only do that once (either at start up or when the interface comes up). it might be enough for those users who just need to login once a day but it's not gonna work the second time.

Unfortunately as you said, we do seem to lack a standard method for captive portal recognition. but still some methods could be argued to be better than others.

That is a good document (are documents like that stored at some searchable place?), however it is describing
a complicated method to work around problems encountered in established hotspot networks.
It would be so much easier if there was some convention like "do a DNS lookup of captive.portal.local" that
could return a CNAME or addresss of a hotspot login page on a network that requires it. Preferrably the
client would be able to detect that it already has logged in, from the response code received when fetching
that page. Anyone wanting to fix their MikroTik hotspot would only need to create a static DNS entry...

I'm not sure why i didn't know about this until now but it seems there have been some attempts for a standard method: https://tools.ietf.org/html/rfc7710
It still however, doesn't seem to address situations in which the user might logout of captive portal at will and would need to re-login later on

Right - a DHCP option. That is one of the solutions I was thinking of.
Well, you could try putting that in your DHCP server, but maybe it is better to first investigate
if this option is used by anyone
Of course the option would put the system only in the "aware of captive portal" state where it
would go back to the login page whenever unexpected things happen that point to a block
in the network. Detection similar to what that Google document explained (error 204), but
without the need to intercept traffic and do funny redirects.
(only Microsoft would make an implementation that works only once...)

Yes, i will try the DHCP method. certainly including it would be better than not to. That however, does not invalidate my initial feature request. the issue still stands and it might be years before we end up with a standard acceptable solution that could deal with all situations.
Thank you for your advises. they were very helpful

I would love a Mikrotik staff opinion on this if possible please.
I was by the way able to develop a very hacky way to distinguish between https traffic passing through walled garden and https traffic being intercepted. I however abandoned that approach since it would cut off the connection at an awkward state without the possibility to send appropriate RST packets, leaving both ends in an endless re-try loop.