Hotspot Odyssey with Apple devices

Good morning everyone,
this post doesn’t want to be only a request for help, but also a help for other users who are in the same problem as me,
and a feedback for the mikrotik developers, that’s why it will be a long post (and I apologize if this causes problems for the moderators)

The scenario is that of a kiosk in southern Italy, facing a beautiful beach,
very busy, with no possibility of coverage or telephone (no telephone, fibre, ADSL, etc) e
without cellular coverage (not even gsm or sos).
Given the need for internet for the activities, the manager of the kiosk
he closed a contract with a small local company which guarantees him through radio links
( at high prices ) an internet connection.
The owner of the kiosk had the idea of creating a hotspot linked to the consumption of
lunches or dinners at your local, with a system that the more you consume at the restaurant and bar the more
Giga of internet traffic you have.

It was decided not to make large investments immediately, but to leave
with a small investment, see how it proceeded and the demand from consumers and then develop it in the future
depending on how it works.

The system consists of n. 1 hEX S, connected by cable to the radio link ( WAN ) and n. 3 Cisco Aironet 3702i in autonomous mode, n. 1 PC Veriton N281G used as a logger.

Step 1
Knowing the problems of Apple devices we have set SSID
short, no spaces, no numbers or special characters, and in open network.
On the mikrotik hex we have activated the hotspot, radius, and User Manager
The system expects that it is the manager who registers users in the system and assigns them
a Profiles and activates them.

Problems encountered
Many devices connected to wifi even without using the network,
brute force attempts on hotspot username and password.
Connection problems for Apple, android and windows

Step 2
We set up the network with a WPA2 TKIP key.
Entered in the Hotspot.Walled_Garden a rule for path /generate_204 (android devices)
We changed the dns name to the Hotspot by adding .lan at the end ( kiosk → kiosk.lan )

Problems encountered
No bruteforce on the hotspot page, wifi bandwidth ok and no longer saturated.
Android devices OK (CPD (captive portal detection) starts automatically and the page for requesting credentials is shown)
, apple and windows always with problems (the connection to the wifi network is there, but no web page is displayed).

Step 3
After spending a week studying the system logs I saw that windows CPD attempted to access the following
url: /connecttest.txt, /canonical.html, *.msftncsi.com, *.msftconnecttest.com, /ncsi.txt, *.msedge.net and sè can’t find them
directly accessible blocks Layer 3 traffic, while the ip and dns system work correctly.

Inserted in the Hotspot.Walled_Garden finally even the windows machines were able to connect!!

Android OK, Windows OK … Apple KO :frowning:

Step 4
I borrowed an ipad and ran a series of tests and found the following problems:
Ios does NOT want the open network with captive portal, it does not want TKIP, it just wants WPA2 AES and WPA3 with captive portal, otherwise the network is blocked when accessing the wifi.
Then it activates the CPD ( captive portal detection ), and here it tries to communicate with a series of servers expecting a specific response.
After various tests I entered the following url /hotspot-detect.html , /success.html , /detect.html , *.apple.com, airport.us, itools.info in the Hotspot.Walled_Garden .

Unfortunately, still 30% of Apple devices had problems accessing the captive portal page to authenticate.
I also had negative feedback from people with vision problems to whom the default theme of the hotspot created problems in recognizing the position of the fields to be filled in with user and password

Step 5
I read everything I found on the internet about the mikrotik hotspot system, until analyzing the present css I found
the class “lite”, and it was enough to change with in all the html files to obtain a high contrast page that was also good for those with vision problems.
I have enabled MAC cookies and set to 3 days.

I devoted myself to understanding how Apple implemented the CPD system, eventually realizing that the real problem is not the CPD,
but the internal dns cache of the device which is updated more and more slowly for the values that remain stable.
Testing with some customers’ iPhone 13s, I saw that no DNS requests were made to the network, but there was a whole series of
connections to servers such as e6858.dscx.akamaiedge.net, captive-cidr.origin-apple.com.akadns.net, xxxxx.dsct.akamaiedge.net .
Not being able to act on the dns, not even on the device and not even on the Hotspot.Walled_Garden, I saw that what united the requests
was the User-Agent which was always of the type: “CaptiveNetworkSupport-407.0.1 wispr” , “CaptiveNetworkSupport/1.0 wispr”, “CaptiveNetworkSupport-355.200.27 wispr”,
“CaptiveNetworkSupport wispr”, etc.

I decided to edit rlogin.html and login.html as reported in old posts

$(if user agent == CaptiveNetworkSupport)

SuccessSuccess $(else) ..... $(endif)

And finally on many recent Apple devices it works!! .. and now a question:

What regex can I use in the if to match all CaptiveNetworkSupport types?
i tried with ^CaptiveNetworkSupport* but it doesn’t work…

For the developers of mikrotik: in the future it would be possible to include in the
Hotspot.Walled_Garden a rule for User-Agent?

Thank you all and sorry for the length of the post