Hotspot Walled Garden https problem

RouterOS General
1

Hi all.
I am trying to work with hotspot, but I have a serious problem.
I need to walled garden all facebook.com, and also .akamaihd.net (for Facebook’s contents). If I insert in walled garden dst-host=.akamaihd.net and I try to get https://fbstatic-a.akamaihd.net/rsrc.php/v2/yG/r/oeyMmgGXkU1.js I get a SSL error, but if I try to get http://fbstatic-a.akamaihd.net/rsrc.php/v2/yG/r/oeyMmgGXkU1.js it works.

The strange thing is that if I insert in walled garden dst-host=* dst-port=443 and I try to get https://fbstatic-a.akamaihd.net/rsrc.php/v2/yG/r/oeyMmgGXkU1.js it works.

Obviously, I cant walled garden all https hosts…

ROS 5.24 on RB493G.

Any ideas?

2

Try inserting the entry into [/ip hotspot walled-garden ip]under dst-host rather than directly into [/ip hotspot walled-garden]

3

Hi TheWiFiGuy,
thank you for the answer.
fbstatic-a.akamaihd.net has tens of IP, is not simple to place all in walled-garden ip.

I founded an other problem, but I don’t know if it is related.
In hotspot ethernet segment, if I am non logged and I nslookup fbstatic-a.akamaihd.net, I obtain a wrong answer:

PC:~ User$ nslookup fbstatic-a.akamaihd.net
Server: 8.8.8.8
Address: 8.8.8.8#53

Non-authoritative answer:
Name: fbstatic-a.akamaihd.net
Address: 0.0.0.8
Name: fbstatic-a.akamaihd.net
Address: 0.0.0.1
Name: fbstatic-a.akamaihd.net
Address: 0.0.0.3
Name: fbstatic-a.akamaihd.net
Address: 0.0.0.4
Name: fbstatic-a.akamaihd.net
Address: 0.0.0.5

This is the correct one:

PC:~ User$ nslookup fbstatic-a.akamaihd.net
Server: 8.8.8.8
Address: 8.8.8.8#53

Non-authoritative answer:
fbstatic-a.akamaihd.net canonical name = fbstatic-a.akamaihd.net.edgesuite.net.
fbstatic-a.akamaihd.net.edgesuite.net canonical name = a1168.dsw4.akamai.net.
Name: a1168.dsw4.akamai.net
Address: 66.171.231.65
Name: a1168.dsw4.akamai.net
Address: 66.171.231.32
Name: a1168.dsw4.akamai.net
Address: 66.171.231.24
Name: a1168.dsw4.akamai.net
Address: 66.171.231.62
Name: a1168.dsw4.akamai.net
Address: 66.171.231.94
Name: a1168.dsw4.akamai.net
Address: 66.171.231.25
Name: a1168.dsw4.akamai.net
Address: 66.171.231.31
Name: a1168.dsw4.akamai.net
Address: 66.171.231.102
Name: a1168.dsw4.akamai.net
Address: 66.171.231.16

Is the problem related?

4

You can post the URL in the walled-garden-ip - it accepts an IP or a Hostname.

If you add the hostname, the Mikrotik will go off and resolve all the associated IPs.

5

Hi gallysoft,

I’m experiencing the same problem as you. Have you found a solution?

By now I’ll tell you what I have discovered till now.

The main problem is that facebook uses several host names and changing IPs and also it uses Akamai for content distribution with dinamically generated DNSs, so it is not possible to enter all the host names/IPs used by facebook in the Walled Garden (WG).

You could enter partial host names with wildcards like *facebook.com or *akamai.net in WG, but the problem here is that Facebook uses also HTTPS protocol and this technique wouldn’t work because WG, which is specific for HTTP and HTTPs protocols, extracts the host name from the HTTP header and the header in HTTPs is encrypted.

So the only chance is the WG IP list. Here, you should enter the IP or the dns of the hosts you want to allow access. The host names here must be fully specified (you can’t use wildcards) because WG IP performs an inverse query to obtain the IP from the DNs and add it to the WG. So this is not a possibility. But you can enter IP ranges in the form of 31.13.24.0/21 and you can get a list of IP ranges used by Facebook at a given moment with this linux command:
whois -h whois.radb.net – ‘-i origin AS32934’ | grep ^route

… But I don’t now yet how to get the IPs for akamai.net and akamaihd.net which are used by facebook …
Any idea?

Another problem I’m having is that if the users enters a Https address in the browser before login he gets redirected by a dinamically generated NAT rule to the hospot login servlet for https (port 64875). By default you will get a “page not found” message in the browser resulting in the user not being redirected to the hotspot login page. In order for this to work you will need to enable HTTPS login in the hotspot profile and install a valid SSL certificate. In my case, as the certificate was generated by myself, I’m getting a security warning in the browser. I wonder if to remove this warning I necessarily need to buy a certificate or there is another way to solve it (ie redirecting https (443) to the http login servlet (64873)).

Anyone has a clue on these matters?

6

Same problem with PayPal. It is not only the multiple ips for the servers, but the very short time that is set in the TTL for the resolution. PayPal can be just a few seconds.
http://wiki.mikrotik.com/wiki/PayPal_with_hotspot_and_walled_garden_bypass

The dns cache is used to allow your requests to the walled garden, but some sites use a dynamic dns to load balance the severs. The walled garden will initially let you through to PayPal, but if you wait more than a few seconds to access anything else on the PayPal site, like make a payment, it may fail. If you wait a couple minutes, which most do filling out the payment page, it will fail.

The script above captures those ip addresses and enters them in the “/ip hotspot walled-garden ip” list so they will be available for more than just a few seconds. Maybe you could modify it for facebook.

7

Thank you SurferTim for your help,

I will have a look to these scripts. Do you know if this is still an issue in RouterOs v6.1?

8

Probably. Unless Mikrotik changes the way the hotspot walled garden works, it will remain a problem. For example, I have found the PayPal dns TTL is about 15 seconds. There is no way you can fill out and submit that PayPal payment page in 15 seconds.

9

This should be better documented, we would’nt have to specullate about how it is implemented and we would save many headaches…

Now I’m trying to guess how all this stuff works…

It makes sense what you say about the use of the DNS cache for applying the rules of the WG. This is how it seems to work for the entries entered in the Walled Garden tab. So now I think that I was wrong when I said that WG “extracts the host name from the HTTP header” and thus this wouldn’t work for HTTPs. May be I got confused by the TTLs of the dns cache and that led me to that wrong conclusion.

However, as routeros v6.1, when adding a walled garden IP entry, several dynamic rules are created in the WG (and FW and NAT) that get refreshed periodically. In my case, there are many dynamic rules for a single destination host with different ips and some ips that aren’t in the DNS cache anymore. So this is why I think that when you enter a host name in the WG IP List it seems to act like the scripts in the link you provided, that is, executing a scheduled job which resolves the host names and adds dynamic rules to WG.

By the way, in my case the list of dynamic rules in WG/firewall and NAT is growing continuously. By the moment I can see about 1000 entries! I wonder if it will make a “flush” in any moment or it will fill all the memory of the router…

10

I think this is my first post on this topic. September, 2010.
http://forum.mikrotik.com/t/walled-garden-and-ssl-sites-intermittent-problem/40829/1

It is in “/ip dns cache” that this all happens. Initially when your web browser tries to access a SSL site, it first does the dns resolution. That puts the ip of the server in the cache. Later, when the web browser tries to access that ip of the SSL site, the hotspot looks through the cache list of ip addresses. If it finds that ip, then it checks the associated domain name and checks the walled garden domain names for that domain.

The problem lies in the fact that PayPal, and other busy sites, use the dns TTL as a form of load balancing. PayPal uses 4 severs at a time, and rotates through those ips every 30 seconds or so.

For examples, presume these are PayPal server ips for today.
x.x.x.20
x.x.x.21
x.x.x.22
x.x.x.23

When your web browser does the initial dns resolution, it gets x.x.x.20. So you go to that payment form page right away, and it works because x.x.x.20 is in the dns cache with a valid domain to bypass the hotspot.

But by the time you fill out the form, and want to submit your payment form, the ip address x.x.x.20 has been dropped from the dns cache because the TTL expired, and you are NOT allowed through the hotspot,

If another user on the same hotspot does a resolution for PayPal, the odds are 1 in 4 that they will put that ip (x.x.x.20) back in the dns cache for 15 seconds or so. So if you are really lucky, you might get let through with your payment form if your timing is almost perfect.

edit: The theory behind my wiki code is that it uses the system scheduler to resolve the domain until it gets all 4 daily PayPal server ips in “/ip hotspot walled-garden ip”. Now, no matter which of those 4 ips your client gets, they will all be in the walled-garden ip list and will bypass the hotspot ok. It no longer matters if the TTL is short and drops from the cache.
http://wiki.mikrotik.com/wiki/PayPal_with_hotspot_and_walled_garden_bypass

If you think it would be as easy as getting those ips and entering them in “/ip hotspot walled-garden ip” manually, that won’t work. I have found they rotate one of those servers out every day or two. Then it will use these ips:
x.x.x.21
x.x.x.22
x.x.x.23
x.x.x.24

If I worked at MikroTik, which I don’t, here is the fix I would try:
I would develop a routine that, when the domain is resolved by the hotspot, and that domain is in the hotspot walled garden, it adds the destination ip to a special address-list with a timeout of 1 day. Then use that address-list as the bypass check. You could make the “/ip hotspot walled-garden ip” that address list by adding an expires field to it. :wink:

11

Of course, you could just do i the way I posted twice.

Using DNS HOSTNAMES in /IP walled-garden-ip results in ROuterOS resolving ALL ips against a domain, every few seconds. We use for 1000’s of sites using Paypal, Facebook and other social media connectors.

We deal with 5,000,000+ logins per month, no issue reported through our helpdesk since implementing.

This works in all 6.x versions, and later 5.x versions.

12

Dont worry, we have over 10,000 entries for some sites with lots of social connectors.

13

OK! Now it works with no duplicate dynamic entries, thanks to Maris at support. You can’t use “dst-host” and “dst-address” in the same entry in “/ip hotspot walled-garden ip”. In V5.x, you will not be able to use both in the same entry. The scripts are in the wiki if you want to give it a try. Using these scripts, I have no fails to PayPal through the walled garden.

@WiFiGuy: This is from that other topic. Does this mean that you have been successful with using dst-host in “/ip hotspot walled-garden ip”? If it resolves PayPal every few seconds, you could end up in the same predicament as I found myself in. It returns different ips 3 out of 4 times. ??

Unless it caches ALL those ips somewhere, it will fail most of the time. The client does one resolution for that domain, and uses that ip for the entire transaction. Under the current scheme, the client ip must match the resolved ip for that domain, and that will happen only a quarter of the time on the average.

add: After dwelling on it for a while, caching the ips in an address list (or similar) for a specific time period as the hotspot dns resolves them (and it matches an entry in the walled garden) is the only efficient way. I feel bad for my dns server by doing it my way, and I don’t resolve the names nearly that often.

This way it is only storing ip addresses that matched an entry in the walled garden AND was issued to a client. That should cut down the wear and tear on the dns server, and the related internet traffic.

14

So it sounds like this has been made to work for Paypal, but has anyone had success with facebook? Can the paypal scripts be modified for facebook and if so how?

Thanks in advance..

15

I still not tried the script but in order to find a rapid solution I did this:

/ip hotspot walled-garden ip
add action=accept comment=AKAMAI disabled=no dst-address=5.178.42.0/24

It solved my problem.

I will check the script and find a solution. I will update this thread.

16

I modified the Paypal script in order to work with Facebook and Akamai.
It worked like a charm!

17

planetcaravan, would you mind sharing your scripts ?
Running into the same problem here

Greetings
steven

18

Hi,

I just used this http://wiki.mikrotik.com/wiki/PayPal_with_hotspot_and_walled_garden_bypass and did an edit in order to use Akamai.
Let me know if it works for you.

19

Still using your “PayPal script” on my hotspots. 2 years later Tim. Saved me a LOT of Grief! :smiley: :smiley: :smiley:

20

Thanks! It is good to hear my script is still useful!

Actually, I was hoping the Mikrotik crew would have come up with a fix to eliminate the need for that script by now.