Hello ,
I’m running proxy+hotspot on my RB411
I want to see the websites my users enter in order to get statics ,
my problem is that all HTTPS websites are not shown - (HTTP websites are shown…)
I have try to do
but still I can’t see the https(443) - is there any way to see them?
Thanks ,
You cannot make a “transparent proxy” for https.
This is only possible for http. You need to configure the proxy on the user’s systems to do this.
You can proxy or do anything you want with SSL sessions. You just need the right tools. In order to inspect it you have to inject yourself into the SSL session by having the proxy server negotiate the SSL with the remote server then turn around and talk to the client via SSL and generate a SSL certificate with a CN for the target hostname on the fly. The trick is that the proxy server needs a root CA that is trusted by the client. The only way to do that is generate a root or preferably an intermediate and install it on all of the clients.
An example of this would be Palo Alto’s SSL Inspection feature. They use it to peek inside SSL sessions for nasties like malware. I wouldn’t be surprised if Squid supports this.
MikroTik does not to my knowledge though.
I wouldn’t be surprised if Squid supports this.<
You are correct in some sense. Using squid, it is possible to determine the target domain/IP of the SSL session, without installation of special certs in the clients.
However, a bit tricky. And NOT possible to see the full URL, domain only.
Installing special certs in the clients allows even more with squid; however, this is a real PITA, and because of cert pinning etc. this does not work for all SSL sessions.
The trick is that the proxy server needs a root CA that is trusted by the client.
Yes, this makes it completely not transparent. Actually quite the opposite. User has to accept shady certificate warnings, and is told to ignore them, very bad advice.
Also, as reinerotto already wrote, the browser makers are now actively working on solutions that make this impossible or
at least warn the users that it is going on. That is of course mostly to detect bad practices at certificate issuers, but it
also works against interception by nosey companies and oppressive regimes.
There are pros and conns to SSL intercept. I.e. SSL-intercept could be useful to better detect malware. Or to do parental control more refined.
Or to do better ad blocking. Which, I guess, is the main reason, quite a few “browser vendors” try hard to make this impossible.
I would say there are pros and cons to the decision to move everything to https even when there is no private data to be protected.
Once it is decided that the connection should be private and encrypted, there is no case for breaking that open and looking into it again.
That would make https equivalent to http again, and a new more secure protocol would have to be developed that really protects private data.
Detecting malware is just some lazy excuse to follow your traffic.
DNS can be used for the things you mentioned, but what I am basically saying, privacy is very important.
You can not detect inline-malware, like injected javascript, using DNS.
You could keep privacy using local intercepting https proxy.
But, privacy (read: protection) of advertisers is the crucial issue here.
I was talking about parental controls. If you use verified SSL, there should be no issue with JavaScript injections anyway. That is why you have trusted sites and their certificates. By telling people to accept “false SSL” errors, you are killing the whole idea behind secure, certified and verified web.
+1 to normis. While as I explained breaking SSL to sniff inside of it is totally possible. I generally try to shift customers away from doing it.
The usable explanation is detecting malicious traffic that may be difficult to detect with a method like DNS. An example would be a botnet application that is performing a cnc operation, although likely configured to make a DNS query it could be operating over IP directly and using SSL to communicate. The trustworthy factor of the certificate doesn’t come into play but it can’t be effectively detected and blocked by a firewall capable of doing deep inspection and analysis unless the SSL session is man in the middled. This justification is warranting enough for me to nod my head and begrudgingly turn on SSL inspection for a customer.
The trick here is do you really want to MITM every SSL connection? Do you want to be responsible to be a point where data passes un-encrypted temporarily? Do you want to be responsible to be a trusted CA to the clients that connect behind it? Have you taken the necessary precautions to protect the root key or hopefully the intermediate root key you are using for SSL interception?
These are important things to think about. The saving bandwidth for web-browsing argument of proxies is largely dead at this point. With CDN nodes and membership programs with the various CDNs you can save traffic in a layer 7 agnostic way not to mention non-video web traffic is negligible at best for all but the most constrained connections.
Some of you have touched on one topic I’d be keen to think about enabling SSL inspection, parental controls. That said the majority of the bad content on the Internet can be blocked by a good upstream DNS service like OpenDNS which comes with a free home product that blocks adult out of the box. For me personally, as my son begins to explore the Internet, I think I’m going to follow the route of DNS protection to catch the most obvious of negative content and actively coach him on how to use and act on the Internet. After all, he’s going to be on the Internet a lot longer than I was and definitely after he is out from under my wing.
As far as the argument that SSL breaks ad-blocking. A good plugin can see the page data and will block or change it there. ABP in Chrome works wonders on HTTPS enabled sites. The argument for HTTPS around ads is to prevent a middle party from injecting an unwanted ad into a web-page. A recent report of it happening is http://webpolicy.org/2015/08/25/att-hotspots-now-with-advertising-injection/ but for a while this was a revenue stream most large ISPs in the US tried until the tech media melted their faces for it. It’s also the reason I don’t use my upstreams DNS servers. I’d rather give away what I’m querying to Google than get sent to an ISP controlled ad-page every time a URL doesn’t resolve to an IP on the public side. TLDR; the advertising side of the argument is that HTTPS prevents more ads than it protects.