Using version 3.7 x86.

Doing transparent proxy, but also using it as normal proxy.

All https addresses don't work if you specify to use the proxy.
If you don't specify obviously it works because ssl isnt and cant be forced through via firewall rules.

The requested URL could not be retrieved

While trying to retrieve the URL: http:443

The following error was encountered:

Unable to determine IP address from host name for

The dnsserver returned:

Name Error: The domain name does not exist.

This means that:

The cache was not able to resolve the hostname presented in the URL.
Check if the address is correct.

Your cache administrator is xxx@xxx.xx.xx.
Generated Mon, 21 Apr 2008 07:37:43 GMT by thavinci.za.net (squid/3.0.STABLE1)

This message in truth comes from my upstream cache, but it sedning this error back to MT because it seems MT somehow looses the address in the request...


Very Odd!!!

Of course they don't work. The key word is "secure" in the HTTPS part. If you are trying to force a connection through a proxy AND trying to keep it secure then you won't succeed.

Remember, in general a proxy is something that acts on behalf of something else. So when using https (or port 443), you really don't want something else in between you and your secure site (like your bank for instance).

Hope this makes sense?

ROFL

I get what you're trying to say.
But the fact is that it should go through.

Brief description:

Working as a normal proxy, squid can tunnel SSL requests when are requested by a HTTP user-agent (Netscape Documentation) vía HTTP proxies.
This involved a HTTP method (CONNECT) for establishing the tunnel.

But in a interception proxy know as transparent proxy as well, the proxy becomes the server for the client and becomes the client for the web server. The connection between the two parts who starts the connection is broken and the identity of each is hidden (SSL), so in this special case the transparent proxy doesn't know how to handle the SSL requests because is not operating as a normal proxy.

Infact Microsoft ISA and squid (the open source product MT uses) allows you to do this.
It is still a secure connection even if you go through a proxy.

The only thing thats not possible is a transparent HTTPS proxy.

Finally, as far as transparently proxing HTTPS (e.g. secure web pages using SSL, TSL, etc.), you can't do it. Don't even ask. For the explanation, do a search for 'man-in-the-middle attack'. Note that you probably don't really need to transparently proxy HTTPS anyway, since squid can not cache secure pages.
2.4 Proxy Authentication

Infact Microsoft ISA and squid (the open source product MT uses) allows you to do this.
It is still a secure connection even if you go through a proxy.
The only thing thats not possible is a transparent HTTPS proxy.

Yes but that's what he was trying to do no?

Anyway, nice to know about ISA and squid. Thanks for this.

No, not trying to do transparent proxy.

I literally only enable web proxy set up basic settings no firewall rules and so.
And then i set my browser manually to point to MT as proxy.
I do have a parent proxy and that's whats giving me the error.

It's literally saying that the request coming from client, (In this case MT), is invalid as it has no url....

Only https://443 as url!

Very weird, sure it's a bug.

But hey let's see ;>

The basic option is, https can't be cache by MT proxy so you will never goto https sites through proxy here. or the https can't be cached by proxy for it's security reason.

Thanks

I am also aware that it cannot be cached.

But the point is that it's a bug.

All major caching software supports this correctly including squid. (What MT Uses)

In a corporate environment if you have you're proxy setup say via 2003 server policies, this bug will now ensure NO-ONE in the organization can visit https sites.


So it's serious. It's not a matter of caching the content rather than forwarding you're request.

Hope, you understand what im trying to say.

ThankX.

thavinci said:

All major caching software supports this correctly including squid. (What MT Uses)

MT Ver 3 proxy is not squid it's their own(MT) invention.

I guarantee you it's squid.

Looks like someone doesn't read there license agreements!

Go read in you're LICENSE file that comes with ... say version 3.7 x86

In fact here it is.... http://www.thavinci.za.net/Downloads/LICENSE.TXT

They even refer you to what Linux packages they use and there respective links....

ie

squid GPL http://www.squid-cache.org/

Almost everything MT uses is Linux based programmes, just fancy well designed front end.

I guarantee you it's squid.

Looks like someone doesn't read there license agreements!

Go read in you're LICENSE file that comes with ... say version 3.7 x86

In fact here it is.... http://www.thavinci.za.net/Downloads/LICENSE.TXT

They even refer you to what Linux packages they use and there respective links....

ie

squid GPL http://www.squid-cache.org/

MIKROTIK ROUTEROS V2.0 SOFTWARE ROUTER SYSTEM

MikroTik RouterOS V2

omg, they do not have ROS v3 License!

lol @ Chupaka , i just think they haven't updated it fully ;p
So it should be free until they specify more correctly :p


Anyways jokes aside, for those interested in how this looks...




And it's very easy to simply change the (squid/3.0STABLE1) at the bottom to whatever you want, including Mikrotik Proxy ;>

And it's very easy to simply change the (squid/3.0STABLE1) at the bottom to whatever you want, including Mikrotik Proxy ;>

I don't think they invented new 'webproxy-test' package in v2 just to change that label =)

Of course they won't only change one thing....

proxy test package is most probably their migration to squid 3.0 versions from 2.6.


....
Later after a crap load of research and going through the MT File system...
....


Did try do more research and couldn't come up with any conclusive proof either way....
I did mount the MT file system and do a "strings" to compare binaries and to youre credit didn't find enough similarities.

Also found this comment on forum to you're credit....

IP Proxy works only as a proxy and does not cache, of course if you set it to use a parent-proxy, the parent will cache. This is mostly used when you need to enforce rules and filter a network.
Web proxy is squid with the limitations.
web proxy-test is a new caching system created by Mikrotik, according to them this is not based on squid and it's supposed to be the fastest with many more features.

So if indeed they did do their own software, im really REALLY surprised.
Would also explain the bug.

Found a temporary work around for the https issue.

We now have to rely on Proxy auto configuration scripts to set the SSL proxy in users browsers to the upstream proxy server the MT itself uses. (in turn bypassing MT) and ONLY using MT for simple http.

Still hasn't been fixed in v3.10.

Haven't fully tested it but looks like ftp isn't supported at all in Mikrotiks proxy!!!

I mean wtf guys!!!

thavinci, report the problem to support department [support@mikrotik.com], if you expect to see the problem fixed.
Yesterday, I was able to reproduce the same problem you are referring to, we will try to fix it.
v3 proxy is written proxy from the scratch by MikroTik, squid is not used there.

I guarantee you it's squid.
Looks like someone doesn't read there license agreements!
Go read in you're LICENSE file that comes with ... say version 3.7 x86
In fact here it is.... http://www.thavinci.za.net/Downloads/LICENSE.TXT

omg, they do not have ROS v3 License!

Faking license in such way won't help you. Read real license from a router, not from your download site.

http://demo2.mt.lv/help/license.html

Thanks for updated license link....
wan't to read through :>

And i didn't "Fake" it.....
Even in latest 3.10 "all_packages" zip file the license file within there is still the "MIKROTIK ROUTEROS V2.0 SOFTWARE ROUTER SYSTEM" license.
Thats where i uploaded my copy from..

So maby it's time to update the license file in that package ;>

Have sent request through to support today.

thavinci, we have fixed some issue with HTTPS, when parent proxy is being used.

Hi I had this problem and I fix it in this way :

1 My RT that link all my radios is in the same net of the proxy machine

On the RT I create a mangle rule for all tcp por 80 mark routing "http"

then on the ip routing I create a destination gateway to my proxy for all http mark

In the proxy I create the nat for tcp port 80 redirect to por of my webproxy.

It's working now !

Thats just HTTP , im rfering to HTTPS and suing it directly, no rules needed.

Yes, but you do not need to go to the users browsers setup.

You cannot transparently cache HTTPS only HTTP therefore it MUST be set on clients brower settings.

Ok after months of sending mails to support i still haven't had an answer on this bug and it STILL exists!

Changed only slightly in output message.

I still don't know why you are trying to cache HTTPS sites? It's not going to work.

I am NOT trying to cache HTTPS.

Please understand in a enterprise environment it is often practice to simply deny all internet traffic by disabling NAT and forcing users to use a proxy via say Active Directory Policy.

Now the result of this bug would be that Users will not be able to access the banking sites unless the Nating is enabled and the proxy bypassed for HTTPS.

This is NOT a desirable option in a large organization as you loose control.

If you look at ISA or squid this is possible and is a need infact....

Here are my findings;

HTTPS via the Mikrotik transparent proxy does NOT work (OK we knew that).

HTTPS via the Mikrotik web browser configured proxy BUT with a parent proxy configured in the web proxy does NOT work (dsl-cache.saix.net).

HTTPS via the Mikrotik web browser configured proxy BUT with NO parent proxy configured in the web proxy DOES work.

I logged into a few banking sites and the connections tab in the Mikrotik proxy clearly showed my PC hitting those sites.

I'm using 3.11 x86

Perhaps we should compare settings?

Ok there we hit the nail on the head! So-to-speak.


The option i have is configured proxy with parent proxy.

Reason being have a squid upstream proxy that we use for logging traffic analysis etc.... Just more powerful.


In fact i think FTP doesn't work via this configuration either...... Haven't Confirmed lately...



Thanks For Input.

This is still broken and i can confirm that FTP also doesn't work!


This has been a long outstanding one!

did you open a ticket and send a supout ?

Twice, once on own once with other issues i had at the time....

HTTPS via the Mikrotik transparent proxy does NOT work (OK we knew that).

My evaluation of 3.17 for use with hotspot actually shows, that HTTPS works for me, with transparent proxy and caching, but NO upstream proxy used until now. Will try that later.
However, I have problems using FTP, too. But that might be considered a "feuture" of hotspot-module

I just ran into same problem with proxying HTTPS to a parent proxy.

I'm on an enterprise network where all direct outgoing traffic is blocked and BlueCoat proxies are used for all HTTP, HTTPS and FTP traffic.

I'm now trying to solve a capacity problem by using Mikrotik RB1000 proxies to offload a WAN link from web traffic and route the traffic between the Mikrotik proxy and the parent bluecoat proxy over a seperate link.

HTTP and FTP seems to work fine, however HTTPS doesn't work. I get the same type of error that others have reported from my parent proxy.

I think I just found the problem. When a client browser requests for instance:

https://www.handelsbanken.se

The initial HTTPS request to the Mikrotik proxy the Proxy CONNECT string looks like this:

CONNECT www.handelsbanken.se:443 HTTP/1.1
Host: www.handelsbanken.se

However, when the Mikrotik sends out it's request to the parent proxy it looks like this:

CONNECT http://www.handelsbanken.se:443 HTTP/1.1
Host: www.handelsbanken.se

So the Mikrotik seems to prepend the CONNECT string with "http://" which I assume is not correct when it's a https page we have requested. Correct behaviour should be to simple pass the CONNECT string untouched to the parent proxy.

This is tested running RouterOS 3.20 on an RB1000.

PS: Like others I'm not trying to achieve any caching of HTTPS traffic, I only need it working for simple forwarding of traffic. Without that working the proxy is unfortunately worthless to me in this type of setups.

Just wanted to let you know that I've verified that the problem IS what I described above.

I just tried telneting directly to our parent proxy and tried to send CONNECT cmds with both syntaxes and using http:// in front of the URL the way the
Mikrotik does do fail.

Working:

mathias@mathias-laptop:~$ telnet www-proxy.xxxxx.se 8080
Trying 153.xx.xxx.xx...
Connected to www-proxy.xxxxx.se.
Escape character is '^]'.
CONNECT www.handelsbanken.se:443 HTTP/1.1
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
Proxy-Connection: keep-alive
Host: www.handelsbanken.se

HTTP/1.1 200 Connection established


NOT Working (the Mikrotik way):

mathias@mathias-laptop:~$ telnet www-proxy.xxxxx.se 8080
Trying 153.xx.xxx.xx...
Connected to www-proxy.xxxxx.se .
Escape character is '^]'.
CONNECT http://www.handelsbanken.se:443 HTTP/1.1
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
Proxy-Connection: keep-alive
Host: www.handelsbanken.se

HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close

I've mailed a bugreport regarding this to support and are hoping for a quick response.

In the meantime I found that proxying FTP accually didn't work with a parent proxy! The Mikrotik proxy happily accepts FTP requests but they are sent out directly, ignoring the defined parent proxy.

As a workaround for the HTTPS problem I used a proxy.pac file to have browsers send https:// requests directly to the parent-proxy and everything else to the Mikrotik proxy. It's not a solution I can put in production, but atleast it let's me continue with some realworld tesing without completely blocking HTTPS for the testusers.

BTW: How large CF memory would you recommend to use on a normal office with about 100 users and a 8/1 Mbps ADSL internet connection? If I had to choose between a 8GB 266X fast memory or a 16GB 133X (bigger but slower memory), what would go for?

Not having much luck with the support yet

The FTP problem is simply acknowledged as works-as-designed and not a bug. They claim that the parent-proxy feature is only supposed to forward HTTP to the parent, not other protocols, but obviously it IS trying to forward HTTPS as well, so I can't understand why it shouldn't forward FTP as well.

The HTTPS bug was first declined as "works-as-it-should-according-to-RFC2616" but I'm still trying to convince them it's not.

According to the following two documents it should work as I have described at least:

http://tools.ietf.org/html/rfc2817
http://tools.ietf.org/html/draft-luoton ... nneling-03

Starting to miss my open-source based firewalls where I could just go into the code and fix the damn bugs myself. An open-sourced community edition of RouterOS would have been great so people could contribute with patches for such trivial problems like this.

Much better answers from the support today

The FTP problem is at least accepted as a feature request, and the HTTPS problem will most likely be fixed in next release (3.21).

Thanks support!

HTTPS problem now fixed in this beta package of 3.21:

http://mikrotik.com/download/temp/netin ... 21-ppc.zip
http://mikrotik.com/download/temp/route ... c-3.21.npk

Thanks a lot Mikrotik!

Horray!!!!

Finally HTTPS Bug fixed in v3.22!!!! Now Just only waiting for the ftp side of things......

Actually not entirely fixed, i have noticed that there have been a few https sites that wont work through a mikrotik proxy but do through ISA and squid.


Will update with more detail when i can....

In connection with previous post.....

V3.23 SSL Proxy Issue:



Have not tested if this still exists in v3.24

FYI: Exact same bug exists in v3.24

Please, contact support (support@mikrotik.com) about HTTPS issues.

so how can i work around this



network--------x86----eth1---adslmodem1
----eth2----adslmodem
----eth3----mailserver
----eth4----x86proxy----------adslmodem


so i have a nat to redirect 80 trafic to proxy
but i want 445 trafic to go throught the same modem as proxy traffic

Old thread. Damn

i can see that so there must be a salution correct

or has it been fixed

can i redirect https through the proxy yes or no????

I never have to. You should never have to as well. It will only pass through or it will break. It does not cache encrypted objects so there would be no point.

RouterOS Proxy doesn't support transparent proxying of https

thanks i found a salution to my problem

thanks i found a salution to my problem

hi xezen
can u tell me what is your soulotion ?