How turn off “universal proxy” ?

RouterOS General
1

Good day everybody,

Mikrotik (951G-2HnD, Router OS 6.7) is configured as router between a couple of internal networks and as accounting system (User Manager). NAT on external interface is off. Device is working fine.
Mikrotik external interface is connected to firewall. Firewall is maintaining bandwidth restriction and site access rules depend of source IP from internal networks. Firewall has NAT on external interface. System is working fine also.

I run proxy on Firewall computer and want to configure some of clients from internal networks use proxy directly.
Here trouble appeared!!
When Mikrotik/Hotspot is recognizing proxy traffic, it redirect it to internal Mikrotik/WebProxy and after to external interface. But in this case information about source IP is lost and firewall is treated traffic as from Mikrotik IP, not from client IP.

Result of searching in different forums gave result – It is Mikrotik “Universal Proxy” feature. (http://wiki.mikrotik.com/wiki/Manual:Customizing_Hotspot).
But I could not find how to turn off this feature in my version of Router OS. As per manuals options was present in v2.7, and after disappeared. Mikrotik Web Proxy is not enabled, but is showing status: running.

Question – how is turn off universal proxy or override it in purpose to allow internal network clients directly work with proxy located on external interface? (without hiding client IP by NAT, transparent proxies between or forwarding connection to external proxy)

PS: I mean “work directly” is LAN computer has information about proxy IP and Firewall/Proxy on external interface has information about client IP.

Hidden NAT on HotSpot !
2

You can redirect (by dst-nat) traffic to proxy(firewall) computer, but anyway client IP will be “lost” from external point of view.
If it is not problem you can disable hotspot/proxy in mikrotik and use just your proxy computer. You can still see internal IPs in your proxy computer log.

3 
/ip hotspot user profile
set X transparent-proxy=no

Change X to the line number of the user profile you are using.

Also check “/ip firewall nat” and insure you are not masquerading the hotspot ips. If you have a rule like this, disable it.

chain=srcnat action=masquerade src-address=192.168.0.0/24

The src-address will be the network for the hotspot.

edit: Insure you have all src-nat and masquerade rules disabled in the Mikrotik router, and you route the hotspot ips back to the Mikrotik from the firewall.

4

Mikrotik has the best hotspot (User Manager) solution if compare with other systems which I have seen/work ( at least 8 ). It contains easy/powerful/fast interface for creating accounts, making reports,… and can simultaneously work with different rules on different interfaces.
I am planning to use Mikrotik only as router/hotspot (for 3 local networks). I need have source clients IP on external interface in purpose to control and manage speed by other systems.
We have slow and float Internet connection speed (64-400kbits) therefore I prefer squid/pool/filters technology for some of clients then IP/queues (squid pool has less packet drop which already passed via our slow connection).
And I could not run all clients via proxy (to some clients I do not have access and some of our http traffic do not work via proxy at all)
Also as next step I want to move bandwidth control to other side of slow external connection. And I have many other nuances.
Therefore using NAT or do not use hotspot as you advice is equal for me to do not use Mikrotik.
But I like this hotspot solution and I need only turn off additional feature of Mikrotik witch I did not activated/ordered - “universal proxy”.

I do not have hotspot user profile. I am using default profile and there are transparent proxy is off.
And I do not src-nat for external interface. And I have back routes in firewall.
If any of you mentioned settings were active – nobody can use internet. Traffic from external Mikrotik IP is blocked by next firewall. :slight_smile:.
Problem is appeared when some of client is trying to use proxy.

New issue with “universal proxy”
Browser (Chrome) on Apple computers is also trying to use proxy first time after start. (Browser does not have any proxy in settings!!!).
In result client is receiving deny from firewall (“universal-proxy”- connection is going from Mikrotik IP!!!-blocked) and after page reload everything is working (because Apple finally follow settings and do not trying to use proxy anymore - connection from client IP - allowed).

5

Hi everybody,
Can anybody advice correct place in Mikrotik firewall where I can add rule in purpose to override detection and forwarding proxy traffic in Mikrotik “Web Proxy” by “Universal Proxy” technique?
But I still need hotspot authentication for proxy traffic.

6

There is no “universal proxy”. There is a transparent proxy used by the hotspot. That is in “/ip hotspot user profile” and “/ip hotspot profile”.

There is a transparent proxy you can enable and enter a firewall nat rule to intercept port 80 requests and redirect to that proxy.
http://wiki.mikrotik.com/wiki/Manual:IP/Proxy#Transparent_proxy_configuration_example

There is a “universal nat” in the hotspot.

7

OK. Forget about terms.

At this moment I have 4 active interfaces

/ip address
add address=192.168.0.4/24 interface=ether1-Optional-Net network=192.168.0.0
add address=172.26.101.4/24 interface=ether4-DMZ network=172.26.101.0
add address=192.168.1.1/24 interface=ether5-Office-Net network=192.168.1.0
add address=192.168.14.1/24 interface=ether3-Public-Net network=192.168.14.0

Interface ether4-DMZ is default gate.

/ip route
add distance=1 gateway=172.26.101.2

NAT for default route is OFF.
But it is ON for ether1-Optional-Net. (Some resources from this net need to be available from local network, without changing configuration of Optional net)

/ip firewall nat
add action=passthrough chain=unused-hs-chain comment="place hotspot rules here" disabled=yes to-addresses=0.0.0.0
add action=masquerade chain=srcnat dst-address=192.168.0.0/24 out-interface=ether1-Optional-Net src-address=192.168.14.
add action=masquerade chain=srcnat dst-address=192.168.0.0/24 out-interface=ether1-Optional-Net src-address=192.168.1.0

And Web Proxy is OFF also.

/ip proxy
set parent-proxy=0.0.0.0

Local network interfaces (ether5-Office-Net and ether3-Public-Net network) have hotspot with UserMan identifications.
Universal NAT is disabled for ether5-Office-Net.

/ip hotspot profile
add dns-name=public.inet hotspot-address=192.168.14.1 html-directory=hotspot.inet login-by=http-chap,https,http-pap name=public.hs.profile use-radius=yes
add dns-name=office.inet hotspot-address=192.168.1.1 html-directory=hotspot.inet login-by=http-chap,https,http-pap name=office.hs.profile use-radius=yes
/ip hotspot
add address-pool=Public-Net-pool disabled=no interface=bridge-Public-Net name=Public-Hotspot profile=public.hs.profile
add disabled=no interface=ether5-Office-Net name=Office-Hotspot profile=office.hs.profile
/ip hotspot user profile
set [ find default=yes ] idle-timeout=10m insert-queue-before=hotspot-users-rules mac-cookie-timeout=3d session-timeout=3h20m

I think configuration is clear. Now let’s make tests.
I will try to open site http://forum.mikrotik.com (159.148.147.201) from computer which connected to ether5-Office-Net and will monitor connections from Mikrotik on my firewall interface (172.26.101.2).

tcpdump -i le0 'tcp[tcpflags] & (tcp-syn) != 0 and host 159.148.147.201'

Test 1. No any proxy settings on client computer.
Result: Site was opened without problem and log is showing no any NAT between client and firewall.
All traffic was originated from original source IP (192.168.1.61)

13:34:46.559538 IP 192.168.1.61.4493 > tuncis.mt.lv.http: Flags [S], seq 1194333346, win 65535, options [mss 1460,nop,wscale 0,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
13:34:47.671713 IP 192.168.1.61.4495 > tuncis.mt.lv.http: Flags [S], seq 4103440868, win 65535, options [mss 1460,nop,wscale 0,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
13:34:48.972472 IP tuncis.mt.lv.http > 192.168.1.61.4493: Flags [S.], seq 3292920672, ack 1194333347, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 715220277 ecr 0], length 0
13:34:49.463431 IP tuncis.mt.lv.http > 192.168.1.61.4495: Flags [S.], seq 348353262, ack 4103440869, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 3446909092 ecr 0], length 0
….

Test 2. Proxy is activated in browser on client computer (172.26.101.2:3128)
Result: After passing hotspot login page – I have error message

Request denied by Proxy Server: 403 Forbidden
Reason: Access to site is blocked. Client address: 172.26.101.4
Client group: default
Target group: 82059
URL: http://forum.mikrotik.com/viewtopic.php?f=2

(transparent proxy is activated on firewall site and all connection from Mikrotik IP is prohibited)
And tcpdump is confirming than connection is now going from Mikrotik IP (router=172.26.101.4, not from client IP):

13:48:02.834014 IP router.59200 > tuncis.mt.lv.http: Flags [S], seq 2390002386, win 14600, options [mss 1460,sackOK,TS val 4294953119 ecr 0,nop,wscale 4], length 0
13:48:02.834115 IP tuncis.mt.lv.http > router.59200: Flags [S.], seq 1329855587, ack 2390002387, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 1343130535 ecr 4294953119], length 0

Something is now making masquerading or proxying traffic inside of Mikrotik.

Test 3. Proxy is activated in client browser. And I deny forum IP inside of Mikrotik proxy and Mikrotik Web proxy is still NOT enabled.

/ip proxy
set parent-proxy=0.0.0.0
/ip proxy access
add action=deny dst-address=159.148.147.201

Result: Client browser show message from Mikrotik:

ERROR: Forbidden
While trying to retrieve the URL http://forum.mikrotik.com/t/how-turn-off-universal-proxy/74429/1
    Access Denied
Your cache administrator is webmaster.
Generated Mon, 03 Mar 2014 14:05:11 GMT by 192.168.1.1 (Mikrotik HttpProxy)

And no any traffic tcpdump logged.

Test 4. I turn off proxy in client browser, but Mikrotik proxy settings the same as in test 3.
Result: Site was opened and tcpdump show correct traffic (no any transparent proxy in Mikrotik)

15:18:07.882427 IP 192.168.1.61.4750 > tuncis.mt.lv.http: Flags [S], seq 809050715, win 65535, options [mss 1460,nop,wscale 0,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
15:18:07.882544 IP tuncis.mt.lv.http > 192.168.1.61.4750: Flags [S.], seq 4037620847, ack 809050716, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2164858298 ecr 0], length 0
15:18:16.735943 IP 192.168.1.61.4752 > tuncis.mt.lv.http: Flags [S], seq 1298546553, win 65535, options [mss 1460,nop,wscale 0,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
15:18:16.739448 IP 192.168.1.61.4755 > tuncis.mt.lv.http: Flags [S], seq 977636447, win 65535, options [mss 1460,nop,wscale 0,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0

Summary:
For some reason Miktotik is forwarding HTTP traffic via internal Web Proxy when client is trying to use outside proxy server.

/ip proxy> print
                 enabled: no  (!!!!!!!)
             src-address: ::
                    port: 8080
            parent-proxy: 0.0.0.0
       parent-proxy-port: 0
     cache-administrator: webmaster
          max-cache-size: unlimited
   max-cache-object-size: 2048KiB
           cache-on-disk: no
  max-client-connections: 600
  max-server-connections: 600
          max-fresh-time: 3d
   serialize-connections: no
       always-from-cache: no
          cache-hit-dscp: 4
             cache-drive: system




/ip proxy> monitor
                 status: stopped
                 uptime: 2h22m42s
               requests: 7412     (!!!!!!!)
                   hits: 0
             cache-used: 0KiB
         total-ram-used: 128KiB
  received-from-servers: 768KiB
        sent-to-clients: 727KiB
   hits-sent-to-clients: 0KiB

As per Mikrotik manual (http://wiki.mikrotik.com/wiki/Manual:Customizing_Hotspot) it is “Universal Proxy technique” (I first time read this term from above link).

So my question – somebody know how to turn off or block above described behavior?

8

If you are not logged in, your http requests will be forwarded to the proxy even if it is disabled.
If you have the transparent proxy disabled in “/ip hotspot user profile”, then the proxy is not used when you are logged in..

I’m using ROS v6.9, and that is how I see it here. Using “/ip proxy monitor”, I show no requests when I’m logged in. I do show requests if I am not logged in, even if the proxy is disabled.

9

Proxy monitor statistic I showed in my previous message only as reference. For me, it does not matter where packets are going when user not logged in.
But as per above posted my test #2 in case if client computer has proxy settings - his traffic is going via Mikrotik proxy server after login procedure. (Try to login with proxy in browser)
As result:

    • I can see message in browser.
  • I can see wrong traffic source in firewall
  • Test 3 is proofing that traffic passing proxy.

And this behavior exactly described in current Mikrotik manual (link was posted above):

“This feature is called "Universal Proxy". If it is detected that a client is using some proxy server, the system will automatically mark that packets with the http hotspot mark to work around the unknown proxy problem, as we will see later on. Note that the port used (64874) is the same as for HTTP requests in the rule #9 (so both HTTP and HTTP proxy requests are processed by the same code).”

And in old manual I found mention about turning off (http://www.mikrotik.com/documentation/manual_2.7/IP/Hotspot.html)

Submenu level : /ip hotspot
Property Description
….
universal-proxy (yes | no; default: no) - whether to intercept the requests to HTTP proxy servers
….

But I could not find now.
So question is still active – how to turn off it or override it in firewall.

10

Nothing new?

11

I found the “solution” here:
http://gregsowell.com/?p=2744

It says:

By default hotspot will proxy all traffic, even that of authenticated users. This will slow authenticated user traffic as well as make the CPU work harder on your router. To bypass this behavior you can use the following NAT rule:

/ip nat firewall
add chain=pre-hotspot dst-address=!local hotspot=auth action=accept

>
> “local” should be set to your local subnet. Be sure to drag this rule to the top.