Traffic/bandwidth optimization and transparent proxy

Hi guys!

Is it possible to do traffic/bandwidth optimization and transparent proxy simultaneously?

My config is:

/ip firewall mangle>pr  
8 X ;;; Route_to_ABCom_backup
     chain=prerouting action=accept src-address-list=winbox_active in-interface=Local 

9   ;;; All Download All
     chain=prerouting action=mark-packet new-packet-mark=all_Albtelekom2 passthrough=yes in-interface=!Local 

10   ;;; Video list prio6
     chain=prerouting action=mark-packet new-packet-mark=prio6_Albtelekom2 passthrough=no src-address-list=Youtube 
     in-interface=Albtelekom2 

11   ;;; Video L7 prio6
     chain=prerouting action=mark-packet new-packet-mark=prio6_Albtelekom2 passthrough=no layer7-protocol=http-video 
     in-interface=Albtelekom2 

12   ;;; Download 5M-10M
     chain=prerouting action=mark-packet new-packet-mark=prio7_Albtelekom2 passthrough=no protocol=tcp 
     in-interface=Albtelekom2 connection-bytes=5000000-10000000 

13   ;;; Download mbi 10M
     chain=prerouting action=mark-packet new-packet-mark=prio8_Albtelekom2 passthrough=no protocol=tcp 
     in-interface=Albtelekom2 connection-bytes=10000000-0 

14   ;;; All other Download Albtelekom2-end
     chain=prerouting action=mark-packet new-packet-mark=prio2_Albtelekom2 passthrough=no in-interface=Albtelekom2 


/ip firewall nat>pr
 X ;;; Transparent Web Cache
     chain=dstnat action=redirect to-ports=800 protocol=tcp in-interface=Local connection-mark=http_conn

but when I activate rule on mangle below rule does not work properly.
Pls, I need any idea to do that.

BR

a) You are redirecting to port 800. Is that a typo?
b) Packets can only have one packet mark. Are you overwriting the http_conn mark? Your mangle rules don’t ever show that mark being applied.
c) Once you proxy chains change, and there are two connections instead of just one. The second connection from the router proxy process does not inherit packet marks.
d) Be careful not to set routing marks on that traffic as it interferes with the redirect.

Yes, I am marking packets on upper rules.

ip firewall mangle print>
3 X ;;; Mark 80 TCP Connection for proxy
     chain=prerouting action=mark-connection new-connection-mark=http_conn passthrough=no protocol=tcp dst-port=80

But does exist any way to do traffic/bandwidth optimization and transparent proxy?

The two as such are completely independent of one another, so generally speaking: yes. Even when you add a proxy there are still IP packets, so you can manipulate them.
You have an error in your configuration. If you need help troubleshooting it you’ll have to post the entire ruleset, and you will have to explain what you mean when you say “traffic/bandwidth optimization” and what you are trying to achieve.

I am trying to mangle with prerouting packets and than to give priority to queue tree.
Also must mangle prerouting tcp port 80 connection for than redirect that to port 800 (port of proxy).

But in this way does not work. Does exist any other way to do so?

Config:

ip firewall mangle print>
;;; Download All
     chain=prerouting action=mark-packet new-packet-mark=all_dw passthrough=yes in-interface=!Local

ip queue tree pr>
name="Download_All_New" parent=global-in packet-mark=all_dw limit-at=0 priority=1 max-limit=3M burst-limit=0 
     burst-threshold=0 burst-time=0s 

name="Prio2" parent=Download_All_New packet-mark=prio2_ABCom limit-at=2M queue=default priority=2 max-limit=3M 
     burst-limit=0 burst-threshold=0 burst-time=0s

Sorry, it was my mistake.

Rules that inhibit each-other are:
ip firewall mangle print>
1-

   ;;; Route_to_ABCom
     chain=prerouting action=mark-routing new-routing-mark=route_to_ABCom 
     passthrough=no src-address-list=Route_to_ABCom 

   ;;; Route_to_Albtelekom2
     chain=prerouting action=mark-routing 
     new-routing-mark=route_to_Albtelekom2 passthrough=no 
     src-address-list=Route to second Gateway

2-

;;; Transparent proxy
     chain=prerouting action=mark-connection new-connection-mark=http_conn 
     passthrough=no protocol=tcp dst-port=80

So, I have separated LAN on 2 pieces, one piece route to one gateway and other piece to other 2nd gateway. But I want to
do transparent web proxy also.
Pls, I need any idea on how may will be it possible.

Thank you again for your help!

In prerouting add rules before the routing marks that match the same traffic as the rule that will redirect in the proxy and have an action of ‘accept’. Now that traffic won’t get a routing mark and goes to the proxy as it is exempt
If you need to add routing marks to the traffic from the proxy to the Internet you can add them in the output mangle chain instead of prerouting. If you need to match reliably for traffic coming from the proxy only (and not other router processes) you can create a mock loopback interface by creating a bridge and adding to ports to it, and assigning an IP address to that loopback interface (can be a /32). You can then use that IP as the source address in the proxy configuration, and match in the output mangle chain based on source address. You have to make sure your WAN source NAT rules include that loopback IP, of course.

But I want a solution that ‘Group IP A’ of LAN to connect with ISP1 and ‘Group IP B’ of LAN to connect with ISP2 via Mikrotik Router with buildin Web Proxy (traffic must not bypassing proxy).

Is it possible to separate traffic based on source IP address after proxy?
Mikrotik With Web Proxy.jpg

To the best of my knowledge that is impossible.

The proxy builds a completely new connection out to the WAN, that connection has absolutely nothing to do - from a routing point of view - with the initial client request. There are no provisions in the built in web proxy to apply any marking to the WAN connection it builds based on the original client request, so in the output mangle queue you are unable to determine whether the original client request came from group A or group B.

So, there is no way to build 2 separated LAN IP groups connected to 2 different ISPs with Web proxy?

Why that works with PCC http://wiki.mikrotik.com/wiki/Manual:PCC ?
Even on this config connection is classified per src-address.

I tried that to put

src-address-list="grpup ip A"

and

src-address-list="grpup ip A"

instead of

per-connection-classifier=both-addresses:2/0

and

per-connection-classifier=both-addresses:2/1

, but no luck.

What if I put group IP A and group IP B to separated physical ports/ip ranges?
What if I build proxy on external server (squid one)?

Pls, I need any way to do that.

Thank you again for helping me.

BR

"Why that works with PCC http://wiki.mikrotik.com/wiki/Manual:PCC ?"

“that” i mean proxy

Because proxies take the ONE original connection and make it TWO: client to server becomes client to proxy and proxy to server. If you have no way to determine in the mangle rules what the client IP was in the unrelated client to proxy connection (and I know of no way to do that), you cannot make routing decisions based on that information when mangle is looking at the proxy to server connection.

Maybe squid has something built in that lets you do that, I am not a squid expert.

Are enough this 2 rules below for doing config like on attached picture (clients and Squid proxy are on the same subnet and squid will not serve as parent proxy for mikrotik web proxy):

ip firewall nat>print
2 X ;;; To Squid
     chain=dstnat action=dst-nat to-addresses=192.168.12.66 to-ports=3128 protocol=tcp src-address=192.168.12.0/24 dst-port=80 

 3   ;;; To Squid
     chain=dstnat action=accept protocol=tcp src-address=192.168.12.66 dst-address=!192.168.12.66 dst-port=80

Or even this line must be entered:

ip firewall mangle add chain=prerouting action=mark-connection new-connection-mark=http_conn passthrough=no protocol=tcp src-address=!192.168.12.66 dst-port=80

Proxy with Squid.jpg