Problems with PCC and Web Proxy

mixig · Thu Oct 27, 2011 3:46 pm

Hi,
i have 2 pppoe connections to my ISP, i configured load balancing over my two pppoe connection (PCC) and that is working great. after that i enabled web proxy on mikrotik (OS v5.4) and added new output chain (line 6 and 7) but the HTTP load balancing does not work. All HTTP traffic goes through proxy but uses only primary pppoe connection. But sometimes I get dfferent ip on the internet (www.whatismyip.com), below is my config:

[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; Mark new inbound connection wan1
chain=input action=mark-connection new-connection-mark=wan1 passthrough=yes connection-state=new in-interface=pppoe-out1

1 ;;; Mark new inbound connection wan2
chain=input action=mark-connection new-connection-mark=wan2 passthrough=yes connection-state=new in-interface=ether4

2 ;;; Mark established inbound connection wan1
chain=prerouting action=mark-connection new-connection-mark=wan1 passthrough=yes connection-state=established in-interface=pppoe-out1

3 ;;; Mark established inbound connection wan2
chain=prerouting action=mark-connection new-connection-mark=wan2 passthrough=yes connection-state=established in-interface=ether4

4 ;;; Mark related inbound connection wan1
chain=prerouting action=mark-connection new-connection-mark=wan1 passthrough=yes connection-state=related in-interface=pppoe-out1

5 ;;; Mark related inbound connection wan2
chain=prerouting action=mark-connection new-connection-mark=wan2 passthrough=yes connection-state=related in-interface=ether4

6 chain=output action=mark-connection new-connection-mark=wan1 passthrough=yes connection-state=new protocol=tcp dst-port=80 per-connection-classifier=both-addresses:2/0

7 chain=output action=mark-connection new-connection-mark=wan2 passthrough=yes connection-state=new protocol=tcp dst-port=80 per-connection-classifier=both-addresses:2/1

8 ;;; Mark new inbound route wan1
chain=output action=mark-routing new-routing-mark=static-wan1 passthrough=no connection-mark=wan1

9 ;;; Mark new inbound route wan2
chain=output action=mark-routing new-routing-mark=static-wan2 passthrough=no connection-mark=wan2

10 ;;; Mark traffic that isn't local with PCC mark rand (2 possibilities) - option 1
chain=prerouting action=mark-connection new-connection-mark=wan1_pcc_conn passthrough=yes connection-state=new dst-address-type=!local in-interface=ether3 per-connecti

11 ;;; Mark traffic that isn't local with PCC mark rand (2 possibilities) - option 2
chain=prerouting action=mark-connection new-connection-mark=wan2_pcc_conn passthrough=yes connection-state=new dst-address-type=!local in-interface=ether3 per-connecti

12 ;;; Mark established traffic that isn't local with PCC mark rand (2 possibilities) - option 1
chain=prerouting action=mark-connection new-connection-mark=wan1_pcc_conn passthrough=yes connection-state=established dst-address-type=!local in-interface=ether3 per-

13 ;;; Mark established traffic that isn't local with PCC mark rand (2 possibilities) - option 2
chain=prerouting action=mark-connection new-connection-mark=wan2_pcc_conn passthrough=yes connection-state=established dst-address-type=!local in-interface=ether3 per-

14 ;;; Mark related traffic that isn't local with PCC mark rand (2 possibilities) - option 1
chain=prerouting action=mark-connection new-connection-mark=wan1_pcc_conn passthrough=yes connection-state=related dst-address-type=!local in-interface=ether3 per-conn

15 ;;; Mark related traffic that isn't local with PCC mark rand (2 possibilities) - option 2
chain=prerouting action=mark-connection new-connection-mark=wan2_pcc_conn passthrough=yes connection-state=related dst-address-type=!local in-interface=ether3 per-conn

16 ;;; Mark routing for PCC mark - option 1
chain=prerouting action=mark-routing new-routing-mark=wan1 passthrough=yes connection-mark=wan1_pcc_conn

I am not a Mikrotik expert, I have read all the topics in the forums regarding PCC and proxy, but to no avail.
Any help would be apreciated

Feklar · Thu Oct 27, 2011 4:45 pm

The problem you are running into is this. Proxy takes all HTTP requests and places them into a different chain. Instead of being forward it is now in output and input. This is because the router itself is servicing the requests and making them out to the internet instead of the end user, so it is no longer forwarding the traffic.

Load balancing and Proxy is a bit tricky, and requires a few extra steps in order to do. Also I'm not sure if it would break anything long term or not since I haven't tried it past the lab environment. You also have to be VERY specific with what traffic you are marking, you generally never want to mess with the output chain and change it's routing. I think this is the method, it's been a while since I've set it up, but should get you on the right path.
1.) Mark in output tcp port 80 connections and use the PCC method. (See you already have this)
2.) Mark for routing in the same way. (See that you already have this)
3.) Use src-nat to change the headers of packets leaving the router on specific interfaces to the right IP address. Once again, be as specific as possible with this. This is because a routing decision has already been made, so it keeps the routes src IP it chose before, you need to NAT it to the correct one. (This appears to be what you are missing.)

A simpler solution is to use a dedicated proxy.

mixig · Fri Oct 28, 2011 9:38 am

I thnik I already done what you said:

here is my routing table:

[admin@MikroTik] /ip route> print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 X S ;;; Static Route - WAN1
0.0.0.0/0 pppoe-out1 1

1 X S ;;; Static Route - WAN2
0.0.0.0/0 ether4 2

2 X S ;;; WAN 1 - Distance 1
0.0.0.0/0 pppoe-out1 1

3 X S ;;; WAN 1 - Distance 2
0.0.0.0/0 ether4 2

4 X S ;;; WAN 2 - Distance 1
0.0.0.0/0 ether4 1

5 X S ;;; WAN 2 - Distance 2
0.0.0.0/0 pppoe-out1 2

6 A S ;;; Default Route - Distance 1
0.0.0.0/0 pppoe-out1 1

7 X S ;;; Default Route - Distance 2
0.0.0.0/0 ether4 2

and here is my NAT config:

3 X ;;; redirekt port 80 na web proxy port 5555
chain=dstnat action=redirect to-ports=5555 protocol=tcp dst-port=80

4 ;;; default configuration
chain=srcnat action=masquerade out-interface=pppoe-out1

5 chain=srcnat action=masquerade out-interface=ether4

Feklar · Fri Oct 28, 2011 4:13 pm

Nope, you need to make your NAT rule very specific. If you look at torch you will see it's trying to send out connections with the wrong IP address for that interface. I would throw in the connection mark you have just to make sure you're not NATing something you don't want to as well.

/ip firewall nat
add action=src-nat src-address=x.x.x.y protocol=tcp dst-port=80 out-interface=ether1 to-address=z.z.z.y connection-mark=wan1
add action=src-nat src-address=z.z.z.y protocol=tcp dst-port=80 out-interface=ether2 to-address=x.x.x.y connection-mark=wan2

It has already decided what route it will use since the routing decision comes before the output chain. Because of that it has already decided what IP address it will use when sending the traffic out, and then you are telling it something different. You need to change it's IP src-address to correct this. You want to be very specific because you are messing with output traffic, you could end up breaking something if you're not careful.

mixig · Fri Oct 28, 2011 5:01 pm

Can you please be more specific what are address x.x.x.y and z.z.z.y (my local network is 192.160.0.0/24), and my public IPs are dynamic, for pppoe1 and pppoe2, and there is missing chain in you config, I guess that chain is also src-nat??

Thanks

Feklar · Fri Oct 28, 2011 5:31 pm

x.x.x.y and z.z.z.y depends on your install. Lets assume that the IP address that is on ether1 is 1.1.1.2 and that the IP on ether2 is 2.2.2.3.

So the rules would look like this. I forgot the chain, but yes it is src-nat

/ip firewall nat
add action=src-nat src-address=2.2.2.3 protocol=tcp dst-port=80 out-interface=ether1 to-address=1.1.1.2 connection-mark=wan1 chain=src-nat
add action=src-nat src-address=1.1.1.2 protocol=tcp dst-port=80 out-interface=ether2 to-address=2.2.2.3 connection-mark=wan2 chain=src-nat

Since you have a dynamic IP it is a bit more tricky. You can script it to update the rules for you on an IP change. Look at the wiki for examples of DDNS scripts and modify them to fit your needs in that case.

gtpro · Fri Oct 28, 2011 9:59 pm

Hi, try this and adjust it to your needs.

/ip firewall mangle
add action=mark-connection chain=input disabled=no in-interface=pppoe-out1 new-connection-mark=pppoe-out1_conn passthrough=yes
add action=mark-connection chain=input disabled=no in-interface=pppoe-out2 new-connection-mark=pppoe-out2_conn passthrough=yes
add action=mark-connection chain=input disabled=no in-interface=pppoe-out3 new-connection-mark=pppoe-out3_conn passthrough=yes
add action=mark-connection chain=input disabled=no in-interface=pppoe-out4 new-connection-mark=pppoe-out4_conn passthrough=yes

add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=pppoe-out1_conn passthrough=yes per-connection-classifier=both-addresses:4/0 protocol=tcp
add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=pppoe-out2_conn passthrough=yes per-connection-classifier=both-addresses:4/1 protocol=tcp
add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=pppoe-out3_conn passthrough=yes per-connection-classifier=both-addresses:4/2 protocol=tcp
add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=pppoe-out4_conn passthrough=yes per-connection-classifier=both-addresses:4/3 protocol=tcp

add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=Local new-connection-mark=pppoe-out1_conn passthrough=yes per-connection-classifier=both-addresses:4/0 protocol=tcp
add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=Local new-connection-mark=pppoe-out2_conn passthrough=yes per-connection-classifier=both-addresses:4/1 protocol=tcp
add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=Local new-connection-mark=pppoe-out3_conn passthrough=yes per-connection-classifier=both-addresses:4/2 protocol=tcp
add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=Local new-connection-mark=pppoe-out4_conn passthrough=yes per-connection-classifier=both-addresses:4/3 protocol=tcp

add action=mark-routing chain=prerouting connection-mark=pppoe-out1_conn disabled=no in-interface=Local new-routing-mark=to_pppoe-out1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe-out2_conn disabled=no in-interface=Local new-routing-mark=to_pppoe-out2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe-out3_conn disabled=no in-interface=Local new-routing-mark=to_pppoe-out3 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe-out4_conn disabled=no in-interface=Local new-routing-mark=to_pppoe-out4 passthrough=yes

add action=mark-routing chain=output connection-mark=pppoe-out1_conn disabled=no new-routing-mark=to_pppoe-out1 passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe-out2_conn disabled=no new-routing-mark=to_pppoe-out2 passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe-out3_conn disabled=no new-routing-mark=to_pppoe-out3 passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe-out4_conn disabled=no new-routing-mark=to_pppoe-out4 passthrough=yes

/ip firewall nat
add action=masquerade chain=srcnat disabled=no out-interface=pppoe-out1
add action=masquerade chain=srcnat disabled=no out-interface=pppoe-out2
add action=masquerade chain=srcnat disabled=no out-interface=pppoe-out3
add action=masquerade chain=srcnat disabled=no out-interface=pppoe-out4

add action=redirect chain=dstnat disabled=no dst-port=80 in-interface=Local protocol=tcp to-ports=3123

mixig · Wed Nov 09, 2011 6:01 pm

Hi gtpro, I took your example and adjusted for my needs but it`s still doesn`t work.

here is the config:

/ip route
add check-gateway=arp comment="WAN 1 - Distance 1" disabled=yes distance=1 dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_pppoe-out1 scope=30 target-scope=10
add check-gateway=arp comment="WAN 1 - Distance 2" disabled=yes distance=2 dst-address=0.0.0.0/0 gateway=ether4 routing-mark=to_pppoe-out1 scope=30 target-scope=10

add check-gateway=arp comment="WAN 2 - Distance 1" disabled=yes distance=1 dst-address=0.0.0.0/0 gateway=ether4 routing-mark=to_pppoe-out2 scope=30 target-scope=10
add check-gateway=arp comment="WAN 2 - Distance 2" disabled=yes distance=2 dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_pppoe-out2 scope=30 target-scope=10

add check-gateway=arp comment="Default Route - Distance 1" disabled=yes distance=1 dst-address=0.0.0.0/0 gateway=pppoe-out1 scope=30 target-scope=10
add check-gateway=arp comment="Default Route - Distance 2" disabled=yes distance=2 dst-address=0.0.0.0/0 gateway=ether4 scope=30 target-scope=10


/ip firewall nat
add action=redirect chain=dstnat comment="redirekt port 80 na web proxy port 8080" disabled=no dst-port=80 protocol=tcp to-ports=8080

add action=masquerade chain=srcnat disabled=no out-interface=pppoe-out1
add action=masquerade chain=srcnat disabled=no out-interface=ether4


/ip firewall mangle
add action=mark-connection chain=input disabled=no in-interface=pppoe-out1 new-connection-mark=pppoe-out1_conn passthrough=yes
add action=mark-connection chain=input disabled=no in-interface=ether4 new-connection-mark=pppoe-out2_conn passthrough=yes

add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=pppoe-out1_conn passthrough=yes per-connection-classifier=both-addresses:2/0 protocol=tcp
add action=mark-connection chain=output disabled=no dst-port=80 new-connection-mark=pppoe-out2_conn passthrough=yes per-connection-classifier=both-addresses:2/1 protocol=tcp

add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=ether3 new-connection-mark=pppoe-out1_conn passthrough=yes per-connection-classifier=both-addresses:2/0 protocol=tcp
add action=mark-connection chain=prerouting disabled=no dst-address-type=!local dst-port=!80 in-interface=ether3 new-connection-mark=pppoe-out2_conn passthrough=yes per-connection-classifier=both-addresses:2/1 protocol=tcp

add action=mark-routing chain=prerouting connection-mark=pppoe-out1_conn disabled=no in-interface=ether3 new-routing-mark=to_pppoe-out1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=pppoe-out2_conn disabled=no in-interface=ether3 new-routing-mark=to_pppoe-out2 passthrough=yes

add action=mark-routing chain=output connection-mark=pppoe-out1_conn disabled=no new-routing-mark=to_pppoe-out1 passthrough=yes
add action=mark-routing chain=output connection-mark=pppoe-out2_conn disabled=no new-routing-mark=to_pppoe-out2 passthrough=yes

with this config my web pages are opening extremely slowly or they not open at all (timeout), and there is no load balancing, traffic is going through ether4 which is also pppoe connection just as first interface pppoe-out1

Any other solution??
Thanks

gtpro · Wed Nov 09, 2011 7:20 pm

HI.

Why do you your routes disabled? And you should add in-interface in

/ip firewall nat
add action=redirect chain=dstnat comment="redirekt port 80 na web proxy port 8080" disabled=no dst-port=80 protocol=tcp to-ports=8080

like

/ip firewall nat
add action=redirect chain=dstnat comment="redirekt port 80 na web proxy port 8080" disabled=no dst-port=80 in-interface=Local protocol=tcp to-ports=8080

mixig · Wed Nov 09, 2011 8:33 pm

i disabled my routes after I tested, I also tried with in-interface but there is no differences

gtpro · Thu Nov 10, 2011 12:38 am

I used to have a configuration working fine, you must be doing something wrong, I am no expert in mikrotik.

dotnet · Tue Mar 19, 2013 11:48 am

I hope it will be work for you.
Bcoz i have solve some problem with this nat for web proxy.
Please don't copy-pase.
Just follow the rule:

/ip firewall nat
add action=redirect chain=dstnat comment="redirekt port 80 na web proxy port 8080" disabled=no dst-port=80 in-interface=!ether4 protocol=tcp to-ports=8080

Best regards

Problems with PCC and Web Proxy

Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Re: Problems with PCC and Web Proxy

Who is online