Need help with PCC, per-traffic load balanc. and web server

Hi!
First of all, sorry about my English, I promess I will try to make this post easier to write and read :slight_smile:

I was following these guides

http://wiki.mikrotik.com/wiki/Manual:PCC
http://wiki.mikrotik.com/wiki/Per-Traffic_Load_Balancing

and I have a little problem with one of my WAN connections, I currently have two WAN: one ordinary ADSL and one SDSL with static IP.
Recently, when I let the PCC work normally, internet becomes unusable from the LAN, so I created a mangle rule to mark all TCP 80 traffic and taking it to the ADSL gateway. The problem is, I have also a web server and with this mangle it gives me timed out connections when I try to get it using the static IP.

I don’t know if the SDSL issue comes from the router configuration (please see below) or the connection itself is lossing packets (pinging 8.8.8.8 gives me about 8% of packet loss).

I will post the configuration, so you can see in a more plain way what I mean and you can give me some advice something about that.


Addresses:

 ip address print
Flags: X - disabled, I - invalid, D - dynamic 
 #   ADDRESS            NETWORK         INTERFACE                              
 0   ;;; default configuration
     10.81.1.1/24       10.81.1.0       ether2-master-local                    
 1   ;;; SDSL WAN
     190.228.137.179/29 190.228.137.176 ether1-gateway                         
 2   [a pptp connection]                                  
 3   ;;; WAN 5 - ADSL
     192.168.1.254/24   192.168.1.0     ether5 - WAN - ADSL 7M 313             
 [some L2TP connections]

NAT

ip firewall nat print
Flags: X - disabled, I - invalid, D - dynamic 
 0   ;;; masquerade wan 1 (SDSL)
     chain=srcnat action=masquerade to-addresses=0.0.0.0 
     out-interface=ether1-gateway 

 1   ;;; masquerade wan 5 (ADSL)
     chain=srcnat action=masquerade to-addresses=0.0.0.0 
     out-interface=ether5 - WAN - ADSL 7M 313 

 2   ;;; HTTP to ProLiant (10.81.1.15)
     chain=dstnat action=dst-nat to-addresses=10.81.1.15 to-ports=80 
     protocol=tcp dst-address=190.228.137.179 dst-port=80 

 3   ;;; FTP to ProLiant (10.81.1.15)
     chain=dstnat action=dst-nat to-addresses=10.81.1.10 to-ports=22 
     protocol=tcp dst-address=190.228.137.179 dst-port=22 

 4   ;;; uTorrent@Killerpc
     chain=dstnat action=dst-nat to-addresses=10.81.1.129 to-ports=47228 
     protocol=tcp dst-address=190.228.137.179 dst-port=47228 

 5   ;;; RDP@Killerpc
     chain=dstnat action=dst-nat to-addresses=10.81.1.129 to-ports=3389 
     protocol=tcp dst-address=190.228.137.179 dst-port=3389 

 6   ;;; uTorrent@Killerpc
     chain=dstnat action=dst-nat to-addresses=10.81.1.129 to-ports=47228 
     protocol=udp dst-address=190.228.137.179 dst-port=47228

Address list:

ip firewall address print
Flags: X - disabled, D - dynamic 
 #   LIST                                          ADDRESS                        
 0   local-networks                                10.81.1.0/24                   
 1   local-networks                                10.81.3.0/24                   
 2   local-networks                                10.81.4.0/24                   
 3   local-networks                                10.81.5.0/24                   
 4   local-networks                                10.81.6.0/24                   
 5   local-networks                                10.81.7.0/24                   
 6   local-networks                                10.81.8.0/24                   
 7   local-networks                                192.168.1.0/24                 
 8   local-networks                                190.228.137.176/29             
 9   local-networks                                192.168.88.0/24                
10   webserver                                     10.81.1.15

from 0 to 6 are layer 2 tunnels
7 is the ADSL WAN
8 is the SDSL WAN
9 is another layer 2 tunnel
10 is the webserver

Mangle:

ip firewall mangle print
Flags: X - disabled, I - invalid, D - dynamic 
 0   chain=prerouting action=accept dst-address=190.228.137.176/29 
     in-interface=ether2-master-local 

 1   chain=prerouting action=accept dst-address=192.168.1.0/24 
     in-interface=ether2-master-local 

 2   ;;; Allow local networks (ping and sharing through L2TP)
     chain=prerouting action=accept src-address-list=local-networks 
     dst-address-list=local-networks 

 3   ;;; Bypass webserver
     chain=prerouting action=mark-routing new-routing-mark=Webserver 
     passthrough=no src-address-list=webserver 

 4   ;;; HTTPS and shoutcast Traffic Routing
     chain=prerouting action=mark-routing 
     new-routing-mark=HTTPS traffic routing passthrough=no protocol=tcp 
     dst-port=443,8004,8002 

 5   ;;; HTTP Traffic Routing
     chain=prerouting action=mark-routing new-routing-mark=HTTP traffic routing 
     passthrough=no protocol=tcp routing-mark=!Webserver dst-port=80 

 6   ;;; Mark connection WAN 7M
     chain=prerouting action=mark-connection new-connection-mark=wan_7M 
     passthrough=no connection-state=new 
     in-interface=ether5 - WAN - ADSL 7M 313 connection-mark=no-mark 

 7   ;;; Mark connection WAN SDSL
     chain=prerouting action=mark-connection new-connection-mark=wan_SDSL 
     passthrough=no connection-state=new in-interface=ether1-gateway 
     connection-mark=no-mark 

 8   ;;; Mark Connection WAN ADSL - 0
     chain=prerouting action=mark-connection new-connection-mark=wan_7M 
     passthrough=no connection-state=new dst-address-type=!local 
     in-interface=ether2-master-local connection-mark=no-mark 
     per-connection-classifier=both-addresses-and-ports:2/0 

 9   ;;; Mark Connection WAN SDSL - 1
     chain=prerouting action=mark-connection new-connection-mark=wan_SDSL 
     passthrough=no connection-state=new dst-address-type=!local 
     in-interface=ether2-master-local connection-mark=no-mark 
     per-connection-classifier=both-addresses-and-ports:2/1 

10   ;;; Mark Routing WAN ADSL
     chain=prerouting action=mark-routing new-routing-mark=to_WAN 7M 
     passthrough=no in-interface=ether2-master-local connection-mark=wan_7M 

11   ;;; Mark Routing WAN SDSL
     chain=prerouting action=mark-routing new-routing-mark=to_WAN SDSL 
     passthrough=no in-interface=ether2-master-local connection-mark=wan_SDSL 

12   ;;; Routing Mark WAN ADSL
     chain=output action=mark-routing new-routing-mark=to_WAN 7M passthrough=no 
     connection-mark=wan_7M 

13   ;;; Routing Mark WAN SDSL
     chain=output action=mark-routing new-routing-mark=to_WAN SDSL 
     passthrough=no connection-mark=wan_SDSL

If I disable the 5th rule, the webserver works fine when requesting the static IP from outside, but Internet works slow as hell from the LAN. This rule with or without “routing-mark=!Webserver” the web server doesn’t works.


Routes:

ip route print detail
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 0 A S  dst-address=0.0.0.0/0 gateway=190.228.137.177 
        gateway-status=190.228.137.177 reachable via  ether1-gateway 
        check-gateway=ping distance=1 scope=30 target-scope=10 
        routing-mark=Webserver 

 1 A S  dst-address=0.0.0.0/0 gateway=192.168.1.1 
        gateway-status=192.168.1.1 reachable via  ether5 - WAN - ADSL 7M 313 
        check-gateway=ping distance=1 scope=30 target-scope=10 
        routing-mark=HTTP traffic routing 

 2 A S  dst-address=0.0.0.0/0 gateway=192.168.1.1 
        gateway-status=192.168.1.1 reachable via  ether5 - WAN - ADSL 7M 313 
        check-gateway=ping distance=1 scope=30 target-scope=10 
        routing-mark=HTTPS traffic routing 

 3 A S  dst-address=0.0.0.0/0 gateway=192.168.1.1 
        gateway-status=192.168.1.1 reachable via  ether5 - WAN - ADSL 7M 313 
        check-gateway=ping distance=1 scope=30 target-scope=10 
        routing-mark=to_WAN 7M 

 4 A S  dst-address=0.0.0.0/0 gateway=190.228.137.177 
        gateway-status=190.228.137.177 reachable via  ether1-gateway 
        check-gateway=ping distance=1 scope=30 target-scope=10 
        routing-mark=to_WAN SDSL 

 5 A S  dst-address=0.0.0.0/0 gateway=190.228.137.177 
        gateway-status=190.228.137.177 reachable via  ether1-gateway 
        check-gateway=ping distance=1 scope=30 target-scope=10 

 6   S  dst-address=0.0.0.0/0 gateway=192.168.1.1 
        gateway-status=192.168.1.1 reachable via  ether5 - WAN - ADSL 7M 313 
        check-gateway=ping distance=2 scope=30 target-scope=10 

 7 ADC  dst-address=10.81.1.0/24 pref-src=10.81.1.1 gateway=ether2-master-local 
        gateway-status=ether2-master-local reachable distance=0 scope=10 

[...a bunch of l2tp connections]

22 ADC  dst-address=190.228.137.176/29 pref-src=190.228.137.179 
        gateway=ether1-gateway gateway-status=ether1-gateway reachable 
        distance=0 scope=10 

23 ADC  dst-address=192.168.1.0/24 pref-src=192.168.1.254 
        gateway=ether5 - WAN - ADSL 7M 313 
        gateway-status=ether5 - WAN - ADSL 7M 313 reachable distance=0 scope=10 

24 ADS  [other l2tp]

Any help would be greatly appreciated :slight_smile:



Edit 01:

I think I have this now working well… running some tests by now. Later I will update this post.

Solved!

I have redid the mangle configuration following some MUM slides.

Internet goes well, the webserver too.

    ip firewall address print
    Flags: X - disabled, D - dynamic
    #   LIST                                       ADDRESS                       
    0   local-networks                             10.81.1.0/24                   
    1   local-networks                             10.81.3.0/24                   
    2   local-networks                             10.81.4.0/24                   
    3   local-networks                             10.81.5.0/24                   
    4   local-networks                             10.81.6.0/24                   
    5   local-networks                             10.81.7.0/24                   
    6   local-networks                             10.81.8.0/24                   
    7   local-networks                             192.168.1.0/24                 
    8   local-networks                             190.228.137.176/29             
    9   local-networks                             192.168.88.0/24               
    10   webserver                                   10.81.1.15                     
    11   LAN                                           10.81.1.0/24

and mangle…

    ip firewall mangle print
    Flags: X - disabled, I - invalid, D - dynamic
    0   chain=prerouting action=accept dst-address=190.228.137.176/29
         in-interface=ether2-master-local

    1   chain=prerouting action=accept dst-address=192.168.1.0/24
         in-interface=ether2-master-local

    2   ;;; Allow local networks
         chain=prerouting action=accept src-address-list=local-networks
         dst-address-list=local-networks

    3 X ;;; Bypass webserver
         chain=prerouting action=mark-routing new-routing-mark=Webserver
         passthrough=no src-address-list=webserver

    4 X ;;; HTTPS Traffic Routing
         chain=prerouting action=mark-routing
         new-routing-mark=HTTPS traffic routing passthrough=no protocol=tcp
         dst-port=443,8004,8002

    5 X ;;; HTTP Traffic Routing
         chain=prerouting action=mark-routing
         new-routing-mark=HTTP traffic routing passthrough=no protocol=tcp
         routing-mark=!Webserver dst-port=80

    6   ;;; Mark connection WAN ADSL to Router
         chain=input action=mark-connection new-connection-mark=ADSL to Router
         passthrough=no connection-state=new
         in-interface=ether5 - WAN - ADSL 7M 313 connection-mark=no-mark

    7   ;;; Mark connection WAN SDSL to Router
         chain=input action=mark-connection new-connection-mark=SDSL to Router
         passthrough=no connection-state=new in-interface=ether1-gateway
         connection-mark=no-mark

    8   ;;; Routing Mark WAN ADSL
         chain=output action=mark-routing new-routing-mark=To ADSL Route
         passthrough=no connection-mark=ADSL to Router

    9   ;;; Routing Mark WAN SDSL
         chain=output action=mark-routing new-routing-mark=To SDSL Route
         passthrough=no connection-mark=SDSL to Router

    10   ;;; Mark connection SDSL to LAN
         chain=forward action=mark-connection new-connection-mark=SDSL to LAN
         passthrough=no in-interface=ether1-gateway connection-mark=no-mark

    11   ;;; Mark connection ADSL to LAN
         chain=forward action=mark-connection new-connection-mark=ADSL to LAN
         passthrough=no in-interface=ether5 - WAN - ADSL 7M 313
         connection-mark=no-mark

    12   ;;; Mark Routing WAN ADSL
         chain=prerouting action=mark-routing new-routing-mark=To ADSL Route
         passthrough=no src-address-list=LAN in-interface=ether2-master-local
         connection-mark=ADSL to LAN

    13   ;;; Mark Routing WAN SDSL
         chain=prerouting action=mark-routing new-routing-mark=To SDSL Route
         passthrough=no src-address-list=LAN in-interface=ether2-master-local
         connection-mark=SDSL to LAN

    14   ;;; Mark Connection WAN ADSL - 0
         chain=prerouting action=mark-connection new-connection-mark=ADSL to LAN
         passthrough=no connection-state=new dst-address-type=!local
         src-address-list=LAN dst-address-list=!local-networks
         in-interface=ether2-master-local connection-mark=no-mark
         per-connection-classifier=both-addresses:3/0

    15   ;;; Mark Connection WAN ADSL - 1
         chain=prerouting action=mark-connection new-connection-mark=ADSL to LAN
         passthrough=no connection-state=new dst-address-type=!local
         src-address-list=LAN dst-address-list=!local-networks
         in-interface=ether2-master-local connection-mark=no-mark
         per-connection-classifier=both-addresses:3/1

    16   ;;; Mark Connection WAN SDSL - 2
         chain=prerouting action=mark-connection new-connection-mark=SDSL to LAN
         passthrough=no connection-state=new dst-address-type=!local
         src-address-list=LAN dst-address-list=!local-networks
         in-interface=ether2-master-local connection-mark=no-mark
         per-connection-classifier=both-addresses:3/2

Now I have deleted the disabled rules. Everything runs fine.

Thanks for reading! I hope that can be useful for someone. :slight_smile:


Edit:


With that configuration the router wasn’t balancing at all and all the traffic went out through the SDSL (distance=1). I remade all the configuration again following the PCC Manual, doing some research, trial and error and finally I had my load balancing working better than I expected.

Now the mangle config is:

    ip firewall mangle print
    Flags: X - disabled, I - invalid, D - dynamic
    0   chain=prerouting action=accept dst-address=190.228.137.176/29
         in-interface=ether2-master-local

    1   chain=prerouting action=accept dst-address=192.168.1.0/24
         in-interface=ether2-master-local

    2   ;;; Allow local networks
         chain=prerouting action=accept src-address-list=local-networks
         dst-address-list=local-networks

    3   ;;; Mark connection WAN ADSL to Router
         chain=prerouting action=mark-connection
         new-connection-mark=ADSL to Router passthrough=no connection-state=new
         in-interface=ether5 - WAN - ADSL 7M 313 connection-mark=no-mark

    4   ;;; Mark connection WAN SDSL to Router
         chain=prerouting action=mark-connection
         new-connection-mark=SDSL to Router passthrough=no connection-state=new
         in-interface=ether1-gateway connection-mark=no-mark

    5 X ;;; Mark connection SDSL to LAN
         chain=forward action=mark-connection new-connection-mark=SDSL to LAN
         passthrough=no in-interface=ether1-gateway connection-mark=no-mark

    6 X ;;; Mark connection ADSL to LAN
         chain=forward action=mark-connection new-connection-mark=ADSL to LAN
         passthrough=no in-interface=ether5 - WAN - ADSL 7M 313
         connection-mark=no-mark

    7   ;;; Mark Connection WAN ADSL - 0
         chain=prerouting action=mark-connection
         new-connection-mark=ADSL to Router passthrough=yes connection-state=new
         dst-address-type=!local in-interface=ether2-master-local
         connection-mark=no-mark
         per-connection-classifier=both-addresses-and-ports:3/0

    8   ;;; Mark Connection WAN ADSL - 1
         chain=prerouting action=mark-connection
         new-connection-mark=ADSL to Router passthrough=yes connection-state=new
         dst-address-type=!local in-interface=ether2-master-local
         connection-mark=no-mark
         per-connection-classifier=both-addresses-and-ports:3/1

    9   ;;; Mark Connection WAN SDSL - 2
         chain=prerouting action=mark-connection
         new-connection-mark=ADSL to Router passthrough=yes connection-state=new
         dst-address-type=!local in-interface=ether2-master-local
         connection-mark=no-mark
         per-connection-classifier=both-addresses-and-ports:3/2

    10   ;;; Mark Routing WAN ADSL
         chain=prerouting action=mark-routing new-routing-mark=To ADSL Route
         passthrough=no in-interface=ether2-master-local
         connection-mark=ADSL to Router

    11   ;;; Mark Routing WAN SDSL
         chain=prerouting action=mark-routing new-routing-mark=To SDSL Route
         passthrough=no in-interface=ether2-master-local
         connection-mark=SDSL to Router

    12   ;;; Routing Mark WAN ADSL
         chain=output action=mark-routing new-routing-mark=To ADSL Route
         passthrough=no connection-mark=ADSL to Router

    13   ;;; Routing Mark WAN SDSL
         chain=output action=mark-routing new-routing-mark=To SDSL Route
         passthrough=no connection-mark=SDSL to Router

That forwards from the slides apparently had no much sense in there. I’m actually comparing the PCC methods “both-addresses” and “both-addresses-and-ports”.

Well, that’s all for now :laughing: