PCC loadbalace and Failover not correct.

thiendn04 · May 30, 2018, 4:12am

Hi everyone,
I have a stupid case.

I have 7 wan pppoe client, and it loadbalacing and failover equal pcc.
When i run full speedtest, it don’t run full all traffic in 7 wan, it have randon 5 wan full speedtest.
Can you give me some solutions.
Thanks.

This is my script.

\

mar/20/2018 09:13:39 by RouterOS 6.40.5

software id = AE0Y-A9D4

/interface vlan
add interface=Bridge_LAN name=TuVan vlan-id=18
add interface=Bridge_LAN name=HanhChanh vlan-id=19
add interface=Bridge_LAN name=KinhDoanh vlan-id=17

/ip address
add address=172.100.10.1/24 interface=Bridge_LAN network=172.100.10.0
add address=172.100.20.1/16 interface=TuVan network=172.100.20.0
add address=172.110.0.1/22 interface=HanhChanh network=172.110.0.0
add address=172.120.0.1/22 interface=KinhDoanh network=172.120.0.0

/ip firewall mangle
add action=mark-connection chain=input in-interface=WAN1 new-connection-mark=WAN1_conn passthrough=yes
add action=mark-connection chain=input in-interface=WAN2 new-connection-mark=WAN2_conn passthrough=yes
add action=mark-connection chain=input in-interface=WAN3 new-connection-mark=WAN3_conn passthrough=yes
add action=mark-connection chain=input in-interface=WAN4 new-connection-mark=WAN4_conn passthrough=yes
add action=mark-connection chain=input in-interface=WAN5 new-connection-mark=WAN5_conn passthrough=yes
add action=mark-connection chain=input in-interface=WAN6 new-connection-mark=WAN6_conn passthrough=yes
add action=mark-connection chain=input in-interface=WAN7 new-connection-mark=WAN7_conn passthrough=yes

add action=mark-routing chain=output connection-mark=WAN1_conn new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN2_conn new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN3_conn new-routing-mark=to_WAN3 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN4_conn new-routing-mark=to_WAN4 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN5_conn new-routing-mark=to_WAN5 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN6_conn new-routing-mark=to_WAN6 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN7_conn new-routing-mark=to_WAN7 passthrough=yes

add action=mark-connection chain=prerouting dst-address-type=!local in-interface=Bridge_LAN new-connection-mark=WAN1_conn passthrough=yes per-connection-classifier=both-addresses-and-ports:7/0
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=Bridge_LAN new-connection-mark=WAN2_conn passthrough=yes per-connection-classifier=both-addresses-and-ports:7/1
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=Bridge_LAN new-connection-mark=WAN3_conn passthrough=yes per-connection-classifier=both-addresses-and-ports:7/2
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=Bridge_LAN new-connection-mark=WAN4_conn passthrough=yes per-connection-classifier=both-addresses-and-ports:7/3
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=Bridge_LAN new-connection-mark=WAN5_conn passthrough=yes per-connection-classifier=both-addresses-and-ports:7/4
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=Bridge_LAN new-connection-mark=WAN6_conn passthrough=yes per-connection-classifier=both-addresses-and-ports:7/5
add action=mark-connection chain=prerouting dst-address-type=!local in-interface=Bridge_LAN new-connection-mark=WAN7_conn passthrough=yes per-connection-classifier=both-addresses-and-ports:7/6

add action=mark-routing chain=prerouting connection-mark=WAN1_conn in-interface=Bridge_LAN new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_conn in-interface=Bridge_LAN new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN3_conn in-interface=Bridge_LAN new-routing-mark=to_WAN3 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN4_conn in-interface=Bridge_LAN new-routing-mark=to_WAN4 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN5_conn in-interface=Bridge_LAN new-routing-mark=to_WAN5 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN6_conn in-interface=Bridge_LAN new-routing-mark=to_WAN6 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN7_conn in-interface=Bridge_LAN new-routing-mark=to_WAN7 passthrough=yes

/ip firewall nat
add action=masquerade chain=srcnat out-interface=WAN1
add action=masquerade chain=srcnat out-interface=WAN2
add action=masquerade chain=srcnat out-interface=WAN3
add action=masquerade chain=srcnat out-interface=WAN4
add action=masquerade chain=srcnat out-interface=WAN5
add action=masquerade chain=srcnat out-interface=WAN6
add action=masquerade chain=srcnat out-interface=WAN7
/ip route
add check-gateway=ping distance=1 gateway=WAN1 routing-mark=to_WAN1
add check-gateway=ping distance=1 gateway=WAN2 routing-mark=to_WAN2
add check-gateway=ping distance=1 gateway=WAN3 routing-mark=to_WAN3
add check-gateway=ping distance=1 gateway=WAN4 routing-mark=to_WAN4
add check-gateway=ping distance=1 gateway=WAN5 routing-mark=to_WAN5
add check-gateway=ping distance=1 gateway=WAN6 routing-mark=to_WAN6
add check-gateway=ping distance=1 gateway=WAN7 routing-mark=to_WAN7

add check-gateway=ping distance=2 gateway=WAN1
add check-gateway=ping distance=3 gateway=WAN2
add check-gateway=ping distance=4 gateway=WAN3
add check-gateway=ping distance=5 gateway=WAN4
add check-gateway=ping distance=6 gateway=WAN5
add check-gateway=ping distance=7 gateway=WAN6
add check-gateway=ping distance=8 gateway=WAN7

macgaiver · May 30, 2018, 4:57am

your implementation doesn’t correspond with best practice example in manual
this is per address-pair load balancing (almost same as per connection load balancing), one connection can use only one gateway, so you will never be able to get speedtest aggregated on all 7 WANs - you need something consistent of many many connection from and to different IPs (like torrent) to utilize this connection aggregation method.

thiendn04 · May 30, 2018, 6:30am

Thank you your imformation.
can you repair for me,please ?

sindy · May 30, 2018, 8:42am

I’ll try to use another words for what @macgaiver already said: speedtest uses a limited number of parallel connections to test the bandwidth. When I was trying on a single WAN, it used only 4 per direction, so I am even a little bit surprised that you could get 5 times the bandwidth of a single connection.

Another point is that if your uplinks are mobile modems from the same operator in the same cell, you may have exhausted the total capacity of the cell.

If you can run Wireshark, read this post from a similar topic and provide your observations regarding how many connections per direction does speedtest use in your case with 7 uplinks combined. Look at times of the GET messages, the last N starting at about the same time are those used to measure upload, N is the number we are interested in. If N<7, it is your answer; if N>=7, the cell capacity exhaustion is the most likely reason.

As for the firewall rules, you waste CPU by using passthrough=yes in the mark-routing rules and by connection-marking packets belonging to already marked connections, so the time the firewall has to burn on each packet is uselessly high.