PCC Load balancing not working

RouterOS General
1

I have following setup
Ether1- > ISP1
Ether5 -> ISP2
Ether2 -> LAN

Following is configuation
0 chain=prerouting action=accept dst-address=111.x.x.x/24
in-interface=ether2 - LAN

1 chain=prerouting action=accept dst-address=182.x.x.x/24
in-interface=ether2 - LAN

2 ;;; WAN->ROS
chain=input action=mark-connection new-connection-mark=ISP1_conn
passthrough=yes in-interface=ether1-WAN connection-mark=no-mark

3 chain=input action=mark-connection new-connection-mark=ISP2_conn
passthrough=yes in-interface=ether5-airtel connection-mark=no-mark

4 ;;; Mark Connection and then Route
chain=prerouting action=mark-connection new-connection-mark=ISP1_conn
passthrough=yes src-address=10.15.15.2-10.15.15.254
dst-address-type=!local in-interface=ether2 - LAN connection-mark=no-mark
per-connection-classifier=both-addresses-and-ports:2/0

5 chain=prerouting action=mark-connection new-connection-mark=ISP2_conn
passthrough=yes src-address=10.15.15.2-10.15.15.254
dst-address-type=!local in-interface=ether2 - LAN connection-mark=no-mark
per-connection-classifier=both-addresses:2/1

6 chain=prerouting action=mark-routing new-routing-mark=to_ISP1
passthrough=yes in-interface=ether2 - LAN connection-mark=ISP1_conn

7 chain=prerouting action=mark-routing new-routing-mark=to_ISP2
passthrough=yes in-interface=ether2 - LAN connection-mark=ISP2_conn

8 chain=output action=mark-routing new-routing-mark=to_ISP1 passthrough=yes
connection-mark=ISP1_conn

9 chain=output action=mark-routing new-routing-mark=to_ISP2 passthrough=yes
connection-mark=ISP2_conn

10 chain=prerouting action=mark-connection new-connection-mark=ISP1_conn
passthrough=yes in-interface=ether1-WAN connection-mark=no-mark

11 chain=prerouting action=mark-connection new-connection-mark=ISP2_conn
passthrough=yes in-interface=ether5-airtel connection-mark=no-mark

===============
Routes

DST-ADDRESS PREF-SRC GATEWAY DISTANCE

0 A S 0.0.0.0/0 111.x.x.x 1
1 A S 0.0.0.0/0 182.x.x.x 1
2 A S 0.0.0.0/0 111.x.x.x.x 1
3 S 0.0.0.0/0 182.x.x.x.x 1
4 X S 0.0.0.0/0 182.x.x.x.x 1


\

Here both routes are not used. At at a time Only WAN1 is used. If I remove and insert Ether1( First WAN), then traffic goes to Ether5(WAN2), and then after inserting Ether1 it works for sometimes and then again traffic does not go via Ether5(WAN2). Both have same 10Mbps links and I want 20Mbps to be used but it only utilizes 10 Mbps and negligible traffic seems to be going on WAN2.

Can someone please guide anything wrong into my config? I have PPPOE server setup on Ether2.

2

Here is my config what is working perfectly:

/ip firewall address-list
add address=172.16.0.0/16 list=PrivateIPs
add address=10.0.0.0/8 list=PrivateIPs
add address=192.168.0.0/16 list=PrivateIPs

/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark \
    in-interface=wan1 new-connection-mark=ISP1_conn
add action=mark-connection chain=prerouting connection-mark=no-mark \
    in-interface=wan0 new-connection-mark=ISP2_conn
add action=mark-connection chain=prerouting connection-mark=no-mark \
    dst-address-list=!PrivateIPs in-interface=lan0 new-connection-mark=ISP1_conn \
    per-connection-classifier=src-port:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark \
    dst-address-list=!PrivateIPs in-interface=lan0 new-connection-mark=ISP2_conn \
    per-connection-classifier=src-port:2/1
add action=mark-routing chain=prerouting connection-mark=ISP1_conn \
    in-interface=lan0 new-routing-mark=to_ISP1
add action=mark-routing chain=prerouting connection-mark=ISP1_conn \
    in-interface=lan1 new-routing-mark=to_ISP1
add action=mark-routing chain=prerouting connection-mark=ISP2_conn \
    in-interface=lan0 new-routing-mark=to_ISP2
add action=mark-routing chain=prerouting connection-mark=ISP2_conn \
    in-interface=lan1 new-routing-mark=to_ISP2
add action=mark-routing chain=output connection-mark=ISP1_conn \
    new-routing-mark=to_ISP1
add action=mark-routing chain=output connection-mark=ISP2_conn \
    new-routing-mark=to_ISP2
add chain=prerouting dst-address=138.xxx.xxx.0/22 in-interface=lan0
add chain=prerouting dst-address=66.xxx.xxx.0/22 in-interface=lan0

/ip firewall nat
add action=masquerade chain=srcnat out-interface=wan1
add action=masquerade chain=srcnat out-interface=wan0


/ip route
add distance=10 gateway=138.xxx.xxx.1 routing-mark=to_ISP1
add distance=10 gateway=66.xxx.xxx.1 routing-mark=to_ISP2
3

what are these two rules for ?

add chain=prerouting dst-address=138.xxx.xxx.0/22 in-interface=lan0
add chain=prerouting dst-address=66.xxx.xxx.0/22 in-interface=lan0

4

They are “accept” rules. The export seems to leave out the “action=accept”
Not 100% sure why they are needed, but it doesn’t work without them.

5

Wouldn’t the accept rules have to be at the top, so they skip processing certain traffic. At the bottom they would seem to have no effect because the default in a chain is accept anyway isnt it?

I agree that if it doesn’t work without them then there must be some purpose, but I cant see what it is though :slight_smile:.

6

I would have thought they would be needed earlier in the prerouting chain -
I’m pretty sure these rules are there to make sure that traffic into ISP1’s IP space only go out ISP1 interface - no PCC.
Same is true for ISP2.

This is needed for the ping tests to check gateway without failing.

Another way to achieve the same result would be to drop those two rules and to add more static routes:
/ip route
add dst=138.xxx.xxx.0/24 gateway=WAN1 routing-mark=to_ISP1
add dst=138.xxx.xxx.0/24 gateway=WAN1 routing-mark=to_ISP2
add dst=66.xxx.xxx.0/24 gateway=WAN2 routing-mark=to_ISP1
add dst=66.xxx.xxx.0/24 gateway=WAN2 routing-mark=to_ISP2

of course use the correct netmask instead of /24

This allows all routing tables to have visibility into the locally-attached WAN segments, regardless of what PCC chooses for the routing table to use. Personally, I find that to be the cleaner solution because you don’t need to put exceptions into the PCC behavior.

7

hmmm yes.
Then I’m trying to see why you have to route into a ISP address space.
Normally with an ISP connection I have a gateway and a small (/29) assigned address range including that gateway, and I route all traffic to the gateway.
I’m not familiar with routing into an ISP’s subnet… not sure why or when that comes into play…

8

The only one that’s really required is the directly connected wan segments. (/29 or whatever mask they may be).

Routing an ISP’s entire CIDR block(s) is mostly an efficiency thing - don’t make targets in ISP1 go "all the way around the Internet’ to get to my ISP2 interface… that’s the thinking there. It’s optional.

9

thanks for clearing up that mystery!
I’m slowly chugging my way through this stuff, routing marks and so on, its so powerful and very interesting.

10

I have two cable modems from the same ISP. My ISP, Charter Comunications, offers 60/5 and 100/5. But the 60M is $50/month and the 100M is $150/month. So the logical choice for me was two 60M modems at $100/month.

As for the routing, in my case, the /22 is simply what the ISP is assigning via dhcp. Each modem is on a /22 subnet. It took a little work to get the modems on different subnets. (Simply changing the MAC address on the WAN until each was assigned a different subnet).

My config, in the order posted, works very well. The only thing missing is two extra default routes, they are dynamic from the DHCP client on each WAN interface. Each WAN DCHP client sets a default route with a distance of 100. This allows the two static routes for “ISP1” and “ISP2” with a distance of 10 to take priority. By letting the DHCP set this for both WANs, if one goes down, there is no need to setup extra packet marking. If the interface is not online, then the default 100 route takes over.

If it helps to see WinBox screen shots, I can post those later this evening.

11

thanks for that.
so your static routes must be crossed over from the dhcp routes, each points to the other ISP?

12

the static routes are there to ensure that the modems don’t cross over, and so that the monitor can watch the gateways.