SCRIPT: Dual WAN Load Balancing with Fail-over

RouterOS General
1

Hi Mikrotik Masters!

I would like to ask for your help for me to implement the following on my two ISPs (dedicated lines). Both has 6 Mbps bandwidth.:

  1. Dual WAN Load-Balancing
  2. Fail-over
  3. Bandwidth Bonding/ Aggregation (optional)

Overview of my set-up:

ether2-WAN1
ether3-WAN2
ether4- bridge for vlan gateways 10,20,30 and 40 going to a switch

Connection is just like this: Rb951G-2HnD <—> Switch


I already tried the tutorials from the following sites, but still fail-over is working but Load-Balancing is not:

Can someone help me with the complete mangle rules, route settings, NAT (masquerade), DNSs, and everything?

Thank you so much.

2

Have you tried the pcc method?
http://wiki.mikrotik.com/wiki/PCC

3

Yes I already tried that, Sir. But to make sure, I’ll try it again later.

What I needed primarily is the load balancing with Fail-over feature on my Dual WAN set-up.

Currently, with above mentioned guides I tried, all results are the same, no traffic is being showed on the WAN2 interface but the Fail-over is working.

I’m using 6.20 (mipsbe) version on my RB951G-2HnD.

4

Hi Caci99,

I already tried again the PCC method, but still bandwidth of the WAN2 is not being used by seeing the Traffic tab.

5

Currently, the script on this guide was applied to my network:

http://mikrotikbook.blogspot.com/2014/04/mikrotik-dual-wan-load-balancing-using.html

6

How are you testing it? From a single computer? Also, post your config in here, so we can have a look at it.

7

Here’s my script.

Both WAN1 and WAN2 has the same Bandwidth which is 6 Mbps Upload and 6 Mbps Download each link.

/interface bridge
add mtu=1500 name=bridge1-LAN


/interface ethernet
set [ find default-name=ether1 ] disabled=yes mac-address=D4:CA:6D:80:A5:8A
set [ find default-name=ether2 ] mac-address=D4:CA:6D:80:A5:8B name=ether2-WAN1
set [ find default-name=ether3 ] mac-address=D4:CA:6D:80:A5:8C name=ether3-WAN2
set [ find default-name=ether4 ] mac-address=D4:CA:6D:80:A5:8D name=ether4-LAN
set [ find default-name=ether5 ] mac-address=D4:CA:6D:80:A5:8E name=ether5-LAN

/interface vlan
add interface=bridge1-LAN l2mtu=1594 name=vlan10-President vlan-id=10
add interface=bridge1-LAN l2mtu=1594 name=vlan20-Recruiting vlan-id=20
add interface=bridge1-LAN l2mtu=1594 name=vlan30-VoIP vlan-id=30
add interface=bridge1-LAN l2mtu=1594 name=vlan40-Employees vlan-id=40

/interface bridge port
add bridge=bridge1-LAN interface=ether4-LAN
add bridge=bridge1-LAN interface=ether5-LAN


/ip address

add address=192.168.1.5/27 comment=“WAN1 GW” interface=ether2-WAN1 network=192.168.1.0
add address=192.168.2.2/30 comment=“WAN2 GW” interface=ether3-WAN2 network=192.168.2.0

add address=172.30.8.1/24 comment=“Management VLAN” interface=bridge1-LAN network=172.30.8.0
add address=172.16.20.1/28 comment=“Recruiting IP Block GW” interface=vlan20-Recruiting network=172.16.20.0
add address=172.16.10.1/27 comment=“President IP Block GW” interface=vlan10-President network=172.16.10.0
add address=172.16.30.1/28 comment=“VoIP IP Block GW” interface=vlan30-VoIP network=172.16.30.0
add address=172.30.40.1/24 comment=“Employees IP Block GW” interface=vlan40-Employees network=172.30.40.0



/ip firewall mangle EXPORT
add action=mark-connection chain=input comment=“Dual WAN Load Balancing w/ Fail Over” in-interface=ether2-WAN1 new-connection-mark=WAN1_mark
passthrough=no
add action=mark-connection chain=input comment=“Dual WAN Load Balancing w/ Fail Over” in-interface=ether3-WAN2 new-connection-mark=WAN2_mark
passthrough=no

add action=mark-routing chain=output comment=“Dual WAN Load Balancing w/ Fail Over” connection-mark=WAN1_mark new-routing-mark=to_ISP1 passthrough=no
add action=mark-routing chain=output comment=“Dual WAN Load Balancing w/ Fail Over” connection-mark=WAN2_mark new-routing-mark=to_ISP2 passthrough=no

add chain=prerouting comment=“Dual WAN Load Balancing w/ Fail Over” dst-address=192.168.1.0/27 in-interface=bridge1-LAN
add chain=prerouting comment=“Dual WAN Load Balancing w/ Fail Over” dst-address=192.168.2.0/30 in-interface=bridge1-LAN

add action=mark-connection chain=prerouting comment=“Dual WAN Load Balancing w/ Fail Over” dst-address-type=!local in-interface=bridge1-LAN
new-connection-mark=WAN1_mark per-connection-classifier=both-addresses-and-ports:2/0
add action=mark-connection chain=prerouting comment=“Dual WAN Load Balancing w/ Fail Over” dst-address-type=!local in-interface=bridge1-LAN
new-connection-mark=WAN2_mark per-connection-classifier=both-addresses-and-ports:2/1

add action=mark-routing chain=prerouting comment=“Dual WAN Load Balancing w/ Fail Over” connection-mark=WAN1_mark in-interface=bridge1-LAN
new-routing-mark=to_ISP1 passthrough=no
add action=mark-routing chain=prerouting comment=“Dual WAN Load Balancing w/ Fail Over” connection-mark=WAN2_mark in-interface=bridge1-LAN
new-routing-mark=to_ISP2 passthrough=no


/ip firewall mangle PRINT

0 ;;; Dual WAN Load Balancing w/ Fail Over
chain=input action=mark-connection new-connection-mark=WAN1_mark passthrough=no in-interface=ether2-WAN1 log=no
log-prefix=“”

1 ;;; Dual WAN Load Balancing w/ Fail Over
chain=input action=mark-connection new-connection-mark=WAN2_mark passthrough=no in-interface=ether3-WAN2 log=no
log-prefix=“”

2 ;;; Dual WAN Load Balancing w/ Fail Over
chain=output action=mark-routing new-routing-mark=to_ISP1 passthrough=no connection-mark=WAN1_mark log=no log-prefix=“”

3 ;;; Dual WAN Load Balancing w/ Fail Over
chain=output action=mark-routing new-routing-mark=to_ISP2 passthrough=no connection-mark=WAN2_mark log=no log-prefix=“”

4 ;;; Dual WAN Load Balancing w/ Fail Over
chain=prerouting action=accept dst-address=192.168.1.0/27 in-interface=bridge1-LAN log=no log-prefix=“”

5 ;;; Dual WAN Load Balancing w/ Fail Over
chain=prerouting action=accept dst-address=192.168.2.0/30 in-interface=bridge1-LAN log=no log-prefix=“”

6 ;;; Dual WAN Load Balancing w/ Fail Over
chain=prerouting action=mark-connection new-connection-mark=WAN1_mark passthrough=yes dst-address-type=!local
in-interface=bridge1-LAN per-connection-classifier=both-addresses-and-ports:2/0 log=no log-prefix=“”

7 ;;; Dual WAN Load Balancing w/ Fail Over
chain=prerouting action=mark-connection new-connection-mark=WAN2_mark passthrough=yes dst-address-type=!local
in-interface=bridge1-LAN per-connection-classifier=both-addresses-and-ports:2/1 log=no log-prefix=“”

8 ;;; Dual WAN Load Balancing w/ Fail Over
chain=prerouting action=mark-routing new-routing-mark=to_ISP1 passthrough=no in-interface=bridge1-LAN
connection-mark=WAN1_mark log=no log-prefix=“”

9 ;;; Dual WAN Load Balancing w/ Fail Over
chain=prerouting action=mark-routing new-routing-mark=to_ISP2 passthrough=no in-interface=bridge1-LAN
connection-mark=WAN2_mark log=no log-prefix=“”


/ip route
add check-gateway=ping comment=“WAN1 GW” distance=1 gateway=192.168.1.1 routing-mark=to_ISP1
add check-gateway=ping comment=“WAN2 GW” distance=1 gateway=192.168.2.1 routing-mark=to_ISP2
add check-gateway=ping comment=“Normal Default Route excep for ‘Distance set to 1’” distance=1 gateway=192.168.1.1
add check-gateway=ping distance=2 gateway=192.168.2.1


/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether2-WAN1
add action=masquerade chain=srcnat out-interface=ether3-WAN2

Thank you.

8

Sorry for late response
I would change the classifier from “both addresses and ports” to “both addresses”
Also, in the route table, for the normal route you are using a different gateway which I can’t figure out where is coming from. I am talking about this:

add check-gateway=ping comment="Normal Default Route excep for 'Distance set to 1'" distance=1 gateway=210.213.67.65
add check-gateway=ping distance=2 gateway=202.78.78.189

I also see that you are using a modem or some other router before the routerboard. Can you bridge those devices so that it will be routerboard to deal directly with the internet conncetion instead of having another one doing NAT in front of it?

And last, how are you testing it? Are you testing it from a single PC? If yes, it is normal that you won’t see any load balancing.

9

HI Caci99!

Sorry, the default gateway IP Addresses has been corrected.

Are you sure I must choose ‘both addresses’ only and not ‘both addresses and ports’? I just want to clarify this because most of the dual wan load balancing script I’ve seen uses ‘both addresses and ports’. This confused me.

Can you explain to me the difference of the two?

Also, what modem are you talking about?

Thank you.

10

Read the wiki about the classifier, and you will get it better. If you divide the traffic with “both addresses and ports” it will mean that you might divide the same traffic in two or more streams. For example, local IP 1.1.1.1 sends requests to public IP 2.2.2.2 from ports 2000 and 2001. Using “both addresses and ports” might divide this traffic in two streams.
Using “both addresses” ensures that traffic from one device to another is kept in one stream, i.e. goes through the same gateway. Both cases may be used anyway, it depends on your needs.

I see you use IP 192.168.1.1 and 192.168.2.1 as gateway, which is typical when you have another modem or router between the internet and router. Since these are private addresses, there is some other device doing NAT before the router. You are doing at least double NAT which is not that good, so if you can avoid it, the better.

Also, it is important to understand how are you testing the PCC? See, the pcc is not perfect load balancing, it merely tries to divide the whole traffic of your network in two streams. If you are testing it from a single PC it is normal to see that only one gateway is used, that’s what pcc does, keeps the same connections on the same gateway. You need to test it from two or more devices in the course of the day.

11

Great! Thanks for the explanation Caci99.

I see you use IP 192.168.1.1 and 192.168.2.1 as gateway, which is typical when you have another modem or router between the internet and router. Since these are private addresses, there is some other device doing NAT before the router. You are doing at least double NAT which is not that good, so if you can avoid it, the better.

I’m not using any modem. The IP Addresses are just for the representation of the script :slight_smile:

Also, it is important to understand how are you testing the PCC? See, the pcc is not perfect load balancing, it merely tries to divide the whole traffic of your network in two streams. If you are testing it from a single PC it is normal to see that only one gateway is used, that’s what pcc does, keeps the same connections on the same gateway. You need to test it from two or more devices in the course of the day.

I’m using PCC on a multiple VLAN (multiple subnets) environment under a single bridge interface.

Also, the computers on the LAN experiencing slow browsing/ URL resolving.

I’m using Google DNS (8.8.8.:sunglasses: as my primary DNS and one of the two DNSs of my Primary ISP. Is there any effect why the secondary ISP is not being utilized because I haven’t used any of the two DNSs of the secondary ISP?

Will there be any adjustments needed with my set-up?

Thank you so much. God bless you.

12

In routerOS, is not recomended to use dns servers from different providers. Stick to one, google for example. The router will send requests to resolve dns names randomly at both servers specified. If they are from diferent providers, might cause problems.

13

In routerOS, is not recomended to use dns servers from different providers. Stick to one, google for example. The router will send requests to resolve dns names randomly at both servers specified. If they are from diferent providers, might cause problems.

I see. My DNSs was set-up like this, first 8.8.8.8 then 2nd DNS of ISP1 and 3rd DNS of ISP2. I also set these DNS under DHCP> Networks on each subnet/ network. Is it improper?

I’m using PCC script (already changed to ‘both addresses’) on a MULTIPLE VLAN (multiple subnets) environment under a single bridge interface.

Can you tell me why the secondary ISP is not being utilized most of the time?

Will there be any adjustments needed with my set-up and requirements?

My requirements is to have a DUAL WAN Load-balancing with Fail-over set-up. I’m not sure if load-balancing can also bond the two equal bandwidth of my two providers (6M + 6M = 12M).

Thank you so much. God bless you.

14

The dns you specified on the dhcp server will get eventually distributed to the devices wich will get ip configuration from dhcp servers. It depends on what clients these devices are using. Windows client, to cut it short, will query the first dns server, and if it can not resolve it, will query the second one. Problem here might be that the ISP might not allow query outside its own network. In load balancing from different ISP these would cause problems.
The best practice in Load Balancing, is to setup one public dns server on the router, and then let the users use the dns cache of the router, either by specifying it on the dhcp server or by doing it transparently:

/ip firewall nat
add chain=dstnat action=redirect to-ports=53 protocol=udp dst-address-type=!local dst-port=53
add chain=dstnat action=redirect to-ports=53 protocol=tcp dst-address-type=!local dst-port=53

These rules will redirect users requests to router’s dns cache. But you should also protect it from outside requests

/ip firewall filter
add chain=input action=drop protocol=tcp in-interface=ether1 dst-port=53
add chain=input action=drop protocol=udp in-interface=ether1 dst-port=53

assuming that ether1 is one of the WAN interfaces. This should be done for ever WAN interface

As for the load balancing issue, have a look at the counters of the rules of PCC in /ip firewall mangle. Are those counters more or less the same? Are they marking approximately the same quantity of packets?

15

Here’s my mangle configuration:

Is the firewall NAT rule is per ISP or only the firewall FILTER rule?

Thank you.

16

Rules number 6 and 7 are those who would divide the traffic in two streams. From your picture it looks like they are not doing it, one has 300 000 packets, the other 150 000 packets. It does not seem right, I mean, the configuration is ok, but it is not doing the separation right, one rule grabs more packets than the other, causing the majority of the traffic go through one gateway.
I don’t know why it is happening. Over the course of a day and more, the quantity of the packets should balance themselves, but it is not. Maybe there is one device doing a lot more traffic and the first rule grabs it first? I don’t know :frowning: .
Try disabling the accept rules, the rules number 4 and 5, and see what happens (just guessing here).

17

Hi Caci99, As I’ve said, I’m using PCC on a multiple VLAN (multiple subnets) environment under a single bridge interface. My requirements is to have a DUAL WAN Load-balancing with Fail-over set-up. I’m not sure if load-balancing can also bond the two equal bandwidth of my two providers (6M + 6M = 12M).

Will there be any adjustments needed with my set-up and requirements for me to achieve above goals?

Is there any adjustments needed on the ‘bridge interface’ since it carries multiple VLANs going to my switch? the bridge interface acts like a tagged port since it is connected to my Layer 2 switch which has multiple VLANs and the Mikrotik handles the gateways of each VLAN that’s why the bridge interface needs to be in trunk/ tagged mode for it to carry multiple VLANs going to the switch.

Thank you.

18

Feels like you are goind after water on the otherside of the river.

If you want to use to WAN connections and have failover also, just insert both routes in the route table with same cost/administrative distance (make sure to have individual IP adresses/subnets on each interface facing the WAN), and ECMP will kick in and load balance on session basis on available paths. To make the fail-over to work, activate the check gateway function on each route and if there is a problem with the next hop, that route is removed from the ECMP paths.

Simple no? Or did I miss any of your pre-reqs?

19

Hi darkiesan!

Thanks for your input on my issue with the Dual WAN Load Balancing with Automatic Fail-over.

Here’s my current routing table:

/ip route
add check-gateway=ping comment=“I-Gate ISP GW” distance=1 gateway=X.X.X.65 routing-mark=to_ISP1
add check-gateway=ping comment=“BayanTel ISP GW” distance=1 gateway=X.X.X.189 routing-mark=to_ISP2
add check-gateway=ping comment=“Normal Default Route excep for ‘Distance set to 1’” distance=1 gateway=X.X.X.65
add check-gateway=ping distance=2 gateway=X.X.X.189

Can you tell me what changes should I made with those entries?

Also in my mangle rule (posted above), is it much better to select ‘both addresses’ or ‘both addresses and ports’?

I’m having an issue with some ports not being allowed by Mikrotik. I want to just ALLOW ALL ports especially port 22. Right now I’m blocking some websites for each VLAN using Layer 7 Protocols only.

Thank you so much.

20

Hello!

My suggestion is to completly drop the mangling and then use:

/ip route
add check-gateway=ping comment=“I-Gate ISP GW” distance=1 gateway=X.X.X.65 routing-mark=to_ISP1
add check-gateway=ping comment=“BayanTel ISP GW” distance=1 gateway=X.X.X.189 routing-mark=to_ISP2
add check-gateway=ping comment=“Normal Default Route excep for ‘Distance set to 1’” distance=1 gateway=X.X.X.65
add check-gateway=ping distance=2 gateway=X.X.X.189

That will have the effect that the "-gate, BayanTel and “Normal default route” will be used in a loadbalanced fashion no matter the destination or protocol.
The last route with distance=2 will only be used if all first three default routes have been failed by the check gateway function.

Does that accomplish what you want to do?