Community discussions

MUM Europe 2020
 
nbencic
just joined
Topic Author
Posts: 3
Joined: Fri Oct 26, 2018 5:45 pm

SCRIPT: Multi WAN failover with multiple host checks

Fri Oct 26, 2018 7:31 pm

Hi all,

I'd like to share my script for multi WAN failover with multi-host checks.
I've used Tomas Kirnak's script, version 1.0.7 as base.

Reason(s) for all these modifications are multiple, but most important one would be failure of Tomas's script in some scenarios.
* Multiple hosts for connectivity check were added

* Sometimes IP tool ( used in Tomas's script ) doesn't work over some gateways/interfaces where we need to specify GW ip address, to overcome this, my modified script creates temporary route for every check-ip and routes it either over interface or IPaddr ( depending on our configuration )

* Speed selector for WAN recovery is added, and by default its set to 'slow', which means it will reduce 'PingFailCountISP*' by 3 for every run when it detects that WAN is working. We can also set it to 'fast', which will reduce 'PingFailCountISP*' to 0 on the first OK run
Route will return to original distance once ping fail counter reaches 0.



# ------------------- header -------------------
# Base script created by 
# Script by Tomas Kirnak, version 1.0.7
# If you use this script, or edit and
# re-use it, please keep the header intact.
#
# For more information and details about
# this script please visit the wiki page at
# http://wiki.mikrotik.com/wiki/Failover_Scripting


# Script modifications and adaptations done by Nikola Bencic v1.0.0
# 26.oct.2018. in Reykjavik
# ------------------- header -------------------

# Fill the WAN interface gateways ( ip address or interface name - preferred is IP address, but for dhcp-client WAN use interface name )
:local GatewayISP1 ether2
:local GatewayISP2 ether1
:local GatewayISP3 ether3
:local GatewayISP4 ether5
:local GatewayISP5 192.168.1.1

# Mark routes with comments and fill their marks ( unique comments only !!! )
:local RouteISP1 [/ip route find comment="WAN1"];
:local RouteISP2 [/ip route find comment="WAN2"];
:local RouteISP3 [/ip route find comment="WAN3"];
:local RouteISP4 [/ip route find comment="WAN4"];
:local RouteISP5 [/ip route find comment="WAN5"];

# Check hosts: opendns, googledns, 0.ntp.europe.pool ( you can add more hosts if needed - but remember to chec firewall rules and whitelist them if you need )
:local PingTargets {"208.67.222.222";"8.8.8.8";"185.19.184.35"}

# Amount of ping fails needed to declare route as faulty
:local FailTreshold 3

# Define the distance increase of a route when it fails ( number should be count of ISPs or greater )
:local DistanceIncrease 5

# -------------- stop editing here --------------

# Declare the global variables
:global PingFailCountISP1
:global OriginalRouteDistanceISP1
:global PingFailCountISP2
:global OriginalRouteDistanceISP2
:global PingFailCountISP3
:global OriginalRouteDistanceISP3
:global PingFailCountISP4
:global OriginalRouteDistanceISP4
:global PingFailCountISP5
:global OriginalRouteDistanceISP5

# This inicializes the PingFailCount variables, in case this is the 1st time the script has ran
:if ([:typeof $PingFailCountISP1] = "nothing") do={
	:set PingFailCountISP1 0
}
:if ([:typeof $PingFailCountISP2] = "nothing") do={
	:set PingFailCountISP2 0
}
:if ([:typeof $PingFailCountISP3] = "nothing") do={
	:set PingFailCountISP3 0
}
:if ([:typeof $PingFailCountISP4] = "nothing") do={
	:set PingFailCountISP4 0
}
:if ([:typeof $PingFailCountISP5] = "nothing") do={
	:set PingFailCountISP5 0
}

# This variable will be used to keep results of individual ping attempts
:local PingResult

# Clean routing table just in case we have a leftover route 
/ip route remove [find comment="FAILOVER CHECK ROUTE"]




###########################################################################################
# Check ISP1

# Check remote ip 3 times
:foreach k,pingTarget in=$PingTargets do={

	:delay 1s;
	/ip route add comment="FAILOVER CHECK ROUTE" distance=1 gateway=$GatewayISP1 dst-address=$pingTarget
	:local testRouteID [/ip route find comment="FAILOVER CHECK ROUTE"];
	/log warning "Pinging $pingTarget thru $GatewayISP1"

	:set PingResult [ping $pingTarget count=1];

 	:if ([/ip route get $testRouteID gateway-status] ~"unreachable"=true) do={

		:set PingFailCountISP1 ($PingFailCountISP1 + 1)
		/log error "Ping to $pingTarget thru $GatewayISP1 FAILED - unreachable gw"

 	} else={

		# remote ping failed, increase fail count isp +1
		:if ($PingResult = 0) do={
			:set PingFailCountISP1 ($PingFailCountISP1 + 1)
			/log error "Ping to $pingTarget thru $GatewayISP1 FAILED - no ping"
		};

		# remote ping passed, decrease fail count isp -1
		:if ($PingResult = 1) do={
			:if ($PingFailCountISP1 > 0) do={
				:set PingFailCountISP1 ($PingFailCountISP1 - 1)
			};
		};

 	}

	/ip route remove [find comment="FAILOVER CHECK ROUTE"]
};

#if ping to all 3 hosts failed, declare route as faulty
:if ($PingFailCountISP1 >= $FailTreshold) do={

	:if ([:typeof $OriginalRouteDistanceISP1] = "nothing") do={
		:set OriginalRouteDistanceISP1 [/ip route get $RouteISP1 distance]
	} 

	:if ($OriginalRouteDistanceISP1 + $DistanceIncrease != [/ip route get $RouteISP1 distance]) do={

		/log warning "ISP1 has a problem en route to $pingTarget - increasing distance of route."
		/ip route set $RouteISP1 distance=($OriginalRouteDistanceISP1 + $DistanceIncrease)
		/log warning "ISP1 Route distance increase finished."
	}
}

:if ($PingFailCountISP1 < $FailTreshold) do={

	/log warning "ISP1 is working ok"

	:if ([:typeof $OriginalRouteDistanceISP1] != "nothing") do={

		:if ([/ip route get $RouteISP1 distance] != $OriginalRouteDistanceISP1) do={

			/log warning "ISP1 can reach $pingTarget again - bringing back original distance of route."

			# if ISP1RecoverSpeed is "fast" - recovery of ISP will be FASTER - but there is potential danger of flapping in route ( if its unstable ISP )
			#
			# if ISP1RecoverSpeed is "slow" - recovery of ISP will be SLOWER - but there we are safe in case of unstable ISP
			:local ISP1RecoverSpeed "slow";

			:if ($ISP1RecoverSpeed = "slow") do={
				/log warning "ISP1 recovery is set to SLOW"
				:for pingRun from 0 to 2 do={
					:if ($PingFailCountISP1 > 0) do={
						:set PingFailCountISP1 ($PingFailCountISP1 - 1)
					}
				}
			} else={
				/log warning "ISP1 recovery is set to FAST"
				:set PingFailCountISP1 0
			}

			:if ($PingFailCountISP1 = 0) do={
				
				/ip route set $RouteISP1 distance=($OriginalRouteDistanceISP1)
				/log warning "ISP1 Route distance returned to original value."
			}
		}
	}
}

###########################################################################################
# Check ISP2

# Check remote ip 3 times
:foreach k,pingTarget in=$PingTargets do={

	:delay 1s;
	/ip route add comment="FAILOVER CHECK ROUTE" distance=1 gateway=$GatewayISP2 dst-address=$pingTarget
	:local testRouteID [/ip route find comment="FAILOVER CHECK ROUTE"];
	/log warning "Pinging $pingTarget thru $GatewayISP2"

	:set PingResult [ping $pingTarget count=1];

 	:if ([/ip route get $testRouteID gateway-status] ~"unreachable"=true) do={

		:set PingFailCountISP2 ($PingFailCountISP2 + 1)
		/log error "Ping to $pingTarget thru $GatewayISP2 FAILED - unreachable gw"

 	} else={

		# remote ping failed, increase fail count isp +1
		:if ($PingResult = 0) do={
			:set PingFailCountISP2 ($PingFailCountISP2 + 1)
			/log error "Ping to $pingTarget thru $GatewayISP2 FAILED - no ping"
		};

		# remote ping passed, decrease fail count isp -1
		:if ($PingResult = 1) do={
			:if ($PingFailCountISP2 > 0) do={
				:set PingFailCountISP2 ($PingFailCountISP2 - 1)
			};
		};

 	}

	/ip route remove [find comment="FAILOVER CHECK ROUTE"]
};

#if ping to all 3 hosts failed, declare route as faulty
:if ($PingFailCountISP2 >= $FailTreshold) do={

	:if ([:typeof $OriginalRouteDistanceISP2] = "nothing") do={
		:set OriginalRouteDistanceISP2 [/ip route get $RouteISP2 distance]
	} 

	:if ($OriginalRouteDistanceISP2 + $DistanceIncrease != [/ip route get $RouteISP2 distance]) do={

		/log warning "ISP2 has a problem en route to $pingTarget - increasing distance of route."
		/ip route set $RouteISP2 distance=($OriginalRouteDistanceISP2 + $DistanceIncrease)
		/log warning "ISP2 Route distance increase finished."
	}
}

:if ($PingFailCountISP2 < $FailTreshold) do={

	/log warning "ISP2 is working ok"

	:if ([:typeof $OriginalRouteDistanceISP2] != "nothing") do={

		:if ([/ip route get $RouteISP2 distance] != $OriginalRouteDistanceISP2) do={

			/log warning "ISP2 can reach $pingTarget again - bringing back original distance of route."

			# if ISP2RecoverSpeed is "fast" - recovery of ISP will be FASTER - but there is potential danger of flapping in route ( if its unstable ISP )
			#
			# if ISP2RecoverSpeed is "slow" - recovery of ISP will be SLOWER - but there we are safe in case of unstable ISP
			:local ISP2RecoverSpeed "slow";

			:if ($ISP2RecoverSpeed = "slow") do={
				/log warning "ISP2 recovery is set to SLOW"
				:for pingRun from 0 to 2 do={
					:if ($PingFailCountISP2 > 0) do={
						:set PingFailCountISP2 ($PingFailCountISP2 - 1)
					}
				}
			} else={
				/log warning "ISP2 recovery is set to FAST"
				:set PingFailCountISP2 0
			}

			:if ($PingFailCountISP2 = 0) do={
				
				/ip route set $RouteISP2 distance=($OriginalRouteDistanceISP2)
				/log warning "ISP2 Route distance returned to original value."
			}
		}
	}
}

###########################################################################################
# Check ISP3

# Check remote ip 3 times
:foreach k,pingTarget in=$PingTargets do={

	:delay 1s;
	/ip route add comment="FAILOVER CHECK ROUTE" distance=1 gateway=$GatewayISP3 dst-address=$pingTarget
	:local testRouteID [/ip route find comment="FAILOVER CHECK ROUTE"];
	/log warning "Pinging $pingTarget thru $GatewayISP3"

	:set PingResult [ping $pingTarget count=1];

 	:if ([/ip route get $testRouteID gateway-status] ~"unreachable"=true) do={

		:set PingFailCountISP3 ($PingFailCountISP3 + 1)
		/log error "Ping to $pingTarget thru $GatewayISP3 FAILED - unreachable gw"

 	} else={

		# remote ping failed, increase fail count isp +1
		:if ($PingResult = 0) do={
			:set PingFailCountISP3 ($PingFailCountISP3 + 1)
			/log error "Ping to $pingTarget thru $GatewayISP3 FAILED - no ping"
		};

		# remote ping passed, decrease fail count isp -1
		:if ($PingResult = 1) do={
			:if ($PingFailCountISP3 > 0) do={
				:set PingFailCountISP3 ($PingFailCountISP3 - 1)
			};
		};

 	}

	/ip route remove [find comment="FAILOVER CHECK ROUTE"]
};

#if ping to all 3 hosts failed, declare route as faulty
:if ($PingFailCountISP3 >= $FailTreshold) do={

	:if ([:typeof $OriginalRouteDistanceISP3] = "nothing") do={
		:set OriginalRouteDistanceISP3 [/ip route get $RouteISP3 distance]
	} 

	:if ($OriginalRouteDistanceISP3 + $DistanceIncrease != [/ip route get $RouteISP3 distance]) do={

		/log warning "ISP3 has a problem en route to $pingTarget - increasing distance of route."
		/ip route set $RouteISP3 distance=($OriginalRouteDistanceISP3 + $DistanceIncrease)
		/log warning "ISP3 Route distance increase finished."
	}
}

:if ($PingFailCountISP3 < $FailTreshold) do={

	/log warning "ISP3 is working ok"

	:if ([:typeof $OriginalRouteDistanceISP3] != "nothing") do={

		:if ([/ip route get $RouteISP3 distance] != $OriginalRouteDistanceISP3) do={

			/log warning "ISP3 can reach $pingTarget again - bringing back original distance of route."

			# if ISP3RecoverSpeed is "fast" - recovery of ISP will be FASTER - but there is potential danger of flapping in route ( if its unstable ISP )
			#
			# if ISP3RecoverSpeed is "slow" - recovery of ISP will be SLOWER - but there we are safe in case of unstable ISP
			:local ISP3RecoverSpeed "slow";

			:if ($ISP3RecoverSpeed = "slow") do={
				/log warning "ISP3 recovery is set to SLOW"
				:for pingRun from 0 to 2 do={
					:if ($PingFailCountISP3 > 0) do={
						:set PingFailCountISP3 ($PingFailCountISP3 - 1)
					}
				}
			} else={
				/log warning "ISP3 recovery is set to FAST"
				:set PingFailCountISP3 0
			}

			:if ($PingFailCountISP3 = 0) do={
				
				/ip route set $RouteISP3 distance=($OriginalRouteDistanceISP3)
				/log warning "ISP3 Route distance returned to original value."
			}
		}
	}
}

###########################################################################################
# Check ISP4

# Check remote ip 3 times
:foreach k,pingTarget in=$PingTargets do={

	:delay 1s;
	/ip route add comment="FAILOVER CHECK ROUTE" distance=1 gateway=$GatewayISP4 dst-address=$pingTarget
	:local testRouteID [/ip route find comment="FAILOVER CHECK ROUTE"];
	/log warning "Pinging $pingTarget thru $GatewayISP4"

	:set PingResult [ping $pingTarget count=1];

 	:if ([/ip route get $testRouteID gateway-status] ~"unreachable"=true) do={

		:set PingFailCountISP4 ($PingFailCountISP4 + 1)
		/log error "Ping to $pingTarget thru $GatewayISP4 FAILED - unreachable gw"

 	} else={

		# remote ping failed, increase fail count isp +1
		:if ($PingResult = 0) do={
			:set PingFailCountISP4 ($PingFailCountISP4 + 1)
			/log error "Ping to $pingTarget thru $GatewayISP4 FAILED - no ping"
		};

		# remote ping passed, decrease fail count isp -1
		:if ($PingResult = 1) do={
			:if ($PingFailCountISP4 > 0) do={
				:set PingFailCountISP4 ($PingFailCountISP4 - 1)
			};
		};

 	}

	/ip route remove [find comment="FAILOVER CHECK ROUTE"]
};

#if ping to all 3 hosts failed, declare route as faulty
:if ($PingFailCountISP4 >= $FailTreshold) do={

	:if ([:typeof $OriginalRouteDistanceISP4] = "nothing") do={
		:set OriginalRouteDistanceISP4 [/ip route get $RouteISP4 distance]
	} 

	:if ($OriginalRouteDistanceISP4 + $DistanceIncrease != [/ip route get $RouteISP4 distance]) do={

		/log warning "ISP4 has a problem en route to $pingTarget - increasing distance of route."
		/ip route set $RouteISP4 distance=($OriginalRouteDistanceISP4 + $DistanceIncrease)
		/log warning "ISP4 Route distance increase finished."
	}
}

:if ($PingFailCountISP4 < $FailTreshold) do={

	/log warning "ISP4 is working ok"

	:if ([:typeof $OriginalRouteDistanceISP4] != "nothing") do={

		:if ([/ip route get $RouteISP4 distance] != $OriginalRouteDistanceISP4) do={

			/log warning "ISP4 can reach $pingTarget again - bringing back original distance of route."

			# if ISP4RecoverSpeed is "fast" - recovery of ISP will be FASTER - but there is potential danger of flapping in route ( if its unstable ISP )
			#
			# if ISP4RecoverSpeed is "slow" - recovery of ISP will be SLOWER - but there we are safe in case of unstable ISP
			:local ISP4RecoverSpeed "slow";

			:if ($ISP4RecoverSpeed = "slow") do={
				/log warning "ISP4 recovery is set to SLOW"
				:for pingRun from 0 to 2 do={
					:if ($PingFailCountISP4 > 0) do={
						:set PingFailCountISP4 ($PingFailCountISP4 - 1)
					}
				}
			} else={
				/log warning "ISP4 recovery is set to FAST"
				:set PingFailCountISP4 0
			}

			:if ($PingFailCountISP4 = 0) do={
				
				/ip route set $RouteISP4 distance=($OriginalRouteDistanceISP4)
				/log warning "ISP4 Route distance returned to original value."
			}
		}
	}
}

###########################################################################################
# Check ISP5

# Check remote ip 3 times
:foreach k,pingTarget in=$PingTargets do={

	:delay 1s;
	/ip route add comment="FAILOVER CHECK ROUTE" distance=1 gateway=$GatewayISP5 dst-address=$pingTarget
	:local testRouteID [/ip route find comment="FAILOVER CHECK ROUTE"];
	/log warning "Pinging $pingTarget thru $GatewayISP5"

	:set PingResult [ping $pingTarget count=1];

 	:if ([/ip route get $testRouteID gateway-status] ~"unreachable"=true) do={

		:set PingFailCountISP5 ($PingFailCountISP5 + 1)
		/log error "Ping to $pingTarget thru $GatewayISP5 FAILED - unreachable gw"

 	} else={

		# remote ping failed, increase fail count isp +1
		:if ($PingResult = 0) do={
			:set PingFailCountISP5 ($PingFailCountISP5 + 1)
			/log error "Ping to $pingTarget thru $GatewayISP5 FAILED - no ping"
		};

		# remote ping passed, decrease fail count isp -1
		:if ($PingResult = 1) do={
			:if ($PingFailCountISP5 > 0) do={
				:set PingFailCountISP5 ($PingFailCountISP5 - 1)
			};
		};

 	}

	/ip route remove [find comment="FAILOVER CHECK ROUTE"]
};

#if ping to all 3 hosts failed, declare route as faulty
:if ($PingFailCountISP5 >= $FailTreshold) do={

	:if ([:typeof $OriginalRouteDistanceISP5] = "nothing") do={
		:set OriginalRouteDistanceISP5 [/ip route get $RouteISP5 distance]
	} 

	:if ($OriginalRouteDistanceISP5 + $DistanceIncrease != [/ip route get $RouteISP5 distance]) do={

		/log warning "ISP5 has a problem en route to $pingTarget - increasing distance of route."
		/ip route set $RouteISP5 distance=($OriginalRouteDistanceISP5 + $DistanceIncrease)
		/log warning "ISP5 Route distance increase finished."
	}
}

:if ($PingFailCountISP5 < $FailTreshold) do={

	/log warning "ISP5 is working ok"

	:if ([:typeof $OriginalRouteDistanceISP5] != "nothing") do={

		:if ([/ip route get $RouteISP5 distance] != $OriginalRouteDistanceISP5) do={

			/log warning "ISP5 can reach $pingTarget again - bringing back original distance of route."

			# if ISP5RecoverSpeed is "fast" - recovery of ISP will be FASTER - but there is potential danger of flapping in route ( if its unstable ISP )
			#
			# if ISP5RecoverSpeed is "slow" - recovery of ISP will be SLOWER - but there we are safe in case of unstable ISP
			:local ISP5RecoverSpeed "slow";

			:if ($ISP5RecoverSpeed = "slow") do={
				/log warning "ISP5 recovery is set to SLOW"
				:for pingRun from 0 to 2 do={
					:if ($PingFailCountISP5 > 0) do={
						:set PingFailCountISP5 ($PingFailCountISP5 - 1)
					}
				}
			} else={
				/log warning "ISP5 recovery is set to FAST"
				:set PingFailCountISP5 0
			}

			:if ($PingFailCountISP5 = 0) do={

				/ip route set $RouteISP5 distance=($OriginalRouteDistanceISP5)
				/log warning "ISP5 Route distance returned to original value."
			}
		}
	}
}
 
User avatar
JaZzSuperman
just joined
Posts: 12
Joined: Sun Oct 09, 2016 9:55 am
Location: South Africa
Contact:

Re: SCRIPT: Multi WAN failover with multiple host checks

Tue Mar 19, 2019 10:22 am

Hi

Do you run this script once?

Or do I need to set this script to run on a schedule? and if so, how often?
 
Frostbyte
Frequent Visitor
Frequent Visitor
Posts: 75
Joined: Mon Dec 25, 2017 1:42 am

Re: SCRIPT: Multi WAN failover with multiple host checks

Mon Apr 08, 2019 11:43 am

Not a true multi-WAN script, this will work for up to 5 gateways.
It's a nice attempt, but there's a lot of hard-coded stuff into it.. which will eventually make you reach the 30.000 character script limit.
There are also ways to tidy-up/compact the variables further and even completely get rid a few of them.
Plus checking against 3 different targets is a little bit of an overkill, imo.

You may check my signature for a suite of scripts that improves on all of that for you, plus reads the configuration from a file.

Who is online

Users browsing this forum: No registered users and 11 guests