Subject: Request for new Watchdog "Watch Address" timeout delay

It would be really really really nice and a much needed feature if the Watchdog "Watch Address" had an option to set a number of missed pings prior to rebooting.

One of my problems is I have many busy APs which have close to 1,000 clients connected (combined total). If I reboot anything in the upstream network between the APs to the actual "Watch Address", the APs reboot (because my network might take greater than 1 minute to reboot) and I dump several hundred clients on the APs - and also have hundreds of client CPEs also rebooting. When this happens it can take a long time for everything to return to normal. A Mikrotik AP does not like to see anything more than 50 client connections max trying to happen at the same time. My APs might see 80 to 200 connections trying to happen at the same time. As a result, it might take up to 10 minutes to recover from a mikrotik AP watchdog reboot.

So - what I ask is "can a new feature be added to the watchdog "Watch Address" section where the default timeout of 60 seconds to automatic reboot could be user selected to something greater like 5 minutes".

There are many things that can cause an upstream network outtage which lasts just over 60 seconds - this can be things like spanning tree, rebooting a core router or switch
. The above asked for software feature would sure solve many problems.


Tom Jones
a WISP up in North Idaho

Tom, /system watchdog does not have such feature and there is no plan to implement it.

But /tool netwatch+system script provides you with the same functionality.
/tool netwatch used to monitor specific address and it has timeouts customization.
/system script can be triggered, when host is not responding, script is very simple "/system reboot".

So you are saying the watchdog does not run at the kernel level and cannot reboot the hardware if its kernel panic'd?

So you are saying the watchdog does not run at the kernel level and cannot reboot the hardware if its kernel panic'd?

No - I am not saying that.

What I was trying to say is this:
-> using winbox, if you insert an IP address in (System -> Watchdog -> Watch Address -> (insert IP Address to ping).
(((This results in an automatic reboot in 60 seconds if the watchdog "Watch Address" does not answer pings)))

It would be a nice feature if this Watch Address had a setting to wait for 5 minutes instead of the default 60 seconds.

NOTE - If the "Watch Address" is an upstream IP address -and- if this address is not reachable for a full 60 seconds - then the board will reboot. Sometimes there may be pause somewhere in the upstream network lasting greater than 60 seconds which would auto-trigger a RB reboot. The pause could be something upstream being rebooted or a potential spanning tree flap. As to what caused the greater than 60 second delay is not the issue - the issue is that it would be nice if the 'Watch Address" had a setting to over-ride the default 60 seconds (only on this Watch Address - and not change any other timers).

Note - from a prior post on this subject - it appears there might be a work around - I have not tested it yet.

Tom Jones

Tom, I meant that question for Sergejs not you. Sorry... I figured the watchdog was to reboot a locked up box when scripts wouldnt work in the first place...

changeip and Tom,

I mean /system watchdog does not have features to control timeouts/packet-size/packet-number for watch-address.

I assume OP was about changing behavior of watch-address timeouts, and netwatch+script will do the same job.
watchdog has only such feature,
no-ping-delay (time; Default: 5m) Specifies how long after reboot not to test and ping watch-address. The default setting means that if watch-address is set and is not reachable, the router will reboot about every 6 minutes.

Re: "The default setting means that if watch-address is set and is not reachable, the router will reboot about every 6 minutes."

Yes - that is correct. The router will repeat rebooting every 6 minutes (5 minute delay and a 1 minute delay ping timeout)

However - when the router has been running - the router will reboot in 1 minute if the ping watchdog address does not respond to pings.

Thus the sequence of a sustained ping watchdog no-respond would be like this
1---router is running and has been running
2---watchdog ping address stops responding
3---router reboots in 60 seconds
4---router is rebooted - waits 5 minutes to start the watchdog
5---the watchdog watch ip address still does not respond and the router reboots 60 seconds later
6---go to step # 4
The sequence is 60 seconds - then 6 minutes - then 6 minutes - then 6 minutes - then 6 minutes (repeats untill the watchdog ip address responds to pings).

Tom Jones

Tom, /system watchdog does not have such feature and there is no plan to implement it.

But /tool netwatch+system script provides you with the same functionality.
/tool netwatch used to monitor specific address and it has timeouts customization.
/system script can be triggered, when host is not responding, script is very simple "/system reboot".

I started looking at the Toools-->Netwatch feature and I don't think this will work. (I am new to scripting)
There is a "Host" ip address
There is an "Interval" section
There is a "Timeout" section
And in the "Down" tab area it appears one or more lines of commands can be placed there.

What I do not see is a section for how-many-missed "Host" pings to trigger the "Down" action.

I suspect I might have to look at something different...
I am now looking at the System-->Scripts section (not Netwatch) for a way to reboot if pings to a remote IP address quit responding which would then force a reboot in 5 minutes.

Any ideas ?

I suspect I need to start looking in the scripting section of this forum...

Tom Jones

This script was originally used by me to check and fallover to a backup routing path but could be modified to simply reboot the router.

http://www.mikrotik-routeros.com/?p=180

It's all commented so even if you write your own hopefully this should give you a better understanding of how the scripting works.

This code waits for 5 dropped pings before it rolls over, you can again modify this process to suit.

OK - I finally got something working to do what I wanted.

First - what this does:

#1 It will ping test an IP address. This can be a remote IP address
#2 If the remote IP address does not answer any pings for about 5 minutes, then the router reboots.


To set it up - follow these steps using Winbox:

Note: In my example I am testing to remote IP address 192.0.2.254
Note: You will need to change anything I have below from 192.0.2.254 to your IP address you wish to test.
Note: Example if you will be testing 10.10.18.5 then change stuff like what is shown below
Note: "NetWatchBoot-192.0.2.254" would be changed to "NetWatchBoot-10.10.18.5"

A - (Optional) - remove the IP address from System-->Watchdog-->Watch Address

B - Go to System-->Scripts
Click on the red "+"
Give your script a Name (in my example I will use "NetWatchBoot-192.0.2.254"
C - In the Source: area paste in the entire line below:
:if ([/ping 192.0.2.254 interval=5 count=60] =0) do={
log info "my ping watchdog is down" ; /system reboot
}

D - Click the Apply button - then click the OK button

At this time - we have a script - it is not running or active yet.

E - Go to Tools-->Netwatch
Click on the red "+"
On the Host tab - type in 192.0.2.254
(Your IP address may be different - it is the same IP address you used from "B" above)
F - Click on the Down tab
Paste in the line below:
/system script run NetWatchBoot-192.0.2.254
G - Click the Apply button - then click the OK button

---- You are done -----

Your system will now auto reboot only after 5 (up to 6) minutes of the remote IP address not pinging.

If you want to disable the auto-reboot function, all you need to do is:
Tools-->Netwatch and disable the line that shows the IP address you are testing to.

-------------------------------------------------------------------------------
What is nice about this is - if you are testing to a server or something and that server takes longer
than 60 seconds to reboot, your mikrotik will not reboot. The remote IP address has to be down for 5 (to 6) minutes
to trigger an automatic reboot.
---------------------
and yes - i know I could of thrown this is the schedular instead of triggering my script from netwatch - but I like it more this way
FYI - This is my first use of Mikrotik script and Mikrotik netwatch

I hope this help others - if it works for you then please post a reply

Tom Jones
A WISP up in North Idaho

Topic is now moved to the scripts section of this forum

http://forum.mikrotik.com/viewtopic.php?f=9&t=56138


RouterOS Community Support [Scripting]
How to auto-reboot if remote IP down for 5 minutes

Tom Jones