Log Monitoring Script

RouterOS Scripting
1

I wanted a way to monitor log files for certain entries. I have created a script based on the various log monitoring scripts I have found. I thought I might share this in case others wanted an alternative.

The only challenge is that, due to an issue with how ROS displays time in log files around midnight if the Router is not set to GMT, entries are odd around midnight. I am in the UK so don't mind setting my Router to GMT.

Any suggested improvements welcome.

Charles

# Written by Charles Hooper - October 2018
# Partly based on https://wiki.mikrotik.com/wiki/Monitor_logs,_send_email_alert_/_run_script
########################################################################################################
# BEGIN SETUP
########################################################################################################
# Create Schedule of the same name and run as often as you want
:local scheduleName "LogMonitor"
########################################################################################################
:local emailAddress "charles@XXXXX"
:local startBuf [:toarray [/log find message~"l2tp-in-" || message~"<l2tp-" || message~"router rebooted" || message~"failed to get valid proposal."]]
########################################################################################################
# for checking time of each log entry
:local logEntryTime
# log message
:local logMessage
# final output
:local output
# END SETUP

########################################################################################################
# FUNCTION #############################################################################################
#  Reformat LOG DATE
#  depending on log date/time, the format may be different. 3 known formats
#  format of jan/01/2002 00:00:00 which shows up at unknown date/time. Using as default
# from https://wiki.mikrotik.com/wiki/Use_Functions_in_CMD_Script
########################################################################################################
:global FormatDateTime do={
   :local input1 [:tostr $1]
#  format of 00:00:00 which shows up on current day's logs
   :if ([:len $input1] = 8 ) do={
      :set input1 ([:pick [/system clock get date] 0 11]." ".$input1)
   } else={
#     format of jan/01 00:00:00 which shows up on previous day's logs
      :if ([:len $input1] = 15 ) do={
        :set input1 ([:pick $input1 0 6]."/".[:pick [/system clock get date] 7 11]." ".[:pick $input1 7 15])
      }
   }
   :return $input1
}
########################################################################################################
# FUNCTION #############################################################################################
# Convert jan/01/2002 00:00:00 to YYYYMMDDHHMMSS #######################################################
########################################################################################################
:global ConvertDateTime do={
   :local input2 [:tostr $1]
   :local months [:toarray "jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec"]
   :local mon 0
   :local convertedDateTime
   :for x from=0 to=([:len $months] - 1) do={
      :if ([:tostr [:pick $months $x]] = [:tostr [:pick $input2 0 3]]) do={
         :if ($x < 9) do={ :set mon ("0" . ($x + 1)) } else={ :set mon ($x + 1) } } }
   :set convertedDateTime ("2018" . $mon . [:pick $input2 4 6] . [:pick $input2 12 14] . [:pick $input2 15 17] . [:pick $input2 18 20])
   :set convertedDateTime [:tonum $convertedDateTime ]
   :return $convertedDateTime 
}

########################################################################################################
# END OF FUNCTIONS
########################################################################################################

########################################################################################################
# START AREA
########################################################################################################

# warn if schedule does not exist
:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
  /log warning "[LOGMON] ERROR: Schedule does not exist. Create schedule and edit script to match name"
}

# get last Run time From Schedule in YYYYMMDDHHMMSS format
:local lastRunTime [/system scheduler get [find name="$scheduleName"] comment]
:set lastRunTime [:tonum $lastRunTime ]

########################################################################################################
# MAIN LOOP
########################################################################################################

# loop through all log entries that have been found
:foreach i in=$startBuf do={
   :set logMessage [/log get $i message]
   :set logEntryTime [ /log get $i time ]
   :local FormattedDate [$FormatDateTime $logEntryTime]
   :local DateToTest [$ConvertDateTime $FormattedDate]
#  Compare datetimes and add to output if newer
   :if ( $lastRunTime < $DateToTest ) do={
      :set output ($output.$FormattedDate." ".$logMessage."\r\n")
   }
}

########################################################################################################
# END AREA
########################################################################################################

/log info "[LogMonitorScript] Run"

:if ([:len $output] != 0 ) do={
#  Update the Schedule entry comment with last run time
   :local NewDateTime ([/system clock get date]." ".[/system clock get time])
   :set NewDateTime [$ConvertDateTime $NewDateTime]
   /system scheduler set [find name="$scheduleName"] comment=$NewDateTime 
#  Convert the LastRunTime to something readable and send email
   :set lastRunTime  ([:pick $lastRunTime 0 4]."-".[:pick $lastRunTime 4 6]."-".[:pick $lastRunTime 6 8]." ".[:pick $lastRunTime 8 10].":".[:pick $lastRunTime 10 12].":".[:pick $lastRunTime 12 14])
   /tool e-mail send to="$emailAddress" subject="$[/system identity get name] MikroTik alert since $lastRunTime" body="$output"
   /log info "[LOGMON] New logs found, send email"
}
2

Instead if sending log info direct to email and study it there, I do use Splunk (or you can use Graylog or other) as a Syslog monitor.
There can you get a much better insight of your logs. Se my Splunk - Mikrotik project here: http://forum.mikrotik.com/t/tool-using-splunk-to-analyse-mikrotik-logs-3-3-graphing-everything/121810/1

3

Hi Charles Hooper,
the script is very good, the original script has timezone bug. Your script seems advanced.
May i know any chance update the script to filter some words inside result ?
Say
:local removeThese {“telnet”;“whatever string you want”}
Because i have some API call. it will trigger many no use msg when using keyword “login”

Sunny

4





I improved the script as follows:

  • replaced the static “2018” in the ConvertDateTime with the year contained in “input2” variable;
  • added a check for “dec/31 → jan/01” in FormatDateTime: if log date is dec/31 year is “year - 1” (past year);
  • added a workaround for the GMT log time bug: in the main loop I also test if the “DateToTest” is greater than present date-time (is in the future, so it’s wrong);
  • other customization specific for my purpose

Going to test it for a couple of days ad back hear with a report :slight_smile:

Best

5

I’ve just been asked whether I have updated my script for ROS7 and, yes, I did that a few weeks ago. It is much simpler now ROS uses a proper date format.

# BEGIN SETUP;

############################################################;

# Create Schedule of the same name and run as often as you want
:local scheduleName "LogMonitor2";
:local scriptName "LogMonitor2";
#:local lastRunTimeDisplay;
:local thisRunTime ([/system clock get date]." ".[/system clock get time]);
:local resultMsg "Nothing Found"
############################################################
:local emailAddress "blah@blah.com";
##############################################################
# for checking time of each log entry
:local logEntryTime
:local DateToTest
# log message
:local logMessage
# final output
:local output ""
# END SETUP

###################################################
# START AREA
#############################################
# warn if schedule does not exist
:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
  /log warning "[$scriptName] ERROR: Schedule does not exist. Create schedule and edit script to match name"
}

# get last Run time From Schedule 
:local lastRunTime [/system scheduler get [find name="$scheduleName"] comment]

#/log info "[$scriptName] thisRunTime = $thisRunTime, lastRunTime = $lastRunTime"
####################################################
# MAIN LOOP
####################################################

:foreach entry in=[/log find] do={
    :local t [/log get $entry time]
    
    :if ($t > $lastRunTime) do={
         :local msg [/log get $entry message]
         #/log info "msg = $msg"
         #/log info "msgTime = $t"
        :if (($msg ~ "blah") || \
             ($msg~"router rebooted") || \
             ($msg~"ether1 Internet link down") \
             ) do={
              :set output ($output." $t $msg\r\n")
        }
    }
}
##############################################################
# END AREA
###############################################################
#  Update the Schedule entry comment with last run time
/system scheduler set [find name="$scheduleName"] comment=$thisRunTime

:if ([:len $output] != 0 ) do={
   # Send Email
   /tool e-mail send to="$emailAddress" subject="$[/system identity get name] MikroTik Log alert since $lastRunTime" body="$output"
   #/log info "[$scriptName] Output\r\n$output"
   :set resultMsg "New logs found, send email"
   /log info "[$scriptName] $resultMsg - This Run >> $thisRunTime, Last Run >> $lastRunTime"
}
#/log info "[$scriptName] $resultMsg"
#/log info "[$scriptName] $resultMsg - This Run >> $thisRunTime, Last Run >> $lastRunTime"
#################################################################
6

To avoid repeated writes to the configuration,
using a global variable (which simply stores the last .ID and consider only those of the last .ID) is better.

7

But would that survive a reboot?

8

Do the logs survive reboot?

9

Er, no

10

So... :upside_down_face:

:global lastLog
:if ([:typeof $lastLog] != "num") do={:set lastLog 0}

{
:local id2num do={:return [:tonum "0x$[:pick $1 1 [:len $1]]"]}

/log
:foreach item in=[find where ([$id2num $".id"] > $lastLog)] do={
    :set lastLog [$id2num $item]

    [...]

}

Extracted from: