Community discussions

MikroTik App
 
howdey57
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Wed Dec 31, 2014 2:36 pm

Log Monitoring Script

Sat Oct 20, 2018 2:45 pm

I wanted a way to monitor log files for certain entries. I have created a script based on the various log monitoring scripts I have found. I thought I might share this in case others wanted an alternative.

The only challenge is that, due to an issue with how ROS displays time in log files around midnight if the Router is not set to GMT, entries are odd around midnight. I am in the UK so don't mind setting my Router to GMT.

Any suggested improvements welcome.

Charles
# Written by Charles Hooper - charles@futurescope.co.uk - October 2018
# Partly based on https://wiki.mikrotik.com/wiki/Monitor_logs,_send_email_alert_/_run_script
########################################################################################################
# BEGIN SETUP
########################################################################################################
# Create Schedule of the same name and run as often as you want
:local scheduleName "LogMonitor"
########################################################################################################
:local emailAddress "charles@XXXXX"
:local startBuf [:toarray [/log find message~"l2tp-in-" || message~"<l2tp-" || message~"router rebooted" || message~"failed to get valid proposal."]]
########################################################################################################
# for checking time of each log entry
:local logEntryTime
# log message
:local logMessage
# final output
:local output
# END SETUP

########################################################################################################
# FUNCTION #############################################################################################
#  Reformat LOG DATE
#  depending on log date/time, the format may be different. 3 known formats
#  format of jan/01/2002 00:00:00 which shows up at unknown date/time. Using as default
# from https://wiki.mikrotik.com/wiki/Use_Functions_in_CMD_Script
########################################################################################################
:global FormatDateTime do={
   :local input1 [:tostr $1]
#  format of 00:00:00 which shows up on current day's logs
   :if ([:len $input1] = 8 ) do={
      :set input1 ([:pick [/system clock get date] 0 11]." ".$input1)
   } else={
#     format of jan/01 00:00:00 which shows up on previous day's logs
      :if ([:len $input1] = 15 ) do={
        :set input1 ([:pick $input1 0 6]."/".[:pick [/system clock get date] 7 11]." ".[:pick $input1 7 15])
      }
   }
   :return $input1
}
########################################################################################################
# FUNCTION #############################################################################################
# Convert jan/01/2002 00:00:00 to YYYYMMDDHHMMSS #######################################################
########################################################################################################
:global ConvertDateTime do={
   :local input2 [:tostr $1]
   :local months [:toarray "jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec"]
   :local mon 0
   :local convertedDateTime
   :for x from=0 to=([:len $months] - 1) do={
      :if ([:tostr [:pick $months $x]] = [:tostr [:pick $input2 0 3]]) do={
         :if ($x < 9) do={ :set mon ("0" . ($x + 1)) } else={ :set mon ($x + 1) } } }
   :set convertedDateTime ("2018" . $mon . [:pick $input2 4 6] . [:pick $input2 12 14] . [:pick $input2 15 17] . [:pick $input2 18 20])
   :set convertedDateTime [:tonum $convertedDateTime ]
   :return $convertedDateTime 
}

########################################################################################################
# END OF FUNCTIONS
########################################################################################################

########################################################################################################
# START AREA
########################################################################################################

# warn if schedule does not exist
:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
  /log warning "[LOGMON] ERROR: Schedule does not exist. Create schedule and edit script to match name"
}

# get last Run time From Schedule in YYYYMMDDHHMMSS format
:local lastRunTime [/system scheduler get [find name="$scheduleName"] comment]
:set lastRunTime [:tonum $lastRunTime ]

########################################################################################################
# MAIN LOOP
########################################################################################################

# loop through all log entries that have been found
:foreach i in=$startBuf do={
   :set logMessage [/log get $i message]
   :set logEntryTime [ /log get $i time ]
   :local FormattedDate [$FormatDateTime $logEntryTime]
   :local DateToTest [$ConvertDateTime $FormattedDate]
#  Compare datetimes and add to output if newer
   :if ( $lastRunTime < $DateToTest ) do={
      :set output ($output.$FormattedDate." ".$logMessage."\r\n")
   }
}

########################################################################################################
# END AREA
########################################################################################################

/log info "[LogMonitorScript] Run"

:if ([:len $output] != 0 ) do={
#  Update the Schedule entry comment with last run time
   :local NewDateTime ([/system clock get date]." ".[/system clock get time])
   :set NewDateTime [$ConvertDateTime $NewDateTime]
   /system scheduler set [find name="$scheduleName"] comment=$NewDateTime 
#  Convert the LastRunTime to something readable and send email
   :set lastRunTime  ([:pick $lastRunTime 0 4]."-".[:pick $lastRunTime 4 6]."-".[:pick $lastRunTime 6 8]." ".[:pick $lastRunTime 8 10].":".[:pick $lastRunTime 10 12].":".[:pick $lastRunTime 12 14])
   /tool e-mail send to="$emailAddress" subject="$[/system identity get name] MikroTik alert since $lastRunTime" body="$output"
   /log info "[LOGMON] New logs found, send email"
}
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3291
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Log Monitoring Script

Sat Oct 20, 2018 5:47 pm

Instead if sending log info direct to email and study it there, I do use Splunk (or you can use Graylog or other) as a Syslog monitor.
There can you get a much better insight of your logs. Se my Splunk - Mikrotik project here: viewtopic.php?f=2&t=137338
 
chukinsun
just joined
Posts: 3
Joined: Wed Aug 09, 2017 11:52 am

Re: Log Monitoring Script

Tue Jul 09, 2019 8:43 am

Hi Charles Hooper,
the script is very good, the original script has timezone bug. Your script seems advanced.
May i know any chance update the script to filter some words inside result ?
Say
:local removeThese {"telnet";"whatever string you want"}
Because i have some API call. it will trigger many no use msg when using keyword "login"

Sunny
 
bandini981
newbie
Posts: 25
Joined: Fri Jul 19, 2019 11:46 am

Re: Log Monitoring Script

Thu Oct 14, 2021 9:00 am

I wanted a way to monitor log files for certain entries. I have created a script based on the various log monitoring scripts I have found. I thought I might share this in case others wanted an alternative.

The only challenge is that, due to an issue with how ROS displays time in log files around midnight if the Router is not set to GMT, entries are odd around midnight. I am in the UK so don't mind setting my Router to GMT.

Any suggested improvements welcome.

Charles
# Written by Charles Hooper - charles@futurescope.co.uk - October 2018
# Partly based on https://wiki.mikrotik.com/wiki/Monitor_logs,_send_email_alert_/_run_script
########################################################################################################
# BEGIN SETUP
########################################################################################################
# Create Schedule of the same name and run as often as you want
:local scheduleName "LogMonitor"
########################################################################################################
:local emailAddress "charles@XXXXX"
:local startBuf [:toarray [/log find message~"l2tp-in-" || message~"<l2tp-" || message~"router rebooted" || message~"failed to get valid proposal."]]
########################################################################################################
# for checking time of each log entry
:local logEntryTime
# log message
:local logMessage
# final output
:local output
# END SETUP

########################################################################################################
# FUNCTION #############################################################################################
#  Reformat LOG DATE
#  depending on log date/time, the format may be different. 3 known formats
#  format of jan/01/2002 00:00:00 which shows up at unknown date/time. Using as default
# from https://wiki.mikrotik.com/wiki/Use_Functions_in_CMD_Script
########################################################################################################
:global FormatDateTime do={
   :local input1 [:tostr $1]
#  format of 00:00:00 which shows up on current day's logs
   :if ([:len $input1] = 8 ) do={
      :set input1 ([:pick [/system clock get date] 0 11]." ".$input1)
   } else={
#     format of jan/01 00:00:00 which shows up on previous day's logs
      :if ([:len $input1] = 15 ) do={
        :set input1 ([:pick $input1 0 6]."/".[:pick [/system clock get date] 7 11]." ".[:pick $input1 7 15])
      }
   }
   :return $input1
}
########################################################################################################
# FUNCTION #############################################################################################
# Convert jan/01/2002 00:00:00 to YYYYMMDDHHMMSS #######################################################
########################################################################################################
:global ConvertDateTime do={
   :local input2 [:tostr $1]
   :local months [:toarray "jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec"]
   :local mon 0
   :local convertedDateTime
   :for x from=0 to=([:len $months] - 1) do={
      :if ([:tostr [:pick $months $x]] = [:tostr [:pick $input2 0 3]]) do={
         :if ($x < 9) do={ :set mon ("0" . ($x + 1)) } else={ :set mon ($x + 1) } } }
   :set convertedDateTime ("2018" . $mon . [:pick $input2 4 6] . [:pick $input2 12 14] . [:pick $input2 15 17] . [:pick $input2 18 20])
   :set convertedDateTime [:tonum $convertedDateTime ]
   :return $convertedDateTime 
}

########################################################################################################
# END OF FUNCTIONS
########################################################################################################

########################################################################################################
# START AREA
########################################################################################################

# warn if schedule does not exist
:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
  /log warning "[LOGMON] ERROR: Schedule does not exist. Create schedule and edit script to match name"
}

# get last Run time From Schedule in YYYYMMDDHHMMSS format
:local lastRunTime [/system scheduler get [find name="$scheduleName"] comment]
:set lastRunTime [:tonum $lastRunTime ]

########################################################################################################
# MAIN LOOP
########################################################################################################

# loop through all log entries that have been found
:foreach i in=$startBuf do={
   :set logMessage [/log get $i message]
   :set logEntryTime [ /log get $i time ]
   :local FormattedDate [$FormatDateTime $logEntryTime]
   :local DateToTest [$ConvertDateTime $FormattedDate]
#  Compare datetimes and add to output if newer
   :if ( $lastRunTime < $DateToTest ) do={
      :set output ($output.$FormattedDate." ".$logMessage."\r\n")
   }
}

########################################################################################################
# END AREA
########################################################################################################

/log info "[LogMonitorScript] Run"

:if ([:len $output] != 0 ) do={
#  Update the Schedule entry comment with last run time
   :local NewDateTime ([/system clock get date]." ".[/system clock get time])
   :set NewDateTime [$ConvertDateTime $NewDateTime]
   /system scheduler set [find name="$scheduleName"] comment=$NewDateTime 
#  Convert the LastRunTime to something readable and send email
   :set lastRunTime  ([:pick $lastRunTime 0 4]."-".[:pick $lastRunTime 4 6]."-".[:pick $lastRunTime 6 8]." ".[:pick $lastRunTime 8 10].":".[:pick $lastRunTime 10 12].":".[:pick $lastRunTime 12 14])
   /tool e-mail send to="$emailAddress" subject="$[/system identity get name] MikroTik alert since $lastRunTime" body="$output"
   /log info "[LOGMON] New logs found, send email"
}





I improved the script as follows:
  • replaced the static "2018" in the ConvertDateTime with the year contained in "input2" variable;
  • added a check for "dec/31 -> jan/01" in FormatDateTime: if log date is dec/31 year is "year - 1" (past year);
  • added a workaround for the GMT log time bug: in the main loop I also test if the "DateToTest" is greater than present date-time (is in the future, so it's wrong);
  • other customization specific for my purpose
Going to test it for a couple of days ad back hear with a report :)

Best

Who is online

Users browsing this forum: Bing [Bot], marcelofares and 34 guests