Community discussions

MikroTik App
 
abbio90
Member Candidate
Member Candidate
Topic Author
Posts: 163
Joined: Fri Aug 27, 2021 9:16 pm
Contact:

Log Filter

Wed May 24, 2023 1:13 am

Good evening everyone, I took this script from Github and I adapted it to my needs.
It works perfectly, but every day at 00:00 and 02:00, I get the latest notification of the day.

For example, if the latest notification is sent to me at 20:32, every day at 00:00 and at 02:00 it is sent to me again, I don't understand in which part of the script there are problems.


#----------------------------------------
#inserire qui il botID
:local bot "123456789:abcdefghilmnopqrstuvz"
#----------------------------------------
#inserire qui il chatID1 (es: "123456789")
:local ChatID1 "123456789"
#----------------------------------------
#inserire si o no se si vuole utilizzare il chatID2 (es. "si")
:local useChatID2 "si"
#----------------------------------------
#inserire qui il chatID2 (es: "123456789")
:local ChatID2 "123456789"
#----------------------------------------
#inserire si o no se si vuole utilizzare il chatID3 (es: "no")
:local useChatID3 "no"
#----------------------------------------
#inserire qui il chatID3 (es: "123456789")
:local ChatID3 "123456789"
#----------------------------------------
#inserire nome azienda
:local azienda "VidaSPA"
#----------------------------------------
#PAROLE CHIAVE DA CERCARE
#inserire qui la parola chiave 1 
:local key1 "failure"
#----------------------------------------
#inserire qui la parola chiave 2 
:local key2 "loop"
#----------------------------------------
#inserire qui la parola chiave 3 
:local key3 "excessive"
#----------------------------------------
#inserire qui la parola chiave 4 
:local key4 "Netwatch_"
#----------------------------------------
#inserire qui la parola chiave 5 
:local key5 "pppoe-out_EOLO: connected"
#----------------------------------------
#inserire qui la parola chiave 6 
:local key6 "pppoe-out_EOLO: disconnected"
#----------------------------------------
#inserire qui la parola chiave 7 
:local key7 "STATO PEER EDOK="
#----------------------------------------
#inserire qui la parola chiave 8 
:local key8 "DuckDNS: Current IP"
#----------------------------------------
#inserire qui la parola chiave 9 
:local key9 "non usato"
#----------------------------------------
#inserire qui ogni qunto tempo eseguire lo script
:local myRunTime 60s
#----------------------------------------
#inserire qui uno slogan pubblicitario
:local slogan "Sistema di alert by foisfabio.it"



#----------NO MODIFY THIS SECTION----------

#inserire il nome identificativo della macchina
:local myRouterName ([/system identity get name]) 
#----------------------------------------

#----------------------------------------
:local myserver "([/system identity get name])"

:local startBuf [:toarray [/log find message~$key1 || message~$key2 || message~$key3 || message~$key4 || message~$key5 || message~$key6 || message~$key7 || message~$key8 || message~$key9 ]]
#----------------------------------------
:local scheduleName "LogFilter2.0"
:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
  /log error "[LogFilter2.0] Alert : lo Scheduler non esiste. Creo lo scheduler"

  /system scheduler add name=$scheduleName interval=$myRunTime start-date=Jan/01/1970 start-time=startup on-event="system script run LogFilter2"

  /log warning "[LogFilter2.0] Alert : Scheduler creato ."
}
#----------------------------------------

# get last time
:local lastTime [/system scheduler get [find name="$scheduleName"] comment]
# for checking time of each log entry
:local currentTime
# log message
:local message
 
# final output
:local output
:local myMac
:local myHost
:local ptr1
:local ptr2
:local emoji "\E2\9C\85"

:local keepOutput false
# if lastTime is empty, set keepOutput to true DISATIVVATO PER PROVA (disativando non funge)
:if ([:len $lastTime] = 0) do={
:set keepOutput true
}


:local counter 0
# loop through all log entries that have been found
:foreach i in=$startBuf do={
 
# loop through all removeThese array items
  :local keepLog true
  :foreach j in=$removeThese do={
#   if this log entry contains any of them, it will be ignored
    :if ([/log get $i message] ~ "$j") do={
      :set keepLog false
    }
  }
  :if ($keepLog = true) do={
   
   :set message [/log get $i message]

#   LOG DATE
#   depending on log date/time, the format may be different. 3 known formats
#   format of jan/01/2002 00:00:00 which shows up at unknown date/time. Using as default
    :set currentTime [ /log get $i time ]
#   format of 00:00:00 which shows up on current day's logs
   :if ([:len $currentTime] = 8 ) do={
     :set currentTime ([:pick [/system clock get date] 0 11]." ".$currentTime)
    } else={
#     format of jan/01 00:00:00 which shows up on previous day's logs
     :if ([:len $currentTime] = 15 ) do={
        :set currentTime ([:pick $currentTime 0 6]."/".[:pick [/system clock get date] 7 11]." ".[:pick $currentTime 7 15])
      }
   }


#   if keepOutput is true, add this log entry to output
   :if ($keepOutput = true) do={
   
      :if ( [:find $message " assigned"] > 0 ) do={
	    :set ptr1 ([:find $message "to"] +3)
	    :set ptr2 [:len $message]
	    :set myMac [:pick $message $ptr1 $ptr2 ]
	    :set myHost [/ip dhcp-server lease get [find where mac-address=$myMac] host-name]
	    :set message ($message." name=".$myHost)
     }
	

      :set output ($output."Data e Ora: ".$currentTime. " \E2\9A\A0 \E2\9A\A0 \E2\9A\A0 \E2\9A\A0--> ".$message."\r\n")
 
  }

    :if ($currentTime = $lastTime) do={
     :set keepOutput true
     :set output ""
   }
  }
  :if ($counter = ([:len $startBuf]-1)) do={
   :if ($keepOutput = false) do={    
     :if ([:len $message] > 0) do={
        :set output ($output."Data e Ora: ".$currentTime. " \E2\9A\A0 \E2\9A\A0 \E2\9A\A0 \E2\9A\A0--> ".$message."\r\n")
      }
    }
  }
  :set counter ($counter + 1)
}


if ([:len $output] > 0) do={

	/system scheduler set [find name="$scheduleName"] comment=$currentTime

	:set output ("Azienda: ".$azienda."\r\n"."Dispositivo: ".$myRouterName."\r\n".$output."\r\n".$slogan)
	 
	:do { 
		:local requestUrl "https://api.telegram.org/bot$bot/sendMessage";
		:local httpData "{\"chat_id\": \"$ChatID1\", \"text\": \"$output\"}";
		/tool fetch url=$requestUrl http-data=$httpData http-header-field=content-type:application/json http-method=post  keep-result=no
	} on-error={
		# log errori http post (disabilitato)
		#:log info "finished with error";
	}
:delay 3s
:if ($useChatID2="si") do={
:local requestUrl "https://api.telegram.org/bot$bot/sendMessage";
		:local httpData "{\"chat_id\": \"$ChatID2\", \"text\": \"$output\"}";
		/tool fetch url=$requestUrl http-data=$httpData http-header-field=content-type:application/json http-method=post  keep-result=no
	} else={
		# log errori http post (disabilitato)
		:log warning "[LogFilter2.0] ChatID2 non utilizzato";
	}
:delay 3s
:if ($useChatID3="si") do={
:local requestUrl "https://api.telegram.org/bot$bot/sendMessage";
		:local httpData "{\"chat_id\": \"$ChatID3\", \"text\": \"$output\"}";
		/tool fetch url=$requestUrl http-data=$httpData http-header-field=content-type:application/json http-method=post  keep-result=no
	} else={
		# log errori http post (disabilitato)
		#:log warning "[LogFilter2.0] ChatID3 non utilizzato";
	}
}
}
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Wed May 24, 2023 2:00 pm

Bel posto il Lago di Garda.
Il lupo perde il pelo ma non il vizio, ancora con questo metodo dei commenit che usa la NAND invece di usare le variabili globali....
viewtopic.php?p=1003884#p1003398
[…]
:local lastTime [/system scheduler get [find name="$scheduleName"] comment]
[…]
	/system scheduler set [find name="$scheduleName"] comment=$currentTime
[…]
Perhaps it could depend on the fact that at 2 am, Italian time, the format of the logs changes from xx:xx:xx to xxx/xx/xxxx xx:xx:xx and since the comment only says xx:xx:xx the script send again the logs another time?....
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Wed May 24, 2023 2:37 pm

It works perfectly, but every day at 00:00 and 02:00, I get the latest notification of the day.
My script is the same base as yours, but adapted to my needs,
viewtopic.php?t=151953#p1003030

I have the same problem you mentioned: It repeats events of the previous day, same time but with day and month of the current day.



..
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Wed May 24, 2023 3:48 pm

Perhaps it could depend on the fact that at 2 am, Italian time, the format of the logs changes from xx:xx:xx to xxx/xx/xxxx xx:xx:xx and since the comment only says xx:xx:xx the script send again the logs another time?....
???

When I have time I check that, but probably timezone is involved... 2H is actual timezione offset on italy & co.
 
abbio90
Member Candidate
Member Candidate
Topic Author
Posts: 163
Joined: Fri Aug 27, 2021 9:16 pm
Contact:

Re: Log Filter

Wed May 24, 2023 4:03 pm

ipsec script was copied from this one i found on github. so by copying I also copied the errors. I'll try to change this. more than anything I don't understand how to solve the continuous sending of messages at 00:00 and 02:00
 
abbio90
Member Candidate
Member Candidate
Topic Author
Posts: 163
Joined: Fri Aug 27, 2021 9:16 pm
Contact:

Re: Log Filter

Wed May 24, 2023 4:04 pm

could this be solved by uncommenting it and using the global variable?
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Wed May 24, 2023 5:24 pm

ipsec script was copied from this one i found on github. so by copying I also copied the errors. I'll try to change this. more than anything I don't understand how to solve the continuous sending of messages at 00:00 and 02:00
My time zone is the same as Italy (from Spain) and the times previously handled by the script are modified thanks to the "anydate2isodate" function (you have it in my previous post) courtesy of @rextended which converts "any" time to full format.

hh:mm:ss -> YYYY-MM-DD hh:mm:ss
MMM/DD hh:mm:ss -> YYYY-MM-DD hh:mm:ss
MMM/DD/YYYY hh:mm:ss -> YYYY-MM-DD hh:mm:ss
...
etc.

I also don't understand the problem of repeating events before 00:00 and 02:00, the rest of the day works OK.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Wed May 24, 2023 5:27 pm

Hello again,

I have tested this code (original script) without any modification (only Telegram token and chat_id) and made a simulation by setting the router time to 23:58:00. When reaching 00:00:00:00 no event from the previous day has been repeated, it has worked as expected.

@abbio90, try with this code to see if it works well for you.
:local myserver ("\E2\84\B9"." "."MikroTik"." ".[/system identity get name])
:local scheduleName "LogFilter"
:local bot          "XXXXXXXXXXXXXXXX:XXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXX"
:local ChatID       "XXXXXXXXX"
:local startBuf     [:toarray [/log find message~"login failure" || message~"logged in" || message~"rebooted" || message~"loop" || message~"fcs" || message~"excessive" || message~"system"]]
:local removeThese  [:toarray ""]

:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
    /log warning "[LogFilter] Alert : Schedule does not exist. Creating schedule ...."
    /system scheduler add name=$scheduleName interval=60s start-date=may/24/2023 start-time=12:00:00 on-event=LogFilter
    /log warning "[LogFilter] Alert : Schedule created ."
}

:local lastTime     [/system scheduler get [find name="$scheduleName"] comment]
:local currentTime  ""
:local message      ""
:local output       ""

:local keepOutput false
:if ([:len $lastTime] = 0) do={:set keepOutput true}

:local counter 0

:foreach i in=$startBuf do={

    :local keepLog true
    :foreach j in=$removeThese do={
        :if ([/log get $i message] ~ "$j") do={:set keepLog false}
    }

    :if ($keepLog = true) do={
        :set message [/log get $i message]
        :set currentTime [ /log get $i time ]

        :if ([:len $currentTime] = 8 ) do={
            :set currentTime ([:pick [/system clock get date] 0 11]." ".$currentTime)
        } else={
            :if ([:len $currentTime] = 15 ) do={
                :set currentTime ([:pick $currentTime 0 6]."/".[:pick [/system clock get date] 7 11]." ".[:pick $currentTime 7 15])
            }
        }

        :if ($keepOutput = true) do={:set output ($output.$currentTime." ".$message." %0A%0A ")}
        :if ($currentTime = $lastTime) do={:set keepOutput true; :set output ""}
    }                                                                                                                            

    :if ($counter = ([:len $startBuf])-1) do={
        :if ($keepOutput = false) do={
            :if ([:len $message] > 0) do={:set output ($output.$currentTime." ".$message)}
        }
    }

    :set counter ($counter + 1)
}

if ([:len $output] > 0) do={
    /system scheduler set [find name="$scheduleName"] comment=$currentTime
    /tool fetch url="https://api.telegram.org/bot$bot/sendMessage\?chat_id=$ChatID&text=$myserver%0A%0A$output" keep-result=no
}
Maybe @rextended is right and the problem is going to be the dates, I can't find another explanation.

BR.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Thu May 25, 2023 10:22 am

Well, the test performed tonight with the original script has been failed, some events of the previous day have been repeated tonight at 00:00 and 02:00.

I think the problem then cannot be the date. I don't know where it fails.
 
abbio90
Member Candidate
Member Candidate
Topic Author
Posts: 163
Joined: Fri Aug 27, 2021 9:16 pm
Contact:

Re: Log Filter

Thu May 25, 2023 5:39 pm

yes, I uncommented it and save to a global variable, but the result is the same. In your opinion, changing the rules of the date and updated to 7.10 with the new ISO format of the date, could it be solved?
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Thu May 25, 2023 6:06 pm

yes, I uncommented it and save to a global variable, but the result is the same. In your opinion, changing the rules of the date and updated to 7.10 with the new ISO format of the date, could it be solved?
On a Miktotik with v7.9.1 I tried tonight to leave the date format of the original script: may/25/2023 00:00:00:00 and the events repeated again at 00:00 and
02:00. Next night I will try on a Mikrotik, original script with v7.10beta+ to see the results.

I'm no expert, but I think you can only make progress by trying different options.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Fri May 26, 2023 1:02 am

Nothing, the same problem persists.
:(
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Fri May 26, 2023 1:09 am

try to set the device to GMT +00:00...... (and disable auto-detect)
 
abbio90
Member Candidate
Member Candidate
Topic Author
Posts: 163
Joined: Fri Aug 27, 2021 9:16 pm
Contact:

Re: Log Filter

Fri May 26, 2023 1:14 am

Are you saying it could be a time zone related system? What if you try to insert a row where at 00:00 and 02:00 set the keepOutput variable to false?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Fri May 26, 2023 10:46 am

From post #4:
[…] probably timezone is involved... 2H is actual timezione offset on italy & co.
I still do not have time to test it.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Fri May 26, 2023 8:43 pm

try to set the device to GMT +00:00...... (and disable auto-detect)
Ok, I'll do the test tonight.

..
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Fri May 26, 2023 8:45 pm

try to set the device to GMT +00:00...... (and disable auto-detect)
Ok, I'll do the test tonight.

..
Thanks
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Sat May 27, 2023 11:16 am

try to set the device to GMT +00:00...... (and disable auto-detect)
Good morning, I have already performed the tests with the following conditions:

a) Configured System > Clock
- Time: Local time
- Date: Current date
- Time Zone Autodetected: Unchecked
- Time Zone Name: GMT+0

b) System > NTP Server: Unchecked

c) System > NTP Client: Unchecked

d) IP > Cloud
- DDNS Enabled: Checked
- Update Time: Checked

Result: Some LogFilter events at 00:00 and 02:00 were not repeated.

The recommendation of @rextended is good (thanks) but I don't know if the Time Zone configuration could be done in the script itself instead of configuring it in System > Clock so as not to affect the rest of Mikrotik services.

BR.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sat May 27, 2023 12:16 pm

for sure is possible avoid duplication using log index instead of last date.... the original script is not mine and I do not like it... ;)

select * from log where id > last id ;)
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Sat May 27, 2023 12:32 pm

Hello @diamuxin, I was reading your previous post and noticed that the script is looking for the "last log" for the date that was written on the scheduled comment.
Druvis from MikroTik wrote a script that may help you to fix the problem and you could try to your integrate: https://github.com/Druvis-Timma/Mikroti ... coding.rsc
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sat May 27, 2023 12:52 pm

Hello @diamuxin, I was reading your previous post and noticed that the script is looking for the "last log" for the date that was written on the scheduled comment.
Druvis from MikroTik wrote a script that may help you to fix the problem and you could try to your integrate: https://github.com/Druvis-Timma/Mikroti ... coding.rsc
Completely useless in the context of this topic.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sat May 27, 2023 1:07 pm

one telegram message for each new log entry code

:global anydate2isodate do={
    :local dtime [:tostr $1]
    /system clock
    :local cyear [get date] ; :if ($cyear ~ "....-..-..") do={:set cyear [:pick $cyear 0 4]} else={:set cyear [:pick $cyear 7 11]}
    :if (([:len $dtime] = 10) or ([:len $dtime] = 11)) do={:set dtime "$dtime 00:00:00"}
    :if ([:len $dtime] = 15) do={:set dtime "$[:pick $dtime 0 6]/$cyear $[:pick $dtime 7 15]"}
    :if ([:len $dtime] = 14) do={:set dtime "$cyear-$[:pick $dtime 0 5] $[:pick $dtime 6 14]"}
    :if ([:len $dtime] =  8) do={:set dtime "$[get date] $dtime"}
    :if ([:tostr $1] = "") do={:set dtime ("$[get date] $[get time]")}
    :local vdoff [:toarray "0,4,5,7,8,10,11,19"]
    :local MM    [:pick $dtime ($vdoff->2) ($vdoff->3)]
    :if ($dtime ~ ".../../....") do={
        :set vdoff [:toarray "7,11,1,3,4,6,12,20"]
        :set MM    ([:find "xxanebarprayunulugepctovecANEBARPRAYUNULUGEPCTOVEC" [:pick $dtime ($vdoff->2) ($vdoff->3)] -1] / 2)
        :if ($MM>12) do={:set MM ($MM - 12)} ; :if ($MM<10) do={:set MM "0$MM"}
    }
    :return "$[:pick $dtime ($vdoff->0) ($vdoff->1)]-$MM-$[:pick $dtime ($vdoff->4) ($vdoff->5)] $[:pick $dtime ($vdoff->6) ($vdoff->7)]"
}

:global UTF8toURLencode do={
    :local ascii "\00\01\02\03\04\05\06\07\08\09\0A\0B\0C\0D\0E\0F\
                  \10\11\12\13\14\15\16\17\18\19\1A\1B\1C\1D\1E\1F\
                  \20\21\22\23\24\25\26\27\28\29\2A\2B\2C\2D\2E\2F\
                  \30\31\32\33\34\35\36\37\38\39\3A\3B\3C\3D\3E\3F\
                  \40\41\42\43\44\45\46\47\48\49\4A\4B\4C\4D\4E\4F\
                  \50\51\52\53\54\55\56\57\58\59\5A\5B\5C\5D\5E\5F\
                  \60\61\62\63\64\65\66\67\68\69\6A\6B\6C\6D\6E\6F\
                  \70\71\72\73\74\75\76\77\78\79\7A\7B\7C\7D\7E\7F\
                  \80\81\82\83\84\85\86\87\88\89\8A\8B\8C\8D\8E\8F\
                  \90\91\92\93\94\95\96\97\98\99\9A\9B\9C\9D\9E\9F\
                  \A0\A1\A2\A3\A4\A5\A6\A7\A8\A9\AA\AB\AC\AD\AE\AF\
                  \B0\B1\B2\B3\B4\B5\B6\B7\B8\B9\BA\BB\BC\BD\BE\BF\
                  \C0\C1\C2\C3\C4\C5\C6\C7\C8\C9\CA\CB\CC\CD\CE\CF\
                  \D0\D1\D2\D3\D4\D5\D6\D7\D8\D9\DA\DB\DC\DD\DE\DF\
                  \E0\E1\E2\E3\E4\E5\E6\E7\E8\E9\EA\EB\EC\ED\EE\EF\
                  \F0\F1\F2\F3\F4\F5\F6\F7\F8\F9\FA\FB\FC\FD\FE\FF"
    :local UTF8toURLe {"00";"01";"02";"03";"04";"05";"06";"07";"08";"09";"0A";"0B";"0C";"0D";"0E";"0F";
                       "10";"11";"12";"13";"14";"15";"16";"17";"18";"19";"1A";"1B";"1C";"1D";"1E";"1F";
                       "+";"21";"22";"23";"24";"25";"26";"27";"28";"29";"2A";"2B";"2C";"-";".";"2F";
                       "0";"1";"2";"3";"4";"5";"6";"7";"8";"9";"3A";"3B";"3C";"3D";"3E";"3F";
                       "40";"A";"B";"C";"D";"E";"F";"G";"H";"I";"J";"K";"L";"M";"N";"O";
                       "P";"Q";"R";"S";"T";"U";"V";"W";"X";"Y";"Z";"5B";"5C";"5D";"5E";"_";
                       "60";"a";"b";"c";"d";"e";"f";"g";"h";"i";"j";"k";"l";"m";"n";"o";
                       "p";"q";"r";"s";"t";"u";"v";"w";"x";"y";"z";"7B";"7C";"7D";"~";"7F";
                       "80";"81";"82";"83";"84";"85";"86";"87";"88";"89";"8A";"8B";"8C";"8D";"8E";"8F";
                       "90";"91";"92";"93";"94";"95";"96";"97";"98";"99";"9A";"9B";"9C";"9D";"9E";"9F";
                       "A0";"A1";"A2";"A3";"A4";"A5";"A6";"A7";"A8";"A9";"AA";"AB";"AC";"AD";"AE";"AF";
                       "B0";"B1";"B2";"B3";"B4";"B5";"B6";"B7";"B8";"B9";"BA";"BB";"BC";"BD";"BE";"BF";
                       "C0";"C1";"C2";"C3";"C4";"C5";"C6";"C7";"C8";"C9";"CA";"CB";"CC";"CD";"CE";"CF";
                       "D0";"D1";"D2";"D3";"D4";"D5";"D6";"D7";"D8";"D9";"DA";"DB";"DC";"DD";"DE";"DF";
                       "E0";"E1";"E2";"E3";"E4";"E5";"E6";"E7";"E8";"E9";"EA";"EB";"EC";"ED";"EE";"EF";
                       "F0";"F1";"F2";"F3";"F4";"F5";"F6";"F7";"F8";"F9";"FA";"FB";"FC";"FD";"FE";"FF"
                      }
    :local string $1
    :if (([:typeof $string] != "str") or ($string = "")) do={ :return "" }
    :local lenstr [:len $string]
    :local constr ""
    :for pos from=0 to=($lenstr - 1) do={
        :local urle ($UTF8toURLe->[:find $ascii [:pick $string $pos ($pos + 1)] -1])
        :local sym $urle
        :if ([:len $urle] = 2) do={:set sym "%$[:pick $urle 0 2]" }
        :set constr "$constr$sym"
    }
    :return $constr
}

:global lastLog
:if ([:typeof $lastLog] != "num") do={:set lastLog 0}

{
:local tgBot    "XXXXXXXXXXXXXXXX:XXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXX"
:local tgChatID "XXXXXXXXX"
:local tgPrefix "\E2\84\B9 MikroTik $[/system identity get name] $[/system resource get board-name]"
:local wtopics  "critical|error|warning"
:local utopics  "ipsec"
:local mkwd     "login failure|logged in|loop|down|fcs|excessive|system|rebooted|ipsec|ike2"
:local ukwd     "unwanted|example|phase1 negotiation failed"
:local trim     256

:local id2num do={:return [:tonum "0x$[:pick $1 1 [:len $1]]"]}
:local tgmessg ""
:local emotico ""
:local temp    ""
:local tempmsg ""

/log
:foreach item in=[find where (((([$id2num $".id"] > $lastLog) and (buffer=memory)) and (!([:tostr $topics]~"($utopics)"))) \
                             and \
                             ((([:tostr $topics]~"($wtopics)") or (message~"($mkwd)")) and (!(message~"($ukwd)"))))] do={
    :set lastLog [$id2num $item]
    :set emotico ""
    :local temp [:tostr [get $item topics]]
    :if ($temp~"critical") do={:set emotico "\E2\98\A0\20"}
    :if ($temp~"error")    do={:set emotico "$emotico\E2\9D\8C\20"}
    :if ($temp~"warning")  do={:set emotico "$emotico\E2\9A\A0\20"}
    :set tempmsg [get $item message]
    :if ([:len $tempmsg] > $trim) do={:set tempmsg ("$[:pick $tempmsg 0 ($trim - 1)]\E2\80\A6") }
    :set tgmessg [$UTF8toURLencode ("$tgPrefix\r\n$[$anydate2isodate [get $item time]] $emotico$tempmsg")]
    /tool fetch url="https://api.telegram.org/bot$tgBot/sendMessage\3Fchat_id=$tgChatID&text=$tgmessg" keep-result=no
}

}
EDIT 1,2,3 and 4: bugfix
EDIT 5: Separate the two version of the script on 2 posts.
EDIT 6: https://forum.mikrotik.com/viewtopic.php?p=1004648#p1004648
EDIT 7: Exclude on default ipsec error messages phase1 negotiation failed, added the possibility to exclude topics regardless are critical or error or warning
(thanks to @S8T8 for the idea)
EDIT 8: Misspell error
EDIT 9: Added the trim parameter to cut the message at wanted length, the default is 256 (thanks to @S8T8 for the idea)
EDIT 10: The trim now ignore the critical / error / warning icons
EDIT 11: Replaced "..." with unicode character 0xE2 0x80 0xA6 "…"
EDIT 12: The "trim" now trim to one less character to add "..." at the end (on this way the message, without considering critical / error / warning icons, are exactly of that length)
Last edited by rextended on Mon Jun 12, 2023 2:32 pm, edited 12 times in total.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Sat May 27, 2023 2:39 pm

Impressive work Rex, thank you very much, really.

I'm away with the family right now, I'll try it as soon as I can.

BR.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sat May 27, 2023 2:41 pm

one unique telegram message with all new log entry code

:global anydate2isodate do={
    :local dtime [:tostr $1]
    /system clock
    :local cyear [get date] ; :if ($cyear ~ "....-..-..") do={:set cyear [:pick $cyear 0 4]} else={:set cyear [:pick $cyear 7 11]}
    :if (([:len $dtime] = 10) or ([:len $dtime] = 11)) do={:set dtime "$dtime 00:00:00"}
    :if ([:len $dtime] = 15) do={:set dtime "$[:pick $dtime 0 6]/$cyear $[:pick $dtime 7 15]"}
    :if ([:len $dtime] = 14) do={:set dtime "$cyear-$[:pick $dtime 0 5] $[:pick $dtime 6 14]"}
    :if ([:len $dtime] =  8) do={:set dtime "$[get date] $dtime"}
    :if ([:tostr $1] = "") do={:set dtime ("$[get date] $[get time]")}
    :local vdoff [:toarray "0,4,5,7,8,10,11,19"]
    :local MM    [:pick $dtime ($vdoff->2) ($vdoff->3)]
    :if ($dtime ~ ".../../....") do={
        :set vdoff [:toarray "7,11,1,3,4,6,12,20"]
        :set MM    ([:find "xxanebarprayunulugepctovecANEBARPRAYUNULUGEPCTOVEC" [:pick $dtime ($vdoff->2) ($vdoff->3)] -1] / 2)
        :if ($MM>12) do={:set MM ($MM - 12)} ; :if ($MM<10) do={:set MM "0$MM"}
    }
    :return "$[:pick $dtime ($vdoff->0) ($vdoff->1)]-$MM-$[:pick $dtime ($vdoff->4) ($vdoff->5)] $[:pick $dtime ($vdoff->6) ($vdoff->7)]"
}

:global UTF8toURLencode do={
    :local ascii "\00\01\02\03\04\05\06\07\08\09\0A\0B\0C\0D\0E\0F\
                  \10\11\12\13\14\15\16\17\18\19\1A\1B\1C\1D\1E\1F\
                  \20\21\22\23\24\25\26\27\28\29\2A\2B\2C\2D\2E\2F\
                  \30\31\32\33\34\35\36\37\38\39\3A\3B\3C\3D\3E\3F\
                  \40\41\42\43\44\45\46\47\48\49\4A\4B\4C\4D\4E\4F\
                  \50\51\52\53\54\55\56\57\58\59\5A\5B\5C\5D\5E\5F\
                  \60\61\62\63\64\65\66\67\68\69\6A\6B\6C\6D\6E\6F\
                  \70\71\72\73\74\75\76\77\78\79\7A\7B\7C\7D\7E\7F\
                  \80\81\82\83\84\85\86\87\88\89\8A\8B\8C\8D\8E\8F\
                  \90\91\92\93\94\95\96\97\98\99\9A\9B\9C\9D\9E\9F\
                  \A0\A1\A2\A3\A4\A5\A6\A7\A8\A9\AA\AB\AC\AD\AE\AF\
                  \B0\B1\B2\B3\B4\B5\B6\B7\B8\B9\BA\BB\BC\BD\BE\BF\
                  \C0\C1\C2\C3\C4\C5\C6\C7\C8\C9\CA\CB\CC\CD\CE\CF\
                  \D0\D1\D2\D3\D4\D5\D6\D7\D8\D9\DA\DB\DC\DD\DE\DF\
                  \E0\E1\E2\E3\E4\E5\E6\E7\E8\E9\EA\EB\EC\ED\EE\EF\
                  \F0\F1\F2\F3\F4\F5\F6\F7\F8\F9\FA\FB\FC\FD\FE\FF"
    :local UTF8toURLe {"00";"01";"02";"03";"04";"05";"06";"07";"08";"09";"0A";"0B";"0C";"0D";"0E";"0F";
                       "10";"11";"12";"13";"14";"15";"16";"17";"18";"19";"1A";"1B";"1C";"1D";"1E";"1F";
                       "+";"21";"22";"23";"24";"25";"26";"27";"28";"29";"2A";"2B";"2C";"-";".";"2F";
                       "0";"1";"2";"3";"4";"5";"6";"7";"8";"9";"3A";"3B";"3C";"3D";"3E";"3F";
                       "40";"A";"B";"C";"D";"E";"F";"G";"H";"I";"J";"K";"L";"M";"N";"O";
                       "P";"Q";"R";"S";"T";"U";"V";"W";"X";"Y";"Z";"5B";"5C";"5D";"5E";"_";
                       "60";"a";"b";"c";"d";"e";"f";"g";"h";"i";"j";"k";"l";"m";"n";"o";
                       "p";"q";"r";"s";"t";"u";"v";"w";"x";"y";"z";"7B";"7C";"7D";"~";"7F";
                       "80";"81";"82";"83";"84";"85";"86";"87";"88";"89";"8A";"8B";"8C";"8D";"8E";"8F";
                       "90";"91";"92";"93";"94";"95";"96";"97";"98";"99";"9A";"9B";"9C";"9D";"9E";"9F";
                       "A0";"A1";"A2";"A3";"A4";"A5";"A6";"A7";"A8";"A9";"AA";"AB";"AC";"AD";"AE";"AF";
                       "B0";"B1";"B2";"B3";"B4";"B5";"B6";"B7";"B8";"B9";"BA";"BB";"BC";"BD";"BE";"BF";
                       "C0";"C1";"C2";"C3";"C4";"C5";"C6";"C7";"C8";"C9";"CA";"CB";"CC";"CD";"CE";"CF";
                       "D0";"D1";"D2";"D3";"D4";"D5";"D6";"D7";"D8";"D9";"DA";"DB";"DC";"DD";"DE";"DF";
                       "E0";"E1";"E2";"E3";"E4";"E5";"E6";"E7";"E8";"E9";"EA";"EB";"EC";"ED";"EE";"EF";
                       "F0";"F1";"F2";"F3";"F4";"F5";"F6";"F7";"F8";"F9";"FA";"FB";"FC";"FD";"FE";"FF"
                      }
    :local string $1
    :if (([:typeof $string] != "str") or ($string = "")) do={ :return "" }
    :local lenstr [:len $string]
    :local constr ""
    :for pos from=0 to=($lenstr - 1) do={
        :local urle ($UTF8toURLe->[:find $ascii [:pick $string $pos ($pos + 1)] -1])
        :local sym $urle
        :if ([:len $urle] = 2) do={:set sym "%$[:pick $urle 0 2]" }
        :set constr "$constr$sym"
    }
    :return $constr
}

:global lastLog
:if ([:typeof $lastLog] != "num") do={:set lastLog 0}

{
:local tgBot    "XXXXXXXXXXXXXXXX:XXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXX"
:local tgChatID "XXXXXXXXX"
:local tgPrefix "\E2\84\B9 MikroTik $[/system identity get name] $[/system resource get board-name]"
:local wtopics  "critical|error|warning"
:local utopics  "ipsec"
:local mkwd     "login failure|logged in|loop|down|fcs|excessive|system|rebooted|ipsec|ike2"
:local ukwd     "unwanted|example|phase1 negotiation failed"
:local trim     256

:local id2num do={:return [:tonum "0x$[:pick $1 1 [:len $1]]"]}
:local tgmessg ""
:local emotico ""
:local temp    ""
:local tempmsg ""

/log
:foreach item in=[find where (((([$id2num $".id"] > $lastLog) and (buffer=memory)) and (!([:tostr $topics]~"($utopics)"))) \
                             and \
                             ((([:tostr $topics]~"($wtopics)") or (message~"($mkwd)")) and (!(message~"($ukwd)"))))] do={
    :set lastLog [$id2num $item]
    :set emotico ""
    :local temp [:tostr [get $item topics]]
    :if ($temp~"critical") do={:set emotico "\E2\98\A0\20"}
    :if ($temp~"error")    do={:set emotico "$emotico\E2\9D\8C\20"}
    :if ($temp~"warning")  do={:set emotico "$emotico\E2\9A\A0\20"}
    :set tempmsg [get $item message]
    :if ([:len $tempmsg] > $trim) do={:set tempmsg ("$[:pick $tempmsg 0 ($trim - 1)]\E2\80\A6") }
    :set tgmessg "$tgmessg$[$anydate2isodate [get $item time]] $emotico$tempmsg\r\n"
}

:if ($tgmessg != "") do={
    :set tgmessg [$UTF8toURLencode ("$tgPrefix\r\n$tgmessg")]
    /tool fetch url="https://api.telegram.org/bot$tgBot/sendMessage\3Fchat_id=$tgChatID&text=$tgmessg" keep-result=no
}

}
EDIT 1: https://forum.mikrotik.com/viewtopic.php?p=1004648#p1004648
EDIT 2: Exclude on default ipsec error messages phase1 negotiation failed, added the possibility to exclude topics regardless are critical or error or warning
(thanks to @S8T8 for the idea)
EDIT 3: Misspell fixed
EDIT 4: Posted the wrong version of the script
EDIT 5: Posted the correct version of the script
EDIT 6: Added the trim parameter to cut the message at wanted length, the default is 256 (thanks to @S8T8 for the idea)
EDIT 7: The trim now ignore the critical / error / warning icons
EDIT 8: Replaced 3 dots at the end "..." with the unicode character 0xE2 0x80 0xA6 "…"
EDIT 9: The "trim" now trim the message to one less character (on this way, without considering critical / error / warning icons, are exactly of that length with "…" at the end)
Last edited by rextended on Mon Jun 12, 2023 2:36 pm, edited 9 times in total.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sat May 27, 2023 2:43 pm

Impressive work Rex, thank you very much, really.
Thanks, I hope is error free. I do not have Telegram, but the URL "text" is correctly generated.
If you ignore the date conversion and the correct urlencode, are less lines of code for do the same thing... ;)
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Sun May 28, 2023 2:28 am

Impressive work Rex, thank you very much, really.
Thanks, I hope is error free. I do not have Telegram, but the URL "text" is correctly generated.
If you ignore the date conversion and the correct urlencode, are less lines of code for do the same thing... ;)
I confirm that your script works perfect, it is much better to work on the last Id.
I love how summarized the code is. Thanks again.
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Sun May 28, 2023 4:04 am

@rextended testing right now, works also for me!
This should be a pinned topic...
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sun May 28, 2023 1:39 pm

Since I never seen at same time critical that is also error or warning on same log, no matter, but I want fix code for this.
from
:if ($temp~"critical") do={:set emotico "\E2\98\A0\20"}
:if ($temp~"error") do={:set emotico "\E2\9D\8C\20"}
:if ($temp~"warning") do={:set emotico "\E2\9A\A0\20"}
to
:if ($temp~"critical") do={:set emotico "\E2\98\A0\20"}
:if ($temp~"error") do={:set emotico "$emotico\E2\9D\8C\20"}
:if ($temp~"warning") do={:set emotico "$emotico\E2\9A\A0\20"}
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sun May 28, 2023 1:44 pm

@rextended testing right now, works also for me!
This should be a pinned topic...
Thanks, added to my "Fragments of Snippets":
viewtopic.php?p=1004649#p1004649
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Sun May 28, 2023 7:13 pm

Also if I don't have any knowledge in scripts I applied few edits;
- added a dash " - " before a new line of log sent on the text
Example:
- 2023-05-28 12:20:46 ether3 link down
- 2023-05-28 12:21:03 ether3 link up (speed 100M, full duplex)

- changed: and !(message~" -> and !(topic~"
I fell that is easier for me to exclude topics instead of content of the log.

- it's possibile (and easy) to have just the time instead of the full date in the text? (Only if is provided by the script, don't want to waste your time)
Example:
- 12:20:46 ether3 link down

Have a nice Sunday!
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sun May 28, 2023 7:43 pm

- changed: and !(message~" -> and !(topic~"
I fell that is easier for me to exclude topics instead of content of the log.
Nice Idea, I consider that on next version.
But doing that in your way, you can accidentally avoid the reporting of critical / error logs if are critical/error at the same time of your unwanted topic...

- it's possibile (and easy) to have just the time instead of the full date in the text? (Only if is provided by the script, don't want to waste your time)
Example:
- 12:20:46 ether3 link down
Yes, replace the line
:return "$[:pick $dtime ($vdoff->0) ($vdoff->1)]-$MM-$[:pick $dtime ($vdoff->4) ($vdoff->5)] $[:pick $dtime ($vdoff->6) ($vdoff->7)]"
with what you want, like
:return "$[:pick $dtime ($vdoff->6) ($vdoff->7)]"
 
abbio90
Member Candidate
Member Candidate
Topic Author
Posts: 163
Joined: Fri Aug 27, 2021 9:16 pm
Contact:

Re: Log Filter

Mon May 29, 2023 8:57 am

sorry guys but i've been busy. I'll try as soon as I have time. but this script left in chat last, send any event log?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Mon May 29, 2023 10:45 am

sorry guys but i've been busy. I'll try as soon as I have time. but this script left in chat last, send any event log?
Both on post #22 and post #24, choose your preferred version
 
abbio90
Member Candidate
Member Candidate
Topic Author
Posts: 163
Joined: Fri Aug 27, 2021 9:16 pm
Contact:

Re: Log Filter

Tue May 30, 2023 7:40 pm

I tried the script and I adapted it to my needs, and I would say that it works very well.. even if some error alerts are annoying and not very sensible like the ipsec phase 1 which fails when the router fails over, it doesn't make sense. .how could I do to create a list of phrases that should not be communicated on telegram? this script I understand will also work from version 7.10 with the ISO adaptations of the date and time. Right?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Tue May 30, 2023 7:46 pm

will also work from version 7.10 with the ISO adaptations of the date and time. Right?
Yes


Both scripts are updated for:
EDIT x: Exclude on default ipsec error messages phase1 negotiation failed, added the possibility to exclude topics regardless are critical or error or warning
(thanks to @S8T8 for the idea)
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Tue May 30, 2023 9:04 pm

Both scripts are updated for:
EDIT x: Exclude on default ipsec error messages phase1 negotiation failed, added the possibility to exclude topics regardless are critical or error or warning
(thanks to @S8T8 for the idea)
Updated! thx

..
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Wed May 31, 2023 1:25 am

EDIT 2: Exclude on default ipsec error messages phase1 negotiation failed, added the possibility to exclude topics regardless are critical or error or warning
EDIT 3: Misspell fixed
Nice update, this was exactly what I meant!
You mind to spend few words on the last Edit?
Hope to have other suggestion to share.
THANKS!!!

FYI, I'm using it as Scheduler running every 10min.
As I'm not a fan of Telegram I'll check if possible an easy integration with other solutions like Elements (Matrix).
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Wed May 31, 2023 1:29 am

You mind to spend few words on the last Edit?
Really? :lol:

Exclude on defoult ipsec ==>> Exclude on default ipsec
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Sun Jun 04, 2023 5:41 pm

Hi @rextended, using the latest version of the script provided, if possible would be nice to have integrated the possibility to exclude some topics but integrate if associated with others to be included, like MikroTik logging settings.

Let me explain a little bit better;
If we include topic "DHCP" a bunch of notifications are sended, the messages "assigned" "unassigned" must be excluded to avoid this.

Excluding topic "DHCP" also exclud the warnings that are not sent.

The possibility to include topics DHCP but not if is !info would be useful.

Make sense? What do you think?

Have a nice weekend!
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Sun Jun 11, 2023 7:15 pm

Hello all, @rextended, someone may will find useful the option to truncate messages, like logs with more than 30 characters are replaced with "...".
What do you think?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sun Jun 11, 2023 8:54 pm

That someone can truncate the message to the desired length,
what's the problem?
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Sun Jun 11, 2023 9:33 pm

Not a problem at all, just to share an opinion, a long log message could result in 3/4 or more text lines.

If you could give an hint would be appreciated, as always.
Thanks
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Sun Jun 11, 2023 10:01 pm

This item write the message on the buffer:

original code

$[get $item message]

search exact part and replace with:

Simply trim at 30 characters:

replacement code

$[:pick [get $item message] 0 30]
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Mon Jun 12, 2023 10:57 am

Excellent @rextended, shockingly easy and works!
Now, in a perfect world text with more that 30 characters should have "..." at the end and shorter not. I'll try to scratch my head but if you have an hint to point in the right direction... :)
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Mon Jun 12, 2023 12:28 pm

Excellent @rextended, shockingly easy and works!
Now, in a perfect world text with more that 30 characters should have "..." at the end and shorter not. I'll try to scratch my head but if you have an hint to point in the right direction... :)

I don't really understand the point of cutting the message to 30 characters, the interesting thing about the script is to know the complete content of each message (sometimes an IP address can be cut off).

:set tgmessg "$tgmessg$[$anydate2isodate [get $item time]] $emotico$[:pick [get $item message] 0 30]...\r\n\r\n"

Image


I see it better this way

Image
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: Log Filter

Mon Jun 12, 2023 1:11 pm

I get your point, I see more interesting the user than IP in this specific case, "you" can still access to the router and read the full log, if someone had access to the device I would try to do log-in to the router anyway. This was just my point of view, I could be wrong.
On my phone I received log messages with 3/4 or more lines, also if you feel that one line of characters is too short, I find useful also to limit to two lines.

Adding "..." as suggested by @diamuxin will add "..." also to shorter messages (don't know if this was clear...)
Will take some time and tests but I'll try to provide a practical example.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Mon Jun 12, 2023 1:51 pm

Even if it's a modification that I don't agree with, for me as the author, it's simple to make.
So seeing as you kindly asked me, I'll do it.

You will see soon updated scripts... with instructions...
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Mon Jun 12, 2023 2:23 pm

Ok, the scripts are updated.
Default is 256.

the trim do not count, for choice, the critical / warning / error icons.

If the string is trimmed, is added "…" at the end.
Notice: is not 3 dots "..." but the appropiate unicode character "…" !!!

example with uncode ch
example with 3 dots in...
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Mon Jun 12, 2023 6:26 pm

Adding "..." as suggested by @diamuxin will add "..." also to shorter messages (don't know if this was clear...)
I don't understand you, messages of < 30 characters will always be visible, they are not cut (tested) they are only cut > 30 characters. The lines 2, 3 or 4 that you receive in Telegram is for the total length of the message, if you limit it to XX characters, you will receive only the required number. You should adjust it to match the width of your phone.

I love this script, @rextended is making it very very fine.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Mon Jun 12, 2023 6:47 pm

Thanks :mrgreen:
 
User avatar
Sertik
Member
Member
Posts: 425
Joined: Fri Sep 25, 2020 3:30 pm
Location: Russia, Moscow

Re: Log Filter

Wed Jun 21, 2023 3:16 pm

In no way belittling the merits of our esteemed Rextended, we previously made the TLGRM system and TaRaN system, which allows not only to annotate messages in Telegram, but also to control Mikrotik from the chat

https://forummikrotik.ru/viewtopic.php?f=14&t=12659
https://forummikrotik.ru/viewtopic.php?f=14&t=14281

differences TaRan from the TLGRM system:

processing of all messages in the chat during the "circulation" period, and not just the last message
- implementation of the main TaRaN execution and parsing modules as functions that can be added from your scripts
- independent operation of the execution module and the annotation module
- customizable user commands
- the ability to set a list of commands from the Router OS to the user's chat bot for a quick call from the chat using "/"
- support for Cyrillic in the output of messages in the chat and in the error to the list

project mail for personal messages: taranscr@mail.ru
Link to download the script for installing the system on the router and the user manual https://cloud.mail.ru/public/3kpW/FSCYMM4ZL

These systems are designed to work with English and Russian messages

And my friend Brook wrote a very cool graphic bot project for Telegram on Mikrotik:

https://forummikrotik.ru/viewtopic.php?f=14&t=13853
https://t.me/mikRobot_RU
https://habr.com/ru/articles/686252/
https://vc.ru/u/1346081-brook-brooklyn/ ... i-ne-tolko
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Wed Jun 21, 2023 4:00 pm

A small example of how not to program...
:local normalTime (($months->$dateM)."/");
:if ($dateD < 10) do={ :set $normalTime ($normalTime."0".[:tostr $dateD]."/"); } else={ :set $normalTime ($normalTime.[:tostr $dateD]."/"); }
:set $normalTime ($normalTime.[:tostr $dateY]." ");   
:if ($timeH < 10) do={ :set $normalTime ($normalTime."0".[:tostr $timeH].":"); } else={ :set $normalTime ($normalTime.[:tostr $timeH].":"); }
:if ($timeM < 10) do={ :set $normalTime ($normalTime."0".[:tostr $timeM].":"); } else={ :set $normalTime ($normalTime.[:tostr $timeM].":"); }
:if ($timeS < 10) do={ :set $normalTime ($normalTime."0".[:tostr $timeS]); }     else={ :set $normalTime ($normalTime.[:tostr $timeS]); }
This is just an example, the script (the point is whether it works or not in the end doesn't matter) is full of logic and common sense errors.
 
User avatar
Sertik
Member
Member
Posts: 425
Joined: Fri Sep 25, 2020 3:30 pm
Location: Russia, Moscow

Re: Log Filter

Wed Jun 21, 2023 4:12 pm

There are probably errors, but the scripts are working. All their functions are performed. There is no big job without mistakes.
And a clear example of this is RouterOS itself, in which errors are fixed from version to version, and they keep appearing and new ones appear ...

What script did you pull this fragment from?

Although I understand where. These are the first versions of the script, they have been repeatedly corrected and improved. You download my works in the final form. Yes, I do not exclude rationality there, but everything works as it should.

Yes, and look at the work of Brook, you will like it.
Last edited by Sertik on Wed Jun 21, 2023 4:39 pm, edited 1 time in total.
 
User avatar
Sertik
Member
Member
Posts: 425
Joined: Fri Sep 25, 2020 3:30 pm
Location: Russia, Moscow

Re: Log Filter

Wed Jun 21, 2023 4:23 pm

By the way, why doesn't a programmer like you have serious big jobs? Not enough time? So you would take up the development of a universal graphics engine for Telegram bots on Mikrotik. It would be great.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Thu Jun 22, 2023 10:11 am

By the way, why doesn't a programmer like you have serious big jobs?
I already have my own company and working for others is out of the question.
Being a programmer for work is much more stressful than being an ISP...

Filling my free time with other jobs is out of the question, I don't want to be rich, I want to be happy... 8)
 
User avatar
Sertik
Member
Member
Posts: 425
Joined: Fri Sep 25, 2020 3:30 pm
Location: Russia, Moscow

Re: Log Filter

Thu Jun 22, 2023 1:02 pm

I already have my own company and working for others is out of the question.
Being a programmer for work is much more stressful than being an ISP...

Filling my free time with other jobs is out of the question, I don't want to be rich, I want to be happy...
It's not about that at all. I'm not suggesting you work for money. But you're a great programmer and you write a lot of very useful things for Microtics. So why not write something other than individual functions and subroutines absolutely for free for personal happiness and the benefit of others. Even in the field of functions, you can create a complete free product that is very necessary for everyone.
 
User avatar
Sertik
Member
Member
Posts: 425
Joined: Fri Sep 25, 2020 3:30 pm
Location: Russia, Moscow

Re: Log Filter

Thu Jun 22, 2023 1:03 pm

After all, others might want to, but they can't, but you can.
Well, everyone wants to be happy, but since the time of Adam and Eve, no one has yet succeeded.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Thu Jun 22, 2023 1:21 pm

@rextended already makes us happy with impressive contributions, he doesn't need to work any more :lol:
 
User avatar
Sertik
Member
Member
Posts: 425
Joined: Fri Sep 25, 2020 3:30 pm
Location: Russia, Moscow

Re: Log Filter

Thu Jun 22, 2023 1:39 pm

So I want him to please us even more.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11968
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Log Filter

Thu Jun 22, 2023 2:42 pm

But at that point... it's like a job ;)
 
RakeZim
just joined
Posts: 1
Joined: Fri Dec 29, 2023 8:53 am

Re: Log Filter

Wed Jan 03, 2024 4:26 pm

Hi folks I hate to ask you but this seems to be the right place, have any of you for some weird reason updated to RouterOS V7.13?
I did it against my better judgment and now LogFilter is broken

Now I get this on Telegram
RouterOS
2024-01-03 15:35:25
Download from https://api.telegram.org/bot********/sendmessage?chat_id=-*******
my logs also have some unusual activity before it would let me know that the schedule had been updated which it still does but now i have a new line as per below with the Topics, Fetch, Info and Debug
Download from https://api.telegram.org/bot******/sendmessage?chat_id=-*****&text=RouterOS%0A2024-01-03 16:11:33%0Alogin failure for user admin from 00:00:00:00:00:00 via winbox to <nothing> FINISHED
The Logfilter works on another router I have around and it worked on here until I updated from RouterOS 7.11.2 to 7.13
Last edited by RakeZim on Wed Jan 03, 2024 4:41 pm, edited 1 time in total.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Wed Jan 03, 2024 10:21 pm

Hi folks I hate to ask you but this seems to be the right place, have any of you for some weird reason updated to RouterOS V7.13?
I did it against my better judgment and now LogFilter is broken

The Logfilter works on another router I have around and it worked on here until I updated from RouterOS 7.11.2 to 7.13
Please post your script and error messages to analyze where the problem is.
 
RakeZim
just joined
Posts: 1
Joined: Fri Dec 29, 2023 8:53 am

Re: Log Filter

Thu Jan 04, 2024 9:16 am

The script I'm using is as per below I've only altered the Bot ID and ChatID.

I'm not getting an error so much as the output has been warped as per my last post I get a download link to the telegram API with my BOTID, The message I'm supposed to get is appended at the end aka "Login failure for user admin from Mac via Winbox" although the <nothing> part is meant to be the router name.

This is from a working router on V7.11.2
Main-rtr
2024-01-04 11:04:40
login failure for user admin from 172.29.255.255 via api
This is from the updated router on V7.13
RouterOS
2024-01-03 16:00:03
Download from https://api.telegram.org/bot123:abcbac/ ... _id=-12345

:local myserver ([/system identity get name])
:local scheduleName "LogFilter"
:local bot "123:abcbac"
:local ChatID "-12345"
#Here you can specify which events from the logs you need to send.
:local startBuf [:toarray [/log find message~" failure" || message~"loop" || message~"critical" || message~"error" || message~"Excessive Average CPU load" || message~"fcs" || message~"excessive"]]

# warn if schedule does not exist
:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
  /log warning "[LogFilter] Alert : Schedule does not exist. Creating schedule ...."

 /system scheduler add name=$scheduleName interval=300s start-date=Jul/09/2020 start-time=startup on-event=LogFilter

  /log warning "[LogFilter] Alert : Schedule created ."
}

# get last time
:local lastTime [/system scheduler get [find name="$scheduleName"] comment]
# for checking time of each log entry
:local currentTime
# log message
:local message
 
# final output
:local output

:local keepOutput false
# if lastTime is empty, set keepOutput to true
:if ([:len $lastTime] = 0) do={
  :set keepOutput true
}

:local counter 0
# loop through all log entries that have been found
:foreach i in=$startBuf do={
 
# loop through all removeThese array items
  :local keepLog true
  :foreach j in=$removeThese do={
#   if this log entry contains any of them, it will be ignored
    :if ([/log get $i message] ~ "$j") do={
      :set keepLog false
    }
  }
  :if ($keepLog = true) do={
   
   :set message [/log get $i message]

#   LOG DATE
#   depending on log date/time, the format may be different. 3 known formats
#   format of jan/01/2002 00:00:00 which shows up at unknown date/time. Using as default
    :set currentTime [ /log get $i time ]
#   format of 00:00:00 which shows up on current day's logs
   :if ([:len $currentTime] = 8 ) do={
     :set currentTime ([:pick [/system clock get date] 0 11]." ".$currentTime)
    } else={
#     format of jan/01 00:00:00 which shows up on previous day's logs
     :if ([:len $currentTime] = 15 ) do={
        :set currentTime ([:pick $currentTime 0 6]."/".[:pick [/system clock get date] 7 11]." ".[:pick $currentTime 7 15])
      }
   }
    
#   if keepOutput is true, add this log entry to output
   :if ($keepOutput = true) do={
     :set output ($output.$currentTime." ".$message."\r\n")
   }

    :if ($currentTime = $lastTime) do={
     :set keepOutput true
     :set output ""
   }
  }
  :if ($counter = ([:len $startBuf]-1)) do={
   :if ($keepOutput = false) do={    
     :if ([:len $message] > 0) do={
        :set output ($output.$currentTimer." ".$message."\r\n")
      }
    }
  }
  :set counter ($counter + 1)
}

if ([:len $output] > 0) do={
  /system scheduler set [find name="$scheduleName"] comment=$currentTime
  /tool fetch url="https://api.telegram.org/bot$bot/sendmessage\?chat_id=$ChatID&text=$myserver%0A$currentTime%0A$message" keep-result=no;
}
Last edited by RakeZim on Thu Jan 04, 2024 9:19 am, edited 1 time in total.
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Thu Jan 04, 2024 7:24 pm

The script I'm using is as per below I've only altered the Bot ID and ChatID.

I'm not getting an error so much as the output has been warped as per my last post I get a download link to the telegram API with my BOTID, The message I'm supposed to get is appended at the end aka "Login failure for user admin from Mac via Winbox" although the <nothing> part is meant to be the router name.

This is from a working router on V7.11.2
Main-rtr
2024-01-04 11:04:40
login failure for user admin from 172.29.255.255 via api
This is from the updated router on V7.13
RouterOS
2024-01-03 16:00:03
Download from https://api.telegram.org/bot123:abcbac/ ... _id=-12345

:local myserver ([/system identity get name])
:local scheduleName "LogFilter"
:local bot "123:abcbac"
:local ChatID "-12345"
#Here you can specify which events from the logs you need to send.
:local startBuf [:toarray [/log find message~" failure" || message~"loop" || message~"critical" || message~"error" || message~"Excessive Average CPU load" || message~"fcs" || message~"excessive"]]

# warn if schedule does not exist
:if ([:len [/system scheduler find name="$scheduleName"]] = 0) do={
  /log warning "[LogFilter] Alert : Schedule does not exist. Creating schedule ...."

 /system scheduler add name=$scheduleName interval=300s start-date=Jul/09/2020 start-time=startup on-event=LogFilter

  /log warning "[LogFilter] Alert : Schedule created ."
}

# get last time
:local lastTime [/system scheduler get [find name="$scheduleName"] comment]
# for checking time of each log entry
:local currentTime
# log message
:local message
 
# final output
:local output

:local keepOutput false
# if lastTime is empty, set keepOutput to true
:if ([:len $lastTime] = 0) do={
  :set keepOutput true
}

:local counter 0
# loop through all log entries that have been found
:foreach i in=$startBuf do={
 
# loop through all removeThese array items
  :local keepLog true
  :foreach j in=$removeThese do={
#   if this log entry contains any of them, it will be ignored
    :if ([/log get $i message] ~ "$j") do={
      :set keepLog false
    }
  }
  :if ($keepLog = true) do={
   
   :set message [/log get $i message]

#   LOG DATE
#   depending on log date/time, the format may be different. 3 known formats
#   format of jan/01/2002 00:00:00 which shows up at unknown date/time. Using as default
    :set currentTime [ /log get $i time ]
#   format of 00:00:00 which shows up on current day's logs
   :if ([:len $currentTime] = 8 ) do={
     :set currentTime ([:pick [/system clock get date] 0 11]." ".$currentTime)
    } else={
#     format of jan/01 00:00:00 which shows up on previous day's logs
     :if ([:len $currentTime] = 15 ) do={
        :set currentTime ([:pick $currentTime 0 6]."/".[:pick [/system clock get date] 7 11]." ".[:pick $currentTime 7 15])
      }
   }
    
#   if keepOutput is true, add this log entry to output
   :if ($keepOutput = true) do={
     :set output ($output.$currentTime." ".$message."\r\n")
   }

    :if ($currentTime = $lastTime) do={
     :set keepOutput true
     :set output ""
   }
  }
  :if ($counter = ([:len $startBuf]-1)) do={
   :if ($keepOutput = false) do={    
     :if ([:len $message] > 0) do={
        :set output ($output.$currentTimer." ".$message."\r\n")
      }
    }
  }
  :set counter ($counter + 1)
}

if ([:len $output] > 0) do={
  /system scheduler set [find name="$scheduleName"] comment=$currentTime
  /tool fetch url="https://api.telegram.org/bot$bot/sendmessage\?chat_id=$ChatID&text=$myserver%0A$currentTime%0A$message" keep-result=no;
}
Try the last version of @rextended:
viewtopic.php?t=196432#p1004552

It works fine for me with v7.13
 
reyota
just joined
Posts: 3
Joined: Mon Jan 29, 2024 8:22 am

Re: Log Filter

Mon Jan 29, 2024 3:37 pm

one unique telegram message with all new log entry code

:global anydate2isodate do={
    :local dtime [:tostr $1]
    /system clock
    :local cyear [get date] ; :if ($cyear ~ "....-..-..") do={:set cyear [:pick $cyear 0 4]} else={:set cyear [:pick $cyear 7 11]}
    :if (([:len $dtime] = 10) or ([:len $dtime] = 11)) do={:set dtime "$dtime 00:00:00"}
    :if ([:len $dtime] = 15) do={:set dtime "$[:pick $dtime 0 6]/$cyear $[:pick $dtime 7 15]"}
    :if ([:len $dtime] = 14) do={:set dtime "$cyear-$[:pick $dtime 0 5] $[:pick $dtime 6 14]"}
    :if ([:len $dtime] =  8) do={:set dtime "$[get date] $dtime"}
    :if ([:tostr $1] = "") do={:set dtime ("$[get date] $[get time]")}
    :local vdoff [:toarray "0,4,5,7,8,10,11,19"]
    :local MM    [:pick $dtime ($vdoff->2) ($vdoff->3)]
    :if ($dtime ~ ".../../....") do={
        :set vdoff [:toarray "7,11,1,3,4,6,12,20"]
        :set MM    ([:find "xxanebarprayunulugepctovecANEBARPRAYUNULUGEPCTOVEC" [:pick $dtime ($vdoff->2) ($vdoff->3)] -1] / 2)
        :if ($MM>12) do={:set MM ($MM - 12)} ; :if ($MM<10) do={:set MM "0$MM"}
    }
    :return "$[:pick $dtime ($vdoff->0) ($vdoff->1)]-$MM-$[:pick $dtime ($vdoff->4) ($vdoff->5)] $[:pick $dtime ($vdoff->6) ($vdoff->7)]"
}

:global UTF8toURLencode do={
    :local ascii "\00\01\02\03\04\05\06\07\08\09\0A\0B\0C\0D\0E\0F\
                  \10\11\12\13\14\15\16\17\18\19\1A\1B\1C\1D\1E\1F\
                  \20\21\22\23\24\25\26\27\28\29\2A\2B\2C\2D\2E\2F\
                  \30\31\32\33\34\35\36\37\38\39\3A\3B\3C\3D\3E\3F\
                  \40\41\42\43\44\45\46\47\48\49\4A\4B\4C\4D\4E\4F\
                  \50\51\52\53\54\55\56\57\58\59\5A\5B\5C\5D\5E\5F\
                  \60\61\62\63\64\65\66\67\68\69\6A\6B\6C\6D\6E\6F\
                  \70\71\72\73\74\75\76\77\78\79\7A\7B\7C\7D\7E\7F\
                  \80\81\82\83\84\85\86\87\88\89\8A\8B\8C\8D\8E\8F\
                  \90\91\92\93\94\95\96\97\98\99\9A\9B\9C\9D\9E\9F\
                  \A0\A1\A2\A3\A4\A5\A6\A7\A8\A9\AA\AB\AC\AD\AE\AF\
                  \B0\B1\B2\B3\B4\B5\B6\B7\B8\B9\BA\BB\BC\BD\BE\BF\
                  \C0\C1\C2\C3\C4\C5\C6\C7\C8\C9\CA\CB\CC\CD\CE\CF\
                  \D0\D1\D2\D3\D4\D5\D6\D7\D8\D9\DA\DB\DC\DD\DE\DF\
                  \E0\E1\E2\E3\E4\E5\E6\E7\E8\E9\EA\EB\EC\ED\EE\EF\
                  \F0\F1\F2\F3\F4\F5\F6\F7\F8\F9\FA\FB\FC\FD\FE\FF"
    :local UTF8toURLe {"00";"01";"02";"03";"04";"05";"06";"07";"08";"09";"0A";"0B";"0C";"0D";"0E";"0F";
                       "10";"11";"12";"13";"14";"15";"16";"17";"18";"19";"1A";"1B";"1C";"1D";"1E";"1F";
                       "+";"21";"22";"23";"24";"25";"26";"27";"28";"29";"2A";"2B";"2C";"-";".";"2F";
                       "0";"1";"2";"3";"4";"5";"6";"7";"8";"9";"3A";"3B";"3C";"3D";"3E";"3F";
                       "40";"A";"B";"C";"D";"E";"F";"G";"H";"I";"J";"K";"L";"M";"N";"O";
                       "P";"Q";"R";"S";"T";"U";"V";"W";"X";"Y";"Z";"5B";"5C";"5D";"5E";"_";
                       "60";"a";"b";"c";"d";"e";"f";"g";"h";"i";"j";"k";"l";"m";"n";"o";
                       "p";"q";"r";"s";"t";"u";"v";"w";"x";"y";"z";"7B";"7C";"7D";"~";"7F";
                       "80";"81";"82";"83";"84";"85";"86";"87";"88";"89";"8A";"8B";"8C";"8D";"8E";"8F";
                       "90";"91";"92";"93";"94";"95";"96";"97";"98";"99";"9A";"9B";"9C";"9D";"9E";"9F";
                       "A0";"A1";"A2";"A3";"A4";"A5";"A6";"A7";"A8";"A9";"AA";"AB";"AC";"AD";"AE";"AF";
                       "B0";"B1";"B2";"B3";"B4";"B5";"B6";"B7";"B8";"B9";"BA";"BB";"BC";"BD";"BE";"BF";
                       "C0";"C1";"C2";"C3";"C4";"C5";"C6";"C7";"C8";"C9";"CA";"CB";"CC";"CD";"CE";"CF";
                       "D0";"D1";"D2";"D3";"D4";"D5";"D6";"D7";"D8";"D9";"DA";"DB";"DC";"DD";"DE";"DF";
                       "E0";"E1";"E2";"E3";"E4";"E5";"E6";"E7";"E8";"E9";"EA";"EB";"EC";"ED";"EE";"EF";
                       "F0";"F1";"F2";"F3";"F4";"F5";"F6";"F7";"F8";"F9";"FA";"FB";"FC";"FD";"FE";"FF"
                      }
    :local string $1
    :if (([:typeof $string] != "str") or ($string = "")) do={ :return "" }
    :local lenstr [:len $string]
    :local constr ""
    :for pos from=0 to=($lenstr - 1) do={
        :local urle ($UTF8toURLe->[:find $ascii [:pick $string $pos ($pos + 1)] -1])
        :local sym $urle
        :if ([:len $urle] = 2) do={:set sym "%$[:pick $urle 0 2]" }
        :set constr "$constr$sym"
    }
    :return $constr
}

:global lastLog
:if ([:typeof $lastLog] != "num") do={:set lastLog 0}

{
:local tgBot    "XXXXXXXXXXXXXXXX:XXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXX"
:local tgChatID "XXXXXXXXX"
:local tgPrefix "\E2\84\B9 MikroTik $[/system identity get name] $[/system resource get board-name]"
:local wtopics  "critical|error|warning"
:local utopics  "ipsec"
:local mkwd     "login failure|logged in|loop|down|fcs|excessive|system|rebooted|ipsec|ike2"
:local ukwd     "unwanted|example|phase1 negotiation failed"
:local trim     256

:local id2num do={:return [:tonum "0x$[:pick $1 1 [:len $1]]"]}
:local tgmessg ""
:local emotico ""
:local temp    ""
:local tempmsg ""

/log
:foreach item in=[find where (((([$id2num $".id"] > $lastLog) and (buffer=memory)) and (!([:tostr $topics]~"($utopics)"))) \
                             and \
                             ((([:tostr $topics]~"($wtopics)") or (message~"($mkwd)")) and (!(message~"($ukwd)"))))] do={
    :set lastLog [$id2num $item]
    :set emotico ""
    :local temp [:tostr [get $item topics]]
    :if ($temp~"critical") do={:set emotico "\E2\98\A0\20"}
    :if ($temp~"error")    do={:set emotico "$emotico\E2\9D\8C\20"}
    :if ($temp~"warning")  do={:set emotico "$emotico\E2\9A\A0\20"}
    :set tempmsg [get $item message]
    :if ([:len $tempmsg] > $trim) do={:set tempmsg ("$[:pick $tempmsg 0 ($trim - 1)]\E2\80\A6") }
    :set tgmessg "$tgmessg$[$anydate2isodate [get $item time]] $emotico$tempmsg\r\n"
}

:if ($tgmessg != "") do={
    :set tgmessg [$UTF8toURLencode ("$tgPrefix\r\n$tgmessg")]
    /tool fetch url="https://api.telegram.org/bot$tgBot/send ... t=$tgmessg" keep-result=no
}

}
EDIT 1: viewtopic.php?p=1004648#p1004648
EDIT 2: Exclude on default ipsec error messages phase1 negotiation failed, added the possibility to exclude topics regardless are critical or error or warning
(thanks to @S8T8 for the idea)
EDIT 3: Misspell fixed
EDIT 4: Posted the wrong version of the script
EDIT 5: Posted the correct version of the script
EDIT 6: Added the trim parameter to cut the message at wanted length, the default is 256 (thanks to @S8T8 for the idea)
EDIT 7: The trim now ignore the critical / error / warning icons
EDIT 8: Replaced 3 dots at the end "..." with the unicode character 0xE2 0x80 0xA6 "…"
EDIT 9: The "trim" now trim the message to one less character (on this way, without considering critical / error / warning icons, are exactly of that length with "…" at the end)
I tried to apply this script to CRS with OS version 7.12, but the script sent all the last logs that appeared in CRS and did not select what was written in :local mkwd
What do you think is the cause?
 
User avatar
diamuxin
Member
Member
Posts: 319
Joined: Thu Sep 09, 2021 5:46 pm
Location: Alhambra's City

Re: Log Filter

Tue Jan 30, 2024 3:11 pm

I tried to apply this script to CRS with OS version 7.12, but the script sent all the last logs that appeared in CRS and did not select what was written in :local mkwd
What do you think is the cause?
Run the script from CLI (between {...}) and check if it dumps any errors, the script works fine for me on a RB4011 and others.

Try to check the script policies that look like this:
policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon. ("ftp" is mandatory)
 
reyota
just joined
Posts: 3
Joined: Mon Jan 29, 2024 8:22 am

Re: Log Filter

Wed Jan 31, 2024 6:16 am

I tried to apply this script to CRS with OS version 7.12, but the script sent all the last logs that appeared in CRS and did not select what was written in :local mkwd
What do you think is the cause?
Run the script from CLI (between {...}) and check if it dumps any errors, the script works fine for me on a RB4011 and others.

Try to check the script policies that look like this:
policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon. ("ftp" is mandatory)
Sorry, I forgot something important in the script description, namely what you want to monitor in the log and what you don't want. I thought this script was the same as the script created by reextended previously in the post viewtopic.php?t=151953 but it turned out to be different. Thank you for the response

Who is online

Users browsing this forum: baragoon, diamuxin, Ellaham, rextended and 21 guests