Community discussions

MikroTik App
 
pe1chl
Forum Guru
Forum Guru
Topic Author
Posts: 10195
Joined: Mon Jun 08, 2015 12:09 pm

Get log line from memory log

Wed Mar 17, 2021 1:31 pm

To work around some problem I would like to write a script that looks at the memory log to check for some error message.
(due to the lack of a "script" action in /system logging, which would be much better: when a log occurs with that target, call a specified script with log info as a parameter)

I can do things like :put [/log find where topics=ipsec,error] and it will show a list of IDs when there is a match, however I see no way to get the corresponding messages as there appears to be no /log get command.
Unfortunately there also appears to be no way to find the N most recent messages in that /log find query (I want to look only at the last one). But that could be overcome by looking only at the last item of the returned array.
I need the actual message text because it contains an IP address that I need to isolate from it for further action.

Is it in any way possible to do this? Maybe someone has a reference to an example script that uses this?
 
2frogs
Forum Veteran
Forum Veteran
Posts: 713
Joined: Fri Dec 03, 2010 1:38 am

Re: Get log line from memory log

Wed Mar 17, 2021 2:40 pm

 
pe1chl
Forum Guru
Forum Guru
Topic Author
Posts: 10195
Joined: Mon Jun 08, 2015 12:09 pm

Re: Get log line from memory log

Wed Mar 17, 2021 5:59 pm

Thanks! It turns out that /log get does exist. It does not auto-complete with tab, strange. But it works.
And the /print as-value trick also works.
I remembered that I had seen examples but could not find them using the search....
 
elico
Member Candidate
Member Candidate
Posts: 143
Joined: Mon Nov 07, 2016 3:23 am

Re: Get log line from memory log

Tue Mar 23, 2021 10:12 pm

Thanks! It turns out that /log get does exist. It does not auto-complete with tab, strange. But it works.
And the /print as-value trick also works.
I remembered that I had seen examples but could not find them using the search....
To find the ids you can use:
:put [find .id ]
or loop the [find .id ] somehow
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19099
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Get log line from memory log

Wed Mar 24, 2021 12:41 am

Hi there, I would very much like to use your idea for the UPS monitoring that goes on and available in the LOG.
I want to detect a log entry and then send it to my telegram bot.
I need help with what to do on the put section

Would this be the right approach?

(1) Step one would entail adding via CLI these two commands
...... Added via NEW TERMINAL
/system logging action add memory-lines=100 memory-stop-on-full=no name=logParse target=memory
/system logging add action=logParse disabled=no prefix="" topics=ups,info

(2) Then via system script, I can use the provided stock script as is, being self contained and refers to logParse - this will create a stored buffer.

(3) Step 3, I can use the provided top section of stock script on the third item (the parser script) and only need to customize one of the #check examples for my needs.. This script will find the words in the text highlighted. The action I require is the following bit of script including a fetch
:local sub1 ([/system clock get time])
:local sub2 ([/system clock get date])
/tool fetch "https://api.telegram.org/bot<yourtokencode>/sendMessage\?chat_id=<botcode>&text=Router+went+offline+at+@sub1+on+$sub2"
.........................
TWO QUESTIONS:
i. How do I put this in the PUT section below
ii. what is the purpose of the beep frequency lines, do I even need them??

............... # Check for Power failure
:if ([:find [:tostr $logMessage] "USB UPS AC power off"] != "") do={
:beep frequency=90 length=500ms
:beep frequency=130 length=500ms
:put ("A Power Failure has occurred. Take some action")
}
# End check for Power Failure

Where I would need help is defining the :put command ! I imagine this where I put in the fetch command???
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Get log line from memory log

Wed Mar 24, 2021 5:34 am

pe1chl say:
Unfortunately there also appears to be no way to find the N most recent messages in that /log find query (I want to look only at the last one).
Is it in any way possible to do this? Maybe someone has a reference to an example script that uses this?
Part of my last script solve that problem from topic.
Source from [WorkARound] IPSec PH2 "ready to send" without Active Peer, User cannot connect, Win10 #809 or #0 [SOLVED]

1 line description: I search in logs all entry with IP and pick username from all results, if empty I receive empty results (better then error).
local UserWithProblem
foreach i in=[log find where message~"$BadPolicyIP" message~"logged" topics~"l2tp"] do={
  local LogWithUserWithProblem [log get $i message]
  set $UserWithProblem [pick $LogWithUserWithProblem 0 [:find $LogWithUserWithProblem " "]]
}

# From all similar log in/out logs I have last one and login of l2tp user is:
put $UserWithProblem
I hope it's give some new idea for you
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Get log line from memory log

Wed Mar 24, 2021 5:55 am

anav say:
Would this be the right approach?
Probably yes, most cases I try test step after step.

i. How do I put this in the PUT section below
Like that:
# Check for Power failure
:if ([:find [:tostr $logMessage] "USB UPS AC power off"] != "") do={
:beep frequency=90 length=500ms
:beep frequency=130 length=500ms
:put ("A Power Failure has occurred. Take some action")
:local sub1 ([/system clock get time])
:local sub2 ([/system clock get date])
/tool fetch "https://api.telegram.org/bot<yourtokencode>/sendMessage\?chat_id=<botcode>&text=Router+went+offline+at+@sub1+on+$sub2"
}
# End check for Power Failure

ii. what is the purpose of the beep frequency lines, do I even need them??
:beep do a sound at internal buzzer/speaker.
You not play Super Mario theme at RouterOS ?? Realy ?
Where I would need help is defining the :put command ! I imagine this where I put in the fetch command???
In scripting you must feal the magic of creator, you can put anything anywhere and .. you found that bugs :) or success of doing something like you think, or bugs again. This is like loop.

I hope add some info here.
BR Marcin (SiB) Przysowa
 
pe1chl
Forum Guru
Forum Guru
Topic Author
Posts: 10195
Joined: Mon Jun 08, 2015 12:09 pm

Re: Get log line from memory log

Wed Mar 24, 2021 10:31 am

Yeah, I already found the solution to my problem. my mistake was to just start scribbling away in the console and trying things like:
/log print where...
:put [/log find ...]
but when I tried /log [tab] to see what can be done, there was NO /log get showing up. so I thought /log get would not be implemented.
however this turns out to be a peculiarity of that tab expansion, /log get really does exist and I could use it.

In my case I am working around a problem with IPsec in combination with NAT. I have a client system which is a MikroTik router connected via a WiFi link to a home internet connection with NAT active (not MikroTik), and then setting up an L2TP/IPsec connection to the central router.
Sometimes when that WiFi link is bad the IPsec link times out on our end and not on the client end, and/or the NAT router sees a new UDP port 500 session before the other one was cleared and takes another source port number.
Because of that, the two sides of IPsec no longer get in agreement with eachother, and that situation is permanent. It apparently is an issue with this particular NAT implementation because other links never have this issue here.

I found that by blocking the traffic from that system for a couple of minutes and then allowing it again, the connection is established "fresh" and this issue is worked around.
So I wrote this and scheduled it every 2 minutes:
# scan log buffer for ipsec error messages
# when error is "phase1 negotiation failed due to time up", temporarily block the sender

:global lastTime;

:local currentBuf [ :toarray [ /log find topics=ipsec,error and message~"phase1 negotiation failed due to" ] ] ;
:local currentLineCount [ :len $currentBuf ] ;

:if ($currentLineCount > 0) do={
    :local currentTime [/log get [ :pick $currentBuf ($currentLineCount -1) ] time ] ;
    if ($currentTime != $lastTime) do={
        :set lastTime $currentTime ;
        :local currentMessage [/log get [ :pick $currentBuf ($currentLineCount -1) ] message ] ;
        :if ($currentMessage~"due to time up") do={
            :local ipaddress [:pick $currentMessage ([:find $currentMessage "<=>" ]+3) 99 ] ;
            :set ipaddress [:pick $ipaddress 0 [:find $ipaddress "[" ] ];
            :local activepeers [ :toarray [ /ip ipsec active-peers find where remote-address=$ipaddress and state=established ] ] ;
            :if ( [ :len $activepeers ] = 0) do={
                :log info "Temporarily blocking $ipaddress due to errors" ;
                /ip firewall address-list add list=blocked address=$ipaddress timeout="00:05:00" ;
            }
        }
    }
}
The address-list "blocked" will block all traffic for that IP both in input and output chain.

It appears to work fine, I saw the problem happen 2 times and it was nicely recovered.
Before, I would see the link is down and 100 messages on the console with "phase1 negotiation failed due to time up" and the connection still down.

The message is matched in two parts because blocking the IP will result in a new message "phase1 negotiation failed due to send error" and this should be regarded as the new "last message" but not trigger another blocking.
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Get log line from memory log

Wed Mar 24, 2021 11:47 am

This topic is interesting because I see similar problems with some IPSec tunnels.
Sometimes we must do action like
  • re-enable policies,
  • re-enable proposals,
  • remove active peer
  • disable PH1&2 IPSec for 5m and then enable it.
Those 4 type of actions works on differ problems.
 
pe1chl
Forum Guru
Forum Guru
Topic Author
Posts: 10195
Joined: Mon Jun 08, 2015 12:09 pm

Re: Get log line from memory log

Wed Mar 24, 2021 12:12 pm

In that case you may be able to use a similar automatic script.
I still think a welcome new feature would be a "script" target for error logging.
Like this:
/system logging
add action=script topics=ipsec,error script="ipsecerrorscript"

Then whenever a message with topic ipsec,error is generated the script is run and it gets the log item info in some predefined variables (like with other event scripts).
That means the script can watch along the errors and act upon them immediately instead of the crude method of scheduling it and having to peek in the memory log.
That would also make it easier to send some important messages to a chat service instead of e-mail or syslog.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19099
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Get log line from memory log

Wed Mar 24, 2021 1:20 pm

I guess I missed the purpose of the PUT command or sequence?
Are you saying I dont need it in my particular script??

Also should this work for ensuring no files are added
# Check for Power failure
:if ([:find [:tostr $logMessage] "USB UPS AC power off"] != "") do={
:beep frequency=90 length=500ms
:beep frequency=130 length=500ms
:put ("A Power Failure has occurred. Take some action")
:local sub1 ([/system clock get time])
:local sub2 ([/system clock get date])
/tool fetch "https://api.telegram.org/bot<yourtokencode>/sendMessage\?chat_id=<botcode>&text=Router+went+offline+at+@sub1+on+$sub2"
keep-result=no
}
# End check for Power Failure
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Get log line from memory log

Wed Mar 24, 2021 3:09 pm

pe1chl say:
I still think a welcome new feature would be a "script" target for error logging.
That means the script can watch along the errors and act upon them immediately instead of the crude method of scheduling it and having to peek in the memory log.
That would also make it easier to send some important messages to a chat service instead of e-mail or syslog.
But I have that scripts... many scripts related to LTE and IPSec and others problems with RouterOS what are not fix by years.

One of simplest versions who react on not active, not touch dynamic users and do remove ph1 who reconnect all stuff. This is one who is start to build own version.

local EmailTo marcin.przysowa@domain.tld

# Monitor the not active ipsec who are not Dynamic like l2tp-ipsec from template
# Script can be agressive becasue he can detect your many Policies works on one Peer.

foreach NotActive in=[ip ipsec policy find where !active !template] do={

  local AffectedPeerName [ip ipsec policy get $NotActive peer  ]
  local AffectedPeerComment [ip ipsec policy get $NotActive comment ]
  local AffectedPeerLocalSA [ip ipsec policy get $NotActive sa-src-address ]
  local AffectedPeerRemoteSA [ip ipsec policy get $NotActive sa-dst-address ]
  local Message ("Customer DC: IPSec PH2 !Active Peer: ".$AffectedPeerName." SA:".$AffectedPeerLocalSA."<->".$AffectedPeerRemoteSA." with Comment: ".$AffectedPeerComment)
  log print file=ipsec.txt
  tool e-mail send to=$EmailTo subject=$Message file=ipsec.txt
  #ip ipsec active-peers remove [find local-address=$AffectedPeerLocalSA remote-address=$AffectedPeerRemoteSA]
  delay 10s
}
 
accarda
Member Candidate
Member Candidate
Posts: 208
Joined: Fri Apr 05, 2019 4:06 pm
Location: Italy

Re: Get log line from memory log

Wed Mar 24, 2021 3:39 pm

I still think a welcome new feature would be a "script" target for error logging.
Like this:
/system logging
add action=script topics=ipsec,error script="ipsecerrorscript"

Then whenever a message with topic ipsec,error is generated the script is run and it gets the log item info in some predefined variables (like with other event scripts).
It will be definitively great if among the other existing methods there will be such trigger in logging to a script for pushing updates and info as soon as they are intercepted in log.
Hopefully maybe one day...
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Get log line from memory log

Wed Mar 24, 2021 3:40 pm

anav say:
I guess I missed the purpose of the PUT command or sequence?
put do a display, print, println function. For me it's stupid name but not I create it.
put "This will be display only as echo on terminal, no to logs, no in popup etc. . Only if you run me from terminal/ssh then I display you this text."
put "I do nothing without print output of parameters like: $SomeParameter"
put ("Text on left: " . $MessageFromVariable . " then of sentence.")

Are you saying I dont need it in my particular script?
Your are a god of this script, script do what you enter/write at him. You give him a logic. Logic with :put is only for display stuff into terminal for debug mostly.

Also should this work for ensuring no files are added
What file ?

# Check for Power failure
:if ([:find [:tostr $logMessage] "USB UPS AC power off"] != "") do={
:beep frequency=90 length=500ms
:beep frequency=130 length=500ms
:put ("A Power Failure has occurred. Take some action")
:local sub1 ([/system clock get time])
:local sub2 ([/system clock get date])
/tool fetch "https://api.telegram.org/bot<yourtokencode>/sendMessage\?chat_id=<botcode>&text=Router+went+offline+at+@sub1+on+$sub2" keep-result=no
}
# End check for Power Failure
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19099
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Get log line from memory log

Wed Mar 24, 2021 4:47 pm

The file is one created in FILES by the fetch tool.
The keep-file=no just eliminates the file from being placed in the FILE menu.
Okay good to go......... Will give the script a try.

OKAY
1. I have no idea if the buffer part of the script works (the first long self-contained part)
2. The parser script works in that if I run the script I do get a telgram notice.

However, I have no idea if the whole thing is actually working and I discovered my angst arises from.
THERE IS NO SCHEDULE FOR THE BUFFER SCRIPT, how does it get activated?
HOW DOES THE parser script get activated??

I am assuming I run the first Buffer script on a schedule of lets say every 5 minutes and it triggers the parser script when required??
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Get log line from memory log

Wed Mar 24, 2021 5:37 pm

I use a method with 2parts
  • script who do his job and I never know how long he will work or if he works in loop
  • scheduler as watchdog, who check and monitor that script is still running or not
scheduler can be run every 3s, 5s, 15s, 1m, what ever you want - he just do fast action
#Script WatchDog for script name:
local ScriptName check_ipsec

local ScriptRuningInstances [:len [system script job find script=$ScriptName]]
if ( $ScriptRuningInstances = 0) do={/system script run $ScriptName};
if ( $ScriptRuningInstances = 1) do={ :nothing };
if ( $ScriptRuningInstances >= 2) do={system script job remove [find script=$ScriptName]};


Final script must have proper name, in this scenario: check_ipsec
And I am sure that script be run only ones and if he still work then I do nothing :) yes, this is buildin feature :)

Who is online

Users browsing this forum: No registered users and 27 guests