Community discussions

MikroTik App
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Logging prefix is a mess SUP-105353 SUP-144261

Sun Aug 06, 2017 8:49 pm

I do log packet from my mikrotik's to Splunk.
This works nice, except I have problem to categorize package.

Here is a list of prefix I have found:
certificate,debug
certificate,info
dhcp,critical,error
dhcp,debug
dhcp,debug,packet
dhcp,debug,state
dhcp,info
dhcp,warning
dns
dns,packet
e-mail,debug
firewall,info
interface,info
ipsec
ipsec,debug
ipsec,debug,packet
ipsec,error
ipsec,info
l2tp,debug
l2tp,debug,packet
l2tp,info
l2tp,ppp,debug
l2tp,ppp,debug,packet
l2tp,ppp,error
l2tp,ppp,info
l2tp,ppp,info,account
ntp,debug
ntp,debug,packet
pptp,debug
pptp,debug,packet
pptp,info
pptp,ppp,debug
pptp,ppp,debug,packet
pptp,ppp,error
pptp,ppp,info
pptp,ppp,info,account
radvd,debug
route,debug
route,debug,calc
route,debug,event
script,error
snmp
snmp,debug
ssh,debug
ssh,debug,packet
ssh,info
sstp,packet
system,e-mail,error
system,error,critical
system,info
system,info,account
upnp
It looks like its on format:
module,severity,info, eks ssh,debug,packet
But that is only half true.
What about:
system,error,critical is that module,severity,severity?
system,e-mail,error module,module,severity?
ipsec here is severity missing
pptp,ppp,info,account module,module,severity,info?

Why no just clean this up to only use module, severity, info.
Eks:
e-mail,error, blabla other info
On all message use severity.

E-mail should be its own module, not listed under system.

Hope some one can clean this up. It would make Splunk application much more easy.

Jo
Last edited by Jotne on Mon Feb 19, 2024 3:00 pm, edited 2 times in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Fri Jul 13, 2018 2:50 pm

Still nothing has happen to this.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Thu Apr 18, 2019 10:04 am

I am still waiting for this to be fixed (cleaned up)
Should not be to hard??
If it can not be done whit 6.x, add it to the 7.x version of ros
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Wed Jun 17, 2020 5:34 pm

I can see that v7 beta has not fixed anything regarding log format.
 
neutronlaser
Member
Member
Posts: 445
Joined: Thu Jan 18, 2018 5:18 pm

Re: Logging prefix is a mess

Fri Sep 11, 2020 5:19 am

I don't think anyone cares.
 
jarda
Forum Guru
Forum Guru
Posts: 7756
Joined: Mon Oct 22, 2012 4:46 pm

Re: Logging prefix is a mess

Fri Sep 11, 2020 7:29 am

Seems so. But the idea is not bad, I like it.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Fri Sep 11, 2020 11:16 am

When using external logging tools like Splunk to analyse logs, this old and messy format gives a lot of extra work.
I have sent this request two times to MikroTik so they know about it.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10183
Joined: Mon Jun 08, 2015 12:09 pm

Re: Logging prefix is a mess

Wed Jan 13, 2021 11:56 am

I have filed a feature request some time ago to allow more control over the logging.
Of course the best would be when there is much more detail about the log message in the prefix, probably even up to a unique identifier of each message.
(so you don't have to rely on pattern matching of the message text to separate the individual error messages for the same category)
When each message has a unique category it would also be possible to suppress certain messages while showing detailed output of some category for some reason (when not using Splunk but only the internal logging handler).

Lacking that, I have proposed to add regexp matching capability to the logging topics matcher, but of course more detailed topics would be best.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Wed Dec 08, 2021 3:00 pm

Still not fixed in v7.1
 
infabo
Long time Member
Long time Member
Posts: 586
Joined: Thu Nov 12, 2020 12:07 pm

Re: Logging prefix is a mess

Thu Jun 02, 2022 2:09 pm

ROS-internally they just throw all into the same bin and call it "topics". So no distinction between module and severity.
But they seem to already separate module/severity when logging to syslog-server (see https://help.mikrotik.com/docs/display/ROS/Log).
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Sat Oct 01, 2022 11:02 am

There are some small changes in 7.x but as you see in the list below, its not possible to see what severity level each type of log lines have.
Example some DNS just shows DNS. IT should be in an equal comma separated format. Example l2tp,ppp,info,account compare to l2tp,info. Severity at 3rd or 2end field??

Look at these two:
system,critical,info
system,error,critical

What critically are those messages?

Taken from 7.5 logs:
bridge,info
bridge,stp
dhcp,debug
dhcp,debug,packet
dhcp,debug,state
dhcp,info
dhcp,warning
dns
dns,error
dns,packet
dns,warning
e-mail,info
firewall,info
info
interface,info
ipsec
ipsec,error
ipsec,info
l2tp,info
l2tp,ppp,error
l2tp,ppp,info
l2tp,ppp,info,account
netwatch,info
ntp,warning
poe-out,info
route,bgp,error
route,bgp,info
route,ospf,info
route,ospf,warning
script,error
script,info
snmp
ssh,info
system,critical,info
system,error,critical
system,info
system,info,account
system,info,critical
upnp
wireguard,debug
wireless,info
Make it some like this:
Module,Severity,Type,Other
 
pe1chl
Forum Guru
Forum Guru
Posts: 10183
Joined: Mon Jun 08, 2015 12:09 pm

Re: Logging prefix is a mess

Sat Oct 01, 2022 12:06 pm

As written above, I would propose adding another identifier which is unique for the message. E.g. an 8-digit hex number. That would be last in the sequence.
This allows to suppress one particular message or to match it in a script. The number would be assigned once to each specific message and never change. Maybe the first digit of the number would indicate the severity, then some digits for the module, and the remainder the message number.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Sat Oct 01, 2022 12:41 pm

I like that idea, an ID in form of a number of text.
If you look at logg message for a Cisco Router/Firewall, all message have their own ID:
%CDP_PD-4-POWER_OK
%DOT11-4-BA_FLUSH
%DOT11-6-ROAMED
%EVT-4-WRN
%HA_EM-6-LOG
%ILPOWER-7-DETECT
%LINEPROTO-5-UPDOWN
%LINK-3-UPDOWN
%LINK-5-CHANGED
%LINK-6-UPDOWN
%SW_MATM-4-MACFLAP_NOTIF
%SYS-3-HARIKARI
%SYS-5-CONFIG_I
%SYS-5-RELOAD
etc
They are in the form
Module-Severity-Type_of_message.

Some like I requested above
Last edited by Jotne on Sat Oct 01, 2022 12:53 pm, edited 1 time in total.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10183
Joined: Mon Jun 08, 2015 12:09 pm

Re: Logging prefix is a mess

Sat Oct 01, 2022 12:50 pm

Yes, most professional systems have such a structure. E.g. VMS, IBM operating systems, even Microsoft Windows has error numbers.
Essential in my opinion is that the number is only an addition for programming purposes (scripting, logging in external systems, etc) and not a replacement as it is in Microsoft Windows.
(something went wrong, error code 89abcdef. which you then have to google to see what it even means. there should always be a message text)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Sat Oct 01, 2022 12:58 pm

I am not very happy with the Microsoft logging part.

Yes, each message has its own ID, so that is ok, but:

XML will give large/long messages.

Standard logs: hard to see what belongs to what and they do add an message telling: "This message was genereted due to ++++".
Same text added to same EventCode id. Just makes the message longer without giving any thing extra.

And if you install Swedish or other language on a server running Citrix or other Multi user system) the log message are mixed in English and Swedish....
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Sun Nov 06, 2022 8:31 pm

The best solution to make logging great again for syslog:

Just use:

rfc 5424 (released March 2009)

There all modules are separated by correct name at correct location in the message.
Messages that belongs together have same ID
etc...


Please MT do follow standards.
https://www.rfc-editor.org/rfc/rfc5424
 
User avatar
smyers119
Member Candidate
Member Candidate
Posts: 232
Joined: Sat Feb 27, 2021 8:16 pm
Location: USA

Re: Logging prefix is a mess

Mon Nov 07, 2022 11:13 pm

mikrotik already has a checkbox to enable rfc3164 compatibility.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Tue Nov 08, 2022 11:18 am

How do you know?
 
User avatar
Znevna
Forum Guru
Forum Guru
Posts: 1347
Joined: Mon Sep 23, 2019 1:04 pm

Re: Logging prefix is a mess

Tue Nov 08, 2022 11:26 am

syslog001.JPG
bsd-syslog (yes|no; Default: ) whether to use bsd-syslog as defined in RFC 3164
https://wiki.mikrotik.com/wiki/Manual:System/Log
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Tue Nov 08, 2022 6:13 pm

Does no help at all. BSD logs are even worse . Look at these DHCP debug logs.

BSD logs
2022-11-08T17:06:35.639612+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     Client-Id = 01-00-0C-AA-9B-EA-66
2022-11-08T17:06:35.639683+01:00 <31>Nov  8 17:06:34 OVA MikroTik: dhcp-client on bridge received ack with id 3916435468 from 10.11.10.1
2022-11-08T17:06:35.639744+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     ciaddr = 0.0.0.0
2022-11-08T17:06:35.639831+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     yiaddr = 10.11.10.141
2022-11-08T17:06:35.639884+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     siaddr = 10.11.10.1
2022-11-08T17:06:35.639956+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     chaddr = 00:0C:AA:9B:EA:66
2022-11-08T17:06:35.640060+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     Subnet-Mask = 255.255.254.0
2022-11-08T17:06:35.640144+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     Router = 10.11.10.1
2022-11-08T17:06:35.640210+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     Domain-Server = 10.11.10.1
2022-11-08T17:06:35.640285+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     NTP-Server = 10.11.10.1
2022-11-08T17:06:35.640349+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     Address-Time = 86400
2022-11-08T17:06:35.640431+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     Msg-Type = ack
2022-11-08T17:06:35.640493+01:00 <31>Nov  8 17:06:34 OVA MikroTik:     Server-Id = 10.11.10.1
2022-11-08T17:06:35.640567+01:00 <30>Nov  8 17:06:34 OVA MikroTik: dhcp-client on bridge got IP address 10.11.10.141
2022-11-08T17:06:35.640631+01:00 <31>Nov  8 17:06:34 OVA MikroTik: dhcp-client on bridge entering <bound> state
2022-11-08T17:06:37.048188+01:00 <30>Nov  8 17:06:36 OVA MikroTik: user jotne logged in from 10.11.10.32 via winbox
2022-11-08T17:06:37.231730+01:00 <30>Nov  8 17:06:36 OVA MikroTik: local query: #1 upgrade.mikrotik.com. A
2022-11-08T17:06:37.260367+01:00 <30>Nov  8 17:06:36 OVA MikroTik: done query: #1 upgrade.mikrotik.com 159.148.172.226
Without BSD
2022-11-08T17:08:43.989247+01:00 <13>dhcp,debug,packet MikroTik:     Client-Id = 01-00-0C-AA-9B-EA-66
2022-11-08T17:08:43.989411+01:00 <13>dhcp,debug,packet MikroTik: dhcp-client on bridge received ack with id 1310283069 from 10.11.10.1
2022-11-08T17:08:43.989411+01:00 <13>dhcp,debug,packet MikroTik:     flags = broadcast
2022-11-08T17:08:43.989506+01:00 <13>dhcp,debug,packet MikroTik:     ciaddr = 0.0.0.0
2022-11-08T17:08:43.989571+01:00 <13>dhcp,debug,packet MikroTik:     yiaddr = 10.11.10.141
2022-11-08T17:08:43.989619+01:00 <13>dhcp,debug,packet MikroTik:     siaddr = 10.11.10.1
2022-11-08T17:08:43.989698+01:00 <13>dhcp,debug,packet MikroTik:     chaddr = 00:0C:AA:9B:EA:66
2022-11-08T17:08:43.989748+01:00 <13>dhcp,debug,packet MikroTik:     Subnet-Mask = 255.255.254.0
2022-11-08T17:08:43.989859+01:00 <13>dhcp,debug,packet MikroTik:     Router = 10.11.10.1
2022-11-08T17:08:43.990017+01:00 <13>dhcp,debug,packet MikroTik:     Domain-Server = 10.11.10.1
2022-11-08T17:08:43.990017+01:00 <13>dhcp,debug,packet MikroTik:     NTP-Server = 10.11.10.1
2022-11-08T17:08:43.990017+01:00 <13>dhcp,debug,packet MikroTik:     Address-Time = 86400
2022-11-08T17:08:43.990114+01:00 <13>dhcp,debug,packet MikroTik:     Msg-Type = ack
2022-11-08T17:08:43.990203+01:00 <13>dhcp,debug,packet MikroTik:     Server-Id = 10.11.10.1
2022-11-08T17:08:43.990315+01:00 <13>dhcp,info MikroTik: dhcp-client on bridge got IP address 10.11.10.141
2022-11-08T17:08:43.990362+01:00 <13>dhcp,debug,state MikroTik: dhcp-client on bridge entering <bound> state
As you see with BSD, you do not see what module data belongs to, but it sends the router name as extra info.
Without BSD it sends DHCP, DEBUG, PACKET

It would help allot if an ID was sent as part of the DHCP request (or other logs that are multiple lines) to make sure what belongs togeather.
 
belanger
just joined
Posts: 13
Joined: Sat Feb 19, 2022 5:03 am

Re: Logging prefix is a mess

Wed Jan 18, 2023 11:54 pm

Using telegraf I tried BSD or non-BSD logging - it's not working. Telegraf complains.

2023-01-18T21:52:38Z E! [inputs.syslog] Error in plugin: expecting an alphanumeric tag (max 32 characters) [col 34]

Only mikrotik gives me such trouble. Anything changed in syslog since the 90s? :D

Unless someone has an idea, guess I'll open a ticket w/ mikrotik. Thank!
 
belanger
just joined
Posts: 13
Joined: Sat Feb 19, 2022 5:03 am

Re: Logging prefix is a mess

Thu Jan 19, 2023 12:03 am

Holy crap - I just noticed this thread started years ago :shock: Wish I knew this before buying any mikrotik product.

Do I get this right, Mikrotik thinks logging isn't important? Is there a second option that I'm not aware? Cloud logging perhaps?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Thu Jan 19, 2023 12:52 am

I did make a support request for it some times ago, and MT responded that they sad that they will look at it.
In may year of making Splunk for Mikrotik, I have worked around it.

But it would love that MT fixes this.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10183
Joined: Mon Jun 08, 2015 12:09 pm

Re: Logging prefix is a mess

Thu Jan 19, 2023 11:45 am

It probably is a "work in progress". That is MikroTik language for: we know that it is important but we do not have the resources to work on it.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Sun Jan 22, 2023 3:48 pm

As of comment in 7.8beta post. No need to change it, just add a new logging option that do follow a more standard format.

DHCP logs are another system that fails that are in the same range. SUP-105355
For the logging I always like to get, host_name and class_id (MSFT 5.0 etc). That can you can get when enable DHCP debug message.
A complete set of one DHCP request looks like this (with timestamp not shown)
dhcp,debug MikroTik: DHCP-vlan20-Guest received request id 2761716067 from 0.0.0.0 '1:c4:5d:83:59:7c:41'
dhcp,debug,packet MikroTik:     flags = broadcast
dhcp,debug,packet MikroTik:     ciaddr = 0.0.0.0
dhcp,debug,packet MikroTik:     chaddr = C4:5D:83:59:7C:41
dhcp,debug,packet MikroTik:     Host-Name = "GalaxyWatch3-CC40"
dhcp,debug,packet MikroTik:     Address-Request = 10.10.20.142
dhcp,debug,packet MikroTik:     Msg-Type = request
dhcp,debug,packet MikroTik:     Server-Id = 10.10.20.1
dhcp,debug,packet MikroTik:     Parameter-List = Subnet-Mask,Router,Interface-MTU,Auto-Proxy-Config,NTP-Server,Domain-Name,Domain-Server,Host-Name
dhcp,debug,packet MikroTik:     Max-DHCP-Message-Size = 576
dhcp,debug,packet MikroTik:     Client-Id = 01-C4-5D-83-59-7C-41
dhcp,debug MikroTik: lease offered, addressed to me
dhcp,info MikroTik: DHCP-vlan20-Guest assigned 10.10.20.142 for C4:5D:83:59:7C:41 GalaxyWatch3-CC40
dhcp,debug MikroTik: DHCP-vlan20-Guest sending ack with id 2761716067 to 255.255.255.255
dhcp,debug,packet MikroTik:     flags = broadcast
dhcp,debug,packet MikroTik:     ciaddr = 0.0.0.0
dhcp,debug,packet MikroTik:     yiaddr = 10.10.20.142
dhcp,debug,packet MikroTik:     siaddr = 10.10.20.1
dhcp,debug,packet MikroTik:     chaddr = C4:5D:83:59:7C:41
dhcp,debug,packet MikroTik:     Subnet-Mask = 255.255.255.0
dhcp,debug,packet MikroTik:     Router = 10.10.20.1
dhcp,debug,packet MikroTik:     Domain-Server = 10.10.10.1
Using commands in Splunk or other tools I can join those message looking for first to last message in one request.
This works fine in a system with low DHCP request. But when there are many requests at more or less the same time, they will be merged together.
A solution for this would be that all message belongs to same request. Then we could just join those message to one request.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Sun Jan 22, 2023 4:07 pm

I'd add "don't use multi-line log entries" (DHCP being the worst oftener): one event, one entry.
That I do agree to. It will solve the DHCP multiline logg.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: Logging prefix is a mess

Sun Jan 22, 2023 4:20 pm

I'd add "don't use multi-line log entries" (DHCP being the worst oftener): one event, one entry.
That I do agree to. It will solve the DHCP multiline logg.
LOL. Yes, your example above is what I was talking about. It clutters the log when reading for a single client's thing & no doubt must be a nightmare to parse.

They have multi cores CPUs, so violating "one event, one entry", make matching lines impossible in some cases no doubt.
 
txfz
Frequent Visitor
Frequent Visitor
Posts: 57
Joined: Tue Mar 10, 2020 9:02 am

Re: Logging prefix is a mess

Thu Feb 23, 2023 12:15 pm

I've been going back and forth trying different log systems, and always get reminded by the useless log format of RouterOS. At least map the severity levels automatically instead of requiring a fixed value. I could work around that by creating separate log actions, but that's stupid.
 
Wublide
newbie
Posts: 30
Joined: Sun Feb 18, 2018 11:00 pm

Re: Logging prefix is a mess

Thu Feb 23, 2023 1:56 pm

I've been going back and forth trying different log systems, and always get reminded by the useless log format of RouterOS. At least map the severity levels automatically instead of requiring a fixed value. I could work around that by creating separate log actions, but that's stupid.

on my syslog server all logs are under notice severity/level
 
pe1chl
Forum Guru
Forum Guru
Posts: 10183
Joined: Mon Jun 08, 2015 12:09 pm

Re: Logging prefix is a mess

Thu Feb 23, 2023 2:40 pm

That is not realted to the topic, it is the broken syslog agent that only allows the raw message to be sent including its topics, or the "bsd format" with a fixed facility and severity can be selected but the topics are not sent. We need a setting inbetween that.
 
txfz
Frequent Visitor
Frequent Visitor
Posts: 57
Joined: Tue Mar 10, 2020 9:02 am

Re: Logging prefix is a mess

Thu Feb 23, 2023 4:48 pm

To actually contribute constructively I would like to see support for RFC 5424, with severity levels mapped probably to their corresponding syslog level. Use preferrably one topic for the application name.

https://www.rfc-editor.org/rfc/rfc5424

Edit: Ok, my bad, the severity level does get set automatically when using the BSD Syslog option. Though, as mentioned, some other data is omitted instead.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Logging prefix is a mess

Mon Feb 19, 2024 2:42 pm

Edit: Ok, my bad, the severity level does get set automatically when using the BSD Syslog option. Though, as mentioned, some other data is omitted instead.
Example for logs not using BSD syslog (log part in red are tags I do add to the logs)
dns serial=yyyyyyy MikroTik: done query: #56718 brb.duolingo.com. 54.230.111.35
Example for logs using BSD syslog
Feb 19 13:24:32 RB951 serial=xxxxxxxx MikroTik: done query: #16955 brb.duolingo.com. 54.230.111.35
As you see, non-BSD add what module that do sends the logs. This is a must since there are many types of modules sending logs.

The BSD on the other hand removes the module information but adds the Time of log and identity of the router.

@ Mikrotik. Please fix this and use RFC 5424. What can we to to help?

Who is online

Users browsing this forum: freemannnn and 67 guests