Community discussions

MikroTik App
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 03, 2024 5:11 pm

I'm running Splunk on a Synology too, but as a VM under Ubuntu Linux, not containerized.
Works OK in general, had 1 or 2 occasions where the 4GB assigned memory fell short and things fell apart ;-)

>> After a while the logging to splunk stops ...

Splunk generates a ton of logging messages that might give you an indication why something "stops" working. Did you check any of these ?
(with a container, you'll have to open a shell I guess)

/opt/splunk/var/log/splunk

Are you not exceeding the 500Mbytes daily limit ??
Top menu "Settings" then "Licensing" (under the "System" section)
Hi jvanhambelgium
Did you find anything could help resolve this error?
I never had an issue. This is my/a response somebody else.
Just make sure you do not exceed the 500MByte limit on daily basis or Splunk will stop logging.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3274
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 03, 2024 8:51 pm

Just a tip.
You can request a free 10GB/day license (Developer License) from Splunk. It will give you all function on Splunk with 10GB/day compare to 500MB/day and limited functions (no alerts, no cluster +++) . Only down side is that you need to request a new license every 6 month.

https://dev.splunk.com/enterprise/dev_license/
 
jult
Frequent Visitor
Frequent Visitor
Posts: 52
Joined: Sat Dec 26, 2020 1:16 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 05, 2024 5:04 pm

But this is a remote, off-premise, storage/processing option. Nice, but that would cost you extra data/traffic to/from your WAN as well, and I don't think that's a good idea. It would even interfere with all the intended/normal traffic.
 
mooglez
just joined
Posts: 3
Joined: Mon Jan 22, 2024 4:10 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jan 22, 2024 4:17 pm

Just installed this to try out today.

Running Splunk 9.1 on Windows 10. Currently have log events for few hours in Splunk.

When I go to the dashboard "MikroTik DNS requests", resource usage goes absolutely wild.
It's basically consuming all available RAM and CPU for ~10 minutes.

I also noticed that many of the other dashboards are also quite slow to load, but don't consume everything for a long time.
Any idea what might be going wrong here?
 
mooglez
just joined
Posts: 3
Joined: Mon Jan 22, 2024 4:10 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Tue Jan 23, 2024 3:29 pm

Search job inspector results for a "last 15 minutes" search in the "MikroTik DNS requests" dashboard:
This search has completed and has returned 118 results by scanning 243 events in 223.991 seconds

The following messages were returned by the search subsystem:

info : Search finalized.
info : The term '"dns* query from*#"' contains a wildcard in the middle of a word or string. This might cause inconsistent results if the characters that the wildcard represents include punctuation. Learn More 
(SID: admin__admin__MikroTik__RMD5ecf8a6ae83683ff5_1706015926.479) search.log Job Details Dashboard

Execution costs
Duration (seconds)	Component	Invocations	Input count	Output count
0.00	 command.eval	6	236	236
0.00	 command.fields	6	236	236
46.92	 command.lookup	3	118	118
0.05	 command.postprocess	1	118	118
0.00	 command.presort	3	118	118
0.23	 command.search	6	118	236
0.09	 command.search.expand_search	2	-	-
0.00	 command.search.calcfields	2	243	243
0.00	 command.search.evalfilter	2	243	243
0.00	 command.search.expand_search.calcfield	2	-	-
0.00	 command.search.expand_search.fieldaliaser	2	-	-
0.00	 command.search.expand_search.indexed_fields	2	-	-
0.00	 command.search.expand_search.kv	2	-	-
0.00	 command.search.expand_search.lookup	2	-	-
0.00	 command.search.expand_search.sourcetype	2	-	-
0.00	 command.search.fieldalias	2	243	243
0.00	 command.search.filter	2	243	118
0.00	 command.search.index	5	-	-
0.00	 command.search.index.usec_1_8	272	-	-
0.00	 command.search.index.usec_512_4096	2	-	-
0.17	 command.search.lookups	2	243	243
0.05	 command.search.rawdata	2	-	-
0.02	 command.search.kv	2	-	-
0.00	 command.search.parse_directives	2	-	-
0.00	 command.search.summary	3	-	-
0.00	 command.search.tags	2	118	118
0.00	 command.search.track_sourcetypes	3	-	-
0.00	 command.search.typer	2	118	118
0.00	 command.sort	1	50,000	118
0.02	 command.timeliner	1	118	118
0.08	 dispatch.check_disk_usage	5	-	-
0.00	 dispatch.createdSearchResultInfrastructure	1	-	-
0.00	 dispatch.evaluate.eval	4	-	-
0.00	 dispatch.evaluate.fields	2	-	-
0.00	 dispatch.evaluate.lookup	2	-	-
0.09	 dispatch.evaluate.search	2	-	-
0.00	 dispatch.evaluate.sort	2	-	-
37.25	 dispatch.fetch.rcp.phase_0	5	-	-
0.00	 dispatch.finalWriteToDisk	1	-	-
47.16	 dispatch.localSearch	1	-	-
176.34	 dispatch.preview.snapshot	5	-	-
0.00	 dispatch.readEventsInResults	1	-	-
47.16	 dispatch.stream.local	3	-	-
0.00	 dispatch.timeline	1	-	-
0.03	 dispatch.tmpevents	2	-	-
0.29	 dispatch.writeStatus	52	-	-
0.13	 startup.configuration	2	-	-
0.70	 startup.handoff	2	-	-
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3274
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 26, 2024 3:57 pm

Splunk runs much better/faster on Linux. Its created for Linux and ported to Windows.
SSD disk are nearly a must when data is growing.

You can in the script turn off modules that your do not need or that gives problem like to much DNS (but then you will not see DNS logs)

How much do you log a day? You can see that in the Splunk License info page.
 
mooglez
just joined
Posts: 3
Joined: Mon Jan 22, 2024 4:10 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jan 29, 2024 11:16 am

Splunk runs much better/faster on Linux. Its created for Linux and ported to Windows.
SSD disk are nearly a must when data is growing.

You can in the script turn off modules that your do not need or that gives problem like to much DNS (but then you will not see DNS logs)

How much do you log a day? You can see that in the Splunk License info page.
I'm currently logging about 20 to 30M a day. 425k events in the last 24h, of which 400k are DNS.
Splunk is running on an SSD.

I was mostly wondering if there was some problem with the version of Splunk (9.1.2) I am using and the latest version of the script.
But it seems that nobody else is having issues with it, so it quite probably must be something at my end then.

My main reason for sending the logs to Splunk was to get DNS and DHCP logs over to analyze, so would really not want to disable DNS module.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3274
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Tue Jan 30, 2024 11:05 am

20-30M a day is not much so a simple server should handle that. (also a windows server)
 
JosipTopic
newbie
Posts: 43
Joined: Mon Apr 06, 2020 10:21 pm
Location: Zagreb

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 31, 2024 2:49 am

Hello, i just have a question. This link on the beginig of this thread, for download splunk app for mikrotik, is that the first one(oldest)? Where can be found updated one? Thanks?
 
snowdogging
just joined
Posts: 16
Joined: Tue Dec 20, 2016 6:23 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 31, 2024 10:56 pm

still kind of works on v7.13.3

I had to remove the capsman code. Getting error: expected end of command (line 290 column 50)
Also had to set command history to false. That portion results in a hard interruption and crash.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3274
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Feb 01, 2024 9:30 am

Hello, i just have a question. This link on the beginig of this thread, for download splunk app for mikrotik, is that the first one(oldest)? Where can be found updated one? Thanks?
What have the link do you refer to. The app that I have created under section 1g- IF so there are a link to download it, and also a git repository that always will be the latest updated.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3274
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Feb 01, 2024 9:32 am

still kind of works on v7.13.3

I had to remove the capsman code. Getting error: expected end of command (line 290 column 50)
Also had to set command history to false. That portion results in a hard interruption and crash.
Since I do not have capsmann its som hard to test for me. Will try to look at the code and see whats going wrong.
The command history should work. Has tested it on 17.3.1, but will try 17.3.3 as well.
 
snowdogging
just joined
Posts: 16
Joined: Tue Dec 20, 2016 6:23 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Feb 01, 2024 10:15 pm

Cool. Yeah I don't have capsman either so can't really help. Let me know if I can provide more detail on command history crash. I might pull the script apart to see exactly what command causes it.
 
snowdogging
just joined
Posts: 16
Joined: Tue Dec 20, 2016 6:23 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Feb 01, 2024 10:19 pm

I turned command history back on and it no longer crashes. I did manually pull the code out and ran in terminal. The crash might have something to do with the missing global "cmd" on first run.

Quick questions:
* What log prefixes besides FI_D_port-test are valid. Specifically, what types besides F? Is N nat or does it not matter?
* WireGuard Errror dashboard (sp). How do I trigger this?

Impressive app btw....thanks.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3274
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Feb 04, 2024 11:23 am

Something new in 7.13+ makes the CAPsMANN part fail, even if its run in a do={} group.
To fix this I have updated scripts to 5.5 where CAPsMANN has been separated to an external script.

If you do not like to update the script, just remove the CAPsMANN part of the script and it will work.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3274
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Feb 09, 2024 6:40 pm

Great news. v4.0 are on the way.
Most importante change is that all loggs will be tracked by a unique serial number. This way even if you have many routers behind one single nat or routers with same name, it would be easy to separate all the devices.

To prepare for the new version, you can just run (copy/past to terminal) the log update script found in 2.a It will add the routerboard serial number to the log message. If the device does not have a serial number it will create one. You do the update and the old version will still work and you are prepared for the 4.0 version that needs the serial number to work. Logs size will increase some due to the serial number adds around 18 bytes.

Script has also been updated to 5.6 where just serial number are removed from the system info part, since its part of all messages.

Hope to release 4.0 in not to long time.

Who is online

Users browsing this forum: No registered users and 2 guests