Page 2 of 5

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:00 pm
by Hunty
I've attached two screenshot

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:03 pm
by Hunty
please note that I've inserted "Mikrotik" under System/Logging

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:13 pm
by Jotne
All apps needs to be in
$SPLUNK_HOME/etc/apps
So on windows you should have:
C:\Program Files\Splunk\etc\apps\MikroTik

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:14 pm
by Hunty
All apps needs to be in
$SPLUNK_HOME/etc/apps
So on windows you should have:
C:\Program Files\Splunk\etc\apps\MikroTik
Yes

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:16 pm
by Jotne
Uninstall everything.
Install follow post #1 step by step.

You should also see
C:\Program Files\Splunk\etc\apps\MikroTik\default
C:\Program Files\Splunk\etc\apps\MikroTik\metadata
etc
Not
C:\Program Files\Splunk\etc\apps\MikroTik\MikroTik\default
If that does not work, I will try to do an install my self from the #1 post and test it.

PS no need to quote post above you.

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:23 pm
by Hunty
are you sure?
I've attached a screenshot of the content of the folder

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:43 pm
by Jotne
You have restarted Splunk after app install?
All looks correct.

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:50 pm
by Hunty
Yes several times

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 8:55 pm
by Jotne
You have installed Splunk free as in post #1, or do you use Splunk before to some else?
I have seen problem with installed version that using other index.

From the picture above, it does not seem that splunk does the filed extraction.

If you like, I can try teamviewer to see what is wrong.
Not able to post a private message to you, so post an email so I can get in touch with you.

Re: Using Splunk to analyse MikroTik logs 2.3 (Graphing everything)

Posted: Thu Nov 15, 2018 9:08 pm
by Hunty
thanks for your help, but I'll try tomorrow with the linux VM but I've to solve first why the 514 port is not available even if I followed your guide to install the app with a non root user

Re: Using Splunk to analyse MikroTik logs 2.4 (Graphing everything)

Posted: Tue Nov 20, 2018 12:20 pm
by Jotne
2.4 Released

Nearly all code are rewritten to get better speed and make it cleaner.
Dark Theme makes a big visual change.

# v2.4 (20.11.2018)
# Updated "MikroTik Hotspot login/logout information" to show IP
# Fixed when inn interface= unknown
# Updated view 2.4 to handel more hits
# Updated "MikroTik DNS" to not view revers lookup "site!=*.in-addr.arpa"
# Rewritten "Microtik Traffic" Error in all calculation
# Fixed data rounding and fixed typo
# Fixed formating in "MikroTik Remote Connection"
# Set permission view the view to show in app only
# Added System Changes as a new default menu
# Fixed missing host in "MikroTik Uptime"
# Added Host to "MikroTik Traffic"
# Added view "MikroTik Wifi strength"
# Added view "MikroTik System Changes"
# Dark theme needs >=7.2
# Removed global time (use default time)
# Removed searchWhenChanged="true" (default)
# Cleaned code
# Fixed error in "13. OSCam config changes"
# Added Sprakline to "MikroTik Device List"
2.4 Device list.jpg
.
2.4 System Changes.jpg
.
2.4 Traffic.jpg

Re: Using Splunk to analyse MikroTik logs 2.4 (Graphing everything)

Posted: Sat Nov 24, 2018 12:06 am
by jareckib12
Hi,
First - thx for update.
Second - in MikroTik DNS request view, client filtering does not work. When selecting any item in addition to "any" does not show any results.

Jarecki

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sun Nov 25, 2018 10:40 am
by Jotne
Good catch,

I have updated to 2.5

# 2.5 (25.11.2018)
# Change all "if" test to use "coalesce"
# Fixed error in "MikroTik DNS request"
# Moved more to base search
# Removed some code not needed in "MikroTik Web Proxy"
# Fixed error with src_port in dest_ip dropdown in "MikroTik Firewall Rules"

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sun Nov 25, 2018 9:26 pm
by sherics
Hello,

I have installed it completely and except the Traffic, everything work.

In the traffic I see just few MBs, even if I download 500MB or 1GB, it does not shows up there, just few % of the downloaded amount.

I do not have a public IP on my internal network, the public IP is on the WAN port, ether1, as a standard home router, other clients are on WiFi on first VLAN and 2 computers on second VLAN.

Do you have an idea what is wrong?

Thank you.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Nov 26, 2018 6:08 pm
by Jotne
I did download an 1GB file from here: http://www.ovh.net/files/
And it showed up correctly.

Do you have Fasttrack on?
If so try to disable it, it may be that packed are not accounted when Fasttrack is on.
https://www.youtube.com/watch?v=6LaqhDm6PHI

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Nov 26, 2018 8:45 pm
by MSandoval
Hello Jotne and the whole community.
First I want to tell you, good job, really good jobs, and thanks for sharing with us Jotne.

Secondly I have a question, in version 2.4 I see in the record that wrote "List of devices" this function indicates that it already supports multi-router log ?, in such case as it is identified in each module to which router belongs each record?

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Nov 26, 2018 9:32 pm
by Jotne
You are correct, it does support as many routers as you like to add.
Going away from SNMP to Syslog only was driven by the simpler way to do thing.
With SNMP, you need to set up the monitor system to request SNMP from the device.
This is ok for singel router ans small system.
But if you like to monitor a router across public internet, you end up in a security risk by open for SNMP.

Whit using script and Syslog this is a one way communication. All data are sent from the device to the monitor system.
No need to open ports. Same script for all routers. No need to configure any configuration on the monitoring system for each router.

I have four routers/host that sends log to my sentral log server.

On every view you can select host to view only that host.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Nov 26, 2018 9:41 pm
by MSandoval
Great, you're right, forget that each module has a drop-down menu Hosts. I'm going to try it and anything I write. Thanks again.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Nov 26, 2018 10:59 pm
by sherics
I did download an 1GB file from here: http://www.ovh.net/files/
And it showed up correctly.

Do you have Fasttrack on?
If so try to disable it, it may be that packed are not accounted when Fasttrack is on.
https://www.youtube.com/watch?v=6LaqhDm6PHI
Well, I forgot about fastrack... without fastrack it works now, but unfortunately without fastrack my router is on 95-99% CPU while I download/upload anything; and the speed is lowered for 300mbit/s... With fastrack enable, the cpu is approx on 70% on full gigabit connection, about 90MB/s real speed. Well, after 4 years, I think, I need to purchase a more powerul and new router :)

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Nov 26, 2018 11:22 pm
by Jotne
It may also be that you could configure your router to use hardware offloading. Depending on type and software version.
But old boxes do have less power so upgrade may be the only option.

Its a good point to now that traffic monitoring does not work when fast track is enabled, so I will mention that in the first post.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Fri Nov 30, 2018 3:45 pm
by MSandoval
Hello everyone, I have a problem with module MikroTik_Traffic section Public IP. when reviewing this, I found a small error when declaring the variable host, in this case that variable is capitalized Host, it does that the section does not work, changing this I achieved that it works correctly.
<title>Public IP</title>
        <search base="base_search">
          <query>
            search
              Host="$Host$"         >>>   change with host="$Host$"
            | eval ip_in=if("$direction$"=="in",src_address,dst_address)
            | eval ip_out=if("$direction$"!="in",src_address,dst_address)
Image

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Fri Nov 30, 2018 5:28 pm
by Jotne
Thanks for the feedback :)

It will be fixed in 2.6. For others you can edit det file and correct the typo.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Fri Dec 07, 2018 4:18 am
by ariefwido
Hello there,

This is my first time using splunk and I have no result on dashboard anyway also I did every step on the post #1, any idea why this happen?
The logs already show up on the splunk but the MikroTik app dashboard have no result at all.

Thanks and appreciate your help.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Fri Dec 07, 2018 11:51 pm
by Jotne
/system logging add action=logserver prefix=MikroTik topics=dhcp
/system logging add action=logserver prefix=MikroTik topics=!debug
I would guess you have typed wrong prefix. Any other word than MikroTik would brake the index of the data.
Make sure its 100% equal with capital M and K

Cut and Past is the best option to get it correct.

Do a search like this in Splunk, change to your MikroTik Routers IP, what is the output?
index=* host=192.168.88.1 | rex "^\S+\s(?<prefix>\S+)\s" | stats count by prefix

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 1:23 am
by ariefwido
Hello,

I did copy and paste that command on cli.

The result prefix search on attachment
Search MT.JPG
And then I found something that on the search section if I remove module=xxx then I got the result on the dashboard.
For the example on the device list dashboard I use this
No Module.JPG
instead of your originally script
With Module.JPG
I think that module=xx didn't work on my splunk search. Any idea?

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 9:38 am
by Jotne
Strange.

Can you post output of sourcetype=mikrotik script=sysinfo
Make sure you have Smart Mode selected (see circle on picture)
Click the arrow to expand one post so I see the extraction. >
.
test_output.jpg

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 10:33 am
by ariefwido
Hi Jotne,

Here is the output and just different from yours.
test_output_1.JPG

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 12:26 pm
by Jotne
I see two strange things.
1. It seems that Splunk does not handle the date/time correctly since its shown within your event.
2.I do not see the information from router that shows where it comes from and type (ipsec/DNS/DHCP) (debug packets)

Is this a clean Splunk installation, followed the steps above?

You are running on a 951G a common box, I have a 941 and 750Gr3 and some other.
Your RouterOS software 6.43.4 is the same as I do run, so should be ok

Can you post the last lines of the output on the Router of /log print and /log print detail
Just cut and paste the line, so I do see how it looks like.

On mine
11:21:32 script,info script=pool pool=default-dhcp used=1 total=245
and
time=11:21:32 topics=script,info
message="script=pool pool=default-dhcp used=1 total=245"
I do miss the stuff in bold from your logg message and would like to see how it looks like on the router to compare what Splunk sees.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 12:52 pm
by ariefwido
Hi Jotne,

Yes, this is fresh install splunk and I did several time remove my VM and install again to make sure that.

Here is the output
/log print

17:47:19 firewall,info FW_INTERNAL forward: in:PJX out:BRX-LAN, proto TCP (ACK), 10.99.100.102:7332->10.121.61.108:52380, len 40

/log print detail

time=17:48:19 topics=firewall,info message="FW_INTERNAL forward: in:PJX out:BRX-LAN, proto TCP (ACK), 10.99.100.102:7332->10.121.61.108:52380, len 40"

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 1:09 pm
by Jotne
This looks correct, so it have to be some wrong with Splunk implementation since message looks different there.
Several other has used this, so should not be an big error in the code.

If you tyoe index=* in splunk, do you see any message that have the module tag coming from the router?

Like this
firewall,info
PS If you set time to: real time 1-minute window you should see data live as they arrive.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 1:28 pm
by ariefwido
Unfortunately I didn't see that message on my splunk,
test_output_2.JPG
Any idea what is happening on my splunk?

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 2:10 pm
by Jotne
You have some strange in your message that I have not see with other: RTZPKN02

Can you post this? /system logging export

How did you install the files in Splunk?

Why do you get Des 9 in your log, I am still at Des 8?
Your logs has two different time stamp.
See if all clock is equal everywhere. Router, Computer ++

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 2:23 pm
by ariefwido
Sorry the Dec 9 is from date server, I already change the NTP :D

Here is the output
 /system logging export
# dec/08/2018 19:21:37 by RouterOS 6.43.4
# software id = 29W1-FTPT
#
# model = 951G-2HnD
# serial number = 642E05A9020A
/system logging action
add name=syslog remote=10.99.100.77 remote-port=7514 src-address=10.122.82.200 \
    target=remote
add bsd-syslog=yes name=logserver remote=10.100.10.105 src-address=\
    10.122.82.200 target=remote
/system logging
add action=syslog disabled=yes topics=info,error,interface,warning
add action=logserver prefix=MikroTik topics=dhcp
add action=logserver prefix=MikroTik topics=!debug

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 2:44 pm
by Jotne
Seems that you do not have a clean install.
You are logging to several system at the same time.
It should work.

Try this: Remove all logg line and add this:
/system logging action
add name=myserver remote=10.100.10.105 target=remote
/system logging
add action=myserver prefix=MikroTik topics=!debug
add action=myserver prefix=MikroTik topics=dhcp
Is this your Splunk server? 10.100.10.105
If not, do you relay your message (rslyslog or other server)?
Do you send your log message passing trough several routers?

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 3:45 pm
by ariefwido
Seems that you do not have a clean install.
You are logging to several system at the same time.
It should work.

Try this: Remove all logg line and add this:
/system logging action
add name=myserver remote=10.100.10.105 target=remote
/system logging
add action=myserver prefix=MikroTik topics=!debug
add action=myserver prefix=MikroTik topics=dhcp
Is this your Splunk server? 10.100.10.105
If not, do you relay your message (rslyslog or other server)?
Do you send your log message passing trough several routers?
Ok I will reinstall my splunk VM again and change all log line and I will tell you the result

And yes my log message passing through several routers.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 3:46 pm
by Jotne
But is this your Splunk server? 10.100.10.105
Or do you send data to an rsyslog or other syslog server, that then sends it to your Splunk server?

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Sat Dec 08, 2018 5:30 pm
by ariefwido
Hi Jotne,

It seems I found the problem, the problem is marking the BSD Syslog on log remote action.
test_output_3.JPG
Finally the result is come.

Thanks and very appreciate your help.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Tue Dec 11, 2018 1:53 pm
by WeWiNet
Hi Jotne,

Wanted to say thank you, very nice job.
Also to highlight that this tutorial works perfect on MacOS 10.14.
I just followed your tutorial and installed it with the Splunk Enterprise version
and all is working perfect (Ok I had to restart my machine once as splunk did not launch first time correctly).

I now try to make sense out of all that data and nice graphs ... :-)

PS: How can you know how much data you log per day (which is the limitation of the free version)?

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Tue Dec 11, 2018 2:52 pm
by Jotne
Thanks.

You find license information her:

Settings->Licensing
There you see this for free version
Licensed daily volume 500 MB

Select:
Usage-Report->Previous 30 days

Here you will see how much of the license you use each day, last 30 days.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Tue Jan 01, 2019 11:47 pm
by zandhaas
Thank You for this post and all the work to get all the information in Graphs.

Only I had a hard time to get all the information in Splunk.
After three hours of trying a lot of different things I finaly discoverd that I missed the last "}" in the Router script.

Perhaps you can change the post where the script is too make the beginning and end of the script more clear.

Regards Peter

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Wed Jan 02, 2019 5:50 pm
by Jotne
Thanks for the feedback.
Added some space in the script to make it better to see start end.

Next time you can click Select ALL, behind the Code: at the top of the script and you get all that is needed.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Thu Jan 03, 2019 10:37 am
by Hunty
Thank You for this post and all the work to get all the information in Graphs.

Only I had a hard time to get all the information in Splunk.
After three hours of trying a lot of different things I finaly discoverd that I missed the last "}" in the Router script.

Perhaps you can change the post where the script is too make the beginning and end of the script more clear.

Regards Peter
I had the same problem,
now everything works

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Thu Jan 03, 2019 11:06 am
by Hunty
I'm seeing two problems:
The script reports a cpu higher than usual, it detects the cpu loads when the scripts is running, so instead of reading a normal 10% load, it reads a load near to 100%

The second is the Disk graph.
I've attached two screenshot

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Thu Jan 03, 2019 11:21 am
by Hunty
I've modified the script in order to read the cpu load at the beginning, now the readings are correct
# This script is used to send data to Splunk using syslog.
#===================================

# Collect system resource
# ----------------------------------
:local cpuload ([/system resource get cpu-load]);
:local freemem ([/system resource get free-memory]/1000000);
:local totmem ([/system resource get total-memory]/1000000);
:local freehddspace ([/system resource get free-hdd-space]/1000000);
:local totalhddspace ([/system resource get total-hdd-space]/1000000);
:local up ([/system resource get uptime]);
:log info message="script=resource free_memory=$freemem MB total_memory=$totmem MB free_hdd_space=$freehddspace MB total_hdd_space=$totalhddspace MB cpu_load=$cpuload uptime=$up";



# Collect accounting traffic
# ----------------------------------
# Take a snapshoot
if ([/ip accounting get enabled]=yes) do={
/ip accounting snapshot take
# Send data to loggin server
foreach logline in=[/ip accounting snapshot find] do={:log info message="$[/ip accounting snapshot print as-value from=$logline]"}};

# Finding dynmaic lines used in uPnP
# ----------------------------------
:foreach logline in=[/ip firewall nat find dynamic=yes] do={:log info message="$[/ip firewall nat print as-value from=$logline]"};

# Collect system information
# ----------------------------------
:local version ([/system resource get version]);
:local board ([/system resource get board-name]);
:local model ([/system routerboard get model]);
:local serial ([/system routerboard get serial-number]);
:log info message="script=sysinfo version=\"$version\" board-name=\"$board\" model=\"$model\" serial=$serial";

# Collect system health
# ----------------------------------
:local voltage ([/system health get voltage]/10);
:local temperature ([/system health get temperature]);
:log info message="script=health voltage=$voltage V temperature=$temperature C";

# Sends wireless client data to log server
# ----------------------------------
:foreach logline in=[/interface wireless registration-table find] do={:log info message="$[/interface wireless registration-table print  as-value from=$logline]"};

# Collect DHCP Pool information
# ----------------------------------
/ip pool {
   :local poolname
   :local pooladdresses
   :local poolused
   :local minaddress
   :local maxaddress
   :local findindex
   :local tmpint
   :local maxindex


 #  :put ("IP Pool Statistics")
 #  :put ("------------------")

# Iterate through IP Pools
   :foreach p in=[find] do={

      :set poolname [get $p name]
      :set pooladdresses 0
      :set poolused 0


#   Iterate through current pool's IP ranges
      :foreach r in=[:toarray [get $p range]] do={

#      Get min and max addresses
         :set findindex [:find [:tostr $r] "-"]
         :if ([:len $findindex] > 0) do={
            :set minaddress [:pick [:tostr $r] 0 $findindex]
            :set maxaddress [:pick [:tostr $r] ($findindex + 1) [:len [:tostr $r]]]
         } else={
            :set minaddress [:tostr $r]
            :set maxaddress [:tostr $r]
         }

#       Convert to array of octets (replace '.' with ',')
         :for x from=0 to=([:len [:tostr $minaddress]] - 1) do={
            :if ([:pick [:tostr $minaddress] $x ($x + 1)] = ".") do={
               :set minaddress ([:pick [:tostr $minaddress] 0 $x] . "," . \
                                       [:pick [:tostr $minaddress] ($x + 1) [:len [:tostr $minaddress]]]) }
         }
         :for x from=0 to=([:len [:tostr $maxaddress]] - 1) do={
            :if ([:pick [:tostr $maxaddress] $x ($x + 1)] = ".") do={
               :set maxaddress ([:pick [:tostr $maxaddress] 0 $x] . "," . \
                                       [:pick [:tostr $maxaddress] ($x + 1) [:len [:tostr $maxaddress]]]) }
         }

#      Calculate available addresses for current range
         :if ([:len [:toarray $minaddress]] = [:len [:toarray $maxaddress]]) do={
            :set maxindex ([:len [:toarray $minaddress]] - 1)
            :for x from=$maxindex to=0 step=-1 do={
#             Calculate 256^($maxindex - $x)
               :set tmpint 1
               :if (($maxindex - $x) > 0) do={
                  :for y from=1 to=($maxindex - $x) do={ :set tmpint (256 * $tmpint) }
               }
               :set tmpint ($tmpint * ([:tonum [:pick [:toarray $maxaddress] $x]] - \
                                                    [:tonum [:pick [:toarray $minaddress] $x]]) )
               :set pooladdresses ($pooladdresses + $tmpint)
#         for x
            }

#      if len array $minaddress = $maxaddress
         }

#      Add current range to total pool's available addresses
         :set pooladdresses ($pooladdresses + 1)

#   foreach r
      }

          :set poolused [:len [used find pool=[:tostr $poolname]]]
#   Send data
    #      :log info message=("pool=" . $poolname  . " used=" . $poolused . " total=" . $pooladdresses)
          :log info message=("script=pool pool=$poolname used=$poolused total=$pooladdresses")

# foreach p
   }
# /ip pool
}

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Thu Jan 03, 2019 3:10 pm
by Jotne
Good idea moving the cpu reading to the top. I have updated fist post view new version.
PS mine du not give much difference in CPU when script is running. Maybe you device is some under-powered or you have som wrong in your configuration (fasttrack or hw acceleration missing)

I see that MB is wrongly reported due to dividing on 1000000 and not 1048576 (1024*1024). Corrected in the script.
Since graph is in percentage it should not make any change to the view.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Fri Jan 18, 2019 3:08 am
by fengyuclub
Good idea moving the cpu reading to the top. I have updated fist post view new version.
PS mine du not give much difference in CPU when script is running. Maybe you device is some under-powered or you have som wrong in your configuration (fasttrack or hw acceleration missing)

I see that MB is wrongly reported due to dividing on 1000000 and not 1048576 (1024*1024). Corrected in the script.
Since graph is in percentage it should not make any change to the view.
No matter how it is set, my splunk can't get my ccr1016 data. Splunk is a virtual machine ubuntu server in the LAN, ccr on what you said, every 5 minutes running script can see a lot of log information generated, but still no data into splunk, there is nothing I did not do Is it? Please forgive my english, from google translation.

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Fri Jan 18, 2019 8:29 am
by Jotne
No need to quote message above you, only part of it when needed. Always use Post Reply button under the post.

Are you 100% sure you have tagged the packet with MikroTik? There are no firewall?
Try in the search page and search for a star last 15 min. Do you get any?
*

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Jan 21, 2019 9:50 am
by Hunty
After three hours of trying a lot of different things I finaly discoverd that I missed the last "}" in the Router script.
Check this!

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Jan 21, 2019 3:30 pm
by zandhaas
I'm Using Splunk for a couple of weeks now.

In the Firewall Rule section I see beside the attacks from the large big spooky internet also local adresses appear as a result of the "FW_Drop_All_From_Wan" rules.
and that are mainly request with dest_port 53 (DNS).

Is it possible to filter out the local addresses?

Re: Using Splunk to analyse MikroTik logs 2.5 (Graphing everything)

Posted: Mon Jan 21, 2019 6:38 pm
by Jotne
You should not see local address in the outside inn block rule.

Can you post an example like this:
2019-01-21 17:25:25	FW_Drop_all_from_WAN	input	ether1-Wan	(unknown 0)	00:05:00:01:00:01	TCP	104.131.145.9	45167	92.31.200.211	2082	San Francisco	United States
And yes, you can get rid of the message in two ways.
1. Add a rule on the fw above the outside in block rule that block the specific ip/port of your choice.
2. Modify Splunk to exclude the ip/port you like.

First is the simplest solution.