Community discussions

MikroTik App
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Tue Nov 02, 2021 11:50 am

Version 3.9 13.06.2023
.
NB, there seems to be an error with script running in routerOS 7.13.3, so upgrade to 7.13.4 or later if you have 7.13.3
.
Top_logo.jpg
Using Splunk to monitor and graph various data from our MikroTik Routers is a nice and free way to help you showing what is going on in your network.
Splunk is free to use for logging up to 500MB pr day.
You can request a 10GB/day developer license here: https://dev.splunk.com/enterprise/dev_license/

NB logging large amount of Accouning, DNS or firewall rules quickly eats up license, so I do recommend to turn off Accouning/DNS logging to start with.

Splunk can be used to monitor multiple devices. No ports needs to be opened (like with SNMP monitoring). All data are sent from the device to the Splunk monitor (using sctipt and syslog). Devices could be all around the world.

PS:
Traffic monitoring does not work correctly while fast track is enabled (and its removed in v7.x of RouterOS. Changed to Kid control). Turn fast track off and you may loose throughput, so its something you should consider when using this type of monitoring. How to disable fast track: https://www.youtube.com/watch?v=6LaqhDm6PHI

latest changes
# Added to GIT

# 3.9 (13.06.2023)
# Added discovery type to "MikroTik Neighbor"
# Added new version of "MikroTik DHCP layout"
# Added CapsMann fixes by barrthek
# Updated dhcp_table to work better with new firmat (no more debug logs)
# Changed to use dhcp_table for client name in "MikroTik Wifi connection" and "MikroTik Wifi strength"

Installation
On your PC
1) Works on Windows and Linux, but use Linux (clearly the best choice and also used in all post here)
-----------------------------------
1a) Download and install Splunk (Windows or Linux(Ubuntu recommended))
PS you need an account to download. It's free to create.
https://www.splunk.com/en_us/download/s ... prise.html
PS you need to create an account to download the file. Free to download and use (up to 500MB/day)
PS remember to set timezone on Windows/Linux, or else logging time will be wrong.


1b) PS: To install Splunk as a non root user, recommended. (needs an external syslog reciver)
Splunk setup:
viewtopic.php?t=179960#p888802
rsyslog setup
viewtopic.php?t=179960#p888803

Splunk can run as root user, but not recommended.

1c) Change to free license group. Very important to do before 30 day of use. !!!!!!!!!!!!!!!!!!!!
Web gui:
1d) Settings->licensing->Change license group->Free licnse->Save

1e) Open Windows Firewall for UDP on Windows (On linux its not blocked)
Web gui:
Start->type "adv"->Select:Widows Firewall with Advanced Security->Sect Inbound rules->Right Click "Inbound Rules">New Rule-Port-Next->UDP->Specific local ports->514->Next->Next->Next->Name "syslog"

1f) Allow UDP 514 (syslog), only if you run Splunk as root and not using rsyslog server. (rsyslog can be used when splunk runs as root or as non-root user.)
If running Splunk as non ROOT user or like to use external syslog reciever, see 1b for non-root)
Web gui:
Setting->Datainputs->Add new (behind the UDP)->Port 514->Next->Sourcetype type syslog and select syslog->Next-Submit

1g) Get the Splunk
You can use one of the two option
...1. Download the file from the forum here:
MikroTik3.9.rar
.
...2 Get the GIT repository
https://github.com/Jotne/MikroTik

1h)Installing the Splunk
...1 If you get the file form the from the forum, extract the compressed file to get the Mikrotik.spl file and install it using Splunk Gui:.
From Start page in Splunk, click the gear behind Apps or
from top meny click Apps->Manage Apps
Then select Install app from file and select the spl file

...2 Download it from GIT.
If you download it from GIT copy folder with all content to %Splunk%/etc/apps. so it becomes:
%Splunk%/etc/apps/MikroTik/default/...
%Splunk%/etc/apps/MikroTik/metadata/...
%Splunk%/etc/apps/MikroTik/README
.....

1i) A restart of Splunk may be needed.
Web gui:
Settings->Server controls->Restart Splunk

1j) Upgrade form previous version.
Some time files are renamed, so if you have not change any original files, just delete the MikroTik folder.
No logged data will be deleted.
If you have custom dashboards, menus, saved search (reports) etc, you need to merge the configuration files.
They are normal stored in "local" folder.


2a) Syslog
You need to make your Router able to send Syslog messages.

Cut and past the following to a terminal window.

Set the log server: (change IP to your log server)
/system logging action add name=logserver target=remote remote=192.168.1.50 remote-port=514
Sets the correct tag MikroTik so data can be picked up. It also set the RouterBoard serial to all packet so multiple routers behind nat can be identified. If its not a routerBard and do not have a serial, a random hex number of 12 digits are set,
{
:local serial na
:foreach id in=[/system/logging/find action prefix~"MikroTik"] do={
/system logging remove numbers=$id
}
:do {
	:set serial ([/system routerboard get serial-number])
} on-error={
	:set serial ([:rndstr from=ABCDEF0123456 length=12])
}
# Log hotspot at debug level
/system logging add action=logserver prefix="serial=$serial MikroTik" topics=hotspot

# Do not log debug, packet nor snmp, but all other will be logged.
/system logging add action=logserver prefix="serial=$serial MikroTik" topics=!debug,!packet,!snmp

# Do not log info internal logs, only warning and error logs
/system/logging/disable numbers=[/system logging/find where topics~"info"] 
}
PS Do NOT select BSD Syslog. It will mess up the logging format. BSD adds date and severity, but in the same time remove the prefix that tells what modules that do send logs to syslog.


2b) Select what rules to log
NB Do not use more than 20 charters, or else it start to clip other part of the log!!!!!!!!!!!
To log the Firewall and Nat rules, you need to turn on logging and add Log Prefix (under action).
Do not log more than needed. Logging rules like defconf: accept established,related rules will flod your log,
Below is a sample on how to name the log rules. You do not need to follow this rule, but it makes it more uniform.

Rule name logging
==================

Format:
x_y_z

x=<where and direction>
y=<what to do>
z=<name/info>

Example
-------
Filter Rule Forard allow HTTP
FF_A_Http

Filter Route Input Drop ICMP
FI_D_Icmp

Nat HTTP
ND_DE_Http

Mangle Mark HTTP packets
MF_MP_Http


Filter Rule
------------------
x=
FF Filter Forward
FI Filter Input
FO Filter Output
FX Filter Custom list

y=
A  Accept
AD Add to dst address list
AS Add to src address list
D  Dropp
F  Fast track
J  Jump
L  Log
P  Passthrough
RJ Reject
RT Return
T  Tarpit

Nat Rule
------------------
x=
ND Dest nat
NS Source nat

y=
A  Accept
AD Add to dst address list
AS Add to src address list
DE Dst-nat
J  Jump
L  Log
M  Masquerade
N  Netmap
P  Passthrough
RE Redirect
RT Return
SA same
S  Src-nat

Raw
------------------
x=
RP Filter Raw Prerouting
RO Filter Raw Output

y=
A  Accept
AD Add to dst address list
AS Add to src address list
F  Fast track
D  Dropp
J  Jump
L  Log
N  No track
P  Passthrough
RT Return

Mangle
------------------
x=
MF Mangle Forward
MI Mangle Input
MP Mangle Postrouing
MR Mangle Prerouting

y=
A  Accept
AD Add to address list
AS Add to dst address list
CD Change DSCP
CM Change MSS
CT Change TTL
CL Clear DF
F  Fast track
J  Jump
L  Log
MC Marc connection
MP Mark packets
MR Mark routing
P  Passthrough
RT Return
RO Route
S  Set proirity
SP Sniff PC
ST Sniff TZSP
SI Strip IPv4 options

2d) You should at least log this rule "defconf: drop all not coming from LAN" with this prefix: FI_D_port-test
Web gui:
IP->Firewall->selec:defconf: drop all not coming from LAN->Log:v->Log Prefix:FI_D_port-test
This will populate the MikroTik Live attack view.


2e) Accounting (new version in 3.5)
To get accounting data, you need to turn on Kid Control on the MikroTik router. (MikroTik Traffic dashboard)
Cli:
/ip kid-control
add fri=0s-1d mon=0s-1d name=Monitor sat=0s-1d sun=0s-1d thu=0s-1d tue=0s-1d wed=0s-1d

2f) Scripts

Main Script:

To get all the other data like Traffic accounting, uPnP, System health, System resources and DHCP pool information you need this script on the MikroTik. Create this script with name Data_to_Splunk_using_Syslog and cut and past code using gui.
In the top of the script, you can set a module to true/false. If you do not use wifi, set :local Wireless false

You can from the main script adjust what it should collect or not collect by change between true and false in the first part of the script,
# Collect information from Mikrotik RouterOS
# Jotne 2024
# Script name=Data_to_Splunk_using_Syslog
:log info message="script=version ver=5.6"
# ----------------------------------

# Auto update syslog server. 5.3-5.4.
# Change <your syslog dns name> to the dns of your syslog server.
# The update is disabled by default.  Remove the # from the two next line to use it.

#:local mySyslog [resolve <your syslog dns name>]
#/system/logging/action/set [find where name="logserver"] remote=$mySyslog


# What data to collect.  Set to false to skip the section 
# ----------------------------------
:local SystemResource true
:local SystemInformation true
:local SystemHealth true
:local TrafficData true
:local AccountData true
:local uPnP true
:local Wireless true
:local AddressLists true
:local DHCP true
:local Neighbor true
:local InterfaceData true
:local CmdHistory true
:local CAPsMANN false

:local Routing true
:local OSPF false
:local BGP false

:local PPP true
:local IPSEC true

# Get RouterOS main version (used to run different script on different version)
:local train [:tonum [:pick [/system resource get version] 0 1]] 

# Collect system resource
# ----------------------------------
:if ($SystemResource) do={
	/system resource
	:local cpuload [get cpu-load]
	:local freemem ([get free-memory]/1048576)
	:local totmem ([get total-memory]/1048576)
	:local freehddspace ([get free-hdd-space]/1048576)
	:local totalhddspace ([get total-hdd-space]/1048576)
	:local up [get uptime]
	:local sector [get write-sect-total]
	:log info message="script=resource free_memory=$freemem MB total_memory=$totmem MB free_hdd_space=$freehddspace MB total_hdd_space=$totalhddspace MB cpu_load=$cpuload uptime=$up write-sect-total=$sector"
}


# Make some part only run every hours
# ----------------------------------
:global Hour
:local run false
:local hour [:pick [/system clock get time] 0 2]
:if ($Hour != $hour) do={
	:global Hour $hour
	:set run true
}


# Get NTP status
# ----------------------------------
:local ntpstatus ""
:if ([:len [/system package find where !disabled and name=ntp]] > 0 or [:tonum [:pick [/system resource get version] 0 1]] > 6) do={
    :set ntpstatus [/system ntp client get status]
} else={
    :if ([:typeof [/system ntp client get last-update-from]] = "nil") do={
        :set ntpstatus "using-local-clock"
    } else={
        :set ntpstatus "synchronized"
    }
}
:log info message="script=ntp status=$ntpstatus" 


# Get interface traffic data for all interface
# ----------------------------------
:if ($TrafficData) do={
	:foreach id in=[/interface find] do={
		:local output "$[/interface print stats as-value where .id=$id]"
		:set ( "$output"->"script" ) "if_traffic"
		:log info message="$output"
	}
}


# Get traffic data v2 (Kid Control)
# ----------------------------------
:if ($AccountData) do={
	:foreach logline in=[/ip kid-control device find] do={
		:local output "$[/ip kid-control device get $logline]"
		:set ( "$output"->"script" ) "kids"
		:log info message="$output"
	}
}


# Finding dynmaic lines used in uPnP
# ----------------------------------
:if ($uPnP) do={
	:foreach logline in=[/ip firewall nat find where dynamic=yes and comment~"^upnp "] do={
		:local output "$[/ip firewall nat print as-value from=$logline]"
		:set ( "$output"->"script" ) "upnp"
		:log info message="$output" 
	}
}


# Collect system information 5.5 added ID for non routerBoard 5.6 Remvoed serial
# ----------------------------------
:local model na
:local ffirmware na
:local cfirmware na
:local ufirmware na
:if ($SystemInformation and $run) do={
	:local version ([/system resource get version])
	:local board ([/system resource get board-name])
	:local identity ([/system identity get name])
	:do {
		:if ($board!="CHR" OR $board!="x86") do={
			/system routerboard
			:set model ([get model])
			:set ffirmware ([get factory-firmware])
			:set cfirmware ([get current-firmware])
			:set ufirmware ([get upgrade-firmware])
		}
	} on-error={}
	:log info message="script=sysinfo version=\"$version\" board-name=\"$board\" model=\"$model\" identity=\"$identity\" factory-firmware=\"$ffirmware\" current-firmware=\"$cfirmware\" upgrade-firmware=\"$ufirmware\""
}


# Collect system health
# ----------------------------------
:if ($train > 6 and $SystemHealth) do={
	# New version (RouterOS >6)
	:foreach id in=[/system health find] do={
		:local health "$[/system health get $id]"
		:set ( "$health"->"script" ) "health"
		:log info message="$health"
	}
} else={
	# Old version (RouterOS 6 or older)
	:if (!([/system health get]~"(state=disabled|^\$)")) do={
		:local health "$[/system health get]"
		:set ( "$health"->"script" ) "health"
		:log info message="$health"
	}
}



# Sends wireless client data to log server 
# ----------------------------------
:if ($Wireless && [:len [/int find where type=wlan]]>0) do={
	/interface wireless registration-table
	:foreach i in=[find] do={
		:log info message=".id=$i;ap=$([get $i ap]);interface=$([get $i interface]);mac-address=$([get $i mac-address]);signal-strength=$([get $i signal-strength]);tx-rate=$([get $i tx-rate]);uptime=$([get $i uptime]);script=wifi"
	}
}


# Count IP in address-lists
#----------------------------------
:if ($AddressLists) do={
	:local array [ :toarray "" ]
	:local addrcntdyn [:toarray ""] 
	:local addrcntstat [:toarray ""] 
	:local test
	:foreach id in=[/ip firewall address-list find] do={
		:local rec [/ip firewall address-list get $id]
		:local listname ($rec->"list")
		:local listdynamic ($rec->"dynamic")
		:if (!($array ~ $listname)) do={ :set array ($array , $listname) }
		:if ($listdynamic = true) do={
			:set ($addrcntdyn->$listname) ($addrcntdyn->$listname+1)
		} else={
			:set ($addrcntstat->$listname) ($addrcntstat->$listname+1)}
	}
	:foreach k in=$array do={
		:log info message=("script=address_lists list=$k dynamic=".(($addrcntdyn->$k)+0)." static=".(($addrcntstat->$k)+0))}
}


# Get MNDP (CDP) Neighbors
# ----------------------------------
:if ($Neighbor and $run) do={
	:foreach neighborID in=[/ip neighbor find] do={
		:local nb [/ip neighbor get $neighborID]
		:local id [:pick ("$nb"->".id") 1 99]
		:foreach key,value in=$nb do={
			:local newline [:find $value "\n"]
			:if ([$newline]>0) do={
				:set value [:pick $value 0 $newline]
			}
			:log info message="script=neighbor nid=$id $key=\"$value\""
		}
	}
}


# Collect DHCP Pool information
# ----------------------------------
:if ($DHCP and $run) do={
	/ip pool {
		:local poolname
		:local pooladdresses
		:local poolused
		:local minaddress
		:local maxaddress
		:local findindex

# Iterate through IP Pools
		:foreach pool in=[find] do={
			:set poolname [get $pool name]
			:set pooladdresses 0
			:set poolused 0

# Iterate through current pool's IP ranges
			:foreach range in=[:toarray [get $pool range]] do={

# Get min and max addresses
				:set findindex [:find [:tostr $range] "-"]
				:if ([:len $findindex] > 0) do={
					:set minaddress [:pick [:tostr $range] 0 $findindex]
					:set maxaddress [:pick [:tostr $range] ($findindex + 1) [:len [:tostr $range]]]
				} else={
					:set minaddress [:tostr $range]
					:set maxaddress [:tostr $range]
				}

# Calculate number of ip in one range
				:set pooladdresses ($maxaddress - $minaddress)

# /foreach range
			}

# Test if pools is used in DHCP or VPN and show leases used
			:local dname [/ip dhcp-server find where address-pool=$poolname]
			:if ([:len $dname] = 0) do={
# No DHCP server found, assume VPN
				:set poolused [:len [used find pool=[:tostr $poolname]]]
			} else={
# DHCP server found, count leases
				:local dname [/ip dhcp-server get [find where address-pool=$poolname] name]
				:set poolused [:len [/ip dhcp-server lease find where server=$dname]]}

# Send data
			:log info message=("script=pool pool=$poolname used=$poolused total=$pooladdresses")

# /foreach pool
		}
# /ip pool
	}
}


# Get detailed command history RouterOS >= v7
# ----------------------------------
:if ($train > 6 and $CmdHistory) do={
	:global cmd
	:local f 0
	:foreach i in=[/system history find] do={
		:if ($i = $cmd) do={ :set f 1 }
		:if ($f != 1) do={
			:log info message="StartCMD"
			:log info message=[/system history get $i]
			:log info message="EndCMD"
		}
	}
	:global cmd  [:pick [/system history find] 0]
}


# Test if CAPsMANN is installed and run script 5.5
# ----------------------------------
:if ( ([:len [/interface find where type="cap"]] > 0) and $CAPsMANN) do={ 
	/system script run CAPsMANN
}



# Collect routing information
# ----------------------------------
:if ($Routing) do={
	/ip route
	:foreach id in=[find] do={
		:local route "$[get $id]"
		:set ( "$route"->"script" ) "route"
		:log info message="$route"
	}
}

:if ($OSPF) do={
	/routing ospf neighbor
	:foreach id in=[find] do={
		:local ospf "$[get $id]"
		:set ( "$ospf"->"script" ) "ospf"
		:log info message="$ospf"
	}
}

:if ($BGP) do={
	/routing bgp session
	:foreach id in=[find] do={
		:local bgp "$[get $id]"
		:set ( "$bgp"->"script" ) "bgp"
		:log info message="$bgp"
	}
}


# Collect PPP/IPSEC
# ----------------------------------
:if ($PPP) do={
	/ppp active
	:foreach id in=[find] do={
		:local ppp "$[get $id]"
		:set ( "$ppp"->"script" ) "ppp"
		:log info message="$ppp"
	}
}

:if ($IPSEC) do={
	/ip ipsec active-peers
	:foreach id in=[find] do={
		:local ipsec "$[get $id]"
		:set ( "$ipsec"->"script" ) "ipsec"
		:log info message="$ipsec"
	}
}

# End Script


CAPsMANN scrip. Only needed if you are using CAPsMANN
# CAPsMANN script

:local capsregistered ([/caps-man registration-table print count-only])
 
/caps-man interface
:local name
:local mac
 
# ignore all master interfaces
:foreach i in=[find where master-interface="none"] do={
	:set name [get $i name]
	:set mac [get $i radio-mac]
	:local counter ([/caps-man registration-table print count-only  where interface=$name])
	:log info message="script=caps-man name=$name counter=$counter"
}
:log info message="script=caps-man capsregistered=$capsregistered"
#added to log frequency&chennel
:if ( ([:len [/interface find where type="cap"]] > 0) and $CAPsMANN and $run) do={
:foreach i in=[/caps-man interface find where master-interface="none"] do={
	:local name [/caps-man interface get $i name]
	:local channel [/caps-man interface get $i current-channel]
	:log info "$name: selected channel $channel"
	}
}

2g) Then schedule the script to run every 5 minutes:
/system scheduler
add interval=5m name="Data_to_Splunk" on-event=Data_to_Splunk_using_Syslog

3) Starting up
3a) Startup information
Some parts of the scripts runs only every hour, and some jobs in Splunk runs once a day.
So it will take time before all devices er named correctly. To speed things up some, do this:
After running for some hour and you see that data are coming, in Splunk go to:
Apps->Mikrotik->Reports and run both Device table updater and DHCP table updater by clicking on Open In Search behind the app


4) Debugging
4a) See if any data are comming inn to splunk at all. Do a search in Splunk for:
index=*
4b). Test if data has correct tag "MikroTik" (Capital M & T) Do a search in Splunk for:
index=* | table _time sourcetype _raw
You should see correct time, sourcetype should show "mikrotik" and _raw should show data

4c). See that _raw does contain only data and not time and other info. Do a search in Splunk for:
index=* | table  _raw
dns MikroTik: done query: #640030 adservice.google.no 216.58.211.2
dhcp,debug,packet MikroTik:     Client-Id = 01-6C-3B-6B-88-34-3F
firewall,info MikroTik: FI_D_port-test input: in:ether1 out:(unknown 0), src-mac 00:05:00:01:00:01, proto TCP (SYN), 47.118.40.92:52503->92.220.205.91:2376, len 40
If you see date, format of packet from Mikrotik has BDS set or Rsyslog is not setup correctly.

4d). Verify that all files has same user:group (root:root or splunk:splunk if run as non-root user)

4e). Look for error written in file or error in file name. inputs.conf not input.conf

4f). Read trough all steps on how to install if some does not work

4g). License problems.
Not convert license to Free license before 30 days or indexing more than 500MB/day?
The Free license will prevent searching if there are 3 license warnings in a rolling 60 day window. If that happens, Splunk Free continues to index your data but disables search functionality. You will regain search when you are below 3 license violation warnings in a 60 day period. See About license violations.
How to solve this+
1. Convert to free and wait 30 days if you did not convert it.
2. Passing 3 times? Reduce license <500MB and wait 60 days.
3. Reintall Splunk
4. Get a Free 10GB/day developer license ( https://dev.splunk.com/enterprise/dev_license )

4h) Limit inout. Data comes from two sources in Splunk.
1. The log setup. DNS and other stuff. To remove DNS change to
/system logging add action=logserver prefix=MikroTik topics=!debug,!packet,!snmp,!dns
2. From the logging script: Data_to_Splunk_using_Syslog
Change true to false each block you like to stop.

4i) 1. You see you get Duplicate Values under host. This may be that you did change name on one device ore have multiple host with same name.
Install Lookup Editor (a Splunk addon app) in Splunk if you do not already have done. Got to Apps -> Lookup Editor, select device_kvstore and open it. Remove the duplicate/old items.

4j) Make sure you not have setup BSD syslog. System->Logging->Action->Logserver->Make sure BSD are not selected.

4k) Make sure time are correct on the router. NTP running and connected to a server.

4x). Still problems: ask here :)
You do not have the required permissions to view the files attached to this post.
Last edited by Jotne on Mon Feb 19, 2024 2:32 pm, edited 73 times in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 11:50 am

1Mikrotik Firewall.jpg
2DNS Live view.jpg
3Volt_temperature.jpg
4Resources.jpg
You do not have the required permissions to view the files attached to this post.
Last edited by Jotne on Tue Nov 02, 2021 1:27 pm, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 11:51 am

Netwatch

This part shows how to use the MikroTik Netwatch dashboard.
Idea with this part are to monitor one or many IP and get a good logging information of up/down time of a watched IP.

Setup.
Lets say you like to monitor a WireGuard VPN tunnel. There are noe traces of up down status for WireGuard in the RouterOS, so using Netwarch to see if remote IP is up or down is a way to see status of it.

Script
System->Scripts->Add script.
Name: Netwatch
Script:
####################################
# Netwatch script
#
# Used as both up and down script
# Created Jotne 2021 v1.5
#
####################################
:local Host $host
/tool netwatch
:local Status [get [find where host="$Host"] status]
:local Comment [get [find where host="$Host"] comment]
:local Interval [get [find where host="$Host"] interval]
:local Since [get [find where host="$Host"] since]
:log info "script=netwatch watch_host=$Host comment=\"$Comment\" status=$Status interval=$Interval since=\"$Since\""

Tools->Netwatch
Add Host ip. For WireGuard, that would be ip on the other side of the tunnel.
Host: 10.0.0.2
Up: Netwatch
Down: Netwatch
Comment: WG-Tunnel-22 (This name is important to set, since this will identify what this Netwatch do watch.)
/tool netwatch
add comment=WG-Tunnel-22 down-script=Netwatch host=10.0.0.2 up-script=Netwatch

You can ass as many netwatch IP as you like. It will take resource from the router, so do not add to many that test to often.
.
Netwatch.jpg
You do not have the required permissions to view the files attached to this post.
Last edited by Jotne on Tue Jun 28, 2022 5:24 pm, edited 8 times in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 11:51 am

Placeholder p1
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 11:52 am

How to install Splunk as a non root user.
Its a security risk to run everything as a root user, so if you can, you should use a dedicated user for your program.

This tutorial will show how to install Splunk as a user with name splunk on your Ubuntu server (may work on other as well)

Download latest Splunk Enterprise to you /tmp folder

Create the splunk user:
sudo useradd -c "splunk user" -m -s /bin/bash -U -d /opt/splunk splunk
Log in a the splunk user:
sudo su - splunk
Download Splunk Linux tgz file to /tmp folder

Extract the Splunk software to /opt folder (name of file will change with new version):
tar xvzf /tmp/splunk-8.0.3-a6754d8441bf-Linux-x86_64.tgz -C /opt
Start your Splunk server (accept license agrement and set a password for Spkunk admin user):
~/bin/splunk start
Add a user/password to login to Splunk.

As a root user, make Splunk autostart with user splunk as a startup script:
sudo /opt/splunk/bin/splunk enable boot-start -user splunk
You should now be up and running on port 8000 (can be changed)
Remember to use splunk user whenever you change/add files or do anything else with Splunk from the CLI
sudo su - splunk
PS:
If you run Splunk as a non root user then you can not use UDP/514 as a syslog receiver port in Splunk.
Since all port below 1024 need root permission to work.

Workarounds.
1. Send syslog to other port above 1023, like 1514 for UDP syslog. (need to change many routers to send to correct port)
2. Set up a local syslog server like r-syslog and let Splunk read the r-syslog log files.
viewtopic.php?p=677233#p793342
Last edited by Jotne on Tue Jun 28, 2022 5:25 pm, edited 4 times in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 11:53 am

How to install RSyslog for Linux

If you do use Splunk as a non root (recomended) user, you need an external Syslog server.

This is how to set it up using Ubuntu server. Should work on most version.

rsyslog comes default with Ubuntu so no need to install any extra software.


PS do not modify these file to use other location. If you do so you will need to modify udp.conf rsyslog and inputs.conf splunk for every upgrade.

Copy these two files to /etc/rsyslog.d/

udp.conf (sets up rsyslog to accept Sylog on udp/514)
# rsyslog.d/udp.conf
#
# This receives UDP syslog on port 514 and stores it in a reliable format
# in /data/syslog/udp/:
#
#  - A subdirectory is created for each logging host.
#    The directory name is the source IP address of the received log message.
#    The log filename is made up of date and hour of file creation.
#    A new log file is created for each logging host every hour.
#
#  - Each log message is prefixed with a locally generated timestamp.
#
#  - Then the raw, undecoded text of the incoming syslog message is written.
#    If the message is missing <PRI> then a default <PRI> is prepended.
#    This ensures that both old tyle (RFC-3164) and new style (RFC-5424)
#    syslog messages can be recognised and parsed from the log files without
#    risk of loosing information. The raw messages also include the original
#    host name and time stamp from the sender.
#
#  - Each written log message is guaranteed to end in a single newline
#    character.
#

# Stupidly this is set to "on" in the default rsyslog.conf file
# Do our best to negate the effect.
$RepeatedMsgReduction off

module(load="imudp")


# format
template(name="RawFormat" type="list") {
  property(name="timegenerated" dateformat="rfc3339")
  constant(value=" ")
  constant(value="<")
  property(name="pri")
  constant(value=">")
  property(name="rawmsg-after-pri" droplastlf="on")
  constant(value="\n")
}

# file name
template(name="udp_split_filename" type="list") {
  constant(value="/data/syslog/udp/")
  property(name="fromhost-ip")
  constant(value="/")
  property(name="$year")
  property(name="$month")
  property(name="$day")
  constant(value="-")
  property(name="$hour")
  #property(name="$minute")
  constant(value=".log")
}

# rule set
ruleset(name="udp_split") {
  action(type="omfile"
    template="RawFormat"
    createDirs="on"
    dirCreateMode="0755"
    fileCreateMode="0644"
    dynaFile="udp_split_filename"
  )
}

# setting
input(type="imudp" port="514" ruleset="udp_split")

tcp.conf (sets up rsyslog to accept Syslog on tcp/1514) PS MikroTik only sends UDP syslog, so this part is not needed
# rsyslog.d/tcp.conf
#
# This receives TCP syslog on port 1514 and stores it in a reliable format
# in /data/syslog/tcp/:
#
#  - A subdirectory is created for each logging host.
#    The directory name is the source IP address of the received log message.
#    The log filename is made up of date and hour of file creation.
#    A new log file is created for each logging host every hour.
#
#  - Each log message is prefixed with a locally generated timestamp.
#
#  - Then the raw, undecoded text of the incoming syslog message is written.
#    If the message is missing <PRI> then a default <PRI> is prepended.
#    This ensures that both old tyle (RFC-3164) and new style (RFC-5424)
#    syslog messages can be recognised and parsed from the log files without
#    risk of loosing information. The raw messages also include the original
#    host name and time stamp from the sender.
#
#  - Each written log message is guaranteed to end in a single newline
#    character.
#

# Stupidly this is set to "on" in the default rsyslog.conf file
# Do our best to negate the effect.
$RepeatedMsgReduction off

module(load="imtcp")


# format
template(name="RawFormat" type="list") {
  property(name="timegenerated" dateformat="rfc3339")
  constant(value=" ")
  constant(value="<")
  property(name="pri")
  constant(value=">")
  property(name="rawmsg-after-pri" droplastlf="on")
  constant(value="\n")
}

# file name
template(name="tcp_split_filename" type="list") {
  constant(value="/data/syslog/tcp/")
  property(name="fromhost-ip")
  constant(value="/")
  property(name="$year")
  property(name="$month")
  property(name="$day")
  constant(value="-")
  property(name="$hour")
  #property(name="$minute")
  constant(value=".log")
}

# rule set
ruleset(name="tcp_split") {
  action(type="omfile"
    template="RawFormat"
    createDirs="on"
    dirCreateMode="0755"
    fileCreateMode="0644"
    dynaFile="tcp_split_filename"
  )
}

# settings
input(type="imtcp" port="1514" ruleset="tcp_split")

Create the following folders
mkdir /data
mkdir /data/syslog
mkdir /data/syslog/tcp
mkdir /data/syslog/udp
Change folder rights to syslog and restart rsyslog
chown -R syslog:syslog /data/syslog
service rsyslog restart
run ss or netstat as root user to see that rsylog is running
ss -pultn | grep syslog
udp   UNCONN 0      0                                0.0.0.0:514        0.0.0.0:*     users:(("rsyslogd",pid=5532,fd=8))
udp   UNCONN 0      0                                   [::]:514           [::]:*     users:(("rsyslogd",pid=5532,fd=9))
tcp   LISTEN 0      25                               0.0.0.0:1514       0.0.0.0:*     users:(("rsyslogd",pid=5532,fd=6))
tcp   LISTEN 0      25                                  [::]:1514          [::]:*     users:(("rsyslogd",pid=5532,fd=7))
netstat -pultn | grep rsyslog
tcp        0      0 0.0.0.0:1514            0.0.0.0:*               LISTEN      1459/rsyslogd
tcp6       0      0 :::1514                 :::*                    LISTEN      1459/rsyslogd
udp        0      0 0.0.0.0:514             0.0.0.0:*                           1459/rsyslogd
udp6       0      0 :::514                  :::*                                1459/rsyslogd
To make Splunk read rsyslog data make this file: %SplunkHome%/etc/system/local/inputs.conf
[monitor:///data/syslog/udp/.../*.log]
sourcetype = rsyslog
host_segment=4

[monitor:///data/syslog/tcp/.../*.log]
sourcetype = rsyslog
host_segment=4
Test your server:
echo '<14>sourcehost message text' | nc -v -u -w 0 127.0.0.1 514
This should create a folder /data/syslog/udp/127.0.0.1 with a *.log file

Clean UP
Since rsyslog does not delete anything, you need a script that delete old files

Create file /etc/cron.d/rsyslog_cleanup with:
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

UDPLOGDIR=/data/syslog/udp/.
TCPLOGDIR=/data/syslog/tcp/.

# Age are n+1 days
DELETE_AGE=2

# Every hour, as user syslog, clean out ancient log files
00 * * * *  root /usr/bin/find ${UDPLOGDIR} -mtime +${DELETE_AGE} \( -name \*.log -o -name \*.log.gz \) -print -delete 2>&1
00 * * * *  root /usr/bin/find ${TCPLOGDIR} -mtime +${DELETE_AGE} \( -name \*.log -o -name \*.log.gz \) -print -delete 2>&1
Last edited by Jotne on Wed Jul 20, 2022 9:56 am, edited 11 times in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 11:55 am

Placeholder P2
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 11:57 am

Placeholder P3
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 02, 2021 12:18 pm

Hi Jotne,
Are you working on / thinking about incorporating Netflow into your app/dashboard design already ? It can be right next to all the existing stuff anyway.
Offcourse there is also the question about the backend ingesting netflow data from Mikrotik device.
Because today I have 2 separate pages with some Netflow pages found on Github, might be interesting to start including it in the general design ?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Nov 06, 2021 10:45 pm

Yes I have been thinking about netflow, since MT is removing IP accounting.
Will see how much time I find to do some testing.

IP accounting are sent over Syslog, so no need for extra setup anywhere. Scripts take care of sending data.
Netflow on the other hand, need some server to receive data and an extra port (can not be sent over syslog port).
Netflow server also needs to be stup and communicate with Splunk.
Netflow plugins for Splunk also seems to cost money... https://splunkbase.splunk.com/app/489/
So there will be a more complex solution using Netflow.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Nov 06, 2021 11:17 pm

I'm using the Splunk App for Stream, which is capable of many things including netflow-decoding.
Its free as far as I know, I've been using is more then 1 year, actually never failed on me. But I agree installation + adapting config-file was not click-click-click ready ;-) but certainly not overly complex.

https://splunkbase.splunk.com/app/1809/

-------
Targeted full packet capture to NAS for forensic investigation of raw packets. Aggregate data using familiar SPL aggregation methods to reduce the volume of data indexed. Capture Flow-type records, including NetFlow v5, v9, jFlow, and sFlow, and IPFIX, and send Flow Records directly into your Indexers, with optional filtering and aggregation.
-------
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 07, 2021 12:27 am

That seems to look better.
Could you give me a quick guide on how to get netflow in on 9995 to Splunk, I could update the app to show the data.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 07, 2021 10:14 am

Ok,
Let's start with some github-references where I downloaded some dashboards. You probably will have to pick & match what you want to include in your own dashboard

https://github.com/JohnnyMirza/Splunk_Netflow
https://github.com/danucalovj/Splunk-Netflow-Analyzer
https://github.com/lucas-alados/netflow ... dashboards

I followed the guides on Splunkbase itself for installing the Stream_TA package, its not that complex.

https://docs.splunk.com/Documentation/S ... dIPFIXdata

Contents of my /opt/splunk/etc/apps/Splunk_TA_stream/default/streamfwd.conf
(not too sure why I added the customs with their flow-id, I think it was during the time I wanted IPFIX to work (which is buggy on 6.x ROS) but v5 works fine.
I agree you miss out on some field, but "the basics" are there.

[streamfwd]
port = 8889
ipAddr = 127.0.0.1

netflowReceiver.0.ip = X.X.X.X (=IP of the sending interface on the Mikrotik)
netflowReceiver.0.port = 9995
netflowReceiver.0.decoder = netflow


netflowElement.0.enterpriseid = 14988
netflowElement.0.id = 225
netflowElement.0.termid = netflow.postNATSourceIPAddress

netflowElement.1.enterpriseid = 14988
netflowElement.1.id = 226
netflowElement.1.termid = netflow.postNATDestinationIPAddress

netflowElement.2.enterpriseid = 14988
netflowElement.2.id = 227
netflowElement.2.termid = netflow.postNAPTSourceTransportPort

netflowElement.3.enterpriseid = 14988
netflowElement.3.id = 228
netflowElement.3.termid = netflow.postNAPTDestinationTransportPort




And I think that this should be about it. The binary is started also with the rest of Splunk
(I'm running it as root, not the smartest thing to do I guess ;-)
(needed to reboot my NAS on October 12 hence te low uptime)

root 1748 1023 0 Oct12 ? 00:00:00 /bin/sh -c /opt/splunk/etc/apps/Splunk_TA_stream/linux_x86_64/bin/streamfwd
root 1749 1748 0 Oct12 ? 01:14:27 /opt/splunk/etc/apps/Splunk_TA_stream/linux_x86_64/bin/streamfwd


Hope this helps already a bit...

On the Mikrotik, under IP > Flows I've enabled it, selected all possible field.
I have 1 "target" (=IP of the listener Streams/Splunk) and selected "v5"
That's it...
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 07, 2021 11:01 am

Hi, I try to upgrade from 3.2 and reading the instructions says :

1j) Upgrade form previous version.
Some time files are renamed, so if you have not change any original files, just delete the MikroTik folder.
No logged data will be deleted.
If you have custom dashboards, menus, saved search (reports) etc, you need to merge the configuration files.
They are normal stored in "local" folder.

HOW do I delete "the MikroTik folder" thru the splunk interface ???
I am running splunk in docker on synology, not that familiar with splunk, it just works here ;-)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 07, 2021 4:52 pm

HOW do I delete "the MikroTik folder" thru the splunk interface ???
It can not be deleted trough Splunk, you need to delete the folder maualy:
~/etc/apps/MikroTik
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 07, 2021 9:26 pm


netflowReceiver.0.ip = X.X.X.X (=IP of the sending interface on the Mikrotik)
What if you have many devices sending netflow?
Can you open it so it listen for any IP?
netflowElement.1.enterpriseid = 14988
netflowElement.1.id = 226
netflowElement.1.termid = netflow.postNATDestinationIPAddress
What are these section?
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 07, 2021 9:38 pm


netflowReceiver.0.ip = X.X.X.X (=IP of the sending interface on the Mikrotik)
What if you have many devices sending netflow?
Can you open it so it listen for any IP?
netflowElement.1.enterpriseid = 14988
netflowElement.1.id = 226
netflowElement.1.termid = netflow.postNATDestinationIPAddress
What are these section?
I think you can add multiple , like netflowReceiver.0.ip=X.X.X.X
netflowReceiver.1.ip=Y.Y.Y.Y

As each of these instances has fields for IP, port etc , I think its purpose it to be able to add multiple.
Never tried it ;-) I only have 1 Mikrotik

These sections on
netflowElement.1.id = 226
netflowElement.1.termid = netflow.postNATDestinationIPAddress
etc..

I think I added them while playing to get IPFIX working (IPFIX works with periodic templates of fields being transmitted).
I believe the *default* "dictionary" did not recognise certain Netflow field, hence the possibility to add new ones.
To be honest I don't know anymore. They are not present in the "default" files so for sure some custom work.

Perhaps it something to do with this thread viewtopic.php?t=99152
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 8:19 am

I think you can add multiple , like netflowReceiver.0.ip=X.X.X.X
I did hope for a solution where you have a netflow listener, and that it does not care about where data is coming from.
Would be hard to maintain and setup if you have a lot of routers.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 8:56 am

I think you can add multiple , like netflowReceiver.0.ip=X.X.X.X
I did hope for a solution where you have a netflow listener, and that it does not care about where data is coming from.
Would be hard to maintain and setup if you have a lot of routers.
I quickly tested and it seems not to like 0.0.0.0 ;-)
I agree this add some task to the admin, but I'm not even sure this Stream-plugin would "scale" well if you have hundreds of routers sending Netflow data to it.
Then you probably have to run it on a separate box and not on the same "Splunk" host that like I do.
As an "ip accounting" replacement, I would enable netflow only on the Internet-facing devices or some centralized Internet breakout boxes.
Enabling netflow for all internal interfaces on a larger network will generate quite some flows.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 9:21 am

I re-tested and 0.0.0.0 seems to work.
Because Splunk is running on my NAS I really need to be patient, it's quite heavy ;-)

Anyway, I've adapted my config to

netflowReceiver.0.ip = 0.0.0.0
netflowReceiver.0.port = 9995
netflowReceiver.0.decoder = netflow


And then killed the streamfwd-processes and restarted Splunk. Since then, events seems to arrive just fine.
The "exporter_ip" field indicates which element is sending the Netflow data.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 5:47 pm

So I upgraded to this latest version.
I am not familiar enough with splukt to tune this, but it is getting really sloooooooow
Any hints in how to get better performance ?
removing old data ?
some indexing and how ?
getting a warning :

Storage engine migration recommended

If your instance uses the MMAPv1 storage engine,

how do I find out if my docker instance uses this ??
free splunk license btw
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 9:22 pm

There a LOT of interesting documents on Splunk that are easy to read.
Including some to limit the retention for example. I only keep about 30days I believe. I'm running it on a NAS, together with 5 other VM's and 15+ containers so I have to make choices consuming nearly all 16GB that is in my NAS.


Concerning the storage-engine, see for example below

https://docs.splunk.com/Documentation/S ... ateKVstore

It seems I'm also using the "old" MMAPv1 but I don't have such messages. I'm running 8.2.0 (I'm not updating to release 8.2.3 since the fixes are not interesting to me)

For licensing, go to "Settings" (on the top menu) , then "Licensing" en there you'll see what you have, what volume of the 500MB/day you've used etc.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 9:30 pm

tnx,

I am using docker on synology. Splunk has 2GB out of 4GB availiable RAM ...

running portainer to manage and watchtower to automaticly upgrade docker images when they are availiable ...

I surely would like some options to autoclean log if older than xx days ...
It would probably make splunk a lot faster here
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 9:39 pm

In the bin-folder where the splunk binaries are, issue #./splunk show kvstore-status
It will provide the type of store.

This member:
backupRestoreStatus : Ready
date : Mon Nov 8 20:15:19 2021
dateSec : 1636398919.648
disabled : 0
guid : 6D5FAC74-4CDC-4F5C-AEA7-16B04EB7F9AE
oplogEndTimestamp : Mon Nov 8 20:15:17 2021
oplogEndTimestampSec : 1636398917
oplogStartTimestamp : Mon Oct 4 06:20:29 2021
oplogStartTimestampSec : 1633321229
port : 8191
replicaSet : 6D5FAC74-4CDC-4F5C-AEA7-16B04EB7F9AE
replicationStatus : KV store captain
standalone : 1
status : ready
storageEngine : mmapv1


I quickly performed migration to wiredTiger using the document-link earlier. It took only few minutes without issues.Just follow the procedure.


backupRestoreStatus : Ready
date : Mon Nov 8 20:39:04 2021
dateSec : 1636400344.346
disabled : 0
guid : 6D5FAC74-4CDC-4F5C-AEA7-16B04EB7F9AE
oplogEndTimestamp : Mon Nov 8 20:39:01 2021
oplogEndTimestampSec : 1636400341
oplogStartTimestamp : Mon Nov 8 20:32:08 2021
oplogStartTimestampSec : 1636399928
port : 8191
replicaSet : 6D5FAC74-4CDC-4F5C-AEA7-16B04EB7F9AE
replicationStatus : KV store captain
standalone : 1
status : ready
storageEngine : wiredTiger
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 9:45 pm

Upgraded here too ;-)

and now try to find a way to limit and auto clean data to speedup things
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 08, 2021 10:31 pm

Upgraded here too ;-)

and now try to find a way to limit and auto clean data to speedup things
Splunk community has tons of information.

https://community.splunk.com/t5/Getting ... m-p/495331

I'll think you'll find answers there to set some limits.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Nov 09, 2021 4:15 pm

Ok, I am trying to find the correct indexes.conf...
I tried to do so in the webinterface but no luck thru setting reduction on 90 days ...

finally found it in /opt/splunk/etc/system/local/indexes.conf
needed to restart splunk to get this active
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 14, 2021 8:49 pm

Seems to be that there will be a working Traffic accounting for v7.x without need to use netflow.
Will be out in next version if all is ok:
viewtopic.php?p=890978#p890978
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Nov 14, 2021 10:42 pm

Seems to be that there will be a working Traffic accounting for v7.x without need to use netflow.
Will be out in next version if all is ok:
viewtopic.php?p=890978#p890978
With the netflow you do have some more insight in the port-usage too and not just IP's.
Ideal to possibly pick up certain abnormal "flows"
I agree there is a lot TCP/443 these days but still...
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 15, 2021 9:18 am

I know NetFlow is a much more in depth analyze tool and gives information about every packet.
My goal is to deliver some that is simple and many can use to monitor their routers.
Kid Control and IP Accounting, gives information about who is downloading/uploading, how much and when.
Should be enough for most small/medium network admins.

Will have a look at NetFlow later to see if I can get it to work in a simple way with Splunk.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 15, 2021 2:18 pm

Here is a view that combine accounting with kid control.
You can see how much data my Chromecast downloads. 18MB last 4 hour (backgrround images).
At the same time it shows device (kid) control status. If its not in any group, its just used to monitor traffic (dynamic). It can be set to a group with various status, open, blocked manually, blocked due to time limit, blocked due to rate limit etc.

PS Kid Control should be renamed to Device traffic control, since its not just kids you like to block, it may be other devices as well.

.
Kid Accounting2.jpg
You do not have the required permissions to view the files attached to this post.
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Nov 15, 2021 8:30 pm

I agree, there are many uses of this device tracking and control that extend beyond kids. I can also see potential for enhancing it even more with a few more features - just a few useful ones I have thought of:

- The ability to be able to create a simple queue per host that includes the IPv4 address and IPv6 addresses - since "Devices" in kid control tracks this, there could easily be a "rate limit" setting in there. Note this is different from rate limiting per kid because sometimes you might want to limit per device like this.
- The ability to dynamically place the IPv4 and IPv6 addresses for a single device or kid into an address list, that way they could be flexibly used in firewall rules.
- Groups of "kids" could be created for things like departments of a company and used to populate address lists to allow creation of firewall rules based on department

Companies could use these features to restrict what some employees can do compared to others, and to provide an audit log of who had what IPv4 and IPv6 addresses at a given time. Kid-control has many practical use cases outside of restricting kids.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Dec 11, 2021 5:22 pm

Next version will have better health and works better with 7.1

Here is an example on Routers giving PSU State
.
psu_state.jpg
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.4 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Dec 20, 2021 9:21 am

# Script version 4.8
# Change to kid kontroll for accounting (needs to be fixed)
# Fixed possibility to turn off account data
# Updated health section to get all health info on old and new system to work better with 7.x

To upgrade, just cut/past the script to all router. (script found in first post)

NB If you do use accounting from 6.x, do not upgrade this script with also update the main Splunk version to minimum 3.5
This is due to change from accounting to kid control, since accounting does not work in RouterOS 7.x
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Dec 20, 2021 2:39 pm

Upgraded to 3.5

Happy Xmas 🎄 🎁

# 3.5 (20.12.2021)
# Changed from IP Accounting to Kid Control to get accounting data to work with 7.x RouterOS
# Renamed "MikroTik Volt/Temperature" to "MikroTik Health"
# Added more info to "Mikrotik Health"

Since the new app now uses Kid control to collect accounting data, you need to know the following.
1. To use accounting, you need at Script at least on v4.8 or larger.
2. You will no longer see historical data from old accounting.
3. To get Kid Control data see section 2e) in first post.

Upgrade can be done by just replacing old files and restart Splunk
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu Dec 23, 2021 1:58 pm

Next version will have a dashboard for Netwatch. With that you can keep track of when devices goes up and down.
It can also be used to monitor the stateless Wireguard VPN that can not be monitored as normal VPN can.
.
netwatch.jpg
You do not have the required permissions to view the files attached to this post.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu Dec 23, 2021 3:15 pm

Thanks Jotne!
Also usable for example to monitor ZeroTier participants on your "cloud" LAN.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu Dec 23, 2021 3:40 pm

Also usable for example to monitor ZeroTier participants on your "cloud" LAN.
Do ZeroTier work more or less like Wireguard with no logging on connecting/up/down etc?
If yes, this can be used for ZeroTier as well.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu Dec 23, 2021 5:45 pm

Also usable for example to monitor ZeroTier participants on your "cloud" LAN.
Do ZeroTier work more or less like Wireguard with no logging on connecting/up/down etc?
If yes, this can be used for ZeroTier as well.
So it seems, my "interface" "zerotier1" is always UP ,but with Netwatch I can ping/test "remote endpoints" that also participate in the ZeroTier network.
I get "up / down" notifications through Netwatch the moment I switch on the ZeroTier VPN app on my Android phone
Off course conceptually ZeroTier is a bit different from WireGuard
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Jan 02, 2022 6:32 pm

There are many solution.

With Splunk you have 100% control of everything. You server, your setup. And free (up to 500MB/day)
Store as much data as long as you like.
 
maga79
just joined
Posts: 6
Joined: Fri Mar 01, 2019 11:57 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Jan 11, 2022 8:45 am

NB logging large amount of Accouning, DNS or firewall rules quickly eats up license, so I do recommend to turn off Accouning/DNS logging to start with.
How to turn off DNS logging ? When I disable Accounting function , log server still receive the dns request .
Thanks.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Jan 11, 2022 10:57 pm

DNS logs comes from the Router log, so to stop it change from:
/system logging add action=logserver prefix=MikroTik topics=!debug,!packet,!snmp
to
/system logging add action=logserver prefix=MikroTik topics=!debug,!packet,!snmp,!dns
Last edited by Jotne on Tue Jun 28, 2022 5:26 pm, edited 1 time in total.
 
maga79
just joined
Posts: 6
Joined: Fri Mar 01, 2019 11:57 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Jan 18, 2022 6:03 pm

Splunk versino : 8.2.4
License Type :Free license group
Volume used today 3 MB (0.524% of quota)
Mikrotik schedule : 5 minutes
When I login in the Mikrotik logs on Splunk , running search mikrotik on splunk ,There is no log record in splunk server.
After I restart Splunk service on web . splunk server will received the log from RB4011.
I need manually estart Splunk service after splunk server running 15 minutes everyday.
Why splunk need do that ? Is there something wrong with splunk server?How can i check the splunk server is running normally?
Thanks
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue Jan 18, 2022 10:19 pm

Mine never needs to be restarted.
Have one version where Splunk listen on port 514 (not recommended as it needs to be root)
Other version have rsyslog server as input and Splunk reads rsyslog logs.
Both running fine.'

Do you pass any firewall on the way from MikroTik to the Splunk server?
What do you run Splunk on? Linux (recommend Ubuntu) on a dedicated (pri 1) server or vmware (pri 2) are the best options.
Avoid using Splunk on Windows.
 
maga79
just joined
Posts: 6
Joined: Fri Mar 01, 2019 11:57 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed Jan 19, 2022 5:02 am

Mine never needs to be restarted.
Have one version where Splunk listen on port 514 (not recommended as it needs to be root)
Other version have rsyslog server as input and Splunk reads rsyslog logs.
Both running fine.'

Do you pass any firewall on the way from MikroTik to the Splunk server?
What do you run Splunk on? Linux (recommend Ubuntu) on a dedicated (pri 1) server or vmware (pri 2) are the best options.
Avoid using Splunk on Windows.
Port 514 and port 8000
udp UNCONN 0 0 0.0.0.0:514 0.0.0.0:* users:(("splunkd",pid=364916,fd=57))
tcp LISTEN 0 128 0.0.0.0:8000 0.0.0.0:* users:(("splunkd",pid=364916,fd=138))

Status: active

To Action From
-- ------ ----
22/tcp ALLOW Anywhere
514 ALLOW Server IP address
8000/tcp ALLOW Anywhere
8090/tcp ALLOW Anywhere
443/tcp ALLOW Anywhere
22/tcp (v6) ALLOW Anywhere (v6)
8000/tcp (v6) ALLOW Anywhere (v6)
8090/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)

Splunk run on the Ubuntu 20.04 server version which was build on ESXi 6.7.0.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed Jan 19, 2022 8:06 am

Should work.
Only comment is that you should not run splunk as root user, and use rsyslog to listen on port 514.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed Jan 19, 2022 8:36 am

My Splunk is also running for some years now. Only goes down when I apply Splunk updates of some sort. Rock solid!
I'm running on Ubuntu 18.0.4.5 LTS
I suggest you start digging in the LOG-directory of you Splunk.

/opt/splunk/var/log

From there you have folders "watchdog" and "splunk"

You could investigate watchdog.log and in the spluink-folder many logfile reside like health.log , splunkd.log , web_access.log etc,etc

Splunk-specific troubleshooting might be slightly outside the scope of this forum ;-)
The Splunk community-forums are a better place for that.
 
maga79
just joined
Posts: 6
Joined: Fri Mar 01, 2019 11:57 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Jan 22, 2022 7:04 pm

Thanks for help Jotne and jvanhambelgium.
After I reinstall the Splunk server, all the things works well .
But Some graph in APP didn't work.
i can get the graph for Mikrotik CAPsMAN Wifi Connection and "Mikrotik CAPsMAN Number of Clients pr AP" , but there is no graph for"Mikrotik CAPsMAN Channel Usage"
I also setup the ":local CAPsMANN true" but did no work.
Did i miss some configration on Mikrotik script?

Our customer always want to know which client PC or mobile phone take maximum flow in specified time range ,with destination IP and protocol.
How can I the traffice flow graph in app?
If I could define the application data type , maybe the traffic flow grapy would be more visualble.
Thanks.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Jan 22, 2022 10:20 pm

Some times MT do change stuff, so it does not work. Since I do not have capsman, I need some help to debug it.
Can you post a list of log line here?

Example output of:
index=* "caps,info"
 
norooznoroozi85
just joined
Posts: 3
Joined: Sun Feb 06, 2022 8:25 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Feb 07, 2022 1:45 pm

hi
I added MikroTik logs 3.5 in my Splunk & done all configuration

in search tab in main splunk i can see my log from my mikrotik with host="192.168.XX.XX" command
but in app "MikroTik logs 3.5" I can not see any information or log

does this app listen on port:514??
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Feb 07, 2022 5:57 pm

Did you see the debug section 2h-2?

2h) Debugging
1. See if any data are coming inn to splunk at all.
index=*
2. Test if data has correct tag "MikroTik" (Capital M & T)
index=* | table _time sourcetype _raw
Follow this section 100%
2b) Then select what modules to log.

Splunk can listen on port 514, but not recommended since it need to run as root.
Use Rsyslog to listen on 514.
Just follow to tutorial step by step.
 
norooznoroozi85
just joined
Posts: 3
Joined: Sun Feb 06, 2022 8:25 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed Feb 09, 2022 1:02 pm

hi again
I have see the log but i wnat to see the log in your dashboard "Miktotik loge 3.5"
thanks for your answer
I receive log from Mikrtotik in search , with command line index=* , but, I donot have any data in "Miktotik loge 3.5"

this item is ok "Test if data has correct tag "MikroTik" (Capital M & T)"

if possible i can send you a photo
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed Feb 09, 2022 1:33 pm

Can you post some line output of
index=* | fillnull value="-" | table _time index sourcetype _raw
That do contains some data from router?

Do you run as this:
Splunk as root and port 514 open to Splunk
or
Splunk as non root, Splunk getting data from rsyslog that listen in 514
 
norooznoroozi85
just joined
Posts: 3
Joined: Sun Feb 06, 2022 8:25 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Feb 14, 2022 8:00 am

excuse me I confuse a lot
what do you mean by "slunk as root"??
I used Splunk 8.2 on windows server 2022 and my MikroTik Router is CCR1036 , V6.48.6

if possible , I will send you picture!!

I have my MikroTik log in my splunk but I want to see my log in your dashboard "MikroTik log 3.5"
Last edited by norooznoroozi85 on Mon Feb 14, 2022 9:01 am, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Feb 14, 2022 8:22 am

You did not post output of my command above. With that I can see if logs looks like what I expect.

I did forget to ask on what platform you do run Splunk.

Some of my first information in my first post:
Installation
1) On your PC Works on Windows and Linux, but use Linux (clearly the best choice and also used in all post here)
It should work, but I may not be able to help with windows version on the same lever as on Linux (recommended)
Linux has normal user and root user.

If you only have one server, I would suggest to install VmWare Workstation, then setup a Linux server (example Ubuntu 20.04)
Follow all steps in post above to get Splunk installed.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun Feb 27, 2022 5:50 pm

New version with Netwatch logging is not the way. See this post in this thread on how it works:
viewtopic.php?p=888800#p888800
 
thebulgarian
just joined
Posts: 1
Joined: Fri Mar 20, 2015 5:42 am
Location: Plovdiv

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Mar 07, 2022 10:12 pm

Hello everyone!

First of all thanks for this excellent tool @Jotne, I love it!

I have a little problem, I'm unable to get my CHR to visualize on my dashboard. All my other Mikrotik devices are showing correct except CHR. I have 2 CHR - 1 is 6.49.4 and the other is 7.1.3
My Splunk is recieving data, I can search for 10.0.0.56 and 10.0.0.57 and i have data, but I dont see it on the Dashboard
Here is export of my configurations in case you want and have time to help.

CHRv7.1.3:
/interface bridge
add name=bridge1 protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] disable-running-check=no
set [ find default-name=ether2 ] disable-running-check=no
set [ find default-name=ether3 ] disable-running-check=no
set [ find default-name=ether4 ] disable-running-check=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
set 1 name=serial1
/system logging action
add name=72 remote=10.0.0.72 src-address=10.0.0.57 target=remote
/interface bridge port
add bridge=bridge1 interface=ether1
/ipv6 settings
set disable-ipv6=yes
/ip cloud
set update-time=no
/ip dhcp-client
add interface=bridge1
/system hardware
set allow-x86-64=yes
/system identity
set name=CHRv7_x86_64
/system logging
add action=72 prefix=MikroTik topics=!debug,!packet,!snmp
/system ntp client
set enabled=yes
/system ntp client servers
add address=10.0.0.1
/system scheduler
add interval=5m name="Data to Splunk" on-event=Data_to_Splunk_using_Syslog \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    start-date=mar/07/2022 start-time=19:08:47
/system script
add dont-require-permissions=no name=Data_to_Splunk_using_Syslog owner=admin \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    source="# Collect information from Mikrotik RouterOS\r\
    \n# Jotne 2021\r\
    \n:log info message=\"script=version ver=4.8\"\r\
    \n# ----------------------------------\r\
    \n\r\
    \n\r\
    \n# What data to collect.  Set to false to skip the section \r\
    \n# ----------------------------------\r\
    \n:local SystemResource true\r\
    \n:local SystemInformation true\r\
    \n:local SystemHealth true\r\
    \n:local TrafficData true\r\
    \n:local AccuntData true\r\
    \n:local uPnP true\r\
    \n:local Wireless false\r\
    \n:local AddressLists true\r\
    \n:local DHCP true\r\
    \n:local Neighbor true\r\
    \n:local InterfaceData true\r\
    \n:local CmdHistory true\r\
    \n:local CAPsMANN false\r\
    \n\r\
    \n\r\
    \n# Collect system resource\r\
    \n# ----------------------------------\r\
    \n:if (\$SystemResource) do={\r\
    \n\t/system resource\r\
    \n\t:local cpuload [get cpu-load]\r\
    \n\t:local freemem ([get free-memory]/1048576)\r\
    \n\t:local totmem ([get total-memory]/1048576)\r\
    \n\t:local freehddspace ([get free-hdd-space]/1048576)\r\
    \n\t:local totalhddspace ([get total-hdd-space]/1048576)\r\
    \n\t:local up [get uptime]\r\
    \n\t:local sector [get write-sect-total]\r\
    \n\t:log info message=\"script=resource free_memory=\$freemem MB total_mem\
    ory=\$totmem MB free_hdd_space=\$freehddspace MB total_hdd_space=\$totalhd\
    dspace MB cpu_load=\$cpuload uptime=\$up write-sect-total=\$sector\"\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Make some part only run every hours\r\
    \n# ----------------------------------\r\
    \n:global Hour\r\
    \n:local run false\r\
    \n:local hour [:pick [/system clock get time] 0 2]\r\
    \n:if (\$Hour != \$hour) do={\r\
    \n\t:global Hour \$hour\r\
    \n\t:set run true\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get NTP status\r\
    \n# ----------------------------------\r\
    \n:local ntpstatus \"\"\r\
    \n:if ([:len [/system package find where !disabled and name=ntp]] > 0 or [\
    :tonum [:pick [/system resource get version] 0 1]] > 6) do={\r\
    \n    :set ntpstatus [/system ntp client get status]\r\
    \n} else={\r\
    \n    :if ([:typeof [/system ntp client get last-update-from]] = \"nil\") \
    do={\r\
    \n        :set ntpstatus \"using-local-clock\"\r\
    \n    } else={\r\
    \n        :set ntpstatus \"synchronized\"\r\
    \n    }\r\
    \n}\r\
    \n:log info message=\"script=ntp status=\$ntpstatus\" \r\
    \n\r\
    \n\r\
    \n# Get interface traffic data for all interface\r\
    \n# ----------------------------------\r\
    \n:if (\$TrafficData) do={\r\
    \n\t:foreach id in=[/interface find] do={\r\
    \n\t\t:local output \"\$[/interface print stats as-value where .id=\$id]\"\
    \r\
    \n\t\t:set ( \"\$output\"->\"script\" ) \"if_traffic\"\r\
    \n\t\t:log info message=\"\$output\"\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get traffic data v2 (Kid Control)\r\
    \n# ----------------------------------\r\
    \n:if (\$AccuntData) do={\r\
    \n\t:foreach logline in=[/ip kid-control device find] do={\r\
    \n\t\t:local output \"\$[/ip kid-control device get \$logline]\"\r\
    \n\t\t:set ( \"\$output\"->\"script\" ) \"kids\"\r\
    \n\t\t:log info message=\"\$output\"\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Finding dynmaic lines used in uPnP\r\
    \n# ----------------------------------\r\
    \n:if (\$uPnP) do={\r\
    \n\t:foreach logline in=[/ip firewall nat find where dynamic=yes and comme\
    nt~\"^upnp \"] do={\r\
    \n\t\t:local output \"\$[/ip firewall nat print as-value from=\$logline]\"\
    \r\
    \n\t\t:set ( \"\$output\"->\"script\" ) \"upnp\"\r\
    \n\t\t:log info message=\"\$output\" \r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Collect system information\r\
    \n# ----------------------------------\r\
    \n:local model na\r\
    \n:local serial na\r\
    \n:local ffirmware na\r\
    \n:local cfirmware na\r\
    \n:local ufirmware na\r\
    \n:if (\$SystemInformation and \$run) do={\r\
    \n\t:local version ([/system resource get version])\r\
    \n\t:local board ([/system resource get board-name])\r\
    \n\t:if (\$board!=\"CHR\") do={\r\
    \n\t\t/system routerboard\r\
    \n\t\t:set model ([get model])\r\
    \n\t\t:set serial ([get serial-number])\r\
    \n\t\t:set ffirmware ([get factory-firmware])\r\
    \n\t\t:set cfirmware ([get current-firmware])\r\
    \n\t\t:set ufirmware ([get upgrade-firmware])\r\
    \n\t}\r\
    \n\t:local identity ([/system identity get name])\r\
    \n\t:log info message=\"script=sysinfo version=\\\"\$version\\\" board-nam\
    e=\\\"\$board\\\" model=\\\"\$model\\\" serial=\$serial identity=\\\"\$ide\
    ntity\\\" factory-firmware=\\\"\$ffirmware\\\" current-firmware=\\\"\$cfir\
    mware\\\" upgrade-firmware=\\\"\$ufirmware\\\"\"\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Collect system health\r\
    \n# ----------------------------------\r\
    \n:if (\$SystemHealth) do={\r\
    \n\t:do {\r\
    \n\t\t# New version\r\
    \n\t\t:foreach id in=[/system health find] do={\r\
    \n\t\t\t:local health \"\$[/system health get \$id]\"\r\
    \n\t\t\t:set ( \"\$health\"->\"script\" ) \"health\"\r\
    \n\t\t\t:log info message=\"\$health\"\r\
    \n\t\t}\r\
    \n\t} on-error={\r\
    \n\t\t# Old version\r\
    \n\t\t:if (!([/system health get]~\"(state=disabled|^\\\$)\")) do={\r\
    \n\t\t\t:local health \"\$[/system health get]\"\r\
    \n\t\t\t:set ( \"\$health\"->\"script\" ) \"health\"\r\
    \n\t\t\t:log info message=\"\$health\"\r\
    \n\t\t}\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Sends wireless client data to log server \r\
    \n# ----------------------------------\r\
    \n:if (\$Wireless && [:len [/int find where type=wlan]]>0) do={\r\
    \n\t/interface wireless registration-table\r\
    \n\t:foreach i in=[find] do={\r\
    \n\t\t:log info message=\".id=\$i;ap=\$([get \$i ap]);interface=\$([get \$\
    i interface]);mac-address=\$([get \$i mac-address]);signal-strength=\$([ge\
    t \$i signal-strength]);tx-rate=\$([get \$i tx-rate]);uptime=\$([get \$i u\
    ptime]);script=wifi\"\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Count IP in address-lists\r\
    \n#----------------------------------\r\
    \n:if (\$AddressLists) do={\r\
    \n\t:local array [ :toarray \"\" ]\r\
    \n\t:local addrcntdyn [:toarray \"\"] \r\
    \n\t:local addrcntstat [:toarray \"\"] \r\
    \n\t:local test\r\
    \n\t:foreach id in=[/ip firewall address-list find] do={\r\
    \n\t\t:local rec [/ip firewall address-list get \$id]\r\
    \n\t\t:local listname (\$rec->\"list\")\r\
    \n\t\t:local listdynamic (\$rec->\"dynamic\")\r\
    \n\t\t:if (!(\$array ~ \$listname)) do={ :set array (\$array , \$listname)\
    \_}\r\
    \n\t\t:if (\$listdynamic = true) do={\r\
    \n\t\t\t:set (\$addrcntdyn->\$listname) (\$addrcntdyn->\$listname+1)\r\
    \n\t\t} else={\r\
    \n\t\t\t:set (\$addrcntstat->\$listname) (\$addrcntstat->\$listname+1)}\r\
    \n\t}\r\
    \n\t:foreach k in=\$array do={\r\
    \n\t\t:log info message=(\"script=address_lists list=\$k dynamic=\".((\$ad\
    drcntdyn->\$k)+0).\" static=\".((\$addrcntstat->\$k)+0))}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get MNDP (CDP) Neighbors\r\
    \n# ----------------------------------\r\
    \n:if (\$Neighbor and \$run) do={\r\
    \n\t:foreach neighborID in=[/ip neighbor find] do={\r\
    \n\t\t:local nb [/ip neighbor get \$neighborID]\r\
    \n\t\t:local id [:pick (\"\$nb\"->\".id\") 1 99]\r\
    \n\t\t:foreach key,value in=\$nb do={\r\
    \n\t\t\t:local newline [:find \$value \"\\n\"]\r\
    \n\t\t\t:if ([\$newline]>0) do={\r\
    \n\t\t\t\t:set value [:pick \$value 0 \$newline]\r\
    \n\t\t\t}\r\
    \n\t\t\t:log info message=\"script=neighbor nid=\$id \$key=\\\"\$value\\\"\
    \"\r\
    \n\t\t}\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Collect DHCP Pool information\r\
    \n# ----------------------------------\r\
    \n:if (\$DHCP and \$run) do={\r\
    \n\t/ip pool {\r\
    \n\t\t:local poolname\r\
    \n\t\t:local pooladdresses\r\
    \n\t\t:local poolused\r\
    \n\t\t:local minaddress\r\
    \n\t\t:local maxaddress\r\
    \n\t\t:local findindex\r\
    \n\r\
    \n# Iterate through IP Pools\r\
    \n\t\t:foreach pool in=[find] do={\r\
    \n\t\t\t:set poolname [get \$pool name]\r\
    \n\t\t\t:set pooladdresses 0\r\
    \n\t\t\t:set poolused 0\r\
    \n\r\
    \n# Iterate through current pool's IP ranges\r\
    \n\t\t\t:foreach range in=[:toarray [get \$pool range]] do={\r\
    \n\r\
    \n# Get min and max addresses\r\
    \n\t\t\t\t:set findindex [:find [:tostr \$range] \"-\"]\r\
    \n\t\t\t\t:if ([:len \$findindex] > 0) do={\r\
    \n\t\t\t\t\t:set minaddress [:pick [:tostr \$range] 0 \$findindex]\r\
    \n\t\t\t\t\t:set maxaddress [:pick [:tostr \$range] (\$findindex + 1) [:le\
    n [:tostr \$range]]]\r\
    \n\t\t\t\t} else={\r\
    \n\t\t\t\t\t:set minaddress [:tostr \$range]\r\
    \n\t\t\t\t\t:set maxaddress [:tostr \$range]\r\
    \n\t\t\t\t}\r\
    \n\r\
    \n# Calculate number of ip in one range\r\
    \n\t\t\t\t:set pooladdresses (\$maxaddress - \$minaddress)\r\
    \n\r\
    \n# /foreach range\r\
    \n\t\t\t}\r\
    \n\r\
    \n# Test if pools is used in DHCP or VPN and show leases used\r\
    \n\t\t\t:local dname [/ip dhcp-server find where address-pool=\$poolname]\
    \r\
    \n\t\t\t:if ([:len \$dname] = 0) do={\r\
    \n# No DHCP server found, assume VPN\r\
    \n\t\t\t\t:set poolused [:len [used find pool=[:tostr \$poolname]]]\r\
    \n\t\t\t} else={\r\
    \n# DHCP server found, count leases\r\
    \n\t\t\t\t:local dname [/ip dhcp-server get [find where address-pool=\$poo\
    lname] name]\r\
    \n\t\t\t\t:set poolused [:len [/ip dhcp-server lease find where server=\$d\
    name]]}\r\
    \n\r\
    \n# Send data\r\
    \n\t\t\t:log info message=(\"script=pool pool=\$poolname used=\$poolused t\
    otal=\$pooladdresses\")\r\
    \n\r\
    \n# /foreach pool\r\
    \n\t\t}\r\
    \n# /ip pool\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get detailed command history RouterOS >= v7\r\
    \n# ----------------------------------\r\
    \n:if ([:tonum [:pick [/system resource get version] 0 1]] > 6 and \$CmdHi\
    story) do={\r\
    \n\t:global cmd\r\
    \n\t:local f 0\r\
    \n\t:foreach i in=[/system history find] do={\r\
    \n\t\t:if (\$i = \$cmd) do={ :set f 1 }\r\
    \n\t\t:if (\$f != 1) do={\r\
    \n\t\t\t:log info message=\"StartCMD\"\r\
    \n\t\t\t:log info message=[/system history get \$i]\r\
    \n\t\t\t:log info message=\"EndCMD\"\r\
    \n\t\t}\r\
    \n\t}\r\
    \n\t:global cmd  [:pick [/system history find] 0]\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Test if CAPsMANN is installed, if yes, run capsmann script.\r\
    \n# ----------------------------------\r\
    \n:if ( ([:len [/interface find where type=\"cap\"]] > 0) and \$CAPsMANN) \
    do={ /system script run capsman }\r\
    \n\r\
    \n\r\
    \n# End Script"
/tool romon
set enabled=yes

CHRv6.49.4
/interface bridge
add name=bridge1 protocol-mode=none
/system logging action
add name=72 remote=10.0.0.72 src-address=10.0.0.56 target=remote
/interface bridge port
add bridge=bridge1 interface=ether1
/ip cloud
set update-time=no
/ip dhcp-client
add disabled=no interface=bridge1
/system clock manual
set time-zone=+02:00
/system identity
set name=CHR_x86_64
/system logging
add action=72 disabled=yes prefix=MikroTik topics=critical
add action=72 disabled=yes prefix=MikroTik topics=account
add action=72 disabled=yes prefix=MikroTik topics=health
add action=72 disabled=yes prefix=MikroTik topics=interface
add action=72 disabled=yes prefix=MikroTik topics=info
add action=72 prefix=MikroTik topics=!debug,!packet,!snmp
/system note
set note="\r\
    \n   _____ _    _ _____     __ _  _   \r\
    \n  / ____| |  | |  __ \\   / /| || |  \r\
    \n | |    | |__| | |__) | / /_| || |_ \r\
    \n | |    |  __  |  _  / | '_ \\__   _|\r\
    \n | |____| |  | | | \\ \\ | (_) | | |  \r\
    \n  \\_____|_|  |_|_|  \\_\\ \\___/  |_|  \r\
    \n                    ______          \r\
    \n                   |______|         \r\
    \n"
/system ntp client
set enabled=yes primary-ntp=10.0.0.1 secondary-ntp=10.0.200.0
/system ntp server
set enabled=yes
/system scheduler
add interval=5m name="Data to Splunk" on-event=Data_to_Splunk_using_Syslog \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    start-date=mar/03/2022 start-time=14:56:37
/system script
add dont-require-permissions=no name=Data_to_Splunk_using_Syslog owner=admin \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    source="# Collect information from Mikrotik RouterOS\r\
    \n# Jotne 2021\r\
    \n:log info message=\"script=version ver=4.8\"\r\
    \n# ----------------------------------\r\
    \n\r\
    \n\r\
    \n# What data to collect.  Set to false to skip the section \r\
    \n# ----------------------------------\r\
    \n:local SystemResource true\r\
    \n:local SystemInformation true\r\
    \n:local SystemHealth true\r\
    \n:local TrafficData true\r\
    \n:local AccuntData true\r\
    \n:local uPnP true\r\
    \n:local Wireless false\r\
    \n:local AddressLists true\r\
    \n:local DHCP true\r\
    \n:local Neighbor true\r\
    \n:local InterfaceData true\r\
    \n:local CmdHistory true\r\
    \n:local CAPsMANN false\r\
    \n\r\
    \n\r\
    \n# Collect system resource\r\
    \n# ----------------------------------\r\
    \n:if (\$SystemResource) do={\r\
    \n\t/system resource\r\
    \n\t:local cpuload [get cpu-load]\r\
    \n\t:local freemem ([get free-memory]/1048576)\r\
    \n\t:local totmem ([get total-memory]/1048576)\r\
    \n\t:local freehddspace ([get free-hdd-space]/1048576)\r\
    \n\t:local totalhddspace ([get total-hdd-space]/1048576)\r\
    \n\t:local up [get uptime]\r\
    \n\t:local sector [get write-sect-total]\r\
    \n\t:log info message=\"script=resource free_memory=\$freemem MB total_mem\
    ory=\$totmem MB free_hdd_space=\$freehddspace MB total_hdd_space=\$totalhd\
    dspace MB cpu_load=\$cpuload uptime=\$up write-sect-total=\$sector\"\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Make some part only run every hours\r\
    \n# ----------------------------------\r\
    \n:global Hour\r\
    \n:local run false\r\
    \n:local hour [:pick [/system clock get time] 0 2]\r\
    \n:if (\$Hour != \$hour) do={\r\
    \n\t:global Hour \$hour\r\
    \n\t:set run true\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get NTP status\r\
    \n# ----------------------------------\r\
    \n:local ntpstatus \"\"\r\
    \n:if ([:len [/system package find where !disabled and name=ntp]] > 0 or [\
    :tonum [:pick [/system resource get version] 0 1]] > 6) do={\r\
    \n    :set ntpstatus [/system ntp client get status]\r\
    \n} else={\r\
    \n    :if ([:typeof [/system ntp client get last-update-from]] = \"nil\") \
    do={\r\
    \n        :set ntpstatus \"using-local-clock\"\r\
    \n    } else={\r\
    \n        :set ntpstatus \"synchronized\"\r\
    \n    }\r\
    \n}\r\
    \n:log info message=\"script=ntp status=\$ntpstatus\" \r\
    \n\r\
    \n\r\
    \n# Get interface traffic data for all interface\r\
    \n# ----------------------------------\r\
    \n:if (\$TrafficData) do={\r\
    \n\t:foreach id in=[/interface find] do={\r\
    \n\t\t:local output \"\$[/interface print stats as-value where .id=\$id]\"\
    \r\
    \n\t\t:set ( \"\$output\"->\"script\" ) \"if_traffic\"\r\
    \n\t\t:log info message=\"\$output\"\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get traffic data v2 (Kid Control)\r\
    \n# ----------------------------------\r\
    \n:if (\$AccuntData) do={\r\
    \n\t:foreach logline in=[/ip kid-control device find] do={\r\
    \n\t\t:local output \"\$[/ip kid-control device get \$logline]\"\r\
    \n\t\t:set ( \"\$output\"->\"script\" ) \"kids\"\r\
    \n\t\t:log info message=\"\$output\"\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Finding dynmaic lines used in uPnP\r\
    \n# ----------------------------------\r\
    \n:if (\$uPnP) do={\r\
    \n\t:foreach logline in=[/ip firewall nat find where dynamic=yes and comme\
    nt~\"^upnp \"] do={\r\
    \n\t\t:local output \"\$[/ip firewall nat print as-value from=\$logline]\"\
    \r\
    \n\t\t:set ( \"\$output\"->\"script\" ) \"upnp\"\r\
    \n\t\t:log info message=\"\$output\" \r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Collect system information\r\
    \n# ----------------------------------\r\
    \n:local model na\r\
    \n:local serial na\r\
    \n:local ffirmware na\r\
    \n:local cfirmware na\r\
    \n:local ufirmware na\r\
    \n:if (\$SystemInformation and \$run) do={\r\
    \n\t:local version ([/system resource get version])\r\
    \n\t:local board ([/system resource get board-name])\r\
    \n\t:if (\$board!=\"CHR\") do={\r\
    \n\t\t/system routerboard\r\
    \n\t\t:set model ([get model])\r\
    \n\t\t:set serial ([get serial-number])\r\
    \n\t\t:set ffirmware ([get factory-firmware])\r\
    \n\t\t:set cfirmware ([get current-firmware])\r\
    \n\t\t:set ufirmware ([get upgrade-firmware])\r\
    \n\t}\r\
    \n\t:local identity ([/system identity get name])\r\
    \n\t:log info message=\"script=sysinfo version=\\\"\$version\\\" board-nam\
    e=\\\"\$board\\\" model=\\\"\$model\\\" serial=\$serial identity=\\\"\$ide\
    ntity\\\" factory-firmware=\\\"\$ffirmware\\\" current-firmware=\\\"\$cfir\
    mware\\\" upgrade-firmware=\\\"\$ufirmware\\\"\"\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Collect system health\r\
    \n# ----------------------------------\r\
    \n:if (\$SystemHealth) do={\r\
    \n\t:do {\r\
    \n\t\t# New version\r\
    \n\t\t:foreach id in=[/system health find] do={\r\
    \n\t\t\t:local health \"\$[/system health get \$id]\"\r\
    \n\t\t\t:set ( \"\$health\"->\"script\" ) \"health\"\r\
    \n\t\t\t:log info message=\"\$health\"\r\
    \n\t\t}\r\
    \n\t} on-error={\r\
    \n\t\t# Old version\r\
    \n\t\t:if (!([/system health get]~\"(state=disabled|^\\\$)\")) do={\r\
    \n\t\t\t:local health \"\$[/system health get]\"\r\
    \n\t\t\t:set ( \"\$health\"->\"script\" ) \"health\"\r\
    \n\t\t\t:log info message=\"\$health\"\r\
    \n\t\t}\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Sends wireless client data to log server \r\
    \n# ----------------------------------\r\
    \n:if (\$Wireless && [:len [/int find where type=wlan]]>0) do={\r\
    \n\t/interface wireless registration-table\r\
    \n\t:foreach i in=[find] do={\r\
    \n\t\t:log info message=\".id=\$i;ap=\$([get \$i ap]);interface=\$([get \$\
    i interface]);mac-address=\$([get \$i mac-address]);signal-strength=\$([ge\
    t \$i signal-strength]);tx-rate=\$([get \$i tx-rate]);uptime=\$([get \$i u\
    ptime]);script=wifi\"\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Count IP in address-lists\r\
    \n#----------------------------------\r\
    \n:if (\$AddressLists) do={\r\
    \n\t:local array [ :toarray \"\" ]\r\
    \n\t:local addrcntdyn [:toarray \"\"] \r\
    \n\t:local addrcntstat [:toarray \"\"] \r\
    \n\t:local test\r\
    \n\t:foreach id in=[/ip firewall address-list find] do={\r\
    \n\t\t:local rec [/ip firewall address-list get \$id]\r\
    \n\t\t:local listname (\$rec->\"list\")\r\
    \n\t\t:local listdynamic (\$rec->\"dynamic\")\r\
    \n\t\t:if (!(\$array ~ \$listname)) do={ :set array (\$array , \$listname)\
    \_}\r\
    \n\t\t:if (\$listdynamic = true) do={\r\
    \n\t\t\t:set (\$addrcntdyn->\$listname) (\$addrcntdyn->\$listname+1)\r\
    \n\t\t} else={\r\
    \n\t\t\t:set (\$addrcntstat->\$listname) (\$addrcntstat->\$listname+1)}\r\
    \n\t}\r\
    \n\t:foreach k in=\$array do={\r\
    \n\t\t:log info message=(\"script=address_lists list=\$k dynamic=\".((\$ad\
    drcntdyn->\$k)+0).\" static=\".((\$addrcntstat->\$k)+0))}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get MNDP (CDP) Neighbors\r\
    \n# ----------------------------------\r\
    \n:if (\$Neighbor and \$run) do={\r\
    \n\t:foreach neighborID in=[/ip neighbor find] do={\r\
    \n\t\t:local nb [/ip neighbor get \$neighborID]\r\
    \n\t\t:local id [:pick (\"\$nb\"->\".id\") 1 99]\r\
    \n\t\t:foreach key,value in=\$nb do={\r\
    \n\t\t\t:local newline [:find \$value \"\\n\"]\r\
    \n\t\t\t:if ([\$newline]>0) do={\r\
    \n\t\t\t\t:set value [:pick \$value 0 \$newline]\r\
    \n\t\t\t}\r\
    \n\t\t\t:log info message=\"script=neighbor nid=\$id \$key=\\\"\$value\\\"\
    \"\r\
    \n\t\t}\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Collect DHCP Pool information\r\
    \n# ----------------------------------\r\
    \n:if (\$DHCP and \$run) do={\r\
    \n\t/ip pool {\r\
    \n\t\t:local poolname\r\
    \n\t\t:local pooladdresses\r\
    \n\t\t:local poolused\r\
    \n\t\t:local minaddress\r\
    \n\t\t:local maxaddress\r\
    \n\t\t:local findindex\r\
    \n\r\
    \n# Iterate through IP Pools\r\
    \n\t\t:foreach pool in=[find] do={\r\
    \n\t\t\t:set poolname [get \$pool name]\r\
    \n\t\t\t:set pooladdresses 0\r\
    \n\t\t\t:set poolused 0\r\
    \n\r\
    \n# Iterate through current pool's IP ranges\r\
    \n\t\t\t:foreach range in=[:toarray [get \$pool range]] do={\r\
    \n\r\
    \n# Get min and max addresses\r\
    \n\t\t\t\t:set findindex [:find [:tostr \$range] \"-\"]\r\
    \n\t\t\t\t:if ([:len \$findindex] > 0) do={\r\
    \n\t\t\t\t\t:set minaddress [:pick [:tostr \$range] 0 \$findindex]\r\
    \n\t\t\t\t\t:set maxaddress [:pick [:tostr \$range] (\$findindex + 1) [:le\
    n [:tostr \$range]]]\r\
    \n\t\t\t\t} else={\r\
    \n\t\t\t\t\t:set minaddress [:tostr \$range]\r\
    \n\t\t\t\t\t:set maxaddress [:tostr \$range]\r\
    \n\t\t\t\t}\r\
    \n\r\
    \n# Calculate number of ip in one range\r\
    \n\t\t\t\t:set pooladdresses (\$maxaddress - \$minaddress)\r\
    \n\r\
    \n# /foreach range\r\
    \n\t\t\t}\r\
    \n\r\
    \n# Test if pools is used in DHCP or VPN and show leases used\r\
    \n\t\t\t:local dname [/ip dhcp-server find where address-pool=\$poolname]\
    \r\
    \n\t\t\t:if ([:len \$dname] = 0) do={\r\
    \n# No DHCP server found, assume VPN\r\
    \n\t\t\t\t:set poolused [:len [used find pool=[:tostr \$poolname]]]\r\
    \n\t\t\t} else={\r\
    \n# DHCP server found, count leases\r\
    \n\t\t\t\t:local dname [/ip dhcp-server get [find where address-pool=\$poo\
    lname] name]\r\
    \n\t\t\t\t:set poolused [:len [/ip dhcp-server lease find where server=\$d\
    name]]}\r\
    \n\r\
    \n# Send data\r\
    \n\t\t\t:log info message=(\"script=pool pool=\$poolname used=\$poolused t\
    otal=\$pooladdresses\")\r\
    \n\r\
    \n# /foreach pool\r\
    \n\t\t}\r\
    \n# /ip pool\r\
    \n\t}\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Get detailed command history RouterOS >= v7\r\
    \n# ----------------------------------\r\
    \n:if ([:tonum [:pick [/system resource get version] 0 1]] > 6 and \$CmdHi\
    story) do={\r\
    \n\t:global cmd\r\
    \n\t:local f 0\r\
    \n\t:foreach i in=[/system history find] do={\r\
    \n\t\t:if (\$i = \$cmd) do={ :set f 1 }\r\
    \n\t\t:if (\$f != 1) do={\r\
    \n\t\t\t:log info message=\"StartCMD\"\r\
    \n\t\t\t:log info message=[/system history get \$i]\r\
    \n\t\t\t:log info message=\"EndCMD\"\r\
    \n\t\t}\r\
    \n\t}\r\
    \n\t:global cmd  [:pick [/system history find] 0]\r\
    \n}\r\
    \n\r\
    \n\r\
    \n# Test if CAPsMANN is installed, if yes, run capsmann script.\r\
    \n# ----------------------------------\r\
    \n:if ( ([:len [/interface find where type=\"cap\"]] > 0) and \$CAPsMANN) \
    do={ /system script run capsman }\r\
    \n\r\
    \n\r\
    \n# End Script\r\
    \n"
Thanks!
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu Mar 17, 2022 8:12 am

@jotne

I've never used Splunk before but thought I would give it a try for home.

Under MikroTik Device Traffic (although I properly have kid-control enabled), the hostname is blank for all entries, even though I can see in kid-control devices the hostname is shown for some of those. Also instead of showing all IP addresses the device has, it only shows the first one in the list. Other than that most features seem to work.

I am using CAPsMAN as well but there is no data displayed there.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu Mar 17, 2022 8:39 am

Let it run for least on day. Some script are just run every 24 hours.
 
User avatar
gpmendive
just joined
Posts: 2
Joined: Sun Mar 27, 2022 3:47 pm
Location: Ciudad de Buenos Aires

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Apr 02, 2022 7:47 pm

Thanks a lot Jotne for your App for Splunk!
It´s my first installation of Splunk and your guide proved to be very helpful.

I first installed both on Ubuntu server running on a physical PC. I tried it for several days and in performed great.
Then I installed on same OS but running in a ProxmoxVE Virtual Machine; great performance also!
I will stick to the latter configuration.

Thanks again to you, and also to the other members who contributed to this topic.
 
User avatar
LogicalNZ
Frequent Visitor
Frequent Visitor
Posts: 59
Joined: Sat Oct 19, 2013 6:35 am
Location: New Zealand
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Apr 16, 2022 11:03 am

I’m really interested in Splunk.

I have been using GreyLog for a couple of years for Syslog management on my Tiks. Can someone help me understand when using Splunk with a Tik, what would be the advantage over GreyLog?
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Apr 18, 2022 5:09 pm

My 2 cents,
I don't think there are any advantages. In theory with GreyLog you can do the same (or should be able to do so), but obviously the key is that for Mikrotik Jotne has provided some nice app/dashboards to work with.
 
MeMB
just joined
Posts: 1
Joined: Fri Mar 25, 2022 8:48 am

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed Apr 20, 2022 1:21 pm

Do i need to add any apps to Splunk before proceeding so that V3.5 can communicate with Splunk.
I have followed the start of the tutorial, however my Mik does not even show in the device list.
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Apr 23, 2022 12:40 pm

What do you get when search for
index=*
Do you use rsyslog or are you running Splunk as root and listen on port 514?
See section
3b) Debugging
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Apr 23, 2022 12:44 pm

I have been using GreyLog for a couple of years for Syslog management on my Tiks. Can someone help me understand when using Splunk with a Tik, what would be the advantage over GreyLog?
GreyLog and Splunk are the two mayor log receiving system.
One is 100% free, other is free up to 500MB log / day.

Do a google for
graylog vs splunk
I do like Splunk, since its what I can :)
 
User avatar
siscom
Member Candidate
Member Candidate
Posts: 192
Joined: Tue May 26, 2009 6:37 pm
Location: Malta, EU.

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue May 03, 2022 10:59 am

Hi Jotne,

1 - Thank you for this app - very clean and useful.
2 - Problem - VPN Connections not working well

This was all working ok until I introduced a 2FA solution. What happens is that on a user accessing the L2TP/IPsec server, a request is sent to a radius server that sends a request to a service which then sends a request to the users's mobile for auth. Once this happens, the service sends a reply back to radius which in turn advises the router to admit the user. All this takes time which means the logging is not within the same few seconds it usually is. This seems to cause Splunk/Mikrotik app to lose the login.

If I log in using the same user without 2FA (tried with both PPTP & L2TP/IPsec), Splunk logs it 100% and if you look at the router log, only the timing is the difference.

I tried to modify the MAX_TIMESTAMP_LOOKAHEAD value in the props.conf in the Mikrotik app directory (restarted Splunk) but this made no difference.
Any idea what could be causing this?

Rgds,
Mark
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue May 03, 2022 8:29 pm

Do you see the logs in splunk?
index=*
Can you post some example line?
 
juancipolletti
just joined
Posts: 2
Joined: Fri Jul 27, 2018 5:37 pm

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed May 04, 2022 8:33 pm

Hi Jotne,I'm new to the forum and to splunk, I have my lab upstairs with an ubuntu server with rsyslog and a mikrotik rb3011/6.42.9v router, the logs arrive in my rsyslog, but I can't see them in splunk. I would appreciate any help.
 
User avatar
siscom
Member Candidate
Member Candidate
Posts: 192
Joined: Tue May 26, 2009 6:37 pm
Location: Malta, EU.

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed May 04, 2022 9:37 pm

Hi,

Thank you for the reply. I tried to run the Search with the index=* parameter but there is too much data being displayed. Any way to get something only related to the logins?

Rgds,
Mark.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed May 04, 2022 9:56 pm

@siscom try some search like this.
index=* sourcetype=mikrotik  eventtype IN (*tp_connection_from,*tp_user_logged_in,ppp_authentication_failed,l2tp_user_logged_out)
or
index=* sourcetype=mikrotik  ppp
Without seeing what you get its not easy,
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.5 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed May 04, 2022 10:03 pm

Hi Jotne,I'm new to the forum and to splunk, I have my lab upstairs with an ubuntu server with rsyslog and a mikrotik rb3011/6.42.9v router, the logs arrive in my rsyslog, but I can't see them in splunk. I would appreciate any help.
Do this search give any Mikrotik data?
index=*
You have followed this part 100%
viewtopic.php?p=888803#p888803
Do this file exist:
%SplunkHome%/etc/system/local/inputs.conf
Under
Settings->Data Inputs->Files & Directories you should see rsyslog section and a number of files it sees.
udp.png
.
You do not have the required permissions to view the files attached to this post.
Last edited by Jotne on Mon May 16, 2022 11:16 pm, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu May 05, 2022 11:44 am

Upgraded to 3.6
# 3.6 (05.05.2022)
# NB Delete old app (copy custom made config) before install v3.6
# Change data to store in Mikrotik index, instead of default index
# Change how rsyslog handles data. Did fail if there was more than one type of input
# Updeted script in "MikroTik DHCP to Static"
# Uses new Index, important to look at macros.conf and set correct index.
# Added colors to "MikroTik Admin user login"
This version no longer uses default index (main). It will create its own index name (mikrotik)
I do suggest that you remove old installed version before install the new version due to index change.
New data will be stored in new index and old data will remain in old main index. App will search both.

If you for some reason needs another custom index name, you can do that as well. Just edit the macros.conf to indicate what index to search,
 
tahir491
just joined
Posts: 1
Joined: Sat May 07, 2022 1:02 pm

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat May 07, 2022 1:19 pm

Hi Every one
I'm unable to find Mikrotik.spl file can someone please help for finding this file
 
zandhaas
Frequent Visitor
Frequent Visitor
Posts: 73
Joined: Tue Dec 11, 2018 11:02 pm
Location: The Netherlands

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat May 07, 2022 3:47 pm

The Mikrotik.SPL file is packaged as a RAR file and you can download this under paragraph 1G in the first post of this thread.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat May 07, 2022 6:43 pm

File is find in section 1g) in the first post. And I did forget to upload 3.6 when I had written that it was upgraded. Fixed now.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue May 10, 2022 2:22 pm

Small bug found that will come in 3.7 if you do use rsyslog.

Quick fix to get it to work.

Change this part of props.conf from
[rsyslog]
TRUNCATE = 10000
TRANSFORMS-dns = remove_dns_query,remove_dns_answer
TRANSFORMS-force_mikrotik = force_mikrotik_st,force_mikrotik_ix

To
[rsyslog]
TRUNCATE = 10000
TRANSFORMS-dns = remove_dns_query,remove_dns_answer
TRANSFORMS-force_mikrotik = force_mikrotik_st,force_mikrotik_ix
SEDCMD-clean_header = s/\d{4}-\d\d-\d\d.*?<\d+>//
SEDCMD-clean_end = s/#015$//
 
User avatar
jwshields
just joined
Posts: 6
Joined: Wed Aug 05, 2020 2:34 am
Location: Seattle, WA, USA
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue May 10, 2022 6:54 pm

I work with Splunk daily as part of my day job, managing a medium sized cluster of around ~50 nodes, consuming around 5TB of events per day. I absolutely love getting my hands dirty in apps and building things for Splunk.
Is this the same Mikrotik App that was out about 1.5-2 years ago, and has just been updated? Or a completely different app?

Are you interested in open-sourcing the app and hosting it on [your choice of a publicly hosted Git platform]? I would love to contribute and work on this app if at all possible, and if there is interest.
Let me know if you're open to open-sourcing the Splunk app, or collaborating together.
You should be able to DM me or email me on here, but if you're interested, let me know. Would love to chat more about this! Your screenshots look so beautiful!

- Jared
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Tue May 10, 2022 7:05 pm

I would have no problem to make this app better and working together :)
Yes its the same app that has been around since at least 2017.

My level of programming skill is not at a high level, but know some and also working with splunk as a main work.
We have 50+ Splunk server and 1+ TB a day from 3k+ servers +++
 
eaf
just joined
Posts: 3
Joined: Tue Apr 19, 2022 10:20 pm

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed May 11, 2022 10:53 pm

I have a question regarding the below snippet. What is [/ip kid-control device find] supposed to print/return? For me it prints nothing. And kid control has been enabled as per the instructions.

Also a couple of typos here: "$AccuntData" and "dynmaic".

:if ($AccuntData) do={
:foreach logline in=[/ip kid-control device find] do={
:local output "$[/ip kid-control device get $logline]"
:set ( "$output"->"script" ) "kids"
:log info message="$output"
}
}


# Finding dynmaic lines used in uPnP
# ----------------------------------
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed May 11, 2022 11:29 pm

Here is output of my settings
/ip/kid-control> export
# may/11/2022 22:25:17 by RouterOS 7.2.3
# software id = E4B6-AAAA
#
# model = RouterBOARD 750G r3
# serial number = xxxxx
/ip kid-control
add fri=0s-1d mon=0s-1d name=Monitor sat=0s-1d sun=0s-1d thu=0s-1d tue=0s-1d wed=0s-1d

To see the actual data
 /ip kid-control device/print 
Flags: D, L - LIMITED; I - INACTIVE
Columns: NAME, MAC-ADDRESS, IDLE-TIME, RATE-DOWN, RATE-UP
 #    NAME                       MAC-ADDRESS        IDLE-TIME   RATE-DOWN  RATE-UP 
 0 D                             00:05:00:01:00:01  2s          0bps       0bps    
 1 D                             AC:CA:54:00:AA:CC  2s          0bps       0bps    
 2 D                             00:1D:EC:06:AA:83  2s          0bps       0bps    
 3 D  vuxxxx-xxx                00:1D:EC:AA:92:6D  1s          0bps       0bps    
 4 D                             00:40:8C:DF:AA:44  3m37s       0bps       0bps    
 5 D                             90:B1:1C:8E:AA:6E  0s          49.7kbps   32.6kbps
 6 D                             90:B1:1C:68:AA:D1  0s          29.9kbps   4.8kbps 
 7 D                             EE:9D:21:79:AA:CC  3m50s       0bps       0bps    
 8 D                             4C:5E:0C:0E:AA:F5  39s         0bps       0bps    
 9 D  HUAWEI_P_smart_Z-3006e00b  74:AA:09:53:A7:E9  5s          0bps       0bps    
10 D  ESP_4E46B6                 CC:50:E3:AA:46:B6  1s          0bps       0bps    
11 D  S21-pol-zovatela-rtes     BA:E8:4C:AA:DF:FD  31s         0bps       0bps    
12 D  Pulsecf129d87a0            A4:CF:12:AA:87:A0  3s          0bps       0bps    
13 D                             5C:83:8F:0C:AA:AC  11s         0bps       0bps    
14 D                             C4:AD:34:B1:AA:CE  49s         0bps       0bps    
15 D                             00:C0:B7:C2:AA:0F              0bps       0bps    
16 D  raspberrypi                00:13:EF:AA:2F:E3  1m54s       0bps       0bps 


If you do not see anything, I am not sure what is wrong.
Should work both on both RuterOS 6 and 7.
 
eaf
just joined
Posts: 3
Joined: Tue Apr 19, 2022 10:20 pm

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu May 12, 2022 12:02 am

Ah, sorry, some major misunderstanding of MT scripting on my side. "/ip kid-control device find" isn't supposed to print anything. ":put [/ip kid-control device find]" will.

I was just trying to see what kind of log lines that snippet was supposed to produce. Apparently something like this:

.id=*1;activity=i.scdn.co;blocked=false;bytes-down=6980168;bytes-up=742091;disabled=false;dynamic=true;idle-time=00:00:01;inactive=false;ip-address=192.168.88.7;limited=false;mac-address=74:D6:37:70:58:41;name=amazon-cd1e20631;rate-down=504;rate-up=920;user=
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu May 12, 2022 10:56 pm

Hi Guys,

An interesting discussion is going on here. I have been looking for this type of solution for a while. However, I have few questions for @jotnet

1. Can a report be generated for the individual device?

2. Can the script log the address list so I can perform an audit on affected IPs.

3. I dont understand your naming convention...are you referring to the comment given to the rules?

4. What if the minimum requirement for the splunk and can any of the AWS t2 micro or nano be sufficient?

I look forward to your reply.

Regards
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Fri May 13, 2022 12:04 am

1. Can a report be generated for the individual device?

Sure, in the dashboard there is a "Host" selector if you have multiple Mikrotiks


2. Can the script log the address list so I can perform an audit on affected IPs.

"Log the address list" ? In the dashboard you'll see the IP-address of the device generating the traffic. So you, you can trace it back yes.


3. I dont understand your naming convention...are you referring to the comment given to the rules?

You can name the FW-rules how you want, but he gives a proposal that could be interesting.
I run completely different naming and this works fine, just make sure you do not exceed 20characters.



4. What if the minimum requirement for the splunk and can any of the AWS t2 micro or nano be sufficient?

Hmm, these are *very* lightweight ... t2 micro = 1vCPU and 1GBytes RAM and a t2 nano = 1vCPU and 512Mbytes RAM
That's gonna be a problem I think. I run Splunk on a VM (on my Synology NAS) with 1vCPU and 4Gbytes RAM and it is quite slow and collector only data from 1 Mikrotik.
Good enough for "hobby" but if I compare it to the performance of the Splunk systems I have access to professionally its a bit of a joke ;-)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Fri May 13, 2022 8:18 am


1. Can a report be generated for the individual device?
Yes
2. Can the script log the address list so I can perform an audit on affected IPs.
Yes
3. I dont understand your naming convention...are you referring to the comment given to the rules?
Yes its the naming of filter/nat rules to make it easier to se what is what int the graphical view
4. What if the minimum requirement for the splunk and can any of the AWS t2 micro or nano be sufficient?
Not sure, but it depends on the amount of logged each day and how often you search the data.
Running the 500MB/day limit (lots of stuff logged) does run fine on and older linux PC (16GB ram)
Using SSD as storage speeds up the dashboard view.

If there are some function missing Splunk is very flexible and can add almost any thing you like.
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Fri May 13, 2022 11:45 am

HI Guys,

Thank you for the brilliant response. I am a product manager and security expert. I love to deploy Mikrotik firewall and bandwidth management in the cooperate or enterprise environment, In most cases, some of our customers have replaced their Cisco ASA with our customized Mikrotik Security Boxes. However, one of the issues we keep having is a customized logging system for reporting, be it for regulatory compliance reasons or internal policies.

Splunk appears like software with such flexibility. I will like to work with someone with the experience on how to customize Splunk to suit the range of reporting required by our customers. One of such is having address-list on Splunk. We have various address-lists with captured src or dst address that violates certain policy and the administrator may want to do a drill-down.

e.g if we have the policy to capture the src ip of those using VPN to bypass content filtering, the admin will want to know the users and be able to take necessary actions to forestall such action. Mikrotik firewall can capture such IP in address-list for future evaluation on splunk.

Also, does Splunk provide domain names of destination hosts when generating reports?

I look forward to your usual brilliant contribution
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Fri May 13, 2022 1:40 pm

Splunk can do all you ask about and more. Nearly unlimited possibility. It do cost allot of money if you put it inn to a large scale company, but may be the best and most flexible solution out there.

For your IP address list, you can hav tem in a csv file that Splunk uses. This fil can be updated automatically if need.
You can then make alerts or reports that checks data logged against this table and send alerts of make graphs.

Domain name are normally part of your DNS solution, so you can just make Splunk do a lookup at your DSN servers for domain name,
Or you can add it as an identity to the router it self and make Splunk read it from there.

To use Splunk, you need to get data inn to it (File/Agents/Syslog/HTTP requests/Scripts +++++), then the next step is to graph it.
This can be an application you download (free or paid) or you can make dashboard your self.

I my organisation I am in charge og logging and using Splunk to handle it. We do get 1TB + of data a day from 3-4000 server, 1000+ switches/routers ++++++++. This shows a basic overview of our Splunk design (50+ Splunk dedicated servers).
.
splunk.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Fri May 13, 2022 2:23 pm

Splunk can do all you ask about and more. Nearly unlimited possibility. It do cost allot of money if you put it inn to a large scale company, but may be the best and most flexible solution out there.

For your IP address list, you can hav tem in a csv file that Splunk uses. This fil can be updated automatically if need.
You can then make alerts or reports that checks data logged against this table and send alerts of make graphs.

Domain name are normally part of your DNS solution, so you can just make Splunk do a lookup at your DSN servers for domain name,
Or you can add it as an identity to the router it self and make Splunk read it from there.

To use Splunk, you need to get data inn to it (File/Agents/Syslog/HTTP requests/Scripts +++++), then the next step is to graph it.
This can be an application you download (free or paid) or you can make dashboard your self.

I my organisation I am in charge og logging and using Splunk to handle it. We do get 1TB + of data a day from 3-4000 server, 1000+ switches/routers ++++++++. This shows a basic overview of our Splunk design (50+ Splunk dedicated servers).
.
splunk.png
Wow!

I did a lookup your profile to see i could do you a DM. Mine is holler4eva@gmail.com. Lets take this further.

Regards

Abiola
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 1:34 pm

Hi Guys,

I have been able to launch an EC2 instance (on the free tier for testing purposes) on AWS and installed splunk successfully. However, I am having issues on the rsyslog section of the installation process. see my questions below:

Where is /etc/rsyslog drirectory ? - in the root user account or splunk (/opt/splunk) i couldnt locate it.

Where are the new syslog directory be created ? root user account or splunk ?

REgards
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 2:18 pm

Did try to send you an email, but did not get delivered, so did try one more just now.

I do use Ubuntu and there all rsyslog are installed as default in folder /etc/rsyslog.d/ and as user root.
In the config there are settings that points to where to store syslog data, udp.conf that points to folder /data/syslog/udp/ .

So rsyslog runs as root and Splunk runs as Splunk user.
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 3:25 pm

HI,

I just replied your email. BTW, thank you for the clarification. I am already on it. I will revert incase of any issue.

Thank you

Regards
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 4:24 pm

I am getting error from splunk web interface...

The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. What does this mean?

I am using AWS t2micro. It comes with 8GB storage
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 4:51 pm

I am having error is adding UDP/514 to Data Inputs in Splunk. See attached
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 7:03 pm

Ahh, If you use rsyslog, it uses UDP port 514, so you can not add it to Splunk. Only one app can use one given port. And since you need to be root to use port 514 (<1024), the app needs to run as root. And since its not recommended to run Splunk as root, I let rsyslog get the data by it listen to port 514.
Splunk do get log data from Splunk by reading (following) folder
/data/syslog/udp/*
In file
splunk/etc/system/local/inputs.conf
there are a section that collects the rsyslog data.
[monitor:///data/syslog/tcp/.../*.log]
sourcetype = rsyslog
host_segment = 4
Or you can see it here:
Settings->Data Input->Files & Directories
/data/syslog/udp/.../*.log	
Segment
rsyslog
default
11711	
system
Enabled | Disable	Delete
Since this number 11711 do increase, it shows that it reads new data.
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 9:48 pm

I followed your instrction based on steps 1b and 1f.

I installed splunk as non root and deploy rsyslog.

1f required we specify dns/514 for syslog as the the sourcetype.

Are you trying to say 1f is not required?

please reply
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sun May 15, 2022 11:08 pm

Updated section 1f, to make clear your can not use UDP/514 in Splunk if Splunk is not run as root. You then need external rsyslog server.
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 4:36 pm

Hey Guys,

I need your help. My dashboard isnt reading data despite my router keeps loggin data to splunk.

When I issued the command sudo vim /data/syslog/udp/41.X.X.193 (where 41.X.X.193) is my router's IP), see the output below

" ============================================================================
" Netrw Directory Listing (netrw v165)
" /data/syslog/udp/41.X.X.193
" Sorted by name
" Sort sequence: [\/]$,\<core\%(\.\d\+\)\=\>,\.h$,\.c$,\.cpp$,\~\=\*$,*,\.o$,\.obj$,\.info$,\.swp$,\.bak$,\~$
" Quick Help: <F1>:help -:go up dir D:delete R:rename s:sort-by x:special
" ==============================================================================
../
./
20220515-15.log
20220515-16.log
20220515-17.log
20220515-18.log
20220515-19.log
20220515-20.log
20220515-21.log
20220515-22.log
20220515-23.log
20220516-00.log
20220516-01.log
20220516-02.log
20220516-03.log
20220516-04.log
20220516-05.log
20220516-06.log
20220516-07.log
20220516-08.log
20220516-09.log
20220516-10.log
20220516-11.log
20220516-12.log
20220516-13.log

I believe this is a piece of evidence that my Mikrotik is logging data to Splunk. If yes, what could be the possible reason the dashboard is empty.

KIndly assist with your brilliant suggestion.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 5:31 pm

Whats inn those files? paste some lines. See section 3b Debuging

If syslog folder has data, it could be one of two.
Not using MikroTik tag (Capital M and capital T)
Splunk not reading data.

What do search for
index=*
give
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 7:10 pm

Whats inn those files? paste some lines. See section 3b Debuging

If syslog folder has data, it could be one of two.
Not using MikroTik tag (Capital M and capital T)
Splunk not reading data.

What do search for
index=*
give
See the line from the syslog file
2022-05-16T16:00:00.259726+00:00 <13>firewall,info MikroTik: AT_Accpt_Https Allowed Traffic: in:BV-Br out:ether2-WAN, src-mac d8:b0:53:1b:97:a6, proto TCP (SYN), 192.168.10.81:50032->216.58.223.202:443, len 60
2022-05-16T16:00:00.259989+00:00 <13>firewall,info MikroTik: SrcNat_Masq srcnat: in:(unknown 0) out:ether2-WAN, src-mac d8:b0:53:1b:97:a6, proto TCP (SYN), 192.168.10.81:50032->216.58.223.202:443, len 60
2022-05-16T16:00:00.260448+00:00 <13>firewall,info MikroTik: RS_Drop_Youtube Restricted Site: in:BV-Br out:ether2-WAN, src-mac d8:b0:53:1b:97:a6, proto TCP (ACK,PSH), 192.168.10.81:50032->216.58.223.202:443, NAT (192.168.10.81:50032->41.X.X.193:50032)->216.58.223.202:443, len 569

About index=*, see below:
ubuntu@ip-172-31-28-110:~$ index=*
ubuntu@ip-172-31-28-110:~$ index=* | table _time sourcetype _raw

Command 'table' not found, did you mean:

command 'ptable' from deb xcrysden (1.6.2-3build1)
command 'tabble' from deb tabble (0.43-3)

Try: sudo apt install <deb name>

ubuntu@ip-172-31-28-110:~$

About caps for M & T. it is confirmed OK.

I am suspecting the right of Splunk. OR what do you think
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 7:22 pm

It seems that rsyslog data looks fine.
But to get the firewall data in firewall dashboard correctly, you should name the rule as in section. 2c

In splunk go to this setting:
Settings->Data Input->Files & Directories
Do you see a line starting with?
/data/syslog/udp/.../*.log

This command
index=* | table _time sourcetype _raw
should be run in Splunk search window, not in linux command line
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 8:02 pm

It seems that rsyslog data looks fine.
But to get the firewall data in firewall dashboard correctly, you should name the rule as in section. 2c

In splunk go to this setting:
Settings->Data Input->Files & Directories
Do you see a line starting with?
/data/syslog/udp/.../*.log

NO. there is no line like that

This command
index=* | table _time sourcetype _raw
should be run in Splunk search window, not in linux command line
I ran the command, it did not return any result
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 8:17 pm

Then you have skipped this part in rsyslog setup:

To make Splunk read rsyslog data make this file: %SplunkHome%/etc/system/local/inputs.conf
[monitor:///data/syslog/udp/.../*.log]
sourcetype = rsyslog
host_segment=4

[monitor:///data/syslog/tcp/.../*.log]
sourcetype = rsyslog
host_segment=4
NB Splunk needs to restart to read new config file.

Its important to follow all steps and read all information, line by line.
Last edited by Jotne on Mon May 16, 2022 11:16 pm, edited 1 time in total.
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 8:29 pm

Then you have skipped this part in rsyslog setup:

To make Splunk read rsyslog data make this file: %SplunkHome%/etc/system/local/input.conf
[monitor:///data/syslog/udp/.../*.log]
sourcetype = rsyslog
host_segment=4

[monitor:///data/syslog/tcp/.../*.log]
sourcetype = rsyslog
host_segment=4
NB Splunk needs to restart to read new config file.

Its important to follow all steps and read all information, line by line.
splunk@ip-172-31-28-110:~$ cat /opt/splunk/etc/system/local/input.conf
[monitor:///data/syslog/udp/.../*.log]
sourcetype = rsyslog
host_segment=4

[monitor:///data/syslog/tcp/.../*.log]
sourcetype = rsyslog
host_segment=4
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 8:54 pm

In folder (in linux)
/opt/splunk/bin
run
./splunk btool inputs list | grep udp
You should see:
[monitor:///data/syslog/udp/.../*.log]
[udp]
 
User avatar
bholler
Trainer
Trainer
Posts: 95
Joined: Wed Feb 09, 2005 10:22 pm
Location: Nigeria
Contact:

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 9:25 pm

In folder (in linux)
/opt/splunk/bin
run
./splunk btool inputs list | grep udp
You should see:
[monitor:///data/syslog/udp/.../*.log]
[udp]
Only saw [udp]

[monitor:///data/syslog/udp/.../*.log] isnt there.

Oops
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon May 16, 2022 9:39 pm

Then splunk does not see the file.
/opt/splunk/etc/system/local/inputs.conf
It can be a permission settings, if its there.
If I do run Splunk a non-root user, f.eks as a splunk user, I make sure all files under /opt/splunk has same rights.
 
mareka
just joined
Posts: 9
Joined: Mon Sep 23, 2013 9:38 pm

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Jun 06, 2022 4:32 pm

At first, this is wery good job. It take a while to undestand how it works and i missing caps man script etc in this thread but still, good job.
I am using ROS 7 and i have small enhancement for CapsMan to work correctly with Caps man dashboards.
Could you Jotne please update your app by there steps?

In file C:\Program Files\Splunk\etc\apps\MikroTik\default\eventtypes.conf
search
[capsman_disconnected]
search = "caps,info * disconnected,"
[capsman_connected]
search = "caps,info * connected, signal"

change it to
[capsman_disconnected]
search = "caps,info * disconnected," OR "caps,debug * disconnected,"
[capsman_connected]
search = "caps,info * connected, signal" OR "caps,debug * connected, signal"


In file C:\Program Files\Splunk\etc\apps\MikroTik\default\props.conf
search
EXTRACT-mikrotik_caps-man2 = caps,info.*?(?<mac>(?:\w\w:){5}\w\w)@(?<ap>.*?) (?:(?:dis)?connected|rejected), (?<reason>.*?)(?:\(\d+\)| (?<strength>-\d+))?$
EXTRACT-mikrotik_caps-man_frequency = caps,info.*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<widt>\d+)\/(?<standard>[^\(]+)\((?<dBm>\d+)dBm

change it to
EXTRACT-mikrotik_caps-man2 = caps,(info|debug).*?(?<mac>(?:\w\w:){5}\w\w)@(?<ap>.*?) (?:(?:dis)?connected|rejected), (?<reason>.*?)(?:\(\d+\)| (?<strength>-\d+))?$
EXTRACT-mikrotik_caps-man_frequency = caps,(info|debug).*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<widt>\d+)?-?(?<extensionChannel>\w+)\/(?<standard>[^\(]+)\((?<dBm>\d+)dBm

Then restart splunk.
i am pretty sure on linux will be same path structure from splunk install directory so every user can change it itself.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Jun 06, 2022 5:59 pm

All updated. But could you post a log line that this will match on?
EXTRACT-mikrotik_caps-man_frequency = caps,(info|debug).*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<widt>\d+)?-?(?<extensionChannel>\w+)\/(?<standard>[^\(]+)\((?<dBm>\d+)dBm
I do not have/use capsman, so any help is appreciated. :)
 
mareka
just joined
Posts: 9
Joined: Mon Sep 23, 2013 9:38 pm

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Wed Jun 15, 2022 6:33 pm

Hello
the parse modification is not perfect because i am not understand wifi standards deep enought. But it works with probably one mistake in standard detection and that is the place of my misunderstand of wifi stamdards vs Mikrotik.

caps,info MikroTik: AP 2 - 5GHz: selected channel 5680/20-eeCe/ac/DP(27dBm)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Thu Jun 16, 2022 3:50 am

Do you say that the reges is not correct? When I test regex:
caps,(?:info|debug).*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<widt>\d+)?-?(?<extensionChannel>\w+)\/(?<standard>[^\(]+)\((?<dBm>\d+)dBm
On this data
caps,info MikroTik: AP 2 - 5GHz: selected channel 5680/20-eeCe/ac/DP(27dBm)
using regex101.com it looks correct.
 
mareka
just joined
Posts: 9
Joined: Mon Sep 23, 2013 9:38 pm

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Fri Jun 17, 2022 2:53 pm

No, regex which i posted is correct, it parses the line but grouping may be incorrect.
When i use web you posted i will get these groups
ap=AP 2 - 5GHz
frequency=5680
widt=20
extensionChannel=eeCe
standard=ac/DP
dBm=27

i as i can see now i have typo in "widt" which may be width. Sorry about that.
Back to root case. In previos post i spoke about this grou standard=ac/DP because i have no idea if /DP is part of wifi AC standard ot it is it's own category (group)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Mon Jun 20, 2022 1:01 pm

I do agree that it may be wrong. Updated to this:
caps,(?:info|debug).*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<widt>\d+)?-?(?<extensionChannel>\w+)\/(?<standard>[^\/]+)\/\S+\((?<dBm>\d+)dBm
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: &#128204; Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) &#128190; &#128736; &#128187; &#1282

Sat Jun 25, 2022 9:25 pm

NB. If you upgrade Splunk to 9.0, you will get some warnings. I have locked at them, and there are no big error.
It will be fixed in next version of MikroTik app for Splunk :)
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Jul 24, 2022 2:49 pm

Hi,
I've switched over to the RB5009 on 7.3.1 and the logging throws me the error below on execution of your script 4.8
Some sections seem to give output, like address-lists & kid-control.

script error: error - contact MikroTik support and send a supout file (10)

Did you test on 7.3.1 already ?
I'll cut the script into smaller sections and execute section-per-section to see if something comes up.





My 7.3.1 release does not like the section below, it throws the error on the console immediately after execution it.

# Get detailed command history RouterOS >= v7
# ----------------------------------
:if ([:tonum [:pick [/system resource get version] 0 1]] > 6 and $CmdHistory) do={
:global cmd
:local f 0
:foreach i in=[/system history find] do={
:if ($i = $cmd) do={ :set f 1 }
:if ($f != 1) do={
:log info message="StartCMD"
:log info message=[/system history get $i]
:log info message="EndCMD"
}
}
:global cmd [:pick [/system history find] 0]
}
Last edited by jvanhambelgium on Sun Jul 24, 2022 3:10 pm, edited 2 times in total.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5325
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Jul 24, 2022 2:54 pm

Hi,
I've switched over to the RB9005 ...
Where did you find that one ?? :lol:
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Jul 24, 2022 3:11 pm

Hi,
I've switched over to the RB9005 ...
Where did you find that one ?? :lol:
Coming from a secret lab @ Mikrotik somewhere in the very,very,very distant future :lol:
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Jul 24, 2022 6:45 pm

I have tested it on bot 7.2.3 and 7.4 without any problem.

What do you see when you cut and past this to terminal.

{
:local CmdHistory true
# Get detailed command history RouterOS >= v7
# ----------------------------------
:if ([:tonum [:pick [/system resource get version] 0 1]] > 6 and $CmdHistory) do={
:global cmd
:local f 0
:foreach i in=[/system history find] do={
:if ($i = $cmd) do={ :set f 1 }
:if ($f != 1) do={
:log info message="StartCMD"
:log info message=[/system history get $i]
:log info message="EndCMD"
}
}
:global cmd [:pick [/system history find] 0]
}
}
If you after run this part look at Environment in script list, then try change some (add a static DNS host), then run the script. It should increase the global cmd for each time some is done on the router.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Jul 24, 2022 7:42 pm

So initially I executed and these are the env-vars

Hour = 18
cmd = *238B

Then I added Netflow-config + target and re-ran

Hour = 18
cmd = *2391


Now I have to say, it has been some time since that error popped up...the last one about 2 hours ago.
That is the same time I reconfigged the "logging target" settings. Hmm, probably fixed, but thanks the feedback!
Screenshot from 2022-07-24 18-39-42.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Fri Jul 29, 2022 11:52 pm

Just updated my Splunk to 9.0.0.1-9e907cedecb1 and indeed getting the "error" about the possibel insecure query. I'm sure you'll clean it up in a next release.
However, I think I'm seeing some issues with the DHCP-pool section.
With my RB5009, I've modified the config and now I'm using VLAN's internally on my network and therefore created 7 IP-pools representing lab areas.
This does not seem to reflect in the Splunk dashboard, where I would be expecting various 192.168.10.x , 192.168.20.x etc like the Winbox pool-configuration shows correctly.

Screenshot from 2022-07-29 22-43-18.png
Screenshot from 2022-07-29 22-44-43.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Jul 30, 2022 9:30 pm

1. I do see you get Duplicate Values under host. This may be that you did change name on one device ore have multiple host with same name.
Install Lookup Editor in Splunk if you do not already have done. Got to Apps -> Lookup Editor, select device_kvstore and open it. Remove the duplicate line.

2. The warning about security issue is already fixed in next version. Just ignore the message, it just comes from Dashboards updates table.

3. DHCP dashboard are correct. One pool many contain several different subnet, so just pool name are used,
Here is an example:
pool.png
Here is an example from a test router:
dhcp.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Jul 31, 2022 8:46 pm

Hi Jotne,
Cleaned up device_kvstore, thx for the tip.
Can you please explain how you arranged you DHCP-settings on your test-system ?

> Are you using multiple DHCP-server instances (1 for each VLAN ?) This seems mandatory to me to get it working.
> Are you using multiple pools ?? (/ip pool) or 1 pool and did add several IP-ranges ?? -> This does not work, VM's in my VLAN's are assigned a faulty IP unless I create 1 POOL for EACH VLAN subnet

Because in Splunk I get following.
Why it is written like

"DHCP-VLAN20(172.29.45.254)" ?? This VLAN20 = 192.168.20.0/24 and it has a DHCP-server running on 192.168.20.254 (= gateway for that subnet)
Why would it be displaying this 172.29.45.254 which is my "VLAN1" / Bridge-IP ?
Screenshot from 2022-07-31 19-43-34.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Jul 31, 2022 10:03 pm

No I understand what you asks for :)

I do use a simple setup with one pool pr each vlan. What I would say most do.

IP you see after the scope is the host IP. The reason for having it there is that if you have two routers with both have same pool name (default setup), Splunk would mix each scope together and make a big mess.

Have now changed the tag from using host (ip) to (host_name). Will be in next release.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Fri Aug 05, 2022 3:45 pm

Hi,
I just upgraded to the 3.6 version by upgrading the .spl in my running splunt (on docker/synology)
I see "This dashboard version is missing. Update the dashboard version in source" in the dashboard...
Any way to fix this ? I am not a splunk guru so please help

(It seems my splunk was auto updated to 9 ???)

Eddie
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Fri Aug 05, 2022 11:43 pm

Its nothing to worry about. Its just some Splunk added in the new 9.x version. I have updated it for all dashboard the will be released as next version.

To fix it you self:
Click Edit for the Dashboard to fix, then click source and change first line from:
<form theme="dark">
to
<form version="1.1" theme="dark">
Save, and then repeat for all dashboard.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 06, 2022 7:14 am

Its nothing to worry about. Its just some Splunk added in the new 9.x version. I have updated it for all dashboard the will be released as next version.

To fix it you self:
Click Edit for the Dashboard to fix, then click source and change first line from:
<form theme="dark">
to
<form version="1.1" theme="dark">
Save, and then repeat for all dashboard.
Tnx, I can do that...
But, on "mikrotik_device_list"

I get
We've identified a potential security risk
This dashboard is attempting to execute an SPL query that we've flagged as risky.Learn more
The flagged commands are:

    outputlookup
I did not find a way to fix that ..
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 06, 2022 8:26 am

PS
No need to quote the whole post above you. Use Post Reply button below post.

I have fixed that as well. Its just a warning that a Dashboard writes an update back to a KV database or CSV file. Nothing to worry about.
Edit MicroTik Device List, find and remove:
| outputlookup device_table
It was you to make sure the KV database of devices was created as soon as possible so devices gets populated. There is an scheduled task that runes every 24 hours that update it. So for new user, it will take a day before dashboard show host instead of IP.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 06, 2022 8:29 am

tnx,
indeed, after removing that line the error is gone :D
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 3:55 pm

hmm,
something very strange did happen ...
I did upgrade my gateway router from 6.x to 7.4 this morning, all seemed to work fine
splunk does not display firewall rules anymore for this device !!!
It is in de devicelist and updated
DNS requests are displayed
DHCP requests are NOT displayed (no results found)
Firewall info is NOT displayed (no results found)
Device traffic is displayed (using kid control)
Interface traffic is displayed
Live attack is empty (waiting for data) ...
any way to debug this ?
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 4:30 pm

hmm,
something very strange did happen ...
I did upgrade my gateway router from 6.x to 7.4 this morning, all seemed to work fine
splunk does not display firewall rules anymore for this device !!!
It is in de devicelist and updated
DNS requests are displayed
DHCP requests are NOT displayed (no results found)
Firewall info is NOT displayed (no results found)
Device traffic is displayed (using kid control)
Interface traffic is displayed
Live attack is empty (waiting for data) ...
any way to debug this ?
That's a major jump from 6.x to 7.4
I suggest you check each and every requirement to get it working in the first place. Like on the first page of the post to make sure everything is OK.
A bit like a new install.
For example, my latest update from 7.3 to 7.4 mysteriously removed some zerotier-interface etc. I had to recreate it again.
Do you have log-output when the script runs ?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 7:27 pm

@eddieb

What is the output from this search:
index=* sourcetype=mikrotik
| fillnull value=NULL 
| stats count by index module script
| sort -count
It should give a list of all module and script that sends log to splunk from mikrotik
index....	module	script	count
mikrotik	dns	NULL	19983
mikrotik	script	kids	2923
mikrotik	firewall	NULL	2815
mikrotik	dhcp	NULL	2533
mikrotik	script	if_traffic	1198
mikrotik	script	traffic	942
mikrotik	script	address_lists	665
mikrotik	script	neighbor	396
mikrotik	script	upnp	387
mikrotik	script	resource	182
mikrotik	script	monitor	148
mikrotik	script	ntp	145
mikrotik	script	version	145
mikrotik	ipsec	NULL	96
mikrotik	script	health	74
mikrotik	script	wifi	74
mikrotik	script	pool	49
mikrotik	script	sysinfo	49
mikrotik	script	uncounted	37
mikrotik	wireless	NULL	16
mikrotik	upnp	NULL	4
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 9:25 pm

is this what you want to see ?
Screenshot 2022-08-07 at 20.24.18.png
You do not have the required permissions to view the files attached to this post.
 
mfedotov
just joined
Posts: 18
Joined: Mon Oct 25, 2021 3:32 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 11:06 pm

I just installed the latest Splunk (9.0.0.1) and the app 3.6, still playing with it to see, but I noticed that my Mikrotik is sending the firewall logs with some extra information (prio 1->0), which breaks the parser a bit and makes it believe that this whole string from start of destination IP address up to the len is just the destination IP, and does not collect the destination port at all.
The example of the log entry is below (IP/MAC addresses blanked):
firewall,info drop_in input: in:wan out:(unknown 0), src-mac XX:XX:XX:XX:XX:XX, proto TCP (SYN), XX.XXX.XXX.XX:18568->XXX.XXX.XXX.XXX:32400, prio 1->0, len 60
I tried to update the regex I found, but without proper knowledge it was a shot in the dark and did not work for me. Would you mind to update your rule to parse this case?

Thanks
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 11:19 pm

@eddieb

Post looks correct, but it may be that some format has changed.
Can you post output of:
index=* sourcetype=mikrotik module=dhcp OR module=firewall
| dedup module punct
Just post 10-20 events, so I can look at the format.

It may be the same problem as mfedotov sees
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 11:24 pm

@eddieb

Post looks correct, but it may be that some format has changed.
Can you post output of:
index=* sourcetype=mikrotik module=dhcp OR module=firewall
| dedup module punct
Just post 10-20 events, so I can look at the format.

It may be the same problem as mfedotov sees
I'm running 7.3.1 release and do not have any issue / new "prio" field. So perhaps the 7.4 introduced this change. Not present in 7.3.1

firewall,info MikroTik: IP4-IN-DYNAMIC-DROP input: in:ISP out:(unknown 0), proto TCP (SYN), 91.240.118.229:41773->82.241.131.121:3196, len 44
host = 172.30.45.254 index = mainsource = udp:20514 sourcetype = mikrotiksplunk_server = splunky
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 11:40 pm

@mfedotov

Not sure when firewall has added the prio and what it does mean. My test router running 7.2.3 do not logg this type of message.
From your post I do not see the MikroTik tag that needs to be in the log to make sure Splunk gives it correct source type'. See section 2b) in first post.
Understanding Regex is hard, understand other users regex harder, and understanding this regex is very hard :)

Edit
MikroTik/default/props.conf
find
EXTRACT-mikrotik_firewall_info1 = .....

Try to change the regex after the = to:
^firewall,info\s[Mm]ikro[Tt]ik:\s(?:(?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(?:\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(?:\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(?::(?<src_port>[^-]+))?->\[?(?<dest_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))(?:\]?:(?<dest_port>[^,]+))?,\s(?:NAT[^,]+, )?(?:prio[^,]+,\s)?len\s(?<length>\d+) 
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 07, 2022 11:54 pm

@jvanhambelgium
I do se now that you just did try to help and have no problem. Since you do not use Post Reply but quote the whole post above you. I did missunderstand.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 8:13 am

@jotne
removed
Last edited by eddieb on Mon Aug 08, 2022 10:49 am, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 10:16 am

For some reason, do your firewall show connection state. So here is a new and updated regex

Edit
MikroTik/default/props.conf
find
EXTRACT-mikrotik_firewall_info1 = .....

Change the regex after the = to:
^firewall,info\s[Mm]ikro[Tt]ik:\s(?:(?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(?:\sconnection-state:(?<connection_state>\S+))?(?:\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(?:\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(?::(?<src_port>[^-]+))?->\[?(?<dest_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))(?:\]?:(?<dest_port>[^,]+))?,\s(?:NAT[^,]+, )?(?:prio\s(?<prio>[^,]+),\s)?len\s(?<length>\d+)
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 10:48 am

@jotne
tnx, now my firewall entries are visible again.
but stil no DHCP entries visible ...
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 12:22 pm

@jotne
Here is some output from this search

index=* sourcetype=mikrotik module=dhcp

removed
Last edited by eddieb on Mon Aug 08, 2022 2:39 pm, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 2:31 pm

That did not get me what I am looking for.

Post some of this output instead:
index=*
      sourcetype=mikrotik
      module=dhcp
      host="*"
      | rex "Parameter-List = (?<ParameterList>.*)"
      | makemv delim="," ParameterList
      | eval identity=coalesce(identity,host)
      | transaction host  maxspan=1s
| search "received request"
| table eventtype _raw
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 2:39 pm

@jotne
on request ;-)
remove
Last edited by eddieb on Mon Aug 08, 2022 4:49 pm, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 2:59 pm

Everything looks correct.
Last test:
index=*
sourcetype=mikrotik
module=dhcp
host="*"
| rex "Parameter-List = (?<ParameterList>.*)"
| makemv delim="," ParameterList
| eval identity=coalesce(identity,host)
| transaction host startswith="eventtype=dhcp_received_request" endswith="eventtype=dhcp_domain_server" maxspan=1s
| stats  count as Num_Hits values(*) as * latest(_time) as last_time by chaddr
If possible, I can log inn to your Splunk and see directly. (post your email or another form for contact)
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 08, 2022 4:48 pm

@jotne
removed
Last edited by eddieb on Tue Aug 09, 2022 5:26 pm, edited 1 time in total.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 09, 2022 5:05 pm

For some reason, do your firewall show connection state. So here is a new and updated regex
Edit
MikroTik/default/props.conf
find
EXTRACT-mikrotik_firewall_info1 = .....
Change the regex after the = to:
^firewall,info\s[Mm]ikro[Tt]ik:\s(?:(?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(?:\sconnection-state:(?<connection_state>\S+))?(?:\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(?:\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(?::(?<src_port>[^-]+))?->\[?(?<dest_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))(?:\]?:(?<dest_port>[^,]+))?,\s(?:NAT[^,]+, )?(?:prio\s(?<prio>[^,]+),\s)?len\s(?<length>\d+)
@Jotne, I've updated to 7.4.1 and now indeed there is more information in each of the FW-logging lines.
Does you updated REGEX handles this? It seems to only handle "connection-state" ?
Looking at a typical firewall-log rule now :

IP4-FWD-SHELLY-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 98:cd:ac:1f:36:3c, proto TCP (SYN), 172.29.45.25:18233->23.251.142.183:80, len 44

So now I'm getting a connection-mark and connection-state fields. I indeed apply this marker in traffic coming using a mangle-rule
The "connection-mark" is NOT present in each of the logging lines, ALL of them have the "connection-state" value
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 09, 2022 9:03 pm

I need a lot of firewall logs to make sure script does not break. So will have a look at it.

@Mikrotik: Where in the change logs does it mention that the log format has changed?

@jvanhambelgium
Can you give me some example firewall logs? And how do you mark some packet?
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 09, 2022 9:46 pm

I need a lot of firewall logs to make sure script does not break. So will have a look at it.

@Mikrotik: Where in the change logs does it mention that the log format has changed?
In addition, I've checked both mangle-rules.
I do not have specific "logging" enabled on these rules btw.
Screenshot from 2022-08-09 20-37-43.png

I yet have to understand why & when this extra data shows up in the drop-logs, but it seems for outbound packets. (which makes sense since my mangle rule specifies outbound ISP-interface)
These drops are from 2 Unify AP's that would love to call home ;-)

17:12:34 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:38242->34.210.237.89:443, len 60
17:12:35 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:38242->34.210.237.89:443, len 60
17:12:37 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:42706->44.236.10.9:443, len 60
17:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:42706->44.236.10.9:443, len 60
17:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:56604->54.201.115.248:443, len 60
17:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:42932->44.241.83.169:443, len 60
18:12:34 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:43520->44.241.83.169:443, len 60
18:12:35 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:43520->44.241.83.169:443, len 60
18:12:37 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:57196->54.201.115.248:443, len 60
18:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:57196->54.201.115.248:443, len 60
18:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:43302->44.236.10.9:443, len 60
18:12:39 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:38842->34.210.237.89:443, len 60
19:12:34 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:57808->54.201.115.248:443, len 60
19:12:35 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:57808->54.201.115.248:443, len 60
19:12:37 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:44136->44.241.83.169:443, len 60
19:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:44136->44.241.83.169:443, len 60
19:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:43916->44.236.10.9:443, len 60
19:12:39 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:39456->34.210.237.89:443, len 60
20:12:34 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:58424->54.201.115.248:443, len 60
20:12:35 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:58424->54.201.115.248:443, len 60
20:12:37 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:44752->44.241.83.169:443, len 60
20:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:44752->44.241.83.169:443, len 60
20:12:38 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:40070->34.210.237.89:443, len 60
20:12:39 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 80:2a:a8:50:b6:41, proto TCP (SYN), 172.29.45.247:44534->44.236.10.9:443, len 60
20:20:27 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:46828->54.171.230.55:443, len 60
20:20:28 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:46828->54.171.230.55:443, len 60
20:20:30 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:46828->54.171.230.55:443, len 60
20:20:35 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:46828->54.171.230.55:443, len 60
20:20:43 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:46828->54.171.230.55:443, len 60
20:20:59 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:46828->54.171.230.55:443, len 60
20:21:27 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:42310->34.254.182.186:443, len 60
20:21:28 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:42310->34.254.182.186:443, len 60
20:21:30 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:42310->34.254.182.186:443, len 60
20:21:34 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:42310->34.254.182.186:443, len 60
20:21:43 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:42310->34.254.182.186:443, len 60
20:21:59 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:42310->34.254.182.186:443, len 60
20:22:27 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:40986->54.247.62.1:443, len 60
20:22:28 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:40986->54.247.62.1:443, len 60
20:22:30 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:40986->54.247.62.1:443, len 60
20:22:35 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:40986->54.247.62.1:443, len 60
20:22:43 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:40986->54.247.62.1:443, len 60
20:22:59 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:40986->54.247.62.1:443, len 60
20:23:27 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:43302->34.243.160.129:443, len 60
20:23:28 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:43302->34.243.160.129:443, len 60
20:23:30 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:43302->34.243.160.129:443, len 60
20:23:34 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:43302->34.243.160.129:443, len 60
20:23:43 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:43302->34.243.160.129:443, len 60
20:23:59 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:43302->34.243.160.129:443, len 60
20:24:27 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:33074->54.217.10.153:443, len 60
20:24:28 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:33074->54.217.10.153:443, len 60
20:24:30 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:33074->54.217.10.153:443, len 60
20:24:34 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:33074->54.217.10.153:443, len 60
20:24:43 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:33074->54.217.10.153:443, len 60
20:24:59 firewall,info IP4-FWD-UNIFI-DROP forward: in:Bridge-LAN out:ISP, connection-mark:ALL_UP_CONN connection-state:new src-mac 02:11:32:25:51:92, proto TCP (SYN), 172.29.45.239:33074->54.217.10.153:443, len 60
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 09, 2022 9:59 pm

This should do:
^firewall,info\s[Mm]ikro[Tt]ik:\s(?:(?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(?:\sconnection-mark:(?<connection_mark>\S+))?(?:\sconnection-state:(?<connection_state>\S+))?(?:\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(?:\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(?::(?<src_port>[^-]+))?->\[?(?<dest_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))(?:\]?:(?<dest_port>[^,]+))?,\s(?:NAT[^,]+, )?(?:prio\s(?<prio>[^,]+),\s)?len\s(?<length>\d+)
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 09, 2022 10:58 pm

I updated this one too, works fine on 7.4 logging
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 09, 2022 11:22 pm

I confirm the new regex does its magic on 7.4.1 too.
Thanks!
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 09, 2022 11:41 pm

You are welcome. It will be added to upcoming 3.7. You can just edit the firewall dashboard and cut/past and save with this, to get the new firewall fields shown:
<form version="3.7" theme="dark">
  <label>MikroTik Firewall Rules</label>
  <description>Data collected from Syslog</description>
  <!--Version
  2.4.1 Fixed error with src_port in dest_ip dropdown
  2.5.1 Changed to "Coalesce"
  2.6.1 Sourt by count
  2.7.1 Cleaned searh
  2.7.2 useother=false to remove base level
  2.7.3 Added counters
  2.8.1 Added "output" color
  2.9.0 Added identity
  3.0.0 Fixed missing identity
  3.1.0 Added Free text search
  3.2.0 Added source port
  3.3.0 Fixed missing City
  3.3.1 Change from table to fields
  3.4.0 Fixed missing color
  3.7.0 Added dashboard version
  3.7.1 Added connection state/mark
  ######################################################
  #
  # Mikrotik Add-On for Splunk
  #
  # Copyright (C) 2022 Jotne
  # All Rights Reserved
  #
  ######################################################
  -->
  <search id="base_search">
    <query>
      `index`
      sourcetype=mikrotik
      module=firewall
      $Free$
      host="$host$"
      rule="$rule$"
      chain="$chain$"
      in_if="$in_if$"
      out_if="$out_if$"
      src_mac="$src_mac$"
      protocol="$protocol$"
      src_ip="$src_ip$"
      src_port="$src_port$"
      dest_ip="$dest_ip$"
      dest_port="$dest_port$"
      | fillnull value="-"
      | search 
        connection_state="$connection_state$"
        connection_mark="$connection_mark$"
      | eval host_name=coalesce(identity,host)
      | fields _time host rule chain in_if out_if src_mac protocol src_ip dest_ip src_port dest_port host_name connection_state connection_mark
    </query>
  </search>
  <search base="base_search">
    <query>
      | stats dc(dest_port) as count1 dc(src_ip) as count2 count as count3
    </query>
    <finalized>
      <set token="DestPort_count">$result.count1$</set>
      <set token="SrcIP_count">$result.count2$</set>
      <set token="Total_count">$result.count3$</set>
    </finalized>
  </search>
  <fieldset submitButton="false">
    <input type="time">
      <label>Time Range</label>
      <default>
        <earliest>-15m</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="dropdown" token="host">
      <label>Host</label>
      <search base="base_search">
        <query>
          | eval data=host
          | stats count by data host_name
          | eval info=host_name."-".data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="rule">
      <label>Rule</label>
      <search base="base_search">
        <query>
          | eval data=rule
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="chain">
      <label>Chain</label>
      <search base="base_search">
        <query>
          | eval data=chain
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="in_if">
      <label>Input Interface</label>
      <search base="base_search">
        <query>
          | eval data=in_if
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
          </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="out_if">
      <label>Output Interface</label>
      <search base="base_search">
        <query>
          | eval data=out_if
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
           </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="src_mac">
      <label>Source MAC</label>
      <search base="base_search">
        <query>
          | eval data=src_mac
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="protocol">
      <label>Protocol</label>
      <search base="base_search">
        <query>
          | eval data=protocol
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="src_ip">
      <label>Source IP</label>
      <search base="base_search">
        <query>
          | eval data=src_ip
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="src_port">
      <label>Source port</label>
      <search base="base_search">
        <query>
          | eval data=src_port
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="dest_ip">
      <label>Destination IP</label>
      <search base="base_search">
        <query>
          | eval data=dest_ip
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="dest_port">
      <label>Destination port</label>
      <search base="base_search">
        <query>
          | eval data=dest_port
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="connection_state">
      <label>Connection State</label>
      <search base="base_search">
        <query>
          | eval data=connection_state
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="connection_mark">
      <label>Connection Mark</label>
      <search base="base_search">
        <query>
          | eval data=connection_mark
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="text" token="Free">
      <label>Free Text</label>
      <default>*</default>
    </input>
  </fieldset>
  <row>
    <panel>
      <chart>
        <title>src_ip #$SrcIP_count$</title>
        <search base="base_search">
          <query>
            | timechart count by src_ip limit=20 useother=false
          </query>
        </search>
        <option name="charting.axisTitleX.visibility">collapsed</option>
        <option name="charting.chart">column</option>
        <option name="charting.chart.stackMode">stacked</option>
        <option name="charting.drilldown">none</option>
      </chart>
    </panel>
    <panel>
      <chart>
        <title>dest_port #$DestPort_count$</title>
        <search base="base_search">
          <query>
            | timechart count by dest_port limit=20 useother=false
          </query>
        </search>
        <option name="charting.axisTitleX.visibility">collapsed</option>
        <option name="charting.chart">column</option>
        <option name="charting.chart.stackMode">stacked</option>
        <option name="charting.drilldown">none</option>
      </chart>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Number of entries #$Total_count$</title>
        <search base="base_search">
          <query>
            | iplocation src_ip
            | eval City=if(City="","Unknown",City)
            | rex field=rule "^(?&lt;type&gt;.)(?&lt;dir&gt;.)_(?&lt;act&gt;.+?)_"
            | table _time type dir act rule chain in_if out_if src_mac protocol src_ip src_port dest_ip dest_port connection_state connection_mark City Country
            | sort -_time</query>
        </search>
        <option name="count">50</option>
        <option name="wrap">false</option>
        <format type="color" field="chain">
          <colorPalette type="map">{"input":#F8BE34,"dstnat":#6DB7C6,"forward":#53A051,"output":#006D9C,"srcnat":#AF575A,"prerouting":#F1813F}</colorPalette>
        </format>
        <format type="color" field="protocol">
          <colorPalette type="map">{"47":#006D9C,"TCP":#65A637,"UDP":#D93F3C,"ICMP":#F8BE34}</colorPalette>
        </format>
        <format type="color" field="rule">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="in_if">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="out_if">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="src_ip">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="src_port">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="dest_ip">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="dest_port">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="City">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="Country">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="src_mac">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="type">
          <colorPalette type="map">{"F":#53A051,"R":#006D9C,"N":#F8BE34,"M":#AF575A}</colorPalette>
        </format>
        <format type="color" field="act">
          <colorPalette type="map">{"D":#DC4E41,"RE":#DC4E41,"T":#DC4E41,"DE":#62B3B2,"M":#006D9C,"J":#F8BE34,"AS":#5A4575,"A":#53A051,"L":#FFFFFF}</colorPalette>
        </format>
        <format type="color" field="dir">
          <colorPalette type="map">{"I":#F8BE34,"F":#53A051,"O":#B6C75A,"D":#53A051,"X":#006D9C,"S":#AF575A,"P":#F1813F}</colorPalette>
        </format>
        <format type="color" field="connection_state">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="connection_mark">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
      </table>
    </panel>
  </row>
</form>
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 22, 2022 7:39 pm

@Jotne: thank you very much for the good work.

In my installation everything works as described except the MirkoTik Wifi connection dashboard. I have 2 MikroTik systems with wireless interfaces (no CAPSMan). Further I activated ":local Wireless true" in the collection script. The dashboard Wifi Signal Strength is correctly filled but not the connection dashboard. Do you have an idea? Is the part of the script (registration-table etc.) just for the Signal Strength dashboard. Is the other dashboard filled based on syslog information (the association and connection attempts exists in my splunk installation)?

Regarding Splunk: do you have a good idea to handle authentication and event-messaging (are there good alternatives except buying a commercial license?)

Thanks in advance!
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 22, 2022 8:39 pm

Do you see any line with run this search:
index=* connected signal strength
You should get some like this:
wireless,info MikroTik: 74:aa:09:53:A7:E9@wlan1: connected, signal strength -67
wireless,info MikroTik: 74:aa:09:53:A7:E9@wlan1: connected, signal strength -77
wireless,info MikroTik: 6C:aa:D6:C2:4E:03@wlan2: connected, signal strength -82
wireless,info MikroTik: 62:aa:61:F1:2E:9B@wlan1: connected, signal strength -76
Regarding Splunk: do you have a good idea to handle authentication and event-messaging (are there good alternatives except buying a commercial license?)
Do you mean how to add user/password login to Splunk, when you only have free license?
I solved this by using basic authentication function in HAProxy (a free reverse proxy)
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 22, 2022 9:37 pm

Thanks.

Yes, I can see results based on this search. I also see the results in the "mikrotik_wifi_strength" dashboard, but I can not see data in the mikrotik_wifi_connection dashboard.

Example:
2022-08-22T18:07:11.648980+02:00 <13>wireless,info MikroTik: AA:BB:DE:AD:BE:EF@wlan4: connected, signal strength -79
The filter looks for:
eventtype=wifi_connected OR eventtype=wifi_disconnected_*
Strange.

Ok, I will also add a reverse proxy with a basic-auth - better than nothing.

Thanks for your support.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 22, 2022 11:17 pm

This is some that is fixed in upcoming 3.7
# Fixed splunk read from rsyslog data
Replace props.conf with this:
[source::udp:514]
TRUNCATE = 10000
TRANSFORMS-dns = remove_dns_query,remove_dns_answer
TRANSFORMS-force_mikrotik = force_mikrotik_st,force_mikrotik_ix

[syslog]
TRUNCATE = 10000
TRANSFORMS-dns = remove_dns_query,remove_dns_answer
TRANSFORMS-force_mikrotik = force_mikrotik_st,force_mikrotik_ix

[rsyslog]
TRUNCATE = 10000
TRANSFORMS-dns = remove_dns_query,remove_dns_answer
TRANSFORMS-force_mikrotik = force_mikrotik_st,force_mikrotik_ix
SEDCMD-clean_header = s/\d{4}-\d\d-\d\d.*?<\d+>//
SEDCMD-clean_end = s/#015$//

[mikrotik]
MAX_TIMESTAMP_LOOKAHEAD = 23
SHOULD_LINEMERGE = false
DATETIME_CONFIG = CURRENT


EXTRACT-mikrotik_bridge = bridge,stp (?:\w+:\s)?(?<int>\S+)\s(?<status>\w+)

EXTRACT-mikrotik_caps-man1 = script=caps-man name=(?<ap>.*?) counter=(?<counter>\d+)
EXTRACT-mikrotik_caps-man2 = caps,(?:info|debug).*?(?<mac>(?:\w\w:){5}\w\w)@(?<ap>.*?) (?:(?:dis)?connected|rejected), (?<reason>.*?)(?:\(\d+\)| (?<strength>-\d+))?$
EXTRACT-mikrotik_caps-man_frequency = caps,(?:info|debug).*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<widt>\d+)?-?(?<extensionChannel>\w+)\/(?<standard>[^\/]+)\/\S+\((?<dBm>\d+)dBm

EXTRACT-mikrotik_dhcp_vlan = dhcp,debug.*: (?<dhcp_server>.*?) received request

EXTRACT-mikrotik_dns_src = dns,packet.*from\s(?<src_ip>[^:]+):(?<src_port>\d+)
EXTRACT-mikrotik_dns_site = dns,packet.*question:\s(?<site1>[^:]+):(?<site_type>[^:]+):(?<site_direction>[^:]+)
EXTRACT-mikrotik_dns_record_1 = dns.*query from\s(?<src_ip>[^:]+):\s#(?<ses_id>\d+)\s(?<site>\S+)\.\s(?<record_type>\S+)
EXTRACT-mikrotik_dns_record_2 = dns,packet.*\s<(?<site2>[^:]+):(?<record_type>[^:]+):(?<record_id>[^=]+)=(?<record_info>[^>]+)
EXTRACT-mikrotik_dns_record_3 = dns.* done query:\s#(?<ses_id>\d+)\s(?<site>\S+)\s(?<dest_ip>\S+)

# EXTRACT-mikrotik_firewall_info1 = firewall,info\s[Mm]ikro[Tt]ik:\s(?:(?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(?:\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(?:\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(?::(?<src_port>[^-]+))?->\[?(?<dest_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))(?:\]?:(?<dest_port>[^,]+))?,\s(?:NAT.*)?len\s(?<length>\d+)
EXTRACT-mikrotik_firewall_info1 = ^firewall,info\s[Mm]ikro[Tt]ik:\s(?:(?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(?:\sconnection-mark:(?<connection_mark>\S+))?(?:\sconnection-state:(?<connection_state>\S+))?(?:\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(?:\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(?::(?<src_port>[^-]+))?->\[?(?<dest_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))(?:\]?:(?<dest_port>[^,]+))?,\s(?:NAT[^,]+, )?(?:prio\s(?<prio>[^,]+),\s)?len\s(?<length>\d+)


EXTRACT-mikrotik_hotspot_user_scrip_mac = hotspot[^:]+:\s(?<user>\S+)\s\(?:(?:(?<src_ip>\d+\.\d+\.\d+\.\d+)|(?<mac>(?:..:){5}..))\)
EXTRACT-mikrotik_hotspot_user_login_type = hotspot.*trying to log in by (?<log_in_by>\S+)

EXTRACT-mikrotik_interface = interface,info.*? (?<int>\S+) link (?<change>\S+)(?: \(speed (?<speed>[^,]+), (?<duplex>\S+) duplex)?

EXTRACT-mikrotik_ip_accounting = ^\d+\s+(?<src_ip>\d+\.\d+\.\d+\.\d+)\s+(?<dest_ip>\d+\.\d+\.\d+\.\d+)\s+(?<packet>\d+)\s+(?<byte>\d+)(?:\s+(?<src_user>\S+)\s+(?<dest_user>\S+))?

EXTRACT-mikrotik_ipsec_ip_1 = ipsec,error.*? (?<src_ip>\d+\.\d+\.\d+\.\d+) (?:failed to|parsing|phase1)
EXTRACT-mikrotik_ipsec_ip_2 = Invalid exchange type \d+ from\s(?<src_ip>[^[]+)
EXTRACT-mikrotik_ipsec_ip_3 = ipsec.*? not registered for (?<src_ip>[^[]+)
EXTRACT-mikrotik_ipsec_ip_4 = ipsec,error.*? phase1 negotiation failed.*?(?<dest_ip>\d+\.\d+\.\d+\.\d+).*?>(?<src_ip>\d+\.\d+\.\d+\.\d+)

EXTRACT-mikrotik_l2tp_src_ip = first L2TP UDP packet received from\s(?<src_ip>\S+)
EXTRACT-mikrotik_l2tp_user1 = l2tp,ppp,info.*? (?<user>\S+) logged
EXTRACT-mikrotik_l2tp_user2 = l2tp,ppp,info.*? <l2tp-(?<user>[^>]+)>:\s

EXTRACT-mikrotik_modul_status = ^(?:.*?<\d+>)?(?<module>[^\s,]+),?(?<info1>.*?)(?:,(?<severity>(?:debug|info|warning|error|critical)))?(?:,(?<info2>(?:packet|account|state|calc|event)))?\s

EXTRACT-mikrotik_pppoe = pppoe.*?(?<user>\S+) logged (?<logged>[^,]+), (?<src_ip>\d+\.\d+\.\d+\.\d+)?(?:(?<time>\d+) (?<rcv_bytes>\d+) (?<sent_bytes>\d+) (?<rcv_packets>\d+) (?<sent_packet>\d+))?

EXTRACT-mikrotik_pptp_src_ip = TCP connection established from\s(?<src_ip>\S+)

EXTRACT-mikrotik_session = (?<user>\S+)\slogged out, (?<seconds>\d+) (?<bytes_in>\d+) (?<bytes_out>\d+) (?<packets_in>\d+) (?<packets_out>\d+)

EXTRACT-mikrotik_ssh_user = ssh,debug .*(?:user|auth req):\s(?<user>\S+)\s

EXTRACT-mikrotik_system_action = system,info .*:\s(?<change>.*)\s(?<action>[\S]+)\sby\s(?<user>\S+)
EXTRACT-mikrotik_system_info = system,info,account .*user\s(?<user>\S+)\slogged in from\s(?<user_ip>\S+)\svia\s(?<system>\S+)

EXTRACT-mikrotik_upnp_service = script,info .*nextid.*: (?<service>[^;]+);

EXTRACT-mikrotik_user_logged_in_1 = (?<user>\S+)\slogged (?:in(?:,| )|out (?:from)?)(?: (?<user_ip>\S+))?(?:\s?via\s(?<system>\S+))?
EXTRACT-mikrotik_user_logged_in_2 = login failure for user\s(?<user>\S+)\sfrom\s(?<user_ip>\S+)(?:\svia\s(?<system>\S+))?
EXTRACT-mikrotik_user_logged_in_3 = user\s(?<user>\S+)\sauthentication

EXTRACT-mikrotik_web-proxy = web-proxy,account\s\S+\s(?<src_ip>\d+\.\d+\.\d+\.\d+)\s(?<status>\w+)(?:\s(?<id>\d+))?\s(?<url>\S+)

EXTRACT-mikrotik_wireless_1 = wireless,info (?:\w+:\s)?(?<mac>\w\w:\w\w:\w\w:\w\w:\w\w:\w\w)@(?<int>[^:]+):\s(?<status>\w+)(?:, signal strength\s(?<strength>\S+))?
EXTRACT-mikrotik_wireless_strenght = signal-strength=(?<signal_strength>-\d+).*tx-rate=(?<tx_rate>[\d.]+)

EXTRACT-mikrotik_wireguard_error = wireguard,debug\s\S+\s(?<interface>\S+)\s(?<public_key>\S+)\s(?<error>.*?)\s\(


EVAL-channel = (frequency-if(match(standard,"a"),5000,2407))/5
EVAL-src_port = if(isnull(src_port) AND module=="firewall","na",src_port)
EVAL-src_mac = if(isnull(src_mac) AND module=="firewall","na",src_mac)
EVAL-dest_port = if(isnull(dest_port) AND module=="firewall","na",dest_port)
EVAL-rule = case(isnull(rule) AND module=="firewall","missing_or_too_long_Prefix_max_20_characters",len(rule)>20 AND module=="firewall","too_long_Prefix_max_20_characters",1=1,rule)

LOOKUP-automatic_device_table = device_table host

LOOKUP-automatic_dhcp_table = dhcp_table mac OUTPUTNEW
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 23, 2022 12:05 am

Great @Jotne - it works ;-)

Thanks. Hope I can contribute something good in the future.
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 23, 2022 11:16 pm

Since I renewed the props.conf I can't see the firewall logs anymore. They were there before the change - do you have an idea?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Wed Aug 24, 2022 8:47 am

I guess you have upgrade the router as well. There are new connection state/mark fields in 7.4+ logs. When I am done testing som Routing protocol BGP/OSPF, I will release next version. You can try this dashboard and see if it works:
<form version="3.7" theme="dark">
  <label>MikroTik Firewall Rules</label>
  <description>Data collected from Syslog</description>
  <!--Version
  2.4.1 Fixed error with src_port in dest_ip dropdown
  2.5.1 Changed to "Coalesce"
  2.6.1 Sourt by count
  2.7.1 Cleaned searh
  2.7.2 useother=false to remove base level
  2.7.3 Added counters
  2.8.1 Added "output" color
  2.9.0 Added identity
  3.0.0 Fixed missing identity
  3.1.0 Added Free text search
  3.2.0 Added source port
  3.3.0 Fixed missing City
  3.3.1 Change from table to fields
  3.4.0 Fixed missing color
  3.7.0 Added dashboard version
  3.7.1 Added connection state/mark
  ######################################################
  #
  # Mikrotik Add-On for Splunk
  #
  # Copyright (C) 2022 Jotne
  # All Rights Reserved
  #
  ######################################################
  -->
  <search id="base_search">
    <query>
      `index`
      sourcetype=mikrotik
      module=firewall
      $Free$
      host="$host$"
      rule="$rule$"
      chain="$chain$"
      in_if="$in_if$"
      out_if="$out_if$"
      src_mac="$src_mac$"
      protocol="$protocol$"
      src_ip="$src_ip$"
      src_port="$src_port$"
      dest_ip="$dest_ip$"
      dest_port="$dest_port$"
      | fillnull value="-"
      | search 
        connection_state="$connection_state$"
        connection_mark="$connection_mark$"
      | eval host_name=coalesce(identity,host)
      | fields _time host rule chain in_if out_if src_mac protocol src_ip dest_ip src_port dest_port host_name connection_state connection_mark
    </query>
  </search>
  <search base="base_search">
    <query>
      | stats dc(dest_port) as count1 dc(src_ip) as count2 count as count3
    </query>
    <finalized>
      <set token="DestPort_count">$result.count1$</set>
      <set token="SrcIP_count">$result.count2$</set>
      <set token="Total_count">$result.count3$</set>
    </finalized>
  </search>
  <fieldset submitButton="false">
    <input type="time">
      <label>Time Range</label>
      <default>
        <earliest>-15m</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="dropdown" token="host">
      <label>Host</label>
      <search base="base_search">
        <query>
          | eval data=host
          | stats count by data host_name
          | eval info=host_name."-".data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="rule">
      <label>Rule</label>
      <search base="base_search">
        <query>
          | eval data=rule
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="chain">
      <label>Chain</label>
      <search base="base_search">
        <query>
          | eval data=chain
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="in_if">
      <label>Input Interface</label>
      <search base="base_search">
        <query>
          | eval data=in_if
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
          </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="out_if">
      <label>Output Interface</label>
      <search base="base_search">
        <query>
          | eval data=out_if
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
           </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="src_mac">
      <label>Source MAC</label>
      <search base="base_search">
        <query>
          | eval data=src_mac
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="protocol">
      <label>Protocol</label>
      <search base="base_search">
        <query>
          | eval data=protocol
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="src_ip">
      <label>Source IP</label>
      <search base="base_search">
        <query>
          | eval data=src_ip
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="src_port">
      <label>Source port</label>
      <search base="base_search">
        <query>
          | eval data=src_port
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="dest_ip">
      <label>Destination IP</label>
      <search base="base_search">
        <query>
          | eval data=dest_ip
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="dest_port">
      <label>Destination port</label>
      <search base="base_search">
        <query>
          | eval data=dest_port
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="connection_state">
      <label>Connection State</label>
      <search base="base_search">
        <query>
          | eval data=connection_state
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="dropdown" token="connection_mark">
      <label>Connection Mark</label>
      <search base="base_search">
        <query>
          | eval data=connection_mark
          | stats count by data
          | eval info=data." (".count.")"
          | sort -count
        </query>
      </search>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>info</fieldForLabel>
      <fieldForValue>data</fieldForValue>
    </input>
    <input type="text" token="Free">
      <label>Free Text</label>
      <default>*</default>
    </input>
  </fieldset>
  <row>
    <panel>
      <chart>
        <title>src_ip #$SrcIP_count$</title>
        <search base="base_search">
          <query>
            | timechart count by src_ip limit=20 useother=false
          </query>
        </search>
        <option name="charting.axisTitleX.visibility">collapsed</option>
        <option name="charting.chart">column</option>
        <option name="charting.chart.stackMode">stacked</option>
        <option name="charting.drilldown">none</option>
      </chart>
    </panel>
    <panel>
      <chart>
        <title>dest_port #$DestPort_count$</title>
        <search base="base_search">
          <query>
            | timechart count by dest_port limit=20 useother=false
          </query>
        </search>
        <option name="charting.axisTitleX.visibility">collapsed</option>
        <option name="charting.chart">column</option>
        <option name="charting.chart.stackMode">stacked</option>
        <option name="charting.drilldown">none</option>
      </chart>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Number of entries #$Total_count$</title>
        <search base="base_search">
          <query>
            | iplocation src_ip
            | eval City=if(City="","Unknown",City)
            | rex field=rule "^(?&lt;type&gt;.)(?&lt;dir&gt;.)_(?&lt;act&gt;.+?)_"
            | table _time type dir act rule chain in_if out_if src_mac protocol src_ip src_port dest_ip dest_port connection_state connection_mark City Country
            | sort -_time</query>
        </search>
        <option name="count">50</option>
        <option name="wrap">false</option>
        <format type="color" field="chain">
          <colorPalette type="map">{"input":#F8BE34,"dstnat":#6DB7C6,"forward":#53A051,"output":#006D9C,"srcnat":#AF575A,"prerouting":#F1813F}</colorPalette>
        </format>
        <format type="color" field="protocol">
          <colorPalette type="map">{"47":#006D9C,"TCP":#65A637,"UDP":#D93F3C,"ICMP":#F8BE34}</colorPalette>
        </format>
        <format type="color" field="rule">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="in_if">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="out_if">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="src_ip">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="src_port">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="dest_ip">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="dest_port">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="City">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="Country">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="src_mac">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="type">
          <colorPalette type="map">{"F":#53A051,"R":#006D9C,"N":#F8BE34,"M":#AF575A}</colorPalette>
        </format>
        <format type="color" field="act">
          <colorPalette type="map">{"D":#DC4E41,"RE":#DC4E41,"T":#DC4E41,"DE":#62B3B2,"M":#006D9C,"J":#F8BE34,"AS":#5A4575,"A":#53A051,"L":#FFFFFF}</colorPalette>
        </format>
        <format type="color" field="dir">
          <colorPalette type="map">{"I":#F8BE34,"F":#53A051,"O":#B6C75A,"D":#53A051,"X":#006D9C,"S":#AF575A,"P":#F1813F}</colorPalette>
        </format>
        <format type="color" field="connection_state">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
        <format type="color" field="connection_mark">
          <colorPalette type="sharedList"></colorPalette>
          <scale type="sharedCategory"></scale>
        </format>
      </table>
    </panel>
  </row>
</form>
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Wed Aug 24, 2022 9:15 am

@jotne
I see some strange things happen ...
1 of my devices stops displaying in splunk after approx 7 days of working fine.
Only a reboot of the device and a restart of splunk (docker on synology) makes everything work again.
6 other devices don't show this behaviour. All devices run ROS 7.4 or 7.4.1. All devices run the 3.6 scripts
any suggestions ?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Wed Aug 24, 2022 10:06 am

I have seen sometimes syslog stops working. Not sure if its syslog server not receiving or device not able to send.
What syslog server do you use? Splunk/514? rsyslog? other?
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Wed Aug 24, 2022 10:41 am

splunk is listening on udp/3514 (514 is in use by syno itself, rsyslog is running on 2514)

on the syno :
# netstat -aunp | grep 514
udp        0      0 0.0.0.0:2514            0.0.0.0:*                           31440/syslog-ng     
udp        0      0 0.0.0.0:3514            0.0.0.0:*                           21088/splunkd       
6 devices are still working, the 7th (my gateway) stops beining visible in splunk.
the gateway still logs to rsyslog (udp/2514)
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Thu Aug 25, 2022 6:08 pm

You can try this dashboard and see if it works
Thanks - unfortunately it does not work.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 27, 2022 9:12 am

I will soon release 3.7 You can try that and see if it fixes you problem.

Here is a view of the routing dashboard. It will show all type of routes that are used. Separate dashboards for OSPF and BGP will be included.
.
Route List.png
You do not have the required permissions to view the files attached to this post.
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 27, 2022 10:01 am

The new routing dashboard looks very good.

I will wait (I am still not on routeros 7.4.2) - I hope I will find some time at this weekend, maybe I can figure something out.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 27, 2022 11:39 am

Min script in first post updated to 4.9 (needed to work with 3.7 MikroTik app for Splunk)

# 4.9 Added part to collect routing information

Script can be updated regardless of if you update main app or not.
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 27, 2022 2:29 pm

EXTRACT-mikrotik_firewall_info1 = ^firewall,info\s[Mm]ikro[Tt]ik:\s(?:(?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(?:\sconnection-mark:(?<connection_mark>\S+))?(?:\sconnection-state:(?<connection_state>\S+))?(?:\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(?:\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(?::(?<src_port>[^-]+))?->\[?(?<dest_ip>(?:\d+\.\d+\.\d+\.\d+|[^\]]+))(?:\]?:(?<dest_port>[^,]+))?,\s(?:NAT[^,]+, )?(?:prio\s(?<prio>[^,]+),\s)?len\s(?<length>\d+)
In the current props.conf the EXTRACT-mikrotik_firewall_info1 section changed. Is there a way, how I can manually verify the filter? I can not get any data based on it.

If I add the old filter to the new props.conf it also does not work anymore.
EXTRACT-mikrotik_firewall_info1 = firewall,info\sMikroTik:\s((?<rule>[^:]+)\s)?(?<chain>.+?):\sin:(?<in_if>.*?)\sout:(?<out_if>[^,]+),(\ssrc-mac\s(?<src_mac>[^,]+),)?\sproto\s(?<protocol>\S+)(\s\((?<flag>[^\)]*)\))?,\s\[?(?<src_ip>(\d+\.\d+\.\d+\.\d+|[^\]]+))\]?(:(?<src_port>[^-]+))?->\[?(?<dest_ip>(\d+\.\d+\.\d+\.\d+|[^\]]+))(\]?:(?<dest_port>[^,]+))?,\s(NAT.*)?len\s(?<length>\d+)
This is in the logs:
2022-08-27T13:25:40.391228+02:00 <13>firewall,info MikroTik: FI_D_port-test input: in:pppoe-out1 out:(unknown 0), proto TCP (SYN), XX.YY.165.97:52819->C.A.B.172:34975, len 44
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 27, 2022 10:56 pm

Just remove the ^ in front of the regex and it will work. I see you have the date info still in the log line.
I will make sure 3.7 works with and without the added front info since it may vary some from one syslog server to another.

To see how it works:
https://regex101.com/r/1Kzr3b/1

And here to see the complexity:
https://www.debuggex.com/r/cFElMmqQa5sYVgYW
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 27, 2022 11:34 pm

Version 3.7 is now released.

Mayor changes in this release are the new routing dashboards. Since I do not have a high lever knowledge of routing there may be various bugs or stuff that are missing. Test and creation of dashboard was done using EVE-NG. So if you have OSPF or BGP routing please test this out give me feedback. i you are using other routing protocol that should be included, shout out and we can look inn to it. For personal question or stuff that you do not like to post public use this email degire4559@xitudy.com

New file are found under section 1g
Download, extract and replace old app (if you have not med personal changes)

PS Main script needs to be updated to 4.9 to get routing information.

What has changes since 3.6

# 3.7 (27.08.2022)
# Fixed Splunk read from rsyslog data
# Added view "MikroTik WireGuard Error"
# Updated some Capsman search
# Added version info in header to conform with Splunk v9.0
# Fixed transaction due to rsyslog in "MikroTik Neighbor"
# Added DNS for interface for "MikroTik Neighbor"
# Removed outputlookup from "MikroTik Device List"
# Fixed metadata config
# Removed clients with 0 bytes in "MikroTik Device Traffic"
# Added script version in "MikroTik Device List"
# Removed x/y axes in "MikroTik VPN Connections"
# Moved version info to VERSION.txt
# Change App name from "MikroTik 3.7" to "MikroTik App for Splunk"
# Added ?: to regex to speed up extraction
# Updated Firewall regex to handle prio and connection-state fields
# Script updated to #4.9 to collect routing information
# Added new dashboard "MikroTik Route List"
# Added new dashboard "MikroTik OSPF Neighbors"
# Added new dashboard "MikroTik BPG Neighbors"

Here are Routing, OSPF and BGP dashboards.
.
Route List.png
.
OSPF.png
.
BGP.png
You do not have the required permissions to view the files attached to this post.
Last edited by Jotne on Sun Aug 28, 2022 10:49 am, edited 1 time in total.
 
dmq
newbie
Posts: 26
Joined: Mon Feb 07, 2022 10:46 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 28, 2022 10:01 am

Thanks @Jotne. Looks very good.

Firewall rules logs are working again (thanks for the hint to the regex site, I already knew it but I have not thought enough).

Routing dashboard is cool.

Have a nice day.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sun Aug 28, 2022 11:07 am

thanks @jotne
I just updated the .spl and the script on my main gateway. Look good
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 29, 2022 12:03 am

Too bad the "Kid Control" is actually not accurate on my 7.4.1 box.
Not too sure what is happening, but the use-case is simple. On my ShieldTV I've just watched a 16Gbytes movie. This is served from my NAS.

So it only recorded 107 megabytes of data...
The 4K movie runs at an average of 24megabit/sec .... so .....we're not going to cut it with 107MB ;-)

We'll I'll do some reference-testing with iperf between hosts and see if counters represent reality...then I'll open a case for that.

Even with this fine set of dashboards ; crap in = crap out ;-)
Screenshot from 2022-08-28 22-54-31.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 29, 2022 8:07 am

I have not testet on 7.4.1 but on 7.2.3 and other version it works fine.
Here I did download a 1GB file from here:
https://speed.hetzner.de/
and the result looks great.
.
1GB.png
It may be fail in config or maybe your device?
Not seen other complains about this.
Could it be some like fasttrack enabled?
You do not have the required permissions to view the files attached to this post.
 
ferdytao
newbie
Posts: 29
Joined: Mon Sep 26, 2016 8:51 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 29, 2022 5:48 pm

Where I can find the capsman script?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 29, 2022 8:04 pm

Section
2f) Main Collector Script
in first post.
 
ferdytao
newbie
Posts: 29
Joined: Mon Sep 26, 2016 8:51 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 30, 2022 12:38 am

I mean this part, it invoke a "capsman" script
# Test if CAPsMANN is installed, if yes, run capsmann script.
# ----------------------------------
:if ( ([:len [/interface find where type="cap"]] > 0) and $CAPsMANN) do={ /system script run capsman }
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Tue Aug 30, 2022 8:06 am

My fault. It was missing (again)
Added to section 2g.

If I have had a CAPSman setup, I maybe could have added this part to the main script.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 03, 2022 2:06 pm

Don't know what is wrong, after a couple of days that all works as expected, devices stop being visible in spluk ...
Only restarting splunk seems to help.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 03, 2022 2:22 pm

I have not testet on 7.4.1 but on 7.2.3 and other version it works fine.
Here I did download a 1GB file from here:
https://speed.hetzner.de/
and the result looks great.
.
1GB.png

It may be fail in config or maybe your device?
Not seen other complains about this.
Could it be some like fasttrack enabled?
I've ran a couple of iperf3 testing and indeed the accounting/KidControl-data seems to match! So I'm not sure what happened here to draw my false conclusion ;-)
Carry on ;-)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 03, 2022 5:43 pm

Don't know what is wrong, after a couple of days that all works as expected, devices stop being visible in spluk ...
I do not say its a docker problem, but I always runs Splunk on clean Ubuntu (18.04 or newer) and its very stable.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 03, 2022 5:44 pm

@Jotne,
Can you include in your next release the version-tag ?
Screenshot from 2022-09-03 16-42-38.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 03, 2022 10:19 pm

Its already included in 3.7 that was released some day ago.
If you still se it, in what dashboard?
You can just edit dashboard select source and change first line from:
<form theme="dark">
to
<form version="3.7" theme="dark">
This is just a stupid warning that was added in splunk 9.0 that was released in juni. You can ignore it.
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 03, 2022 11:23 pm

no need to quote preceding post - use "Post Reply"
True, I've added it manually so no problem for me. I know its just cosmetic thing.
Indeed I've started seeing it since running Splunk 9.x release.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 04, 2022 9:06 am

my docker splunk was updated to 9.0.1 on 2022-09-01
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 04, 2022 9:21 am

@jotne
really strange, restarting the splunk container fixed displaying my gateway router in splunk.
just that 1 disappears, all other devices do not show this behaviour.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 04, 2022 6:40 pm

Not sure why. Are some device at different distanse? (passing other routers etc)
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 04, 2022 6:43 pm

No, not really, all the same class C network here @home ...
Some passing thru a switch, some directly connected.
The gateway that is stops being visible is producing the most traffic, that is the only difference
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Sep 05, 2022 12:30 pm

the gateway stopped again after 23 times ... not visible in Splunk right now.
@jotne any suggestions ?
gateway is functioning normally, sending syslog to an other remote syslog thru udp/2514
splunk is listening on udp/3514
at least 6 other devices are sending to that port and are visible in splunk
Screenshot 2022-09-05 at 11.28.40.png
screenshot from last 32hours ...
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Sep 05, 2022 5:21 pm

Set up an Ubuntu server (No container) add rsylog. Then on the gateway send syslog to New rsylog and Splunk to see if on or both stops recieving data.

If you use my post to setup syslog, you can see if it recieve data by looking in folder /data/syslog/udp/ip of device/file
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Sep 05, 2022 5:44 pm

tnx for the suggestion. I am not going to that for now. I don't have any hardware to run this on.
I am running this in in docker on my synology DS420+ for several reasons and I am not going to use more hardware.
I had no problems before, have to learn to live with this,
tnx for your effort so far.
 
mducharme
Trainer
Trainer
Posts: 1777
Joined: Tue Jul 19, 2016 6:45 pm
Location: Vancouver, BC, Canada

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Fri Sep 09, 2022 10:12 am

Too bad the "Kid Control" is actually not accurate on my 7.4.1 box.
Not too sure what is happening, but the use-case is simple. On my ShieldTV I've just watched a 16Gbytes movie. This is served from my NAS.
It is very possible that Kid Control does not properly account for traffic that is bridged, and not routed. If your NAS is on the same subnet as your ShieldTV, this traffic would be bridged, and might not be included in the Kid Control statistics. However, this traffic also would not probably be included in the older RouterOS v6 IP accounting statistics, since I believe it had the same limitation.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Tue Sep 13, 2022 10:55 am

Although I updated the script and application today, I have always felt that the visualization of interface traffic is not very good. Where should I adjust it?
2022-09-13_15-52-13.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Tue Sep 13, 2022 8:43 pm

Not sure what you like to change. I am always open for improvement :)
It show 10,796.5 = 10GB I may round the data to not show so many decimal.
From your picture it seems that you have already changed some, like title the size has modded text and graph some compare the what I do see.
.
interface.png
You do not have the required permissions to view the files attached to this post.
 
pospanko
Member Candidate
Member Candidate
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Mon Oct 10, 2022 11:22 pm

Hi

I'm trying to log current interface traffic - current usage.
This is my script:
:if ($Traffic) do={
	:foreach id in=[/interface find] do={
        :local output "$[/interface monitor-traffic [find where .id=$id] once as-value]"
		:set ( "$output"->"script" ) "traffic"
		:log info message="$output"
	}
}
Problem is that I copied Interface Traffic page to new one (cloned) but I'm not getting all of the interfaces on the list. I changed only few things but I get only half of them. As I can see, all interfaces with large amount of traffic aren't listed.
Just started with this so I'm still learning...
I use only fp_rx_bits_per_second and fp_tx_bits_per_second values from response.
Is there maybe anyone who can create page for that?
Thanks.

EDIT
I think that problem is 6.45.9 MT version. On 6.49.6 everything works ok. I'll try to upgrade...
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Tue Oct 11, 2022 8:25 am

You should at least upgrade. 6.45.9 is more then 2 years old. :)
 
AMeador1
just joined
Posts: 22
Joined: Tue Jan 04, 2022 2:13 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Thu Nov 10, 2022 5:19 pm

You said:
Using Splunk to monitor and graph various data from our MikroTik Routers is a nice and free way to help you showing what is going on in your network.
Splunk is free to use for logging up to 500MB pr day.
But I can't seem to find an option at the site that show this. Too many products/services/pricing models. Maybe it its there and I just missed it.
I'm looking at getting monitoring going on my small WISP so this is of interest. I'm working with The Dude as well - curious to compare - just don't need another bill right now.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Thu Nov 10, 2022 11:12 pm

See section about free license in the link:
https://docs.splunk.com/Documentation/S ... nklicenses

More here and how to convert to free license:
https://docs.splunk.com/Documentation/S ... al_License

Only thing its not very clear about is that if you do not convert to Free license before 60 days, you will get inn to big problem with your splunk.
So if you like to use splunk free, you NEEEED to convert it before 60 days has gone.

PS links was found quickly using google search :)

Did change the first post to reflect that its 60 days to convert not 30 as before.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.7 (Graphing everything) 💾 🛠 💻 📊

Sat Nov 26, 2022 3:41 pm

Main script updated to 5.1 (see section 2g)
Added CAPsMANN to script (no need for separate CapsMANNScript)
Added more Routing information
Removed on-error by useing main RouterOS version to run correct script
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Sun Nov 27, 2022 9:54 am

Version 3.8 is now released.

More remote connection dashboards. Fixed problem in firewall rules when packet mark was added to logging by Mikrotik.
Lots of other small fixes.

New file are found under section 1g
Download, extract and replace old app (if you have not med personal changes)

PS Main script needs to be updated to at least 4.9 to get routing information. (latest is 5.1)

What has changes since 3.7

# 3.8 (27.11.2022)
# Added new dashboard "MikroTik IPsec connections"
# Added new dashboard "MikroTik PPP connections"
# Change from bytes to delta_bytes in table in "MikroTik Device Traffic"
# Removed 0 fields to make math correct in "MikroTik Device Traffic"
# Hide parameter list in "MikroTik DHCP request"
# Change transaction to 100ms in "MikroTik DHCP request"
# Set dropdown to request as default in "MikroTik DHCP request"
# Added packet mark in "MikroTik Firewall Rules"
# Added new panel with optional input in "MikroTik Firewall Rules"
# Fixed missing packet-marking i firewall extraction
# Added disabled status and added VPN route in "MikroTik Route List"
# Fixed "Device table updater" to better follow firmware upgrade/downgrade
# Fixed saved search rights to not show up everywhere
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Sun Nov 27, 2022 1:35 pm

Thanks Jotne! for this new release. Both updated 5.1 script & 3.8 Splunk-app are working fine over here!
 
TheNetworkBerg
just joined
Posts: 15
Joined: Mon Sep 30, 2019 9:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Mon Nov 28, 2022 9:06 pm

Hi Jotne,

Awesome thread and really appreciate the work you have put into the script. I would like to test out the logging for myself, would it be okay if I made a video on your script if I get everything working correctly? I will definitely give you a shoutout and put links to this thread in a YouTube video. Here is a link to my YouTube channel where I cover various other MikroTik configurations (https://www.youtube.com/c/thenetworkberg). Another custom script I covered a few years ago was Nathan1's HA script. I just don't want to create a video if you are planning on doing that yourself or feel unhappy with someone else doing it. Hoping to hear back from you soon, wanted to send you a message but couldn't seem to do that on your profile. Have a great evening/day

Kind Regards,
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Tue Nov 29, 2022 7:57 am

Hi

I am not much of a video guy, nor do I have equipment for that. So If you like to do it, you are welcome to do so.
Have seen your videos that are very good, so looking forward to see the result :)

PS If you like to get in contact with me for question, you can reach me at this temp mail coribi3326@xegge.com
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Thu Dec 29, 2022 3:58 am

The same two ccr1016s have the same firmware version 6.48.6, but one does not work after upgrading the 5.1 script. There is also a CCR1009-8G-1S that has the same problem. I checked the update log and other files, but I can’t see where the problem is. Can you tell me what the possible problem is?Currently temporarily restored to 4.9.
2022-12-29_09-52-42.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Thu Dec 29, 2022 5:07 pm

Not sure why 5.1 does not work for you. Just tested it on a blank 6.49 router and all works.

Try in Splunk to search for:
index=* host=your_router_ip
And also see in first post:
4) Debugging

Also, what does not work? You see the router in the device list. PS it may take a day for all to show correctly.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Dec 30, 2022 3:04 am

Using the 5.1 script, you can’t see any action in the log after manually running it, which can be understood as not running. After manually running the 4.9 script, you can see the output immediately.
5.1 Is this paragraph in the script the cause
“# Get RouterOS main version (used to run different script on different version)
:local train [:tonum [:pick [/system resource get version] 0 1]]”
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Dec 30, 2022 12:02 pm

PS, no need to quote the post above you, use "Post Reply"

Then I do not get what is wrong. v5.1 are coming as a part of the script, so you do get some data to splunk.
What are the output of:
index=* host=192.168.0.1 | fillnull value="-" | stats count by module script | sort -count
For one of my router it looks like this:
module	script	count
dns	-	127149
firewall	- 18190
script	kids	15744
dhcp	-		12544
script	address_lists	3213
script	neighbor	2776
script	if_traffic	1771
script	route	989
script	upnp	491
script	health	382
ipsec	-	255
script	ntp	192
script	resource	192
script	version	192
script	ipsec	109
script	pool	72
script	ppp	44
upnp	-	33
script	sysinfo	24
l2tp	-	10
script	IPSEC_failed	8
script	-	2
info	-	1
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 06, 2023 10:24 am

index=* host=10.0.0.1 | fillnull value="-" | stats count by module script | sort -count
script 5.1

dhcp	-	186239
dns	-	1017
script	-	12
system	-	4
e-mail	-	1


script4.9

dhcp	-	517849
script	kids	115519
script	neighbor	6456
script	upnp	4935
script	if_traffic	4785
script	address_lists	4060
script	route	3625
dns	-	3124
script	pool	300
script	health	145
script	ntp	145
script	resource	145
script	version	145
script	-	36
script	sysinfo	12
upnp	-	10
system	-	8
e-mail	-	2


Changed to script v5.1 at 9:00 in the morning, and it will not be visible in the device list.
2023-01-06_16-35-07.png
You do not have the required permissions to view the files attached to this post.
 
akabyshev
just joined
Posts: 1
Joined: Mon Jan 09, 2023 2:00 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Mon Jan 09, 2023 3:06 pm

Hi Jotne,
Great work you've done here, thanks!
Did you consider putting everything on github? People here could help with localization, also having a versions history on the script could help a bunch.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Tue Jan 10, 2023 12:25 pm

@fengyuclub

Can looks like some breaks the script getting data on your router.
If you can test part by part of the the script, we can see where it goes wrong.

Example Resources are missing, so test this first.

Section looks like this:
:if ($SystemResource) do={
	/system resource
	:local cpuload [get cpu-load]
	:local freemem ([get free-memory]/1048576)
	:local totmem ([get total-memory]/1048576)
	:local freehddspace ([get free-hdd-space]/1048576)
	:local totalhddspace ([get total-hdd-space]/1048576)
	:local up [get uptime]
	:local sector [get write-sect-total]
	:log info message="script=resource free_memory=$freemem MB total_memory=$totmem MB free_hdd_space=$freehddspace MB total_hdd_space=$totalhddspace MB cpu_load=$cpuload uptime=$up write-sect-total=$sector"
}
To test it out do the following.
1. Remove first line up to first {, so remove here :if ($SystemResource) do=
2. Change :log info message= to :put
3. Optional, remove all tabs in front of all lines to not get extra info when cut/paste to terminal. If there are two tabs it shows commands etc.
4. Then copy paste to terminal and see what you get out.

Example here would be
{
	/system resource
	:local cpuload [get cpu-load]
	:local freemem ([get free-memory]/1048576)
	:local totmem ([get total-memory]/1048576)
	:local freehddspace ([get free-hdd-space]/1048576)
	:local totalhddspace ([get total-hdd-space]/1048576)
	:local up [get uptime]
	:local sector [get write-sect-total]
	:put "script=resource free_memory=$freemem MB total_memory=$totmem MB free_hdd_space=$freehddspace MB total_hdd_space=$totalhddspace MB cpu_load=$cpuload uptime=$up write-sect-total=$sector"
}
Output on my test router:
script=resource free_memory=83 MB total_memory=128 MB free_hdd_space=102 MB total_hdd_space=128 MB cpu_load=8 uptime=6d17
:15:37 write-sect-total=163153
You can also set everything in the script to false, then on by on to true for testing.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Tue Jan 10, 2023 12:26 pm

Great work you've done here, thanks!
Did you consider putting everything on github?
Thanks, I am thinking of that, but still my git knowledge are some limited :)
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Thu Jan 12, 2023 3:28 am

Tested the script on several mikrotik, but again there is no output, as shown in this picture. 'print' will output information
2023-01-12_09-21-49.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Thu Jan 12, 2023 5:17 pm

You can not put multiple command line that belongs together to the terminal windows. They must be wrapped in {}.
The red colored variable show that rest of data are missing.

It looks like that you did not cut and past my post. See my example.

Tested on 6.49 and 7.6

How it looks like for me:
.
script.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 13, 2023 4:53 am

Sorry I didn't read your post carefully
script=resource free_memory=1575 MB total_memory=1952 MB free_hdd_space=428 MB total_hdd_space=512 MB cpu_load=12 uptime=4w3d04:50:39 write-sect-total=
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 13, 2023 12:09 pm

Ok, then we know that first part of the script does not break. Then there are only 17 parts more to test.

What you can do it so se all part below to false, then test script, change on to true, test script, change next to true etc. SystemResource seems to be fine.
:local SystemResource true
:local SystemInformation true
:local SystemHealth true
:local TrafficData true
:local AccuntData true
:local uPnP true
:local Wireless true
:local AddressLists true
:local DHCP true
:local Neighbor true
:local InterfaceData true
:local CmdHistory true
:local CAPsMANN false
:local Routing true
:local OSPF false
:local BGP false
:local PPP true
:local IPSEC true
,

If I had access to the router (vpn), I could have tested it out.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Sat Jan 14, 2023 8:17 am

Tried both true and false, there is no prompt or output in the log
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Sat Jan 14, 2023 11:34 am

Then do as I write in post above. Split script in to 18 parts. Remove if test, change log to print and test one by one, as we did with resources.
If you can give me ssh access, I can test it out and find out what is wrong. You can reach me on email removed
Last edited by Jotne on Tue Jan 17, 2023 12:11 pm, edited 1 time in total.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Mon Jan 16, 2023 9:29 am

The ssh access permission has been sent to your email.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Mon Jan 16, 2023 12:01 pm

Deleted
Last edited by Jotne on Wed Jan 18, 2023 11:35 am, edited 2 times in total.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Tue Jan 17, 2023 3:31 am

Has been sent
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Tue Jan 17, 2023 12:11 pm

Deleted
Last edited by Jotne on Wed Jan 18, 2023 11:35 am, edited 2 times in total.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 18, 2023 3:10 am

Has been sent
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 18, 2023 11:42 am

I do post answer here, so other with same problem now what to do.

Error found. You are running 6.48.6, that for some reason fails on this command.
:foreach i in=[find where master-interface="none"] do={
I have tested the script on 6.49.x and most 7.x version and there it runs fine. I guess it would have worked if you have had CapsMann installed.
But to get it to work, just remove from the script the section that start with:
# Test if CAPsMANN is installed.
and all down to but not included:
# Collect routing information
and it should work
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 20, 2023 2:26 am

From the logs of your router, I do see there are some serious error in DHCP request dashboard that I need to look inn to.
It seems to join many request together.

Problem are that one DHCP request are made up of many log messages. No one of these message do contain any unique ID, so it becomes hard to see what belongs together. So if MT reads this message please add ID to all DHCP message where each message that belongs together have same ID.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 20, 2023 3:01 am

Thank you for discovering the problem. After I enabled system/package/wireless, the problem was solved. Since I use the router as a hardrouter, there is a separate unifi control to take over the ap device, so the wireless module is naturally disabled. It should be the problem. As for the dhcp problem you mentioned, I don't think it affects anything. It should be that I didn't find the problem.
2023-01-20_09-04-42.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Jan 20, 2023 12:02 pm

Look at DHCP-> MikroTik DHCP Request.
There should only be one line for each request, not multiline with many different IP and different MAC.

This is due to many DHCP request more or less at the same time, so Debug message are clogged togeather.
It can be solved by only looking at Info message, but then we do loose DHCP Request Name, Class_Id etc.

Until MikroTik does add ID for each DHCP request, I will change the view to a simple default view, that only shows VlAN, IP and MAC, with an option to show more details (if server are not heavy loaded)

For the error in CapsMan part of the script, I am not sure if I will fix it, since 6.x are going away and it works on 6.49 and later (7.x)
It may be enough to post a not about removing that section for older OS or upgrade the OS

This should be 30+ lines not 3...
.
dhcp.png
You do not have the required permissions to view the files attached to this post.
 
barrthek
just joined
Posts: 7
Joined: Wed Feb 01, 2023 5:01 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.6 (Graphing everything) 💾 🛠 💻 📊

Wed Feb 22, 2023 8:48 pm

Hi problem with "script error: error - contact MikroTik support and send a supout file (10)" in "#Get detailed command history RouterOS" is probalby related to number of enrities in /system history - if there are too many entries, an error message appears (even executing "/system history print"). Rebooting router clears this history and temporaily solving problem.
So initially I executed and these are the env-vars

Hour = 18
cmd = *238B

Then I added Netflow-config + target and re-ran

Hour = 18
cmd = *2391


Now I have to say, it has been some time since that error popped up...the last one about 2 hours ago.
That is the same time I reconfigged the "logging target" settings. Hmm, probably fixed, but thanks the feedback!
 
barrthek
just joined
Posts: 7
Joined: Wed Feb 01, 2023 5:01 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Thu Apr 06, 2023 12:53 am

I had a problem with proper dispalying channels for 2.4 ghz wifi in capsman. I solved that by updating entry in proper.conf to:
EXTRACT-mikrotik_caps-man_frequency = caps,(?:info|debug).*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<width>\d+)-?(?<extensionChannel>[a-zA-Z]+)?\/(?<standard>[a-zA-Z]{2})[^\d]*(?<dBm>\d+)dBm 
However logs with "caps,info" with frequency appear very rarely.
So, I added an additional script to log it:
:foreach i in=[/caps-man interface find where master-interface="none"] do={
  :local name [/caps-man interface get $i name]
  :local channel [/caps-man interface get $i current-channel]
  :log info "$name: selected channel $channel"
}
but I have no idea how to properly add information from "script,info" to module=caps eventtype=capsman_channel :(
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Thu Apr 06, 2023 12:02 pm

I would like to help out, but since I do not use capsmann, its not easy to see what is going on.
But If you like, I can have a look at your logs and try to enhance the capsmann logs.
You can send me an email on hison88097@fectode.com
 
barrthek
just joined
Posts: 7
Joined: Wed Feb 01, 2023 5:01 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri Apr 07, 2023 4:39 pm

I would like to help out, but since I do not use capsmann, its not easy to see what is going on.
But If you like, I can have a look at your logs and try to enhance the capsmann logs.
You can send me an email on hison88097@fectode.com
Thanks Jotne, but i think i solved this:
list of changes:

EXTRACT-mikrotik_caps-man_frequency in props.conf to:
EXTRACT-mikrotik_caps-man_frequency = (?:caps|script),(?:info|debug).*?: (?<ap>[^:]+): selected channel (?<frequency>\d+)\/(?<width>\d+)-?(?<extensionChannel>[a-zA-Z]+)?\/(?<standard>[a-zA-Z]{2})[^\d]*(?<dBm>\d+)dBm
capsman_channel in eventtypes.conf to:
[capsman_channel]
search = "*,info *: selected channel"

Base search in "Mikrotik CAPsMAN Channel Usage" (i leave both options - generated by caps and by script)
<search id="base_search">
    <query>
      `index`
      sourcetype=mikrotik
      module="script" OR "caps"
      eventtype=capsman_channel
      host="$Host$"
      ap="$AP$"
      | eval host_name=coalesce(identity,host)
      | fields _time host_name host ap channel frequency
      | sort 0 -_time
    </query>
  </search>
i've also modified script (Capsmann) to log every hour frequency & channel :
# Test if CAPsMANN is installed.
# ----------------------------------
:if ( ([:len [/interface find where type="cap"]] > 0) and $CAPsMANN) do={ 
	:local capsregistered ([/caps-man registration-table print count-only])
	 
	/caps-man interface
	:local name
	:local mac
	 
	# ignore all master interfaces
	:foreach i in=[find where master-interface="none"] do={
		:set name [get $i name]
		:set mac [get $i radio-mac]
		:local counter ([/caps-man registration-table print count-only  where interface=$name])
		:log info message="script=caps-man name=$name counter=$counter"
	}
	:log info message="script=caps-man capsregistered=$capsregistered"
#added to log frequency&chennel
:if  ($CAPsMANN and $run) do={
	:foreach i in=[/caps-man interface find where master-interface="none"] do={
		:local name [/caps-man interface get $i name]
		:local channel [/caps-man interface get $i current-channel]
		:log info "$name: selected channel $channel"
		}
	}
}
Logs looks like that:
script,info MikroTik: AP_Ground-2: selected channel 5220/20-eeCe/ac/P(23dBm)
script,info MikroTik: AP_Ground-1: selected channel 2447/20-Ce/gn(20dBm)
script,info MikroTik: AP_Gralnia-2: selected channel 5240/20-eeeC/ac/P(23dBm)
script,info MikroTik: AP_Gralnia-1: selected channel 2412/20-Ce/gn(20dBm)
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Sat Apr 08, 2023 12:12 am

@Jotne


;)
Last edited by rextended on Sat Apr 08, 2023 2:53 pm, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Mon Apr 10, 2023 11:04 am

Main script updated to 5.2 (see section 2f)
Added more CAPsMANN logging
list of changes:
@barrthek can you test if script in section 2f works correctly with CAPsMANN
 
barrthek
just joined
Posts: 7
Joined: Wed Feb 01, 2023 5:01 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Tue Apr 25, 2023 1:03 am

Main script updated to 5.2 (see section 2f)
Added more CAPsMANN logging
list of changes:
@barrthek can you test if script in section 2f works correctly with CAPsMANN
Hi Jotne,

The script has been tested and looks ok on my configuration (hEX PoE + 2 cAP ac with CAPsMAN and 3 Wi-Fi networks).
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri May 12, 2023 1:37 pm

Tested script on 7.10beta and it seem to work fine. Since only hour are used, the new date format should not give any problem.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Fri May 12, 2023 2:38 pm

[…] Since only hour are used […]
(as predicted...)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.8 (Graphing everything) 💾 🛠 💻 📊

Tue Jun 13, 2023 2:25 pm

Version 3.9 is now released.

To get the new version, download it from Git:
https://github.com/Jotne/MikroTik

By adding it to Git, I do hope it would be easier for other to collaborate :D



What has changes since 3.8

# 3.9 (13.06.2023)
# Added discovery type to "MikroTik Neighbor"
# Added new version of "MikroTik DHCP layout"
# Added CapsMann fixes by barrthek
# Updated dhcp_table to work better with new firmat (no more debug logs)
# Changed to use dhcp_table for client name in "MikroTik Wifi connection" and "MikroTik Wifi strength"
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jun 14, 2023 7:06 pm

The guide says

"1h) Extract the spl file" ...

Where can I find the .spl file on github ???
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jun 14, 2023 8:48 pm

File uploaded, but its not needed. Just download the app from GIT and copy the folder mikrotik to %SplunkHome%/etc/apps and restart splunk.
 
User avatar
k6ccc
Forum Guru
Forum Guru
Posts: 1490
Joined: Fri May 13, 2016 12:01 am
Location: Glendora, CA, USA (near Los Angeles)
Contact:

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jun 19, 2023 12:03 am

Just download the app from GIT and copy the folder mikrotik to %SplunkHome%/etc/apps and restart splunk.
I'm looking at: https://github.com/Jotne/MikroTik
What am I supposed to download?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jun 19, 2023 8:31 am

Use: Code -> Download ZIP
.
git.png
.
Format of folder should be

%Splunk-Home&/etc/app/MikroTik
- appserver
- default
- metadata
- README
- static
- README.md
You do not have the required permissions to view the files attached to this post.
 
User avatar
k6ccc
Forum Guru
Forum Guru
Posts: 1490
Joined: Fri May 13, 2016 12:01 am
Location: Glendora, CA, USA (near Los Angeles)
Contact:

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jun 19, 2023 7:56 pm

Thanks. I have never use github for anything other than one project that I beta which test uses github for bug reports - just writing comments.

I'm doing this on a Windows server, so I assume the extract from the zip should go into C:\Program Files\Splunk. The root of the zip is MikroTik-main with the directories you mention below that. Does the MikroTik-main and it sub-folders go into the Splunk directory, or skip the MikroTik-main and just put the sub-folders into the Splunk directory?

I followed the directions in the first post in this thread which says to extract the zip and copy that all to the Splunk directory (MikroTik-mail and it's sub-folders), but I get absolutely no indication in Splunk that I accomplished anything by doing so (yes, restarted Splunk several times). But I also really have no idea what I am expecting to see...
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jun 19, 2023 9:50 pm

You can use Mikrotik-main, but I do suggest you remove the main, so the folder should be

C:\Program Files\Splunk\etc\apps\MikroTik\default
C:\Program Files\Splunk\etc\apps\MikroTik\metadata
C:\Program Files\Splunk\etc\apps\MikroTik\<+ the rest of folders>

PS restart Splunk after adding the files.

I did added splunk to git so if other like to help out or add stuff it would simpler to do so.

You can also download the rar file in first post under section 1g.
The unpack the file and you get a Mikrotik.spl
Go then to you Splunk->Apps-Manage apps->Install app from file
Splunk will then install the app for you.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jun 19, 2023 11:33 pm

I used to download the .spl and load it thru the splunk interface ...
Last week I downloaded the git stuff and tried to upload it manually into the directory inside the splunk docker that I run.
I noticed that I needed to correct the user/group manually to get things working again ...
Normally splunk did that but by uploading the files it has to be done by hand to match the system ...
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Tue Jun 20, 2023 1:33 pm

You can use chown -R on the folder to sett correct user/group.
Installing it using git will be som manual work.
You can use git pull and point it directly to your folder, but that will make a risk of overwriting stuff you do manually.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jun 26, 2023 12:26 pm

I recently re-deployed a splunk9.0.5 machine on the intranet, and then forwarded all the previous logs to this splunk. The data can be indexed, but the dashboard in the application has no data, and even basic dhcp requests can be seen No, why is this? Is the splunk version too new?
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Jun 29, 2023 5:19 am

@Jotne
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Jun 30, 2023 4:32 am

You can use chown -R on the folder to sett correct user/group.
Installing it using git will be som manual work.
You can use git pull and point it directly to your folder, but that will make a risk of overwriting stuff you do manually.
please help me
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Jun 30, 2023 8:49 am

No need to multipost. How does it look like when you follow section 4) in first post?
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Jun 30, 2023 10:30 am

I set it up step by step according to the steps in your post, and it started working, and the data was indexed, which is great.

There is another problem. I have been using a high temperature warning script, but now it seems that it cannot be used after v7. After reading many posts in the forum, I can't find it available. I saw an article about the early warning after the power supply failure, which can be used for reference, but will not be modified. I implore @Jotne to give other solutions. In addition, splunk has alarms for sending emails, as well as webhooks, but how to implement specific webhooks? There are no similar posts on the forum.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Jun 30, 2023 11:40 pm

If you have full version of Splunk, you can use it to send email alerts on various condition, but Splunk Free does not include alerts, authenticated logins, and a number of other enterprise features available only to customers of paid versions.
There is an other option, you can ask for a developer version that gives you all option and 10GB logs pr day. You only need to request it every 6 month.

Regarding the script, make a new post an I am sure some know how to fix it. As long as you can see it on the router that some is wrong and as long as it still has internet, the router can use many option to warn the user. I do use pushbullet to get phone notification.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Jul 01, 2023 5:24 am

thank you for your reply. I just applied for the 10G enterprise license according to your post. I read the official splunk manual about webhook, but I didn't find a solution. I'm looking for other ways. In fact, this notification is to solve the high temperature reminder caused by the power outage of the air-conditioning equipment, or there is a better solution.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Jul 01, 2023 8:35 pm

As long as the Router sees the temperatur:
You can:

* Use alert from Splunk to send email (needs full or develop license). May be some work to set it up
* Send email from Router. Search gmail setup on Router OS
* Send pushbullet from Router (to browser og phone)
* Send telegram
* There may be other way to do it as well...
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jul 03, 2023 3:38 am

My mikrotik is ccr1016, and the /system/health/print information is as follows:
  system health/pr
Columns: NAME, VALUE, TYPE
# NAME VALUE TYPE
0 cpu-temperature 50 C
1 fan1-speed 5730 RPM
2 fan2-speed 0 RPM
3 board-temperature1 28C
4 psu1-state ok
5 psu2-state fail
How should I write the alarm script in splunk? I tried the following method, but it didn't work. The mail setting in the router and the mail setting of splunk are all successful. pushbullet was not considered due to some kind of censorship.
host="10.0.0.1" board-temperature1 value>26
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jul 03, 2023 10:52 am

You need to make sure splunk sees the temperature. MikroTik app -> Overview -> MikroTik Health.
Or from search (you can post the output and I can see what you see)
`index`
sourcetype=mikrotik
module=script
script=health
You should se some like this (depends on router)
script,info MikroTik: .id=*e;name=temperature;script=health;type=C;value=38
script,info MikroTik: .id=*d;name=voltage;script=health;type=V;value=25
For this I can make an alert:
`index`
sourcetype=mikrotik
module=script
script=health
host=192.168.88.1
name=temperature
| where value>50
Save as Alert. Set how often and email etc.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Jul 06, 2023 5:52 am

According to your suggestion, I changed it slightly and it works normally.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jul 12, 2023 3:29 pm

Updated "MikroTik Interface Traffic" to give graph with correct scale regarding Mbps and better handling of multiple hosts.
PS this can be download for the git repository.
.
bps.png
You do not have the required permissions to view the files attached to this post.
 
lordimac
just joined
Posts: 14
Joined: Fri Mar 04, 2022 11:24 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jul 31, 2023 3:38 pm

Hello and thanks for the great app! Works like a charm. But where is the difference to the Splunk Technology Add-On for Mikrotik RouterOS which is located here: https://splunkbase.splunk.com/app/3845
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jul 31, 2023 9:08 pm

You can look at it your self here: https://github.com/schose/TA-routeros
The TA app has not been updated for more than 3 years, and are mainly used to normalize data using field extraction.
There are no dashboard in this app. You do not need it if you are using the app from this page.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Aug 25, 2023 1:12 pm

One of my ccr1016-12G doesn’t record data to splunk anymore. I can see the error message in the mikrotik log, as shown in the figure below, every time it is 998. It’s very strange. I try to stop and delete the corresponding one according to the error log field. "# Count IP in address-lists" and "upnp", but the error message is still there, I don't know what to do.
2023-08-25_17-56-55.png
2023-08-25_18-07-26.png
You do not have the required permissions to view the files attached to this post.
 
barrthek
just joined
Posts: 7
Joined: Wed Feb 01, 2023 5:01 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Aug 25, 2023 11:57 pm

fengyuclub

Hi problem with "script error: error - contact MikroTik support and send a supout file (10)" in "#Get detailed command history RouterOS" is probalby related to number of enrities in /system history - if there are too many entries, an error message appears (even executing "/system history print"). Rebooting router clears this history and temporaily solving problem.
 
barrthek
just joined
Posts: 7
Joined: Wed Feb 01, 2023 5:01 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 26, 2023 12:04 am

Hi Jonte,
I've just updated the app to 3.9 and you probably forgot to change capsman_channel in eventtypes.conf to:
[capsman_channel]
search = "*,info *: selected channel"
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 26, 2023 10:14 am

When you change from
search = "caps,info *: selected channel"
to
search = "*,info *: selected channel"

You are sure it still works correctly and does not hit other stuff than capsman?
I do not have capsman, so can not test it.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Aug 26, 2023 11:18 am

Updated "MikroTik Device Traffic" to give correct graphs if router is rebooted or counter is reset.
Get your update from git.
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 28, 2023 5:26 am

fengyuclub

Hi problem with "script error: error - contact MikroTik support and send a supout file (10)" in "#Get detailed command history RouterOS" is probalby related to number of enrities in /system history - if there are too many entries, an error message appears (even executing "/system history print"). Rebooting router clears this history and temporaily solving problem.
After restarting the router, the error message is gone, but splunk still has not received the data
 
User avatar
fengyuclub
Frequent Visitor
Frequent Visitor
Posts: 98
Joined: Mon Dec 09, 2013 8:50 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 28, 2023 6:07 am

Updated "MikroTik Device Traffic" to give correct graphs if router is rebooted or counter is reset.
Get your update from git.
After the update, the above problem has been solved,but "script error: error - contact MikroTik support and send a supout file (10)" This error message still exists
 
barrthek
just joined
Posts: 7
Joined: Wed Feb 01, 2023 5:01 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Aug 28, 2023 10:25 pm

When you change from
search = "caps,info *: selected channel"
to
search = "*,info *: selected channel"

You are sure it still works correctly and does not hit other stuff than capsman?
I do not have capsman, so can not test it.
Yes,
During my testing, I observed that there are two types of logs related to the "selected channel":

When CAPsMAN changes a channel, it is logged as "caps,info *" – for example, when you modify the "channel" in the CapsMan configuration. This type of log is irregular and sometimes occurs only once every several days.
Logs triggered by a script are logged as "script,info *" and occur regularly every 1 hour.
Therefore, the proposed change covers both situations,and does not hit other stuff.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 02, 2023 5:49 pm

I am running splunk for a while now and with some help from @jotne now and than it works rather good ...
BUT
As I am running splunk in docker on a synology I experiance some very strange things.
After a while the logging to splunk stops ...
I have a couple of devices sending log to the splunk machine.
Especially my gateway, a ccr1009. has this problem. the log is not received by splunk after a while.
This gateway is sending more log than any other device in my network ..
Anyone have suggestions in how to debug or fix this ? (other than getting a non-docker-synology solution) ?
Eddie
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Sep 02, 2023 6:27 pm

I'm running Splunk on a Synology too, but as a VM under Ubuntu Linux, not containerized.
Works OK in general, had 1 or 2 occasions where the 4GB assigned memory fell short and things fell apart ;-)

>> After a while the logging to splunk stops ...

Splunk generates a ton of logging messages that might give you an indication why something "stops" working. Did you check any of these ?
(with a container, you'll have to open a shell I guess)

/opt/splunk/var/log/splunk

Are you not exceeding the 500Mbytes daily limit ??
Top menu "Settings" then "Licensing" (under the "System" section)
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 03, 2023 8:54 am

Hi @jvanhambelgium

tnx for your answer.
I did check the license volume more than once and that should not be the problem. According to the spluk webinterface the daily volume stays below 200M
I will be looking into that logfiles, a quick look in splunkd.log shows an error that occors every couple of seconds ...
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py" [modular_input:349] [execute] [25520] Modular input: Splunk Assist exit with exception: Traceback (most recent call last):
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"   File "/opt/splunk/etc/apps/splunk_assist/bin/assist/modular_input.py", line 342, in execute
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"     self.do_run(input_definition["inputs"])
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"   File "/opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py", line 66, in do_run
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"     if not should_run(self.logger, self.session_key):
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"   File "/opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py", line 27, in should_run
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"     sh = is_search_head(log, session_key)
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"   File "/opt/splunk/etc/apps/splunk_assist/bin/assist/serverinfo.py", line 153, in is_search_head
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"     cluster_mode = get_cluster_mode(log, session_key)
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"   File "/opt/splunk/etc/apps/splunk_assist/bin/assist/serverinfo.py", line 257, in get_cluster_mode
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"     raiseAllErrors=True
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"   File "/opt/splunk/lib/python3.7/site-packages/splunk/rest/__init__.py", line 646, in simpleRequest
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py"     raise splunk.LicenseRestriction
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py" splunk.LicenseRestriction: [HTTP 402] Current license does not allow the requested action
09-03-2023 05:50:02.625 +0000 ERROR ExecProcessor [1599 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/splunk_assist/bin/uiassets_modular_input.py" .
I am interested in the way you run splunk inside ubuntu, how did you get that working ?
runnning syno virtual machine manager ? and than a plain ubuntu image ? and than a normal ubutu splunk install ?
I am running latest DSM on a 920+ with enough resources
 
User avatar
jvanhambelgium
Forum Veteran
Forum Veteran
Posts: 985
Joined: Thu Jul 14, 2016 9:29 pm
Location: Belgium

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 03, 2023 9:29 am

I am interested in the way you run splunk inside ubuntu, how did you get that working ?
runnning syno virtual machine manager ? and than a plain ubuntu image ? and than a normal ubutu splunk install ?
I am running latest DSM on a 920+ with enough resources
Indeed, just like that. I'm running on 918+ on DSM 6.2.4
I'm running Ubuntu 18.04.5 LTS (I'm running this Splunk VM for several years already) , but I would suggest you look at 20.04(.06) LTS.
(I might try to upgrade from 18.04 > 20.04 LTS soon, but I'll take a backup first ;-)

Installing Splunk on Linux is pretty simple.

Eg.
https://medium.com/@dannyopara/installi ... 45982038c3

And there are dozens of similar tutorials.
Once installed, the approach is similar offcourse.

Concerning your errors, be aware that Splunk generates quite a lot of logging and it's a complex framework, I also have some weird looking errors fly by from time to time but things works. You might try to Google about your specific error and this might point to the Splunk forums perhaps with some explanations & fixes.
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 03, 2023 9:36 am

tnx,
will try that later, have to do family things today ;-)
I will go for debian as I am more familiar with that than ubuntu ...
running several RPI with debian over here
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Sep 04, 2023 12:16 pm

Ok,
I did a fresh debian install on the syno inside a VM. That went fine, It has is own IP and SSH.
I did install the splunk.deb and it runs.
Pointed my devices to the new IP and all seems to work.
Now just wait a couple of days and see if the lost UDP traffic problem is solved
tnx so far
Eddie
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Sep 06, 2023 8:07 am

After the update, the above problem has been solved,but "script error: error - contact MikroTik support and send a supout file (10)" This error message still exists
Do you have more than one script? Do you run latest ROS?
 
jult
Frequent Visitor
Frequent Visitor
Posts: 52
Joined: Sat Dec 26, 2020 1:16 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 10, 2023 12:40 am

Version 3.9 13.06.2023

:local AccuntData true

:if ($AccuntData) do={
You do realize it's "AccountData" ?
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 10, 2023 8:37 am

After 6 days I allready can tell that running Splunk inside VM on a Synology works a lot better than running it in Docker.
In Docker I experienced UDP drops, log records were send but just didn't reach Splunk.
In VM this does not happen and all devices sending logs appear inside Splunk.
SO, do not use Docker on Synology, install Virtual Machine Manager and use a debian based OS. Install Splunk and of you go !
tnx for the suggestion @jhvanhambelgium and oof course @jotne for all the effort
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 10, 2023 10:37 am

You do realize it's "AccountData" ?
Fixed the script. Strange there are not more errors ;)
 
eddieb
Member
Member
Posts: 305
Joined: Thu Aug 28, 2014 10:53 am
Location: Netherlands

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 10, 2023 10:55 am

Fixed the script. Strange there are not more errors ;)
Did you not update the script version ?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Sep 10, 2023 5:41 pm

No, since it was just one letter and it does not have any function difference/bug fix, I did not update the number. No need to upgrade.
 
markerasmus
just joined
Posts: 16
Joined: Tue Oct 17, 2023 12:19 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Oct 21, 2023 4:22 am

Thank you for putting this together I have it working using your guide - had a few little issues but nothing I could not work around
 
markerasmus
just joined
Posts: 16
Joined: Tue Oct 17, 2023 12:19 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Oct 21, 2023 9:55 pm

How can I get to see the Interface - but it show throughput but not total data.

How can I get to see total data used? per interface

Thanks again for the hard work - just wish there was more then 500mb daily limited I reached it seems trying to log firewall rules is a killer
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Oct 21, 2023 10:44 pm

Thanks for the feedback.

I will look at a way to calculate/show total traffic pr interface over a given periode.

Regarding the 500MB/day, there is a way around. You can request and developer license.
https://dev.splunk.com/enterprise/dev_license/

+
It give you 10GB/day
It gives you all Splunk function

-
It only works for 6 month, but you can re request a new license every 6 month.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Oct 22, 2023 3:57 pm

Added graph to show tx and rx pie chart so you can see total traffic (over selected periode) transfered pr interface.

Updated files in the gir repository.
.
traffic.png
You do not have the required permissions to view the files attached to this post.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Oct 22, 2023 5:28 pm

I see a smal bug that is not easy to fix. If interface counters has been reset or router rebooted within the time frame selected the counter will be wrong. This since it the counter increments all the time, except for reset and the the calculation between start and stop of a time frame will not show the correct gap.
 
markerasmus
just joined
Posts: 16
Joined: Tue Oct 17, 2023 12:19 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Oct 22, 2023 10:26 pm

Thank you so much thats awesome
Added graph to show tx and rx pie chart so you can see total traffic (over selected periode) transfered pr interface.

Updated files in the gir repository.
.
traffic.png
 
FlippinTurt
just joined
Posts: 15
Joined: Sun Aug 20, 2023 10:48 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Tue Nov 07, 2023 2:19 am

I am having an issue in ROS 7.11 with the 'Data_to_Splunk_using_Syslog' script, in that it doesn't output anything to the log, and of course doesn't get picked up by splunk
I can confirm there are log messages making it into splunk, and have tested other scripts which work in 7.11.

Wondering if you have any idea what might be breaking it in this version, or if I am missing something?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Nov 16, 2023 8:07 am

I am having an issue in ROS 7.11
Script is tested and working with all version up to 7.13beta2

Try to split up the script in parts add the { } around the part and cut / past to terminal to see if some break.
You need to include the corresponding enable line like :local SystemResource true
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Nov 16, 2023 8:10 am

Script updated to 5.3

Only a small addition to enable use of DNS for syslog. Set your DNS name so it resolve it to an IP within the script.
This helps out if router is on public IP and syslog server are on a dynamic IP
# Auto update syslog server. 5.3  (if not needed, put a # in front of next two lines)
:local mySyslog [resolve <your syslog dns name>]
/system/logging/action/set [find where name="logserver"] remote=$mySyslog
 
FlippinTurt
just joined
Posts: 15
Joined: Sun Aug 20, 2023 10:48 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Thu Nov 16, 2023 10:09 am

I am having an issue in ROS 7.11
Script is tested and working with all version up to 7.13beta2

Try to split up the script in parts add the { } around the part and cut / past to terminal to see if some break.
You need to include the corresponding enable line like :local SystemResource true
Cheers
Tested it out, and removed the wifi section and seems to work fine now (probably should have said I am using an ax router, so wifiwave2.) Tried changing the script but didn't get anything back so just removed it and now works perfect

Thanks for all your work on this!
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Nov 17, 2023 8:27 pm

My problem is that I do not have all MikroTik routers so can not test all hardware. Same its also hard to test all the overwhelming number of configuration option on the router.
 
FlippinTurt
just joined
Posts: 15
Joined: Sun Aug 20, 2023 10:48 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Nov 19, 2023 9:21 pm

My problem is that I do not have all MikroTik routers so can not test all hardware. Same its also hard to test all the overwhelming number of configuration option on the router.
Totally get that, especially now there are a few different types out there haha
I managed to get wifi working using the following, if you're interested (ap changed to ssid, signal-strength is now signal) :)
# Sends wireless client data to log server 
# ----------------------------------
:if ($Wireless) do={
	/interface wifiwave2 registration-table
	:foreach i in=[find] do={
		:log info message=".id=$i;ap=$([get $i ssid]);interface=$([get $i interface]);mac-address=$([get $i mac-address]);signal-strength=$([get $i signal]);tx-rate=$([get $i tx-rate]);uptime=$([get $i uptime]);script=wifi"
	}
}
 
blinderix
newbie
Posts: 48
Joined: Wed Apr 08, 2009 8:55 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Nov 19, 2023 9:31 pm

Thank you for you efforts!
Usind kid control to watch the traffic.
On the dashboard "Mikrotik Device Traffic" there is a column "client". When the host is in the same network as splunk server, there is shown the device name or hostname. But when the host is remote using VPN, the "client" cell is IP address. Is there any change to show the hostname of the device when the router is on a remote network? Or manually attach names to corresponding ip's?
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Nov 24, 2023 1:01 pm

The client name are just looked up using DNS lookup, so client on remote system will not show up. Not sure if there are any good solution for this. Maybe import all DNS from remote site.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sat Nov 25, 2023 8:54 am


Totally get that, especially now there are a few different types out there haha
Since there are new types, and script will fail if it does not exists, I need to test if the type of internet do exists.
Can you post a output of
/interface; :foreach i in=[find] do={:put [get $i type]}
It would also be interesting to see this output for the new wifi naming on 7.13 beta, so please post :)

Can you also do an output of:
/interface/wireless/registration-table print
I do not see the SSID field on standard wifi
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Sun Dec 17, 2023 11:54 pm

Seems that most stuff works in 7.13, but are some skeptical if all the wireless scrips works, since there are more than one menu to go inn too.
And since I do not have all types of model, it would be nice with some feedback if there are things that do not work.
 
FlippinTurt
just joined
Posts: 15
Joined: Sun Aug 20, 2023 10:48 pm

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Dec 18, 2023 12:22 am


Totally get that, especially now there are a few different types out there haha
Since there are new types, and script will fail if it does not exists, I need to test if the type of internet do exists.
Can you post a output of
/interface; :foreach i in=[find] do={:put [get $i type]}
It would also be interesting to see this output for the new wifi naming on 7.13 beta, so please post :)

Can you also do an output of:
/interface/wireless/registration-table print
I do not see the SSID field on standard wifi
Apologies, didn't get a notification for the reply.

7.12.1
 /interface; :foreach i in=[find] do={:put [get $i type]}
ether
ether
ether
ether
ether
wifi
wifi
wifi
bridge

wireless doesn't exist on ax (Possibly some other newer ones?) so had to use 'wifiwave2'
/interface/wifiwave2/registration-table> /interface/wifiwave2/registration-table print                                      
Flags: A - AUTHORIZED
Columns: INTERFACE, SSID, MAC-ADDRESS, UPTIME, SIGNAL
#   INTERFACE   SSID            MAC-ADDRESS        UPTIME      SIGNAL
0 A AX_5        SSID  11:11:11:11:11:11 3d9h20m36s  -50   
1 A AC_2.4_IoT  IoT_SSID         11:11:11:11:11:11 1d9h19m27s  -29   
2 A AC_2.4      GSSID     11:11:11:11:11:11  8m39s       -65 

7.13
/interface; :foreach i in=[find] do={:put [get $i type]}
ether
ether
ether
ether
ether
wifi
wifi
wifi
bridge
 
jult
Frequent Visitor
Frequent Visitor
Posts: 52
Joined: Sat Dec 26, 2020 1:16 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Dec 27, 2023 9:55 pm

Script updated to 5.3

Only a small addition to enable use of DNS for syslog. Set your DNS name so it resolve it to an IP within the script.
This helps out if router is on public IP and syslog server are on a dynamic IP
# Auto update syslog server. 5.3  (if not needed, put a # in front of next two lines)
:local mySyslog [resolve <your syslog dns name>]
/system/logging/action/set [find where name="logserver"] remote=$mySyslog
Just an FYI: This broke my setup (nothing would arrive at splunk..). The remote logging command was already using an IP as per your instructions under 2a, so this is generally not needed. (Had to indeed comment this out.)
Added graph to show tx and rx pie chart so you can see total traffic (over selected periode) transfered pr interface.
I'm still missing an easy way to see *all* traffic I+O per IP per x time *only* to/from the WAN interface, so the actual internet traffic usage per device.
Guess I really need netflow for that?
Been trying to set that up, but the latest Splunk version and apps are entirely different from what is instructed above, so I'm not getting anywhere thus far with netflow data showing up. Splunk seems waaaay overkill for what I would want to do/see.

I'm surprised this (internet data/LAN-IP) is not even logged by mikrotik itself. This is what most users will want to know. I have several switches connected to the router that has just one port to/from the internet modem, so interface traffic as such is not useful, as internal traffic is of no value for what I would want to know. Traffic to/from the web is what counts (and has a cost).
Last edited by jult on Thu Dec 28, 2023 12:50 pm, edited 1 time in total.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Dec 29, 2023 2:57 pm

Just an FYI: This broke my setup (nothing would arrive at splunk..). The remote logging command was already using an IP as per your instructions under 2a, so this is generally not needed. (Had to indeed comment this out.)
I think you did forget to set the DNS server. If not se, it will not resolve IP.
:local mySyslog [resolve <your syslog dns name>]
/system/logging/action/set [find where name="logserver"] remote=$mySyslog
Eks your Syslog server = goolge.com it should be:
:local mySyslog [resolve google.com]
/system/logging/action/set [find where name="logserver"] remote=$mySyslog
In next version I will disable this option as default and also make it more clear that you need to add your DNS to the syslog server.


Regarding traffic logging. This is not some easy thing to do and will require lots of space and resources to log everything. Netflow is one way to do it, but that require another setup than just syslog. Also with the new private mode on lots of hardware like iPhone that takes a new IP every time and do not give information to the DHCP server with Device IP, its hard to get any valid information.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Dec 29, 2023 3:07 pm

Since I have several devices with dynamic public IP, I think it would be a better solution to use device name only or together with serial number, instead of IP that do changes to identify the device.

This will require a mayor rewrite of lots of the code so it need to bee thought trough to not break other stuff.
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Fri Dec 29, 2023 3:13 pm

Script updated to 5.4 that only set the auto syslog server (dns) to not as default.
I do reformende that you do use it if your Syslog server do not have static IP
 
ellipsis
just joined
Posts: 1
Joined: Wed Dec 20, 2023 10:47 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Mon Jan 01, 2024 9:15 am

I'm running Splunk on a Synology too, but as a VM under Ubuntu Linux, not containerized.
Works OK in general, had 1 or 2 occasions where the 4GB assigned memory fell short and things fell apart ;-)

>> After a while the logging to splunk stops ...

Splunk generates a ton of logging messages that might give you an indication why something "stops" working. Did you check any of these ?
(with a container, you'll have to open a shell I guess)

/opt/splunk/var/log/splunk

Are you not exceeding the 500Mbytes daily limit ??
Top menu "Settings" then "Licensing" (under the "System" section)
Hi jvanhambelgium
Did you find anything could help resolve this error?
 
jult
Frequent Visitor
Frequent Visitor
Posts: 52
Joined: Sat Dec 26, 2020 1:16 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 03, 2024 1:47 pm

Regarding traffic logging. This is not some easy thing to do and will require lots of space and resources to log everything. Netflow is one way to do it, but that require another setup than just syslog.
Yes, for the total traffic I've used LibreNMS for years, it does that pretty accurately using their Traffic > Billing plugin. With that at least you get very accurate internet data traffic usage per day and can get a monthly usage average per day, and alerting for when you tend to go over that average and such. Works pretty well, just using SNMP from the mikrotik core router.

Unfortunately this does not specify which LAN IP was responsible for which traffic usage. You can however track down which interface (physical) was using the data at what time using libreNMS and SNMP output from the mikrotik, in many cases that info would be sufficient to research the culprit, except for when Wireless APs are on those ports and the size of your network is beyond a simple home network scope..
Also with the new private mode on lots of hardware like iPhone that takes a new IP every time and do not give information to the DHCP server with Device IP, its hard to get any valid information.
Yes, I have simply demanded our users to switch off private mode when they want to use our wifi, and I did that using some DHCP trickery and some shell scripts. They can't go online when they try and use private mode again. Yes, it's a BOFH method, but it works if your userbase is not that large ;)
 
User avatar
Jotne
Forum Guru
Forum Guru
Topic Author
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 03, 2024 2:41 pm

, and I did that using some DHCP trickery and some shell scripts. They can't go online when they try and use private mode again.
This will just be a temporary solution. What if they deny? I am not sure if you can turn this of pr network. So if you then take your device to a public net, you are at more risk.

Step 1. Add function for private mode.
Step 2. Add private mode as standard. This is now true for iPhone + some other. Will be more later. Android/Windows++
Step 3. Remove possibility to disable private mode.

At step 3, you can not force the user to turn off private mode.
 
jult
Frequent Visitor
Frequent Visitor
Posts: 52
Joined: Sat Dec 26, 2020 1:16 am

Re: 📌 Tool: Using Splunk to analyse MikroTik logs 3.9 (Graphing everything) 💾 🛠 💻 📊

Wed Jan 03, 2024 4:25 pm

, and I did that using some DHCP trickery and some shell scripts. They can't go online when they try and use private mode again.
This will just be a temporary solution. What if they deny? I am not sure if you can turn this of pr network. So if you then take your device to a public net, you are at more risk.

Step 1. Add function for private mode.
Step 2. Add private mode as standard. This is now true for iPhone + some other. Will be more later. Android/Windows++
Step 3. Remove possibility to disable private mode.

At step 3, you can not force the user to turn off private mode.
All those new devices (Android too) have the option to choose private mode off *per WiFi AP*. So you let them default to private mode if they wish, just not on your network.

Who is online

Users browsing this forum: No registered users and 7 guests