This thread is deprecated use new thread:
Version 3.3 13.08.2021
Using Splunk to monitor and graph various data from our MikroTik Routers is a nice and free way to help you showing what is going on in your network.
Splunk is free to use for logging up to 500MB pr day.
NB logging large amount of Accouning, DNS or firewall rules quickly eats up license, so I do recommend to turn off Accouning/DNS logging to start with.
It can be used to monitor multiple devices. No ports needs to be opened (like with SNMP monitoring). All data are sent from the device to the Splunk monitor. Devices could be all around the world.
PS:
Traffic monitoring does not work correctly while fast track is enabled (and its removed in v7.x of RouterOS). Turn it off and you may loose throughput, so its something you should think about when using this type of monitoring. How to disable it: https://www.youtube.com/watch?v=6LaqhDm6PHI
What's new
A lot have changed since previous version. So much that its better to replace all MikroTik files in the Splunk server instead of trying to update files.
Before data was collected in three ways.
- Syslog
- SNMP
- Scripts (remote)
Problem with SNMP is that you have to add duplicate configuration in Syslog to handle each box you monitor.
Problem with script is that you need to setup a job on the Splunk server that SSH inn to each box to get data.
The script tools did not log host IP, so you did not now from what box data was coming from.
SNMP and Scipts has been removed and change inn to local script on the Router that sends outs needed data to the Splunk using Syslog.
This way all data get correctly marked and you know for what Router you gets data from.
latest changes
3.3 (13.08.2021)
New KV store (used to store info about Devices and DHCP info
Changed from table to fields in base search for all dashboard to speed up searches.
Added more DNS extraction
Updated script to 4.4 (Added firmware info)
Added frimware info "MikroTik Device List"
Fixed lookup "MikroTik Wifi strength"
Added new DoH dashboard "MikroTik DoH information"
Change table format "MikroTik Interface Changes"
Fixed typo in eventtype "MikroTik VPN Connections"
Added user local ip "MikroTik VPN Connections"
Fixed missing search in 2b, duplicate entry in user-name, added 1b login info "MikroTik VPN Connections"
Fixed to not show false logged in user "MikroTik VPN Connections"
Fixed html formatting in "Mikrotik DHCP to Static"
Fixed missing City in "Mikrotik Firewall Rules"
Moved "Mikrotik uPnP" to Connections menu
Added table view to "Mikrotik uPnP"
Installation
1) On your PC Works on Windows and Linux, but use Linux (clearly the best choice)
1a) Download and install Splunk (Windows or Linux(Ubuntu recommended))
PS you need an account to download. It's free to create.
https://www.splunk.com/en_us/download/splunk-enterprise.html
PS you need to create an account to download the file. Free to download and use (up to 500MB/day)
PS remember to set timezone on Windwows/Linux, or else logging time will be wrong.
1b) PS: To install Splunk as a non root user, recomended. (needs an external syslog reciever)
Splunk setup:
https://forum.mikrotik.com/viewtopic.php?p=677233#p677233
rsyslog setup
https://forum.mikrotik.com/viewtopic.php?p=677233#p793342
Splunk can fine run as root user, but not recommended.
1c) Change to free license group. Very important to do before 30 day of use. !!!!!!!!!!!!!!!!!!!!
Web gui:
1d) Settings->licensing->Change license group->Free licnse->Save
1e) Open Windows Firewall for UDP on Windows (On linux its not blocked)
Web gui:
Start->type "adv"->Select:Widows Firewall with Advanced Security->Sect Inbound rules->Right Click "Inbound Rules">New Rule-Port-Next->UDP->Specific local ports->514->Next->Next->Next->Name "syslog"
1f) Allow UDP 514 (syslog) (if running Splunk as non ROOT user or like to use external syslog reciever, see 1b for non-root)
Web gui:
Setting->Datainputs->Add new (behind the UDP)->Port 514->Next->Sourcetype type syslog and select syslog->Next-Submit
1g) Download the Splunk spl file:
MikroTik3.3.rar (62.3 KB)
1h) Extract the spl file
From Start page in Splunk, click the gear behind Apps or
from top meny click Apps->Manage Apps
Then select Install app from file and select the spl file
1i) A restart of Splunk may be needed.
Web gui:
Settings->Server controls->Restart Splunk
1j) Upgrade form previous version.
Since no files are renamed, just data added, you should be fine by just replacing current files.
You can delete the MikroTik folder if you like. No logged data will be deleted.
If you have custom dashboards, menus, saved search (reports) etc, you need to merge the configuration files.
2) On Your MikroTik Router
Before you setup logging, you should make an unique identifier of your route. Important if you have more than one router to monitor.
/system identity set name=Router-London-22
2a) Syslog
You need to make your Router able to send Syslog messages.
Web gui:
System->Logging->Action->Add New->Name (your server name)->Type:Remote->Remote Address:ip your syslog->Ok
Cli
/system logging action add name=logserver target=remote remote=192.168.1.50 remote-port=514
PS Do NOT select BSD Syslog. It will mess up the logging format.
2b) Then select what modules to log.
I do suggest that you send all DHCP logs including debug and all other logs that are not debug.
It is very important to name the prefix like this "MikroTik" and not "mikrotik" or some other.
Splunk uses the MikroTik prefix to find out what type of syslog data that is coming to it.
Uppercase T and uppercase M, rest are lowercase
Web gui:
System->Logging->Rules->Add new->Topics:dhcp->Prefix:MikroTik->action:your syslog server->Ok
System->Logging->Rules->Add new->Topics:!debug,!packet,!snmp->Prefix:MikroTik->action:your syslog server->Ok
Cli:
/system logging add action=logserver prefix=MikroTik topics=dhcp
/system logging add action=logserver prefix=MikroTik topics=!debug,!packet,!snmp
/system logging add action=logserver prefix=MikroTik topics=hotspot
To not fill up internal logs with firwall logs etc, remove firewall from memory logs
/system logging set [find topics="info"] topics=info,!firewall
PS Hotspot is not needed if you do not use it.
2c) Select what rules to log
NB Do not use more than 20 charters, or else it start to clip other part of the log!!!!!!!!!!!
To log the Firewall and Nat rules, you need to turn on logging and add Log Prefix (under action).
Do not log more than needed. Logging rules like defconf: accept established,related rules will flod your log,
Below is a sample on how to name the log rules. You do not need to follow this rule, but it makes it more uniform.
Rule name logging
==================
Format:
x_y_z
z=name/info
Example
-------
Filter Rule Forard allow HTTP
FF_A_Http
Filter Route Input Drop ICMP
FI_D_Icmp
Nat HTTP
ND_DE_Http
Mangle Mark HTTP packets
MF_MP_Http
Filter Rule
------------------
x=
FF Filter Forward
FI Filter Input
FO Filter Output
FX Filter Custom list
y=
A Accept
AD Add to dst address list
AS Add to src address list
D Dropp
F Fast track
J Jump
L Log
P Passthrough
RJ Reject
RT Return
T Tarpit
Nat Rule
------------------
x=
ND Dest nat
NS Source nat
y=
A Accept
AD Add to dst address list
AS Add to src address list
DE Dst-nat
J Jump
L Log
M Masquerade
N Netmap
P Passthrough
RE Redirect
RT Return
SA same
S Src-nat
Raw
------------------
x=
RP Filter Raw Prerouting
RO Filter Raw Output
y=
A Accept
AD Add to dst address list
AS Add to src address list
F Fast track
D Dropp
J Jump
L Log
N No track
P Passthrough
RT Return
Mangle
------------------
x=
MF Mangle Forward
MI Mangle Input
MP Mangle Postrouing
MR Mangle Prerouting
y=
A Accept
AD Add to address list
AS Add to dst address list
CD Change DSCP
CM Change MSS
CT Change TTL
CL Clear DF
F Fast track
J Jump
L Log
MC Marc connection
MP Mark packets
MR Mark routing
P Passthrough
RT Return
RO Route
S Set proirity
SP Sniff PC
ST Sniff TZSP
SI Strip IPv4 options
2d) You should at least log this rule "defconf: drop all not coming from LAN" with this prefix: FI_D_port-test
Web gui:
IP->Firewall->selec:defconf: drop all not coming from LAN->Log:v->Log Prefix:FI_D_port-test
This will populate the MikroTik Live attack view.
2e) Accounting
To get accounting data, you need to enable it on the MikroTik router. (MikroTik Traffic dashboard)
Web gui:
IP-> Accounting -> Enable Accounting -> mark Threshould:2560 OK
Cli:
/ip accounting set enabled=yes threshold=2560
2f) Main Collector Script
To get all the other data like Traffic accounting, uPnP, System health, System resources and DHCP pool information you need this script on the MikroTik. Create this script with name Data_to_Splunk_using_Syslog and cut and past code using gui.
In the top of the script, you can set a module to true/false. If you do not use wifi, set :local Wireless false
# Collect information from Mikrotik RouterOS
# Jotne 2021
:log info message="script=version ver=4.7"
# ----------------------------------
# What data to collect. Set to false to skip the section
# ----------------------------------
:local SystemResource true
:local SystemInformation true
:local SystemHealth true
:local TrafficData true
:local AccuntData true
:local uPnP true
:local Wireless true
:local AddressLists true
:local DHCP true
:local Neighbor true
:local InterfaceData true
:local CmdHistory true
:local CAPsMANN false
# Collect system resource
# ----------------------------------
:if ($SystemResource) do={
/system resource
:local cpuload [get cpu-load]
:local freemem ([get free-memory]/1048576)
:local totmem ([get total-memory]/1048576)
:local freehddspace ([get free-hdd-space]/1048576)
:local totalhddspace ([get total-hdd-space]/1048576)
:local up [get uptime]
:local sector [get write-sect-total]
:log info message="script=resource free_memory=$freemem MB total_memory=$totmem MB free_hdd_space=$freehddspace MB total_hdd_space=$totalhddspace MB cpu_load=$cpuload uptime=$up write-sect-total=$sector"
}
# Make some part only run every hours
# ----------------------------------
:global Hour
:local run false
:local hour [:pick [/system clock get time] 0 2]
:if ($Hour != $hour) do={
:global Hour $hour
:set run true
}
# Get NTP status
# ----------------------------------
:local ntpstatus ""
:if ([:len [/system package find where !disabled and name=ntp]] > 0 or [:tonum [:pick [/system resource get version] 0 1]] > 6) do={
:set ntpstatus [/system ntp client get status]
} else={
:if ([:typeof [/system ntp client get last-update-from]] = "nil") do={
:set ntpstatus "using-local-clock"
} else={
:set ntpstatus "synchronized"
}
}
:log info message="script=ntp status=$ntpstatus"
# Get traffic data (accounting data)
# ----------------------------------
:if ([:tonum [:pick [/system resource get version] 0 1]] < 7 and $AccuntData) do={
# Test if fasttrack is enabled and give warning
:if ([/ip firewall filter find where (action=fasttrack-connection && !disabled)] != "") do={
:log info message=("script=traffic,fasttrack=1")
} else={
:log info message=("script=traffic,fasttrack=0")
}
# Test if accounting is enabled and if yes, get data
:if ([/ip accounting get enabled]=yes) do={
/ip accounting snapshot take
# Send data to loggin server
:foreach logline in=[/ip accounting snapshot find] do={
:local output "$[/ip accounting snapshot print as-value from=$logline]"
:set ( "$output"->"script" ) "traffic"
:log info message="$output"
}
}
}
# Get interface traffic data for all interface
# ----------------------------------
:if ($TrafficData) do={
:foreach id in=[/interface find] do={
:local output "$[/interface print stats as-value where .id=$id]"
:set ( "$output"->"script" ) "if_traffic"
:log info message="$output"
}
}
# Finding dynmaic lines used in uPnP
# ----------------------------------
:if ($uPnP) do={
:foreach logline in=[/ip firewall nat find where dynamic=yes and comment~"^upnp "] do={
:local output "$[/ip firewall nat print as-value from=$logline]"
:set ( "$output"->"script" ) "upnp"
:log info message="$output"
}
}
# Collect system information
# ----------------------------------
:local model na
:local serial na
:local ffirmware na
:local cfirmware na
:local ufirmware na
:if ($SystemInformation and $run) do={
:local version ([/system resource get version])
:local board ([/system resource get board-name])
:if ($board!="CHR") do={
/system routerboard
:set model ([get model])
:set serial ([get serial-number])
:set ffirmware ([get factory-firmware])
:set cfirmware ([get current-firmware])
:set ufirmware ([get upgrade-firmware])
}
:local identity ([/system identity get name])
:log info message="script=sysinfo version=\"$version\" board-name=\"$board\" model=\"$model\" serial=$serial identity=\"$identity\" factory-firmware=\"$ffirmware\" current-firmware=\"$cfirmware\" upgrade-firmware=\"$ufirmware\""
}
# Collect system health
# ----------------------------------
:if ($SystemHealth) do={
:if (!([/system health get]~"(state=disabled|^\$)")) do={
:local voltage ([/system health get voltage]/10)
:local temperature ([/system health get temperature])
:log info message="script=health voltage=$voltage V temperature=$temperature C"
}
}
# Sends wireless client data to log server
# ----------------------------------
:if ($Wireless && [:len [/int find where type=wlan]]>0) do={
/interface wireless registration-table
:foreach i in=[find] do={
:log info message=".id=$i;ap=$([get $i ap]);interface=$([get $i interface]);mac-address=$([get $i mac-address]);signal-strength=$([get $i signal-strength]);tx-rate=$([get $i tx-rate]);uptime=$([get $i uptime]);script=wifi"
}
}
# Count IP in address-lists
#----------------------------------
:if ($AddressLists) do={
:local array [ :toarray "" ]
:local addrcntdyn [:toarray ""]
:local addrcntstat [:toarray ""]
:local test
:foreach id in=[/ip firewall address-list find] do={
:local rec [/ip firewall address-list get $id]
:local listname ($rec->"list")
:local listdynamic ($rec->"dynamic")
:if (!($array ~ $listname)) do={ :set array ($array , $listname) }
:if ($listdynamic = true) do={
:set ($addrcntdyn->$listname) ($addrcntdyn->$listname+1)
} else={
:set ($addrcntstat->$listname) ($addrcntstat->$listname+1)}
}
:foreach k in=$array do={
:log info message=("script=address_lists list=$k dynamic=".(($addrcntdyn->$k)+0)." static=".(($addrcntstat->$k)+0))}
}
# Get MNDP (CDP) Neighbors
# ----------------------------------
:if ($Neighbor and $run) do={
:foreach neighborID in=[/ip neighbor find] do={
:local nb [/ip neighbor get $neighborID]
:local id [:pick ("$nb"->".id") 1 99]
:foreach key,value in=$nb do={
:local newline [:find $value "\n"]
:if ([$newline]>0) do={
:set value [:pick $value 0 $newline]
}
:log info message="script=neighbor nid=$id $key=\"$value\""
}
}
}
# Collect DHCP Pool information
# ----------------------------------
:if ($DHCP and $run) do={
/ip pool {
:local poolname
:local pooladdresses
:local poolused
:local minaddress
:local maxaddress
:local findindex
# Iterate through IP Pools
:foreach pool in=[find] do={
:set poolname [get $pool name]
:set pooladdresses 0
:set poolused 0
# Iterate through current pool's IP ranges
:foreach range in=[:toarray [get $pool range]] do={
# Get min and max addresses
:set findindex [:find [:tostr $range] "-"]
:if ([:len $findindex] > 0) do={
:set minaddress [:pick [:tostr $range] 0 $findindex]
:set maxaddress [:pick [:tostr $range] ($findindex + 1) [:len [:tostr $range]]]
} else={
:set minaddress [:tostr $range]
:set maxaddress [:tostr $range]
}
# Calculate number of ip in one range
:set pooladdresses ($maxaddress - $minaddress)
# /foreach range
}
# Test if pools is used in DHCP or VPN and show leases used
:local dname [/ip dhcp-server find where address-pool=$poolname]
:if ([:len $dname] = 0) do={
# No DHCP server found, assume VPN
:set poolused [:len [used find pool=[:tostr $poolname]]]
} else={
# DHCP server found, count leases
:local dname [/ip dhcp-server get [find where address-pool=$poolname] name]
:set poolused [:len [/ip dhcp-server lease find where server=$dname]]}
# Send data
:log info message=("script=pool pool=$poolname used=$poolused total=$pooladdresses")
# /foreach pool
}
# /ip pool
}
}
# Get detailed command history RouterOS >= v7
# ----------------------------------
:if ([:tonum [:pick [/system resource get version] 0 1]] > 6 and $CmdHistory) do={
:global cmd
:local f 0
:foreach i in=[/system history find] do={
:if ($i = $cmd) do={ :set f 1 }
:if ($f != 1) do={
:log info message="StartCMD"
:log info message=[/system history get $i]
:log info message="EndCMD"
}
}
:global cmd [:pick [/system history find] 0]
}
# Test if CAPsMANN is installed, if yes, run capsmann script.
# ----------------------------------
:if ( ([:len [/interface find where type="cap"]] > 0) and $CAPsMANN) do={ /system script run capsman }
# End Script
If you do use CAPsMan, create a script with name capsman and add this code.
:local capsregistered ([/caps-man registration-table print count-only])
/caps-man interface
:local name
:local mac
# ignore all master interfaces
:foreach i in=[find where master-interface="none"] do={
:set name [get $i name]
:set mac [get $i radio-mac]
:local counter ([/caps-man registration-table print count-only where interface=$name])
:log info message="script=caps-man name=$name counter=$counter"
}
:log info message="script=caps-man capsregistered=$capsregistered"
2g) Then schedule the script to run every 5 minutes:
/system scheduler
add interval=5m name="Data to Splunk" on-event=Data_to_Splunk_using_Syslog
2h) Debugging
- See if any data are comming inn to splunk at all.
index=*
- Test if data has correct tag "MikroTik" (Capital M & T)
index=* | table _time sourcetype _raw
You should see corect time, sourcetype should show "mikrotik" and _raw should show data
- See that _raw does contain only data and not time and other info
index=* | table _raw
dns MikroTik: done query: #640030 adservice.google.no 216.58.211.2
dhcp,debug,packet MikroTik: Client-Id = 01-6C-3B-6B-88-34-3F
firewall,info MikroTik: FI_D_port-test input: in:ether1 out:(unknown 0), src-mac 00:05:00:01:00:01, proto TCP (SYN), 47.118.40.92:52503->92.220.205.91:2376, len 40
If you see date, format of packet from Mikrotik has BDS set or Rsyslog is not setup correctly.
- Read trough all steps on how to install if some does not work
- Still problems: ask here
Example screen shots:
