Which self hosted log server / analyzer do you recommend?

I am in the process of looking for a solution to capture all logs from mikrotiks (and other servers).
I would like it to be self hosted and possibly open source, with a web interface to manage data and if possible to analyze incoming data from mikrotiks and trigger notifications.

Any personal experience shared (pros and cons) would be appreciated!

  • ELK-stack, preferably with Grafana on top.
  • Graylog
  • NetXMS has the availability for parsing syslog and creating events/notifications, but not usable for high volume analytics.

http://forum.mikrotik.com/t/tool-using-splunk-to-analyse-mikrotik-logs-3-3-graphing-everything/121810/1

+1

Triggering notifications is not possible with the free version of Splunk.

It is not a self hosted open source product though.
I need to use their cloud where I am limited with certain quota

their cloud ? no why would do you think that?
The Splunk Enterprise (500MB/day limit) runs on your own server, nothing to do with their cloud.
I have it running on a Linux Ubuntu VM on my NAS.
Performance is not fantastic (hey, it’s a NAS…) but more then fast enough

You are right that it is not open-source.

Otherwise some ELK-stack (Elasticearch / Logstash / Kibana) would be an alternative.

splunk runs here on my syno 920+ in docker …
works for me …

and out of your experience the 500mb/day limit in how many devices sending data would that translate to? Roughly

Depends on your level of logging offcourse.
In my case, I have flows coming in from 1 (home) Mikrotik and through the excellent script from Jotne (and I’m logging almost all firewall rules, not only drops) and I also ingress Netflow v5 data
from the same Mikrotik.
At the moment I am at :

Licensed daily volume 500 MB
Volume used today 17 MB (3.428% of quota)

In terms of events, I have about

mikrotik => 55,895,439 entries in de db
stream:netflow => 21,824,448 entries in the db

I have 6 MT devices reporting to splunk …
2 gateways
and 4 AP
on 2 locations connected to 100Mbit fiber internet
management with ipsec tunnel …
all using jotne’s fine scripts

about 50 wifi clients in both networks

today in 16 hours time I have used about 6% of the license