I would like to transport the data shown under Firewall > Connections (mentioned here: https://wiki.mikrotik.com/wiki/Manual:I ... n_tracking) to another computer for further analysis and monitoring. Here's what I know based on my research on this topic:
- I've watched some tutorials on YouTube that show how to log to remote IPs and stand up Syslog servers on those computers to consume Mikrotik logs. It gives me either too much data or part of the data I'm looking for, but not exactly what I need. (e.g. this video https://www.youtube.com/watch?v=FERuk-gWxKY)
- Firewall > Connections show everything what I want. I just want to capture that and send it to a remote computer where it'll be consumed and analyzed.
- I'd like to avoid adding any unnecessary overhead to the processing, so that it doesn't spike up my routers CPU.
Here's the design of the log analysis I'm thinking about:
I'd like to send the "Firewall > Connections" data to a remote computer where they'll be consumed by an Apache Kafka. I'll then have a script (Python) that'll consume data off of Kafka, do some analysis and then persist in a time series database of some kind. Another script will plot it in a web interface. Welcome ideas if someone have already done this type of work before.