Reading NetFlow Data with Python

RouterOS General
1

I’m trying to analyze the raw NetFlow data coming from a Mikrotik using Python.
The Mikrotik is configured to send IPFIX data to my machine.
It looks like one of the few available pieces of code that can analyze IPFIX data is found in PyPi: https://pypi.org/project/ipfix/#description
I’m having trouble using it though.

I’m trying something like this:

import socket
import ipfix.message

sock = socket.socket(socket.AF_INET, socket.SOCK_DRAM)
sock.bind(('0.0.0.0', 1234))
msg = ipfix.message.MessageBuffer()

while True:
    data, addr = sock.recvfrom(1024)
    msg.from_bytes(data)
    print(msg)

I think there’s probably more to printing “msg” than just “print(msg)”, but it won’t even get that far. It crashes on the “msg.from_bytes(data)” line, saying: “ipfix.template.IpfixDecodeError: (‘Illegal message length7’,)”.

Anyone either know how to use the ipfix python library, or any other, easier way to read NetFlow data coming from a Mikrotik?

2

Take a look at https://code.google.com/archive/p/flowd/

You have a python example code and it works nice!

3

Thanks.

I haven’t tried the Python part of it yet, but the collector seems to be working well with Mikrotik’s Netflow V9. I should be able to use that.

Netflow V9 is not super efficient though, as it sends me a ton of fields I don’t care about. IPFIX is nicer, as you can specify the fields you want. The flowd project had IPFIX on the ToDo list, but the project seems to have been abandoned in 2010, before implementing the IPFIX support. Before I get too deep into integrating flowd into my project, are there any IPFIX collectors anyone knows about?