I am setting up a traffic analysis system for my network and have configured Traffic Flow (IPFIX) forwarding for interface ether2 (my WAN interface) of my RB5009 router. When I analyze the flow records, the field netflow.network.direction shows the value 'unknown'.
Any idea why that is the case? Anything I need to setup in order for the direction field to show if the traffic is inbound or outbound?
Re: IPFIX direction field unknown
0 views on my post so far
posting this comment in hopes of increased reach
posting this comment in hopes of increased reach
Re: IPFIX direction field unknown
Ok, I have to admit that I am not an expert in IPFIX, but have used various netflow versions for many years, so...
IPFIX (and Netflow v9) are based on the idea that they do not have static data models like Netflow 5, but transmit data based on templates that are sent with the flow data. The sending party can choose whatever fields they want to send and implement that in their code as Information Elements. Thes templates show the Information Elements that are being actually being sent.
From a packet capture of an actual IPFIX stream you can see the 2 templates that are sent by the Mikrotik device
and
None of these seem to match your "netflow.network.direction" value.
Also if you look in the Mikrtotik IPFIX documentation, there is not a matching field.
It would seem that Mikrotik just doesnt implement the field you are looking for. It may be too expensive for them to calculate on the fly or something. There are fields such as "IP_NEXT_HOP" that should let you infer this.
IPFIX (and Netflow v9) are based on the idea that they do not have static data models like Netflow 5, but transmit data based on templates that are sent with the flow data. The sending party can choose whatever fields they want to send and implement that in their code as Information Elements. Thes templates show the Information Elements that are being actually being sent.
From a packet capture of an actual IPFIX stream you can see the 2 templates that are sent by the Mikrotik device
Code: Select all
Template (Id = 258, Count = 37)
Template Id: 258
Field Count: 37
Field (1/37): IP_PROTOCOL_VERSION
Field (2/37): FIRST_SWITCHED
Field (3/37): LAST_SWITCHED
Field (4/37): systemInitTimeMilliseconds
Field (5/37): PKTS
Field (6/37): BYTES
Field (7/37): L4_SRC_PORT
Field (8/37): L4_DST_PORT
Field (9/37): INPUT_SNMP
Field (10/37): OUTPUT_SNMP
Field (11/37): PROTOCOL
Field (12/37): IP_TOS
Field (13/37): TCP_FLAGS
Field (14/37): DST_MAC
Field (15/37): DESTINATION_MAC
Field (16/37): SOURCE_MAC
Field (17/37): SRC_MAC
Field (18/37): IP_SRC_ADDR
Field (19/37): IP_DST_ADDR
Field (20/37): IP_NEXT_HOP
Field (21/37): SRC_MASK
Field (22/37): DST_MASK
Field (23/37): IP_TTL
Field (24/37): IS_MULTICAST
Field (25/37): IP_HEADER_LEN
Field (26/37): ipTotalLength
Field (27/37): UDP_LENGTH
Field (28/37): TCP_SEQ_NUM
Field (29/37): TCP_ACK_NUM
Field (30/37): TCP_WINDOW_SIZE
Field (31/37): IGMP_TYPE
Field (32/37): ICMP_IPv4_TYPE
Field (33/37): ICMP_IPv4_CODE
Field (34/37): postNATSourceIPv4Address
Field (35/37): postNATDestinationIPv4Address
Field (36/37): postNAPTSourceTransportPort
Field (37/37): postNAPTDestinationTransportPort
Code: Select all
Template (Id = 259, Count = 34)
Template Id: 259
Field Count: 34
Field (1/34): IP_PROTOCOL_VERSION
Field (2/34): FIRST_SWITCHED
Field (3/34): LAST_SWITCHED
Field (4/34): systemInitTimeMilliseconds
Field (5/34): PKTS
Field (6/34): BYTES
Field (7/34): L4_SRC_PORT
Field (8/34): L4_DST_PORT
Field (9/34): INPUT_SNMP
Field (10/34): OUTPUT_SNMP
Field (11/34): PROTOCOL
Field (12/34): IP_TOS
Field (13/34): TCP_FLAGS
Field (14/34): DST_MAC
Field (15/34): DESTINATION_MAC
Field (16/34): SOURCE_MAC
Field (17/34): SRC_MAC
Field (18/34): IPV6_SRC_ADDR
Field (19/34): IPV6_DST_ADDR
Field (20/34): IPV6_NEXT_HOP
Field (21/34): IPV6_SRC_MASK
Field (22/34): IPV6_DST_MASK
Field (23/34): IP_TTL
Field (24/34): IS_MULTICAST
Field (25/34): IP_HEADER_LEN
Field (26/34): ipTotalLength
Field (27/34): UDP_LENGTH
Field (28/34): TCP_SEQ_NUM
Field (29/34): TCP_ACK_NUM
Field (30/34): TCP_WINDOW_SIZE
Field (31/34): IGMP_TYPE
Field (32/34): ICMP_IPv6_TYPE
Field (33/34): ICMP_IPv6_CODE
Field (34/34): FLOW_LABEL
None of these seem to match your "netflow.network.direction" value.
Also if you look in the Mikrtotik IPFIX documentation, there is not a matching field.
It would seem that Mikrotik just doesnt implement the field you are looking for. It may be too expensive for them to calculate on the fly or something. There are fields such as "IP_NEXT_HOP" that should let you infer this.
Re: IPFIX direction field unknown
Going down the rabbit hole a little further...
I'm going to guess the value "netflow.network.direction" you are hoping for is "flowDirection" from the original IPFIX RFC
Sounds interesting, right? Who wouldn't want to know if their flow is in or out?
Well, the problem is that this is a property of a unidirectional flow.
Let's take the example of a simple http connection between a Host1 and a Host2. This will create 2 unidirectional flows, one from Host1 to Host2 egress and one from Host2 to Host1 ingress. What you probably want to know is which host is the Server and which is the Client so that you can assign a bidirectional flow direction. To do this you need to apply more data than you have here. You could naively assume that any flow "egress" from your WAN port is towards a server and initiated by the client (if you have effective and restrictive firewall filters, I hope), or you could guess that any destination port < 1024 is toward the server etc, etc.
The basic point is:
1) IPFIX sends you info on unidirectional flow information
2) It turns out the "direction" of a unidirectional flow is not very interesting
2) A lot of information we would like depends on creating bidirectional flow info (top talkers etc)
3) Unidirectional flow "direction" doesn't help much to create bidirectional flow Client - Server "direction"
4) Creating bidirectional flows is not trivial and usually uses a lot of data outside of IPFIX, like assumptions about your network setup, and uses a bunch of code on whatever collector you use.
Also, you can derive flow "direction" from other IPFIX data
P.S. if youre interested in interface stats or direction through the router they are usually based on INPUT_SNMP and OUTPUT_SNMP
I'm going to guess the value "netflow.network.direction" you are hoping for is "flowDirection" from the original IPFIX RFC
Code: Select all
5.11.6. flowDirection
Description:
The direction of the Flow observed at the Observation Point.
There are only two values defined.
0x00: ingress flow
0x01: egress flow
Abstract Data Type: unsigned8
Data Type Semantics: identifier
ElementId: 61
Status: current
Well, the problem is that this is a property of a unidirectional flow.
Let's take the example of a simple http connection between a Host1 and a Host2. This will create 2 unidirectional flows, one from Host1 to Host2 egress and one from Host2 to Host1 ingress. What you probably want to know is which host is the Server and which is the Client so that you can assign a bidirectional flow direction. To do this you need to apply more data than you have here. You could naively assume that any flow "egress" from your WAN port is towards a server and initiated by the client (if you have effective and restrictive firewall filters, I hope), or you could guess that any destination port < 1024 is toward the server etc, etc.
The basic point is:
1) IPFIX sends you info on unidirectional flow information
2) It turns out the "direction" of a unidirectional flow is not very interesting
2) A lot of information we would like depends on creating bidirectional flow info (top talkers etc)
3) Unidirectional flow "direction" doesn't help much to create bidirectional flow Client - Server "direction"
4) Creating bidirectional flows is not trivial and usually uses a lot of data outside of IPFIX, like assumptions about your network setup, and uses a bunch of code on whatever collector you use.
Also, you can derive flow "direction" from other IPFIX data
P.S. if youre interested in interface stats or direction through the router they are usually based on INPUT_SNMP and OUTPUT_SNMP