Community discussions

MikroTik App
 
User avatar
remontti
just joined
Topic Author
Posts: 10
Joined: Mon Aug 31, 2015 11:27 pm

Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Tue Sep 12, 2023 10:49 pm

I would like to discuss the implementation of improvements to MikroTik's IP Traffic Flow (NetStream), specifically the addition of "src as" (source AS) and "dst as" (destination AS) information in flow records. These details are essential for understanding and optimizing network traffic and are common features in other network monitoring solutions, as demonstrated below.

Comparison with Other Manufacturers:

To illustrate the importance of this information, let's compare MikroTik's IP Traffic Flow with examples from other manufacturers:

Other Manufacturers:
src as       =             12345
dst as       =             54321
MikroTik - Exemplo Atual:
src as       =             N/A
dst as       =             N/A

Benefits of Implementation:

1. BGP Traffic Analysis: Including "src as" and "dst as" will enable network administrators to easily identify BGP (Border Gateway Protocol) relationships between sources and destinations. This is crucial for optimizing routing and improving network efficiency.

2. Security: By tracking AS information, it is possible to identify malicious activities, such as DDoS attacks originating from a specific AS. This strengthens network security measures.

3. Troubleshooting: When troubleshooting connectivity or performance issues, AS data helps isolate problems within service providers or external networks.

4. Traffic Management: These details allow for more efficient allocation of network resources based on traffic sources and destinations.

Recommended Action:

We strongly recommend that MikroTik seriously considers implementing "src as" and "dst as" information in future updates of IP Traffic Flow (NetStream). This enhancement will provide MikroTik users with a more comprehensive and valuable view of network traffic, aligning with industry best practices.

We appreciate your attention to this suggestion and are open to collaborating on the implementation and testing of this feature. We believe that this improvement will significantly benefit all MikroTik users.

Thanks.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10248
Joined: Mon Jun 08, 2015 12:09 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Tue Sep 12, 2023 11:24 pm

In my opinion even more important is to extend the byte counters to 64 bits. (now they are 32 bits which really does not cut it with today's network speeds)
 
guipoletto
Member Candidate
Member Candidate
Posts: 195
Joined: Mon Sep 19, 2011 5:31 am

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Wed Sep 13, 2023 12:21 am

This, and also:

https://github.com/akvorado/akvorado/issues/417

https://github.com/pavel-odintsov/fastnetmon/issues/985

Proper support for sampling-rate and template formatting would go a long way into making Mikrotik a first-class citizen.
AFAIK, some projects hack together _something_ to read flows from Mikrotik, but then you have to have one collector for Mikrotik, and one collector for _(everything else)_
 
pe1chl
Forum Guru
Forum Guru
Posts: 10248
Joined: Mon Jun 08, 2015 12:09 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Wed Sep 13, 2023 4:11 pm

What do you mean, support for template formatting? When you select IPFIX you can mostly select the members of the template (Ok, some check marks enable multiple fields) but when your collector cannot parse a template that has fields it does not want, I'd say the blame is on the collector.
I wrote a simple collector using a standard parser in Perl (Net::Flow) and it parses the IPFIX flow without issue. I did not use packet sampling yet, first I need to understand what that is and if I want that.
(my purpose is to collect a log of all connections made and how much data was transferred, so in my own Perl code I just pick some fields out of the data the router sends, format them to printable, and log them in a file)
 
juniorespow
just joined
Posts: 2
Joined: Thu Feb 24, 2022 10:06 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Wed Sep 13, 2023 6:13 pm

Please Mikrotik, this ASN data in Flows is vitally important for accurate analysis of a well-monitored infrastructure
 
kendryleite
just joined
Posts: 1
Joined: Tue Sep 12, 2023 11:13 pm
Contact:

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Wed Sep 13, 2023 9:53 pm

Please MikroTik we need flows for better efficiency in the structure, with this we can detect types of attacks, we can prevent these types of attacks.
 
User avatar
kadosc
just joined
Posts: 6
Joined: Mon Mar 24, 2014 3:52 am
Location: Araranguá, SC, BR
Contact:

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Wed Sep 13, 2023 9:55 pm

I would like to discuss the implementation of improvements to MikroTik's IP Traffic Flow (NetStream), specifically the addition of "src as" (source AS) and "dst as" (destination AS) information in flow records. These details are essential for understanding and optimizing network traffic and are common features in other network monitoring solutions, as demonstrated below.

Comparison with Other Manufacturers:

To illustrate the importance of this information, let's compare MikroTik's IP Traffic Flow with examples from other manufacturers:

Other Manufacturers:
src as       =             12345
dst as       =             54321
MikroTik - Exemplo Atual:
src as       =             N/A
dst as       =             N/A

Benefits of Implementation:

1. BGP Traffic Analysis: Including "src as" and "dst as" will enable network administrators to easily identify BGP (Border Gateway Protocol) relationships between sources and destinations. This is crucial for optimizing routing and improving network efficiency.

2. Security: By tracking AS information, it is possible to identify malicious activities, such as DDoS attacks originating from a specific AS. This strengthens network security measures.

3. Troubleshooting: When troubleshooting connectivity or performance issues, AS data helps isolate problems within service providers or external networks.

4. Traffic Management: These details allow for more efficient allocation of network resources based on traffic sources and destinations.

Recommended Action:

We strongly recommend that MikroTik seriously considers implementing "src as" and "dst as" information in future updates of IP Traffic Flow (NetStream). This enhancement will provide MikroTik users with a more comprehensive and valuable view of network traffic, aligning with industry best practices.

We appreciate your attention to this suggestion and are open to collaborating on the implementation and testing of this feature. We believe that this improvement will significantly benefit all MikroTik users.

Thanks.
This really is a need to us, any other vendors does it, come on guys !!
 
User avatar
DuhBatista
just joined
Posts: 16
Joined: Mon Oct 01, 2018 6:24 pm
Location: Brazil

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Thu Sep 14, 2023 6:54 am

This struggle has been going on for a long time and has always been requested in this community. Everyone who uses flow requests this type of sampling for ASN. Mikrotik does not indicate whether it has a release date for this support. Flow is as important as having a firewall rule configured on the box. Mikrotik Please pay attention to us mortals.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10248
Joined: Mon Jun 08, 2015 12:09 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Thu Sep 14, 2023 9:30 am

It is probably not that easy to implement...
 
User avatar
remontti
just joined
Topic Author
Posts: 10
Joined: Mon Aug 31, 2015 11:27 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Mon Sep 18, 2023 9:48 pm

It is probably not that easy to implement...
They released version 7, claiming they would revolutionize it. I'm just contributing ideas, while they "sleep," ISPs are migrating to other manufacturers because they don't implement basic things in RouterOS. :(
 
pe1chl
Forum Guru
Forum Guru
Posts: 10248
Joined: Mon Jun 08, 2015 12:09 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Tue Sep 19, 2023 11:43 am

They released version 7, claiming they would revolutionize it.
I agree that v7 in general has been a disappointment. It has been delayed far too long, and the revolutionary new routing engine is unfinished and shows little progress.
It seems that MikroTik has moved away from the small ISP market and is now more interested in the home- and maybe some small-business market.
For my usage it is good enough (I use BGP only on isolated networks) but still there are obvious shortcomings that I run in to and that take forever to fix.
E.g. the standard for IPFIX states the byte counters are 64-bit but in RouterOS they are 32-bit. I asked to change that years ago and at some point I could live by the fact that this would only happen on v7 because such work was no longer done on v6, but came v7 and the problem wasn't solved. Disappointing.
 
 
llamajaja
Member Candidate
Member Candidate
Posts: 198
Joined: Sat Sep 30, 2023 3:11 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Sun Oct 08, 2023 11:16 am

Dont blame Normis and the staff doing the work. Such decisions are made at higher levels, aka the amount of resources allocated to working ON MT development etc... which would cut into the (millionaire) owners profits in the myopic short term...... Heresy!!

So much unrealized potential is the sense I get when reading many expert postst here......... For me, it does all I need well, except the ability to put address lists in routing rules and the infamous zero trust cloudflare tunnel as an options package for all users.
 
User avatar
dzievamarcos
just joined
Posts: 4
Joined: Tue Jan 30, 2024 10:22 pm
Location: Iguazu Falls, Brazil

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Wed Jan 31, 2024 12:20 am

They released version 7, claiming they would revolutionize it.
I agree that v7 in general has been a disappointment. It has been delayed far too long, and the revolutionary new routing engine is unfinished and shows little progress.
It seems that MikroTik has moved away from the small ISP market and is now more interested in the home- and maybe some small-business market.
For my usage it is good enough (I use BGP only on isolated networks) but still there are obvious shortcomings that I run in to and that take forever to fix.
E.g. the standard for IPFIX states the byte counters are 64-bit but in RouterOS they are 32-bit. I asked to change that years ago and at some point I could live by the fact that this would only happen on v7 because such work was no longer done on v6, but came v7 and the problem wasn't solved. Disappointing.
Your wish was granted
What's new in 7.14beta8 (2024-Jan-22 21:07):
*) traffic-flow - use 64bit counters for v9 and IPFIX flows;

Look, I confess to you, I also expected more, but I'm not disappointed, v7 has a lot of good features, for example, the containers run very easily on both the CCR2116 and an X86, without any problems, I was surprised by the performance , but it is still not possible to reserve a percentage of usage for the CPU as is done with memory.

Mikrotik team, please add this feature:
viewtopic.php?p=1018901&hilit=containers+cpu#p1018901
 
pe1chl
Forum Guru
Forum Guru
Posts: 10248
Joined: Mon Jun 08, 2015 12:09 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Wed Jan 31, 2024 12:00 pm

Yes, I saw it that finally the counters are 64-bit, hooray! That means I do no longer have to set a 1-minute timeout on flows...
However it will be some time before this version gets installed on our work router, currently running 7.12.1
(too many "introduced in 7.13" in the changelog lately, I'll wait for things to stabilize, maybe 7.15.1 or 7.14.3 or so...)
 
User avatar
remontti
just joined
Topic Author
Posts: 10
Joined: Mon Aug 31, 2015 11:27 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Thu Feb 08, 2024 3:44 pm

We still don't have information about AS
# nfcapd -E -p 3055 -w /tmp/
Verbose log level: 3
Add flow source: ident: none, IP: any IP, flowdir: /tmp
Bound to IPv4 host/IP: any, Port: 3055
Init v1
Init v5/v7: Default sampling: 1
Init v9: Max number of v9 tags enabled: 105, default sampling: 1
Init IPFIX: Max number of ipfix tags enabled: 91, default sampling: 1
Startup nfcapd.
Process_ipfix: New ipfix exporter: SysID: 1, Observation domain 0 from: 192.168.87.85


Flow Record: 
  Flags        =              0x00 NETFLOW v10, Unsampled
  Elements     =                 8: 1 2 4 10 12 15 20 22 
  size         =               168
  engine type  =                 0
  engine ID    =                 0
  export sysid =                 1
  first        =     1707388953391 [2024-02-08 07:42:33.391]
  last         =     1707388953391 [2024-02-08 07:42:33.391]
  received at  =     1707399770738 [2024-02-08 10:42:50.738]
  proto        =                17 UDP
  tcp flags    =              0x00 ........
  src port     =              5678
  dst port     =              5678
  src tos      =                 0
  in packets   =                 1
  in bytes     =               178
  src addr     =       172.18.18.1
  dst addr     =   255.255.255.255
  input        =                 1
  output       =                 0
  src mask     =                 0 /0
  dst mask     =                 0 /0
  fwd status   =                46
  dst tos      =                22
  direction    =                 0
  biFlow Dir   =              0x00 
  end reason   =              0x00 
  ip next hop  =           0.0.0.0
  ip exporter  =     192.168.87.85

Flow Record: 
  Flags        =              0x00 NETFLOW v10, Unsampled
  Elements     =                 8: 1 2 4 10 12 15 20 22 
  size         =               168
  engine type  =                 0
  engine ID    =                 0
  export sysid =                 1
  first        =     1707388956640 [2024-02-08 07:42:36.640]
  last         =     1707388956640 [2024-02-08 07:42:36.640]
  received at  =     1707399773860 [2024-02-08 10:42:53.860]
  proto        =                17 UDP
  tcp flags    =              0x00 ........
  src port     =             57621
  dst port     =             57621
  src tos      =                 0
  in packets   =                 1
  in bytes     =                72
  src addr     =     192.168.87.84
  dst addr     =    192.168.87.255
  input        =                 1
  output       =                 0
  src mask     =                 0 /0
  dst mask     =                 0 /0
  fwd status   =                21
  dst tos      =               225
  direction    =                 0
  biFlow Dir   =              0x00 
  end reason   =              0x00 
  ip next hop  =           0.0.0.0
  ip exporter  =     192.168.87.85

CFile Block Header: type: 3, size: 368, NumRecords: 3
Ident: 'none' Flows: 2, Packets: 2, Bytes: 250, Sequence Errors: 0, Bad Packets: 0, Blocks: 0
Terminating nfcapd.

I agree that v7 in general has been a disappointment. It has been delayed far too long, and the revolutionary new routing engine is unfinished and shows little progress.
It seems that MikroTik has moved away from the small ISP market and is now more interested in the home- and maybe some small-business market.
For my usage it is good enough (I use BGP only on isolated networks) but still there are obvious shortcomings that I run in to and that take forever to fix.
E.g. the standard for IPFIX states the byte counters are 64-bit but in RouterOS they are 32-bit. I asked to change that years ago and at some point I could live by the fact that this would only happen on v7 because such work was no longer done on v6, but came v7 and the problem wasn't solved. Disappointing.
Your wish was granted
What's new in 7.14beta8 (2024-Jan-22 21:07):
*) traffic-flow - use 64bit counters for v9 and IPFIX flows;

Look, I confess to you, I also expected more, but I'm not disappointed, v7 has a lot of good features, for example, the containers run very easily on both the CCR2116 and an X86, without any problems, I was surprised by the performance , but it is still not possible to reserve a percentage of usage for the CPU as is done with memory.

Mikrotik team, please add this feature:
viewtopic.php?p=1018901&hilit=containers+cpu#p1018901
 
pe1chl
Forum Guru
Forum Guru
Posts: 10248
Joined: Mon Jun 08, 2015 12:09 pm

Re: Suggestion for Enhancement to MikroTik's IP Traffic Flow (Source/Destination AS)

Thu Feb 08, 2024 4:11 pm

We still don't have information about AS
Of course not. It was not in the release notes, so why should it be there?
This topic is only a suggestion for a new feature.

Who is online

Users browsing this forum: No registered users and 2 guests