Save Interface Traffic to File

markerasmus · Tue Oct 17, 2023 12:32 pm

Good day,

I have limited download bandwidth and need to know when I am reaching my limit - I have seen a few different options such as splunk - I have a few Arm64 board which splunk core does not fully support.

how can I just get this info saved to disk from different interfaces? save to file every 5 min - I would then need to export the file Daily / Weekly / Monthly

Wantraffic.jpg

Thank you in advance

markerasmus · Tue Oct 17, 2023 1:43 pm

I found a script that does this

{
:global time ([/system clock get time]);
:global hour [:pick $time 0 2];
:global min [:pick $time 3 5];
:global sec [:pick $time 6 8];
:global stamp ($hour . "_" . $min . "_" . $sec);
:global id ([/system identity get name]);
:global numports 100;
:global name "";
:global rx "";
:global tx "";
:global line "Interface Name, Avg RX Byte, Avg TX Byte";
:global line2 "";
:global filenumber 1;
:global buffer "";
:global newLength 0;
:global debug "";

:set $buffer ($line);

:foreach INTERFACE in=[/interface find] do={

delay 2s;

# record the parameters
:set name [/interface get $INTERFACE name];
:set rx [/interface get $INTERFACE rx-byte];
:set tx [/interface get $INTERFACE tx-byte];

# assemble them into a line
:set line2 ($name . "," . $rx . "," . $tx);

# add the buffer length and the string length
:set newLength ([:len $buffer] + [:len $line2]);

# check if the buffer would now be larger than the variable size limit
:if ($newLength > 4095) do={

# it is. write out the current line buffer (without the latest line)
# first construct the file name
:local fileName ($id . "_BW_" . $stamp . "_" . $filenumber . ".txt");

# create the file as it doesn't exist yet
/file print file=$fileName;

delay 2s;

# write the buffer into it
/file set $fileName contents="$buffer";

# increase the file number
:set $filenumber ($filenumber + 1);

# reset the line buffer to just the last line that hasn't been written to a file yet
:set $buffer ($line . "\n" . $name . "," . $rx . "," . $tx);
:set $newLength 0;
} else= {
# variable size limit would not be exceeded. add line to buffer
[:set $buffer ($buffer . "\n" . $line2)]
}
}

# write out the last buffer
:local fileName ($id . "_BW_" . $stamp . "_" . $filenumber . ".txt");
/file print file=$fileName;

delay 2s;

/file set $fileName contents="$buffer";
set $debug ("Im here");
}

from here - viewtopic.php?t=128864#

markerasmus · Tue Oct 17, 2023 1:45 pm

Next question

how do I ftp these files?

Mikrotik RouterOS 7.1
/tool/fetch address="ubuntu" src-path="/testfile.txt" user=ubuntu mode=ftp password="123456789" dst-path="/home/ubuntu/Data-Exchange/test2.txt" port=21 upload=yes

this just does a single file how can I export a folder and then delete once sent over?

Thanks again

rextended · Tue Oct 17, 2023 1:53 pm

The counters are preserved even on reboot, but (obviously) the intermediate values between two updates are lost if the device is rebooted.

example code

/ip firewall layer7-protocol
{
    :if ( ([:len [find where name="ether1-total-upload"]] = 0) or ([:len [find where name="ether1-total-download"]] = 0) ) do={
        remove [find where name~"^ether1-total-(upload|download)\$"]
        add name=ether1-total-upload regexp=0
        add name=ether1-total-download regexp=0
    }
    :local tupid [find where name="ether1-total-upload"]
    :local tdnid [find where name="ether1-total-download"]
    set $tupid regex=([:tonum [get $tupid regex]] + [/interface get ether1 tx-byte])
    set $tdnid regex=([:tonum [get $tdnid regex]] + [/interface get ether1 rx-byte])
   /interface reset-counters ether1
}

reset stored counters code

/ip firewall layer7-protocol
{
        remove [find where name~"^ether1-total-(upload|download)\$"]
        add name=ether1-total-upload regexp=0
        add name=ether1-total-download regexp=0
}

read current values code

/ip firewall layer7-protocol
{
    :if ( ([:len [find where name="ether1-total-upload"]] = 0) or ([:len [find where name="ether1-total-download"]] = 0) ) do={
        remove [find where name~"^ether1-total-(upload|download)\$"]
        add name=ether1-total-upload regexp=0
        add name=ether1-total-download regexp=0
    }
    :local tupid [find where name="ether1-total-upload"]
    :local tdnid [find where name="ether1-total-download"]
    :local currUpload   [:tonum [get $tupid regex]]
    :local currDownload [:tonum [get $tdnid regex]]
    :put "Current Upload is $currUpload and current Download is $currDownload"
}

markerasmus · Tue Oct 17, 2023 4:01 pm

Thank you so much for the reply

I cobbled together this code from what I read on the forums - it's creating a file and sending it via FTP to a server - now to find how to import these files into a new gui

just to be clear I am no developer can copy past code at best

{
:global fetchdate [/system clock get date]
:global time ([/system clock get time]);
:global hour [:pick $time 0 2];
:global min [:pick $time 3 5];
:global sec [:pick $time 6 8];
:global stamp ($fetchdate . "_" . $hour . "_" . $min . "_" . $sec);
:global id ([/system identity get name]);
:global numports 100;
:global name "";
:global rx "";
:global tx "";
:global line "Interface Name, Avg RX Byte, Avg TX Byte";
:global line2 "";
:global filenumber 1;
:global buffer "";
:global newLength 0;
:global debug "";
# Set params for FTP
:local ftpuser "user"
:local ftppass "pass"
:local ftpaddr "192.168.1.24"


:set $buffer ($line);

:foreach INTERFACE in=[/interface find] do={

delay 2s;

# record the parameters
:set name [/interface get $INTERFACE name];
:set rx [/interface get $INTERFACE rx-byte];
:set tx [/interface get $INTERFACE tx-byte];

# assemble them into a line
:set line2 ($name . "," . $rx . "," . $tx);

# add the buffer length and the string length
:set newLength ([:len $buffer] + [:len $line2]);

# check if the buffer would now be larger than the variable size limit
:if ($newLength > 4095) do={

# it is. write out the current line buffer (without the latest line)
# first construct the file name
:local fileName ($id . "_BW_" . $stamp . "_" . $filenumber . ".txt");

# create the file as it doesn't exist yet
/file print file=$fileName;

delay 2s;

# write the buffer into it
/file set $fileName contents="$buffer";

# increase the file number
:set $filenumber ($filenumber + 1);

# reset the line buffer to just the last line that hasn't been written to a file yet
:set $buffer ($line . "\n" . $name . "," . $rx . "," . $tx);
:set $newLength 0;
} else= {
# variable size limit would not be exceeded. add line to buffer
[:set $buffer ($buffer . "\n" . $line2)]
}
}

# write out the last buffer
:local fileName ($id . "_BW_" . $stamp . "_" . $filenumber . ".txt");
/file print file=$fileName;

delay 2s;

/file set $fileName contents="$buffer";
set $debug ("Im here");


# Upload files to FTP
:delay 15s
/tool fetch address=$ftpaddr src-path=$fileName user=$ftpuser password=$ftppass port=21 upload=yes mode=ftp dst-path=/home/ubuntu/data/mikrotik/$fileName

:delay 15s
# Remove files from device
/file remove [find where name=$fileName]


}

markerasmus · Tue Oct 17, 2023 4:04 pm

Thank you for the reply

If I am pulling data from the interface
:set rx [/interface get $INTERFACE rx-byte];
:set tx [/interface get $INTERFACE tx-byte];
does that still stay after reboot? is it it better to pull from the firewall rules?

The counters are preserved even on reboot, but (obviously) the intermediate values between two updates are lost if the device is rebooted.

example code

/ip firewall layer7-protocol
{
    :if ( ([:len [find where name="ether1-total-upload"]] = 0) or ([:len [find where name="ether1-total-download"]] = 0) ) do={
        remove [find where name~"^ether1-total-(upload|download)\$"]
        add name=ether1-total-upload regexp=0
        add name=ether1-total-download regexp=0
    }
    :local tupid [find where name="ether1-total-upload"]
    :local tdnid [find where name="ether1-total-download"]
    set $tupid regex=([:tonum [get $tupid regex]] + [/interface get ether1 tx-byte])
    set $tdnid regex=([:tonum [get $tdnid regex]] + [/interface get ether1 rx-byte])
   /interface reset-counters ether1
}

reset stored counters code

/ip firewall layer7-protocol
{
        remove [find where name~"^ether1-total-(upload|download)\$"]
        add name=ether1-total-upload regexp=0
        add name=ether1-total-download regexp=0
}

read current values code

/ip firewall layer7-protocol
{
    :if ( ([:len [find where name="ether1-total-upload"]] = 0) or ([:len [find where name="ether1-total-download"]] = 0) ) do={
        remove [find where name~"^ether1-total-(upload|download)\$"]
        add name=ether1-total-upload regexp=0
        add name=ether1-total-download regexp=0
    }
    :local tupid [find where name="ether1-total-upload"]
    :local tdnid [find where name="ether1-total-download"]
    :local currUpload   [:tonum [get $tupid regex]]
    :local currDownload [:tonum [get $tdnid regex]]
    :put "Current Upload is $currUpload and current Download is $currDownload"
}