Page 1 of 1

Automated Usage Monitor script - no usermanager required

Posted: Sun Feb 01, 2009 2:55 am
by omega-00
Hello All,

Just finished ironing out what I believe are the last bugs in my usage monitoring script.
This script is handy for anyone who wants to track IP based usage each month without requiring the mikrotik user manager or radius server.

http://wiki.mikrotik.com/wiki/Automated ... sermanager

All data is stored in the queue comment field each hour so doesn't matter if the router is rebooted, I've also included some simple scripts to help you generate the range of queues required.

All site details are emailed at the end of the month to client and host emails along with percentage warnings (70% 90% 100%) being sent through when limits are reached.

Example email:
Full monthly usage for *CLIENT*
In this month this site has downloaded 5575 MB, which is 10% of the 50 GB monthly download allowance.

Traffic Monitor System,
accounts@yourisp.com
Please report any errors in this message to technical@yourisp.com
Please feel free to contact me if you have any problems implementing the script.
Only known problem is that I don't believe the queues will work correctly if you already have a queue tree setup (this uses simple queues).

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Feb 01, 2009 4:30 pm
by jordantrx
Hello All,

Just finished ironing out what I believe are the last bugs in my usage monitoring script.
This script is handy for anyone who wants to track IP based usage each month without requiring the mikrotik user manager or radius server.

http://wiki.mikrotik.com/wiki/Automated ... sermanager

All data is stored in the queue comment field each hour so doesn't matter if the router is rebooted, I've also included some simple scripts to help you generate the range of queues required.

All site details are emailed at the end of the month to client and host emails along with percentage warnings (70% 90% 100%) being sent through when limits are reached.

Example email:
Full monthly usage for *CLIENT*
In this month this site has downloaded 5575 MB, which is 10% of the 50 GB monthly download allowance.

Traffic Monitor System,
accounts@yourisp.com
Please report any errors in this message to technical@yourisp.com
Please feel free to contact me if you have any problems implementing the script.
Only known problem is that I don't believe the queues will work correctly if you already have a queue tree setup (this uses simple queues).
Most of my queue's are handeled by my CPE's so is this to be put on the CPE or was it made for the AP? Just wondering. -Jordan

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Feb 02, 2009 2:49 am
by omega-00
This is really made for back on the AP, or back on a central router where all your traffic passes through.
It's not made for speedlimiting (although you can do that on any queue you want) but merely to keep a record of all the traffic being used by each IP address.

It would also work fine as a secondary method of accounting to compare against radius logs.

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Nov 16, 2009 6:58 pm
by AJStevens
This sounds like what I need, or close to it.

I looked on the Wiki and found in the Talk a new version that "on peak / off peak accounting - rate limiting when over limit - adjustable on and off peak" which sounds fantastic.

I wish it was formatted though, reading through it is a little bit tricky, you've obviously put a lot of thought into it.

I'm looking for a script for my own RB, which has two LANS, multiple WANs/ISPs, a Primary (Specific Download Usage per Month), and others (Acceptable Usage Polices). I use Packet Marking to direct traffic over the relavent links, my local servers should use the Primary link, however to avoid going over the set Download Usage, it should warn me, and then finally direct LAN->WAN Heavy (Web) traffic via another provider at the 80% mark, and rate-limit it to ensure it's only being used for small connection specific stuff (SMTP) for the remainder of the month.

Peak allowance is 8GB (9am to 6pm, Monday to Friday), Night is 1TB (2am to 6am every day), off-peak is 100GB (all other times). For ease of use, it should also be able to handle adding pre-paid allowance (and adding excess usage allowance, which costs more) one off for the month, and a starting "already used allowance" for current month (reset at start of next month).

It's also very useful to get a report to identify an IP on the network that's causing a lot of downloads, as well as generally what individual IP-based usage is.

Any chance of such a modification/new version of your script for this?

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Nov 16, 2009 8:04 pm
by omega-00
I'm looking for a script for my own RB, which has two LANS, multiple WANs/ISPs, a Primary (Specific Download Usage per Month), and others (Acceptable Usage Polices). I use Packet Marking to direct traffic over the relavent links, my local servers should use the Primary link, however to avoid going over the set Download Usage, it should warn me, and then finally direct LAN->WAN Heavy (Web) traffic via another provider at the 80% mark, and rate-limit it to ensure it's only being used for small connection specific stuff (SMTP) for the remainder of the month.

Peak allowance is 8GB (9am to 6pm, Monday to Friday), Night is 1TB (2am to 6am every day), off-peak is 100GB (all other times). For ease of use, it should also be able to handle adding pre-paid allowance (and adding excess usage allowance, which costs more) one off for the month, and a starting "already used allowance" for current month (reset at start of next month).
I'll break it down into relevant sections for you to start having a look over yourself and see if I can come up with some of the code pre-done for you tomorrow (its 4:30am here :-P)

- two LANS | No problem the simple queues just go by whatever IP's you wish to track
- Multiple WANs/ISPs | Once again no prob as we're just watching the traffic to those IP's doesn't matter where its going out.
- Specific download usage per month | this might be a little tricky if you have set quotas on each link as regardless of which method of load balancing you use you'll always get differences in usage. As for the users, each used can have an assigned on and off peak quota.
- WAN Heavy traffic via another provider. | hmm tricky, might need a single script on one of the pppoe interfaces to help with that however for starters we can get the data monitoring up then go from there.
- on-peak 8GB 9-6, Night 1TB 2-6, off-peak 100GB 6pm-2am | that can be done, mine currently works with 2 data limits but I see no major problem changing to 3.
- adding pre-paid allowance | thats ok however its going to be manual (aka you edit users data and change from 10 to 20/ X to Y)
- at the end of the month i've got mine to reset all users back to 15/15gb

Re: Automated Usage Monitor script - no usermanager required

Posted: Tue Nov 17, 2009 12:00 pm
by AJStevens
Wow, that would be so great.

Yeah, could do with "Local Time" under users details in posts.

- two LANS | No problem the simple queues just go by whatever IP's you wish to track
Well, anything from those two LANs basically, the report also needs to split into the two LANs and list IPs under it for usage.

- Multiple WANs/ISPs | Once again no prob as we're just watching the traffic to those IP's doesn't matter where its going out.
That's because it's only download we're interested in, not upload right? Ok, but it's only one ISP that has a Monthly Usage, the others are just "acceptable usage policy", so unlimited within reason really. Hence we're not interested in any downloads that come from those other ISPs, only the Monthly Usage one (or be able to specify your ISPs, which are metered and which are unlimited).

- Specific download usage per month | this might be a little tricky if you have set quotas on each link as regardless of which method of load balancing you use you'll always get differences in usage. As for the users, each used can have an assigned on and off peak quota.
Using a packet mark for ISP1,ISP2,ISP3, load balancing is more directed traffic, certain LAN IPs/Ports are directed over certain ISPs. For the metered ISP1, upload is using the packet mark, it's then split into two routing marks ISP1a and ISP1b which is then sent over either line 1 or line 2, which at the ISP1 are then put together and onto the internet. Similiarly for download, they send it down over both lines (two pppoe connections). The other ISPs are just a single pppoe, or an ethernet interface (VM cable).
For downloads from ISP1, it's metered, with a quota a month, I'm not fussed about dividing it up internally to "users", there's only one user "the local network". This is more my own monitoring and fail-safe to avoid going over the quota, then dishin it out to users internally.

- WAN Heavy traffic via another provider. | hmm tricky, might need a single script on one of the pppoe interfaces to help with that however for starters we can get the data monitoring up then go from there.
I wouldn't want the pppoe shutdown, just redirect the heavy traffic (say a big download of Windows Updates or something). Instead of packet marking it for ISP1, override and re-mark it for ISP2, but only if we've reached the danger zone of usage (80%).

- on-peak 8GB 9-6, Night 1TB 2-6, off-peak 100GB 6pm-2am | that can be done, mine currently works with 2 data limits but I see no major problem changing to 3.
Cool, was worried that might be complicated.

- adding pre-paid allowance | thats ok however its going to be manual (aka you edit users data and change from 10 to 20/ X to Y)
Yeah, obviously, not sure where you'd do it, smome command to write to a text file stored on the RB?

- at the end of the month i've got mine to reset all users back to 15/15gb
As I said before, it's not really "users", it's just a single user, the local network (LANs), though an IP-based report of usage woud help identify where the bandwidth is being used on the network, ideally LAN IP, monthly amount, and itemised above a threshold, say downloads over 1MB in size, and the WAN IP (hostname would be better, but probably unlikely)

Oh and to be able to set the usage period (I believe mine is 1st - last day of Mnth, which could be 28th, 29th, 30th, 31st)

Re: Automated Usage Monitor script - no usermanager required

Posted: Sat Jan 09, 2010 5:46 am
by omega-00
So yeah. I got busy and didn't get around to this *shame* however I'm currently working on v1.5 of the script which will make the whole setup process a lot more efficient.

Re: Automated Usage Monitor script - no usermanager required

Posted: Sat Feb 20, 2010 8:09 am
by grasmuis
Hi

i have 'n problem running the script when it need to set commnd to assign a value for the :local combinedtext .
it says expected variable name (line 1 column 6)

Re: Automated Usage Monitor script - no usermanager required

Posted: Tue Feb 23, 2010 11:14 am
by Muqatil
:local works only inside scripts...
try
{ :local combinedtext;
:set combinedtext "Hello World";}

Re: Automated Usage Monitor script - no usermanager required

Posted: Thu Feb 25, 2010 6:55 am
by omega-00
The combinedtext value shouldn't be modified anyway, it's a dynamically built list of all the data usages for reporting.
You can however modify the section of email it appears in (under the relevant script)

I've also upgraded the script as a whole and have the new version (with improved variable handling and an easy setup procedure) listed here:

http://www.mikrotik-routeros.com/?p=24

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Mar 08, 2010 8:46 pm
by snoozer
hi,

i was trying to use the latest script from the website above but had/have big problems with the emails. one thing is that i can not force the email function in ROS to NOT use TLS or to accept a self signed cert from the smtp server. if i use a server which does not support TLS i get at least something to happen. unfortunately something is still going wrong.

i had fixed a problem in the "manualusagereport". there was this line:
  :set text "$combinedtext \n $sitename: $percentage% - $megsdowncurrent MB used - Allowance is $gigs GB"

which does not work, i get an error saying: system,e-mail,error Error sending e-mail: invalid BODY

i changed that line to this:
  :set text "$combinedtext
$sitename: $percentage% - $megsdowncurrent MB used - Allowance is $gigs GB"
so there is no \n for the new line in it anymore and it works.

the "monthlyreport" does send one email to the 1st queue,s email address but not to any others. i tried again to find the problem with a tcp dump but it seems the mail function in ROS or the script is mixing things up. the communication i see looks as if the script is trying to send multiple bodys in the wrong smtp session. but i can not say for sure, i am not expert in scripting on ROS at all.

if someone has this working on ROS3.30 please let me know how you got it working. here it does not work with ROS3.30 or 3.28.

thanks
Jan

Re: Automated Usage Monitor script - no usermanager required

Posted: Wed Mar 10, 2010 11:30 am
by snoozer
omega-00,

is there any advise you can give as to how to solve the problem ?

Jan

Re: Automated Usage Monitor script - no usermanager required

Posted: Thu Mar 11, 2010 6:08 pm
by snoozer
ok, i found somethin strange, just for the sake of ruling out that the used smtp server is the problem i installed qmail on a system to testing. now it works perfectly fine. with the other mailserver (of our upstream provider) i get this log entry many times:

echo: e-mail,debug recv: 354-please wait\00

ROS 3.30 does not react how the mailserver would expect i wonder ?
3yz Positive Intermediate reply
The command has been accepted, but the requested action is being
held in abeyance, pending receipt of further information. The
SMTP client should send another command specifying this
information. This reply is used in command sequence groups (i.e.,
in DATA).
see RFC 5321
http://tools.ietf.org/html/rfc5321

Jan

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Mar 29, 2010 11:56 pm
by yffud
Great Idea,

Used this as the basis of a bandwidth jail concept where once users have gone over there allotted daily bandwidth there queue is limited to provide better sharing of resources across all users.

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Nov 06, 2011 5:26 pm
by n21roadie
I am having difficulty in trying to implement this script into the AP using OS5.6
[test@testAP] > :local content
[test@testAP] > :local i
[test@testAP] > :local sitename
[test@testAP] > :local gigs
[test@testAP] > :local email
[test@testAP] > :local megstotal
[test@testAP] > :local totalcurrent
[test@testAP] > :local bytesdowncurrent
[test@testAP] > :local bytesupcurrent
[test@testAP] > :local megsdowncurrent
[test@testAP] > :local megsupcurrent
[test@testAP] > :local percentage
[test@testAP] > :local lastwarning
[test@testAP] > :local warninglevel
[test@testAP] > :local warn
[test@testAP] > :local update
[test@testAP] > :local newwarning
[test@testAP] > :local bytesdownsaved
[test@testAP] > :local bytestotal
[test@testAP] >
[test@testAP] > #Script based on Automated billing script at http://wiki.mikrotik.com/wiki/AutomatedBill
ing
[test@testAP] > #Details:
[test@testAP] > #This script checks all current simple queues and using values stored in the queue comme
nt will allow you keep track of usage by each site
[test@testAP] > #The details stored in the queue are broken down and used to determine when a user shoul
d be sent a usage warning at 50 75 90 and 100%
[test@testAP] > #You can create/modify a new entry as long as you follow this format
[test@testAP] > #1) If you just want to name an entry you may do so like "XYZ Router" in the queue name
[test@testAP] > #2) If you want to determine a data limit, enter it like "sitename!gigabytelimit#emailad
dress!00#0000" into the comment.
[test@testAP] > #3) If you just want to enter a comment on its own, you can do this so long as you don't
have an "!" in the field
[test@testAP] > #Breaking it down: sitename is a descriptive name, gigabyte limit is the monthy limit, e
mail address is the address you want notified
[test@testAP] > #00 is the last percentage warning level for this user, 0000 is the default value for a
new data store
[test@testAP] > #This version created by Andrew Cox - http://www.accessplus.com.au
[test@testAP] > #v1.4 Updated 15/June/2009
[test@testAP] > :log info "------ Begining Daily Usage Reports -------"
[test@testAP] >
[test@testAP] > #For each queue in the list
[test@testAP] > :foreach i in=[/queue simple find comment !=""] do={
{...
{... #Pull comment out of queue and divide up accordingly
{... :set content [/queue simple get $i comment]
expected variable name (line 4 column 6)
[test@testAP] >
[test@testAP] > #Determine variables from comment
[test@testAP] > #Format is: sitename ! gigsallowed # who-to-email ! last warning level(0-50-75-90-99)
[test@testAP] > :if ([:find $content "!"] != "") do={
syntax error (line 1 column 14)
[test@testAP] > :local pos1 [:find $content "!"]
syntax error (line 1 column 23)
[test@testAP] > :local pos4 [:len $content]
syntax error (line 1 column 22)
[test@testAP] > :local pos2 ([:find [:pick $content ($pos1+1) $pos4] "#"]+$pos1+1)
syntax error (line 1 column 31)
[test@testAP] > :local pos3 ([:find [:pick $content ($pos2+1) $pos4] "!"]+$pos2)
syntax error (line 1 column 31)
[test@testAP] > :set sitename [:pick $content 0 ($pos1)]
expected variable name (line 1 column 8)
[test@testAP] > :set gigs [:pick $content ($pos1+1) $pos2]
expected variable name (line 1 column 8)
[test@testAP] > :set email [:pick $content ($pos2+1) ($pos3+1)]
expected variable name (line 1 column 8)
[test@testAP] > :set totalcurrent [/queue simple get $i bytes]
expected variable name (line 1 column 8)
[test@testAP] > :set lastwarning [:pick $content ($pos3+2) ($pos3+4)]
expected variable name (line 1 column 8)
[test@testAP] > :local pos5 [:find $totalcurrent "/"]
syntax error (line 1 column 23)
[test@testAP] > :local pos6 [:len $totalcurrent]
syntax error (line 1 column 22)
[test@testAP] > :set bytesupcurrent ([:pick $totalcurrent 0 ($pos5)])
expected variable name (line 1 column 8)
[test@testAP] > :set bytesdowncurrent ([:pick $totalcurrent ($pos5+1) $pos6])
expected variable name (line 1 column 8)
[test@testAP] > :set megsupcurrent ($bytesupcurrent / 1048576)
expected variable name (line 1 column 8)
[test@testAP] > :set bytesdownsaved ([:pick $content ($pos3+5) $pos4])
expected variable name (line 1 column 8)
[test@testAP] > :set bytestotal ($bytesdowncurrent + $bytesdownsaved)
expected variable name (line 1 column 8)
[test@testAP] > :set megsdowncurrent ($bytestotal / 1048576)
expected variable name (line 1 column 8)
[test@testAP] >
[test@testAP] > #Begin calculating usage percentage
[test@testAP] > :set percentage ( ( $bytestotal * 100) / ($gigs * 1073741824 ) )
expected variable name (line 1 column 8)
[test@testAP] > :log info "$sitename: $percentage%"
expected end of command (line 1 column 14)
[test@testAP] > :if ([$percentage] < 50) do={ :set warninglevel "00" }
syntax error (line 1 column 10)
[test@testAP] > :if ([$percentage] > 50) do={
syntax error (line 1 column 10)
[test@testAP] > :if ([$percentage] < 75) do={ :set warninglevel "50" }
syntax error (line 1 column 12)
[test@testAP] > :if ([$percentage] > 75) do={ :set warninglevel "75" }
syntax error (line 1 column 12)
[test@testAP] > }
[test@testAP] > :if ([$percentage] > 75) do={
syntax error (line 1 column 10)
[test@testAP] > :if ([$percentage] < 90) do={ :set warninglevel "75" }
syntax error (line 1 column 12)
[test@testAP] > :if ([$percentage] > 90) do={ :set warninglevel "90" }
syntax error (line 1 column 12)
[test@testAP] > }
[test@testAP] > :if ([$percentage] > 90) do={
syntax error (line 1 column 10)
[test@testAP] > :if ([$percentage] < 100) do={ :set warninglevel "90" }
syntax error (line 1 column 12)
[test@testAP] > :if ([$percentage] > 100) do={ :set warninglevel "99" }
syntax error (line 1 column 12)
[test@testAP] > }
[test@testAP] >
[test@testAP] > # Parse warning necessity
[test@testAP] > :if ([$warninglevel] > $lastwarning ) do={ :set warn "true" ; :set update "true" }
syntax error (line 1 column 9)
[test@testAP] > :if ([$warninglevel] = $lastwarning ) do={ :set warn "false" ; :set update "false" }
syntax error (line 1 column 9)
[test@testAP] > :if ([$warninglevel] < $lastwarning ) do={ :set warn "false" ; :set update "true" }
syntax error (line 1 column 9)
[test@testAP] >
[test@testAP] > #Update Warning Levels
[test@testAP] > :if ([$update] = true ) do={ :set newwarning $warninglevel } else={ :set newwarning $la
stwarning }
syntax error (line 1 column 9)
[test@testAP] > :if ([$warn] = true ) do={
syntax error (line 1 column 9)
[test@testAP] > #Check for email address in queue comment
[test@testAP] > :if ([$email] != "" ) do={
syntax error (line 1 column 9)
[test@testAP] > /tool e-mail send to="$email" from="noreply@example.com" server="1.3.3.7" subject="$si
tename: Usage at $percentage" body="This message is to inform you of the current usage for $sitename
expected end of command (line 1 column 25)
[test@testAP] > The current warning trigger is $warninglevel%.
expected command name (line 1 column 1)
[test@testAP] >
[test@testAP] > This site has downloaded $megsdowncurrent MB, which is over $percentage% of the $gigs GB
monthly download allowance.
expected command name (line 1 column 1)
[test@testAP] >
[test@testAP] > This is an Automatically generated E-mail that is sent out when users reach 50%, 75%, 90
% and 100% of their cap.
expected command name (line 1 column 1)
[test@testAP] >
[test@testAP] > Traffic Monitor System,
expected command name (line 1 column 1)
[test@testAP] > accounts@example.com"
syntax error (line 1 column 9)
[test@testAP] > }
[test@testAP] > #Send email to support/accounts also
[test@testAP] > /tool e-mail send to="accounts@example.com" from="noreply@example.com" server="1.3.3.7
" subject="$sitename: Usage at $percentage" body="Current usage for $sitename - trigger is $warninglevel%.

expected end of command (line 1 column 100)
[test@testAP] > $megsdowncurrent MB, which is over $percentage% of the $gigs GB monthly download allowan
ce.
syntax error (line 1 column 2)
[test@testAP] > Traffic Monitor System"
expected command name (line 1 column 1)
[test@testAP] > :log info "Sent Warning Level $warninglevel% to $email"
expected end of command (line 1 column 31)
[test@testAP] > }
[test@testAP] > #Set new warning level on queue comment
[test@testAP] > /queue simple set $i comment="$sitename!$gigs#$email!$newwarning#$bytestotal"
syntax error (line 1 column 20)
[test@testAP] > /queue simple reset-counters $i
syntax error (line 1 column 31)
[test@testAP] > }
[test@testAP] > }
[test@testAP] >
As usual I must be omitting something in trying to use this script to monitor clientsusage on a AP with static IP's (10.100.50.x) using pppoe server on the AP.

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Nov 06, 2011 5:38 pm
by fewi
You can't just paste scripts into the CLI. The scope is different. Either surround the whole script with a pair of matching brackets ('{ }'), or paste the script content into an actual script object in "/system script" and execute it from there.

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Nov 06, 2011 7:03 pm
by n21roadie
Thanks Fewi I have pasted into /system/script
Quick trick to generate queues
#change the target address range to match what you want to generate.
:local x
:for x from 2 to 254 do={/queue simple add target-address="10.2.100.$x" queue="default/default"}
would generate queues for 10.0.0.2-10.0.0.254 for you.
How do i modify this to pppoe clients on a 10.100.50.x pool

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Nov 06, 2011 7:41 pm
by fewi
You change "10.2.100.$x" to "10.100.50.$x".

Not to be disparaging or mean, but if you understand the script that little (and what you asked is a very, very, very fundamental thing) I don't think it's a good idea to run it. Applying configuration you don't actually understand to your router is asking for trouble. What do you do when a paying customer reports an issue? You can't even troubleshoot it because you can't be sure what the routing is doing.

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Nov 06, 2011 7:57 pm
by n21roadie
Yes I agree i do have very little knowledge of scripts but I am using a test AP to configure this and teach myself and there is no customers connected and when it's working i can transfer to a active AP and plan B is if issues occur then, from my backup's is to roll back to config before this,

Re: Automated Usage Monitor script - no usermanager required

Posted: Tue Nov 08, 2011 1:14 am
by omega-00
Script has been updated a number of times since this original post.

Newest revision lives here: http://www.mikrotik-routeros.com/?p=24

Re: Automated Usage Monitor script - no usermanager required

Posted: Fri Nov 11, 2011 6:33 pm
by n21roadie
Many thanks for updated version but i have some more questions
(1) How do i edit this script to monitor pppoe connections from the AP,
(2) For queue of 10.100.4.1 to 10.100.4.200 what is the syntax for this,

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Dec 18, 2011 9:33 pm
by n21roadie
Has anyone got this script working on OS V5.9?

Re: Automated Usage Monitor script - no usermanager required

Posted: Sat Dec 31, 2011 5:49 pm
by n21roadie
I have some of the script working (?) on 5.9 and there is no syntax errors when testing in terminal but it does not include usage figures in the email report I just get in the email body?

"Site Usage report - Runtime: 15:01:00


Regards,"

On the AP it uses PPPoE server and this AP uses 10.100.6.1 static for pppoe server - 10.100.6.2-10.100.6.253 pool for CPE clients and I have set static IP's for clients which i find easier to monitor on the Dude, below is a screen shot of simple queues and traffic from one customer but so far no usage listed in the email report?




{
/system scheduler
add comment="" disabled=no interval=1h name=Overseer on-event="/system script run overseer" policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive start-date=jan/01/1970 start-time=00:30:00
/system script
add name=overseer policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive source=":global found
:global mailserver\r\
:global adminaddress\r\
:global fromaddress\r\
:local isfqdn\r\
:set isfqdn \"yes\"
: /* <![CDATA[ */ (function(){try{var s,a,i,j,r,c,l=document.getElementById(__cf_email__);a=l.className;if(a){s='';r=parseInt(a.substr(0,2),16);for(j=2;a.length-j;j+=2){c=parseInt(a.substr(j,2),16)^r;s+=String.fromCharCode(c);}s=document.createTextNode(s);l.parentNode.replaceChild(s,l);}}catch(e){}})(); /* ]]> */ </script>\"
 : /* <![CDATA[ */ (function(){try{var s,a,i,j,r,c,l= document.getElementById(__cf_email__);a=l.className;if(a){s='';r=parseInt(a.substr(0,2),16);for(j=2;a.length-j;j+=2){c=parseInt(a.substr(j,2),16)^r;s+=String.fromCharCode(c);}s=document.createTextNode(s);l.parentNode.replaceChild(s,l);}}catch(e){}})(); /* ]]> */ </script>\"
/system script run monthend
:if ([\$found] = \"true\") do={/system script run monthlyreport; :log info \"--Completed Monthly Report--\"} 
else={ /system script run usagereport; :log info \"--Completed Usage Report--\"}" 
add name=monthend policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive source=":local date
:local time
:local day
:local month
:local year
:local hour
:global found \"\"
:set date [/system clock get date]
:set time [/system clock get time]
:set day [:pick \$date 4 6]
:set hour [:pick \$time 0 2]
:if ([\$day] = \"01\" ) do={ :if ([\$hour] = \"00\" ) do={:set found \"true\" } else={ :set found \"false\" } } else={ :set found \"false\" }"
add name=usagereport policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive source=":global mailserver
:global adminaddress
:global fromaddress
:local content
:local i
:local sitename
:local gigs
:local email
:local megstotal
:local totalcurrent
:local bytesdowncurrent
:local bytesupcurrent
:local megsdowncurrent
:local megsupcurrent
:local percentage
:local lastwarning
:local warninglevel
:local warn
:local update
:local newwarning
:local bytesdownsaved
:local bytestotal
:if ([:find \$content \"!\"] != \"\") do={\r\
:local pos1 [:find \$content \"!\"]\r\
:local pos4 [:len \$content]\r\
:local pos2 ([:find [:pick \$content (\$pos1+1) \$pos4] \"#\"]+\$pos1+1)\r\
:local pos3 ([:find [:pick \$content (\$pos2+1) \$pos4] \"!\"]+\$pos2)\r\
:set sitename [:pick \$content 0 (\$pos1)]\r\
:set gigs [:pick \$content (\$pos1+1) \$pos2]\r\
:set email [:pick \$content (\$pos2+1) (\$pos3+1)]\r\
:set totalcurrent [/queue simple get \$i bytes]\r\
:set lastwarning [:pick \$content (\$pos3+2) (\$pos3+4)]\r\
:local pos5 [:find \$totalcurrent \"/\"]\r\
:local pos6 [:len \$totalcurrent]\r\
:set bytesupcurrent ([:pick \$totalcurrent 0 (\$pos5)])\r\
:set bytesdowncurrent ([:pick \$totalcurrent (\$pos5+1) \$pos6])\r\
:set megsupcurrent (\$bytesupcurrent / 1048576)\r\
:set bytesdownsaved ([:pick \$content (\$pos3+5) \$pos4])\r\
:set bytestotal (\$bytesdowncurrent + \$bytesdownsaved)\r\
:set megsdowncurrent (\$bytestotal / 1048576)\r\
:set percentage ( ( \$bytestotal * 100) / (\$gigs * 1073741824 ) )\r\
:log info \"\$sitename: \$percentage%\"\r\
:if ([\$percentage] < 50) do={ :set warninglevel \"00\" }\r\   :if ([\$percentage] > 50) do={\r\
:if ([\$percentage] < 75) do={ :set warninglevel \"50\" }\r\     :if ([\$percentage] > 75) do={ :set warninglevel \"75\" }\r\
  }\r\
  :if ([\$percentage] > 75) do={\r\
    :if ([\$percentage] < 90) do={ :set warninglevel \"75\" }\r\     :if ([\$percentage] > 90) do={ :set warninglevel \"90\" }\r\
  }\r\
  :if ([\$percentage] > 90) do={\r\
    :if ([\$percentage] < 100) do={ :set warninglevel \"90\" }\r\     :if ([\$percentage] > 100) do={ :set warninglevel \"99\" }\r\
  }\r\
\r\
 :if ([\$warninglevel] > \$lastwarning ) do={ :set warn \"true\" ; :set update \"true\" }\r\
 :if ([\$warninglevel] = \$lastwarning ) do={ :set warn \"false\" ; :set update \"false\" }\r\
 :if ([\$warninglevel] < \$lastwarning ) do={ :set warn \"false\" ; :set update \"true\" }\r\
\r\
 :if ([\$update] = true ) do={ :set newwarning \$warninglevel } else={ :set newwarning \$lastwarning }\r\
 :if ([\$warn] = true ) do={\r\
 :if ([\$email] != \"\" ) do={\r\
  /tool e-mail send server=\$mailserver from=\$fromaddress to=\"\$email\" subject=\"\$sitename: Usage at \$percentage\" body=\"This message is to inform you of the current usage for \$sitename\r\
The current warning trigger is \$warninglevel%.\r\
\r\
This site has downloaded \$megsdowncurrent MB, which is over \$percentage% of the \$gigs GB monthly download allowance.\r\
\r\
This is an Automatically generated E-mail that is sent out when users reach 50%, 75%, 90% and 100% of their cap.\r\
\r\
Traffic Monitor System,\r\
\$fromaddress\"\r\
}\r\
  /tool e-mail send server=\$mailserver from=\$fromaddress to=\$adminaddress subject=\"\$sitename: Usage at \$percentage\" body=\"Current usage for \$sitename - trigger is \$warninglevel%.\r\
\$megsdowncurrent MB, which is over \$percentage% of the \$gigs GB monthly download allowance.\r\
Traffic Monitor System\"\r\
:log info \"Sent Warning Level \$warninglevel% to \$email\"\r\
}\r\
/queue simple set \$i comment=\"\$sitename!\$gigs#\$email!\$newwarning#\$bytestotal\"\r\
/queue simple reset-counters \$i\r\
}\r\
}"
add name=monthlyreport policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive source=":global mailserver\r\
:global adminaddress\r\
:global fromaddress\r\
:local content\r\
:local i\r\
:local sitename\r\
:local gigs\r\
:local email\r\
:local megstotal\r\
:local totalcurrent\r\
:local bytesdowncurrent\r\
:local bytesupcurrent\r\
:local megsdowncurrent\r\
:local megsupcurrent\r\
:local percentage\r\
:local bytesdownsaved\r\
:local bytesdowntotal\r\
\r\
:log info \"------ Begining Monthly Reports -------\"\r\
:foreach i in=[/queue simple find comment !=\"\"] do={\r\
\r\
:set content [/queue simple get \$i comment] \r\
\r\
:if ( [ :find \$content \"!\" ] != \"\" ) do={\r\
:local pos1 [:find \$content \"!\"]\r\
:local pos4 [:len \$content]\r\
:local pos2 ([:find [:pick \$content (\$pos1+1) \$pos4] \"#\"]+\$pos1+1)\r\
:local pos3 ([:find [:pick \$content (\$pos2+1) \$pos4] \"!\"]+\$pos2)\r\
:set sitename [:pick \$content 0 (\$pos1)]\r\
:set gigs [:pick \$content (\$pos1+1) \$pos2]\r\
:set email [:pick \$content (\$pos2+1) (\$pos3+1)]\r\
:set totalcurrent [/queue simple get \$i bytes]\r\
:local pos5 [:find \$totalcurrent \"/\"]\r\
:local pos6 [:len \$totalcurrent]\r\
:set bytesupcurrent ([:pick \$totalcurrent 0 (\$pos5)])\r\
:set bytesdowncurrent ([:pick \$totalcurrent (\$pos5+1) \$pos6])\r\
:set megsupcurrent (\$bytesupcurrent / 1048576)\r\
:set bytesdownsaved ([:pick \$content (\$pos3+5) \$pos4])\r\
:set bytesdowntotal (\$bytesdowncurrent + \$bytesdownsaved)\r\
:set megsdowncurrent (\$bytesdowntotal / 1048576)\r\
:set percentage ( ( \$bytesdowntotal * 100 ) / ( \$gigs * 1073741824 ) )\r\
:log info \"\$sitename: \$percentage%\"\r\
:if ([\$email] != \"\" ) do={\r\
/tool e-mail send server=\$mailserver from=\$fromaddress to=\$email subject=\"\$sitename: Monthly Report\" body=\"This message is to inform you of the full monthly usage for \$sitename\r\
\r\
In this month this site has downloaded \$megsdowncurrent MB, which is \$percentage% of the \$gigs GB monthly download allowance.\r\
\r\
This is an Automatically generated E-mail that is sent out at the end of each month.\r\
\r\
Traffic Monitor System,\r\
\$fromaddress\"\r\
}\r\
  /tool e-mail send server=\$mailserver from=\$fromaddress to=\$adminaddress subject=\"\$sitename: Monthly Report\" body=\"Full monthly usage for \$sitename\r\
In this month this site has downloaded \$megsdowncurrent MB, which is \$percentage% of the \$gigs GB monthly download allowance.\r\
\r\
Traffic Monitor System,\r\
Please report any errors in this message to \$fromaddress\"\r\
:log info \"Sent monthly report for \$sitename to \$email\"\r\
/queue simple set \$i comment=\"\$sitename!\$gigs#\$email!00#0000\"\r\
/queue simple reset-counters \$i\r\
}\r\
}"
add name=manualusagereport policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive source=":global mailserver\r\
:global adminaddress\r\
:global fromaddress\r\
:local content\r\
:local i\r\
:local sitename\r\
:local gigs\r\
:local email\r\
:local megstotal\r\
:local totalcurrent\r\
:local bytesdowncurrent\r\
:local bytesupcurrent\r\
:local megsdowncurrent\r\
:local megsupcurrent\r\
:local percentage\r\
:local lastwarning\r\
:local warninglevel\r\
:local warn\r\
:local update\r\
:local newwarning\r\
:local bytesdownsaved\r\
:local bytestotal\r\
:local text\r\
:local combinedtext\r\
:set combinedtext \"\"\r\
:log info \"------ Begining Manual Usage Reports -------\"\r\
:foreach i in=[/queue simple find comment !=\"\"] do={\r\
:set content [/queue simple get \$i comment] \r\
:if ([:find \$content \"!\"] != \"\") do={\r\
:local pos1 [:find \$content \"!\"]\r\
:local pos4 [:len \$content]\r\
:local pos2 ([:find [:pick \$content (\$pos1+1) \$pos4] \"#\"]+\$pos1+1)\r\
:local pos3 ([:find [:pick \$content (\$pos2+1) \$pos4] \"!\"]+\$pos2)\r\
:set sitename [:pick \$content 0 (\$pos1)]\r\
:set gigs [:pick \$content (\$pos1+1) \$pos2]\r\
:set email [:pick \$content (\$pos2+1) (\$pos3+1)]\r\
:set totalcurrent [/queue simple get \$i bytes]\r\
:set lastwarning [:pick \$content (\$pos3+2) (\$pos3+4)]\r\
:local pos5 [:find \$totalcurrent \"/\"]\r\
:local pos6 [:len \$totalcurrent]\r\
:set bytesupcurrent ([:pick \$totalcurrent 0 (\$pos5)])\r\
:set bytesdowncurrent ([:pick \$totalcurrent (\$pos5+1) \$pos6])\r\
:set megsupcurrent (\$bytesupcurrent / 1048576)\r\
:set bytesdownsaved ([:pick \$content (\$pos3+5) \$pos4])\r\
:set bytestotal (\$bytesdowncurrent + \$bytesdownsaved)\r\
:set megsdowncurrent (\$bytestotal / 1048576)\r\
:set percentage ( ( \$bytestotal * 100) / (\$gigs * 1073741824 ) )\r\
:log info \"\$sitename: \$percentage% - \$megsdowncurrent MB used - Allowance is \$gigs GB\"\r\
:set text \"\$combinedtext \ \$sitename: \$percentage% - \$megsdowncurrent MB used - Allowance is \$gigs GB\"\r\
:set combinedtext \"\$text\"\r\
 }\r\
}\r\
:log info \"------ Ending Manual Usage Reports -------\"\r\
:local time [/system clock get time]\r\
/tool e-mail send server=000.134.198.000 to=\"xxxxxxx@hotmail.com\" subject=(\"Manual usage report\" ,[/system identity get name]) from=\"xxxxxxxxx@xxxxxx.net\" body=\"Site Usage report - Runtime: \$time\r\
\$text\r\
\r\
Regards,\r\
\$fromaddress\""

add name=gen-usage-queues policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive source=\
"#change the target address range to match what you want to generate.\r\
:local x\r\
:for x from 1 to 254 do={/queue simple add target-address=\"10.100.6.\$x\" queue=\"default/default\"}"
}

Re: Automated Usage Monitor script - no usermanager required

Posted: Sat Dec 31, 2011 8:11 pm
by omega-00
The email section was incorrectly printed due to some address obfuscation code on the website, I fixed this about a week ago; my apologies.

Re: Automated Usage Monitor script - no usermanager required

Posted: Sat Dec 31, 2011 10:24 pm
by n21roadie
The email section was incorrectly printed due to some address obfuscation code on the website, I fixed this about a week ago; my apologies.
Which site is the most upto date,

http://www.mikrotik-routeros.com/?p=24

or


http://wiki.mikrotik.com/wiki/Automated ... sermanager

Many Thanks

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Jan 01, 2012 1:43 am
by omega-00
The first (my site), is where I apply all fixes and respond to comments currently.

Re: Automated Usage Monitor script - no usermanager required

Posted: Sun Jan 01, 2012 6:25 pm
by n21roadie
The first (my site), is where I apply all fixes and respond to comments currently.
I tried the code on your site but was getting syntax errors but as the script starting with
/system scheduler
add comment="" disabled=no interval=1h name=Overseer on-event="/system script run overseer" policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive start-date=jan/01/1970 start-time=00:30:00
/system script
add name=overseer policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive source=":global found
:global mailserver\r\
:global adminaddress\r\
:global fromaddress\r\
:local isfqdn\r\
:set isfqdn \"yes\".................................................................
.........................................................................................
as in my above post I had fixed the syntax errors, but this time with each /tool e-mail send server= lines i inserted my email config and this script is working but not fully yet?

With generation of queue's I have used the same IP range as PPPoE server (10.100.6.0) on this AP.
Will this script work this way?, after generating the simple queue's ip range i momentarily disconnected each pppoe client and then in the queues selected it's pppoe interface. also i notice the queues show the dynamic pppoe's from the pppoe server.
252    name="Client_08" target-addresses=10.100.6.253/32 interface=<pppoe-Client_08> parent=none direction=both priority=8 queue=default/default limit-at=0/0  max-limit=0/0 burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s total-queue=default-small
254  D name="<pppoe-Client_08>" interface=<pppoe-Client_08> parent=none direction=both priority=8 queue=default-small/default-small limit-at=512k/4M  max-limit=512k/4M burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s total-queue=default-small
Queue 252 is the script generated and i changed the interface to the pppoe and traffic is shown, on queue 254 is dynamic generated from pppoe show the limits set for the client but does not shown any traffic.

So far I receive emails from the script but there is no usage figures included in the email body?

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Jan 02, 2012 9:18 pm
by n21roadie
The main install script which i used does not have any syntax errors but some of the scripts which it generated has some errors, I have fixed the syntax errors but unsure if I have ommited some configuration, below is my part of my "usagereport" script but i get "no such item" this also happens with the full script.
[test@testAP] > {
{... :global mailserver  
{... :global adminaddress
{... :global fromaddress 
{... :local content     
{... :local i      
{... :local sitename
{... :local gigs    
{... :local email
{... :local megstotal
{... :local totalcurrent
{... :local bytesdowncurrent
{... :local bytesupcurrent  
{... :local megsdowncurrent
{... :local megsupcurrent  
{... :local percentage   
{... :local lastwarning
{... :local warninglevel
{... :local warn        
{... :local update
{... :local newwarning
{... :local bytesdownsaved
{... :local bytestotal    
{... :if ([:find $content "!"] != "") do=
{... :local pos1 [:find $content "!"]    
{... :local pos4 [:len $content]     
{... :local pos2 ([:find [:pick $content ($pos1+1) $pos4] "#"]+$pos1+1)
{... :local pos3 ([:find [:pick $content ($pos2+1) $pos4] "!"]+$pos2)  
{... :set sitename [:pick $content 0 ($pos1)]                        
{... :set gigs [:pick $content ($pos1+1) $pos2]
{... :set email [:pick $content ($pos2+1) ($pos3+1)]
{... :set totalcurrent [/queue simple get $i bytes] 
{... :set lastwarning [:pick $content ($pos3+2) ($pos3+4)]
{... :local pos5 [:find $totalcurrent "/"]                
{... :local pos6 [:len $totalcurrent]     
{... :set bytesupcurrent ([:pick $totalcurrent 0 ($pos5)])
{... :set bytesdowncurrent ([:pick $totalcurrent ($pos5+1) $pos6])
{... :set megsupcurrent ($bytesupcurrent / 1048576)               
{... :set bytesdownsaved ([:pick $content ($pos3+5) $pos4])
{... :set bytestotal ($bytesdowncurrent + $bytesdownsaved) 
{... :set megsdowncurrent ($bytestotal / 1048576)         
{... :set percentage ( ( $bytestotal * 100) / ($gigs * 1073741824 ) )
{... :log info "$sitename: $percentage%"                             
{... :if ([$percentage] < 50) do={ :set warninglevel "00" }
{... :if ([$percentage] > 50) do={                         
{{... :if ([$percentage] < 75) do={ :set warninglevel "50" }
{{... :if ([$percentage] > 75) do={ :set warninglevel "75" }
{{... }                                                     
{... :if ([$percentage] > 75) do={
{{... :if ([$percentage] < 90) do={ :set warninglevel "75" }
{{... :if ([$percentage] > 90) do={ :set warninglevel "90" }
{{... }                                                     
{... :if ([$percentage] > 90) do={
{{... :if ([$percentage] < 100) do={ :set warninglevel "90" }
{{... :if ([$percentage] > 100) do={ :set warninglevel "99" }
{{... }                                                      
{...   
{... :if ([$warninglevel] > $lastwarning ) do={ :set warn "true" ; :set update "true" }
{... :if ([$warninglevel] = $lastwarning ) do={ :set warn "false" ; :set update "false" }
{... :if ([$warninglevel] < $lastwarning ) do={ :set warn "false" ; :set update "true" } 
{...                                                                                    
{... :if ([$update] = true ) do={ :set newwarning $warninglevel } else={ :set newwarning $lastwarning }
{... }                                                                                                 
do: 
interrupted
           no such item
[test@testAP] > 

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Jan 02, 2012 10:42 pm
by n21roadie
Solved, now I finally have usage reports for clients.
245    ;;; Client_55!50#[nohide]XXXXXXX@hotmail.com[/nohide]!00#0000
      name="Client_55" target-addresses=10.100.6.246/32 interface=all parent=none direction=both priority=8 queue=default/default limit-at=0/0 
      max-limit=256k/2M burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s total-queue=default 

Re: Automated Usage Monitor script - no usermanager required

Posted: Tue Jan 03, 2012 11:11 pm
by n21roadie
Client_343: 7% - 3608 MB used - Allowance is 50 GB
Is the figure from above 3608MB for both upload and download or just download only?

Re: Automated Usage Monitor script - no usermanager required

Posted: Wed Jan 04, 2012 12:06 am
by omega-00
By default the script only counts download, however I included the variables for both in case anyone wanted to be able to combine them.

Re: Automated Usage Monitor script - no usermanager required

Posted: Wed Jan 04, 2012 12:46 am
by n21roadie
By default the script only counts download, however I included the variables for both in case anyone wanted to be able to combine them.
Can you give me a example of the variable to count upload and download.

I have read a few times and looked at examples on the website but so far cannot figure out how to set a variable to count uploads, this is important for me as with clients whom i suspect have torrent clients installed are uploading at max for long periods and need to figure if they are using skype video or torrent uploads, is there a way to tell and would the number of established connections be a clue.


Many thanks

Re: Automated Usage Monitor script - no usermanager required

Posted: Tue May 07, 2013 4:48 am
by paulgushue
Hi Omega-00;

I own a WISP in Canada and have 4 head ends and 50 or 60 MT APs. I need "Automated Usage Monitor script" and a port hole my customers can log in to and see their usage from the first of the month and monthly.

I would like to offer you $1000.00 Canadian dollars or Australian dollars which ever you prefer to enable my company to do the above.
Money can be wired to you or email transfer (if possible). I can give you access to whatever resources you need. Logins to the MT 1200, 750G (X2) and the 493G, teamviewer or vnc to a computer at every head end to do as you need.
I am hands on and been banging my head against the wall with the data overages for years. I have too many hours trying to learn how to do what you can do!

My website is G1HiSpeed.com and I am starting other projects in Newfoundland Canada too.

Paul Gushue
COO 1690112 Ontario Inc
O/A G1HiSpeed.com

Re: Automated Usage Monitor script - no usermanager required

Posted: Thu May 09, 2013 5:22 am
by omega-00
Hi Paul,

I've sent you contact details via your website form.

Kind Regards,
Omega-00 (AC)

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Feb 16, 2015 10:39 am
by rayson
This is great

I am only wondering if the new version of RouterOS 6.27 as of typing this has been used with this script?
I need to implement this and am very new to Mikrotik. I am going to play around and try get it to work.

Re: Automated Usage Monitor script - no usermanager required

Posted: Mon Jun 29, 2015 2:10 pm
by Nomis
This is great

I am only wondering if the new version of RouterOS 6.27 as of typing this has been used with this script?
I need to implement this and am very new to Mikrotik. I am going to play around and try get it to work.

Re: Automated Usage Monitor script - no usermanager required

Posted: Thu Aug 25, 2016 6:53 pm
by RayZor13
I am struggling, script seems to work but it never ends, maybe my format is wrong on the queues? Any help?