Community discussions

MikroTik App
 
derr12
Member
Member
Topic Author
Posts: 411
Joined: Fri May 01, 2009 11:32 pm

scripting a Bandwidth hog penalty box.

Fri Mar 15, 2013 12:45 am

Hi guys. Im trying to figure out how to impliment this on transparent shaper.

A user has been running High sustained data rate for hours and is hogging bandwidth. I want to be able to identify high sustained traffic ip's and place them into a "penalty box" where they get say, 60% of thier allowed bandwidth.

I get how i would penalize the IP by simply adding a simple queue at a reduced rate for x Hours but....
how would a script be able to identify bandwidth rates to an IP address on a bridged mikrotik? torch?

Has anyone tried something like this before?
This kind of script would be well above and beyond my skill to make. I would be willing to pay someone for this job.
 
SamWCL
Frequent Visitor
Frequent Visitor
Posts: 75
Joined: Mon Apr 20, 2009 1:18 pm
Location: Nelson, NZ

Re: scripting a Bandwidth hog penalty box.

Fri Mar 15, 2013 3:08 pm

Bump!
 
derr12
Member
Member
Topic Author
Posts: 411
Joined: Fri May 01, 2009 11:32 pm

Re: scripting a Bandwidth hog penalty box.

Fri Mar 15, 2013 5:20 pm

had a chat with Butch Evans, He suggested about the only way to moniter usage in this setup would be with simple queues. so...

I guess I should start by adding simple queues for every IP on the subnet this transparent shaper will be shaping(unlimited speed by default), then i can start working on the script to moniter usage and adjust on the fly.

I can see this taking up a lot of resources. Anybody tried something like this before? how did it pan out?
 
derr12
Member
Member
Topic Author
Posts: 411
Joined: Fri May 01, 2009 11:32 pm

Re: scripting a Bandwidth hog penalty box.

Fri Mar 15, 2013 7:12 pm

I havnt ruled out the possibility of paying someone to script this for us either if anyone is interested?
 
derr12
Member
Member
Topic Author
Posts: 411
Joined: Fri May 01, 2009 11:32 pm

Re: scripting a Bandwidth hog penalty box.

Fri Mar 15, 2013 8:48 pm

I got to thinking, having a bajillion simple queues for a script to check might not be the way to go... its messy and they execute in order.

What if the script watches the wan link bandwidth. If it reaches 90% of its capability, the script would fire up torch and watch for streams to/from the local subnet. If they are high bitrate for whatever amount of time you have the script create a simple queue for this user @ 2m/1m or whatever.

Have another script clear all the simple queues after x hours or after total bandwidth usage drops down below whatever speed you want it to stop managing hogs.

anybody got any thoughts? Any experienced scripters think this is possible?
 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: scripting a Bandwidth hog penalty box.

Fri Mar 15, 2013 11:29 pm

Why put penalties in the first place?

Why don't you just set up simple queues in the first place, and have them all under one parent that specifies the total bandwidth you have available.

If you do that, your users are never going to encounter the "hogging" situation to begin with - the bandwidth will be divided among all active users until they fill up your whole bandwidth, and when a new user comes along, all other users' bandwidth will automatically decrease to accommodate the new user.

By using "max-limit", you can specify an upper limit, if "unlimited" is not what you want each user to have... and using "limit-at", you can (in essence) declare a minimum speed certain clients must have.
 
derr12
Member
Member
Topic Author
Posts: 411
Joined: Fri May 01, 2009 11:32 pm

Re: scripting a Bandwidth hog penalty box.

Sat Mar 16, 2013 12:53 am

Im trying to sell this as a NetEQ replacement which uses a penalty-box type approach. PCQ ensures fairness by equally sharing whats left over amungst all users yes, but what we have to gain with the penalty box idea is an overall bandwidth increase for the other non-penalized users. It is a small but noticeable improvement

Consider 4 users, all have 10mbit service on a 20mbit backbone.

user 1 has been bit torrenting all day @ full speed. users 2 3 and 4 come on and fire up netflix in HD. If i penalize the heavy user 1 and give him 2 megs for the next 2 hours, users 2,3, and 4 now get 6mbit each instead of 5 for the next 2 hours. hog might be unhappy, but chances are he doesnt notice and users 2 3 and 4 can still watch netflix HD without interruption.

When you scale up to the ISP level you can see how this would become a good thing.

The reason im trying to switch to RouterOS is because NetEQ's do not prioritize traffic wourth a crap. If one of our wireless backbones is full, VOIP sucks hard (dropped calls audio jitters). Mikrotik QOS fixes that. Plus a routerboard is waaaaay cheaper than a NetEQ.


PS; I might have forgotten to mention that Users have their bandwidth capped via modem configs or pppoe servers farther down the line.
 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: scripting a Bandwidth hog penalty box.

Sat Mar 16, 2013 4:05 am

The biggest obstacle here is the fact that "full speed" is too loosely defined for any semi-efficient script to actually be written.

After you have a simple queue per user, you can get a counter of all bytes that a certain user has downloaded up to a point, and you can reset said counter every time you end the period over which you analyze user's bandwidth.

But here's the tricky part... if speed varies over time, when do you do the analysis? Every time the speed changes? That's pretty much "always", i.e. the router itself will hog too much CPU.

If you have a fixed max-limit for each user, you can instead perform the analysis at regular intervals against that limit.

e.g. if you check all queues every 3 hours, and you want to penalize anyone who's been at full speed for more than 2 hours straight, the check a script would do would go something along the lines of
[max-limit (in bytes)]*60*60*2 < [total-tx-bytes]
(I'm not being hasty in trying to write a script, as we're still figuring out what would suit you AND can be implemented)
 
derr12
Member
Member
Topic Author
Posts: 411
Joined: Fri May 01, 2009 11:32 pm

Re: scripting a Bandwidth hog penalty box.

Sat Mar 16, 2013 6:28 pm

Thanks for the consideration;

Yes, the way i see it we would only need to start checking queues after ether1 hits 60mbits. Forgive my psuedo scripting:
schedualer can execute the first script every 30mins; If ether1>60mbits then run script "check users" which would then run;
[max-limit (in bytes)]*60*60*2 < [total-tx-bytes]

If a user is exceeding 5mbits (averaged) for the last 30 mins then you would change the max-limit on his simple queue.
If the ether1 interface goes to less than 60mbits we can reset all max limits to 0 (unlimited)

That process makes sense to me, the only problem as stated before with that approach is how it will perform in areas where i have a thousand IP addresses or more to moniter. I guess the only way to find out how it works is to field test it.
 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: scripting a Bandwidth hog penalty box.

Wed Mar 20, 2013 11:20 pm

When is each counter being reset? That's vital, because it also determines how calculations are calibrated.

With that in mind...
If a user is exceeding 5mbits (averaged) for the last 30 mins then you would change the max-limit on his simple queue.
I take it that this means that each user's counter will be reset every 30 mins, right? In that case, the
[max-limit (in bytes)]*60*60*2 < [total-tx-bytes]
is redundant, as it can never be reached.
If the ether1 interface goes to less than 60mbits we can reset all max limits to 0 (unlimited)
I'm not sure I get this... don't you mean when the interface goes to more than 60mbits, reset the max limits of all users and the interface counter itself?
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6263
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: scripting a Bandwidth hog penalty box.

Thu Mar 21, 2013 2:43 pm

as initially suggested IMHO best solution is to implement QoS where you can wary a lot of options in a way you want to.

so for example if you want the effect that web is always fast - mark it in mangle add connection mark and set up queue for fist several MB of connection unlimited, after that move that traffic to other queue that is throttled down.

web pages will fly, but longer downloads will be.. erm, longer. you can vary with the limitations when connection is moved to limited queue etc.

creating some weird script can work initially but that will not scale well when customer count increases. When customers notice the interval. When you hit the problem that some customer is opening, for example, larger youtube video and is caught by your script. So you will end up setting check interval lower and lower until you hit state when you management script is using more power than actually managing user traffic.

Who is online

Users browsing this forum: Maxfadeev699 and 23 guests