So this is probably the most sought after "code" in this and other bandwidth management communities. But I'd like to see it done, and actually be usable for the community.

A QOS that will keep track of your customers usage long enough to deprioritize/slow them in comparison to others and your main pipe. Very NetEq like.

We all know how some customers will "hog" your pipes, and others will just "browse". What would be nice is to find a way that (without being application specific) we can slow down/lower their bandwidth cap and lower the priority of those Hogs. Thus allowing your basic browsing customer to not be effected as much and keeping your pipe from being maxed out all the time.

Our requirements are to do this in one machine at the head end, and still be able to NAT and 1:1 publics to privates in this head end router. I know some people would like to be able to do this in a stand alone box as a bridge.

Can it be done? Is someone here knowledgable enough to make it happen? And last but not least, will you make it available for the "community"? Ideas are welcome, or even someone able to step up and make sence of all of these examples of basic shaping that don't quite do what really is needed.

There was an example at some point in the forum, I think, wich kept evidence of total-bytes.
Combine that with simple queues, or queue tree, and you might have something to work on.

When I find that link maybe I'll try put it here.

syntax error

What you're asking *can* be done with Mikrotik.

You just need some imagination, some Time, and Money.

I did exactly what you're 'Challenging' - and more - about 2 years ago.

I call it VX and it works very well (for my circumstances).

What were your circumstances? i.e. what time frame(s), what restrictions, what penalties for offenders?

My critera were :-

Mikrotik APs, OpenWRT APs, DD-WRT APs.
Basic Client Speed limiting (guaranteed rates)
Slightly decreased Speeds after 30 seconds of excessive use
Subtantially decreased Speeds after 30 minutes of excessive use
Client Total Usage (bandwidth) limits per day/week/month (hard Cap)
Auto-cutoff (after 2 emails) for excessive usage, OR bill for excess usage
Automated HotSpots
Tickets
Automated Billing

And *everything* else that i could think of that required my Time, and that a computer could do.

e.g.
1. customer hasn't paid in time. Computer offers them a way to pay. No phone call. They can pay, or not. If Not, no internet, unless they pay.

2. automated SNR stats collection - i can see if a Tree has grown in front of the CPE, or a Radio card is failing on an AP.

Sounds simple, but my objective was to reduce the support calls to zero, and VX has almost done that for me. I still get some calls, but mainly for sales and new enquiries.

Of course, your network has to Work Properly before VX can do anything at all.

VX works very well for me.

An interesting side effect of effective bandwidth management/payment control is that when you have it, and you lose a customer, it is generally because they either do not pay, or require huge amounts of bandwidth to do what they do.

I occasionally lose those types of customers to my competition, who have no bandwidth management at all, which is absolutely Fine by me

The Back Channel is a larger number of Customers that were with my Competition, who pay on time, and use a 'normal' amount of bandwidth, but just want a stable connection.

Happiness is the Result.

(No. VX isn't Free or Open Source. It would cost you 50% of your subscriptions and be a bargain at that)

I don't think you did do "exactly" what I challenged. Your VX seems to reduce by a fixed amount, after a certain time frame. I am hopefull to find a way to dynamically deprioritize a client after a certain ammount of time or bandwidth consumed in reference to others and the Main pipe. OR something close to that.

The quote above kind of illustrates what I am not looking for.. 50% of anything for a "program" is NOT a bargain. It's a rip, and a joke.

When I find that link maybe I'll try put it here.

Jorj, Thank you. I will continue to STF as well. Please do post if you find it.

http://wiki.mikrotik.com/wiki/Limiting_ ... ser_levels

Check this.

You can adapt it to manage different kinds of traffic (HTTP, FTP, P2P), with enough work.
Just an example, tangent somehow to your request.

Your VX seems to reduce by a fixed amount, after a certain time frame

Funny. I don't recall ever giving you a demo.

VX dynamically alters the QoS for each and every client as and when needed based on what they are doing, what they paid for, what you got (and where), and what policies you set for the network.

I can't really sell it as i'd have to swap from Wisp to Software House, so i really should stop banging on about it.

What i *can* say with certainty is that what you are looking for can be done.

50% of anything for a "program" is NOT a bargain

I know it's terrible

Wisps tend not to have the time or money to afford SAP.

No you did not give me a "demo"... your above requirements clearly show time based.

But regardless, I doesn't help other then it can be done. I will figure out a way, AND post it. There are many examples of others systems. Maybe I can spin some of their designs into what I hope to get.

Thanks

You'll find most things in our universe are time based ....

Good luck with the project and i hope it comes out better than you expect.

anal post redacted.

Now your not making a whole lot of sense. But it's not, nor was, relevant to my "challenge"

Thanks

Offer some money as the prize for your Challenge.

You might get more interest.

nasty post redacted

You 'done it' yet ?

C'mon. get on with it.

Stop reading silly posts on forums.

Post useful comments. This is for help, not for "small talk".


( nothing personal, no offence. )

I did have some time after reading all those forums to find a solution. I think I may be on to something, but I have to finalize it and test it.

I'm a pretty busy person, so hopfully I can get to it this weekend. Anyone interested in checking code or theory can send me a PM. Adrian pretty well ruined my desire to paste it in this thread.

Thanks

PMs are disabled on this board. If you find a solution, please post it on the wiki.

Adrian pretty well ruined my desire to paste it in this thread.

Sorry.

My thoughts on this was to use radius accounting to keep track of the bytes, php do do periodic bytecounts, then pop them into a different pcq if usage exceeded certain limits. It's not real time, but it would be close enough.

Brill.

That will certainly work.

Last guy to post made it. That's almost all of it.

Mikrotik. Radius, Hotspot (or whatever you prefer).

Count the bytes moving at whatever interval your server can handle (Time will always be a feature - we ditched Analogue Computers at the start of all this computer madness).

If a Client has moved more bytes than the limit you set them, in the time-frame you set for them - up or down - put them on a different queue on the mikrotik.

What could be easier ?

Implementing it is a bit harder, but it works pretty well.

syntax error

Wooo.

Now the Edges of The Problem are being tickled.

silly post redacted

Adrian,

I just got home from my busy day of 18 hours of a full time job and "trying" to get out and do something for myself. Only to find your on here bashing me and my "challenge". I am full aware of what it will take to get this done, and YES I was on here asking the community to see if someone had done it before (read first post) or if the community was up to the challenge of making it.

So far all any one has heard from you is your doing it wrong, or right,... Or it can be done... WELL, put up, or shut up!! How about that..

I am NOW working on my own solution.. Just NOW. Since there wasn't much of a reply to my challenge. I will use whatever I can find to implement what "I" need for my network. We aren't that large, and we do a pretty good jod of managing what we need to thank you very much. BUT, I am always looking for better ways to do things.

If you have no reasonable input in to "how" to do this, or YES, "code" to make it happen, you really have nothing to add to this thread any more. My support line is on my hip every day (it doesn't ring much), and I am amazing thanks again.. But I am not an MT guru, nor claim to be. I came here for advice, help, and people to bounce ideas off of. But I won't sit here and get harped on just becuase I didn't reply to your posts soon enough or the way you tried to pry information off of me either. (your comments remind me of a bully and a munipulitive punk) Your assumptions have been totally wrong.

To the mods (if there is any) Disabling PM's is rather ridiculous; I surely don't want to post my email to others on the forum to corespond with them. And at this point, I won't be ingauging in this topic any further because of the responses I have gotten from a member, that wishes to stir the pot and not contribute to this thread.

It's no wonder why I havn't started a thread on this forum yet, until now.

hey there gunther. if you need any help or would like to throw some ideas around I would like to have to help you with it. You already have my email. I am the guy that recently was added to your facebook from honduras.

adrian is just being an @$$hole. If he has a solution and dont wont to share it or help out he has no buisness even being on this forum

looks like he is out to just make money and a living

or shut up!

OK. Maybe.

If he has a solution and dont wont to share it or help out he has no buisness even being on this forum

Damn right.

I should never have started the SolarTik thread cos i shouldn't share anything.
http://forum.mikrotik.com/viewtopic.php?f=3&t=42858

Giving stuff away for free is a terrible idea.

the way you tried to pry information off of me either

?!?!?! Like what ?

I doubt you will need anything from me since you are so smart an I am so dumb. Yeah I saw the other thread where you were giving out info on how to make a mk power controller. I happened to see it after I posted what I did because based on what I read here in this thread it was agrivating. I have worked on the QOS with MK several times and come up with some decent solutions. always seems like if i could stack a couple MK inline I could do it easier than with only one device.

We are not asking for someone to give all the answers. We all know how many long hours it takes to come up with somthing good that works. If you want to point in the right direction that is fine.

I just wish we could come up with somthing as a community with a community effort. Some people are super smart in some areas and in others they are not.

If you check out all my post here I have tried to answer questions where I could and have posted some of the soulutions I have come up with.

You ask a question about a ptp link , ospf , bgp or many other questions you get lots of answers. You ask about QOS and people think you want a handout.

You ask a question about a ptp link , ospf , bgp or many other questions you get lots of answers. You ask about QOS and people think you want a handout.

The reason for that is obvious.

MT and all the others make radio stuff that Works. Most of the time it is point-and-shoot.
All the hard work there has already been done (the radio side).

Basic Networking is pretty easy too, and there is a mountain of info freely available on how to do it.

Bandwidth Management is a much more complex and difficult problem - basically because it means balancing what you have to sell with what People Want.

When 'People' and 'What They Expect for their Money' are involved, things go ape.

QoS and how it's done is what differentiates the WISPs.

QoS is where the money really really is.

you are so smart an I am so dumb

I truly have no idea which of us is smarter, nor do i care.

Smart and Dumb have no real meaning at all - it depends entirely on the situation and who wants what.

I only use this forum as an outlet for my sad ramblings. I'm not looking for answers.

Sometimes it seems i give people the answers they want, and for free. Sometimes not.

http://wiki.mikrotik.com/wiki/Limiting_ ... ser_levels

Check this.

You can adapt it to manage different kinds of traffic (HTTP, FTP, P2P), with enough work.
Just an example, tangent somehow to your request.

I have a script I wrote almost identical to that except that it worked with usermanager. I also have one that I wrote and posted here somewhere to set several groups of users and have trees for each group that controlled the the amount of bytes they could download per con before it dropped them to a subqueue with lower data rate or priority, but maintained the overall bandwith per user to what is established in there plan. it worked fine for tcp but had a bug or two when it came to udp.

what i would like to do is

be able to set my pipe size
have 4 groups of speeds say , 256k 512k 1m and 1.5m
have it check each ip in each group for say 10 sec and if they download >x in 10 seconds set a timer for 30 sec, else reset timer to check agian in 10sec.

when timer is up they have download >x total since first count change them to a lower subqueue (speed and priority)set new timer for 30 sec and if at the end of those 30 sec the total is >x continue the process until they are at 30% plan rate. else reset counters and start all over.

be able to use nat if nessasary , but mostly just routing.

be able to set someone to a dedicated priority for dedicated bandwith. (not in any plan, they bought dedicated not shared service)

keep a running total on upload and download per user.

if p2p can be cut out completely per group and given lower priority per group.

if voip can be set to a higher priority per group also

you could add other stuff like dns and other protocols that need higher priority.

I'l be in touch mps01k.. It's been a long weekend so far.

I will add, since I am here, that MT is almost based intirely on "free" subsets.. There is nothing wrong with handing out answers or methods to make certain things work. I have no reason to try and "charge" for my code.. I don't own it and never will!! Someone else could just as well make up the exact same thing and claim it is theirs.

Just so you know, I don't use PPPoE, nor Radius.. This has to be entirely self contained within MT for it too work. So whatever Adrian has (which seems radius based) won't do me ANY good to begin with.

My largest task is trying to figure out weither to use Queues (simple/tree) or PCQ. I have something for simple I have drawn up, but still have to code out. PCQ is a different beast, and I don't like that I can't "see" the individual (per client) traffic to make sure it is even working.

Hell, maybe mps01k and I can come up with the latest and greatest QOS ever for MT, and charge the hell out of everyone to get it.. We could be Millionares in no time LOL. "Giving stuff away for free is a terrible idea." Well, what do you think most of MT is? You just paid for a "package".. There is more to life then money. And I personally enjoy helping others Adrian. I have delt with plenty of A-holes in my passings. You can ramble all you want, it won't change anything, nor your attitude towards others.

The only problem i forsee is the issue of scaleability.

If it's all contained inside the 1 router, what happens when you got 1000 users and multiple internet feeds ?

Is it more a question of making the 1 router balance it's own traffic more 'fairly', or is it a centralised 'system' for controlling the traffic on *all* routers more fairly that you're after ?

Maybe i got this all arse-about face.

Scalability is a concern. But if I can work this the way I would like to, you could have the same code at each border router.

It is (or I think it should be) a border router, to manage the "total" bandwidth going from/to your clients. If we can prioritize traffic via individual client management and by utilizing MT's own priority functions, I think it can work pretty easily.

Personally I don't think you can just mangle traffic to get this to work "dynamically". You almost need to be able to prioritize clients as a whole and individually, slow them down (gradually) after they have been maxing out their connection, and deprioritize them.

For now i need somthing that can handle 300 customers and around 20mb. I am sure that if this does what we want then the cpu of the hardware is goint to come into to play as you scale.

One thing that i was thinking is using traffic monitor and if say total bandwith usage is under 80% it keeps up with where you would be on the deprioritation scale but does not actually lower you until actual bandwith is over 80% total for say 30 seconds or so. I have done somthing similiar to this with simple queues before. i had all customers bandwith set to what they pay for then i had a pcq above all the simple queues. when pipe bandwith was not being used it activated the PCQ and when it went over the top it cut off PCQ and left them with what they had in there simple queues. So basiclly everone had say 1.5mb until system saturated and then it cut everybody to there normal speed of say 512k until Pipe flow went down to 60% then it released the PCQ again. It was like a Brake. It was dynamic but I didnt peticulary like it.

p.s. i would like to do all this on the border also. I am sure not everbody has the same needs, but I could emplement some sort of protocol QOS on every tower MK router there. and on the border do a per user QOS. I am using PPOE termination on each tower and routing to my border.

My largest goal is to slow down the "hogs" so that my normal customers won't have such a hard time.

The plan is to do this with Priorities and slowing down those "hogs" individually, allowing others to use normal/higher priority and normal max speeds. This coupled with standard limits on the Pipe, and with any luck the whole pipe doesn't get so full we have as much packet loss due to discards on the main pipe.

We are getting ready to upgrade to fiber and 10meg service. With additional 10 meg pretty soon for a total of 20. So you and I may be in about the same boat. I would guess one of the newer RB1100's would be fine for quite a bit of Pipe and customers on one machine. Plus I hope to NOT mangle packet flows which will help a lot on processing needs.

I'm pretty sure you'll be looking at something else as the numbers go up, but hey - if it works for now, good.

Very good that you're thinking about it now, and not when there are 299 complaints.

The first issue is how to identfy individual Clients.

*if* you have a fully routed network, then you can go on the IP addresses.

If not, then maybe the Border router is also doing PPoE or whatever.

First thing is to decide How the Client is reliably identified.

Second thing is whether you want to sell different services to different users.

If all users have the same service, then the scripting will be a lot easier.

Lastly you need to decide what the parameters are - like how long can a User Max out their Current service level before being dropped to a Lower service level, and what they have to do (or stop doing) to get back to a Normal service level.

Coding or scripting is the easy part, and there are many capable scripters out there.

Working out WTF it should do is the hard bit.

Entirely routed with Statics on the customer CPE. We already have a fixed high limit on each customer based on the plan they purchase.

I will sit down over the week, maybe tomorrow night and work up my plan to see if it works

Statics & routed. Good.

It might make things easier if you have Queues called something like OK-1M, SLOW-1M etc.

Then if they have bought the 1Mb package, and they go over whatever limit, you just need to change the first part of the name of the queue they're in.

That way you could also see at a glance who was meant to be going slow, but still see what package they bought.

Then run 2 'checking' scripts periodically - one to see if they should be swapped to the Slow lane, the other to see if they should be swapped back to the Fast lane.

If you need to store any data anywhere (like how long they have been in the Slow Lane), you can use the Comments field.

...which is not that silly if centralized with API interfaces on a Server

Hi guys, would you please reply to my question. I don't know how to make a queue tree. please help,my question must be very simple for you.

http://forum.mikrotik.com/viewtopic.php ... 32#p221932

thanks

I returned into the challenge of gunther01.
Everyone was looking for the most ingegnous solution for it.
But why dont you just use Bursts?
http://wiki.mikrotik.com/wiki/Manual:Queues_-_Burst While this will make some fluctuactions of the limits, will ensure a better navigation against file download.
Which is what you are looking for...

Bursts do not work with pcq queues.

do not use pcq queues then

Did you get anywhere with this Gunther ?

I'm not asking you to share or anything (and i'd be surpirsed if you did after the reception you got) just curious if you made any progress.