Community discussions

MikroTik App
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Dynamic Queue Tree Max Limit Using Bras / IP Profiles In PHP

Thu Oct 27, 2011 12:31 pm

Being fairly new to router boards I wanted a method to dynamically change my queue tree max limits based on my IP profile. So if my line sync changed and I moved into a new IP profile I did not have to manually change my queue trees to reflect the new rate.

I investigated doing this via the API but found it to be quite difficult and not ideal for someone new to RouterOS.

I in the end settled on scripting a SSH session in PHP.

I currently run some screen scraping PHP cron jobs that connect to my ISP's management pages (clueless on AAISP, a UK independent internet provider) and grab relevant information, one piece of information that is retrieved is my current IP / Bras profile.

Using this information and an available PHP SSH library I created a script to talk to my RB and automatically update the max limit on my relevant download queue trees.

I though I would share this here in the hope it is useful for other people.

First of you will need to install the following SSH library for PHP details on how to do this are listed here: http://kevin.vanzonneveld.net/techblog/ ... _with_php/

You can then include the following PHP script (or modify it) to make the changes to your queue trees.

You will need to call this script from another PHP file using the following method : update_queue_profile($profile) where $profile is the IP / Bras profile rate in megabits per second, for example 9,10,8,7.1. It expect this in that kind of format and not in kilobits.

Obviously I don't know how you will get your rate in megabits as I get mine from screen scraping a webpage at my ISP's end, you may wish do get your actual sync rate from your modem via SNMP and do some calculations based on that or you may get a direct feed from your ISP via xml or the like.

If people would like to see my screen scraping scrips for AAISP or for my Zyxel router (I get information from both and insert them into a mysql database) please let me know.

My script is rather basic and I have no doubt can be improved.

You will need to change the variables $rbUser to reflect your routeros username and $rbPass for the password. You will also need to change the $queuetrees array to reflect the name of your queue trees.
<?php
function update_queue_profile($profile){

   $kbrate=calculate_rate($profile);
   if($kbrate>1000){
        //basic check if we have a profile that is less than 1M something is up so dont make change
        $rbUser = "RouterOS UserName";
        $rbPass = "RouterOS Password";

        $queuetrees = array('"Incoming queue"','"Critical download"','"High-pri interactive download"','"Low-pri interactive download"','"Low-pri non-marked download"','"Low-pri non-interactive download"');

        if (!function_exists("ssh2_connect")) die("function ssh2_connect doesn't exist");

        if(!($con = ssh2_connect("192.168.0.254", 22))){
            echo "fail: unable to establish connection\n";
        } else {
            // try to authenticate with username root, password secretpassword
            if(!ssh2_auth_password($con, $rbUser, $rbPass)) {
                echo "fail: unable to authenticate\n";
            } else {
                foreach($queuetrees as $q){

                        if (!($stream = ssh2_exec($con,  "/queue tree set " .$q ." max-limit=" .$kbrate ."k" ))) {
                            echo "fail: unable to execute command\n";
                        } else {
                            stream_set_blocking($stream, true);
                            fclose($stream);
                        }
                }
            }
        }

   }else{
   }

}

function calculate_rate($profile){
        //convert to float and check if sane
        $profileF = floatval($profile);
        if ($profileF>0){
        //we did not get 0 so we make the assumption (maybe wrong) that we are in Mbits so values like 10, 11, 8,7.1 etc
        //we need to convert this to kilobits as this will make things easier then take 95% of the value (overheads) and pass
        //this value back.

                $kbits = $profileF*1000;
                $per95 = ($kbits/100)*95;

                return $per95;
        }else{
        //if we end up with zero just pass this back
        //and debug upstream
                return 0;
        }
}

?>

 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Thu Oct 27, 2011 7:16 pm

I investigated doing this via the API but found it to be quite difficult and not ideal for someone new to RouterOS.
What part exactly do you find difficult? The fact that it's a different syntax?

Try my API client... it allows you to enter commands with RouterOS' shell syntax, with the only significant caveat that you must explicitly write argument names (as required by the API).

If it's something else, please say what it is.
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Thu Oct 27, 2011 7:18 pm

I investigated doing this via the API but found it to be quite difficult and not ideal for someone new to RouterOS.
What part exactly do you find difficult? The fact that it's a different syntax?

Try my API client... it allows you to enter commands with RouterOS' shell syntax, with the only significant caveat that you must explicitly write argument names (as required by the API).

If it's something else, please say what it is.
I must admit I had not seen your client, I will take a look thanks.
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Fri Oct 28, 2011 12:34 pm

I investigated doing this via the API but found it to be quite difficult and not ideal for someone new to RouterOS.
What part exactly do you find difficult? The fact that it's a different syntax?

Try my API client... it allows you to enter commands with RouterOS' shell syntax, with the only significant caveat that you must explicitly write argument names (as required by the API).

If it's something else, please say what it is.
boen_robot,

I have done some work with your API client and it is very nice to work with, however I cannot find a way to perform the following ssh command, I am sure there is something I am missing but any help would be appreciated.

Command is: /queue tree set numbers=16,17 max-limit=5500k

Thanks, Blake
 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Fri Oct 28, 2011 7:32 pm

Try to use IDs instead of plain numbers, i.e.
new Request('/queue tree set numbers=*' . dexhex(16) . ',*' . dexhex(17) . ' max-limit=5500k'); 
Also make sure those are indeed the IDs. Just because they appear 16th in Winbox doesn't guarantee the ID is *10.

If you know the name of the queue, try to use that instead of the ID.

And last but not least, make sure your API user has writing privileges (last time I tried to diagnose, that turned out to be the problem).

P.S. When a command fails because RouterOS doesn't like it for whatever reason, you'll get error responses... seeing what they say helps in finding what must be changed to make RouterOS accept it. If the client itself doesn't like it, an exception will be thrown.

P.S.S. Love how explicit you are about the skepticism thing on Twitter ;-) . I usually save it for YouTube comments, but that's probably because I don't use social networks.
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Mon Oct 31, 2011 3:26 pm

Try to use IDs instead of plain numbers, i.e.
new Request('/queue tree set numbers=*' . dexhex(16) . ',*' . dexhex(17) . ' max-limit=5500k'); 
Also make sure those are indeed the IDs. Just because they appear 16th in Winbox doesn't guarantee the ID is *10.

If you know the name of the queue, try to use that instead of the ID.

And last but not least, make sure your API user has writing privileges (last time I tried to diagnose, that turned out to be the problem).

P.S. When a command fails because RouterOS doesn't like it for whatever reason, you'll get error responses... seeing what they say helps in finding what must be changed to make RouterOS accept it. If the client itself doesn't like it, an exception will be thrown.

P.S.S. Love how explicit you are about the skepticism thing on Twitter ;-) . I usually save it for YouTube comments, but that's probably because I don't use social networks.
Thanks for the information, I have had a chance to have a play around but it is still not working for me :( (I have made sure the user has full admin rights!)

I am getting !trap errors but no debug information, I have the following code to trap errors and debug. Am I missing anything that would be give me further debug information?

Error Trap Code
foreach($responses as $response) {
        echo $response->getTag() . "\n";
        echo $response->getType() . "\n";
        foreach ($response->getUnrecognizedWords() as $r){
                echo $r;
        }
}
I assume from your code line above you meant dechex (i.e decimal to hex ) I have tried this with no joy and have tried using the name= parameter as well, it is very strange almost if queues behave differently in the api.

Oh thanks for the comments on skepticism! I am a die hard skeptic! I think everything should have an evidence base and we live our lives through science! Plus it is always better to learn and explore the unknown with a rational head rather than a mystical one
 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Mon Oct 31, 2011 4:15 pm

I assume from your code line above you meant dechex (i.e decimal to hex )
Opps... yes. That's what I meant.
I have tried this with no joy and have tried using the name= parameter as well, it is very strange almost if queues behave differently in the api.
I meant use the name at the numbers argument. The argument "numbers" is the equivalent of not writing anything in shell, i.e. the shell
/queue tree set something
is equivalent to the API
/queue tree set numbers=something
(which also works in shell BTW... it's just the former form that doesn't work with the API)
I am getting !trap errors but no debug information, I have the following code to trap errors and debug. Am I missing anything that would be give me further debug information?

Error Trap Code
foreach($responses as $response) {
        echo $response->getTag() . "\n";
        echo $response->getType() . "\n";
        foreach ($response->getUnrecognizedWords() as $r){
                echo $r;
        }
}
The error responses are just like all other responses - they contain arguments/properties. getUnrecognizedWords() is currently only useful in !fatal responses, because they don't have a "structure" behind them (and would theoretically be useful if future versions of RouterOS have new API specific stuff). A full debugging code should include the arguments:
foreach($responses as $response) {
        echo $response->getTag() . "\n";
        echo $response->getType() . "\n";
        foreach ($response->getAllArguments() as $name => $value){
                echo "{$name}: {$value}\n";
        }
        foreach ($response->getUnrecognizedWords() as $r){
                echo "{$r}\n";
        }
}
 
Last edited by boen_robot on Tue Nov 01, 2011 5:12 pm, edited 4 times in total.
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Mon Oct 31, 2011 4:25 pm

I meant use the name at the numbers argument. The argument "numbers" is the equivalent of not writing anything in shell, i.e. the shell
/queue tree set something
is equivalent to the API
/queue tree set numbers=something
(which also works in shell BTW... it's just the former form that doesn't work with the API)
I think everything has just clicked into place! I was trying to user numbers= with actual numbers that were clearly not mapping to the items that I thought or were just causing errors. Then I was thinking clearly I cant use numbers= with a name it must be name= and then that was not working. But with that succinct explanation I now understand. Changed my code to sent numbers=nameofqueue and it is all working!

Thank you for your time and effort with this, I will update my script to reflect the use of your api.

Thanks again for your help.

P.S Thanks for the debug code and the explanation that also helped and did return more relevant error messages than I was getting before!

P.P.S Thanks for writing the PHP API!
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Wed Nov 02, 2011 4:04 pm

Here is as promised an updated version that I am now using.

This uses the PHP API by boen_robot and requires it to be installed.

I am using this over my SSH implementation as this is the correct way of doing this (via the mikrotik api) rather than my work around.
<?php
namespace PEAR2\Net\RouterOS;
require_once 'PEAR2/Net/RouterOS/Autoload.php';

function update_queue_profile($profile){

   $kbrate=calculate_rate($profile);
   if($kbrate>1000){
        //basic check if we have a profile that is less than 1M something is up so dont make change
        $rbHost = "RB IP ADDRESS";
        $rbUser = "RB USER ID";
        $rbPass = "RB PASSWORD";

        $client = new Client($rbHost,$rbUser,$rbPass);

        $queuetrees = array('"Incoming queue"','"Critical download"','"High-pri interactive download"','"Low-pri interactive download"','"Low-pri non-marked download"','"Low-pri non-interactive download"');

        foreach($queuetrees as $q){
                $responses = $client->sendSync(new Request('/queue tree set numbers=' . $q . ' max-limit=' . $kbrate . 'k'));
        }
   }
}

function calculate_rate($profile){
        //convert to float and check if sane
        $profileF = floatval($profile);
        if ($profileF>0){
                $kbits = $profileF*1000;
                $per95 = ($kbits/100)*95;
                return $per95;
        }else{
        //if we end up with zero just pass this back
        //and debug upstream
                return 0;
        }
}
?>

 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Thu Nov 03, 2011 4:01 pm

Note that since you're setting the same rate, it would be more efficient to wrap up everyone at one point.

Also, if you don't care about the responses, using sendAsync() is more efficient, since there is no response processing.
<?php
namespace PEAR2\Net\RouterOS;
require_once 'PEAR2/Net/RouterOS/Autoload.php';

function update_queue_profile($profile){

   $kbrate=calculate_rate($profile);
   if($kbrate>1000){
        //basic check if we have a profile that is less than 1M something is up so dont make change
        $rbHost = "RB IP ADDRESS";
        $rbUser = "RB USER ID";
        $rbPass = "RB PASSWORD";

        $client = new Client($rbHost,$rbUser,$rbPass);

        $queuetrees = array('Incoming queue','Critical download','High-pri interactive download','Low-pri interactive download','Low-pri non-marked download','Low-pri non-interactive download');

        $client->sendAsync(new Request('/queue tree set numbers="' . implode(',', $queuetrees) . '" max-limit=' . $kbrate . 'k', 'p'));
   }
}

function calculate_rate($profile){
        //convert to float and check if sane
        $profileF = floatval($profile);
        if ($profileF>0){
                $kbits = $profileF*1000;
                $per95 = ($kbits/100)*95;
                return $per95;
        }else{
        //if we end up with zero just pass this back
        //and debug upstream
                return 0;
        }
} 
(I'm not 100% sure the comma syntax works for queue names, but it's worth a try... and I'm sure on the sendAsync() thing)
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Thu Nov 03, 2011 6:34 pm

Note that since you're setting the same rate, it would be more efficient to wrap up everyone at one point.

Also, if you don't care about the responses, using sendAsync() is more efficient, since there is no response processing.

(I'm not 100% sure the comma syntax works for queue names, but it's worth a try... and I'm sure on the sendAsync() thing)
Interestingly the Async method does not work, however if I send the exact same instruction normally it does work...

Fail
$client->sendAsync(new Request('/queue tree set numbers="' . implode(',', $queuetrees) . '" max-limit=' . $kbrate . 'k','p'));
 
Work
$responses = $client->sendSync(new Request('/queue tree set numbers="' . implode(',', $queuetrees) . '" max-limit=' . $kbrate . 'k'));
 
Very strange indeed!
 
User avatar
boen_robot
Forum Guru
Forum Guru
Posts: 2400
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Thu Nov 03, 2011 7:05 pm

Is there an exception, or does it all finish OK, but with no effect visisble in the router after the PHP script is finished?

If the latter... that's most odd... sounds like it might be some sort of a PHP bug related to buffering*... or maybe it's a RouterOS thing**.

I'll have to look into it more deeply. Thanks.

* Keeping the buffer in until responses are requested and forgetting to flush before the script is finished.
** RouterOS declines to perform an action before the client receives the response

(Note to self: How on earth am I going to write Unit tests for this?)
 
GhostSeven
just joined
Topic Author
Posts: 13
Joined: Sat Oct 15, 2011 2:48 pm

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In

Fri Nov 04, 2011 2:21 am

It exits cleanly which I will admit is odd. No exceptions.
 
hadarelv
just joined
Posts: 9
Joined: Sat Jan 15, 2022 10:59 am

Re: Dynamic Queue Tree Max Limit Using Bras / IP Profiles In PHP

Mon Jan 24, 2022 10:28 pm

boen_robot, the link to Your API is broken. Is it still available somewhere?

Who is online

Users browsing this forum: No registered users and 7 guests