Pear2/Net_RouterOS API Return Tx-Rate/Bytes-Out and Save Connection for period of time

RouterOS Scripting
1

I’m using Pear2/Net_RouterOS PHP API

i’m developing an application and i have two questions

First :

when i try to get ip/hotspot/host/print its returning limited data not all like Tx/Rate and Bytes-Out but when i used the version 6.32 of mikrotik it was returning only the Bytes-Out value and when i returned back to v5.26 it won’t return it, so how can i return all the data or just the Tx-Rate and Bytes-Out values on my v5.26?

Second :

My script everytime i do anything it says in the Winbox Log : user admin logged in via api then user admin logged out via api, that can lag and bug my Mikrotik i think, but if i saved the connection and used it just once for a period of time that can make sense with no lags or bugs.

My Code :

try {
	$util = new RouterOS\Util(
	    $client = new RouterOS\Client('10.0.0.1', 'admin', 'password')
	);
} catch (Exception $e) {
	die($e);
}

$gethosts = $client->sendSync(new RouterOS\Request('/ip/hotspot/host/print'));

foreach ($gethosts as $host) {

    if ($host->getType() === Response::TYPE_DATA) {

    	var_dump($host);

    }
}

I’ve attached a screenshot of the returned data using the API

Thanks!

2

The only thing I can suggest is adding the “detail” argument. This argument will sometimes add additional properties that are not otherwise returned.

i.e.

$gethosts = $client->sendSync(new RouterOS\Request('/ip/hotspot/host/print detail=""'));

or you could try the other arguments like “bytes”, “packets” and “status”, analogously to the above.

But if even that doesn’t work, then I’m afraid that’s simply a RouterOS bug. One that apparently, as you say, is mostly fixed in 6.32 already (in that at least bytes-out is returned). The solution would be to upgrade to the latest RouterOS version, and report the missing things as bugs if they’re not present there too.

3

The query /print bytes=“” and /print status=“”

Worked and returned most of data but not tx-rate, i’m happy cuz bytes-out returned, the App is almost complete but returning tx-rate will improve it hundred percent, the idea i’m sorting the hosts by the tx-rate, if bytes-out only returned then i’ll sort by bytes-out not tx-rate, the tx-rate is more functional for my App, any idea? btw your genius i wanna thank you in advance.

4

It seems that data is just not available in the menu from the command line, and by extension from the API.

One way to get an almost equally effective “idea” of the rate is to go about this in a more roundabout way… The hosts menu tells you the IP address of each host, both before (“address”) and after login (“to-address”). If you create queues for each logged in user, you can check out the “to-address” from the hotspot hosts, and look up that address in the “/queue simple” menu, where you can see the current rates.

If you want to also capture users that are not logged in, you’d need to let the DHCP server create a queue that is then either deleted or simply is not matched after the login.


But both of these require you to make some assumptions about the router configuration, so if your app is meant to be more generic… I suggest you ask MikroTik to add that to the command line and API, and not add rx/tx rates at all for earlier RouterOS versions.

5

In that case i’m gonna play around with the simple queues to return the tx/rate, i think it’s the last card i have,

The more important thing and the second part of my question is, How to save the connection (run it once),
because my logic is refreshing my App every second to get the returned data in realtime using Jquery/Ajax so i get in touched with the server every second.

Now, in every second i send the request to the server it runs a connection authentication to the mikrotik and it’s logging in the Log like _User admin logged in from _ip_address via Api every second.

That may lag my mikrotik server, specially i figured out that too many problems happen by that logic such as internet disconnect every 15min.

In the C# API i think the connection is being saved for a period of time, Ughhh i dunno Meh :confused:

I hope that you got the idea which i try to achieve.

Any Plans?

6

One way is to use persistent connections, but while my artificial tests don’t show problems, I haven’t tested it “in the wild” myself, nor has anyone reported anything about it (neither “it improved things. Thanks!” or “it broke things. Help!”).

Before actually setting the 5th argument to “true”, make sure PHP runs as either FastCGI or as an Apache module. If you’re using CGI instead, the PHP process just doesn’t persist, and thus can’t save the connection.


Another way is to use Server Sent Events (a.k.a. “EventSource”). I recently managed to successfully use this approach myself.

With that approach, you can’t change commands without reconnecting (because the communication is one way - after the request starts, you only keep reading data), but you can keep a command turned on indefinitely and watch its output. In your case - monitoring rates - this is sufficient. Internet Explorer (and Edge as well) doesn’t support this.


Persistent connections coupled with SSEs will greatly minimize the number of times you’re reconnecting to the router overall.


There’s also Web Sockets, which works similarly to SSEs, except that the communication is two way, so you can ask for another command without even refreshing the page or otherwise make a new HTTP request. This one also works with IE10, IE11 and Edge in addition to the other browsers, and doesn’t require the use of persistent connections, since the server script itself runs indefinitely.

The big problem however is that it requires a special setup at the server to support the protocol. The minimal setup required is for you to have shell access with which you can run in the background a custom PHP file that uses a library such as this one. If your app is intended to be given for 3rd parties to use, the requirement to set up web sockets is going to put a lot of people off.

7

As mentioned, to use Persistent connections i have to set true at the 5th argument of the Client constructor

I did that but it gives me these errors :

$client = new RouterOS\Client('HOST', USER', 'PASSWORD', 8728, true)



exception ‘PEAR2\Cache\SHM\InvalidArgumentException’ with message ‘No appropriate adapter available’ in F:\wamp\www\vendor\pear2\cache_shm\src\PEAR2\Cache\SHM.php:77
Stack trace:

#0 F:\wamp\www\vendor\pear2\net_transmitter\src\PEAR2\Net\Transmitter\TcpClient.php(174): PEAR2\Cache\SHM::factory(‘PEAR2\Net\Trans…’)

#1 F:\wamp\www\vendor\pear2\net_routeros\src\PEAR2\Net\RouterOS\Communicator.php(148): PEAR2\Net\Transmitter\TcpClient->__construct(‘HOST’, 8728, true, NULL, ‘USER/PASSWORD’, ‘’, NULL)
#2 F:\wamp\www\vendor\pear2\net_routeros\src\PEAR2\Net\RouterOS\Client.php(146):
PEAR2\Net\RouterOS\Communicator->__construct(‘HOST’, 8728, true, NULL, ‘USER/PASSWORD’, ‘’, NULL)

#3 F:\wamp\www\events.php(19): PEAR2\Net\RouterOS\Client->__construct(‘HOST’, ‘USER’, ‘PASSWORD’, 8728, true)

Also i’ve tried the EventSource, it’s great way to stream data and i think i’ll move from Ajax Requests to EventSource, it’s a way better, but unfortunetly it’s the same old way, Logging user admin loggedin via api , user admin logged out from api, still same issue, do i need to make Persistent connections to work first then it should not make everysecond authentication?

Also using Web Sockets won’t work because my host PHP version is 5.3 and the Web Sockets requires PHP 5.4

Actually it will be more clearly if you gave me a full code example of the idea, will be appreciated so much.

8

Persistent connections require either the APC or Wincache PHP extension (they are used to actually store the data and do locks across requests). That’s what the error message refers to - neither extension was found on your server.

(I’ll add this to the docs in the section linked above… If you were to read all docs, you’d see it, but partitioned like that, it’s easy to miss…)

For EventSource, you need to set_time_limit(0) at the EventSource URL, or else PHP stops after 30 seconds. EventSource is smart enough to reconnect (and fire a new “open” event), but the extra HTTP request causes an extra login.

If you have both persistent connections and EventSource, then a page refresh would not cause a new login, and neither will accessing the page from multiple browsers/PCs. If you just have EventSource (with set_time_limit(0)), then every page refresh will cause a new login, as will every browser/PC that connects to the web page.

But you have shell access?

If you do, there’s a great chance your host will probably agree to turn on PHP 5.4 for you, if they don’t already allow you to switch it from their control panel somewhere. Ask them.

The case in point was one where logs are shown. You’ll need to alter this to modify your already present queues, rather than just append the data.

Here’s a slightly altered version from the actual code (I’ve removed the project specific stuff):

<?php
use PEAR2\Net\RouterOS;

if (isset($_GET['data'])) {
    header('Content-Type: text/event-stream');

    require_once 'PEAR2_Net_RouterOS-1.0.0b5.phar';

    try {
        $client = new RouterOS\Client('HOST', 'USER', 'PASSWORD');
        
        set_time_limit(0);
        
        $client->sendAsync(
            new RouterOS\Request('/log print detail="" follow', null, 'l'),
            function (RouterOS\Response $logLine) {
                echo 'data: ' . json_encode(
                    array(
                        'type' => $logLine->getType(),
                        'attributes' => $logLine->getIterator()->getArrayCopy()
                    )
                ) . "\n\n";
                ob_flush();
                flush();
            }
        )->loop();
        return;
    } catch (Exception $e) {
        die("event: quit\ndata: {$e->getMessage()}\n\n");
    }
}
header('Content-Type: text/html;charset=UTF-8');
?><!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
        <title>View log</title>
    </head>
    <body>
        <div class="error" id="status"><noscript>JavaScript is disabled</noscript></div>
        <table>
            <thead>
                <tr>
                    <th>Time</th>
                    <th>Topics</th>
                    <th>Message</th>
                </tr>
            </thead>
            <tbody id="items">
            </tbody>
        </table>
        <script type="text/javascript">
            function escapeHtml(text) {
                return text
                    .replace(/&/g, "&")
                    .replace(/</g, "<")
                    .replace(/>/g, ">")
                    .replace(/"/g, """)
                    .replace(/'/g, "'");
            }

            if (!window.EventSource) {
                document.getElementById('status').innerHTML = 'Unsupported browser';
            } else {
                var items = document.getElementById('items');
                var source = new EventSource(location.origin + location.pathname + (location.search || '?') + '&data');
                source.addEventListener('message', function(e) {
                    var data = JSON.parse(e.data);
                    items.innerHTML += "<tr><td>" + escapeHtml(data.attributes.time) + "</td><td>" + escapeHtml(data.attributes.topics) + "</td><td>" + escapeHtml(data.attributes.message) + "</td></tr>"
                }, false);

                source.addEventListener('open', function(e) {
                }, false);
                source.addEventListener('quit', function(e) {
                    document.getElementById('status').textContent = e.data;
                    source.close();
                }, false); 
                source.addEventListener('error', function(e) {
                    if (e.readyState == EventSource.CLOSED) {
                      document.getElementById('status').textContent = 'Connection closed/interrupted';
                    }
                }, false);
            }
        </script>
    </body>
</html>
9

Unfortunetly, my host provider refused to upgrade to PHP 5.4 now so WebSocket won’t help right now until he upgrades to 5.4 and i cannot install APC or WinCahce because i don’t have shell access.

The code example worked and did not Log to the server (saved the connection) but i’ve spent about 3 hours trying to understand the idea of the code example to change it to get hosts/active/etc, but failed.

Especially this part :

function (RouterOS\Response $logLine) {
	echo 'data: ' . json_encode(
	    array(
	        'type' => $logLine->getType(),
	        'attributes' => $logLine->getIterator()->getArrayCopy()
	    )
	) . "\n\n";
	ob_flush();
	flush();
}

Okay, what if i have php files on my web service, and requesting to it using Ajax or EventSource by an html page in my android app.
Getting hosts, active, and other info like system identity/server uptime from those php files and showing them in my html page as results and monitoring them every 1/5 seconds like i said to get results in realtime.

The normal query i’m using :

$gethosts = $util->sendSync(new RouterOS\Request('/ip/hotspot/active/print status=""'));
foreach ($gethosts as $host) {
	// JSON ENCODE the results and send them back to the html page
}

You merged the sendSync request like a callback function RouterOS\Response $logLine.

new RouterOS\Request('/log print detail="" follow', null, 'l'),
            function (RouterOS\Response $logLine) {

What will the query look like if i want to get the Hosts for e.g? and btw i can’t use the PHP code and the HTML in the same page because my android app cannot run PHP so i’m requesting to the PHP in my online web service JsonP

Will that work? if you felt bored or annoyed from me, you don’t have to reply, but i have to thank you so much for being a helping hand.

Thanks.

10

Ah, you see, this here is the part you’re missing/misunderstanding.

sendAsync() is very different from sendSync().
(and I dare say it’s the reason I made this client in the first place, as opposed to patching Denis Basta’s API client)

sendSync() makes a request that eventually ends, and then lets you sift through the results afterwards.

sendAsync() makes a request, but doesn’t wait for it to end. In the meantime, you can start another request, or more importantly, you can start receiving, but only for a little while.

The callback is just one way of processing the potentially infinite stream of responses. That way, when you start receiving responses (with loop() or completeRequest() - in the code above, it was loop()), for each response, the callback is executed, giving you the opportunity to stop the request at that time. If there are no responses, no callback is executed.

There’s actually also another way of receiving replies that you may understand more easily - if there’s no callback, replies collected in a certain time period are stored by the client itself. Once receiving is done, you can inspect the responses collected so far, and when ready, continue to receive more replies.

Replace

        $client->sendAsync(
            new RouterOS\Request('/log print detail="" follow', null, 'l'),
            function (RouterOS\Response $logLine) {
                echo 'data: ' . json_encode(
                    array(
                        'type' => $logLine->getType(),
                        'attributes' => $logLine->getIterator()->getArrayCopy()
                    )
                ) . "\n\n";
                ob_flush();
                flush();
            }
        )->loop();

with

        $client->sendAsync(
            new RouterOS\Request('/log print detail="" follow', null, 'l'),
        );
        $doAnother = $client->loop(5);
        while (true) {
            echo 'data: ' . json_encode(
                array(
                    'type' => 'note',
                    'attributes' => array('time' => '--:--:--', 'topics' => 'PHP-DEBUG', 'message' => 'Loop')
                )
            ) . "\n\n";
            foreach ($client->extractNewResponses('l') as $logLine) {
                echo 'data: ' . json_encode(
                    array(
                        'type' => $logLine->getType(),
                        'attributes' => $logLine->getIterator()->getArrayCopy()
                    )
                ) . "\n\n";
            }
            ob_flush();
            flush();

            if (!$doAnother) {
                break;
            }
            $doAnother = $client->loop(5);
        }

So… What happens with this new code? The client receives responses for 5 seconds. No callbacks are executed. We just enter a “while” loop. Within it, we first write to the output stream a message saying there was a loop() call (this here is just for debugging purposes - so that you see WHEN these happen), then extract all responses collected in those 5 seconds. For each one of them, we write to the event stream, as previously, and we finally flush PHP’s output buffer. Unlike the callback, we can afford to flush after all individual responses are processed, because we know when they end (which is after the foreach). Finally, if the request has not ended during the last loop(), we start receiving replies for 5 more seconds before processing them again with the while loop, or otherwise we break out of the while loop.

If using Ajax, you need to reconnect every 1/5 seconds, which is obviously very performance costly, and will only cause re-logins, unless you have persistent connections.

If using EventSource, the very idea is that you stay connected to the EventSource - you don’t check it every 1/5 seconds, you just connect once, and tell your app what to do each time the server answers. You let the server determine the pace. The EventSource is only disconnected when your app is closed (or when the user navigates to a different web page).

All print commands have a “follow” argument, used the same way as with the logs.

You have no idea how much patience I have when it comes to programming (which is only doubled when it involves my API client) :wink: . Only one person (in another forum) has ever managed to cross the line, and it’s only because after a series of replies (the length of which was enough for a small book to be published…), he was like “Ah, now I get it, thanks! That last thing - you should’ve started with that.” only to return a few days later asking the same initial question that started the whole thing. I much prefer to hear “I’m not sure I get it…”, as opposed to a false “I get it”, as the latter gives me a warm and fuzzy feeling inside that is taken away when I learn it’s false, and how much it hurts is proportional to how much time I spent trying to reach it… as you can imagine, “a small book”'s worth hurts as hell.

I admit I would find it more pleasant if you were to figure certain pieces for yourself (such as the above section), but the stuff I need to explain is equally fun, useful and already fruitful (e.g. the persistent connection requirements - you made me modify the docs on it, and that’s already a win as far as I’m concerned).

11

Hi
I don´t want to hijack this thread but i thought that would be helpfull also.

I installed the below quoted code and it runs amazing fine.

Now for our support support staff I want to show ping times in real time, so I replaced

new RouterOS\Request('/log print detail="" follow', null, 'l'),

with

new RouterOS\Request('/ping address=10.0.10.37', null, 'l'),

and I tried to changing the line

 items.innerHTML += "<tr><td>" + escapeHtml(data.attributes.time) + "</td><td>" + escapeHtml(data.attributes.topics) + "</td><td>" + escapeHtml(data.attributes.message) + "</td></tr>"
                }, false);

But I am lost.

Can someone give me a hint?

Thanks


<?php
use PEAR2\Net\RouterOS;

if (isset($_GET['data'])) {
    header('Content-Type: text/event-stream');

    require_once 'PEAR2_Net_RouterOS-1.0.0b5.phar';

    try {
        $client = new RouterOS\Client('HOST', 'USER', 'PASSWORD');
        
        set_time_limit(0);
        
        $client->sendAsync(
            new RouterOS\Request('/log print detail="" follow', null, 'l'),
            function (RouterOS\Response $logLine) {
                echo 'data: ' . json_encode(
                    array(
                        'type' => $logLine->getType(),
                        'attributes' => $logLine->getIterator()->getArrayCopy()
                    )
                ) . "\n\n";
                ob_flush();
                flush();
            }
        )->loop();
        return;
    } catch (Exception $e) {
        die("event: quit\ndata: {$e->getMessage()}\n\n");
    }
}
header('Content-Type: text/html;charset=UTF-8');
?><!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
        <title>View log</title>
    </head>
    <body>
        <div class="error" id="status"><noscript>JavaScript is disabled</noscript></div>
        <table>
            <thead>
                <tr>
                    <th>Time</th>
                    <th>Topics</th>
                    <th>Message</th>
                </tr>
            </thead>
            <tbody id="items">
            </tbody>
        </table>
        <script type="text/javascript">
            function escapeHtml(text) {
                return text
                    .replace(/&/g, "&")
                    .replace(/</g, "<")
                    .replace(/>/g, ">")
                    .replace(/"/g, """)
                    .replace(/'/g, "'");
            }

            if (!window.EventSource) {
                document.getElementById('status').innerHTML = 'Unsupported browser';
            } else {
                var items = document.getElementById('items');
                var source = new EventSource(location.origin + location.pathname + (location.search || '?') + '&data');
                source.addEventListener('message', function(e) {
                    var data = JSON.parse(e.data);
                    items.innerHTML += "<tr><td>" + escapeHtml(data.attributes.time) + "</td><td>" + escapeHtml(data.attributes.topics) + "</td><td>" + escapeHtml(data.attributes.message) + "</td></tr>"
                }, false);

                source.addEventListener('open', function(e) {
                }, false);
                source.addEventListener('quit', function(e) {
                    document.getElementById('status').textContent = e.data;
                    source.close();
                }, false); 
                source.addEventListener('error', function(e) {
                    if (e.readyState == EventSource.CLOSED) {
                      document.getElementById('status').textContent = 'Connection closed/interrupted';
                    }
                }, false);
            }
        </script>
    </body>
</html>

[/quote]

12

Is there an error? Empty lines? Anything in the browser’s console (F12)?

Also, you say you tried changing

items.innerHTML += "<tr><td>" + escapeHtml(data.attributes.time) + "</td><td>" + escapeHtml(data.attributes.topics) + "</td><td>" + escapeHtml(data.attributes.message) + "</td></tr>"
                }, false);

but I don’t see it changed in the end… You need to use the properties that ping has, and I don’t think it has “message” and “topics”.

BTW,

and it runs amazing fine.

you remind me of Ant-Man, where there was “this chick, who was stupid fine” :laughing: .

13

I Just changed that line to according to the poperties of “API_command_notes” located in mikrotik.com website just to show the three attributes like in your example.

items.innerHTML += “” + escapeHtml(data.attributes.time) + “” + escapeHtml(data.attributes.avg-rtt) + “” + escapeHtml(data.attributes.packet-loss) + “”
}, false);

By the way I see the the API user logs in and out immediately but in your original script NOT.

17:41:03 system,info,account user soporte logged in from 192.168.2.6 via api
17:41:05 system,info,account user soporte logged out from 192.168.2.6 via api

Myabe I have to include in the ping command the “count” parameter?

14

Oh… That…

In JavaScript, “-” is not a valid name for a property accessor, and thus you’re really getting the “avg” property minus the value of a variable called “rtt”.

The workaround for getting the right property is to address it as if it’s an associative array member out of its container, i.e.

items.innerHTML += "<tr><td>" + escapeHtml(data.attributes.time) + "</td><td>" + escapeHtml(data.attributes["avg-rtt"]) + "</td><td>" + escapeHtml(data.attributes["packet-loss"]) + "</td></tr>"
 }, false);

(that’s not technically the right term; JavaScript doesn’t have “associative arrays”, but that here works in the same vain)

15

Boen_robot

It worked as you sugested. Thank you very much
:smiley: