Community discussions

MUM Europe 2020
 
User avatar
boen_robot
Forum Guru
Forum Guru
Topic Author
Posts: 2411
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

[API bug/suggestion] Regex in queries

Thu Nov 17, 2011 4:22 pm

I was recently made aware that the API doesn't support regular expressions in queries. While I don't personally have a need for that (yet), there are others who do, and I believe it's a reasonable feature request.

Reasons for having this feature and use cases are the same for which queries exist in the first place - to limit the amout of memory consumed by API clients, and allow RouterOS to minimize network traffic.

To keep consistency with other operations, using a syntax like
?~name=pattern
would probably be best IMO.

[edit]
Well, I was told that the following syntax works for making print filter by regex in API:
~name~"pattern"
However this appears more like a bug in API rather than a feature, since we have a completely inconsistent syntax from the rest of the API. Not to mention it's not part of the spec.
[edit]
PEAR2_Net_RouterOS(1.0.0b6) - My API client in PHP
(Rate my posts? If you want... no pressure...)
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: [API bug/suggestion] Regex in queries

Mon Nov 21, 2011 10:14 am

hmm

prints all interfaces, so it does not work:
/interface/print
~name~"ether"
prints no interfaces, so it does work:
/interface/print
?name=ether
prints one interface, so it does work:
/interface/print
?name=ether1
as i have been told some long time ago, as it is, in most cases, cheaper for router to send packets than do regexp matching, API will not have this feature. And that has to be done on client side.
 
User avatar
boen_robot
Forum Guru
Forum Guru
Topic Author
Posts: 2411
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: [API bug/suggestion] Regex in queries

Mon Nov 21, 2011 1:40 pm

The example (which you can see on the linked page) was
/ip/dhcp-server/lease/print
=count-only=
~active-address~"10.80."
Maybe it's a problem/"feature" that occurs only in that particular print.
as it is, in most cases, cheaper for router to send packets than do regexp matching, API will not have this feature. And that has to be done on client side.
In terms of router CPU, yes. But overall, in order for the client to process a response, it needs to first receive it in memory (which is also spending some client CPU time right there, not to mention network bandwidth) which may not be good for the client, plus the actual regexp matching itself, which may not be as efficient as RouterOS' regexp.

So although it would be more work for the router, it should be up to the user to decide where will they put the burden.
PEAR2_Net_RouterOS(1.0.0b6) - My API client in PHP
(Rate my posts? If you want... no pressure...)
 
amr
just joined
Posts: 1
Joined: Sun Feb 26, 2012 10:49 pm

Re: [API bug/suggestion] Regex in queries

Sun Feb 26, 2012 10:57 pm

So although it would be more work for the router, it should be up to the user to decide where will they put the burden.
I couldnt agree more! We run routerOS on none mikrotik hardware - I think servers are capable of a little regex :/

Can anyone confirm that regex/partial value matching does not work?

"~key~value" does not filter anything, and "?~key~value" returns nothing.

You can do it via terminal ...
 /ip address print where address~"10.10"
So how is it any extra "burden" via the API?
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: [API bug/suggestion] Regex in queries

Mon Jun 04, 2012 3:59 pm

it adds a lot. As i explained before - it is cheaper to run them on client side than on the server side, taking into account that queries are possible.

I have not seen any powerful router placed somewhere, just to idle around. Usually that is done other way around - you are required to have powerful router, and when you want to automate some tasks, you want as little additional load (not routing/network related) as possible to avoid other hardware upgrade.
 
User avatar
boen_robot
Forum Guru
Forum Guru
Topic Author
Posts: 2411
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: [API bug/suggestion] Regex in queries

Mon Jun 04, 2012 4:30 pm

But regex is available in scripting/SSH... I'd assume that if implemented in the API, it would use the same regex engine, so where does the difference come from?

I mean, in both cases you have a non-routing automated action. The only difference is that with scripting/SSH, the results are not sent over the network, but kept for further processing (which could in turn include a sending over the network, especially if we're talking SSH output). Or am I missing something else?

If the penalty is really high for some scenarios, people could always just... I don't know... not use this feature in those scenarios for example. A warning in the docs would suffice to alert people IMO.
PEAR2_Net_RouterOS(1.0.0b6) - My API client in PHP
(Rate my posts? If you want... no pressure...)
 
chojrak11
Member Candidate
Member Candidate
Posts: 129
Joined: Sun Apr 05, 2009 10:37 am

Re: [API bug/suggestion] Regex in queries

Tue Mar 31, 2015 7:05 am

But regex is available in scripting/SSH...
Yeah that mismatch is weird. So I can put a "burden" on a router using SSH, but I cannot do the same using API? Where's the logic?

You could implement this with at most 30 minutes of coding, and let users decide if it's useful or not. These autocratic arguments seems like corporation which knows better what's best for its customers/users.

And @janisk is well aware that his arguments don't make any sense at all, but for some reason defends them for years. If you really think that regex puts a lot of load on the router, the "~" operator should be removed from scripting as well. And then remove scripting altogether, because it can also cause 100% load, e.g. by using an infinite loop.

Don't consider your customers dumbasses, just please add a feature that is asked for.
 
legrang
just joined
Posts: 22
Joined: Wed Nov 03, 2010 4:05 pm
Location: South Africa
Contact:

Re: [API bug/suggestion] Regex in queries

Sun Jul 12, 2015 9:16 pm

This has come up again, see: http://forum.mikrotik.com/viewtopic.php?p=490459

Moore's law and the recurring questions about this should surely be an indication that this feature should be reconsidered?

Gideon
Gideon le Grange
RouterOS Java API: https://github.com/GideonLeGrange/mikrotik-java
 
User avatar
Deantwo
Member
Member
Posts: 309
Joined: Tue Sep 30, 2014 4:07 pm

Re: [API bug/suggestion] Regex in queries

Tue Jan 19, 2016 9:36 am

as i have been told some long time ago, as it is, in most cases, cheaper for router to send packets than do regexp matching, API will not have this feature. And that has to be done on client side.
Unless the scripting/CLI engine is totally separate from the API engine and work totally different, I don't see how this augment makes any sense. The person that told you this "a long time ago" may have been right at the time, but it is clearly not an issue anymore.

You are also assuming that the client is always more powerful than the router, which does not have to be true at all anymore. I think a RaspberryPi using the API would prefer the work being done on the router.

Anyway, would very much like for an update on this issue/request.
I wish my FTP was FTL.
 
chojrak11
Member Candidate
Member Candidate
Posts: 129
Joined: Sun Apr 05, 2009 10:37 am

Re: [API bug/suggestion] Regex in queries

Wed Jan 20, 2016 12:56 pm

as i have been told some long time ago, as it is, in most cases, cheaper for router to send packets than do regexp matching, API will not have this feature. And that has to be done on client side.
Unless the scripting/CLI engine is totally separate from the API engine and work totally different, I don't see how this augment makes any sense. The person that told you this "a long time ago" may have been right at the time, but it is clearly not an issue anymore.

You are also assuming that the client is always more powerful than the router, which does not have to be true at all anymore. I think a RaspberryPi using the API would prefer the work being done on the router.

Anyway, would very much like for an update on this issue/request.
Come on guys, don't play fool, do this API regex matching already. And when you decide to do it, don't use those crappy GNU Extended Regex syntax anymore, just go for the widespread PCRE-compatible syntax.
 
User avatar
boen_robot
Forum Guru
Forum Guru
Topic Author
Posts: 2411
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: [API bug/suggestion] Regex in queries

Wed Jan 20, 2016 1:11 pm

And when you decide to do it, don't use those crappy GNU Extended Regex syntax anymore, just go for the widespread PCRE-compatible syntax.
In fairness, they're already using POSIX regular expressions in scripting... Switching to PCRE is going to break a lot of scripts, and using PCRE just for the API while using POSIX for scripting is going to be confusing to people.

If the API uses POSIX regular expressions, that's enough IMHO. A switch to PCRE, if it happens at all, should be at v7, for both API and scripting.
PEAR2_Net_RouterOS(1.0.0b6) - My API client in PHP
(Rate my posts? If you want... no pressure...)
 
chojrak11
Member Candidate
Member Candidate
Posts: 129
Joined: Sun Apr 05, 2009 10:37 am

Re: [API bug/suggestion] Regex in queries

Wed Jan 20, 2016 1:14 pm

In fairness, they're already using POSIX regular expressions in scripting... Switching to PCRE is going to break a lot of scripts, and using PCRE just for the API while using POSIX for scripting is going to be confusing to people.

If the API uses POSIX regular expressions, that's enough IMHO. A switch to PCRE, if it happens at all, should be at v7, for both API and scripting.
Good points. I forgot about scripting and L7 matching. I hate POSIX regex, it's wrong on all fields. So let's vote for PCRE for RouterOS v7 :-)
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5950
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: [API bug/suggestion] Regex in queries

Wed Jan 20, 2016 3:51 pm

Please be patient, we will have some new features in ROSv7. :)
 
User avatar
boen_robot
Forum Guru
Forum Guru
Topic Author
Posts: 2411
Joined: Thu Aug 31, 2006 4:43 pm
Location: europe://Bulgaria/Plovdiv

Re: [API bug/suggestion] Regex in queries

Wed Jan 20, 2016 3:55 pm

Please be patient, we will have some new features in ROSv7. :)
Oh? Including this one?

(where's a firework emoticon when you need one...)

Excellent!!!!!!


Knowing that it would come eventually is all anyone wanted to know... As opposed to what we had until now, which was "Nope, won't have it, sorry".
PEAR2_Net_RouterOS(1.0.0b6) - My API client in PHP
(Rate my posts? If you want... no pressure...)
 
legrang
just joined
Posts: 22
Joined: Wed Nov 03, 2010 4:05 pm
Location: South Africa
Contact:

Re: [API bug/suggestion] Regex in queries

Wed Jan 20, 2016 4:07 pm

As API implementation developer (not just API user), it would be really nice if I could have some pre-release info so I can adapt the Java API and be ready to release an update when v7.0 leaves beta *hint* *hint* :D

But good news!
Gideon le Grange
RouterOS Java API: https://github.com/GideonLeGrange/mikrotik-java

Who is online

Users browsing this forum: No registered users and 29 guests