Regex search in String Issue

RouterOS Scripting
1

Hi,
I’m to write my first script in routeros.

For this script I have to find a part in a string and put it in an other variable.
I have made me a regex Filter, but I fail to convert it to the routeros regex.

This is my regex filter:

"id":"(?:\d+)","hostname":"find"

This is the string within I have to find the expression:

data={"serverrequestid":"k1x=nTjeHeSzdVh","clientrequestid":"","action":"infoDnsRecords","status":"success","statuscode":2000,"shortmessage":"DNS records found","longmessage":"DNS Records for this zone were found.","responsedata":{"dnsrecords":[{"id":"11223344","hostname":"*","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"44332211","hostname":"@","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"55667788","hostname":"@","type":"MX","priority":"10","destination":"mail.domain.com","deleterecord":false,"state":"unknown"},{"id":"88776655","hostname":"ftp","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"99001122","hostname":"mail","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"12345678","hostname":"find","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"22110099","hostname":"www","type":"CNAME","priority":"0","destination":"@","deleterecord":false,"state":"unknown"}]}}

When I use the filter (on regex101.com), this is the output from the data above:

"id":"12345678","hostname":"find"

I tried to convert this to routeros regex and put it in an script, like this:

:put [:pick $DNSRecStr [:find $DNSRecStr "\"id\"\\:\"\\~(\\?\\:d\\+\\)\"\\,\"hostname\"\\:\"find\""]] ;

But it doesn’t work, so I think I have an error in this.

2

Wrong approach. Look in this forum for how NordVPN data is extracted from a json record.

Update: link to topic
http://forum.mikrotik.com/t/script-to-change-nordvpn-server-address/151988/11

3

Thank you,
I’m going to look at it.

I’m not a programmer so I asked a friend who knows a little bit about programming, but not about routeros.

We created the following code together, but it did’t work either:

:local dnsRecords ($DNSRecStr->"responsedata"->"dnsrecords");
:local foundRecord false;
:local hostnameToFind find;

:foreach record in=$dnsRecords do={
  :if (($record->"hostname") = $hostnameToFind) do={
    :put ("Found record with hostname: " . $hostnameToFind);
    :put ("ID: " . ($record->"id"));
    :set foundRecord true;
  }
}

:if ($foundRecord = false) do={
  :put ("No record found with hostname: " . $hostnameToFind);
}
4

What is the problem you’re trying to solve? I’m not sure what DNS has to do with NordVPN… e.g. most people want to find the closest server, which there is an approach for as linked above.

RouterOS does not support JSON, so the syntax above isn’t going to work. While perhaps some regex could work, you’d have to be very exact in what want from it.

5

First you find the location of “find” in and then take that position and find again from the beginning the last "ID before that position.

Use :pick to find the location.

6

Its not about NordVPN!
It is about an API on a Server, with that I can edit DNS Records of Domains on this server.

To change a DNS Record I need it’s ID.
I can ask the Server via the API to send me all DNS Records, thats the data you can see in my first post.

Then I have so find the matching ID from the record I want to change. In this example the record (hostname) is called “find”:

{"id":"12345678","hostname":"find","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"}

So I need to get of this Data this exact Record ID “12345678”
This ID can have different length, so it can be 5 Numbers or 8 Numbers or 10 numbers…

Then I can send a change request back to the server via its API with this ID to change the record.

I hope you understand my issue better now, english is not my native language

7

You likely just need to parse the JSON. JSON parsing isn’t built in, so you could use a script to do this. There are several ways to do this, I just posted what I used here: http://forum.mikrotik.com/t/json-parsing-json-to-from-array-type/169717/1

But fundamentally it’s not that easy since you kinda need to know scripting somewhat to deal with a JSON REST API from RouterOS. You might be able to guess at right regex, but as you’d ran into…different regex need for another call & as you can see RouterOS requires some escaping of the regex, which add another element of complexity.

8

Well just to further advocate for mikrotik adding native JSON support…

Sure that’s possible… But it’s a lot of :pick and :find regardless. So for clarity, if it’s really just ONE attribute from ONE response…

:global findHostInJson do={:local s ([:find $x "hostname"]+11); :local r [:pick $x $s 65000]; :local l [:find $r "\""]; :local rv [:pick $r 0 $l]; :return "$rv"}
:global myoutput [/tool/fetch url=... as-value output=user ...]
:put [$findHostInJson ($myoutput->"data")]

While complex, less lines than loading JSON parsing function… But still not fool-proof by any means…

9

My version:

{
:local strstart ";find;"
:local data [:tostr {"id":"12345678","hostname":"find","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"}]
:put "Stored string: $data"
:local found [:find $data $strstart -1]
:put "ID for find: $[:pick $data ($found - 8) $found] "
}
10

I think the problem is /tool/fetch is returning string, so it’s not a RouterOS array… so the :tostr trick may not work. Dunno.

11

@Amm0
Your script doesn’t return anything.

@msatter
Your script does return the whole sting and doesn’t filter


I did my own investigation a little bit, and found this JSON Parser:
https://github.com/Winand/mikrotik-json-parser

When I use the String out of the example:

global content "{\"ParsedResults\": [{\"ParsedText\": \"Hello, world!\"}]}"

The Parser works!

When I use the String from the server, it returns nothing :frowning:
So I think that something with the Sttring from the Server is not ok!?

12

Glad you found a solution. FWIW, that’s what I already said :wink:.

13

No,
I didn’t found a solution.
Please red my last comment completly.

The Parser is not working on the String from the Server!
Only on the “example” String from the developer of the parser.

So I think, somethiong is wrong with the data that comes from the server

EDIT:
Is it possible, that I have to add escape characters to the sting?
And how to add these atomatic?

14

I have now found out, that I have to remove the first part of the string “data=” to get valid JSON data:

The new string looks like this:

{"serverrequestid":"k1x=nTjeHeSzdVh","clientrequestid":"","action":"infoDnsRecords","status":"success","statuscode":2000,"shortmessage":"DNS records found","longmessage":"DNS Records for this zone were found.","responsedata":{"dnsrecords":[{"id":"11223344","hostname":"*","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"44332211","hostname":"@","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"55667788","hostname":"@","type":"MX","priority":"10","destination":"mail.domain.com","deleterecord":false,"state":"unknown"},{"id":"88776655","hostname":"ftp","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"99001122","hostname":"mail","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"22110099","hostname":"find","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"12341234","hostname":"www","type":"CNAME","priority":"0","destination":"@","deleterecord":false,"state":"unknown"}]}}

this string is valid JSON, I have validatet it with jsonlint.com and parsed it on http://json.parser.online.fr/

The issue now is, that the JSON Parser on routeros (that I linked above) doesn’t parse it.

15

Why complicate things unnecessarily, “two lines” of code are enough…

Ignoring if start or not with “data”… (the " are escaped as " for simulating the read of the data)

{
:global dnsrecords “data={"serverrequestid":"k1x=nTjeHeSzdVh","clientrequestid":"","action":"infoDnsRecords","status":"success","statuscode":2000,"shortmessage":"DNS records found","longmessage":"DNS Records for this zone were found.","responsedata":{"dnsrecords":[{"id":"11223344","hostname":"*","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"44332211","hostname":"@","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"55667788","hostname":"@","type":"MX","priority":"10","destination":"mail.domain.com","deleterecord":false,"state":"unknown"},{"id":"88776655","hostname":"ftp","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"99001122","hostname":"mail","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"12345678","hostname":"find","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"22110099","hostname":"www","type":"CNAME","priority":"0","destination":"@","deleterecord":false,"state":"unknown"}]}}”

:local id 0 ; :local hostname "" ; :local type "" ; :local destination ""
:global dnsrecords [:pick $dnsrecords ([:find $dnsrecords "[" -1] + 1) [:find $dnsrecords "]" -1]]
[:parse ":global dnsrecords ; set dnsrecords [:toarray {$dnsrecords}]"]
:foreach item in=($dnsrecords->0) do={
    :set id           ($item->0)
    :set hostname     ($item->1)
    :set type         ($item->2)
    :set destination  ($item->4)
    :if ($hostname = "find") do={
        :put "id:$id, hostname:$hostname, type:$type, destination:$destination"
    }
}

:set dnsrecords
}
Obviously the example works as long as the server response remains uniform…

Output example:
id:12345678, hostname:find, type:A, destination:55.77.233.244

16

@rextended
Thank you so much! Your script is working with the data direct from the server!
On RouterOS 7.11


Yes the data should be uniform, as far as I know. Only the length of the ID could be longer or shorter.
I do some more tests now, but it's looking already very good!

17

Edit:
I found the next issue there :frowning:

When I save the ID as a variable, I can’t use it in the next querry to the server.
I changed your code like this:

{
:global dnsrecords "data={\"serverrequestid\":\"k1x=nTjeHeSzdVh\",\"clientrequestid\":\"\",\"action\":\"infoDnsRecords\",\"status\":\"success\",\"statuscode\":2000,\"shortmessage\":\"DNS records found\",\"longmessage\":\"DNS Records for this zone were found.\",\"responsedata\":{\"dnsrecords\":[{\"id\":\"11223344\",\"hostname\":\"*\",\"type\":\"A\",\"priority\":\"0\",\"destination\":\"55.77.233.244\",\"deleterecord\":false,\"state\":\"unknown\"},{\"id\":\"44332211\",\"hostname\":\"@\",\"type\":\"A\",\"priority\":\"0\",\"destination\":\"55.77.233.244\",\"deleterecord\":false,\"state\":\"unknown\"},{\"id\":\"55667788\",\"hostname\":\"@\",\"type\":\"MX\",\"priority\":\"10\",\"destination\":\"mail.domain.com\",\"deleterecord\":false,\"state\":\"unknown\"},{\"id\":\"88776655\",\"hostname\":\"ftp\",\"type\":\"A\",\"priority\":\"0\",\"destination\":\"55.77.233.244\",\"deleterecord\":false,\"state\":\"unknown\"},{\"id\":\"99001122\",\"hostname\":\"mail\",\"type\":\"A\",\"priority\":\"0\",\"destination\":\"55.77.233.244\",\"deleterecord\":false,\"state\":\"unknown\"},{\"id\":\"12345678\",\"hostname\":\"find\",\"type\":\"A\",\"priority\":\"0\",\"destination\":\"55.77.233.244\",\"deleterecord\":false,\"state\":\"unknown\"},{\"id\":\"22110099\",\"hostname\":\"www\",\"type\":\"CNAME\",\"priority\":\"0\",\"destination\":\"@\",\"deleterecord\":false,\"state\":\"unknown\"}]}}"

    :local id 0 ; :local hostname "" ; :local type "" ; :local destination ""
    :global dnsrecords [:pick $dnsrecords ([:find $dnsrecords "[" -1] + 1) [:find $dnsrecords "]" -1]]
    [:parse ":global dnsrecords ; set dnsrecords [:toarray {$dnsrecords}]"]
    :foreach item in=($dnsrecords->0) do={
        :set id           ($item->0)
        :set hostname     ($item->1)
        :set type         ($item->2)
        :set destination  ($item->4)
        :if ($hostname = "find") do={
            :global DNSid "$id"
        }
    }
:set dnsrecords
}

And I use it in the next querry, it doesn’t work:

:global DNSsubmit [/tool fetch http-method=post http-header-field="Content-Type: application/json" http-data="{\"action\": \"updateDnsRecords\", \"param\": {\"apikey\": \"$APIkey\", \"apisessionid\": \"$SessionId\", \"customernumber\": \"$Number\",\"clientrequestid\": \"\" , \"domainname\": \"$Domain\", \"dnsrecordset\": { \"dnsrecords\": [ {\"id\": \"$DNSid\", \"hostname\": \"$Hostname1\", \"type\": \"$Type\", \"priority\": \"$Prio\", \"destination\": \"$WANip\", \"deleterecord\": \"false\", \"state\": \"yes\"} ]}}}" url="https://server.com/endpoint.php?JSON" as-value output=user] ;

But when I set the ID from your script manual. It works:

:global DNSsubmit [/tool fetch http-method=post http-header-field="Content-Type: application/json" http-data="{\"action\": \"updateDnsRecords\", \"param\": {\"apikey\": \"$APIkey\", \"apisessionid\": \"$SessionId\", \"customernumber\": \"$Number\",\"clientrequestid\": \"\" , \"domainname\": \"$Domain\", \"dnsrecordset\": { \"dnsrecords\": [ {\"id\": \"12345678\", \"hostname\": \"$Hostname1\", \"type\": \"$Type\", \"priority\": \"$Prio\", \"destination\": \"$WANip\", \"deleterecord\": \"false\", \"state\": \"yes\"} ]}}}" url="https://server.com/endpoint.php?JSON" as-value output=user] ;

Do I have to convert the ID to another format?
All other Variables in the querry worked.

18

you can’t define a global variable inside a function without declare first the variable (no matter if is set to 0 or is set to “”)

:global findedID 0
{
:global dnsrecords “data={"serverrequestid":"k1x=nTjeHeSzdVh","clientrequestid":"","action":"infoDnsRecords","status":"success","statuscode":2000,"shortmessage":"DNS records found","longmessage":"DNS Records for this zone were found.","responsedata":{"dnsrecords":[{"id":"11223344","hostname":"*","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"44332211","hostname":"@","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"55667788","hostname":"@","type":"MX","priority":"10","destination":"mail.domain.com","deleterecord":false,"state":"unknown"},{"id":"88776655","hostname":"ftp","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"99001122","hostname":"mail","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"12345678","hostname":"find","type":"A","priority":"0","destination":"55.77.233.244","deleterecord":false,"state":"unknown"},{"id":"22110099","hostname":"www","type":"CNAME","priority":"0","destination":"@","deleterecord":false,"state":"unknown"}]}}”

:local id 0 ; :local hostname "" ; :local type "" ; :local destination ""
:global dnsrecords [:pick $dnsrecords ([:find $dnsrecords "[" -1] + 1) [:find $dnsrecords "]" -1]]
[:parse ":global dnsrecords ; :set dnsrecords [:toarray {$dnsrecords}]"]
:foreach item in=($dnsrecords->0) do={
    :set id           ($item->0)
    :set hostname     ($item->1)
    :set type         ($item->2)
    :set destination  ($item->4)
    :if ($hostname = "find") do={
        :put "id:$id, hostname:$hostname, type:$type, destination:$destination"
        :set findedID $id
    }
}

:set dnsrecords
}

19

Ah ok,
so I have to declare it empty?

:global data ;

and then I can use it in your script like I did or do I have to change someting on yout script?

Edit:
Oh thank you, you were faster :smiley:

20

After the run, you can use $findedID everywhere, also on another script.