Community discussions

MikroTik App
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 794
Joined: Thu Jan 27, 2011 7:29 am

need some scripting help, bounty available

Tue Dec 06, 2022 11:37 pm

I'm not a terrible slouch at mikrotik scripting but this is eating up too much time

I'm pulling this object from a web request:

[;"id":"d66feee0-601e-4b45-8135-2fece1df4a41","identification":{"id":"d66feee0-601e-4b45-8135-2fece1df4a41","status":"active","name":"Customer Name","p
arent":null,"type":"endpoint","suspended":false,"updated":"2022-12-06T05:56:03.035Z"},"description":{"address":"30 Rockafeller Center","note":null,"contact":{"name":null,"phone":null,"email":null},"location":null,"height":null,"elevation":null,"endpoints":[],"deviceCount":6,"d
eviceOutageCount":0,"deviceListStatus":"active","ucrmId":"537","regulatoryDomain":"US","ipAddresses":["100.70.5.93","100.70.9.200","100.70.31.179","100.
70.32.190"],"sla":1},"lastSpeedReport":null,"notifications":{"type":"none","users":[]},"qos":{"aggregation":null,"downloadBurstSize":null,"downloadSpeed
":50000000,"enabled":true,"propagation":null,"uploadBurstSize":null,"uploadSpeed":20000000},"ucrm":{"client":{"id":"254","name":"customer name","isLead
":false},"service":{"id":"537","name":"50M-R","status":1,"activeFrom":"2021-11-28T00:00:00.000Z","tariffId":"24","trafficShapingOverrideEnabled":false}}
]

I need to get this output into variables like so
name=customer name
service_id=537
customer_id=254
service_name=50M-R
status=1
and array of ipaddress=100.70.5.93, 100.70.9.200, 100.70.31.179, 100.70.32.190
downloads-speed = 50000000
upload-speed = 20000000
and really all the variables mapped out, except IP that needs to be in an array so I can interate through it.


this is for a DHCP script and all in the 'local' context.

I'm happy to buy a beer for someone to hack this together for me.

Thanks.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3253
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: need some scripting help, bounty available

Wed Dec 07, 2022 12:40 am

I can get you close for with free functions. I have a library of function I use for things, so if you can use [almost] YAML output is easy. If you remove the $YAML part, the $JSON part returns an Mikrotik array from the JSON out from the the url.
:put [$YAML [ $JSON ([/tool/fetch url="https://wttr.in/Riga+LV?format=j2" output=user as-value]->"data")]]
Which outputs this (since it a nested map, the indentation happens for YAML, output is truncated for forum but converts some weather data for Riga, Lativia):
  current_condition:
    0:
      FeelsLikeC: -7
      FeelsLikeF: 19
      cloudcover: 100
      humidity: 100
      localObsDateTime: 2022-12-06 11:34 PM
      observation_time: 09:34 PM
      precipInches: 0.0
      precipMM: 0.3
      pressure: 1010
      pressureInches: 30
      temp_C: -2
      temp_F: 28
      uvIndex: 1
      visibility: 6
      visibilityMiles: 3
      weatherCode: 326
      weatherDesc:
        0:
          value: Light snow
      weatherIconUrl:
        0:
          value: 
      winddir16Point: SSE
      winddirDegree: 160
      windspeedKmph: 17
      windspeedMiles: 11
  nearest_area:
    0:
      areaName:
        0:
          value: Riga
      country:
        0:
          value: Latvia
      latitude: 56.950
      longitude: 24.100
      population: 742570
      region:
        0:
          value: Riga
      weatherUrl:
        0:
          value: 
  request:
    0:
      query: Lat 56.95 and Lon 24.11
      type: LatLon
  

The follow functions are what to that, so should be able to cut-and-paste. But recommend you understand what you're running. In particular, the script downloads a JSON library from GitHub (not mine, but used for a while) that does all the hard work to convert JSON to Mikrotik array. The last mile to YAML here is easier. You can see the YAML function, be easy enough to change if you wanted different output style.

:do {
    /tool fetch url="https://raw.githubusercontent.com/Winand/mikrotik-json-parser/master/JParseFunctions"
    :import JParseFunctions 
} on-error={:put "could not load JSON support - no internet?"}

### $JSON <string->parser | array->stringify>
:global JSON
:set JSON do={
    :global JSONLoads
    #:if ([:typeof $JSONLoads]="nothing") do={
    #    /tool fetch url=https://raw.githubusercontent.com/Winand/mikrotik-json-parser/master/JParseFunctions
    #    :import JParseFunctions
    #    :delay 5s
    #}
    :if ([:typeof $1]="str") do={
        :local got
        :if ([len [/file find name=$1]] > 0) do={
            :global JSONLoads
            :set $got [$JSONLoads [/file get $1 contents]]
            :return $got
        }
        :set $got [$JSONLoads $1]
        :return $got
    }
    :if ([:typeof $1]="array") do={
        :local tojson do={
            :local ret "{"
            :local firstIteration true
            :foreach k,v in=$1 do={
                if ($firstIteration) do={
                :set $ret ($ret . "\"".$k . "\":")
                } else={
                :set $ret ($ret . "," . "\"". $k . "\":")
                };
                :set $firstIteration false
                :local type [:typeof $v]
                :if ($type = "array") do={
                :set $ret ($ret . [$tojson $v])
                } else={
                :if ($type = "str" || $type = "id" || $type = "time" || $type = "ip" || $type = "ipv6") do={
                    :set $ret ($ret . "\"" . $v . "\"")
                } else {
                    :set $ret ($ret . $v )
                };
                };
            };
            :set $ret ($ret . "}")
            :return $ret;
        };
        :return [$tojson $1]
    }
    :error "Bug: Unhandled code path"
}

### YAML
:global YAML
:set YAML do={
    :global YAML
    :local ar $1
    :local lvl 0
    :local lines ""
    :if ([:typeof $level]="num") do={:set $lvl $level}
    :if ([:typeof $memo]="str") do={:set lines $memo}
    :local puttab do={
        :local r ""
        :for i from=0 to=$1 do={
            :set $r "$r  " 
        }
        :return $r
    }
    :foreach k,v in=$ar do={
        :local line
        :if ([:typeof $v]="array") do={
            :set line ([$puttab $lvl] . "$k:")
            :put "$line"
            [$YAML $v level=($lvl+1) memo=($lines)]
        } else={
            :set line ([$puttab $lvl] . "$k: $v")
            :put "$line"
        }
        #:if (!($2="as-value")) do={
        #}
        :set lines ("$lines\n$line")
        #:put "** line --> $lines"
        #:put ([$puttab $lvl] . "$k:\t\t\t#$([:typeof $v])")
        #:put ([$puttab $lvl] . "$k: $v\t\t#$([:typeof $v])")
    }
    :return $lines
}

:put [$YAML [ $JSON ([/tool/fetch url="https://wttr.in/Riga+LV?format=j2" output=user as-value]->"data")]]
 
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 794
Joined: Thu Jan 27, 2011 7:29 am

Re: need some scripting help, bounty available

Wed Dec 07, 2022 1:57 am

I've adapted this in and I do get yaml formatted output but now I have to deal with that.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3253
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: need some scripting help, bounty available

Wed Dec 07, 2022 3:09 am

I've adapted this in and I do get yaml formatted output but now I have to deal with that.
If you skip the YAML setup, the JSON returns a Mikroitk array so a :foreach could work, but your sample data is actually nested, why YAML has indents. Here is what the JSON from your post looks like:
[user@bigdude] > :put [$JSON myjson.txt]
description=address=30 Rockafeller Center;contact=email=;name=;phone=;deviceCount=6;deviceListStatus=active;deviceOutageCount=0;elevation=;endpoints=;height=;ipAddresses=100.70.5.
93;100.70.9.200;100.70.31.179;100.70.32.190;location=;note=;regulatoryDomain=US;sla=1;ucrmId=537;id=d66feee0-601e-4b45-8135-2fece1df4a41;identification=id=d66feee0-601e-4b45-8135-
2fece1df4a41;name=Customer Name;parent=;status=active;suspended=false;type=endpoint;updated=2022-12-06T05:56:03.035Z;lastSpeedReport=;notifications=type=none;users=;qos=aggregatio
n=;downloadBurstSize=;downloadSpeed=50000000;enabled=true;propagation=;uploadBurstSize=;uploadSpeed=20000000;ucrm=client=id=254;isLead=false;name=customer name;service=activeFrom=
2021-11-28T00:00:00.000Z;id=537;name=50M-R;status=1;tariffId=24;trafficShapingOverrideEnabled=false


[user@bigdude] > :global ja [$JSON myjson.txt]


[user@bigdude] > :foreach k,v in=($ja->0) do={:put "$[:tostr $k] = $[:tostr $v]"}
description = address=30 Rockafeller Center;contact=email=;name=;phone=;deviceCount=6;deviceListStatus=active;deviceOutageCount=0;elevation=;endpoints=;height=;ipAddresses=100.70.
5.93;100.70.9.200;100.70.31.179;100.70.32.190;location=;note=;regulatoryDomain=US;sla=1;ucrmId=537
id = d66feee0-601e-4b45-8135-2fece1df4a41
identification = id=d66feee0-601e-4b45-8135-2fece1df4a41;name=Customer Name;parent=;status=active;suspended=false;type=endpoint;updated=2022-12-06T05:56:03.035Z
lastSpeedReport = 
notifications = type=none;users=
qos = aggregation=;downloadBurstSize=;downloadSpeed=50000000;enabled=true;propagation=;uploadBurstSize=;uploadSpeed=20000000
ucrm = client=id=254;isLead=false;name=customer name;service=activeFrom=2021-11-28T00:00:00.000Z;id=537;name=50M-R;status=1;tariffId=24;trafficShapingOverrideEnabled=false

[user@bigdude] > $YAML ($ja->0)
  description:
    address: 30 Rockafeller Center
    contact:
      email: 
      name: 
      phone: 
    deviceCount: 6
    deviceListStatus: active
    deviceOutageCount: 0
    elevation: 
    endpoints:
    height: 
    ipAddresses:
      0: 100.70.5.93
      1: 100.70.9.200
      2: 100.70.31.179
      3: 100.70.32.190
    location: 
    note: 
    regulatoryDomain: US
    sla: 1
    ucrmId: 537
  id: d66feee0-601e-4b45-8135-2fece1df4a41
  identification:
    id: d66feee0-601e-4b45-8135-2fece1df4a41
    name: Customer Name
    parent: 
    status: active
    suspended: false
    type: endpoint
    updated: 2022-12-06T05:56:03.035Z
  lastSpeedReport: 
  notifications:
    type: none
    users:
  qos:
    aggregation: 
    downloadBurstSize: 
    downloadSpeed: 50000000
    enabled: true
    propagation: 
    uploadBurstSize: 
    uploadSpeed: 20000000
  ucrm:
    client:
      id: 254
      isLead: false
      name: customer name
    service:
      activeFrom: 2021-11-28T00:00:00.000Z
      id: 537
      name: 50M-R
      status: 1
      tariffId: 24
      trafficShapingOverrideEnabled: false


# As an array you can pull out the parts you need...  Again no need to use YAML, the print could work but that YAML code deals with unwinding the nesting (I normally just use it to OUTPUT a more friendly version of an array to the console)

[user@bigdude] > $YAML ($ja->0->"description")
  address: 30 Rockafeller Center
  contact:
    email: 
    name: 
    phone: 
  deviceCount: 6
  deviceListStatus: active
  deviceOutageCount: 0
  elevation: 
  endpoints:
  height: 
  ipAddresses:
    0: 100.70.5.93
    1: 100.70.9.200
    2: 100.70.31.179
    3: 100.70.32.190
  location: 
  note: 
  regulatoryDomain: US
  sla: 1
  ucrmId: 537
But sound like you want it "flat", this function can roughly do that
*recursion is the technique that makes this relatively simple, since it calls itself if it gets an array (expect the ipAddresses it skips that and uses RouterOS-style to keep on same line).
:global FLATPUT do={
  :global FLATPUT
  :foreach i,k in=$1 do={
    :if ([:typeof $k]="array") do={
      :if ($i != "ipAddresses") do={
        :put "$[:tostr $i]=(array)"
        $FLATPUT $k
      } else={
        :put "$[:tostr $i]=$[:tostr $k] ;$[:typeof $k]"
      }
    } else={
      :put "$[:tostr $i]=$[:tostr $k] ;$[:typeof $k]"
    }
  }
}
which get your data looking like this:
[user@bigdude] >  $FLATPUT ($ja->0)
description=(array)
address=30 Rockafeller Center ;str
contact=(array)
email= ;nil
name= ;nil
phone= ;nil
deviceCount=6 ;num
deviceListStatus=active ;str
deviceOutageCount=0 ;num
elevation= ;nil
endpoints=(array)
height= ;nil
ipAddresses=100.70.5.93;100.70.9.200;100.70.31.179;100.70.32.190 ;array
location= ;nil
note= ;nil
regulatoryDomain=US ;str
sla=1 ;num
ucrmId=537 ;str
id=d66feee0-601e-4b45-8135-2fece1df4a41 ;str
identification=(array)
id=d66feee0-601e-4b45-8135-2fece1df4a41 ;str
name=Customer Name ;str
parent= ;nil
status=active ;str
suspended=false ;bool
type=endpoint ;str
updated=2022-12-06T05:56:03.035Z ;str
lastSpeedReport= ;nil
notifications=(array)
type=none ;str
users=(array)
qos=(array)
aggregation= ;nil
downloadBurstSize= ;nil
downloadSpeed=50000000 ;num
enabled=true ;bool
propagation= ;nil
uploadBurstSize= ;nil
uploadSpeed=20000000 ;num
ucrm=(array)
client=(array)
id=254 ;str
isLead=false ;bool
name=customer name ;str
service=(array)
activeFrom=2021-11-28T00:00:00.000Z ;str
id=537 ;str
name=50M-R ;str
status=1 ;num
tariffId=24 ;str
trafficShapingOverrideEnabled=false ;bool
That's pretty close

Who is online

Users browsing this forum: alexantao and 27 guests