Community discussions

MikroTik App
 
User avatar
Amm0
Forum Guru
Forum Guru
Topic Author
Posts: 3510
Joined: Sun May 01, 2016 7:12 pm
Location: California

$YAML - convert arrays to pretty YAML-formated text

Thu Sep 14, 2023 7:36 pm

I created a function long ago that takes an array and converts it to YAML. YAML has the benefit of BOTH being readable in console & still parsable into other formats like JSON etc.
I recently made some updates to it, so thought I share since I use my $YAML function a lot. Code is below, but example might be better.

More info on YAML is here:
https://yaml.org - which is actually written in valid YAML
YAML spec is here: https://yaml.org/spec/


If you have a complex array like this:
:global myarray {
    "system"=[/system resource print as-value];
    "ipaddr"=[/ip address print as-value];
    "iproutes"=[/ip route print as-value];
    "ipconns"=[/ip firewall connection print as-value]}

and then try to ":put" it the console, you get the output below...noting that it's hard to read & and if you put that whole $myarray...impossible to read:

:put ($myarray->"system")       

architecture-name=arm;bad-blocks=0;board-name=RB1100AHx4 Dude Edition;build-time=Sep/13/2023 06:58:09;cpu=ARM;cpu-count=4;cpu-frequency=1400
;cpu-load=25;factory-software=6.41.3;free-hdd-space=70254592;free-memory=864948224;platform=Mikrotik;total-hdd-space=134479872;total-mem
ory=1073741824;uptime=1d02:15:38;version=7.12beta7 (development);write-sect-since-reboot=1791;write-sect-total=1624637

withe $YAML function below loaded, you can replace the :put with $YAML to get more readable output:

$YAML ($myarray->"system")

architecture-name: arm
bad-blocks: 0
board-name: RB1100AHx4 Dude Edition
build-time: "Sep/13/2023 06:58:09"
cpu: ARM
cpu-count: 4
cpu-frequency: 1400
cpu-load: 25
factory-software: 6.41.3
free-hdd-space: 70254592
free-memory: 864948224
platform: Mikrotik
total-hdd-space: 134479872
total-memory: 1073741824
uptime: 1d02:15:38
version: 7.12beta7 (development)
write-sect-since-reboot: 1791
write-sect-total: 1624637

My $YAML function now supports a "file=", which wraps the more complex writing of large files inside the function (now it has to use another :global to support files, but no way to avoid...):

$YAML $myarray file=sample-yaml
/file print where name~"sample-yaml"

Columns: NAME, TYPE, SIZE, CREATION-TIME
# NAME TYPE SIZE CREATION-TIME
0 sample-yaml.txt .txt file 54.5KiB 2023-09-14 09:17:01

And the same "pretty" output should be parsable in most programming editors, and parsable from most languages, using the above file.


$YAML function code

To use, add to scheduler at startup to load automatically.
:global YAML
:global "temp-yaml-cache"
:set  "temp-yaml-cache" [:toarray ""]
:set YAML do={
    :global YAML
    :local ar $1
    :if ([:typeof $file]="str") do={
        :local cacheid [:rndstr length=16 from=ABCDEFGHJKLMNPQRSTUVWXYZabcdefghjkmnpqrstuvwxyz23456789]
        :set ($"temp-yaml-cache"->"$cacheid") $ar 
        :execute file=$file script=":global YAML; \$YAML (\$\"temp-yaml-cache\"->\"$cacheid\"); :set (\$\"temp-yaml-cache\"->\"$cacheid\")"
        :if ($console~"yes|1|true") do={} else={ :return [:nothing] }  
    }
    :local lvl 0
    :if ([:typeof $level]="num") do={:set lvl $level}
    :local puttab do={
        :local r ""
        :if ($1=0) do={:return ""}
        :for i from=1 to=$1 do={
            :set $r "$r  " 
        }
        :return $r
    }
    :foreach k,v in=$ar do={
        :local line
        :if ([:typeof $v]="array") do={
            :local arrmark "$k:"
            :if ([:typeof "$k"]="num") do={:set arrmark "-"}
            :set line ([$puttab $lvl] . $arrmark )
            :put "$line"
            :if ([:len $v]>0) do={
                [$YAML $v level=($lvl+1)]
            }
        } else={
            :local lv $v
            :if (([:typeof $lv]="str")) do={
                if ($lv~"[\\{\\}\\,\\*#\\\?|\\-\\<\\>\\!\\%@\\\\\\:\\&]") do={
                    :if ($lv~"[\"]") do={
                      :set lv "'$lv'"  
                    } else={
                        :set lv "\"$lv\""
                    }
                } else={
                    :if ($lv~"[']") do={
                      :set lv "\"$lv\""  
                    } else={
                      :if ($lv~"[][]") do={
                        :set $lv "'$lv'"
                    } else={
                        :set lv $lv                
                      }
                    }
                }
                :if ([:tostr $lv]="") do={:set lv "\"\""}
            }
            :if ([:typeof $k]="num") do={
                :set line ([$puttab $lvl] . "- $[:tostr $lv]")
            } else={
                :set line ([$puttab $lvl] . "$[:tostr $k]: $[:tostr $lv]")
            }
            :put "$line"
        }
    }
    :return [:nothing]
}


Implementation Notes:
  • The string escaping pretty good, but not 100% (e.g. some strings may not get escaped properly). Script tries to leave strings UNQUOTED, since YAML allows that, and it's more readable. If needed, it should use " or '. There are corner cases where string escaping in $YAML isn't going to catch however.
  • RouterOS "nil" should use "null" — $YAML takes the approach to make that an empty string. This was a stylist choice, since YAML's "null" everywhere made it look like a value was set.
  • I'm told RouterOS v6 is dead, but other than [:rndstr] it should work on V6...

Comments, improvements, or suggestions are welcomed.
Last edited by Amm0 on Thu Sep 14, 2023 8:30 pm, edited 12 times in total.
 
User avatar
Amm0
Forum Guru
Forum Guru
Topic Author
Posts: 3510
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: $YAML - convert arrays in pretty YAML-formated text

Thu Sep 14, 2023 7:49 pm

Critical note: $YAML is a one-way street — it's ONLY RouterOS array TO YAML. There is NOT a YAML file to RouterOS array function.
e.g. parsing YAML is WAY harder than generating...



More Examples...

RouterOS Arrays can be "lists" or "maps"*, and also nested & luckily YAML can also express these differences.
* not sure if there Mikrotik-specific terms — but there are two different kinds of arrays


SO if array is "list" type (e.g. you use the ($arr->0) syntax) , it gets express as "-" in YAML:
$YAML [/container/print as-value]

-
  .id: *38
  arch: arm
  comment: ""
  dns:
  interface: veth-cligames
  logging: true
  mounts:
  name: a5f2ec22-e8fe-499e-8c2d-67de9807bd2d
  os: linux
  root-dir: raid1-part1/wg3
  status: stopped
  tag: "ghcr.io/tikoci/wargames:main"
  workdir: /app/
-
  .id: *38
  arch: arm
  comment: ""
  dns:
  interface: veth-cligames
  logging: true
  mounts:
  name: a5f2ec22-e8fe-499e-8c2d-67de9807bd2d
  os: linux
  root-dir: raid1-part1/wg3
  status: stopped
  tag: "ghcr.io/tikoci/wargames:main"
  workdir: /app/

If it a "map" type, then it will use the value names as the keys. The YAML looks is in this format:
ip,address,set,numbers:
  syntax:
    nested: 0
    nonorm: false
    symbol: ""
    symbol-type: definition
    text: ""
    type: syntax
  type: arg

And $YAML should deal with them being mixed and nested in a large, nested array too:
ip,address,set,interface:
  syntax:
    -
      nested: 0
      nonorm: false
      symbol: Interface
      symbol-type: definition
      text: ""
      type: syntax
    -
      nested: 1
      nonorm: false
      symbol: ""
      symbol-type: definition
      text: "ether1 | ether2 | ether3 | ether4 | ether5 | ..."
      type: syntax
    -
      nested: 2
      nonorm: false
      symbol: back-to-home-vpn
      symbol-type: explanation
      text: back-to-home-vpn
      type: syntax
  type: arg
 
User avatar
Sertik
Member
Member
Posts: 435
Joined: Fri Sep 25, 2020 3:30 pm
Location: Russia, Moscow

Re: $YAML - convert arrays to pretty YAML-formated text

Thu Sep 21, 2023 9:59 am

Thank you very much to the author for this useful feature.
I wanted to ask how YAML differs from the JParsePrint function from the Chupacabra JSON parser set ?
:global fJParsePrint
:if (!any $fJParsePrint) do={ :global fJParsePrint do={
  :global JParseOut
  :local TempPath
  :global fJParsePrint

  :if ([:len $1] = 0) do={
    :set $1 "\$JParseOut"
    :set $2 $JParseOut
   }
   
  :foreach k,v in=$2 do={
    :if ([:typeof $k] = "str") do={
      :set k "\"$k\""
    }
    :set TempPath ($1. "->" . $k)
    :if ([:typeof $v] = "array") do={
      :if ([:len $v] > 0) do={
        $fJParsePrint $TempPath $v
      } else={
        :put "$TempPath = [] ($[:typeof $v])"
      }
    } else={
        :put "$TempPath = $v ($[:typeof $v])"
    }
  }
}}
 
User avatar
Amm0
Forum Guru
Forum Guru
Topic Author
Posts: 3510
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: $YAML - convert arrays to pretty YAML-formated text

Thu Sep 21, 2023 5:05 pm

Not much, both use recursion to parse RouterOS arrays. This produce almost-standard YAML format that used in some config system. JSON won the standards wars but YAML still a nicer format.

The "Chupacabra Print" is pretty similar except it uses a non-standard format & shows types — but does solve similar "making arrays readable".

Who is online

Users browsing this forum: No registered users and 15 guests