šŸ§¬ RouterOS LSP for better syntax checking & command completion in editors like VSCode & NeoVim

RouterOS Scripting
14

Open Questions & Scripting Mysteries

In developing the LSP, I have some "open issues" that limit potential features. If anyone had ideas on any of these topics, feel free to reply.

So to answer @Kentzo question:

TL;DR answer is Yes. But exactly the needed RegEx start to get complicated since it not just looking for simple keywords and tokens...

Now could be JavaScript/TypeScript code too, since that what the LSP is implemented in.

But some "static" analysis is actually need to "fill gaps" in /console/inspect data.

Specifically the following things....

  1. Regular Expression for Types?

Anyone have a list of regular expressions for RouterOS types - like "str", "array", "ip-prefix" etc?

This be useful to produce more semantic tokens in LSP with the RouterOS types. /console/inspect highlight= only get variables (and dir, cmd, etc) - not types. So this limited what "validation" can do in VSCode's "Problems" ("diagnostics" in LSP world) and LSP "missing" some script error that it could find. e.g. since some commands RouterOS LSP can know from request=syntax what the allowed types and for numbers allowed ranges (at least to some degree).

  1. Computing "cwd" is needed to validate args and provide help and more... ā€“ but hard

The LSP currently does not find commands, which is needed to use request=syntax, that allow more "schema" information for found tokens - or do thing like link to "help". The problem is while "dir" and "cmd" can be identify via request=highlight ā€“ how they relate back the "current working directory" of RouterOS is not so easy to parse out, since it computed as it goes.

An example may help explain, RouterOS often uses syntax where "dir" is on separate line, or even using { } blocks:

/ip address
set address=1.1.1.1 interface=ether5
/container {
add remote-image=ghcr.io/tikoci/netinstall interface=veth1
mount add src=/ dst=/ list=mymounts
set netinstall mounts=mymounts
 }

The problem is if, for example, use the "Hover" editor feature, and hovered over "mount" above... the LSP would need to know the path= is "/container" to lookup "explanation" for help info for mount... but that single line has no idea what the parent path is...

I'm really not sure of an algo/heuristic/rule that find the path even... So ideas on doing this be welcomed. Since determine the path is critical to using /console/inspect request=syntax ā€” but it needs to know a path=ip,address for example to do that...

I wrote up what /console/inspect request=syntax can on the readme:

But request=syntax only usable if the "current directory" is known to the LSP is the underlying.

Plan right now to at least look for for simple sequences of dir+cmd+[arg] until within a line (and until a "dir" or "cmd" is found after "cmd")

Theoretical Approach: using [:parse] and the code types...

I have thought to use the[:parse] command (via REST), but I've never written a parser for it and even less is known about it's format... But that is one idea I have in my head about this, not sure it will work... it likely has markers for variables and does know the stack. Now problem is linking editors "position" with the parse command is where this approach, even if possible, set even more tricky. i.e. these no .map file like JS has between RSC transpiled into it's :parse/"code" type.

  1. Finding :local/:global variable scope - all { } blocks are not created equal?

Another thing the LSP does not know about is scope. Again, some RegEx be useful to identify "scoped blocks" but it beyond just regex match { with another } is the problem.

And being able to do be useful since the LSP can offer a "rename variable" refactoring option. For :global, I believe, within a single script it "syntaxly safe" to just rename all found since request=highlight can identify global variables in script.

The issue is :local variable is the scope is not entire clear from the { and } usage and require way more nuance. And I'm limited by docs and while I know most of the scoping rules when I see script but reducing it a RegEx is not so easy.

If anyone had ideas on how to identify a local scope in RegEx that be very useful to the LSP development.

Some example where it's pretty clear:

{
  :local outside "outside"
  /ip address { 
    :local inside "inside"
    :put "ip seeing '$outside' (outside)"
    /interface {
        :local reallyinside "reallyinside"
        :put "my parent is '$inside', but I can see '$outside'"
        ethernet {
            :put "ethernet is '$reallyinside' (reallyinstead)"
        }
    }
  }
  :put "looking from outside i cannot see '$inside' (inside)"
}

which returns the follow, with local variable inhered into the /ip/address.

ip seeing 'outside' (outside)
my parent is 'inside', but I can see 'outside'
ethernet is 'reallyinside' (reallyinstead)
looking from outside i cannot see '' (inside)

Now once functions involved ā€“ local scope does not inherit the parents for local function. So with this:

{
    :local outside "outside"
    :local fn do={
        :put "local function can see '$outside' (outside)"
    }
    :global FN do={ 
        :put "but global cannot see '$outside' (outside)"
    }
    $fn
    $FN
}

you'd get nothing, since function do= blocks get a new "stack" AFAIK.

local function cannot see '' (outside)
nor can global see '' (outside) either

I'll probably add a "global variable rename" support from LSP. But the :local is trickier.

And I'm not sure "local variable rename" is all that useful... But just from a BNF/syntax view, it would be good to identify "local scope", somehow.