Community discussions

MikroTik App
 
User avatar
Amm0
Forum Guru
Forum Guru
Topic Author
Posts: 4962
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

🧬 RouterOS LSP for better syntax checking & command completion in editors like VSCode

Thu Jun 05, 2025 3:08 am

Let's start with "What is a LSP?"
The Language Server protocol is used between a tool (the client) and a language smartness provider (the server) to integrate features like auto complete, go to definition, find all references and alike into the tool
—from https://langserver.org

As discussed in posting below, I created a "working prototype" of an LSP Server for RouterOS script (and configuration). In theory, it should be VERY similar to the CLI and "edit" command on RouterOS itself — just in any modern editor. See post #2 for some background. My intent was not on creating my own LSP for RouterOS - but here we are... And much more could be done. But at high level is will accurately do completion for RouterOS and does same validation the CLI does for syntax error (i.e. like "edit") in tools like Visual Studio Code and NeoVim.

Image


I committed the code discussed in post #3 below to GitHub:

https://github.com/tikoci/lsp-routeros-ts

and if any wanted to contribute to it, feel free to discuss below or submit a "pull request" with any purpose fixes/changes to GitHub

It's written in TypeScript and based on Microsoft's LSP example codes. I've mainly tested it VSCode, but did see it working in NeoVim (nvim). The GitHub project tikoci/lsp-routeros-ts readme has WAY more details on installation and usage, as well as setuping for further development.

While it is not "published" as extension to Microsoft or lang-server.org... To use it in VSCode, it's pretty easy...
  1. Download lsp-routeros-ts.vsix file from https://github.com/tikoci/lsp-routeros-ts/releases – you may have to unzip it after download
  2. Install using Terminal: code --install-extension ~/Downloads/lsp-routeros-ts.vsix - adjust path as needed
  3. Launch VSCode using "code" at terminal (or however)
  4. Select "Extensions" from left most panel, then search and select "RouterOS LSP" - if installed correctly above, it should show up
  5. In main panel on right-side, with "RouterOS LSP" shown, hit the ⚙️ then pick "Settings"
  6. In the Settings, you'll need to specify a router and credentials to use as "backend" to do validations. The provided account only need read,api,rest-api permissions – do not use a "full" admin account – the LSP just need be able to call the read-only "/console/inspect...". The readme has more details on configuration.
  7. The settings should take effect right away (although restarting VSCode cannot hurt) and the LSP will process any *.rsc file that's opened in VSCode. With usual ⌘+Space bring up any "completions" (like CLI would use tabs) in any *.rsc file. You can also select "RouterOS" in bottom right status bar to "force" a file to be processed by the lSP — only files ending in .rsc will automatically load the RouerOS LSP
Image
Image

NeoVim is more complex - but it works. Again the readme on GitHub has more details, especially on setup. With the "RouterOS internals" of how it works discussed more in post 2/3 below — with some quick GIF of the LSP "in action".

I'm sure there are thing broken and MUCH more work could be done. And some known bugs.... I kinda just want to get to some "strawman" with all the needed plumbing to "run" the LSP code (i.e. while there dozens of files, only TWO are the actual code of a few hundred lines).

But if you do try it out, and have a problem feel free to post below (or file issue in GitHub). I cannot promise anything — since I kinda want to more "test the concept" than maintain an LSP.
Last edited by Amm0 on Wed Jun 11, 2025 12:47 am, edited 5 times in total.
 
User avatar
Amm0
Forum Guru
Forum Guru
Topic Author
Posts: 4962
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: Discussion: /console/inspect approach to an LSP for better external config/scripting editing

Sun Jun 08, 2025 5:55 pm

Background

This topic recently arose from a commentary between @Kentzo and me in the 7.20beta thread concerning an "LSP". While many folks may not know what an LSP is... it is an API that most modern text editors use (like VSCode) to do, among other things, syntax checking, and coloring syntax. See https://lang-server.org for more info.

While there are RouterOS code editor plugins, including @Kentzo's, these all use regular expressions. This works...but using regex is not going to 100% accurate. And an LSP can do WAY more things with script/config... depending on how LSP is implemented. For example, LSP can do stuff like "Go to definition" to find where variables are declared, and more (see https://microsoft.github.io/language-se ... geFeatures for possible operations)

Although I've thought about it... I don't have time to actually develop a RouterOS LSP - the initial "plumbing" to create an LSP project framework is more work than I want to take on... But there are plenty of videos on YouTube on how to create an LSP from scratch & imagine LLMs be pretty good at JavaScript and LSP – since 10000000x more common than RouterOS syntax. Plus some RouterOS LSP likely be a useful step to anyone attempting to develop an MCP for AI agents.

So did want to expand on HOW ONE COULD develop a RouterOS LSP using the undocumented /console/inspect request= commands that are built to RouterOS (and thus available via CLI/SSH/REST/API), so that part is possible if one wanted to create a RouterOS LSP. The only downside of using /console/inspect is a future RouterOS LSP needs to use REST and connect to an online router to work, while most LSP work "offline" since LSP are typically embedded/included with a compiler/interpreter. Now the nice part of using /console/inspect as source of LSP code editor is that it be 100% accurate at highlighting errors specific to the version/packages on the "LSP connected" RouterOS device.

Specifically, /console/inspect takes a "request" parameter. This is similar to LSP model where code editor makes requests, and the LSP responds with the metadata for the LSP request. RouterOS "inspect" supports a few "requests" that help in an LSP. Specifically:
/console/inspect request=<tab>
child     completion     error     highlight     self     syntax   
Of main interest to a future LSP are "request=highlight" and "request=completion". While there are dozens of things an LSP can provide to a code editor about a language like RouterOS config/script, "highlight" and "completion" map directly into the LSP APIs.

So request=highlight input=<script/config/line> corresponds with the "Document Highlight" LSP API. RouterOS provides the following data which can "feed" the LSP:
:put [:serialize to=json [/console/inspect request=highlight input="/ip/address/add interface=ether1 address=1.1.1.1/24" as-value ]]
{"highlight":["dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","cmd","cmd","cmd","none","arg","arg","arg","arg","arg","arg","arg","arg","arg","syntax-meta","none","none","none","none","none","none","none","arg","arg","arg","arg","arg","arg","arg","syntax-meta","none","none","none","none","none","none","none","syntax-meta","none","none"],"type":"highlight"}
and via REST it already be JSON.

Each of the elements in the highlight array corresponds to each character in the input= so the above was for the command:
  /ip/address/add interface=ether1 address=1.1.1.1/24
# ddddddddddddCCC-aaaaaaaaaM-------aaaaaaaM-------M--
The values returned from request=highlight mostly match what /terminal/style so it provides EXACTLY the same colors as RouterOS UIs. So some possible values for highlight's array are:
/terminal/style <tab>
ambiguous     comment     error     escaped     none     syntax-meta     syntax-noterm     syntax-old     syntax-val     varname     varname-global     varname-local     style   
And there is also one/most additional ones. Specifically "obj-inactive" and "syntax-obsolete" that I know. Here are some examples of invalid commands provided:
:put [:serialize to=json [/console/inspect request=highlight input="/ip/address bad-param" as-value ]]
{"highlight":["dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","none","obj-inactive","obj-inactive","obj-inactive","obj-inactive","obj-inactive","obj-inactive","obj-inactive","obj-inactive","obj-inactive"],"type":"highlight"}
:put [:serialize to=json [/console/inspect request=highlight input="/ip/address add bad-param" as-value ]]         
{"highlight":["dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","none","cmd","cmd","cmd","none","error","none","none","none","none","none","none","none","none"],"type":"highlight"}
:put [:serialize to=json [/console/inspect request=highlight input="/ip/address add address " as-value ]]
{"highlight":["dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","none","cmd","cmd","cmd","none","arg","arg","arg","arg","arg","arg","arg","syntax-obsolete"],"type":"highlight"}
:put [:serialize to=json [/console/inspect request=highlight input="/ip/address add address=" as-value ]]  
{"highlight":["dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","none","cmd","cmd","cmd","none","arg","arg","arg","arg","arg","arg","arg","syntax-meta"],"type":"highlight"}
:put [:serialize to=json [/console/inspect request=highlight input="/ip/address add address2=" as-value ]]
{"highlight"":["dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","dir","none","cmd","cmd","cmd","none","error","none","none","none","none","none","none","none","none"],"type":"highlight"}

And same request=highlight can be used to VALIDATE RouterOS config since CRITICALLY an element with "error" or "obj-inactive" in highlight's array means the script will NOT work and has, well, a syntax error (at the array index where "error"/"obj-inactive" was found in the array matches the char position of the command/script/config provided in inspect's input= attributed used to get the highlights).

"request=highlight" supports multi-line input, so you can check an entire script with newlines/spaces/etc & look for the "error states" in results from "/console/inspect request=highlight input=<script/config/line>" were found. If none were found, then the syntax is at least valid (although may not work).

The other /console/inspect thing that's useful is request=completion. Similarly, it takes an input= with the script/config to test. With the LSP API having a similar "Text Document Completion" capability, so if editor requests completion, RouterOS can make SAME suggestion as <tab> in CLI does - but in any LSP support editor. See https://microsoft.github.io/language-se ... completion. This one is more complex, and IDK exactly the meaning of all the field, although, "preference"/"show" likely used to determine if something merely valid/possible syntax, or if should be used in <tab> completion.

For example, request=completion returns stuff like:
/console/inspect request=completion input="/ip/address/add interface=ether1 address=1.1.1.1/24"
Columns: TYPE, COMPLETION, STYLE, OFFSET, PREFERENCE, SHOW, TEXT
TYPE        COMPLETION  STYLE        OFFSET  PREFERENCE  SHOW  TEXT                                                                                 
completion  <value>     none             49  -1          no    literal value that consists only of digits, letters and characters -.,:<>/|+_*&^%#@!~
completion              none             51  80          no    whitespace                                                                           
completion  ;           syntax-meta      51  40          no    end of command                                                                       
/console/inspect request=completion input="/ip/address/add interface=ether1 address=1.1.1.1/24c"
Columns: TYPE, COMPLETION, STYLE, OFFSET, PREFERENCE, SHOW, TEXT
TYPE        COMPLETION  STYLE        OFFSET  PREFERENCE  SHOW  TEXT                                                                                 
completion  <value>     none             49  -1          no    literal value that consists only of digits, letters and characters -.,:<>/|+_*&^%#@!~
completion              none             52  80          no    whitespace                                                                           
completion  ;           syntax-meta      52  40          no    end of command          

# and adding a <space> at end of input= does show the attributes that are possible                                                             
/console/inspect request=completion input="/ip/address/add interface=ether1 address=1.1.1.1/24 " 
Columns: TYPE, COMPLETION, STYLE, OFFSET, PREFERENCE, SHOW, TEXT
TYPE        COMPLETION  STYLE        OFFSET  PREFERENCE  SHOW  TEXT                                   
completion  !           none             52          80  no    whitespace                             
completion  broadcast   arg              52          95  yes   Broadcast address                      
completion  comment     arg              52          96  yes   Short description of the item          
completion  copy-from   arg              52          96  yes   Item number                            
completion  disabled    arg              52          96  yes   Defines whether item is ignored or used
completion  netmask     arg              52          95  yes   Network mask                           
completion  network     arg              52          96  yes   Network prefix                         
completion  ;           syntax-meta      52          40  no    end of command

While perhaps other /console/inspect things might be useful to a future LSP, the above would get someone pretty far in creating an LSP for RouterOS. And, my existing tikoci/restraml project on GitHub make HEAVY use of request=child (to determine the AST/"command tree") and request=syntax (which, sometimes, has help or the valid values of an attribute) to generate RouterOS RAML/OpenAPI schema. For example, request=syntax using similar command as above tells you valid values for LAST attribute in input= (so address=):
/console/inspect request=syntax input="/ip/address/add interface=ether1 address=1.1.1.1/24"
Columns: TYPE, SYMBOL, SYMBOL-TYPE, NESTED, NONORM, TEXT
TYPE    SYMBOL     SYMBOL-TYPE  NESTED  NONORM  TEXT                     
syntax  Netmask    definition        0  no      IpNetmask | Num          
syntax  IpNetmask  definition        1  no      A.B.C.D                  
syntax  Num        definition        1  no      0..32    (integer number)
but this get more complex to "map" to the LSP API IMO. Plus "/console/inspect request=syntax" data is NOT actually complete as to "valid values" (shown in text= value). And there is a path= option to request=syntax, that isn't so easy to map from LSP code... but it does have what F1 help would return:
/console/inspect request=syntax path=ip,address,add                                        
Columns: TYPE, SYMBOL, SYMBOL-TYPE, NESTED, NONORM, TEXT
TYPE    SYMBOL     SYMBOL-TYPE  NESTED  NONORM  TEXT                                   
syntax             collection        0  yes                                            
syntax  address    explanation       1  no      Local IP address                       
syntax  broadcast  explanation       1  no      Broadcast address                      
syntax  comment    explanation       1  no      Short description of the item          
syntax  copy-from  explanation       1  no      Item number                            
syntax  disabled   explanation       1  no      Defines whether item is ignored or used
syntax  interface  explanation       1  no      Interface name                         
syntax  netmask    explanation       1  no      Network mask                           
syntax  network    explanation       1  no      Network prefix  

Anyway, thought I'd share what I know since the topic came up & all of this is undocumented and potentially underutilized.
Last edited by Amm0 on Tue Jun 10, 2025 12:33 am, edited 1 time in total.
 
User avatar
Amm0
Forum Guru
Forum Guru
Topic Author
Posts: 4962
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: Discussion: /console/inspect approach to an LSP for better external config/scripting editing

Mon Jun 09, 2025 10:39 am

Initial Implementation Notes

I tried a TypeScript-based sample LSP server from Microsoft, and added REST calls to /console/inspect highlight and completion based on LSP client requests.
Just a bare-bone, hardwire, demo at this point... But concept does work at least to see something in VSCode. You can see it's pull the real interfaces= from RouterOS, and knows about bad syntax from "highlight" (which are "Problems" in VSCode terms).

Image

I disabled any RouterOS plugin, so graphic is using ONLY my experimental LSP (with minimal VSCode extension in same code to cause LSP to be used).

Now where the concept start leave you hanging is type definitions/constaints. For example, ":beep" takes frequency= and duration= - both take "num" types, but "inspect's completion" get you some text describing what allow (and it NOT always consistent):
{
    completion: '<value>',
    offset: '16',
    preference: '-1',
    show: 'false',
    style: 'none',
    text: 'literal value that consists only of digits, letters and characters -.,:<>/|+_*&^%#@!~',
    type: 'completion'
  }
The "inspect highlight" will catch errors, but even CLI does NOT enforce those as "num" (and waits for error), so at some level at least on par with CLI.

Also, I didn't deal with coloring – there are actually a few ways to handle them - so skipped it for now*. And the existing RouterOS editor plugins do most of that pretty well. * Colors come from either being explicitly setting color on specific text range (which is hackish, since colors would be fixed), or would require mapping "highlight" token to LSP's known tokens (which is not one-to-one nor perfect aligned & more complex).

It actually be a lot more work to package up even my LSP sample+REST-API code... One problem is SOMEHOW the connection to RouterOS needs to be configured (include username/password) – which is not something LSP typically do & opens up WHERE to store username/password. I have them hardcoded for testing. Additionally, it not clear how to know if REQUIRE attributes are provide – that I don't know. But it is sum of all the "little things" that make this a bit more tricky than just GIF to be usable by others.

Supports Neovim

And did a test tonight, also works in neovim. This one show it's actually using 7.20beta2 commands from the connect router... since `nvim` knows about EVPN:

Image

I'll checkin the code I have tomorrow, even if it's far from complete.
Last edited by Amm0 on Tue Jun 10, 2025 12:35 am, edited 1 time in total.
 
User avatar
Amm0
Forum Guru
Forum Guru
Topic Author
Posts: 4962
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: Discussion: /console/inspect approach to an LSP for better external config/scripting editing

Tue Jun 10, 2025 12:31 am

Code is posted on GitHub and VSIX file can be downloaded from there with the RouterOS LSP for VSCode. See post #1 and README.md on GitHub

Image


At this point, it should not break anything – since it just reads routeros & can be easily disabled/removed from VSCode. And does a few common "LSP things" - at least to some degree:
  • completion (like "tab completion")
  • semantic tokens (for colorization and based on "highlights" from /console/inspect)
  • diagnostics (shown in a few places, mainly "Problems" in VSCode terms)
  • configuration (required since it controls the connection to RouterOS used by LSP)

Feel free to post any problems, questions, or suggestions with the RouterOS LSP below....