10 years of MikroTik RouterOS scripting, and my .rsc editor is still Notepad. What do you use?

Very Nice ... Thank You!

Where is HomeSite and / or notepad only vote?
(and the question, cant' be an option...)
:sweat_smile:

Sublime Text for its ongoing support, and slow release cycle (quality). Available for Linux, Mac, and Windows.

Syntax highlight for Sublime Text by @Kentzo does the job, although some keywords are missing.

Sublime Text does not have a native LSP (Language Server Protocol) support but it appears that the limitation can be overcome with the help of third party packages. I have not tried it yet, but if that's the case, then Amm0’s RouterOS LSP technically should work with Sublime Text.

For a monitor I only use a single screen with a 3:2 ratio Benq RD280UA. It is large enough to fit several windows comfortably on a single plane. It takes less space on a desk than several monitors.

I've been working on "offline" checking in under-development tikoci/centrs. It's broader than just offline checking since I'm building as TypeScript-Llibrary/CLI/MCP that wraps running commands around the syntax checking (and policy) as a more AI-friendly+safe way for agent to run commands. But the explain operation in centrs actually parse RouterOS script structure, so blocks/"symbols"/paths/operator/etc are identified.

I have not pulled this work in RouterOS LSP yet, but that's idea one day. And when it more complete, my idea is to package centrs's offline explain into a proper highlight.js plugin to improve the RouterOS syntax coloring for things like Discourse.

I'll offer that having the live router is really the only true way to validate the syntax, since syntax does depending on packaging installer/architecture/version. But it is also limited to some extent too: you can only really get colors and first error (if any), so the "online" check is some ways limited too. And while there are many things that could be caught offline, but without knowing the device's exact details, any offline mode be limited.

So my underlying idea is the offline lex'er pulls out the structural elements like blocks, (to enable some "Format Script" option), as well as "path awareness" so that there is map to script file line/col into the active script path. The tracking of RouterOS "cwd" is both tricky and really important since if you want to wire in help text (say from the new manual.mikrotik.com) or be able to "hover" over an attribute to see it's description... it has to know it's parent path. e.g. in simple case like /ip/address { add interface=..., centrs has to track interface is associated with /ip/address, which is needed to locate some future help text for /ip/address/*/interface.

Anyway, point being getting some help text is missing from all code editor approaches today (including my RouterOS LSP). But with the better structured docs from MikroTik, this is more possible, but not easy. So no upgrade from notepad.exe is going to get help links. Colors and perhaps the first error (if any) is what you'd get today.

Also it's worth noting running a parse command on potential script is a quick way to get verdict if a script is correct, since you'll get some evl(...) back, or an error. But this does not help the offline case.

@amm0 Nice timing ) I've been coming at this from the offline side myself for a while. Nothing to show yet.
Your packaging/arch/version point is the real wall. That's where the work is.

@Amm0, your approach deserves respect, especially the amount of personal currency you have invested in the project. I tried to understand it, but it is beyond my neural network's ability to handle the complexity of it. My brain quickly runs out of available swap space. :slightly_smiling_face:.

As a side note, may I recommend TeraTerm for a terminal, an amazing tool.

(amazed nobody has said EMACS yet . . . . )

Some variation of vi here (not sure whar ssh has to do with it thoug, in the choice . . . )

Mostly I seem to create in WebFig/WinBox, maintain in WebFig, and tinker with exusting configs in vi.

I wonder if you could use all that work to generate simple LSP-less, offline, "old-school" syntax grammar for VSCode. That author of MikrotikScript for Sublime Text didn't bother to update the extension in years!

Maybe. At some level it actually become opinion about how much static data to bring into the offline syntax checking...

For the static data part it's based on selected data from @mrz's "cliref" (clunker's shorthands for CLI Reference | RouterOS Manual). So some of centrs heuristics are designed around the available data from MikroTik's "cliref" data. And that that give you is a list of paths and attributes, along with required package/arch data. So my current centrs explain offline syntax analyzer make a few assumption like:

  • top-level "dirs" that do not have commands are always stored in code for offline. e.g. /ip, /system are path identification tokens, and presumed not to have verbs.
  • The "CRUD" operators are presumed available on a path-like object, so get/set/print/add/remove
  • The "irregular verbs" are also included in centrs

Anyway, it been tricky to balance how much can be deduced from parsing logic, vs where some static/saved tree/AST is needed. centrs is still very much under-development, and made more complex since I've tried to design it so there is "progressive truth" (e.g. "look like routeros" —> parses as RouterOS —> path/cmd/attribute match small set of critical path/verbs —> (optional) uses saved "inspect.json/deep-inspect.json data from files or Rosetta's SQLite with versioned-history (which is 100MB+) -> live router using /console/inspect and/or :put [:parse <script-to-test>' as final authority)

Current it does stuff, but still working through all the syntax cases (among many, e.g. :local "var-with-spaces"; :put "$"var-with-spaces"" kinda stuff). The offline explain has richer data available from NPM library and/or --json. But to get the gist of what's coming back "offline" today, here are some "teasers":

$ bun run src/cli.ts explain --tokens '/ip { address add interface=ether1 }'

verdict: pass — 2 statement(s), 0 diagnostic(s)
gate: script · 0 arg(s)
write: true
statements:
  [0,3)        resolved  menu     path=/ip
  [6,34)       resolved  command  path=/ip/address verb=add  via=api-candidate
    centrs: centrs api '<router>' '/ip/address' -X PUT -f 'interface=ether1'
values:
  [28,34)      attribute  name=interface shapes=str
tokens: 12 token(s), 31/36 byte(s) classified (86.1%), class provisional
  [0,3)        dir
  [3,4)        unclassified
  [4,5)        brace
  [5,6)        unclassified
  [6,13)       dir
  [13,14)      unclassified
  [14,17)      cmd
  [17,18)      unclassified
  [18,28)      arg
  [28,34)      value
  [34,35)      unclassified
  [35,36)      brace
evidence: e0 canonicalizeExecuteCommand (direct), e1 analyzeCoordinates (direct), e11 argSpans (heuristic), e12 pathSpans (heuristic), e14 braceSpans (heuristic), e4 resolveVerbs (heuristic), e6 containsWrite (heuristic), e8 classifyExplainTransport (heuristic), e9 valueShapeHints (heuristic)
runtimeAcceptance: not-proven

Tips:
  - [tip/explain-offline-only] No router was given, so this is the canonicalizer's reading alone.
    fix: Completion, schema, and `:parse` evidence come from a live target (`centrs explain <router> '<input>'`), which is phase 2 and not accepted yet.

Another example:

$ bun run src/cli.ts explain --tokens '/ip dhcp-client renew [find name=dhcp1]'

verdict: pass — 1 statement(s), 0 diagnostic(s)
gate: script · 0 arg(s)
write: unknown
statements:
  [0,39)       resolved  command  path=/ip/dhcp-client verb=renew  via=execute
    centrs: centrs execute '<router>' '/ip dhcp-client renew [find name=dhcp1]'
subcommands:
  [22,39)      resolved  command  path=/ip/dhcp-client verb=find  (in /ip/dhcp-client, depth 0)
symbols:
  [28,32)      unknown   field       name="name"  (S8 filter-field: schema-dependent (local if a real menu field, undefined if not))
tokens: 8 token(s), 23/39 byte(s) classified (59.0%), class provisional
  [0,3)        dir
  [3,4)        unclassified
  [4,15)       dir
  [15,16)      unclassified
  [16,21)      cmd
  [21,23)      unclassified
  [23,27)      cmd
  [27,39)      unclassified
evidence: e0 canonicalizeExecuteCommand (direct), e1 analyzeCoordinates (direct), e12 pathSpans (heuristic), e4 resolveVerbs (heuristic), e5 resolveDocument + resolveVerb (heuristic), e6 containsWrite (heuristic), e7 resolveSymbols (heuristic), e8 classifyExplainTransport (heuristic)
runtimeAcceptance: not-proven

You see the glimmer are the data that possible, like it indirectly providing the "REST-able" commands, either using "pure" GET/PUT/..., or using a POST /rest/execute {"source": ...} if you have complex syntax. It also finds "symbols" so variables can be matched between declaration<>assignment<>usage, including the RouterOS type.

It a good question. I have not come back to the question, since the big limitation on more editor features for RouterOS is knowing the path/verb/attributes. When more done with the offline explain, I have to imagine a better "old-school" TextMate-like grammar file is possible. In VSCode, this be a language-configuration.json file, which the LSP-less path to syntax things which borrows from the orginal TextMate "standard".

The underlying issue is RouterOS script is stack-based, so the ability of RegExp "look-back"s still seems tricky business even with LLM agents. Although, I'm curious how well it could "port" centrs's TypeScript to a language-configuration.json. For Sublime, I have not studied it but either port centrs's explain code to Python and/or shelling out to centrs explain --tokens --... <script-in-editor> in some Sublime plug could also work to deal with your loyal Sublime user base :).

But the overall "roadmap", is that the pure RouterOS LSP (which works outside VSCode) gains the ability to do some work without a connected router. And the LSP interface is now pretty standard, it works most places (including emacs, LSP Mode - Language Server Protocol support for Emacs - LSP Mode - LSP support for Emacs).

But in my current plans, the "final test" of the code in centrs explain (offline) would be a [highlight.js] language provider, since, in theory, that should be possible by the current design to provide tokens back to highlightjs. And with highlighjs support, all the markdown rendering (like here), would at least show proper colors. So static regex-based grammar are a bit lower on the list (and likely benefit if/when I try to spin the centrs code for RouterOS syntax into highlightjs since that flush things out I suspect).

To come back to the OP's topic... 10 years of MikroTik RouterOS scripting, and my .rsc editor is still Notepad. What do you use?

I obviously use VSCode. And my extension there will eventually benifit from the "side-quest" in centrs project that trying to consolidate all the other repo's RouterOS code.

The eventual idea is the TikBook VSCode extension is what integrates the various parts into more VSCode-based "controller". For example, supporting multiple connected routers, and minor things like edit any of the "event scripts" directly on RouterOS but get the LSP features while doing it from VSCode. As well as adding "AI Support" so that VSCode's copilot (and ideally other agents like claude with ACP).

With the dicodany being RouterOS LSP works in any LSP-enabled editor, with the VSCode-specific stuff living in TikBook. Both the using the centrs library internally.

Which sorta address the comment:

I tried to understand it, but it is beyond my neural network's ability to handle the complexity of it.

It confuses me too. Basically it's mainly plumbing, so ain't pretty. And once the plumbing done... I think all makes a lot more sense if you saw all the pieces pull back into TikBook running in VSCode. The WIP centrs add multiple router support (using WinBox CDB as it's data source for devices/ports/names and optionally passwords), and support beyond REST for interacting with RouterOS (including mac-telnet, etc.). While RouterOS LSP needs "offline support" and better syntax structure knowledge. Both share the same code in centrs so that be tested independently of VSCode extensions which are much harder to test than a pure TS library/CLI.

At least in my work, when I need to edit in VSCode, most errors come from missed brackets and forgotten $ in front variables. So that alone would be a big help. Then highlighting "known", but not necessary "available" attributes is just a cherry on top.

And that's why this one is tricky... Underlying "missing brackets" is that context matters for {} as they can be "real block", a attribute value with script, same-scoped operator do/else thing, or an array (and either list-type or KV type of array). So you need enough context to surface which kinda thing {} without out being "too picky" on path/verbs, which is one of centrs explains passes to figure out depth.

The current LSP actually doesn't help on @kenzo's issues since /console/inspect can really only tell you where a the script becomes un-parsable. Useful, and entirely possible a } kinda issue. VSCode has native bracket matching, so for simple cases VSCode already shows a color for matching brackets... but the issue is has no idea on embedded forms of scripts="{..."... so this can get messed up.

So mismatched bracket would be clearly visible via the under-development centrs explain logic, since it is actually parses {} blocks (and determined the type of them), so a mismatch be known. One the variables stuff... offline mode does surface symbols (e.g. local/global variables), which is why it needs context on the {} blocks... the lack of matching definition should be visible which indirectly give clues at a "scoping problem" since, in theory, the scoping rules are (mostly) knowable.

e.g. unclosed bracket, but still finds the variables...

bun run src/cli.ts explain --tokens '{:local x; :set x 123; :put $x; :put y; :put $y'

verdict: fail — 1 statement(s), 2 diagnostic(s)
gate: script · 0 arg(s)
write: unknown
statements:
  [0,47)       unknown    structural defect: unbalanced delimiter or string
symbols:
  [8,9)        local     declaration name="x" bindings=b0
  [16,17)      local     assignment  name="x" bindings=b0
  [29,30)      local     reference   name="x" bindings=b0 reaching=[]+unknown
  [46,47)      parameter reference   name="y"
tokens: 9 token(s), 5/47 byte(s) classified (10.6%), class provisional
  [0,1)        brace
  [1,8)        unclassified
  [8,9)        variable-local
  [9,16)       unclassified
  [16,17)      variable-local
  [17,29)      unclassified
  [29,30)      variable-local
  [30,46)      unclassified
  [46,47)      variable-parameter
diagnostics:
  error   [0,1)        explain/canonicalizer/unclosed: unclosed `{` — never closed
  warning [0,47)       explain/canonicalizer/unresolved-statement: structural defect: unbalanced delimiter or string
evidence: e0 canonicalizeExecuteCommand (direct), e1 analyzeCoordinates (direct), e14 braceSpans (heuristic), e2 segmentStatements (direct), e4 resolveVerbs (heuristic), e6 containsWrite (heuristic), e7 resolveSymbols (heuristic)
runtimeAcceptance: not-proven

which shows why this is not done, since the :put y likely have be found as a string for letter y, not a variables... but with the $y showing up later, is confusing since it knows a variable called y later from the :put $y... and [46,47) variable-parameter should be more visible as a uncleared variable. Anyway, it's still very tricky business with invalid structure.

And you close the parens in {:local x; :set x 123; :put $x; :put y; :put $y}, you can see how it's resolving variables, which be a big clue if something was wrong IMO, if UI showed it "presumed" value/type.

 bun run src/cli.ts explain --tokens '{:local x; :set x 123; :put $x; :put y; :put $y}'
verdict: pass — 5 statement(s), 0 diagnostic(s)
gate: script · 0 arg(s)
write: false
statements:
  [1,9)        resolved  command  path=/ verb=local  via=execute
    centrs: centrs execute '<router>' ':local x'
  [11,21)      resolved  command  path=/ verb=set  via=execute
    centrs: centrs execute '<router>' ':set x 123'
  [23,30)      resolved  command  path=/ verb=put  via=execute
    centrs: centrs execute '<router>' ':put $x'
  [32,38)      resolved  command  path=/ verb=put  via=execute
    centrs: centrs execute '<router>' ':put y'
  [40,47)      resolved  command  path=/ verb=put  via=execute
    centrs: centrs execute '<router>' ':put $y'
symbols:
  [8,9)        local     declaration name="x" bindings=b0
  [16,17)      local     assignment  name="x" bindings=b0 value=v0
  [29,30)      local     reference   name="x" bindings=b0 reaching=[v0]
  [46,47)      parameter reference   name="y"
values:
  [18,21)      positional shapes=num
tokens: 26 token(s), 31/48 byte(s) classified (64.6%), class provisional
  [0,1)        brace
  [1,2)        dir
  [2,7)        cmd
  [7,8)        unclassified
  [8,9)        variable-local
  [9,11)       unclassified
  [11,12)      dir
  [12,15)      cmd
  [15,16)      unclassified
  [16,17)      variable-local
  [17,18)      unclassified
  [18,21)      value
  [21,23)      unclassified
  [23,24)      dir
  [24,27)      cmd
  [27,29)      unclassified
  [29,30)      variable-local
  [30,32)      unclassified
  [32,33)      dir
  [33,36)      cmd
  [36,40)      unclassified
  [40,41)      dir
  [41,44)      cmd
  [44,46)      unclassified
  [46,47)      variable-parameter
  [47,48)      brace
evidence: e0 canonicalizeExecuteCommand (direct), e1 analyzeCoordinates (direct), e12 pathSpans (heuristic), e14 braceSpans (heuristic), e4 resolveVerbs (heuristic), e6 containsWrite (heuristic), e7 resolveSymbols (heuristic), e8 classifyExplainTransport (heuristic), e9 valueShapeHints (heuristic)
runtimeAcceptance: not-proven

In a sense that's why I find RegEx gramma still useful. Sure, it cannot express all valid inputs per RouterOS's interpreter, but it's capable of expressing structures of how you should write real code. Maybe it's a signal to rewrite if level of nesting is such that regexes start failing.

But of course being able to provide detailed context for every token is an ambitious and a welcome project :slight_smile:

@kentzo, Perhaps comment in my LSP thread with some examples?. i.e. I didn't mean to hijack the thread._ But some example(s) help sort out what you think should be findable via regex (or centrs). It's currently using a DB with :parse and request=highlight, by version, for a larger set of rextended/eworm/tangentsoft/my-own scripts, and compare "on-line" results against the offline parser results to squeeze out detection of everything possible. (e.g. one high-level goal for agents is get the token offline to match token online, so working on that dimension current). But my current test cases are mostly valid scripts to find syntax various elements/styles/etc.

To be honest, I have not look as much at invalid cases as much yet (other than there are some unit tests with syntactic bad examples). Although you already caused me to find one where centrs offline thinks this valid /ip/address add interface=ether1 /ip/route add gateway=192.168.88.1 with "/ip/address"/"add" being a positional argument, and gateway= associated with the /ip/address. Online this is currently detected, but as example where the invalid cases be useful. _Seemingly being a rule that / cannot start an attribute name... but also case where a richer static dataset (like known attributes names) be helpful but centrs explain is not using a full set from the cliref or inspect.json data which also like see the combined command.

Anyway it's tricky business parsing RouterOS script. But VSCode showing RouterOS variables in its "Symbols" view and/or content menu support for "Go to definition"/etc be very useful IMO. And the LSP knowing the actual "folding blocks" help a ton with mismatched parens I suspect. But there is this pesky problem that something has to parse routeros script.

On this point, you can use the other part of my AI trilogy, tikoci/quickchr, which can bring up a tiny "free" CHR using QEMU, at the same version (and packages/device-mode/etc) at your target. And then in RouterOS LSP point the configuration at local CHR that does a live check. And if you need another version, you can bring up another one in ~30s to 1m (depending on if you need package, device-mode, etc as those all require reboots). This at least get you close to matching a customer system, and if you adding socket/shared/etc interfaces to the quickchr options, the interface count can match.

Since the LSP just need the RouterOS to invoke /console/inspect, the 1MB speed limit is generally not a problem (although with larger script, it make responses slower to update), so getting a "trial" license may be helpful (which quickchr can also apply automatically).

Eventually this be an option in TikBook, but you can "manually" use quickchr today. On a Mac, it's just

brew install qemu bun
bun install -g @tikoci/quickchr

# optional but useful...
quickchr doctor # check everything but should be okay from above too
quickchr completions  # allows tab completion for quickchr commands like list and machine names
 
# wizard that prompts for CHR config options
# - you can pick min CPU and memory if only used for the LSP
# - say "yes" for "background" and  "start"

quickchr setup

It will display the REST API url, which be localhost and a port like 9100. You'd use this in VSCode's RouterOS LSP setting so the baseURL be http://localhost:9100 (or what ever quickchr assigned for REST port, as each CHR gets it own set of ports that forward from localhost to quickchr CHR, and quickchr list will show the assigned ports for each CHR). _They do not startup automatically, so if you reboot, you need to use quickchr start <machinename> after to bring it back up for the RouterOS LSP to use.

The gory details are here quickchr/MANUAL.md at main · tikoci/quickchr · GitHub

Thanks, I did not know quickchr! will try it on my box. Matching packages and device-mode in one wizard is the useful part, that is where my manual setups eat time.

I already keep disposable CHRs in UTM and QEMU, and stress-testt scripts there in safe-mode. TBH, on a 16 GB M1 Pro that has a price. When real work starts, I kill every VM and most of browser tabs. When the Mac sleeps, a VM sometimes wedges. And I have to keep remembering it is running.

The screen-share case is separate, and that is my setup, not your tool. The server or the laptop is not mine. No brew, no qemu, nothing I get to install. I read the config, I edit, I hand it back.

Well, there is https://vscode.dev (or https://github.dev ) which the web version of VSCode, that runs VSCode in the browser's JavaScript. And the RouterOS LSP extension can be installed into the "VSCode for Web".

This however does not change the need for a live router and also requires CORS proxy server over the REST API. But if you install something like NGNIX, Caddy, Traefik, etc. in front of some router (or CHR) that can handle the needed HTTP headers for CORS and restrict access to a router/CHR at your shop.

Certainly non-trivial, and you have then have some router open to internet at least enough to make HTTP connection between browser with VSCode for Web (with RouterOS) and the CORS proxy server that routes to some router's REST API. There are few approaches to securing the connection, like you can layer on X.509 client certificates (which browser will use) to further protect access to (non-production, for LSP use only) router. And/Or, restrict CORS proxy to just /rest/console/inspect and use read-only account. But overall some CORS-enabled approach does work with vscode.dev/etc and RouterOS LSP. But nothing escape the need for some CORS proxy in between RouterOS, since RouterOS does not support CORS headers.

May not really help your case, but /app code-server available in app catalog also presents a VSCode interface in browser, and also works with LSP, and does not need CORS proxy... but you need the container installed on the local network for it run, and the only reason it does not need CORS is the plugin run inside the container under node so CORS does not apply. But the public vscode.dev/github.dev VSCode for Web sites always require CORS between the LSP an router since the LSP extension runs in the browser's JS..