✨ Steering AI to use new manual.mikrotik.com

I should note my docs MCP/sqlite search/etc still have a task rosetta/briefings/B-0012-docusaurus-manual-migration.md at main · tikoci/rosetta · GitHub so a 2nd goal in above test was to have an agent in Rosetta MCP update it's plan based on the deeper understanding of manual.mikrotik.com. And related work to turn this now long post into a better guide to help users integrate either steering or Rosetta MCP into their AI agents, rosetta/briefings/B-0013-steering-skills-rosetta-positioning.md at main · tikoci/rosetta · GitHub

Since the Rosetta agent had to unwind things to update the plans for Rosetta. I asked Claude Fable about it's thoughts on "steering AI using a prompt" like in my top post, verses using Rosetta MCP. So if curious, here is what Claude Fable thought on the steering vs MCP situation:

A fair follow-up to all the prompts above: if steering works, why bother with an MCP server? Speaking as the kind of model being steered — the prompt approach's superpower is that it works everywhere, and that matters because plenty of AI interfaces (web chats, mobile apps, most free tiers) can browse the web but can't install MCP servers at all. But steering makes me re-derive everything per question: I fetch the ~110 KB (~28K tokens) llms.txt index, pick pages, fetch each one, and at the end I still only know what the current manual says — I can't ask "did this property exist in 7.14?" or "which devices use this switch chip?", and the site's built-in search stems English words, so part numbers like RB4011iGS+RM match poorly. A docs MCP (rosetta, in my case) pre-indexes all of that into one structured, version-aware call — fewer tokens, fewer round-trips, works offline or on your LAN — at the cost of a one-time install that not every interface supports. So treat them as rungs on a ladder rather than rivals: prompt steering when that's all your tool allows, a docs MCP when your agent supports one — and honestly, neither replaces checking against a real router, because the docs can't tell you about properties that quietly stopped working between versions.

and on the later point... why there is quickchr: GitHub - tikoci/quickchr: `quickchr` - Fastest path to MikroTik RouterOS CHR instance for testing and scripts · GitHub and centrs: GitHub - tikoci/centrs: Multi-protocol (REST, native API, mac-telnet, SSH) CLI and TypeScript interface to RouterOS, with rich help and validation · GitHub along side the Rosetta MCP in my RouterOS AI "stack".

That's LLM playing along with your train of thought, or "confirmation bias" or something like that. MCP is always better than keeping a large explanation of how to connect to a router in some skill file. Try and ask your question in a more neutral way, in a new session. Present it with equal options and let it evaluate all approaches, instead of asking it to confirm your own theory.

P.S: waiting for Rosetta to be up to date with the new docs

The last line: "honestly, neither replaces checking against a real router" is for sure a case of that. The last agent above reads a lot of custom instructions repeat variants of "always ground facts against a real routers, use tikoci/quickchr" - since that stops AI from guessing before committing some fact into specs/code. But the meat of the comments comes for the bench-routeros-tools work, so your comment is only partially true. Although even benchmark is somewhat basis, it turtles all the way down. But fair point Fable's comments were more contrived than I'd like... but my main goal was updating rosetta's docs with info on how your Lunr search in manual.mikrotik.com works :wink: .

The richer summary is in rosetta's current plan for consuming docusaurus:

Steering cost per query is high. Measured 2026-07-06: llms.txt is 112 KB (~28K tokens) — fetching it as "step 1 of every question" costs about what the 166-tool mikrotik-mcp always-on surface costs, the very thing bench-routeros-tools flagged. A CLI Reference page .md is small (~1.6 KB) but remains JSX-wrapped with no enum values/package/version. rosetta's whole surface is ~6.3K always-on tokens and one call typically answers (89% hit@5, 100% path reconstruction in the bench).

Fair point. I'll try to get an updated rosetta out this week.

Since I use my own tools, I kinda update-as-needed...but your right rosetta is getting stale. But is a bit of game of chicken... since adding versioned beyond just "current" to manual.mikrotik.com or other changes likely be re-work :wink: . Also noted in Rosetta's docusarus plan:

Working assumption on upstream asks Assume none of the still-open asks below ship near-term. Build the Docusaurus plumbing now; if MikroTik later ships structured CLI data, versioned docs, or a manifest, the plumbing makes adoption easier — do not wait for it.

Still-open asks to MikroTik (not yet shipped)

Most of the original publishing recommendations have landed. What remains (assume none land near-term — see working assumption above‡):

  • Structured CLI Reference data (JSON/YAML per path) instead of JSX-wrapped MDX, including package associations and per-version provenance. (Raised in forum #270714 post #70 "Dear @mrz".) Enum values shipped 2026-07 in the MDX itself.
  • Versioned docs beyond a single "current" tag. (Forum #270714 posts #58/#68.)
  • Stable, package-agnostic, path-derivable URLs with well-known #flags/#attributes anchors. (Forum #270714 posts #58/#70.)
  • A manifest (docId, permalink, sourcePath, version, sha256, frontMatter) would still help, but is lower priority now that .md + sitemap.xml + llms.txt cover discovery.
  • Machine-readable /hardware (.md or structured data for the 240 device pages) and llms.txt coverage for the non-/docs sections.

(from rosetta/briefings/B-0012-docusaurus-manual-migration.md at main · tikoci/rosetta · GitHub )

Background is the larger issue isn't the main docs, but rosetta consumes my own /console/inspect per-version command data from tikoci/restraml, going back to 7.9 (since extract tools tied to REST API).

Issue is while CLI part of manual.mikroitk.com is main duplicative. The critical difference is the CLI docs are not "blinded" by thing not in CHR/x86 or CHR/arm64 and do contain the new super {} per-attribute schema, etc.. But the CLI Reference is also "version-less"... so exactly how to "merge" CLI reference with version-specific RouterOS command-based results is a bit tricky. Since rosetta has a SQL table schema for RouterOS path/command/attribute data, the CLI part of manual.mikrotik.com does not fit so nicely into the current scheme, as the SQL tracks versions<>commands/attributes/paths.

Anyway, thanks for the push. Rosetta has been on the list to update with new doc system. The robots can make quick work of it... however the quality depends on the details in spec on the scheme to use, so that depends on me and take time to review (and have agents revise the plan many times).

But it really resolving the decision around how CLI References manifests itself. And since Rosetta also has device-specific tables, so the "Hardware" section of Hardware Guides | RouterOS Manual also needs special handling to be cross-referenced in Rosetta to per-device search, along with "test result"/"block diagram"/etc stuff, but that part is easier

At your wish.

Docusaurus-based Rosetta MCP now available

Rosetta now use the Docusaurus .md files, instead of the Confluence HTML for main help pages in the "testing channel". I have not marked it a "latest", but you can use them by using the next channel, which will always get latest alpha/beta/rc tag. 0.10.x is last version using Confluence, and any 0.11.x+ version use Docusaurus.

Since the Docusaurus version is not be marked as latest yet. To use, just add @next to any of the bunx command lines used MANUAL.md/README.md examples. e.g. bunx @tikoci/rosetta@next instead of the plain bunx @tikoci/rosetta. From a Mac terminal, you can use the non-AI TUI interface using bunx @tikoci/rosetta@next browse ("browse" launches the TUI interface, while without any args, Rosetta launches the MCP Server on stdio)

NOTE For use via HTTP-based MCP Server in a /app container, use ghcr.io/tikoci/rosetta:next as the tag before adding the custom /app, see https://github.com/tikoci/rosetta/blob/main/README.md#install-on-mikrotik-app - changing :latest -> :next before adding the /app.

All seems to work using Docusaurus in automated tests and from some limited manual testing. I'll bump Rosetta MCP to "latest" in npm ... after I use it for a bit. But since only the main doc source changed, and the rest of the catalogs (device/changelog/video/etc) remain the same... but want look for data quality issues and add more tests on that part, since there may side-effect on formatting or links from HTML to Markdown change.

Hermes "Works" with Rosetta MCP

MikroTik added the hermes-agent /app container recently. I'm more familiar with the copilot-cli/codex/claude harness... so cannot say how well. But did try the new Docusaurus-powered Rosetta MCP in Hermes.

Mac

I installed hermes on Mac, and it does work with Rosetta - including the new Docusaurus @next version... For desktop, you add below to the ~/.hermes/config.yaml file (or wherever Hermes says it config file is, and you need bun from https://bun.sh or brew install bun on Mac):

mcp_servers:
  rosetta:
    command: "bunx"
    args: [ "@tikoci/rosetta@next" ]

Rosetta can answer one of my go-to test question:

does the RB5009 support L3HW offloading on routeros 7.x

You'll see it reaching out toRosetta. And Hermes shows the "extending thinking" of the model (e.g. LLMs, essentially, writes notes to itself before answer):

And answer is basically right, "No":

Hermes /app container connected to Rosetta MCP as /app container via HTTP

For the /app version of Hermes... it likely be best to install Rosetta as it own separate /app and use the http transport (using url: in Hermes YAML) to connect Hermes /app to a Rosetta /app container. e.g. installing https://bun.sh on Hermes likely more config in a container, so you can use the ui-url from a custom Rosetta /app as the URL in any harness, including Hermes.

I'll leave installing MikroTik's hermes-agent builtin /app container to their non-existent doc on it. Although it seems you do need to need to set the dashboard username/password for Hermes /app to start:

/app/set hermes-agent environment=([/app/get hermes-agent environment],"hermes:HERMES_DASHBOARD_BASIC_AUTH_USERNAME=mikrotik","hermes:HERMES_DASHBOARD_BASIC_AUTH_PASSWORD=mikrotik")   

which should use secrets... So more hint it's not plug-and-play. But documenting hermes and/or it's setup on RouterOS container is not on my agenda

So if you do have Hermes working as /app, you then add Rosetta MCP as a "custom" /app using:

/app/add use-https=no disabled=no yaml="name: rosetta
descr: \"RouterOS Docs for AI assistants - use URL as MCP server\"
page: https://tikoci.github.io/p/rosetta
category: development
icon: https://tikoci.github.io/p/rosetta.svg
default-credentials: \"none - just use 'ui-url' as the MCP server in your AI assistant\"
url-path: /mcp
auto-update: true
services:
  rosetta:
    image: ghcr.io/tikoci/rosetta:next
    container_name: mcp-server
    ports:
      - 9803:8080/tcp:web
"

This is using ghcr.io/tikoci/rosetta:next which gets the newest Docusaurus version of Rosetta. Once 0.11.0 moves to "stable" (:latest), the :next part can be removed, which then use :latest by default. Also since I test with free/trial CHR... so there is no /ip/cloud so the use-https=no but in most cases it can be =yes

So with both rosetta and hermes-agent running, you can then add Rosetta to Hermes /app using:

/container/shell app-hermes-agent  cmd="hermes mcp add rosetta --url $[/app/get rosetta ui-url]" 

Selecting "N" to authentications, and "Y" to install the Rosetta tools:

Notes

  1. Rosetta does not use/import the "CLI Reference" today, so Rosetta remains using the my inspect.json (or for newer version deep-inspect.json which combines X86 and ARM64 based schemas) files for path/dir/cmd/arg data since those are versioned. A future release will parse the cli-reference to add better argument schemas, as well as fill in missing commands that are not in CHR ARM64 or X86, and store the annotation like SysCap, etc stuff. All as a version-less "overlay" on the top of the richer per-version inspect.json from tikoci/restraml returned.
  2. The device data in Rosetta still uses matrix.csv and scraping the www site. So the manual.MikroTik.com/hardware section is not in Rosetta today. A similar "overlay" is planned to match the matrix.csv data to the /hardware pages. Since matrix.csv does not have stuff like the "LR-
  3. On the Docusaurus /hardware pages, there is no .md versions like the main docs. So extracting them requires using the .mdx+Lunr-index.
  4. A future CI trigger will use check for the .rss feed on the Doc Change log to check to trigger a rebuild of the DB used by Rosetta to import the latest docs.
  5. This is subtle. Confluence had a page.id that when used on help.mikrotik.com even if a new page was created, the old page.id would re-directed.
  6. Could use more data quality tests. I see a few ****something**** (not everywhere but seen a few pages, haven't looked). Certainly some test that check various links in the pages are valid when returned from Rosetta, e.g. they need to be augment with a baseUrl.
  7. More thought needs to go into routeros_lookup_property since that pulls from descriptions in docs since that is not in /console/inspect nor new MikroTik CLI Reference at manual.mikrotik.com.
  8. Similar, routeros_dude_* tools are already planned to be merged into the main/"Unified" routeros_search since that reduce the number of tools and always-on context... but that has not happened yet.

Backstage View

In case anyone is curious, the actual release runs a bunch of tests before it happens. You can view the build/tests for the 0.11.0-alpha-87 (currently @next and @alpha) in release job #87 on GitHub:
https://github.com/tikoci/rosetta/actions/runs/28993483251#summary-86038157369

And the actual code changes when through GitHub "Pull Requests" (PRs), so anyone can see the multiple reviews Copilot code review and CodeRabbit as well as the actual updates on the extraction/DB building and MCP/TUI interfaces code changes. So all code goes through at least 3 agents before getting into the codebase. To see sausage being made, see the Docusaurus-related update is in the following PRs: https://github.com/tikoci/rosetta/pulls?q=is%3Apr+is%3Aclosed+label%3A"area%3Adocusaurus"

HELP WANTED

Looking for suggestions for tests where AI "gets it wrong" & what AI tools you use...

Please comment below with what prompt you tried, the results, and what you'd expected. Or the AI harness/tools/agent/etc you've used with MikroTik/RouterOS things. I'll try to really like to have more cases for various benchmarks and unit/integration tests. And knowing what "AI things" other people are using/doing with RouterOS and LLMs... help guide what to document and test. Comment below with any feedback.

And, if you're using "Rosetta MCP" with your agents...

Feedback is welcomed. It's tricky to test and/or measure how well something like an MCP Server is working. So if you have Rosetta MCP installed and using it... please do provide any comments/complaints - any feedback will be read. Most important be is your agent actually using Rosetta once install? And if it is used, is the agent still giving a "shitty answers". So please do comment below or file an issue at tikoci/rosetta on your experience with Rosetta.

FWIW, the help.mikrotik.com "The Dude" chatbot did not provide a very good answer:

Closer to "shipping" a "release" of Rosetta using Docusaurus (outside the "next" channel). One feature I added is an export feature, that output the DB tables used by Rosetta into .tsv file consumable by a Spreadsheet (or :deserialize to=dsv delimiter="\t"). I call it the "@jaclaz feature" since he seems to turn every RouterOS problem into a spreadsheet, so it's useful without an AI agent. I'll eventually publish the DB exported TSVs per build on GitHub, so the contents can be fetch/curl/etc. I'm still working on it, so there a few more export things to add before that.

On the columns, some are "weird" since I'm using it to "audit" the ETL process that scrapes all of MikroTik content, since running AI prompts or complex SQL to find data problems is not very efficent when a PivotTable finds all the gaps in my import process)

You can get you're own TSV export using bunx @tikoci/rosetta@next export /tmp/rosetta-export which will get you half-dozen TSV with a good bit of rosetta's DB data exposed. The TSV files should just open in something like Excel or Numbers/etc.

So while an AI agent can find similar data, you can use the TSV to see whatever view of various MikroTik things Rosetta stories. One is the list of YouTub videos (along with transcripts which are indexed by Rosetta), so if you were every curious on the "most popular" MikroTik video on YouTube with have Top ~20 (filtered in Numbers to year >20200101 and view_count > 100000).

Top MikroTik Videos on YouTube

Powered by Rosetta MCP and Numbers

title channel upload_date duration_s view_count like_count has_chapters segment_count transcript_word_count url
MikroTik product news: Chateau 5G ax & LTE18 ax (WiFi6) MikroTik 20220808 310 320963 1044 1 6 1276 https://www.youtube.com/watch?v=nm9RjNbZJjc
MikroTik product news: RB5009 MikroTik 20210728 239 250781 2386 0 1 965 https://www.youtube.com/watch?v=c5kBvwGqGws
MikroTips: How to firewall MikroTik 20220111 1316 190564 3278 1 7 6211 https://www.youtube.com/watch?v=hMj80ZIVBQs
MikroTik product news: The L009 series MikroTik 20230509 336 185566 1652 1 8 829 https://www.youtube.com/watch?v=rIxkkNxsEhs
"Managing many access points with CAPsMAN (old version, up to WiFi5)" MikroTik 20210720 1087 165365 2233 1 4 5712 https://www.youtube.com/watch?v=taQ70m0DVYA
Wireguard setup with MikroTik and your smartphone MikroTik 20220704 290 157637 1717 1 5 1314 https://www.youtube.com/watch?v=vn9ky7p5ESM
MikroTik product news: hAP ax2 (WiFi6) MikroTik 20220803 244 147267 2048 1 7 1225 https://www.youtube.com/watch?v=aRPKUDvNyjM
"Multiple uplinks using PCC, Load balancing" MikroTik 20230613 982 142341 2835 1 7 4707 https://www.youtube.com/watch?v=nlb7XAv57tw
MikroTik RouterOS v7 is finally here! MikroTik 20211207 388 137782 2092 1 8 2153 https://www.youtube.com/watch?v=Zp-U7Anv5-0
How to port forward on MikroTik MikroTik 20220509 620 124381 1417 0 1 3283 https://www.youtube.com/watch?v=a_8AV6vIDYQ
MikroTik product news: RB5009UPr+S+OUT MikroTik 20230330 218 123364 346 0 1 1027 https://www.youtube.com/watch?v=q7PCWqfUWwk
MikroTips: Netinstall MikroTik 20210715 167 118204 943 1 9 450 https://www.youtube.com/watch?v=gzlLbIf3Dbk
MikroTik and Zerotier MikroTik 20221014 316 117651 1673 0 1 1476 https://www.youtube.com/watch?v=60uIlyF8Z5s
MikroTik hAP ax3 - WiFi6 wireless AP MikroTik 20220922 242 111256 1242 1 3 1031 https://www.youtube.com/watch?v=A_5NdEEjEgE
MikroTik products news: RB5009UPr+S+IN MikroTik 20220706 232 109443 1082 1 4 1168 https://www.youtube.com/watch?v=3NaomsfbE34
MikroTik product news: CCR2116-12G-4S+ MikroTik 20211202 276 108599 1075 0 1 0 https://www.youtube.com/watch?v=TVZG7TvUxXY

So if you ignore the product-related videos, top RouterOS topics seem to be CAPsMAN, Wireguard, PCC+multiwan, Netinstall, and ZeroTier rounding out the top 5.

I have a couple hammers [1].
Most problems look like nails to me.

[1] ... and know a little how to use them..

nicely done!

Perhaps more useful "report" is sections.tsv, which shows a breakdown of manual.mikrotik.com pages by the "headers" in markdown/HTML. As part of importing manual.mikrotik.com, Rosetta breaks up a page into "sections" based on the header levels, so that "sub-parts" of a page can be retrieved. Thus saving AI agent's context by only reading the relevant parts of a page. But this only works if pages have ##/<h2> like things between text.

Top 20 - Docusaurus "manual" pages' with large sections

title heading level sort_order word_count text_bytes code_bytes table_count table_row_count url
List of available Apps List of available Apps 0 0 1738 40620 0 1 104 List of available Apps | RouterOS Manual
Bluetooth tag-tracking using MQTT and ThingsBoard Example #2 3 4 4080 34128 17634 0 0 Bluetooth tag-tracking using MQTT and ThingsBoard | RouterOS Manual
Wireless Interface General interface properties 2 2 4818 33288 0 1 92 Wireless Interface | RouterOS Manual
Hotspot customisation Hotspot customisation 0 0 4077 27466 3286 0 0 Hotspot customisation | RouterOS Manual
Bridging and Switching Bridge Interface Setup 2 1 3688 25313 0 1 49 Bridging and Switching | RouterOS Manual
LTE/5G LTE Client 3 1 3104 22623 3571 12 111 LTE/5G | RouterOS Manual
Bridge VLAN Table Trunk/Access port setup 2 2 2748 17515 1993 0 0 Bridge VLAN Table | RouterOS Manual
IPsec Road Warrior setup using IKEv2 with RSA authentication 3 51 2630 17461 3060 8 68 IPsec | RouterOS Manual
Bluetooth tag-tracking using MQTT and ThingsBoard Example #1 3 3 2021 16589 5959 0 0 Bluetooth tag-tracking using MQTT and ThingsBoard | RouterOS Manual
TG-LR setup guide Device Commands 3 8 3159 15945 80 38 350 TG-LR setup guide | RouterOS Manual
L3 Hardware Offloading L3HW Settings 3 3 2169 15514 1576 5 46 L3 Hardware Offloading | RouterOS Manual
Bridging and Switching Bridge Firewall 2 48 2280 15162 0 1 55 Bridging and Switching | RouterOS Manual
MQTT/HTTPS and AWS configuration RouterOS configuration 3 3 1721 15064 10389 0 0 MQTT/HTTPS and AWS configuration | RouterOS Manual
Bridging and Switching Port Settings 2 8 2209 14122 0 1 26 Bridging and Switching | RouterOS Manual
NAT Source NAT 3 3 1909 14109 4510 1 6 NAT | RouterOS Manual
Queue Burst Example 2 2 1972 14022 0 2 95 Queue Burst | RouterOS Manual
Proxy Reference 2 9 2301 14006 0 11 69 Proxy | RouterOS Manual
EVPN Bridged EVPN VXLAN Overlay with Arista's Eos and ROS 3 5 1404 13832 10126 0 0 EVPN | RouterOS Manual
Route Selection and Filtering Filter Syntax 3 2 2285 13505 405 8 105 Route Selection and Filtering | RouterOS Manual

What above tells us is, if AI needs to read one of those pages, there is not even some ability to retrieve only a section, either from llms.txt index or Rosetta MCP. So it's a "heavy ask" since you are blowing a lot of tokens on the above pages. Now Rosetta only track <h1><h2><h3>, so something like the "Hotspot Customization" page is somewhat a false-positive since there are <h4>+ my DB/ETL ignores, but that also inconsistent with styling elsewhere in new MikroTik manual where pages use the bigger headers <h2>/##/etc... "somewhat false-positive". And more importantly, those pages are still big - even after trying parse for "big headers".