## Request
Forum user `fischerdouglas` asked whether restraml could include the⦠"ab" nightly builds served at https://mt.lv/nightly-build (thread: https://forum.mikrotik.com/t/https-tikoci-github-io-restraml-to-https-mt-lv-nightly-build-is-to-costly/272302):
> Whether out of fear, laziness, or sheer negligence, MikroTik doesn't include any information or details regarding the "ab" versions... Personally, I trust your tool more than their release notes to know about what changed between releases. Would it be possible for you to include the versions that appear in the nightly build link in your mapper / comparison tool?
`@amm0` replied with viability questions + a strawman:
> Do you know if the mt.lv/nightly-build is actually updated "nightly"? Does it always have all extra-packages? ... It uses different URL schemes since URL goes via redirect to box.com so that's a bit more fragile. Also, it's only .npk files, not CHR disk images. So it not quite "play and play" into the CI. *One solution may be to bring up stable CHR, then upgrade to nightly, then collect the schema files to the `inspect.json`/`deep-inspect.json` files that underpin everything.*
> Perhaps there could be one "nightly" target that one slot that gets replaced with latest nightly...
This issue captures homework + a minimal strawman for that single-slot nightly.
---
## What `mt.lv/nightly-build` actually is (2026-08-13 probe)
- **Redirect:** `https://mt.lv/nightly-build` β 302 β `https://box.mikrotik.com/d/5ce46054d5d2487e8755/` (MikroTik Box = Seafile share, `box.mikrotik.com`). Token `5ce46054d5d2487e8755` appears in HTML/JS `pageOptions` + `og:url`.
- **Today's payload:** `7.25_ab430` (all last_modified `2026-08-13T12:54β13:04Z`, so *today* it did update β but recall in thread that it has historically been irregular).
- **API for listing:** `GET https://box.mikrotik.com/api/v2.1/share-links/5ce46054d5d2487e8755/dirents/?path=/&thumbnail_size=48` returns JSON `dirent_list` (40 files today). Example entries: `routeros-7.25_ab430.npk`, `routeros-x86-7.25_ab430.npk`, `container-7.25_ab430.npk`, `wireless-7.25_ab430.npk`, etc. No `.vdi.zip` / `.qcow2` CHR images at this location.
- **Download URL (verified):** `https://box.mikrotik.com/d/<token>/files/?p=/<file>&dl=1` β 302 β `https://box.mikrotik.com/seafhttp/files/<uuid>/<file>` β works with `curl -L` from CI.
- **Version string:** `7.25_ab430` β note `_<ab><num>` suffix. This **does not match** existing restraml version regexes (`^\d+\.\d+(?:\.\d+)?(?:beta|rc)?\d*$`). So `VERSION_RE` in `scripts/build-docs-index.mjs`, `parseVersion`/`isPreRelease` in `docs/restraml-shared.js`, `deep-inspect.ts` path regex, and `rest2raml.js` all need an `ab` branch if nightly is hosted as a versioned dir.
- **Package coverage (x86):** x86-identifiable files are the unsuffixed ones (`routeros-7.25_ab430.npk` == `routeros-x86-7.25_ab430.npk` by size/id, plus `container-`, `iot-`, `rose-storage-`, `wireless-` unsuffixed). Compared to `all_packages-x86-*` for stable, nightly looks **mostly complete** for x86 extras (container/iot/rose-storage/wireless/zerotier/wifi variants present), but needs a canonical mapping verified per run.
- **Channels are separate:** `upgrade.mikrotik.com/routeros/NEWESTa7.{stable,testing,development,long-term}` currently return `7.23.3`, `7.24rc4`, `7.24rc4`, `7.21.5` β i.e. **nightly is not in any of those feeds**. Discovery must poll the Box API, not `upgrade.mikrotik.com`.
- **No CHR image:** CHR for nightly would have to be synthesized: boot latest stable CHR (`download.mikrotik.com`/`cdn.mikrotik.com` `chr-*.vdi.zip` β `qcow2` + virtio) then *upgrade* via nightly `.npk`s.
### Quick local probes run
```sh
curl -sL -I https://mt.lv/nightly-build # 302 β box.mikrotik.com/d/...
curl -s "https://box.mikrotik.com/api/v2.1/share-links/5ce46054d5d2487e8755/dirents/?path=/&thumbnail_size=48" | jq .
curl -s "https://box.mikrotik.com/d/5ce46054d5d2487e8755/files/?p=%2Frouteros-7.25_ab430.npk&dl=1" -I # 302 β seafhttp (verified)
curl -sS https://upgrade.mikrotik.com/routeros/NEWESTa7.stable # β 7.23.3
# Existing regex check:
node -e "console.log(/^\d+\.\d+(?:\.\d+)?(?:beta|rc)?\d*$/.test('7.25_ab430'))" # β false
```
---
## Viability / cost check
| Factor | Signal |
|---|---|
| **Fragility** | Seafile share behind short-link redirect; token could rotate, API is undocumented (`/api/v2.1/share-links/.../dirents`). Fine as best-effort nightly, not as hard dependency for stable/beta pipelines. |
| **Image shape mismatch** | No nightly `chr-*.vdi.zip`; must add an upgrade step (SCP `.npk` β reboot). Existing extra-packages pipelines already do `scp -P 9122 extra/* admin@localhost:/` + reboot β same primitive works, but nightly needs arch-specific filtering (`x86` unsuffixed + `routeros-x86-*.npk`; `arm64` suffixed). |
| **Repo / Pages size** | `docs/` is already ~2.6 GiB, 62 version dirs, ~60 MiB per version (`schema.raml` 6β8 MiB, `openapi.json` 14β18 MiB, `inspect.json` 1β1.3 MiB, `deep-inspect.json` 5β6 MiB). Keeping *every* `ab` would grow without bound (`ab430` already). Single overwritten slot bounds this at ~one version (~30β60 MiB base + extra) instead. |
| **CI time** | Adds one extra reboot cycle (~2β4 min) + the usual crawl/enrichment. Deep-inspect multi-arch currently ~2 min x86 (KVM) + ~11 min arm64 (TCG, 1024 MiB). Nightly would be similar after upgrade. Needs 1024 MiB RAM (extra-packages rule already). |
| **Website impact** | `docs/docs-index.json` via `scripts/build-docs-index.mjs` + `docs/restraml-shared.js` `fetchVersionList()` / `parseVersion` / `compareVersions` / `isPreRelease` currently do not know `ab`. Need to teach them a synthetic `nightly` slot (e.g. `nightly` or `7.25-ab-nightly`) that sorts as newest and is optionally hidden behind a toggle. `diff.html`/`lookup.html` compare paths would then work unchanged. |
| **Deep-inspect contract** | `deep-inspect.x86.json` / `arm64.json` + `diff-deep-inspect.json` per-arch remains viable β nightly has both arch `.npk` sets available. First iteration could ship x86-only to prove upgrade path. |
Open question from thread that probes didn't fully resolve: is the share updated strictly nightly and does it always contain the full extra-package set? Today it did, and mtimes were spread over ~10 min (build artifact upload window), but @amm0's recollection of gaps suggests the workflow should tolerate missing/sparse nightly payloads gracefully (skip publication, don't fail the main auto pipeline).
---
## Strawman: one overwritten `nightly` slot
Intent: cheapest viable integration that answers the forum request without unbounded storage.
### Published shape
```
docs/nightly/
schema.raml
inspect.json
deep-inspect.json # or deep-inspect.x86.json if starting x86-only
openapi.json
extra/
schema.raml / inspect.json / openapi.json
deep-inspect.x86.json
deep-inspect.arm64.json (P2 β add arm64 after x86 path is green)
diff-deep-inspect.json
nightly.json # provenance: { nightlyVersion: "7.25_ab430", source: "box:/...", builtAt, baseVersion }
```
- Fixed directory name `nightly` (alternatives: `nightly-build`, `ab-nightly` β bikeshed in PR). **Overwritten on each successful nightly build**, not versioned per `ab`.
- `nightly.json` records the underlying `7.25_ab430` so diffs/changelogs can still name the real RouterOS build.
- `docs/docs-index.json` and UI treat `nightly` as a synthetic version: `parseVersion("nightly") -> { major: Infinity }` or special-case sort-to-top, with a badge and `isPreRelease("nightly")==true` so it's hidden unless "Include testing/nightly" is checked.
### Build flow (x86 start)
1. **Discover** nightly version: `302` via `mt.lv/nightly-build` β extract token β `GET /api/v2.1/share-links/{token}/dirents/?path=/` β parse `7.25_abXXX` from `routeros*-*.npk` filenames. If no new version vs `docs/nightly/nightly.json`, exit 0 (no-op).
2. **Boot stable CHR** as today (`qemu-system-x86_64 -enable-kvm -m 1024 -drive ...if=virtio -netdev user,hostfwd=...:80,hostfwd=...:22`).
3. **Download nightly npks** for target arch (today: `curl -L https://box.mikrotik.com/d/<token>/files/?p=/<file>&dl=1` β verified above, redirects to `seafhttp/files/<uuid>/<file>`).
4. **Upgrade CHR:** `scp -P 9122 nightly-x86/*.npk admin@localhost:/` then reboot via `POST /rest/system/reboot`; wait for REST ready (`/rest/system/resource` Γ2 consecutive), verify `/rest/system/package` shows β₯10 packages and `GET /rest/system/resource` `version` matches `7.25_ab430`.
5. **Collect schemas** with existing tooling against upgraded CHR: `bun rest2raml.js`, `bun deep-inspect.ts --live --arch x86 --transport rest`, `bun validate-openapi.ts`, optional `enrich-openapi.ts`. Same `publish-with-retry` pattern (stage `docs/nightly/` + regenerated `docs/docs-index.json`).
6. Schedule: **separate workflow** `nightly.yaml` (daily cron, e.g. `0 6 * * *`, + `workflow_dispatch` with `nightly_version` override), independent of `auto.yaml` so a flaky Box share never blocks stable/beta promotion.
### P2 follow-on
- Add `arm64` nightly: boot `ubuntu-24.04-arm` with `qemu-system-aarch64 -m 1024`, upgrade via `*-arm64.npk` set, produce `deep-inspect.arm64.json` + `diff-deep-inspect.json` under `docs/nightly/extra/` matching `deep-inspect-multi-arch.yaml`.
- UI: add "Include nightly" toggle alongside "Include testing"; nightly sorts first; `diff.html?compare1=7.24rc4&compare2=nightly` etc.
- Consider keeping a one-entry rolling history file (e.g. `docs/nightly/history.json` of last N `ab` builds) if users want to diff between nightlies β but not a per-ab docs dir.
---
## What still needs a live test before merging
- [ ] Confirm `scp` + reboot upgrade reliably moves a stable CHR to the nightly version (and which package set is authoritative for x86 β unsuffixed vs `routeros-x86-*.npk` deduplication).
- [ ] Verify `fetchVersion()` returns `7.25_ab430` verbatim and that existing Bash/JS version comparisons can be extended without breaking `auto.yaml` channel logic.
- [ ] Local dry-run of `build-docs-index.mjs` with a `docs/nightly/` directory present to ensure `VERSION_RE` / `compareVersions` / `latestVersion` handling is sane.
- [ ] Decide `nightly` vs `7.25_ab430` directory naming for URL stability (recommend fixed `nightly`).
---
## Alternatives considered (not recommended)
- **One dir per `ab` (`docs/7.25_ab430/`, `docs/7.25_ab431/`, β¦)** β unbounded growth (β60 Γ number of nightlies), Pages indexing churn, defeats the "one slot" cost goal. Could be revisited only if nightly cadence proves sparse and we explicitly want history.
- **Do nothing** β leaves the gap fischerdouglas flagged: nightly is the only train with no inspect-derived diff. Rosetta already ingests `manual.mikrotik.com` for prose, so the schema side is the missing piece.
---
## References
- Forum thread: https://forum.mikrotik.com/t/https-tikoci-github-io-restraml-to-https-mt-lv-nightly-build-is-to-costly/272302
- Current nightly payload: `box.mikrotik.com/d/5ce46054d5d2487e8755` (token via `mt.lv/nightly-build` 302)
- Existing version channels: `https://upgrade.mikrotik.com/routeros/NEWESTa7.{stable,testing,development,long-term}`
- Related project handling nightly docs: `manual.mikrotik.com` (already tracks main) β `tikoci/rosetta` picks it up; this proposal covers the schema/tool side.
- Prior discussion on xfrm: https://manual.mikrotik.com/docs/cli-reference/interface/xfrm
cc @amm0 β filing as discussed for prioritization against other channel/backfill work (`BACKLOG.md` P1 deep-inspect backfill, P2 merge/openapi tasks).