✨ Steering AI to use new manual.mikrotik.com

Amm0 · June 18, 2026, 5:32pm

LOL. So after my post above, had some updates centrs to "push". The build failed, strangely a DNS error. My DNS works, but from forum there were more wide-spread troubles with *.mikrotik.com that seeming infected GitHub's "Runners" (containers that build/test code in GitHub [MS/Azure] cloud). That's because in "AI trilogy of tools", quickchr is needed to test centrs (but getting a DNS failures downloading images at the current stable/long-term), and rosetta needed to download the doc archive from box.mikrotik.com.

Some might find it interesting look at how the "clunker" handled yesterday's DNS outage:

centrs spotted the DNS issue since integration tests failed (all code is well tested, and reviewed by 2 other "code review" agents, before pushed to main). Since centrs immediate goal was getting the unit/integration test to pass, Claude Code offered to hack /etc/hosts since it thought it was an IPv6 on GitHub issue (it wasn't, but the runner didn't not have IPv6, so not a bad a guess). So I "referred" the matter to quickchr.
quickchr, after prompting, took a deeper look at the problem since its directives essentially steer towards being "always grounded & robust/rich in error handling", especially since it's used in other projects as the test harness for RouterOS in GitHub Actions runners. So the centrs failures were a "bug" in quickchr. Which then triggered it's instructions to "own" being the "CHR expert", so why IPv6 & GitHub runners were not working become it problem, since it should know about CHR related topics. So it's analysis lead to this "memory" that guided the eventual fix:

---
name: chr-ci-ipv6-dualstack
description: "Why CHR-integration on GitHub Actions fails at quickchr resolveVersion — runner DNS returns ESERVFAIL for *.mikrotik.com; FIXED in quickchr via public DNS, not centrs"
metadata: 
  node_type: memory
  type: project
  originSessionId: 18a92444-2298-4743-9fdd-0d554dec0b19
---

**Root cause (confirmed by on-runner probe 2026-06-16, supersedes the earlier
IPv6 theory):** GitHub-hosted `ubuntu-latest` runners' **system DNS resolver
returns `ESERVFAIL` for `upgrade.mikrotik.com` / `download.mikrotik.com`** —
slowly (2–26 s) — via *both* getaddrinfo and c-ares-over-resolv.conf. So a plain
`fetch` times out resolving, or (when the stub hands back only the unreachable
AAAA) fails with Bun's `errno: 0` ConnectionRefused / FailedToOpenSocket. A
**direct query to a public resolver (1.1.1.1 / 8.8.8.8) answers in ~10 ms**.
The "IPv6 happy-eyeballs no-egress" framing was a **red herring** — the fault is
DNS at the runner's stub, not IPv6 egress. This also explains why `getent
ahostsv4` returned empty on the runner (same broken stub).

This killed quickchr's `resolveVersion()` before any CHR booted, failing every
CHR test in the QA `chr-integration` job. Surfaced 2026-06-16 by the first
manual `workflow_dispatch` QA runs that forced CHR-on-GHA to reach MikroTik
(push/PR runs gate the job off, so it had never resolved a version before).

**This is a quickchr bug, NOT a centrs bug — do not work around it in centrs.**
Fixed in quickchr `fetchResilient()` (`src/lib/net.ts`, tikoci/quickchr#1):
resolves the A record via `dns.Resolver.setServers([1.1.1.1,8.8.8.8,1.0.0.1])`
(public DNS, 3 s timeout), connects to the IPv4 literal with `Host` + TLS SNI
preserved, and falls back to a normal fetch when public DNS is blocked. Verified
green end-to-end on the real runner (quickchr's Integration job downloads CHR +
boots via QEMU). centrs needs **no** workflow change — just bump
`@tikoci/quickchr` once the fix is released (>0.4.0).

**Rejected workaround (don't repeat):** an `/etc/hosts` IPv4 pin via
`getent ahostsv4 <host>` in qa.yaml — wrong layer (belongs in quickchr) **and**
non-functional (hit the same broken stub resolver → empty). centrs PR #53 was
closed unmerged.

**How to apply:** if a sister project's CHR-in-CI dies at `resolveVersion`
(slow `ESERVFAIL` or `errno: 0`), route it to quickchr — review/extend
`fetchResilient` there, don't patch the consumer's workflow. See
[[june-gloom-phase2-ci-techdebt]].

Since quickchr rejected centrs hack, and planned to add fallback code to resolve *.mikrotik.com using 1.1.1.1 or 8.8.8.8 or 1.0.0.1... so future DNS issue try alternative resolution and be more IPv6 vs IPv4 aware. But this "PR" shows how the system works to implement the fix... So this included Claude Opus coding the fix to quickchr, while CoPilot (using GPT 5.5) reviews the code changes, with a third-party CodeRabbit code review agent (free from OSS) provides a "third eye" on the changes...

github.com/tikoci/quickchr

fix(net): IPv4 fallback for downloads on IPv6-less egress (CI runners) (#1)

main ← fix/resilient-fetch-ipv4

opened 10:17PM - 16 Jun 26 UTC

mobileskyfi

+315 -10

## Problem A sister project (centrs) hit repeated CI failures where every CHR t…est died at quickchr startup (`resolveVersion`, `errno: 0`). The original "IPv6 no-egress / happy-eyeballs" theory turned out to be a **red herring**. An on-runner DNS probe (GitHub `ubuntu-latest`, 2026-06-16) found the real fault — **the runner's system resolver returns `ESERVFAIL` for `*.mikrotik.com`**, slowly, via every system path: | Resolver | Result | |---|---| | `dns.lookup({family:4})` (getaddrinfo) | ❌ `ESERVFAIL` ~9 s | | `dns.lookup({all})` (getaddrinfo) | ❌ `ESERVFAIL`/`ETIMEOUT` 22–26 s | | `dns.resolve4` (c-ares → resolv.conf) | ❌ `ESERVFAIL` 2–22 s | | normal `fetch` | ❌ times out | | **`Resolver([1.1.1.1, 8.8.8.8]).resolve4`** | ✅ **~10 ms** → `159.148.147.251` | So a plain `fetch` times out resolving, or (when the stub hands back only the unreachable AAAA) fails with Bun's `errno: 0`. The only thing that works is querying a **public resolver directly**. This also explains the centrs `getent ahostsv4` → empty result (same broken stub). ## Fix New `fetchResilient()` (`src/lib/net.ts`), used at the three external call sites (`versions.ts`, `images.ts`, `packages.ts`; `rest.ts`/local 127.0.0.1 untouched): 1. Resolve the A record by querying **public DNS directly** — `dns.Resolver` with `setServers([1.1.1.1, 8.8.8.8, 1.0.0.1])`, bounded by a 3 s timeout — bypassing the runner's broken `resolv.conf` stub. 2. Connect to the **IPv4 literal** with the `Host` header and **TLS SNI** preserved so certificate validation still passes. 3. **Fall back** to a normal `fetch` (system resolver) when public DNS is unavailable (e.g. a network that blocks public resolvers), so it stays correct off CI. 4. HTTP responses and aborts/timeouts pass through unchanged — never retried. ## Verified - **CI on the real runner is green end-to-end**, including the **Integration job** that downloads CHR images and boots them via QEMU (the exact path that was failing): 11/11 + 6/6 + 7/7 + 6/6, 0 fail. - Unit suite **555 pass / 0 fail** (Linux + Windows); `biome`+`tsc` clean; `cspell` clean. - `test/unit/net.test.ts` (9 cases): URL rewriting, connection-failure classification, IPv4 happy path (asserts Host + SNI), 5xx pass-through, no-public-DNS-answer fallback, IPv4-socket-failure fallback, abort pass-through. Download suites mock the public-DNS lookup to stay network-free. ## Docs DESIGN.md decision #9 records the measured root cause and a **sister-project routing rule**: a CI-only `ESERVFAIL`/`errno: 0` download failure is a quickchr concern — don't work around it in the consumer's workflow. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

rosetta does not use quickchr, rather it needs to download the HTML pages from box.microtik.com which was failing yesterday. Since it was pretty clear the issue was on the MikroTik side, and figured it be fixed. I ignored the first failure (but kinda proved it was not a quickchr): Release · tikoci/rosetta@990ca32 · GitHub so I could re-test once report from forum showed it was likely fixed, which it was this morning PDT: Release · tikoci/rosetta@990ca32 · GitHub