refactor(tui): fetch tree-sitter grammars at runtime instead of vendoring

The OpenTUI engine vendored 10 tree-sitter grammars (.wasm + .scm) under
ui-opentui/parsers/ — ~37k checked-in binary lines, the single biggest
addition in the engine diff. opencode (the production reference) vendors
none: it declares grammars as remote URLs and lets OpenTUI fetch + cache
them. OpenTUI supports this natively via TreeSitterClient's dataPath cache.

Migrate to that model:
- parsers.manifest.json (now under src/boundary/) becomes the URL source of
  truth: each grammar is { filetype, aliases, wasm: <release URL>,
  highlights: <.scm URL> }. Grammar versions stay pinned (same release tags);
  .scm sources follow opencode's per-language choices (parser-repo queries
  for python/html where nvim-treesitter's are parser-incompatible).
- parsers.ts: registerVendoredParsers -> registerRemoteParsers. It points the
  global tree-sitter client's cache at HERMES_TUI_PARSER_CACHE via setDataPath
  BEFORE the client initializes, then addDefaultParsers() with the URL configs.
  Registration does zero network; the fetch is lazy on first use of a language
  and degrades to plain text (never throws) when GitHub is unreachable.
- hermes_cli/main.py sets HERMES_TUI_PARSER_CACHE to
  ~/.hermes/cache/opentui-parsers/ (profile-aware via get_hermes_home).
- git rm -r ui-opentui/parsers/ and drop scripts/update-parsers.mjs.
- parsers.test.tsx asserts URL configs are well-formed + cache-dir behavior
  instead of vendored-file existence.

Verified end-to-end on Node 26.3: type-check + lint clean, full ui-opentui
suite (821 tests) green, and a built smoke proves first-use fetch -> cache ->
10 real highlights, cache-hit on rerun, and graceful plain-text degrade when
the grammar URLs are unreachable.

docs/opentui-native-engine.md

@@ -0,0 +1,432 @@
+# OpenTUI native engine — PR documentation
+
+**Branch:** `feat/opentui-native-engine` · **Base:** `origin/main` (merged in; HEAD is at `~main`)
+**New engine root:** `ui-opentui/` (Node 26 + `@opentui/core` 0.4.1 + `@opentui/solid`, Effect at the boundary)
+**Legacy engine root:** `ui-tui/` (React + the `@hermes/ink` fork at `ui-tui/packages/hermes-ink/`)
+
+> This is the canonical in-repo doc for the PR. The companion interactive HTML
+> write-up (`~/projects/opentui-perf-writeup/index.html`) is the case/benchmark
+> deep-dive; this doc is the reviewable text version + the four things review
+> actually needs: **(1) the LoC reduction math, (2) the measured perf deltas,
+> (3) the real UI divergence (with screenshots), (4) the non-core / kitchen-sink
+> change audit.**
+
+This PR adds a from-scratch native terminal UI built on OpenTUI, intended to
+replace the React/Ink TUI **and the Ink fork we maintain alone**. It currently
+ships as a parallel engine (Ink untouched, auto-fallback), selected by
+`HERMES_TUI_ENGINE` env > `display.tui_engine` config > auto (OpenTUI when the
+host is Node ≥ 26.3 with the built bundle, else Ink). **100% parity with the Ink
+TUI is the bar.**
+
+---
+
+## 1. Line-of-code reduction (the headline maintenance win)
+
+All counts are **git-tracked files only** (respects `.gitignore`; `dist/` and
+`node_modules/` are untracked and excluded). Measured live on this branch at
+`~HEAD`. "Code" = `.ts/.tsx/.js/.jsx` only; "total" includes config/json/md.
+
+### What gets *removed* when Ink is retired
+
+| Area | Files | Total lines | Code lines (ts/tsx/js) | Non-blank code |
+|---|---:|---:|---:|---:|
+| `ui-tui/src/` — Ink **consumer app** (our React/Ink view code) | 204 | 40,422 | 40,422 | 33,550 |
+| `ui-tui/packages/hermes-ink/` — **the fork** (`@hermes/ink`) | 148 | 28,167 | 28,113 | 23,718 |
+| **`ui-tui/` whole tree (tracked)** | **362** | **69,320** | **68,831** | **57,545** |
+
+The `ui-tui/` whole-tree number (69,320) also folds in a handful of build
+scripts, `.prettierrc`, `package.json`, etc. The two rows above it are the
+load-bearing split:
+
+- **The fork alone is 28,167 LOC across 148 files** — code we own and can never
+  sync from upstream. Upstream Ink v6.8.0 `src/` is ~7,259 LOC, so the fork's
+  renderer core is **~3.2× the size of stock Ink**. (Cross-checked against the
+  HTML write-up's `ink-fork-analysis.json`: 28,111 LOC / 148 files — the 56-line
+  delta is a single tracked JSON the file-level count includes.)
+- **The consumer app is another 40,422 LOC** — React components/hooks that only
+  exist to drive Ink.
+
+### What gets *added*
+
+| Area | Files | Total lines | Code lines | Non-blank code |
+|---|---:|---:|---:|---:|
+| `ui-opentui/src/` — new engine (app code **+ its own tests**) | 153 | 28,763 | 28,763 | 26,495 |
+|   ↳ non-test (app code only) | 97 | 16,628 | 16,628 | 15,450 |
+|   ↳ tests (`src/test/`) | 56 | 12,135 | 12,135 | 11,045 |
+| Tree-sitter grammars (`python`…`toml`) | 0 | 0 | 0 | 0 |
+| **`ui-opentui/` whole tree (tracked)** | **~170** | **~34,800** | **29,614** | **27,283** |
+
+> Tree-sitter grammars carry **zero repo lines**: the engine declares the 10
+> extra grammars as remote URLs (`src/boundary/parsers.manifest.json`) and
+> OpenTUI fetches+caches each `.wasm`/`.scm` on first use into
+> `~/.hermes/cache/opentui-parsers/` (à la opencode, which vendors none). An
+> earlier revision vendored them as 37,302 checked-in binary lines (10 `.wasm` +
+> 10 `.scm`); that's gone — code lines and total lines now move together.
+
+### The net reduction (code lines, the honest comparison)
+
+| Comparison | Removed (ts/tsx/js) | Added (ts/tsx/js) | Net change |
+|---|---:|---:|---:|
+| **Incl. fork** — retire all of `ui-tui/` vs add `ui-opentui/src` | −68,831 | +28,763 | **−40,068 LOC (−58%)** |
+| **Incl. fork, app-vs-app** (exclude both test suites) | −56,463¹ | +16,628 | **−39,835 LOC (−71%)** |
+| **Excl. fork** — only the Ink *consumer app* vs new engine | −40,422 | +28,763 | **−11,659 LOC (−29%)** |
+| **The fork in isolation** (the unsyncable liability we shed) | −28,113 | — | **−28,113 code lines deleted outright (28,167 incl. its 1 config file)** |
+
+¹ `ui-tui/src` non-test = 28,350 LOC + fork (≈ all 28,113 code lines are non-test;
+it carries only ~54 config lines) = 56,463. (`ui-tui/src` carries 80 test files /
+12,072 LOC; the new engine carries 56 test files / 12,135 LOC.)
+
+**Read it this way:**
+
+- **The cleanest single number: ~−40k code lines net** (retire all of `ui-tui/`,
+  add `ui-opentui/src`). That is a **~58% reduction in the TUI's
+  hand-maintained surface**, and it *includes* the new engine's full 56-file test
+  suite.
+- **The most important number is the fork: −28,167 LOC of unsyncable engine
+  code** disappears. That is the load-bearing maintenance win — it's not just
+  fewer lines, it's lines we are the *sole* maintainer of (own reconciler, ANSI
+  parser, scrollbox, selection/OSC52, hand-rolled memory eviction, Yoga binding).
+- **Even excluding the fork** — i.e. if you imagine upstream Ink were free — the
+  app rewrite is still a net reduction (−11,659 LOC) because the new engine
+  mounts OpenTUI built-ins instead of hand-building components.
+
+### Caveat on the comparison (keep it honest for review)
+
+- These are **whole-tree retirements vs a single source dir add.** If/when Ink is
+  deleted, the `ui-tui/` `package.json`, lockfile, and build scripts go too; the
+  table counts `ui-tui/src` + the fork as the apples-to-apples "hand-maintained
+  TS" figure.
+- **Tree-sitter grammars are NOT vendored.** The 10 extra grammars are declared
+  as remote URLs (`src/boundary/parsers.manifest.json`); OpenTUI fetches each
+  `.wasm`/`.scm` on first use of a language and caches it under
+  `~/.hermes/cache/opentui-parsers/` (profile-aware, set via
+  `HERMES_TUI_PARSER_CACHE` by the launcher). Registration does **zero** network;
+  the fetch is lazy and off the boot critical path, and an unreachable
+  GitHub/air-gapped env degrades that language to plain text — never a throw. This
+  replaces an earlier revision that vendored 37k binary lines, so the repo no
+  longer grows on disk for syntax highlighting. (Trade-off: first-use-per-language
+  needs network to `github.com`/`raw.githubusercontent.com`; pre-seed the cache in
+  a Docker build if you need offline highlighting.)
+- Python/backend LoC is **not** part of this reduction: `tui_gateway/` (~12k LOC)
+  is **shared by both engines** and stays. See §4.
+
+---
+
+## 2. Performance (CPU / latency / memory)
+
+Measured with the `tui-bench` harness driving **both engines on a real PTY
+120×40**, fake gateway feeding deterministic events, `/proc`-sampled identically,
+each SUT under `systemd-run --scope -p MemoryMax=2G -p MemorySwapMax=0`,
+sequential with a load-gate + 10s cooldown. Determinism gate **GREEN**, 71 result
+files, 0 cell errors, 3 reps/cell, `@opentui/core` 0.4.1 native-yoga
+(`libopentui.so`, no `yoga.wasm`). Every number traces to a `summary.<field>` in
+a result dir. Source: `~/projects/opentui-html/bench-numbers.json` (frozen
+2026-06-14, build under test `1ddf7a102` + WIP).
+
+### Scorecard
+
+| Dimension | Winner | Margin | Source cell |
+|---|---|---|---|
+| Streaming frame rate | **OpenTUI** | **~3×** (43 vs 14 fps) | `cpu800.frame_pacing` |
+| Streaming smoothness (interframe p95) | **OpenTUI** | **40ms vs ~220ms** (no ¼-second stalls) | `cpu800.frame_pacing` |
+| Scroll CPU | **OpenTUI** | **~2.7× cheaper** (134–155 vs 403–416 ticks) | `scroll3000.scroll.cpu_ticks` |
+| Cold-start floor | **OpenTUI** | ~97–103 vs ~107–109 MB | `startup.vmhwm_kb` |
+| Session-create latency | **OpenTUI** | ~151–177 vs ~204–229 ms | `startup.session_create_ms` |
+| First-byte paint | Ink | ~93 vs ~122 ms | `startup.first_byte_ms` |
+| Memory @ small/typical | Ink | OpenTUI +30–50 MB | `mem50/100/300.vmhwm` |
+| Memory @ heavy tool output | **OpenTUI** | **crossover** (258–265 vs 280–290 MB) | `results-fat-mem-*` |
+| Layout reflow latency | **Ink** | **~0ms vs ~13ms** (OpenTUI's one honest loss) | `resize3000.resize.reflow_ms` |
+
+### The honest reading
+
+- **OpenTUI wins everything you feel continuously** — frame rate (~3×), scroll
+  CPU (~2.7×), and smoothness (no 200ms hitches; p95 40ms vs ~220ms). This is the
+  lead. The single most user-perceptible difference is the stall-free stream.
+- **Memory: lead with smoothness, not raw RSS.** Ink is lighter at small/typical
+  sizes (OpenTUI carries a ~102 MB irreducible Node+V8+`libopentui.so` floor, so
+  it sits +30–50 MB above Ink there). But it **crosses over** under heavy tool
+  output (mem300: 258–265 MB OpenTUI vs 280–290 MB Ink) because windowing beats
+  Ink's mount-every-row. Real-world: 20 memwatch sessions show a flat ~108 MB
+  floor and ~0 MB/h on long sessions (one 15h session, 0 MB/h; one 4.4h session
+  plateaus flat at ~237 MB with mounted rows pinned at 33).
+- **The one outright loss is layout reflow** (~13ms p50 vs Ink's ~0ms; under a
+  resize storm OpenTUI degrades to ~14fps/~197ms vs Ink ~26fps/~100ms). Heavier
+  native renderables vs Ink's string nodes. This is a real, quantified
+  optimization target — **not** a regression vs current behavior, and **not** the
+  "halved 0.4.0→0.4.1" delta (we measured the absolute 12–15ms only; do not quote
+  "halved" from this run).
+- **The memory fix is engine-agnostic** — a rolling display cap
+  (`HERMES_TUI_MAX_MESSAGES=3000` default) that is display-only and never touches
+  the model's context. Uncapped is a stress config, not real usage (10k msgs
+  uncapped: 793 MB; capped sessions are flat MB/h).
+- **Gut-check vs upstream/opencode: no bugs.** Exactly one frame callback
+  (early-exits cheaply), zero `writeToScrollback` for the transcript (one sticky
+  `<scrollbox>` + reactive `<For>`), native `<markdown streaming>` byte-for-byte
+  parity with live opencode, no reactive-read-outside-tracking-scope (the #1 Solid
+  trap). Source: `docs/plans/opentui-gutcheck-verification.md`.
+
+Full methodology + every cell: see the HTML write-up's benchmark sections and
+`docs/plans/opentui-endgame-benchmark-report.md`.
+
+---
+
+## 3. UI parity — and where the two engines genuinely diverge visually
+
+100% *feature* parity is the bar (matrix in §6), but the two engines are **not**
+visually identical. The Ink TUI renders the transcript as a **box-drawing tree**;
+OpenTUI renders it **flat and marker-based**. This is a deliberate design
+divergence, captured in `ui-opentui/src/view/messageLine.tsx`:
+
+> *"the view is a dark room and gold is the single lamp — it sits on the NEWEST
+> answer's `⚕` and the user's `❯`, nowhere else (older assistant glyphs demote to
+> grey: they merely happened)."*
+
+Real screenshots (saved under `docs/research/opentui-screenshots/`), captured live
+on a real PTY 120×40 via the `tmux-pane-screenshot` workflow — **same session
+resumed in both engines** where possible.
+
+### Legacy Ink — `docs/research/opentui-screenshots/ink-transcript.png`
+
+![Ink transcript](research/opentui-screenshots/ink-transcript.png)
+
+- **Box-drawing tree layout.** Each turn is a nested structure: `└─ Response`,
+  `└─ ▾ Tool calls (1)`, `   └─ ● Terminal("…")` — explicit corner rails and
+  disclosure triangles.
+- **`┊` dotted quote-bar** prefixes assistant prose.
+- **Tool calls collapse by default** behind a `▾ Tool calls (N)` disclosure,
+  nested one rail deeper.
+- **Whole assistant message tinted gold/amber** (body text is colored, not just
+  the marker).
+- Right-edge scrollbar: thin `│` track + `┃`/orange thumb.
+- Status bar: `─ ready │ opus 4.8 fast high │ 0/1m │ [░░░░░░] 0% │ 25s │ voice off │ 1 session ─ ~`
+  — leading dash, pipe-delimited fields, trailing `~`.
+- **No top header bar.**
+
+### New OpenTUI — `docs/research/opentui-screenshots/opentui-transcript.png` (+ `opentui-toolcall.png`)
+
+![OpenTUI transcript](research/opentui-screenshots/opentui-transcript.png)
+
+![OpenTUI tool call](research/opentui-screenshots/opentui-toolcall.png)
+
+- **Flat, marker-based layout.** No tree rails. Assistant = `⚕` (caduceus, gold
+  only on the newest answer), user = `❯` (gold chevron + gold text). Older
+  assistant glyphs demote to grey.
+- **Neutral body text.** Gold is reserved for markers and inline-code accents;
+  prose is grey/white (the "single lamp" rule), so the screen reads calmer than
+  Ink's all-amber blocks.
+- **Tool calls render inline, expanded, on one header line:**
+  `⚕ ▶ delegate_task  Run the shell command `…`  (/agents to monitor)  · 41s  (11 lines)`
+  — marker, `▶` collapse triangle, bold tool name, grey arg preview, hint,
+  `· duration`, `(N lines)` — and the result flows flat directly below (no nesting
+  rail). Per-tool renderers exist (`view/tools/registry.tsx`) — bash/file+diff/
+  read/search/skill/clarify/todo each render differently, not a uniform dump.
+- **Per-block `⧉ copy` affordance** on a quiet footer line under every settled
+  assistant block and user prompt (click → copies that block's source).
+- **Top header bar:** `⚕ Hermes Agent · opentui · ready` + a gold horizontal rule
+  (Ink has none).
+- Status bar (real backend): `● claude-fable-5 │ [▒▒▒] 4% │ …/lively-thrush/hermes-agent (feat/opentui-native-engine)`
+  — green status dot, model, context/token bar, **right-pinned cwd + branch**.
+
+### Divergence summary table
+
+| Aspect | Ink (legacy) | OpenTUI (new) |
+|---|---|---|
+| Transcript structure | Box-drawing **tree** (`└─`, rails) | **Flat**, indented, marker-based |
+| Assistant marker | `└─ Response` rail + `┊` quote-bar | `⚕` caduceus glyph |
+| User marker | (rail) | `❯` gold chevron |
+| Assistant body color | Tinted gold/amber | Neutral grey/white (gold = accents only) |
+| Tool calls | Collapsed `▾ Tool calls (N)`, nested | Inline expanded header + flat result |
+| Per-tool rendering | Largely uniform | Dedicated renderers per tool |
+| Copy affordance | `/copy` command | `/copy` **+ per-block `⧉ copy`** |
+| Header bar | None | `⚕ Hermes Agent · opentui · ready` + rule |
+| Status bar | `─`/`│`-delimited, trailing `~` | dot + bars + right-pinned cwd/branch |
+
+**For review:** the divergence is intentional (a design pass, not an accident),
+but it means "drop-in replacement" is true at the *feature* level, not the
+*pixel* level. A user switching engines will immediately notice the flatter,
+calmer transcript. Worth calling out explicitly so the swap isn't sold as
+visually invisible.
+
+---
+
+## 4. Non-core / kitchen-sink change audit (what review should scrutinize)
+
+Full report: **`docs/research/opentui-noncore-change-audit.md`** (file-by-file,
+commit-by-commit, with `file:line` evidence). Summary below.
+
+This PR's net footprint vs `origin/main` (two-dot diff = exactly this PR's adds,
+no main work re-included):
+
+| Bucket | Files | Net diff |
+|---|---:|---:|
+| UI (`ui-opentui/`, the engine + tests) | 197 | +36,001 / −1 |
+| Docs | 8 | +1,164 / −0 |
+| **Other (the review-flag surface)** | **28** | **+3,218 / −204** |
+
+The 28 "other" files are the only place this PR touches shared Hermes core. They
+classify as:
+
+### ✅ CORE-OPENTUI-NECESSARY (the engine can't work without these; Ink path provably untouched)
+
+- **`hermes_cli/main.py`** (+382/−5) — dual-engine launcher (engine resolution,
+  Node 26 / fnm detection, `_make_opentui_argv`, heap override). Default falls
+  back to Ink unless the host is OpenTUI-ready (`main.py:1685`); OpenTUI is
+  dispatched *around* the Ink bootstrap, never through it (`main.py:1914-1922`).
+- **`scripts/install.sh`** (+78/−1) — `install_opentui` stage, **strictly
+  best-effort** (every failure returns 0; falls back to Ink; Windows/Termux
+  skipped). Ink install path unchanged.
+- **`Dockerfile`** (+21/−11) — Node 22→**26** bump (required by the `node:ffi`
+  renderer) + `ui-opentui` build step. Opt-in; Ink build line preserved. **Caveat:
+  the Node major bump affects the whole image (Ink + web + Playwright)** — the
+  diff self-flags "verify the full image build on Node 26 in CI."
+- **`hermes_cli/_parser.py`** (+16/−2) — bare `--resume` → OpenTUI session picker;
+  `--resume <id>` unchanged.
+- **`tui_gateway/server.py`** (+612/−40) — predominantly opt-in RPCs/fields the
+  new engine calls (`session.peek`, `session.list` filters, `startup.catalog`,
+  `diff_unified`, window-title, skin keys). Each is gated so **the Ink path is
+  byte-for-byte unchanged** (`server.py:3930`, `:4254`, `:10447`). *Note:* this
+  file also carries some of the cost-accounting code (below) — separable.
+
+> `tui_gateway/` (~12k LOC Python) is **shared by both engines** and is **not**
+> removed when Ink is retired. Only the `ui-tui/` frontend tree goes.
+
+### 🚩 FLAG FOR REVIEW — Category C, separable from an OpenTUI PR
+
+These do **not** need to ship with the engine and a reviewer should ask to split
+them out:
+
+1. **Provider-reported-cost accounting** (commits `85546bb9e` + `364b93a4b` +
+   `e01b04de4`) — a coherent feature spanning **11 files**: `agent/usage_pricing.py`,
+   `plugins/model-providers/openrouter/__init__.py`,
+   `agent/transports/chat_completions.py`, `agent/agent_init.py`, `run_agent.py`,
+   `agent/conversation_loop.py`, `agent/account_usage.py`, `hermes_state.py`,
+   `gateway/slash_commands.py`, the cost half of `cli.py`, and the
+   `_get_usage`/`_compact_usage_text` blocks of `tui_gateway/server.py` (+ 5 test
+   files). Strongest evidence: commit `85546bb9e` *"gateway: capture real
+   provider-reported cost (openrouter usage accounting)"* — a provider-accounting
+   rework, not a renderer.
+2. **`plugins/model-providers/openrouter/__init__.py`** — sends
+   `usage:{include:true}`, a provider request-shape change affecting *all*
+   interfaces, not just the TUI (`openrouter/__init__.py:85-90` cites the
+   OpenRouter usage-accounting docs).
+3. **Worktree lock / dirty-tree preservation** (commit `94765e48f`,
+   `cli.py` + `tests/cli/test_worktree.py`, ~145 lines) — git-worktree lifecycle
+   safety plumbing with **zero TUI references** (`cli.py:1391-1545`, `:1635-1713`).
+4. **`tools/clarify_tool.py`** (+16/−4) — docstring/schema-description-only fix
+   (commit `16e408f3f`); applies to every interface, trivially separable.
+
+### ✅ Conversation-loop / role-alternation / prompt-cache correctness verdict: **NO RISK**
+
+Verified: none of `run_agent.py`, `agent/conversation_loop.py`,
+`agent/agent_init.py`, `agent/transports/chat_completions.py` touch
+message-role alternation or the prompt-cache prefix. The
+`conversation_loop.py` added lines grep clean for
+`cache_control|alternation|prompt_cach|api_messages`; the cache/alternation
+machinery (`:57`, `:660-674`, `:759`) is untouched; the PR's insertion at
+`:1809-1879` is purely additive cost bookkeeping after `cost_result`. **Prompt
+caching and strict role alternation are preserved.**
+
+---
+
+## 5. What this does and does NOT fix
+
+**Fixes (structurally, by replacing the rendering substrate):** the renderer bug
+class — layout/scroll/input/copy/mouse/markdown/resize — plus the
+hand-maintained memory-eviction problem (windowing + Solid keyed `<For>`
+unmount→`destroy()`→`free()`), and several long-open feature requests (mouse,
+collapsible tool calls, session title/status bar, double-ESC, chronological
+thinking/tool ordering).
+
+**Does NOT fix:** the gateway is unchanged — the biggest single hotspot file in
+triage is `tui_gateway/server.py`, and whole bug clusters are gateway/Python-side
+(WS write-timeout/RPC pool, MCP-failure startup freezes, shell.exec denylist).
+The engine swap addresses rendering/input/scroll/memory; **gateway bugs ride
+along.** The Effect-boundary hardening does make those failures *visible* (typed
+events → system lines instead of a frozen spinner) and the TUI auto-heals
+(crash → backoff → respawn → resume, capped 3/60s).
+
+---
+
+## 6. Feature parity matrix (vs the Ink TUI)
+
+Verbatim, detailed, surface-by-surface with `file:line` evidence:
+**`docs/plans/opentui-ink-parity-matrix.md`** (interactive/filterable version in
+the HTML write-up). Headline state:
+
+| Surface | State |
+|---|---|
+| Transcript rendering (scrollbox, markdown, code, diffs, collapsible tools, reasoning, chronological order, windowing) | **full parity (9/9)** |
+| Blocking prompts (approval/clarify/sudo/secret/confirm) | **full parity (5/5)** |
+| Theming (skins, light/dark, ANSI-256 norm) | **full parity** |
+| Mouse / copy (tracking, selection, multi-click, OSC52, click-to-expand, wheel accel) | **full parity** |
+| Resilience (crash auto-heal + resume) | **parity++ (exponential backoff)** |
+| Composer / input | near parity — **missing: external editor (Ctrl+G → `$EDITOR`)**; ghost-text autosuggest partial |
+| Slash commands | core parity — **missing: `/setup`, `/redraw`, `/plugins`, `/voice`**; `/undo` prefill + `/image` partial |
+| Status bar / header chrome | almost all closed — **missing: MCP-servers panel, profile-in-prompt** |
+| Agent surfaces | most shipped — **missing: voice indicators, browser/CDP indicator** |
+| Utility commands | **missing: `/redraw`, `/setup`**; rest present |
+
+> The original PR-draft gap list was **substantially stale** — the WIP since
+> shipped context %/token bar, cost, compressions, duration, update banner, todos
+> panel, activity feed, notifications, background-task indicator, **and per-tool
+> renderers** (the "every tool renders the same" claim is false:
+> `view/tools/registry.tsx` has dedicated renderers).
+
+### Genuinely-remaining parity gaps
+
+- [ ] **External editor (Ctrl+G → `$EDITOR`)** — highest-impact missing composer affordance
+- [ ] MCP-servers detail panel; profile-in-prompt marker
+- [ ] Voice indicators (listening/transcribing/REC/STT) + `/voice`
+- [ ] Browser/CDP connection indicator + `/browser`
+- [ ] `/setup` wizard handoff, `/redraw`, `/plugins` hub
+- [ ] Draggable scrollbar; sticky-prompt line
+- [ ] `/undo` prefill into composer; model-picker persist-global toggle; skills-hub install/manage
+
+---
+
+## 7. Rollout, runtime & risks
+
+- **Runtime:** plain Node 26 (FFI floor 26.3+) — one runtime, no Bun. (Note: the
+  upstream OpenTUI docs say "requires Bun"; this engine deliberately runs on Node
+  26's experimental `node:ffi` instead — that's the load-bearing runtime decision.)
+- **Rollback:** Ink is untouched and remains the fallback; reverting is a launcher
+  decision, not a code revert.
+- **Default-engine selection:** auto-picks OpenTUI only when the host is genuinely
+  set up (Node ≥ 26.3 + built bundle), else Ink; explicit env/config bypasses the
+  probe.
+- **Known sharp edges:** `libopentui.so` native-lib distribution (P1 upstream:
+  copies can fill `/tmp`); the Dockerfile Node major bump needs full-image CI
+  verification; tree-sitter grammars are fetched from GitHub on first use and
+  cached in `~/.hermes/cache/opentui-parsers/` — air-gapped hosts get plain-text
+  highlighting until the cache is pre-seeded (the fetch never blocks boot and
+  never throws).
+
+## 8. Try it
+
+```bash
+hermes                              # auto-selects OpenTUI when the host supports it
+HERMES_TUI_ENGINE=opentui hermes    # force the native engine
+HERMES_TUI_ENGINE=ink hermes        # force the legacy Ink engine
+# preview standalone (no backend), Node 26:
+cd ui-opentui && npm install
+node scripts/build.mjs scripts/demo.tsx .demo
+DEMO_TOTAL=120 HERMES_TUI_MAX_MESSAGES=80 \
+  node --experimental-ffi --no-warnings .demo/demo.js   # inside a TTY
+```
+
+Requires Node 26.3+. On older Node / Windows / Termux it auto-falls-back to Ink.
+
+---
+
+## Appendix — source-of-truth files in this repo
+
+| Topic | File |
+|---|---|
+| Non-core change audit (full) | `docs/research/opentui-noncore-change-audit.md` |
+| Feature parity matrix (verbatim) | `docs/plans/opentui-ink-parity-matrix.md` |
+| Benchmark report | `docs/plans/opentui-endgame-benchmark-report.md` |
+| Gut-check verification | `docs/plans/opentui-gutcheck-verification.md` |
+| Ink↔OpenTUI capture asymmetry | `docs/plans/opentui-ink-asymmetry-note.md` |
+| UI screenshots | `docs/research/opentui-screenshots/{ink,opentui}-*.png` |
+| PR description (prose) | `docs/pr-description-main-doc.md` |
+| Interactive write-up | `~/projects/opentui-perf-writeup/index.html` (out-of-repo) |

hermes_cli/main.py

@@ -2281,6 +2281,18 @@ def _launch_tui(
     )
     os.close(active_session_fd)
     env["HERMES_TUI_ACTIVE_SESSION_FILE"] = active_session_file
+    # Tree-sitter grammar cache for the OpenTUI engine: grammars are fetched
+    # from GitHub on first use and cached here (profile-aware). Unset → OpenTUI
+    # falls back to its XDG default ($XDG_DATA_HOME/opentui). See
+    # ui-opentui/src/boundary/parsers.ts.
+    try:
+        from hermes_cli.config import get_hermes_home
+
+        env["HERMES_TUI_PARSER_CACHE"] = str(
+            get_hermes_home() / "cache" / "opentui-parsers"
+        )
+    except Exception:
+        logger.debug("Failed to resolve OpenTUI parser cache dir", exc_info=True)
     env["HERMES_PYTHON_SRC_ROOT"] = os.environ.get(
         "HERMES_PYTHON_SRC_ROOT", str(PROJECT_ROOT)
     )

ui-opentui/parsers/bash/highlights.scm

@@ -1,56 +0,0 @@
-[
-  (string)
-  (raw_string)
-  (heredoc_body)
-  (heredoc_start)
-] @string
-
-(command_name) @function
-
-(variable_name) @property
-
-[
-  "case"
-  "do"
-  "done"
-  "elif"
-  "else"
-  "esac"
-  "export"
-  "fi"
-  "for"
-  "function"
-  "if"
-  "in"
-  "select"
-  "then"
-  "unset"
-  "until"
-  "while"
-] @keyword
-
-(comment) @comment
-
-(function_definition name: (word) @function)
-
-(file_descriptor) @number
-
-[
-  (command_substitution)
-  (process_substitution)
-  (expansion)
-]@embedded
-
-[
-  "$"
-  "&&"
-  ">"
-  ">>"
-  "<"
-  "|"
-] @operator
-
-(
-  (command (_) @constant)
-  (#match? @constant "^-")
-)

ui-opentui/parsers/c/highlights.scm

@@ -1,81 +0,0 @@
-(identifier) @variable
-
-((identifier) @constant
- (#match? @constant "^[A-Z][A-Z\\d_]*$"))
-
-"break" @keyword
-"case" @keyword
-"const" @keyword
-"continue" @keyword
-"default" @keyword
-"do" @keyword
-"else" @keyword
-"enum" @keyword
-"extern" @keyword
-"for" @keyword
-"if" @keyword
-"inline" @keyword
-"return" @keyword
-"sizeof" @keyword
-"static" @keyword
-"struct" @keyword
-"switch" @keyword
-"typedef" @keyword
-"union" @keyword
-"volatile" @keyword
-"while" @keyword
-
-"#define" @keyword
-"#elif" @keyword
-"#else" @keyword
-"#endif" @keyword
-"#if" @keyword
-"#ifdef" @keyword
-"#ifndef" @keyword
-"#include" @keyword
-(preproc_directive) @keyword
-
-"--" @operator
-"-" @operator
-"-=" @operator
-"->" @operator
-"=" @operator
-"!=" @operator
-"*" @operator
-"&" @operator
-"&&" @operator
-"+" @operator
-"++" @operator
-"+=" @operator
-"<" @operator
-"==" @operator
-">" @operator
-"||" @operator
-
-"." @delimiter
-";" @delimiter
-
-(string_literal) @string
-(system_lib_string) @string
-
-(null) @constant
-(number_literal) @number
-(char_literal) @number
-
-(field_identifier) @property
-(statement_identifier) @label
-(type_identifier) @type
-(primitive_type) @type
-(sized_type_specifier) @type
-
-(call_expression
-  function: (identifier) @function)
-(call_expression
-  function: (field_expression
-    field: (field_identifier) @function))
-(function_declarator
-  declarator: (identifier) @function)
-(preproc_function_def
-  name: (identifier) @function.special)
-
-(comment) @comment

ui-opentui/parsers/css/highlights.scm

@@ -1,64 +0,0 @@
-(comment) @comment
-
-(tag_name) @tag
-(nesting_selector) @tag
-(universal_selector) @tag
-
-"~" @operator
-">" @operator
-"+" @operator
-"-" @operator
-"*" @operator
-"/" @operator
-"=" @operator
-"^=" @operator
-"|=" @operator
-"~=" @operator
-"$=" @operator
-"*=" @operator
-
-"and" @operator
-"or" @operator
-"not" @operator
-"only" @operator
-
-(attribute_selector (plain_value) @string)
-(pseudo_element_selector (tag_name) @attribute)
-(pseudo_class_selector (class_name) @attribute)
-
-(class_name) @property
-(id_name) @property
-(namespace_name) @property
-(property_name) @property
-(feature_name) @property
-
-(attribute_name) @attribute
-
-(function_name) @function
-
-((property_name) @variable
- (#match? @variable "^--"))
-((plain_value) @variable
- (#match? @variable "^--"))
-
-"@media" @keyword
-"@import" @keyword
-"@charset" @keyword
-"@namespace" @keyword
-"@supports" @keyword
-"@keyframes" @keyword
-(at_keyword) @keyword
-(to) @keyword
-(from) @keyword
-(important) @keyword
-
-(string_value) @string
-(color_value) @string.special
-
-(integer_value) @number
-(float_value) @number
-(unit) @type
-
-"#" @punctuation.delimiter
-"," @punctuation.delimiter
-":" @punctuation.delimiter

ui-opentui/parsers/go/highlights.scm

@@ -1,123 +0,0 @@
-; Function calls
-
-(call_expression
-  function: (identifier) @function)
-
-(call_expression
-  function: (identifier) @function.builtin
-  (#match? @function.builtin "^(append|cap|close|complex|copy|delete|imag|len|make|new|panic|print|println|real|recover)$"))
-
-(call_expression
-  function: (selector_expression
-    field: (field_identifier) @function.method))
-
-; Function definitions
-
-(function_declaration
-  name: (identifier) @function)
-
-(method_declaration
-  name: (field_identifier) @function.method)
-
-; Identifiers
-
-(type_identifier) @type
-(field_identifier) @property
-(identifier) @variable
-
-; Operators
-
-[
-  "--"
-  "-"
-  "-="
-  ":="
-  "!"
-  "!="
-  "..."
-  "*"
-  "*"
-  "*="
-  "/"
-  "/="
-  "&"
-  "&&"
-  "&="
-  "%"
-  "%="
-  "^"
-  "^="
-  "+"
-  "++"
-  "+="
-  "<-"
-  "<"
-  "<<"
-  "<<="
-  "<="
-  "="
-  "=="
-  ">"
-  ">="
-  ">>"
-  ">>="
-  "|"
-  "|="
-  "||"
-  "~"
-] @operator
-
-; Keywords
-
-[
-  "break"
-  "case"
-  "chan"
-  "const"
-  "continue"
-  "default"
-  "defer"
-  "else"
-  "fallthrough"
-  "for"
-  "func"
-  "go"
-  "goto"
-  "if"
-  "import"
-  "interface"
-  "map"
-  "package"
-  "range"
-  "return"
-  "select"
-  "struct"
-  "switch"
-  "type"
-  "var"
-] @keyword
-
-; Literals
-
-[
-  (interpreted_string_literal)
-  (raw_string_literal)
-  (rune_literal)
-] @string
-
-(escape_sequence) @escape
-
-[
-  (int_literal)
-  (float_literal)
-  (imaginary_literal)
-] @number
-
-[
-  (true)
-  (false)
-  (nil)
-  (iota)
-] @constant.builtin
-
-(comment) @comment

ui-opentui/parsers/html/highlights.scm

@@ -1,13 +0,0 @@
-(tag_name) @tag
-(erroneous_end_tag_name) @tag.error
-(doctype) @constant
-(attribute_name) @attribute
-(attribute_value) @string
-(comment) @comment
-
-[
-  "<"
-  ">"
-  "</"
-  "/>"
-] @punctuation.bracket

ui-opentui/parsers/json/highlights.scm

@@ -1,16 +0,0 @@
-(pair
-  key: (_) @string.special.key)
-
-(string) @string
-
-(number) @number
-
-[
-  (null)
-  (true)
-  (false)
-] @constant.builtin
-
-(escape_sequence) @escape
-
-(comment) @comment

ui-opentui/parsers/manifest.json

@@ -1,15 +0,0 @@
-{
-  "comment": "Tree-sitter grammars vendored beyond @opentui/core's bundled 5 (ts/js/markdown/markdown_inline/zig). Source of truth for scripts/update-parsers.mjs (vendoring) and src/boundary/parsers.ts (boot registration). Curated in docs/plans/opentui-syntax-highlighting-languages.md — cpp deliberately dropped (3.28 MB, ~half the bundle). Aliases are belt-and-braces: core's extToFiletype/infoStringToFiletype already normalize py->python etc., but a literal alias filetype reaching the client still resolves.",
-  "parsers": [
-    { "filetype": "python", "aliases": ["py"], "org": "tree-sitter", "tag": "v0.23.6" },
-    { "filetype": "rust", "aliases": ["rs"], "org": "tree-sitter", "tag": "v0.23.2" },
-    { "filetype": "go", "aliases": [], "org": "tree-sitter", "tag": "v0.23.4" },
-    { "filetype": "bash", "aliases": ["sh", "shell", "zsh"], "org": "tree-sitter", "tag": "v0.23.3" },
-    { "filetype": "json", "aliases": [], "org": "tree-sitter", "tag": "v0.24.8" },
-    { "filetype": "c", "aliases": ["h"], "org": "tree-sitter", "tag": "v0.23.5" },
-    { "filetype": "html", "aliases": [], "org": "tree-sitter", "tag": "v0.23.2" },
-    { "filetype": "css", "aliases": [], "org": "tree-sitter", "tag": "v0.23.2" },
-    { "filetype": "yaml", "aliases": ["yml"], "org": "tree-sitter-grammars", "tag": "v0.7.1" },
-    { "filetype": "toml", "aliases": [], "org": "tree-sitter-grammars", "tag": "v0.7.0" }
-  ]
-}

ui-opentui/parsers/python/highlights.scm

@@ -1,137 +0,0 @@
-; Identifier naming conventions
-
-(identifier) @variable
-
-((identifier) @constructor
- (#match? @constructor "^[A-Z]"))
-
-((identifier) @constant
- (#match? @constant "^[A-Z][A-Z_]*$"))
-
-; Function calls
-
-(decorator) @function
-(decorator
-  (identifier) @function)
-
-(call
-  function: (attribute attribute: (identifier) @function.method))
-(call
-  function: (identifier) @function)
-
-; Builtin functions
-
-((call
-  function: (identifier) @function.builtin)
- (#match?
-   @function.builtin
-   "^(abs|all|any|ascii|bin|bool|breakpoint|bytearray|bytes|callable|chr|classmethod|compile|complex|delattr|dict|dir|divmod|enumerate|eval|exec|filter|float|format|frozenset|getattr|globals|hasattr|hash|help|hex|id|input|int|isinstance|issubclass|iter|len|list|locals|map|max|memoryview|min|next|object|oct|open|ord|pow|print|property|range|repr|reversed|round|set|setattr|slice|sorted|staticmethod|str|sum|super|tuple|type|vars|zip|__import__)$"))
-
-; Function definitions
-
-(function_definition
-  name: (identifier) @function)
-
-(attribute attribute: (identifier) @property)
-(type (identifier) @type)
-
-; Literals
-
-[
-  (none)
-  (true)
-  (false)
-] @constant.builtin
-
-[
-  (integer)
-  (float)
-] @number
-
-(comment) @comment
-(string) @string
-(escape_sequence) @escape
-
-(interpolation
-  "{" @punctuation.special
-  "}" @punctuation.special) @embedded
-
-[
-  "-"
-  "-="
-  "!="
-  "*"
-  "**"
-  "**="
-  "*="
-  "/"
-  "//"
-  "//="
-  "/="
-  "&"
-  "&="
-  "%"
-  "%="
-  "^"
-  "^="
-  "+"
-  "->"
-  "+="
-  "<"
-  "<<"
-  "<<="
-  "<="
-  "<>"
-  "="
-  ":="
-  "=="
-  ">"
-  ">="
-  ">>"
-  ">>="
-  "|"
-  "|="
-  "~"
-  "@="
-  "and"
-  "in"
-  "is"
-  "not"
-  "or"
-  "is not"
-  "not in"
-] @operator
-
-[
-  "as"
-  "assert"
-  "async"
-  "await"
-  "break"
-  "class"
-  "continue"
-  "def"
-  "del"
-  "elif"
-  "else"
-  "except"
-  "exec"
-  "finally"
-  "for"
-  "from"
-  "global"
-  "if"
-  "import"
-  "lambda"
-  "nonlocal"
-  "pass"
-  "print"
-  "raise"
-  "return"
-  "try"
-  "while"
-  "with"
-  "yield"
-  "match"
-  "case"
-] @keyword

ui-opentui/parsers/rust/highlights.scm

@@ -1,161 +0,0 @@
-; Identifiers
-
-(type_identifier) @type
-(primitive_type) @type.builtin
-(field_identifier) @property
-
-; Identifier conventions
-
-; Assume all-caps names are constants
-((identifier) @constant
- (#match? @constant "^[A-Z][A-Z\\d_]+$'"))
-
-; Assume uppercase names are enum constructors
-((identifier) @constructor
- (#match? @constructor "^[A-Z]"))
-
-; Assume that uppercase names in paths are types
-((scoped_identifier
-  path: (identifier) @type)
- (#match? @type "^[A-Z]"))
-((scoped_identifier
-  path: (scoped_identifier
-    name: (identifier) @type))
- (#match? @type "^[A-Z]"))
-((scoped_type_identifier
-  path: (identifier) @type)
- (#match? @type "^[A-Z]"))
-((scoped_type_identifier
-  path: (scoped_identifier
-    name: (identifier) @type))
- (#match? @type "^[A-Z]"))
-
-; Assume all qualified names in struct patterns are enum constructors. (They're
-; either that, or struct names; highlighting both as constructors seems to be
-; the less glaring choice of error, visually.)
-(struct_pattern
-  type: (scoped_type_identifier
-    name: (type_identifier) @constructor))
-
-; Function calls
-
-(call_expression
-  function: (identifier) @function)
-(call_expression
-  function: (field_expression
-    field: (field_identifier) @function.method))
-(call_expression
-  function: (scoped_identifier
-    "::"
-    name: (identifier) @function))
-
-(generic_function
-  function: (identifier) @function)
-(generic_function
-  function: (scoped_identifier
-    name: (identifier) @function))
-(generic_function
-  function: (field_expression
-    field: (field_identifier) @function.method))
-
-(macro_invocation
-  macro: (identifier) @function.macro
-  "!" @function.macro)
-
-; Function definitions
-
-(function_item (identifier) @function)
-(function_signature_item (identifier) @function)
-
-(line_comment) @comment
-(block_comment) @comment
-
-(line_comment (doc_comment)) @comment.documentation
-(block_comment (doc_comment)) @comment.documentation
-
-"(" @punctuation.bracket
-")" @punctuation.bracket
-"[" @punctuation.bracket
-"]" @punctuation.bracket
-"{" @punctuation.bracket
-"}" @punctuation.bracket
-
-(type_arguments
-  "<" @punctuation.bracket
-  ">" @punctuation.bracket)
-(type_parameters
-  "<" @punctuation.bracket
-  ">" @punctuation.bracket)
-
-"::" @punctuation.delimiter
-":" @punctuation.delimiter
-"." @punctuation.delimiter
-"," @punctuation.delimiter
-";" @punctuation.delimiter
-
-(parameter (identifier) @variable.parameter)
-
-(lifetime (identifier) @label)
-
-"as" @keyword
-"async" @keyword
-"await" @keyword
-"break" @keyword
-"const" @keyword
-"continue" @keyword
-"default" @keyword
-"dyn" @keyword
-"else" @keyword
-"enum" @keyword
-"extern" @keyword
-"fn" @keyword
-"for" @keyword
-"gen" @keyword
-"if" @keyword
-"impl" @keyword
-"in" @keyword
-"let" @keyword
-"loop" @keyword
-"macro_rules!" @keyword
-"match" @keyword
-"mod" @keyword
-"move" @keyword
-"pub" @keyword
-"raw" @keyword
-"ref" @keyword
-"return" @keyword
-"static" @keyword
-"struct" @keyword
-"trait" @keyword
-"type" @keyword
-"union" @keyword
-"unsafe" @keyword
-"use" @keyword
-"where" @keyword
-"while" @keyword
-"yield" @keyword
-(crate) @keyword
-(mutable_specifier) @keyword
-(use_list (self) @keyword)
-(scoped_use_list (self) @keyword)
-(scoped_identifier (self) @keyword)
-(super) @keyword
-
-(self) @variable.builtin
-
-(char_literal) @string
-(string_literal) @string
-(raw_string_literal) @string
-
-(boolean_literal) @constant.builtin
-(integer_literal) @constant.builtin
-(float_literal) @constant.builtin
-
-(escape_sequence) @escape
-
-(attribute_item) @attribute
-(inner_attribute_item) @attribute
-
-"*" @operator
-"&" @operator
-"'" @operator

ui-opentui/parsers/toml/highlights.scm

@@ -1,53 +0,0 @@
-; Properties
-;-----------
-
-(bare_key) @type
-
-(quoted_key) @string
-
-(pair
-  (bare_key)) @property
-
-(pair
-  (dotted_key
-    (bare_key) @property))
-
-; Literals
-;---------
-
-(boolean) @boolean
-
-(comment) @comment
-
-(string) @string
-
-[
-  (integer)
-  (float)
-] @number
-
-[
-  (offset_date_time)
-  (local_date_time)
-  (local_date)
-  (local_time)
-] @string.special
-
-; Punctuation
-;------------
-
-[
-  "."
-  ","
-] @punctuation.delimiter
-
-"=" @operator
-
-[
-  "["
-  "]"
-  "[["
-  "]]"
-  "{"
-  "}"
-] @punctuation.bracket

ui-opentui/parsers/yaml/highlights.scm

@@ -1,79 +0,0 @@
-(boolean_scalar) @boolean
-
-(null_scalar) @constant.builtin
-
-[
-  (double_quote_scalar)
-  (single_quote_scalar)
-  (block_scalar)
-  (string_scalar)
-] @string
-
-[
-  (integer_scalar)
-  (float_scalar)
-] @number
-
-(comment) @comment
-
-[
-  (anchor_name)
-  (alias_name)
-] @label
-
-(tag) @type
-
-[
-  (yaml_directive)
-  (tag_directive)
-  (reserved_directive)
-] @attribute
-
-(block_mapping_pair
-  key: (flow_node
-    [
-      (double_quote_scalar)
-      (single_quote_scalar)
-    ] @property))
-
-(block_mapping_pair
-  key: (flow_node
-    (plain_scalar
-      (string_scalar) @property)))
-
-(flow_mapping
-  (_
-    key: (flow_node
-      [
-        (double_quote_scalar)
-        (single_quote_scalar)
-      ] @property)))
-
-(flow_mapping
-  (_
-    key: (flow_node
-      (plain_scalar
-        (string_scalar) @property))))
-
-[
-  ","
-  "-"
-  ":"
-  ">"
-  "?"
-  "|"
-] @punctuation.delimiter
-
-[
-  "["
-  "]"
-  "{"
-  "}"
-] @punctuation.bracket
-
-[
-  "*"
-  "&"
-  "---"
-  "..."
-] @punctuation.special

ui-opentui/scripts/demo.tsx

@@ -16,14 +16,15 @@ import { createCliRenderer } from '@opentui/core'
 import { render } from '@opentui/solid'
 
 import { installMultiClickSelection } from '../src/boundary/multiClickSelect.ts'
-import { registerVendoredParsers } from '../src/boundary/parsers.ts'
+import { registerRemoteParsers } from '../src/boundary/parsers.ts'
 import { createSessionStore } from '../src/logic/store.ts'
 import { App } from '../src/view/App.tsx'
 import { ThemeProvider } from '../src/view/theme.tsx'
 import { materialize } from './fixture.ts'
 
-// Same grammar registration as the live entry so fixture code blocks highlight.
-registerVendoredParsers()
+// Same grammar registration as the live entry so fixture code blocks highlight
+// (fetched+cached on first use; no HERMES_TUI_PARSER_CACHE here → OpenTUI default).
+registerRemoteParsers()
 
 const TOTAL = Number.parseInt(process.env.DEMO_TOTAL ?? '', 10) || 200
 

ui-opentui/scripts/update-parsers.mjs

@@ -1,71 +0,0 @@
-/**
- * Vendor the extra Tree-sitter grammars listed in parsers/manifest.json into
- * parsers/<filetype>/ (wasm + highlights.scm). Plain Node fetch — no Bun
- * (core's update-assets.js is bun-shebanged; its download logic is just
- * fetch+fs, so we do the same two writes ourselves and keep the generated
- * import-module out of the esbuild bundle entirely — registration reads the
- * vendored files by PATH at runtime, see src/boundary/parsers.ts).
- *
- * Idempotent: existing valid files are kept unless --force. Validates wasm
- * magic and a non-empty query so a bad download can never be committed.
- *
- *   node scripts/update-parsers.mjs [--force]
- *
- * The vendored files are COMMITTED (build inputs, like @opentui/core's own
- * assets/) — builds and offline machines never re-download.
- */
-import { Buffer } from 'node:buffer'
-import { mkdir, readFile, writeFile } from 'node:fs/promises'
-import { dirname, join, resolve } from 'node:path'
-import { fileURLToPath } from 'node:url'
-
-const root = resolve(dirname(fileURLToPath(import.meta.url)), '..')
-const parsersDir = join(root, 'parsers')
-const force = process.argv.includes('--force')
-
-const manifest = JSON.parse(await readFile(join(parsersDir, 'manifest.json'), 'utf8'))
-
-const wasmUrl = p =>
-  `https://github.com/${p.org}/tree-sitter-${p.filetype}/releases/download/${p.tag}/tree-sitter-${p.filetype}.wasm`
-const scmUrl = p =>
-  `https://raw.githubusercontent.com/${p.org}/tree-sitter-${p.filetype}/${p.tag}/queries/highlights.scm`
-
-async function fetchBytes(url) {
-  const response = await globalThis.fetch(url)
-  if (!response.ok) throw new Error(`${response.status} ${response.statusText} for ${url}`)
-  return Buffer.from(await response.arrayBuffer())
-}
-
-async function haveValid(path, validate) {
-  try {
-    return validate(await readFile(path))
-  } catch {
-    return false
-  }
-}
-
-const isWasm = bytes => bytes.length > 8 && bytes.subarray(0, 4).toString('latin1') === '\0asm'
-const isQuery = bytes => bytes.length > 0 && bytes.toString('utf8').trim().length > 0
-
-let downloaded = 0
-for (const parser of manifest.parsers) {
-  const dir = join(parsersDir, parser.filetype)
-  await mkdir(dir, { recursive: true })
-  const targets = [
-    { name: `tree-sitter-${parser.filetype}.wasm`, url: wasmUrl(parser), validate: isWasm },
-    { name: 'highlights.scm', url: scmUrl(parser), validate: isQuery }
-  ]
-  for (const target of targets) {
-    const path = join(dir, target.name)
-    if (!force && (await haveValid(path, target.validate))) {
-      console.log(`✓ kept    ${parser.filetype}/${target.name}`)
-      continue
-    }
-    const bytes = await fetchBytes(target.url)
-    if (!target.validate(bytes)) throw new Error(`validation failed for ${target.url}`)
-    await writeFile(path, bytes)
-    downloaded += 1
-    console.log(`↓ fetched ${parser.filetype}/${target.name} (${(bytes.length / 1024).toFixed(0)} KB)`)
-  }
-}
-console.log(`done — ${downloaded} file(s) fetched, ${manifest.parsers.length} grammars vendored`)

ui-opentui/src/boundary/parsers.manifest.json

@@ -0,0 +1,65 @@
+{
+  "comment": "Tree-sitter grammars beyond @opentui/core's bundled 5 (ts/js/markdown/markdown_inline/zig). NOT vendored — declared as remote URLs and fetched+cached at runtime by OpenTUI's TreeSitterClient into HERMES_TUI_PARSER_CACHE (see src/boundary/parsers.ts). This file is the URL source of truth. Grammar versions are pinned via the release tag in each wasm URL; .scm highlight queries follow opencode's per-language source choices (parser-repo queries for python/html where nvim-treesitter's are parser-incompatible, nvim-treesitter master otherwise). cpp deliberately dropped (3.28 MB, ~half the old vendored bundle). Aliases are belt-and-braces: core's extToFiletype/infoStringToFiletype already normalize py->python etc., but a literal alias filetype reaching the client still resolves. To add/refresh a grammar, edit the wasm tag + the highlights URL here — no binaries land in the repo.",
+  "parsers": [
+    {
+      "filetype": "python",
+      "aliases": ["py"],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-python/releases/download/v0.23.6/tree-sitter-python.wasm",
+      "highlights": "https://github.com/tree-sitter/tree-sitter-python/raw/refs/heads/master/queries/highlights.scm"
+    },
+    {
+      "filetype": "rust",
+      "aliases": ["rs"],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-rust/releases/download/v0.23.2/tree-sitter-rust.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/rust/highlights.scm"
+    },
+    {
+      "filetype": "go",
+      "aliases": [],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-go/releases/download/v0.23.4/tree-sitter-go.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/go/highlights.scm"
+    },
+    {
+      "filetype": "bash",
+      "aliases": ["sh", "shell", "zsh"],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-bash/releases/download/v0.23.3/tree-sitter-bash.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/bash/highlights.scm"
+    },
+    {
+      "filetype": "json",
+      "aliases": [],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-json/releases/download/v0.24.8/tree-sitter-json.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/json/highlights.scm"
+    },
+    {
+      "filetype": "c",
+      "aliases": ["h"],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-c/releases/download/v0.23.5/tree-sitter-c.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/c/highlights.scm"
+    },
+    {
+      "filetype": "html",
+      "aliases": [],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-html/releases/download/v0.23.2/tree-sitter-html.wasm",
+      "highlights": "https://github.com/tree-sitter/tree-sitter-html/raw/refs/heads/master/queries/highlights.scm"
+    },
+    {
+      "filetype": "css",
+      "aliases": [],
+      "wasm": "https://github.com/tree-sitter/tree-sitter-css/releases/download/v0.23.2/tree-sitter-css.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/css/highlights.scm"
+    },
+    {
+      "filetype": "yaml",
+      "aliases": ["yml"],
+      "wasm": "https://github.com/tree-sitter-grammars/tree-sitter-yaml/releases/download/v0.7.1/tree-sitter-yaml.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/yaml/highlights.scm"
+    },
+    {
+      "filetype": "toml",
+      "aliases": [],
+      "wasm": "https://github.com/tree-sitter-grammars/tree-sitter-toml/releases/download/v0.7.0/tree-sitter-toml.wasm",
+      "highlights": "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/refs/heads/master/queries/toml/highlights.scm"
+    }
+  ]
+}

ui-opentui/src/boundary/parsers.ts

@@ -2,37 +2,43 @@
  * Extra Tree-sitter grammar registration — the syntax-highlighting language
  * expansion (docs/plans/opentui-syntax-highlighting-languages.md).
  *
- * @opentui/core@0.4.0 bundles only 5 grammars (ts/js/markdown/markdown_inline/
- * zig); everything else rendered plain text. The cure is the public
- * `addDefaultParsers()` API + the grammars vendored under `parsers/<filetype>/`
- * (committed; refresh via `node scripts/update-parsers.mjs`).
+ * @opentui/core@0.4.x bundles only a handful of grammars (ts/js/markdown/
+ * markdown_inline/zig); everything else renders plain text. The cure is the
+ * public `addDefaultParsers()` API fed with REMOTE grammar URLs — OpenTUI's
+ * TreeSitterClient fetches each `.wasm`/`.scm` lazily on first use of a
+ * filetype and caches it under the client's `dataPath`. We do NOT vendor any
+ * binaries (cf. opencode, which checks in zero `.wasm`/`.scm` and lets OpenTUI
+ * fetch+cache). The grammar set + its URLs live in `parsers.manifest.json`.
  *
- * Why paths, not the generated import-module: core's `updateAssets` generates
- * a Bun-flavored module (`import(... { with: { type: "file" } })`) that esbuild
- * can't bundle; its own Node fallback resolves plain file paths anyway, and
- * `FiletypeParserOptions.wasm`/`queries.highlights` accept local paths
- * directly. So registration just points at the vendored files — resolved at
- * RUNTIME by walking up from this module to the package root, which works from
- * both the esbuild bundle (dist/main.js → ../parsers) and vitest's src tree
- * (src/boundary → ../../parsers).
+ * Cache location: `HERMES_TUI_PARSER_CACHE` (set by the Python launcher to
+ * `~/.hermes/cache/opentui-parsers/`, profile-aware via get_hermes_home). When
+ * unset (dev/demo/CI), we leave OpenTUI's default data path
+ * (`$XDG_DATA_HOME/opentui` → `~/.local/share/opentui`) untouched.
  *
- * Must run BEFORE the first `<code>`/`<markdown>` mount (they grab the global
- * tree-sitter client lazily) — the entry imports + calls this at module load,
- * ahead of renderer acquisition. Total: a missing assets dir degrades to
- * core's bundled set (plain text for the extras), never a throw.
+ * `setDataPath()` on the GLOBAL client must run BEFORE the client initializes
+ * (it only mutates `options.dataPath` until init, then the worker boots with
+ * it). `addDefaultParsers()` must run BEFORE the first `<code>`/`<markdown>`
+ * mount (they grab the global client lazily and trigger init). The entry
+ * imports + calls `registerRemoteParsers()` at module load, ahead of renderer
+ * acquisition, so both orderings hold.
+ *
+ * Offline behavior: registration itself does NO network (it only declares the
+ * URL configs). The fetch happens on first highlight of a given language; if it
+ * fails (air-gapped / GitHub unreachable), OpenTUI degrades that filetype to
+ * plain text — never a throw. A registration error likewise degrades the whole
+ * extra set to plain text.
  */
-import { existsSync } from 'node:fs'
-import { dirname, join } from 'node:path'
-import { fileURLToPath } from 'node:url'
-
+import { getTreeSitterClient } from '@opentui/core'
 import { addDefaultParsers } from '@opentui/core'
 
-import manifest from '../../parsers/manifest.json'
+import manifest from './parsers.manifest.json'
 import { getLog } from './log.ts'
 
 interface ManifestParser {
   readonly filetype: string
   readonly aliases: readonly string[]
+  readonly wasm: string
+  readonly highlights: string
 }
 
 /** The registered parser configs (exported shape for tests/diagnostics). */
@@ -43,43 +49,44 @@ export interface RegisteredParser {
   queries: { highlights: string[] }
 }
 
-/** Walk up from this module (bundle: dist/…; tests: src/boundary/…) to the
- *  package root's vendored `parsers/` dir. */
-function findParsersDir(): string | undefined {
-  let dir = dirname(fileURLToPath(import.meta.url))
-  for (let hop = 0; hop < 5; hop += 1) {
-    const candidate = join(dir, 'parsers')
-    if (existsSync(join(candidate, 'manifest.json'))) return candidate
-    dir = dirname(dir)
-  }
-  return undefined
+/** The cache dir for fetched grammar assets, or undefined to use OpenTUI's
+ *  default ($XDG_DATA_HOME/opentui). The launcher sets this per-profile. */
+export function parserCacheDir(): string | undefined {
+  const dir = (process.env.HERMES_TUI_PARSER_CACHE ?? '').trim()
+  return dir.length ? dir : undefined
 }
 
-/** Build the parser configs for every vendored grammar whose files exist. */
-export function vendoredParsers(parsersDir: string | undefined = findParsersDir()): RegisteredParser[] {
-  if (!parsersDir) return []
+/** Build the remote parser configs from the manifest. Pure — no network, no
+ *  filesystem; just declares the URL configs OpenTUI fetches lazily. */
+export function remoteParsers(): RegisteredParser[] {
   const configs: RegisteredParser[] = []
   for (const parser of (manifest as { parsers: ManifestParser[] }).parsers) {
-    const wasm = join(parsersDir, parser.filetype, `tree-sitter-${parser.filetype}.wasm`)
-    const highlights = join(parsersDir, parser.filetype, 'highlights.scm')
-    if (!existsSync(wasm) || !existsSync(highlights)) continue
+    if (!parser.wasm || !parser.highlights) continue
     configs.push({
       filetype: parser.filetype,
       ...(parser.aliases.length ? { aliases: [...parser.aliases] } : {}),
-      wasm,
-      queries: { highlights: [highlights] }
+      wasm: parser.wasm,
+      queries: { highlights: [parser.highlights] }
     })
   }
   return configs
 }
 
-/** Register the vendored grammars with core's global default-parser list.
+/** Point the global tree-sitter client's cache at our profile dir, then
+ *  register the remote grammars with core's global default-parser list.
  *  Returns what was registered (empty on any failure — plain-text fallback). */
-export function registerVendoredParsers(): RegisteredParser[] {
+export function registerRemoteParsers(): RegisteredParser[] {
   try {
-    const parsers = vendoredParsers()
+    const cache = parserCacheDir()
+    if (cache) {
+      // Must precede the client's lazy initialize() (first <code>/<markdown>
+      // mount). Pre-init this only mutates options.dataPath; the returned
+      // promise resolves immediately (no worker yet) so we don't await it.
+      void getTreeSitterClient().setDataPath(cache)
+    }
+    const parsers = remoteParsers()
     if (!parsers.length) {
-      getLog().warn('parsers', 'no vendored tree-sitter grammars found — extras render plain', {})
+      getLog().warn('parsers', 'no remote tree-sitter grammars declared — extras render plain', {})
       return []
     }
     addDefaultParsers(parsers)

ui-opentui/src/entry/main.tsx

@@ -32,7 +32,7 @@ import { getLog } from '../boundary/log.ts'
 import { startMemlog } from '../boundary/memlog.ts'
 import { startMemoryMonitor } from '../boundary/memoryMonitor.ts'
 import { startProactiveGc } from '../boundary/proactiveGc.ts'
-import { registerVendoredParsers } from '../boundary/parsers.ts'
+import { registerRemoteParsers } from '../boundary/parsers.ts'
 import { acquireRenderer } from '../boundary/renderer.ts'
 import { makeAppLayer } from '../boundary/runtime.ts'
 import { nthAssistantResponse } from '../logic/copy.ts'
@@ -68,10 +68,11 @@ import { App } from '../view/App.tsx'
 import { seedLearnedNames } from '../view/composer.tsx'
 import { TerminalChrome } from '../view/terminalChrome.tsx'
 
-// Syntax-highlighting language expansion: register the vendored tree-sitter
+// Syntax-highlighting language expansion: register the remote tree-sitter
 // grammars (python/rust/go/bash/json/c/html/css/yaml/toml) before the first
-// <code>/<markdown> mount initializes the global tree-sitter client.
-registerVendoredParsers()
+// <code>/<markdown> mount initializes the global tree-sitter client. Grammars
+// are fetched from GitHub on first use and cached under HERMES_TUI_PARSER_CACHE.
+registerRemoteParsers()
 import type { SessionPickerOps } from '../view/overlays/sessionPicker.tsx'
 import { ThemeProvider } from '../view/theme.tsx'
 import { makeFakeGatewayLayer, type FakeGatewayController } from './fakeGateway.ts'

ui-opentui/src/test/parsers.test.tsx

@@ -1,49 +1,55 @@
 /**
- * Vendored tree-sitter grammar registration (syntax-highlighting language
- * expansion). Layers:
- *   1. config: every manifest grammar has valid vendored assets (wasm magic,
- *      non-empty query) and the built configs point at existing absolute paths.
+ * Remote tree-sitter grammar registration (syntax-highlighting language
+ * expansion). Grammars are NOT vendored — they're declared as remote URLs in
+ * parsers.manifest.json and fetched+cached by OpenTUI on first use. Layers:
+ *   1. config: every manifest grammar builds a well-formed config — a release
+ *      `.wasm` URL + a highlights `.scm` URL (both https).
  *   2. resolution: core's filetype maps route our curated extensions/fence
  *      labels to the registered filetype ids.
- * Visual color is live-smoke territory (highlighting settles async — see
- * codeBlock.tsx); these tests pin the wiring that makes it possible.
+ * Actual fetch + visual color is live-smoke territory (network + async settle —
+ * see codeBlock.tsx); these tests pin the wiring that makes it possible without
+ * hitting the network.
  */
-import { readFileSync } from 'node:fs'
-import { isAbsolute } from 'node:path'
-
 import { extToFiletype, infoStringToFiletype, pathToFiletype } from '@opentui/core'
 import { describe, expect, test } from 'vitest'
 
-import { registerVendoredParsers, vendoredParsers } from '../boundary/parsers.ts'
+import { parserCacheDir, registerRemoteParsers, remoteParsers } from '../boundary/parsers.ts'
 
 const EXPECTED = ['python', 'rust', 'go', 'bash', 'json', 'c', 'html', 'css', 'yaml', 'toml']
 
-describe('vendored grammar configs', () => {
-  test('all 10 curated grammars resolve with existing absolute asset paths', () => {
-    const configs = vendoredParsers()
+describe('remote grammar configs', () => {
+  test('all 10 curated grammars build configs from the manifest', () => {
+    const configs = remoteParsers()
     expect(configs.map(c => c.filetype).sort()).toEqual([...EXPECTED].sort())
-    for (const config of configs) {
-      expect(isAbsolute(config.wasm)).toBe(true)
-      expect(isAbsolute(config.queries.highlights[0]!)).toBe(true)
+  })
+
+  test('each config carries an https .wasm URL and a non-empty .scm URL', () => {
+    for (const config of remoteParsers()) {
+      expect(config.wasm, config.filetype).toMatch(/^https:\/\/.+\.wasm$/)
+      const scm = config.queries.highlights[0]!
+      expect(scm.startsWith('https://'), config.filetype).toBe(true)
+      expect(scm.endsWith('.scm'), config.filetype).toBe(true)
     }
   })
 
-  test('vendored wasm files carry the wasm magic; queries are non-empty', () => {
-    for (const config of vendoredParsers()) {
-      const wasm = readFileSync(config.wasm)
-      expect(wasm.subarray(0, 4).toString('latin1'), config.filetype).toBe('\0asm')
-      const query = readFileSync(config.queries.highlights[0]!, 'utf8')
-      expect(query.trim().length, config.filetype).toBeGreaterThan(0)
-    }
-  })
-
-  test('registerVendoredParsers registers and reports the full set', () => {
-    const registered = registerVendoredParsers()
+  test('registerRemoteParsers registers and reports the full set', () => {
+    const registered = registerRemoteParsers()
     expect(registered.map(r => r.filetype).sort()).toEqual([...EXPECTED].sort())
   })
 
-  test('a missing assets dir degrades to empty (plain-text fallback), no throw', () => {
-    expect(vendoredParsers('/nonexistent/parsers')).toEqual([])
+  test('parserCacheDir reflects HERMES_TUI_PARSER_CACHE (undefined when unset)', () => {
+    const prev = process.env.HERMES_TUI_PARSER_CACHE
+    try {
+      delete process.env.HERMES_TUI_PARSER_CACHE
+      expect(parserCacheDir()).toBeUndefined()
+      process.env.HERMES_TUI_PARSER_CACHE = '/tmp/hermes-parser-cache'
+      expect(parserCacheDir()).toBe('/tmp/hermes-parser-cache')
+      process.env.HERMES_TUI_PARSER_CACHE = '   '
+      expect(parserCacheDir()).toBeUndefined()
+    } finally {
+      if (prev === undefined) delete process.env.HERMES_TUI_PARSER_CACHE
+      else process.env.HERMES_TUI_PARSER_CACHE = prev
+    }
   })
 })