diff --git a/docs/opentui-memory-story.md b/docs/opentui-memory-story.md new file mode 100644 index 00000000000..63ec531c66e --- /dev/null +++ b/docs/opentui-memory-story.md @@ -0,0 +1,207 @@ +# How the OpenTUI transcript got from 686MB to ~300MB — the full story + +*For: glitch. Branch: `feat/opentui-memory-window`. Everything here is measured, +not vibes; every number has a result JSON in `bench/results/`.* + +--- + +## 1. The cast of characters (the primitives, bottom-up) + +To understand where the memory went, you need to know who's holding it. Six +layers, from the screen up: + +**The terminal grid.** Your terminal is a spreadsheet of character cells. +Nobody pays per-message here — tmux holds ~5MB flat no matter how long the +session is (we measured). The terminal is never the problem. + +**The OpenTUI native renderer (Zig).** A compiled library that owns the +"frame buffer" — the grid of cells about to be painted. Every piece of text the +TUI shows lives in a native **TextBuffer** (the characters + their colors), +viewed through a **TextBufferView**, styled by a **SyntaxStyle**. Each of those +is a **native handle** — a ticket into one global table that has only **65,535 +slots, total, ever** (16-bit indices — like a coat check with 65k hooks). +Destroying a renderable returns its tickets, so the constraint is not "how much +have you ever created" but **"how much is alive right now."** + +**Renderables.** OpenTUI's UI objects — ``, ``, ``, +``, ``. One transcript row (a message with its tool calls, +markdown, code blocks, copy chips) is a *tree* of these: **~16 text renderables +≈ 47 native handles ≈ ~250–340KB of RSS, per row.** This is the number that +drives everything. 1,400 mounted rows × 47 handles = table full = the crash we +root-caused last week. + +**Yoga (the layout engine, WASM).** Every renderable also has a Yoga node — +Yoga is the flexbox calculator that decides where boxes go. OpenTUI ships it +compiled to **WebAssembly**, and WASM has a brutal property: its memory can +**grow but never shrink** back to the OS. So the peak number of +*simultaneously-mounted* renderables sets a high-water mark you pay **forever**, +even after everything is destroyed. (Fun fact from this week's forensics: we +spent two days believing Ink had this disease. It doesn't — our Ink fork swapped +Yoga-WASM for a plain TypeScript port at fork creation. **We** are the ones +running layout in WASM. The accusation was true; we just had the defendant +wrong.) + +**Solid (the view framework).** Renders each store message into a row via +``. The property we exploit: Solid mounts/unmounts *surgically* — remove a +row from what the component returns and Solid destroys exactly that row's +renderables (returning its handles and freeing its Yoga nodes), touching +nothing else. No virtual-DOM diffing, no collateral re-renders. + +**V8 (the JavaScript engine) + the store.** The store keeps every message as JS +strings/objects. V8's garbage collector is *lazy by design*: with the default +8GB ceiling we launch with, it sees no reason to clean up aggressively, so RSS +includes a lot of "collectible but not yet collected" garbage. Cheap to fix, +worth real MB (measured below). + +**The scrollbox.** One detail that fooled everyone at some point: +`viewportCulling` (on by default) skips *drawing* offscreen rows — but they stay +fully **mounted**: handles held, Yoga nodes alive, memory paid. Culling saves +paint time, not memory. That misunderstanding is half the reason the "rolling +store cap" was expected to be enough, and wasn't. + +## 2. Why it was 686MB + +Simple arithmetic. The old TUI mounted **every message in the store** as a full +renderable tree. 2,000 messages × ~16 renderables × (handles + Yoga nodes + +text buffers + V8 objects) ≈ 670–690MB, growing ~300MB per 1,000 messages. And +at ~1,400 rows the handle table filled: first a hard crash (exit 7), then — +after our containment fix — survival with **unstyled text** past that point, +plus a cap clamped from 3,000 rows down to 1,000 as the price of not crashing. + +Ink, meanwhile, sat at ~234MB at the same workload, because Ink only ever +mounts the rows near your viewport (~84–400 live nodes). Its memory is the +*data* plus some caches — not the *view*. + +## 3. The decisions, in order + +### Decision 1: virtualize the view, don't starve the store + +Two ways to cut view memory: keep fewer messages (opencode's answer — they keep +100 and delete the rest from memory; transcript truth lives on their server), or +keep all messages but only *materialize* the ones near the viewport. You vetoed +the first (your p90 session is 182 messages — a 100-row store truncates normal +sessions), so: **windowing**. Notably the OpenTUI devs confirmed this week that +framework-level virtualization is the intended path — the engine doesn't ship +it out of the box, and opencode never built it. We did. + +### Decision 2: exact heights, recorded at unmount — never estimates in your face + +This is the load-bearing idea, and it's where we beat Ink at its own game. + +The hard problem of any virtualized list: an unmounted row still needs to +occupy its correct *height*, or the scrollbar lies and content jumps. Ink +solves it by **guessing** heights and correcting after measurement — those +corrections are precisely the 83–101ms scroll stutters you hate. You explicitly +vetoed "estimate-correction jank" as a model. + +Our advantage: OpenTUI lays out with real, queryable heights. So when a row +scrolls out of the window, we record its **exact laid-out height** (an +`onSizeChange` hook fires inside layout, pre-paint) and replace the row with an +empty `` — a **spacer**: one Yoga node, zero text +buffers, zero native handles. Think of a bookshelf where books you're not +reading are swapped for cardboard sleeves cut to *exactly* the book's +thickness: the shelf never shifts, and you can't tell from across the room. + +The window is your viewport ± one viewport of margin (plus hysteresis so it +doesn't thrash at the edges). Scroll near a spacer and the real row remounts — +at the recorded height, so nothing moves. + +And one **law**, written into the code as `correctionIsLegal`: a spacer's +height may only ever be corrected where you *cannot see it* — fully above the +viewport (with the scroll position compensated in the same frame, so the world +doesn't move) or fully below it. A correction that would shift visible content +is forbidden, structurally. Jank isn't tuned down; it's outlawed. + +### Decision 3 (the S2 insight): adjudicate on *append*, not just on scroll + +S1 alone got 686 → 518MB. Why not more? Because of *when* windowing decided. +S1 re-decided the window when you **scrolled**. But during a streaming burst — +an agent turn dumping hundreds of rows — you don't scroll; rows arrive, each +mounting fully, and only get demoted later. That transient pile-up is mostly +invisible in steady-state numbers… except for Yoga-WASM, where **the transient +peak is permanent** (memory never shrinks). The burst was quietly ratcheting +the floor. + +S2 makes the window recompute on **transcript growth**: while you're pinned at +the bottom, the window anchors to the content *bottom*, so a row that falls +more than a margin behind the live edge becomes a spacer the moment it's +measured — not whenever you next scroll. Measured result: across a 1,500-row +burst, the peak number of simultaneously-mounted rows is **31**. + +Same trick for **resume**: opening a 2,000-message session used to mount all of +it (transient peak again — paid forever). Now resume mounts only the bottom +window; everything above starts as spacers using a line-count estimate, and an +idle-time "measure march" quietly mounts ten rows at a time near the window +edge, records their true heights, and swaps them back — all outside the +viewport, all invisible by the law above. + +### Decision 4: rows that must never be windowed + +Windowing has to know what it's not allowed to touch: +- **Streaming rows** — the native markdown renderer streams incrementally; + unmounting mid-stream would restart it visibly. +- **The bottom 30 rows** — the region you actually live in. +- **Rows under a mouse selection** — the review caught that a lingering + highlight originally froze windowing *forever* (memory regrowing silently). + Fixed: only an active drag pauses swaps, and selected rows get pinned, so + copy is byte-exact while everything else keeps windowing. + +### Decision 5: give back the scrollback (cap 1,000 → 3,000) + +The 1,000-row clamp existed only because mounted-rows == stored-rows and the +handle table dies at ~1,400. With windowing, mounted ≈ 31 regardless of store +size — so the cap went back to the originally-shipped 3,000. It's +windowing-aware: the `HERMES_TUI_WINDOWING=0` escape hatch (which mounts +everything again) keeps the safe 1,000. + +### Decision 6 (measured, not yet shipped as default): right-size the V8 heap + +Running the windowed TUI with a 512MB heap ceiling instead of 8GB forced V8 to +actually collect: another −90MB with zero latency cost. That's queued as a +launcher default change (~1GB), for both engines. + +## 4. The scoreboard + +At 2,000 messages (your real p99 session size — yes, we checked your DB: +median session is 20 messages, p99 is 1,941): + +| | peak memory | scroll p99 (slowest 1-in-100) | +|---|---|---| +| OpenTUI before | 686MB | 16ms | +| + S1 windowing | 518MB | 16ms | +| + S2 append/resume windowing | **300–375MB** | **6ms** | +| Ink (reference) | 229–246MB | ~100ms | + +At the **3,000-message stress** with the restored triple-size scrollback: +**360MB, fully styled, scroll p99 8ms** — a workload that six days ago crashed +the process, and three days ago survived only by dropping syntax colors. + +Scroll got *faster* because there are simply fewer live renderables to walk. +The determinism gate stayed **byte-identical** — the windowed TUI's settled +frame is provably the same pixels as before. And the live smoke (2,000-message +session: full sweep to the top, resize storm, back to bottom) returned a frame +pixel-identical to boot, with deep history fully syntax-highlighted — something +the pre-windowing TUI literally could not do. + +## 5. What's honestly still open + +- The remaining ~60–120MB over Ink is mostly the **store's JS strings** and + process baseline — the view is no longer the problem. The structural fix is + the **thin renderer** (W1): bodies live in the Python gateway (which already + has them in SQLite); the TUI keeps ~300-byte stubs and fetches bodies only + for the window. That also fixes the class of problem neither engine handles + today: a single 10MB tool output. +- Two accepted, documented limits: scrollbar-*jumping* deep into a freshly + resumed session can land on estimate-height rows that snap to true height as + they enter view (normal scrolling doesn't — the margin pre-measures; the idle + march erodes the exposure over time), and a tool you expanded, scrolled far + away from, then returned to will have re-collapsed (state is component-local; + hoisting it to the store is queued). +- Everything is behind `HERMES_TUI_WINDOWING` (default on, `0` = bit-exact old + behavior) — a one-env escape hatch if anything feels off in real use. + +*Where to verify: `bench/results/` (every number above), the design+gates doc +`docs/plans/opentui-transcript-windowing.md`, tests in +`ui-opentui/src/test/window.test.ts` and `transcriptWindow.test.tsx` (the +zero-jank invariants are literal assertions: identical scrollHeight windowed +vs not, byte-stable frames across corrections).*