Re-ran the cells that crashed, against the fixed binary (a939c9a):
- mem3000 otui-capped: crashed_after_stream exit 7 @ ~900MB
→ completed exit 0, vmhwm 859MB
- mem3000 otui-uncapped: crashed_after_stream exit 7
→ completed exit 0, vmhwm 834MB
- slope10000 otui-uncapped: died exit 7 at 3,000 msgs (197d499 result)
→ completed ALL 10,000 msgs, exit 0, vmhwm 1.57GB —
under the 2GB cgroup cap, no cap-hit, no crash
- fresh ink baselines for both cells (mem3000 257MB / slope10k 328MB).
No "Failed to create SyntaxStyle" anywhere; the store cap now binds at the
handle-safe ceiling (1000 rows) long before the native 65,534-slot handle
table exhausts. report.html + report-assets regenerated via render.mjs
(results are append-only; pre-fix runs remain as the baseline).
Root cause of the bench-suite crash (every otui mem3000/slope cell died at
~3000 lumpy fixture msgs, exit 7, ~880MB RSS — not a cgroup kill):
- @opentui/core 0.4.0 routes EVERY native object through ONE global handle
registry with 16-bit slot indices (core src/zig/handles.zig: INDEX_BITS=16,
MAX_SLOTS=65535, slot 0 reserved). Measured on this install: exactly 65,534
live handles; the next createSyntaxStyle() fails. destroy() DOES recycle
slots — exhaustion means LIVE objects.
- Every TextBufferRenderable burns THREE slots in its constructor
(TextBufferRenderable.ts:77-80: TextBuffer + TextBufferView + SyntaxStyle),
so the mount-everything transcript hits the wall at ~1,400 store rows
(~16 text renderables/row x 3 ~ 47 handles/row): "Failed to create
SyntaxStyle" (zig.ts:4554) throws out of a Solid mount effect.
- The crash was MASKED: CliRenderer's own uncaughtException handler
(handleError -> console.show()) allocates the console-overlay
OptimizedBuffer — another handle — so the handler itself threw "Failed to
create optimized buffer: WxH" and Node died with exit 7 (fatal error in
the uncaughtException handler), hiding the real error.
Why not share one SyntaxStyle (the obvious 3->2): the per-buffer style is
load-bearing — native setStyledText (text-buffer.zig) registers each chunk's
color by NAME ("chunk{i}") into the buffer's OWN style, and registration is
name-keyed-overwrite (syntax-style.zig putStyle), so a shared style would
cross-corrupt chunk colors between every styled <text>. Pooling is unsound
at our layer in core 0.4.0.
The fix, at the seams that are ours:
- boundary/nativeHandles.ts (ffiSafe.ts sibling): SyntaxStyle.create() on a
full table DEGRADES to a detached style (native handle 0) instead of
throwing — JS-side styleDefs/mergeStyles (what markdown/code chunk colors
actually use) keep working; all native calls on handle 0 are inert no-ops.
- boundary/renderer.ts: guard the process error listeners createCliRenderer
installs so an exception INSIDE the handler can never exit-7-mask the
original error again (logged honestly; original error stays the story).
- logic/store.ts: HERMES_TUI_MAX_MESSAGES clamped to a handle-safe ceiling
(1000 rows ~ 47k handles ~ 72% of the table on the realistic fixture).
The old default of 3000 was unreachable — the TUI crashed at ~1,400 rows,
before the cap ever bound. Renderable-weight-aware capping is #27's
(virtualization) to do properly; until then the degrade shim backstops
pathological rows.
TODO(upstream) — issue-shaped, for the OpenTUI repo:
(a) a global 64k handle table with a 3-slot cost per text renderable is
too small for transcript-style TUIs (61k renderables ~ 3k messages);
(b) native allocation failures throw out of the render loop with no
degrade path;
(c) handleError allocates (console overlay buffer) and so crashes on the
very condition it is reporting, masking the root cause with exit 7.
Also: eslint now ignores ui-opentui/.bench/** (bench `nodes`-cell build
artifact broke the lint gate) and .gitignore covers it.
Gate: npm run check green, 599 tests (595 baseline + 3 degrade-path tests
+ 1 cap-clamp test).
The OpenTUI /usage went through the slash-worker subprocess, which
resumes the session WITHOUT a live agent — so it could never show
current-session tokens or costs, and what it did show landed as a
full-screen page.
- slash.exec now answers /usage in-process from the live agent:
per-model rows (requests, tokens in/out, cache, provider-reported
cost when present), session totals/context, a one-line 30-day
summary (SessionDB.usage_totals, real costs only) and a one-line
Nous credits gauge (nous_credits_compact_line, refactored out of
nous_credits_lines). ~8 lines instead of a page.
- Unreported costs render as 'not reported by provider' — never
$0.00 — and the 30d summary omits cost when no session in the
window has a provider-reported figure.
- /usage full keeps the detailed legacy CLI page via the worker.
Cost displays were estimates from a pricing table; on OpenRouter the
status bar never reflected what was actually charged. Now cost is
provider-REPORTED only, end to end:
- OpenRouter requests carry usage:{include:true} (profile + legacy
transport paths); the response usage.cost field (credits, 1:1 USD)
is captured per call into agent.session_actual_cost_usd and
persisted to the sessions DB actual_cost_usd column (NULL-safe:
unreported calls never touch the stored value).
- Nous keeps its x-nous-credits-* header capture; the header delta
now surfaces as the session's real cost via real_session_cost_usd.
- Providers that report nothing accumulate NOTHING: cost fields stay
absent/None (the TUI hides its cost segment), never a fabricated
$0.00 and never an estimate. _get_usage, gateway /usage and the
CLI usage page all switched off estimate_usage_cost for display.
- Per-model session accumulator (session_model_usage) records real
per-call counts and provider-reported cost per model.
User feedback: tool/thinking rows did a "v small quick lil jump up and
down" when toggled, worst on the bottom rows.
Root cause (verified live with 10ms tmux capture sampling): the
transcript scrollbox's sticky-bottom re-pin and the scroll anchor fought
AFTER paint. On a toggle near the bottom, the content-height change runs
ScrollBox.recalculateBarProps -> applyStickyStart("bottom") (the user is
at the sticky position, so _hasManualScroll is false), which paints a
fully bottom-pinned frame; the anchor's 4x16ms scrollTo re-asserts then
yanked the viewport back up. The capture burst shows the transient
pinned frame between two anchored ones on every expand — the visible
down-up flick.
Fix at the cause instead of correcting after the effect: suspend
stickyScroll (a runtime get/set property on ScrollBoxRenderable) BEFORE
running the toggle and restore it ~100ms later, once the content height
has settled. With sticky off, the toggle's layout pass leaves scrollTop
untouched — the clicked header's document position is unchanged (content
grows/shrinks below it), so nothing moves and there is nothing left to
flicker; a collapse past the new bottom clamps naturally via the
ScrollBar scrollSize setter. Restoring recomputes the manual-scroll
state from the actual position: still at the bottom -> keeps pinning for
new content; mid-content -> manual-scroll semantics until the user
returns (the same end state the old anchor produced). Rapid re-toggles
inside the window keep the ORIGINAL saved value.
The far-from-bottom anchor guarantee is unchanged (scrollTop is simply
never touched), pinned headlessly in scrollAnchor.test.tsx along with
the suspension sequencing, the clamp-then-re-pin collapse path, and the
double-toggle restore. ffiSafe's tall-diff scroll-cut regression now
drives the negative-y condition explicitly via wheel scrolls (the old
anchor exercised it through the very transient sticky-bottom frames this
fix removes).
Verified live (tmux, real gateway): before — toggling the bottom rows
painted a transient bottom-pinned frame (f141 of a 10ms burst); after —
three toggle bursts produce ONLY the clean before/after states (4
distinct frames in 458 samples), headers hold their row, including the
bottom-most rows.
User feedback: "for all tools, i'd want all their output viewing enabled
to be infinite by default."
Flip envOutputLines (HERMES_TUI_TOOL_OUTPUT_LINES): unset -> Infinity
(was 200); a positive integer RESTORES a cap (e.g. =200); 0 stays
Infinity for back-compat with the old opt-in-unlimited value; garbage ->
Infinity (unrecognized = no cap asked for). The semantic is now "cap
only when the user asked for one".
The store's raw-result preference follows the same rule: envOutputLinesSet
becomes envOutputUnlimited — whenever the cap is unlimited (the default
now) and a gateway tail-capped result_text (omittedNote) arrives with the
always-full raw result on the wire, the raw result wins, since an
uncapped view of a tail would silently miss the head. With an explicit
finite cap the gateway tail + honest omitted note are kept.
Memory safety is unchanged: tool bodies mount only while EXPANDED (rows
default collapsed and free their Yoga nodes on collapse/unmount), and the
rolling HERMES_TUI_MAX_MESSAGES cap bounds the transcript's high-water
mark.
Tests: env.test.ts expectations flipped (unset/garbage -> Infinity, 0
documented as back-compat); tools.test.tsx "flag unset caps at 200"
becomes "unset renders all 250 lines", plus an explicit =50 cap (+note)
test and =200 restored-cap test; the store preference matrix covers
unset/0 (raw wins), =50 (tail+note kept), and no-raw (tail+note, no
crash). Verified live: seq 1 220 expanded renders rows 201-220 with no
"+N more lines" note.
The native TUI prefetches model.options right after session.create (91df32545,
picker instant-open). The handler is network-bound (~3.7s: pricing fetch + Nous
tier check in build_models_payload) and ran on the gateway's main dispatcher
thread, so every fast-path RPC issued in the first seconds after launch —
complete.slash for the '/' dropdown, session.list, config.get — sat unread
behind it. Measured: first '/' dropdown 1718ms at HEAD vs 53ms at 394f45a3d
(pre-prefetch baseline); 52ms after routing model.options onto the existing
RPC thread pool (_LONG_HANDLERS). The /model picker keeps its 29ms cached open.
A patch tool's result is a JSON record whose payload IS the diff. In a verbose
session the gateway redacts + TAIL-caps result_text (_cap_tui_verbose_text),
so the echo arrived under the native diff in two broken shapes: truncated
mid-JSON (unparseable, so the old JSON.parse check failed open), or — for tall
edits — capped PAST the JSON head, which the store's normalizeOutput then
un-escapes into plain lines that duplicate the diff. North star: no raw JSON
in the transcript, ever.
Three layers:
- gateway: when diff_unified ships, result_text drops the in-JSON diff echo
(_result_sans_diff_echo) — small, parseable, carries only the non-diff
signal (success/files_modified/warnings/lsp_diagnostics).
- fileTool diffOutputPlan: anything starting with '{' under a rendered diff is
suppressed regardless of parseability; parseable JSON with real non-diff
signal (error/warning/lsp_diagnostics) renders JUST those as labeled notes;
a non-JSON fragment whose lines echo the rendered diff is suppressed too
(guards older emitters). Plain-text results (lint tails) still render.
Expanding a tall <diff showLineNumbers> pinned to the scrollbox bottom froze
the TUI with ERR_INVALID_ARG_VALUE looping out of CliRenderer.loop every
frame. Root cause: @opentui/core 0.4.0 marshals OptimizedBuffer
fillRect/drawText/setCell* coordinates as u32 in the FFI table while
LineNumberRenderable.renderSelf passes raw screen coordinates — NEGATIVE when
the diff is partially scrolled above the viewport. Bun's FFI silently wraps
negatives (native side bounds-checks them into a no-op); Node's experimental
node:ffi rejects them. bufferDrawBox already uses i32, which is why ordinary
boxes/text scroll fine and only the diff line-background path crashed.
Fix at the seam we own: boundary/ffiSafe.ts patches OptimizedBuffer to clip
fillRect to the non-negative quadrant and skip negative-origin
drawText/setCell*/drawChar before the FFI call (Bun parity). Installed from
boundary/renderer.ts (live) and test/lib/render.ts (headless). TODO(upstream):
widen those FFI params to i32 so this shim can be deleted.
