mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-08 13:12:08 +00:00
tui(diag): Ink 1Hz memwatch collector — OpenTUI-compatible memory trace
Ink had no continuous memory trace (only point-in-time heapdumps + a threshold
monitor), so HERMES_TUI_DIAGNOSTICS=1 gave OpenTUI dogfood data with no Ink
equivalent. Port OpenTUI's memlog collector to Ink so both engines emit
byte-identical ~/.hermes/logs/memwatch/<boot>-<pid>.jsonl traces feeding one
memwatch-report.mjs.
- lib/memlog.ts: 1Hz unref'd sampler, {t,rss_kb,heap_used_kb,external_kb}
(no mounted — Ink has no windowing), 14-day prune, silent-disable on error
- gated by HERMES_TUI_MEMLOG defaulting to the HERMES_TUI_DIAGNOSTICS master
switch (same as OpenTUI — one export covers both engines)
- wired into entry.tsx alongside the existing monitor; stop on beforeExit
- lib/memlog.test.ts: gate/schema/retention/silent-disable (7 tests)
- docs/ink-env-flags.md (new) + docs/opentui-env-flags.md updated
This commit is contained in:
alt-glitch
parent
b0fb2b8b05
commit
4108fe6014
5 changed files with 349 additions and 2 deletions
68
docs/ink-env-flags.md
Normal file
68
docs/ink-env-flags.md
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
# Ink TUI — diagnostic environment flags
|
||||
|
||||
Non-secret behavioral knobs for the Ink engine (`ui-tui/`). These are
|
||||
**environment overrides**, not `.env` secrets — set them in your shell for a
|
||||
session, or `export` them in your shell rc to make them sticky. They mirror the
|
||||
OpenTUI engine's flags (`docs/opentui-env-flags.md`) so a single switch covers
|
||||
both engines.
|
||||
|
||||
| Flag | Default | What it does |
|
||||
|---|---|---|
|
||||
| `HERMES_TUI_DIAGNOSTICS` | off | Master diagnostics switch. Turning it on enables the developer/profiling surface across the TUI — including the memory self-sampler below. One `export HERMES_TUI_DIAGNOSTICS=1` in your shell rc covers **every** session you start, on **either** engine. |
|
||||
| `HERMES_TUI_MEMLOG` | = `HERMES_TUI_DIAGNOSTICS` | In-process 1Hz memory self-sampling (`ui-tui/src/lib/memlog.ts`) → `~/.hermes/logs/memwatch/<boot>-<pid>.jsonl`. Defaults to the master switch; set `=1` / `=0` to force it on/off independently. |
|
||||
|
||||
## What the memory trace captures
|
||||
|
||||
Each Ink session, when sampling is enabled, appends one JSON line per second to
|
||||
its own file under `~/.hermes/logs/memwatch/`, keyed by boot time + pid:
|
||||
|
||||
```json
|
||||
{"t":1781514892,"rss_kb":92148,"heap_used_kb":7234,"external_kb":2378}
|
||||
```
|
||||
|
||||
- `t` — unix seconds.
|
||||
- `rss_kb` — resident set size (the number that matters for the native-RSS-gap
|
||||
story: rss climbing while heap stays flat is the #15141-class signal).
|
||||
- `heap_used_kb` — V8 heap in use.
|
||||
- `external_kb` — off-heap (buffers, native allocations).
|
||||
|
||||
**Ink emits no `mounted` / `peak_mounted` field.** Those are OpenTUI's
|
||||
windowing dev counters; Ink has no windowing, so it logs the rss/heap/external
|
||||
core only. `memwatch-report.mjs` treats `mounted` as optional, so Ink lines
|
||||
aggregate cleanly alongside OpenTUI's.
|
||||
|
||||
## Why this exists — cross-engine memory comparison
|
||||
|
||||
The filename scheme, directory, and line schema are **byte-compatible with
|
||||
OpenTUI's collector** (`ui-opentui/src/boundary/memlog.ts`). Both engines write
|
||||
to the same `~/.hermes/logs/memwatch/` directory, so one aggregator reads both:
|
||||
|
||||
```sh
|
||||
# enable on either/both engines (master switch covers both)
|
||||
export HERMES_TUI_DIAGNOSTICS=1
|
||||
HERMES_TUI_ENGINE=ink hermes --tui # Ink session → its own .jsonl
|
||||
HERMES_TUI_ENGINE=opentui hermes --tui # OpenTUI session → its own .jsonl
|
||||
|
||||
# fleet table across BOTH engines' sessions:
|
||||
cd ~/github/tui-bench && node memwatch-report.mjs
|
||||
```
|
||||
|
||||
This is what makes a true side-by-side **real-world** memory arc possible —
|
||||
cold floor → load → plateau/leak — instead of comparing OpenTUI dogfood traces
|
||||
against an Ink harness with no equivalent data.
|
||||
|
||||
## Cost & safety
|
||||
|
||||
- ~50 bytes/s when on; one `process.memoryUsage()` + one short append per
|
||||
second. The interval is **unref'd** — it never keeps the process alive.
|
||||
- 14-day retention: older traces are pruned (best-effort) at start.
|
||||
- **Every failure path disables the logger silently.** Diagnostics must never
|
||||
break the TUI — this is the one place the "errors propagate" rule is
|
||||
intentionally inverted, matching the OpenTUI collector.
|
||||
- Off by default: regular users write nothing.
|
||||
|
||||
## Getting a meaningful trace
|
||||
|
||||
A short scroll-through won't show growth. For a comparison against OpenTUI's
|
||||
4–5h sessions, drive a tool-heavy 2–3h Ink session as the floor (see
|
||||
`docs/plans/opentui-ink-asymmetry-note.md` for why the harness ≠ dogfood data).
|
||||
|
|
@ -16,9 +16,14 @@ classified by who should ever touch it. The design rule shipped with this doc:
|
|||
| var | default | effect |
|
||||
|---|---|---|
|
||||
| `HERMES_TUI_ENGINE` | auto (`opentui` if Node≥26.3 + built, else `ink`) | Engine pick; also `display.tui_engine` in config.yaml. |
|
||||
| `HERMES_TUI_MOUSE` | on (launcher sets it) | Mouse support (wheel scroll, selection, click-to-expand). **Glitch verdict 2026-06-12: leave as-is — always on, no realistic reason to disable; treat as plumbing, don't document it user-facing.** |
|
||||
| `HERMES_TUI_MOUSE` / `HERMES_TUI_MOUSE_TRACKING` / `HERMES_TUI_DISABLE_MOUSE` | on | Mouse support (wheel scroll, selection, click-to-expand). **Defers to Ink's env surface (`logic/env.ts` `resolveMouseEnabled`):** precedence is `HERMES_TUI_MOUSE_TRACKING` (toggle, force knob) > `HERMES_TUI_DISABLE_MOUSE=1` (legacy kill switch) > `HERMES_TUI_MOUSE` (OpenTUI-native alias, kept — also what the launcher sets) > default on. OpenTUI's renderer mouse is a single boolean, so Ink's granular off\|wheel\|buttons\|all collapses to on/off (the granular mode lives in `display.mouse_tracking` config). |
|
||||
| `HERMES_TUI_SCROLL_SPEED` (alias `CLAUDE_CODE_SCROLL_SPEED`) | native | Wheel-scroll speed multiplier (Ink parity). UNSET → OpenTUI's native scroll acceleration (untouched). A positive value (clamped to (0,20]) installs a constant-multiplier `ScrollAcceleration` on the transcript scrollbox (`view/transcript.tsx`). |
|
||||
| `HERMES_TUI_NO_CONFIRM` | off | Skip the destructive-action confirm step (`/clear`, `/new`) and run immediately (Ink parity, `NO_CONFIRM_DESTRUCTIVE`). Wired at the `confirm` seam (`entry/main.tsx`). |
|
||||
| `HERMES_TUI_MAX_MESSAGES` | ceiling | Scrollback rows kept in the TUI. Can LOWER the ceiling, never raise: 3000 with windowing, 1000 with windowing off (handle-table safety). |
|
||||
| `HERMES_TUI_TOOL_OUTPUT_LINES` | unlimited | Cap expanded tool-output lines (set a number to restore a cap). |
|
||||
| `HERMES_TUI_TOOL_OUTPUTS` | **on** | Keep rich tool-call OUTPUTS (full result body + raw result/args dicts). `=off` drops both the RENDER and the STORE of those bodies (Ink parity: only a one-line context preview + name/duration/error/diff survive) — the memory lever for the OpenTUI-vs-Ink retention asymmetry, and what the bench launches OpenTUI with for the fair engine-overhead comparison (W3). Diffs (file-edit) are KEPT either way. |
|
||||
| `HERMES_TUI_HEAP_MB` | cgroup-aware (default 8192) | V8 `--max-old-space-size` (MB) for BOTH engines. Highest precedence (then `display.tui_heap_mb` config, then the cgroup-75% fallback). Set it LOW for a low-mem session (still cgroup-clamped on top so it never exceeds the container); raise it to lift the ceiling. The low-mem opt-in signal that also arms `HERMES_TUI_PROACTIVE_GC` (W1). |
|
||||
| `HERMES_TUI_PROACTIVE_GC` | = low-`HERMES_TUI_HEAP_MB` (≤4096) | Idle-gated `global.gc()` for the low-mem path. Defaults ON only when a low heap cap is set (so the knobs compose); `=on`/`=off` forces it. Needs `--expose-gc` (the OpenTUI argv now carries it). Never runs mid-stream; tightens cadence above 400MB RSS but stays idle-gated. OpenTUI-only — Ink never GCs proactively (W2). |
|
||||
| `HERMES_TUI_COMPOSER_ROWS` | default rows | Composer height. |
|
||||
|
||||
## 3. Escape hatches & tuning (dev-facing, individually settable)
|
||||
|
|
@ -30,6 +35,8 @@ classified by who should ever touch it. The design rule shipped with this doc:
|
|||
| `HERMES_TUI_WINDOW_STATS` | = `HERMES_TUI_DIAGNOSTICS` | Exposes live/peak mounted-row counters (`globalThis.__hermesTuiWindowStats`) for tui-bench's live-attach reads. |
|
||||
| `HERMES_TUI_MEMLOG` | = `HERMES_TUI_DIAGNOSTICS` | In-process 1Hz memory self-sampling (`boundary/memlog.ts`) → `~/.hermes/logs/memwatch/<boot>-<pid>.jsonl` (rss/heap/external + mounted rows; 14-day retention). Fleet view: `node memwatch-report.mjs` from the tui-bench repo (`github.com/NousResearch/tui-bench`). The "monitor all my sessions" answer: one `export HERMES_TUI_DIAGNOSTICS=1` in your shell rc covers every session. |
|
||||
| `HERMES_TUI_LOG_LEVEL` / `HERMES_TUI_LOG_FILE` | engine defaults | Logging verbosity/destination (`/logs` reads the ring buffer regardless). Deliberately independent of the master switch — support often wants logs without the full diag surface. |
|
||||
| `HERMES_HEAPDUMP_ON_START` | off | Write one V8 heap snapshot at boot (Ink parity). A deliberate baseline-capture escape hatch that BYPASSES the diagnostics master switch; lands at `$HERMES_HOME/logs/opentui-heap-<ts>.heapsnapshot` and echoes the path as a system line (`entry/main.tsx`). |
|
||||
| `HERMES_TUI_NOTIFY` | on | Desktop-notification kill switch (`=0`/`false`/`off` silences the "waiting on you" pings). The ping itself goes through the renderer's native `triggerNotification` (protocol detection + tmux/Zellij wrapping); the window title is not gated by this. |
|
||||
|
||||
## 4. Internal plumbing (set by the launcher/tui-bench/tests — humans never set these)
|
||||
|
||||
|
|
@ -37,10 +44,26 @@ classified by who should ever touch it. The design rule shipped with this doc:
|
|||
|---|---|---|
|
||||
| `HERMES_PYTHON`, `HERMES_PYTHON_SRC_ROOT`, `HERMES_CWD` | launcher / bench | Which gateway python + repo root + cwd the TUI spawns against (the bench's fake-gateway seam). |
|
||||
| `HERMES_TUI_ACTIVE_SESSION_FILE` | launcher/bench | Session handoff file. |
|
||||
| `HERMES_TUI_RESUME`, `HERMES_TUI_PROMPT`, `HERMES_TUI_FAKE` | launcher/tests | Resume-at-boot, seeded prompt, fake-mode. |
|
||||
| `HERMES_TUI_RESUME`, `HERMES_TUI_QUERY`, `HERMES_TUI_PROMPT`, `HERMES_TUI_IMAGE`, `HERMES_TUI_FAKE` | launcher/tests | Resume-at-boot; seeded prompt (`--tui "prompt"`: launcher sets `HERMES_TUI_QUERY`, the engine reads QUERY > the `HERMES_TUI_PROMPT` alias > a bare argv tail — `logic/env.ts` `startupPrompt`); seeded image PATH (`--image`: `HERMES_TUI_IMAGE`, `image.attach`ed before the prompt — `startupImage`, attach in `postSessionSetup`); fake-mode. |
|
||||
| `HERMES_AUTO_HEAPDUMP*` (`_COOLDOWN_MS`/`_MAX_BYTES`), `HERMES_HEAPDUMP_DIR`, `HERMES_HEAPDUMP_MAX_BYTES` | — | **NOT read by the OpenTUI engine (deliberate).** The engine ports Ink's #34095 silent-death early-WARNING (a transcript system line, `boundary/memoryMonitor.ts`) but NOT the auto heap-SNAPSHOT capture — the always-on memlog NDJSON trace is the diagnosis path, and its rss-vs-heap divergence is the better diagnostic for the native-RSS leak class (#15141) a V8 snapshot captures poorly. So the #41948 disk-fill safety set (gate/cooldown/byte-cap/dir) has no consumer here. `HERMES_HEAPDUMP_ON_START` (manual one-shot, §3) is the only heapdump knob the engine honors. |
|
||||
| `HERMES_TUI_RPC_TIMEOUT_MS`, `HERMES_TUI_STARTUP_TIMEOUT_MS` | tests/CI | Protocol timeouts. |
|
||||
| (`ui-tui` only) `HERMES_TUI_MEMSAMPLE_FD/MS` | bench | Ink fd-3 node sampler. |
|
||||
|
||||
## 5. Ink flags NOT ported — handled natively or out of scope
|
||||
|
||||
These exist on the legacy Ink TUI (`ui-tui/`) and are deliberately **not** read
|
||||
by the OpenTUI engine. Documented so a missing flag reads as a decision, not a gap.
|
||||
|
||||
| Ink flag | why not ported |
|
||||
|---|---|
|
||||
| `HERMES_TUI_TRUECOLOR` | OpenTUI core does COLORTERM/truecolor detection natively — the Ink force-truecolor hack is a fork workaround we shed. |
|
||||
| `HERMES_TUI_FORCE_OSC52` | OpenTUI core owns OSC52 clipboard as a primitive; no fallback hint needed. |
|
||||
| `HERMES_TUI_INLINE` / `HERMES_TUI_TERMUX_MODE` / `HERMES_TUI_TERMUX_FAST_ECHO` | Termux/primary-buffer accommodations. OpenTUI's native FFI floor (Node ≥26.3 + `--experimental-ffi`) is absent on Termux, so those sessions stay on **Ink** — these are correctly N/A for the OpenTUI engine. |
|
||||
| `HERMES_TUI_FPS` | Ink FPS overlay; the OpenTUI equivalent is the diag/window-stats surface (`HERMES_TUI_WINDOW_STATS`). Not parity-critical. |
|
||||
| `HERMES_DEV_CREDITS` / `HERMES_DEV_PERF*` | Dev-only throwaway scaffolding (live-spend readout, perf logging) — not user parity. |
|
||||
| `HERMES_BIN` / `HERMES_TUI_GATEWAY_URL` / `HERMES_TUI_SIDECAR_URL` | External-CLI / remote-gateway-URL overrides. OpenTUI spawns its gateway via the Effect boundary (`liveGateway.ts`) and does not shell out to `hermes` or take an external gateway URL. |
|
||||
| `HERMES_VOICE` | Voice mode is tracked on the OpenTUI parity backlog separately, not here. |
|
||||
|
||||
## How the pieces compose (the support script)
|
||||
|
||||
- Regular user, normal day: zero flags, zero diagnostic commands visible.
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import { TERMUX_TUI_MODE } from './config/env.js'
|
|||
import { GatewayClient } from './gatewayClient.js'
|
||||
import { setupGracefulExit } from './lib/gracefulExit.js'
|
||||
import { formatBytes, type HeapDumpResult, performHeapDump } from './lib/memory.js'
|
||||
import { startMemlog } from './lib/memlog.js'
|
||||
import { type MemorySnapshot, startMemoryMonitor } from './lib/memoryMonitor.js'
|
||||
import { openExternalUrl } from './lib/openExternalUrl.js'
|
||||
import { recordParentLifecycle } from './lib/parentLog.js'
|
||||
|
|
@ -108,7 +109,14 @@ if (process.env.HERMES_HEAPDUMP_ON_START === '1') {
|
|||
void performHeapDump('manual')
|
||||
}
|
||||
|
||||
// Fleet memory self-sampling (HERMES_TUI_MEMLOG / diagnostics master switch).
|
||||
// Writes a 1Hz rss/heap/external NDJSON trace to ~/.hermes/logs/memwatch/ in
|
||||
// the SAME format OpenTUI emits, so one memwatch-report.mjs aggregates both
|
||||
// engines. No-op unless enabled. Unref'd interval — never keeps us alive.
|
||||
const stopMemlog = startMemlog()
|
||||
|
||||
process.on('beforeExit', () => stopMemoryMonitor())
|
||||
process.on('beforeExit', () => stopMemlog())
|
||||
|
||||
const [ink, { App }, { logFrameEvent }, { trackFrame }] = await Promise.all([
|
||||
import('@hermes/ink'),
|
||||
|
|
|
|||
139
ui-tui/src/lib/memlog.test.ts
Normal file
139
ui-tui/src/lib/memlog.test.ts
Normal file
|
|
@ -0,0 +1,139 @@
|
|||
import { mkdtempSync, readdirSync, readFileSync, rmSync, utimesSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
import { startMemlog } from './memlog.js'
|
||||
|
||||
const ENV_KEYS = ['HERMES_TUI_MEMLOG', 'HERMES_TUI_DIAGNOSTICS', 'HERMES_HOME'] as const
|
||||
|
||||
const memwatch = (home: string) => join(home, 'logs', 'memwatch')
|
||||
|
||||
describe('startMemlog (Ink 1Hz memory trace, OpenTUI-compatible)', () => {
|
||||
let saved: Record<string, string | undefined>
|
||||
let home: string
|
||||
|
||||
beforeEach(() => {
|
||||
saved = {}
|
||||
for (const k of ENV_KEYS) {
|
||||
saved[k] = process.env[k]
|
||||
delete process.env[k]
|
||||
}
|
||||
home = mkdtempSync(join(tmpdir(), 'hermes-memlog-test-'))
|
||||
process.env.HERMES_HOME = home
|
||||
vi.useFakeTimers()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers()
|
||||
for (const k of ENV_KEYS) {
|
||||
if (saved[k] === undefined) delete process.env[k]
|
||||
else process.env[k] = saved[k]
|
||||
}
|
||||
rmSync(home, { force: true, recursive: true })
|
||||
})
|
||||
|
||||
it('is a no-op (writes nothing) when neither flag is set', () => {
|
||||
const stop = startMemlog()
|
||||
vi.advanceTimersByTime(3000)
|
||||
stop()
|
||||
// dir is never even created
|
||||
expect(() => readdirSync(memwatch(home))).toThrow()
|
||||
})
|
||||
|
||||
it('writes a 1Hz trace when HERMES_TUI_MEMLOG=1', () => {
|
||||
process.env.HERMES_TUI_MEMLOG = '1'
|
||||
const stop = startMemlog()
|
||||
vi.advanceTimersByTime(3000)
|
||||
stop()
|
||||
|
||||
const files = readdirSync(memwatch(home)).filter(f => f.endsWith('.jsonl'))
|
||||
expect(files.length).toBe(1)
|
||||
// filename scheme: <boot15>-<pid>.jsonl, identical to OpenTUI's
|
||||
// (new Date().toISOString() with :/. stripped, sliced to 15 chars →
|
||||
// "2026-06-15T0914"), so memwatch-report.mjs reads both engines.
|
||||
expect(files[0]).toMatch(/^\d{4}-\d{2}-\d{2}T\d{4}-\d+\.jsonl$/)
|
||||
|
||||
const lines = readFileSync(join(memwatch(home), files[0]), 'utf8').trim().split('\n')
|
||||
expect(lines.length).toBe(3) // one per second
|
||||
})
|
||||
|
||||
it('emits the OpenTUI-compatible rss/heap/external schema (no mounted on Ink)', () => {
|
||||
process.env.HERMES_TUI_MEMLOG = '1'
|
||||
const stop = startMemlog()
|
||||
vi.advanceTimersByTime(1000)
|
||||
stop()
|
||||
|
||||
const files = readdirSync(memwatch(home)).filter(f => f.endsWith('.jsonl'))
|
||||
const sample = JSON.parse(readFileSync(join(memwatch(home), files[0]), 'utf8').trim())
|
||||
expect(sample).toHaveProperty('t')
|
||||
expect(sample).toHaveProperty('rss_kb')
|
||||
expect(sample).toHaveProperty('heap_used_kb')
|
||||
expect(sample).toHaveProperty('external_kb')
|
||||
expect(typeof sample.rss_kb).toBe('number')
|
||||
// Ink has no windowing — these OpenTUI-only fields must NOT appear
|
||||
expect(sample).not.toHaveProperty('mounted')
|
||||
expect(sample).not.toHaveProperty('peak_mounted')
|
||||
})
|
||||
|
||||
it('defaults to the HERMES_TUI_DIAGNOSTICS master switch', () => {
|
||||
process.env.HERMES_TUI_DIAGNOSTICS = '1'
|
||||
const stop = startMemlog()
|
||||
vi.advanceTimersByTime(1000)
|
||||
stop()
|
||||
expect(readdirSync(memwatch(home)).filter(f => f.endsWith('.jsonl')).length).toBe(1)
|
||||
})
|
||||
|
||||
it('lets HERMES_TUI_MEMLOG=0 override the master switch (off)', () => {
|
||||
process.env.HERMES_TUI_DIAGNOSTICS = '1'
|
||||
process.env.HERMES_TUI_MEMLOG = '0'
|
||||
const stop = startMemlog()
|
||||
vi.advanceTimersByTime(2000)
|
||||
stop()
|
||||
expect(() => readdirSync(memwatch(home))).toThrow()
|
||||
})
|
||||
|
||||
it('prunes traces older than 14 days at start (keeps recent)', () => {
|
||||
const dir = memwatch(home)
|
||||
// seed an old + a fresh trace before enabling
|
||||
process.env.HERMES_TUI_MEMLOG = '1'
|
||||
// create the dir via a first run, then stop
|
||||
const warm = startMemlog()
|
||||
vi.advanceTimersByTime(1000)
|
||||
warm()
|
||||
|
||||
const old = join(dir, '20000101T000000-99999.jsonl')
|
||||
const fresh = join(dir, '20991231T235959-88888.jsonl')
|
||||
writeFileSync(old, '{}\n')
|
||||
writeFileSync(fresh, '{}\n')
|
||||
const ancient = Date.now() / 1000 - 30 * 24 * 3600
|
||||
utimesSync(old, ancient, ancient)
|
||||
|
||||
// a fresh start triggers pruneOld()
|
||||
const stop = startMemlog()
|
||||
vi.advanceTimersByTime(1000)
|
||||
stop()
|
||||
|
||||
const remaining = readdirSync(dir)
|
||||
expect(remaining).not.toContain('20000101T000000-99999.jsonl')
|
||||
expect(remaining).toContain('20991231T235959-88888.jsonl')
|
||||
})
|
||||
|
||||
it('silently disables on a write failure (never throws, never retries forever)', () => {
|
||||
process.env.HERMES_TUI_MEMLOG = '1'
|
||||
const stop = startMemlog()
|
||||
// first sample writes fine
|
||||
vi.advanceTimersByTime(1000)
|
||||
// now make appendFileSync blow up — the collector must clearInterval, not throw
|
||||
const fs = require('node:fs')
|
||||
const spy = vi.spyOn(fs, 'appendFileSync').mockImplementation(() => {
|
||||
throw new Error('disk full')
|
||||
})
|
||||
expect(() => vi.advanceTimersByTime(1000)).not.toThrow()
|
||||
// interval cleared: further ticks do nothing even after restoring fs
|
||||
spy.mockRestore()
|
||||
expect(() => vi.advanceTimersByTime(5000)).not.toThrow()
|
||||
stop()
|
||||
})
|
||||
})
|
||||
109
ui-tui/src/lib/memlog.ts
Normal file
109
ui-tui/src/lib/memlog.ts
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
/**
|
||||
* memlog — in-process 1Hz memory self-sampling to NDJSON (Ink engine).
|
||||
*
|
||||
* Byte-for-byte the Ink counterpart of OpenTUI's
|
||||
* `ui-opentui/src/boundary/memlog.ts`: every TUI session logs its OWN samples
|
||||
* when enabled, keyed by pid + boot time, into `~/.hermes/logs/memwatch/`.
|
||||
* Both engines write to the SAME directory with the SAME filename scheme and
|
||||
* the SAME line schema so a single `memwatch-report.mjs`
|
||||
* (github.com/NousResearch/tui-bench) aggregates Ink and OpenTUI sessions into
|
||||
* one fleet table. That cross-engine compatibility IS the deliverable — it's
|
||||
* what lets the bench show a true side-by-side real-world memory arc instead of
|
||||
* "OpenTUI has dogfood data, Ink only has the harness."
|
||||
*
|
||||
* Gating: `HERMES_TUI_MEMLOG` — defaults to the `HERMES_TUI_DIAGNOSTICS`
|
||||
* master switch, individually overridable either way. One
|
||||
* `export HERMES_TUI_DIAGNOSTICS=1` in a dev's shell rc therefore covers every
|
||||
* session they ever start, on EITHER engine; regular users write nothing.
|
||||
*
|
||||
* Cost when on: one `process.memoryUsage()` + one short append per second
|
||||
* (~50 bytes/s). The interval is unref'd — it never keeps the process alive.
|
||||
* Every failure path disables the logger silently (diagnostics must never break
|
||||
* the TUI; this is the one place the "errors propagate" rule is intentionally
|
||||
* inverted, matching the OpenTUI collector). Retention: files older than 14
|
||||
* days are pruned at start, best-effort.
|
||||
*
|
||||
* Sample shape (one JSON object per line):
|
||||
* { t, rss_kb, heap_used_kb, external_kb }
|
||||
* Ink has no windowing, so it emits NO `mounted`/`peak_mounted` field (those
|
||||
* are OpenTUI-windowing-specific). The rss/heap_used/external core is the
|
||||
* apples-to-apples comparison — and rss-vs-heap is exactly the native-RSS-gap
|
||||
* signal the memory story is about. `memwatch-report.mjs` treats `mounted` as
|
||||
* optional, so Ink lines aggregate cleanly alongside OpenTUI's.
|
||||
*/
|
||||
import { appendFileSync, mkdirSync, readdirSync, statSync, unlinkSync } from 'node:fs'
|
||||
import { homedir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
|
||||
const RETENTION_DAYS = 14
|
||||
const SAMPLE_MS = 1000
|
||||
|
||||
const truthy = (v?: string) => /^(?:1|true|yes|on)$/i.test((v ?? '').trim())
|
||||
const falsy = (v?: string) => /^(?:0|false|no|off)$/i.test((v ?? '').trim())
|
||||
|
||||
/**
|
||||
* Resolve a per-flag toggle against a default. Mirrors OpenTUI's `envFlag`:
|
||||
* an explicit truthy/falsy value on the flag wins; otherwise the default
|
||||
* (here, the diagnostics master switch) decides. Read per call so a wrapper
|
||||
* that mutates env before launch sees the live value.
|
||||
*/
|
||||
function memlogEnabled(): boolean {
|
||||
const diag = truthy(process.env.HERMES_TUI_DIAGNOSTICS)
|
||||
const raw = (process.env.HERMES_TUI_MEMLOG ?? '').trim()
|
||||
if (truthy(raw)) return true
|
||||
if (falsy(raw)) return false
|
||||
return diag
|
||||
}
|
||||
|
||||
function memwatchDir(): string {
|
||||
const home = process.env.HERMES_HOME?.trim()
|
||||
const base = home && home.length > 0 ? home : join(homedir(), '.hermes')
|
||||
return join(base, 'logs', 'memwatch')
|
||||
}
|
||||
|
||||
function pruneOld(dir: string): void {
|
||||
const cutoff = Date.now() - RETENTION_DAYS * 24 * 3600 * 1000
|
||||
try {
|
||||
for (const name of readdirSync(dir)) {
|
||||
if (!name.endsWith('.jsonl')) continue
|
||||
const p = join(dir, name)
|
||||
try {
|
||||
if (statSync(p).mtimeMs < cutoff) unlinkSync(p)
|
||||
} catch {
|
||||
/* best-effort */
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
/* best-effort */
|
||||
}
|
||||
}
|
||||
|
||||
/** Start the self-sampler (no-op unless enabled). Returns a stop function. */
|
||||
export function startMemlog(): () => void {
|
||||
if (!memlogEnabled()) return () => {}
|
||||
try {
|
||||
const dir = memwatchDir()
|
||||
mkdirSync(dir, { recursive: true })
|
||||
pruneOld(dir)
|
||||
const boot = new Date().toISOString().replace(/[:.]/g, '').slice(0, 15)
|
||||
const file = join(dir, `${boot}-${process.pid}.jsonl`)
|
||||
const timer = setInterval(() => {
|
||||
try {
|
||||
const m = process.memoryUsage()
|
||||
const line = JSON.stringify({
|
||||
t: Math.floor(Date.now() / 1000),
|
||||
rss_kb: Math.floor(m.rss / 1024),
|
||||
heap_used_kb: Math.floor(m.heapUsed / 1024),
|
||||
external_kb: Math.floor(m.external / 1024)
|
||||
})
|
||||
appendFileSync(file, line + '\n')
|
||||
} catch {
|
||||
clearInterval(timer) // a failing diagnostic must not retry forever
|
||||
}
|
||||
}, SAMPLE_MS)
|
||||
timer.unref?.()
|
||||
return () => clearInterval(timer)
|
||||
} catch {
|
||||
return () => {}
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue