mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-07-11 13:41:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

opentui(v6): bench fixture — HERMES_BENCH_TOOL_BODY_LINES knob for fat tool output

Browse source 
The default lumpy-turn fixture's tool bodies are tiny (2/7/18 short lines), so
W3 (HERMES_TUI_TOOL_OUTPUTS=off) shows no RSS delta at realistic sizes — the
saved bytes sit below the ~20MB run-to-run noise floor. This adds an env knob
to scale every tool result body to N lines, making the retention asymmetry
measurable in the bench (a `find /` / big-file-read class fixture).

UNSET = the original tiny bodies, byte-identical — existing benches and the
determinism digest are unaffected. Set HERMES_BENCH_TOOL_BODY_LINES=N (the
bench harness inherits it at fixture-generation time) for a fat run.

Measured with this (mem300, ~100KB/tool, ~27MB retained tool text): OpenTUI
OFF 244-259MB vs ON 260-261MB vs Ink 269-279MB — i.e. W3 OFF is a real
~5-15MB win at heavy output, and OpenTUI's windowing actually beats Ink at
scale (Ink mounts every row; OpenTUI windows).
This commit is contained in:
alt-glitch 2026-06-14 16:12:10 +05:30
parent a70f7f3b7b
commit 1ddf7a1021
1 changed files with 6 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

7
ui-opentui/scripts/fixture.ts
View file

@ -130,7 +130,12 @@ function toolEvents(seed: number, t: number): GatewayEvent[] {
const name = TOOL_NAMES[(seed + t) % TOOL_NAMES.length] ?? 'terminal'
const variant = (seed + t) % 3
// short / capped-16-line / medium result bodies, mixing the render-cost cases.
const bodyLines = variant === 0 ? 2 : variant === 1 ? 18 : 7
// BENCH KNOB: HERMES_BENCH_TOOL_BODY_LINES overrides the body size to N lines
// for ALL tools — the "fat tool output" fixture (e.g. a `find /` dump) used to
// make W3's retention win measurable. UNSET = the original tiny bodies (so the
// default fixture stays byte-identical; existing benches are unaffected).
const fatLines = Number.parseInt(process.env.HERMES_BENCH_TOOL_BODY_LINES ?? '', 10)
const bodyLines = Number.isFinite(fatLines) && fatLines > 0 ? fatLines : variant === 0 ? 2 : variant === 1 ? 18 : 7
const resultText = lines(seed + t * 3, bodyLines)
const context = sentence(seed + t, 4)
// ~half the tools carry a multi-line args block (the expanded-view cost).