mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-22 16:25:58 +00:00
dd418284db
Chased the "stream frame p95 = 60ms with ZERO longtasks" mystery to its actual cause: the default stream chunk had no paragraph breaks, so it grew into one giant ~22KB block that re-rendered fully every flush — defeating the block memoization real streaming relies on. Plain text = 21ms; realistic chunk with `\n\n` breaks (blocks settle, only the tail re-renders) = 23ms. Fixed the default chunk to model real LLM output; a break-less `--chunk` remains available as a single-block worst-case stress. Also hardened the isolated instance so measurements reflect real cost: - Wait for the gateway socket to actually connect before measuring (a booting/ absent backend's reconnect backoff churns the main thread). Exposed via a new __PERF_DRIVE__.connected() probe reading $gateway.connectionState. - Focus emulation + anti-throttle/occlusion flags so a backgrounded perf window isn't frame-throttled (no OS focus stealing). - Generation-guarded the rAF frame recorder so repeated runs don't leave overlapping recorders polluting frame intervals. Baseline re-captured as the median of 5 --spawn runs (darwin-arm64); all three CI scenarios now green and stable. Absolute values are dev-build (noted in _meta) — regression guards, not shipped numbers.
47 lines
1.3 KiB
JSON
47 lines
1.3 KiB
JSON
{
|
|
"_meta": {
|
|
"note": "Median of 5 runs, darwin-arm64, `npm run perf -- --spawn` (dev renderer, gateway waited-for-connected, realistic block-settling stream chunk). Absolute values are dev-build (React dev mode is ~3x prod) so treat them as regression guards, not shipped numbers; re-baseline per device with `npm run perf -- --update-baseline`. Tolerances are loose to absorb cross-machine variance.",
|
|
"platform": "darwin-arm64",
|
|
"node": "v24.11.0",
|
|
"updated": "2026-07-19T21:24:19.047Z"
|
|
},
|
|
"scenarios": {
|
|
"stream": {
|
|
"tolerance": {
|
|
"tolFrac": 0.6,
|
|
"tolAbs": 5
|
|
},
|
|
"metrics": {
|
|
"longtasks_n": 1,
|
|
"longtask_max_ms": 145,
|
|
"frame_p95_ms": 23.5,
|
|
"frame_p99_ms": 26.9,
|
|
"slow_frames_33": 1,
|
|
"intermut_p95_ms": 48.2
|
|
}
|
|
},
|
|
"keystroke": {
|
|
"tolerance": {
|
|
"tolFrac": 0.6,
|
|
"tolAbs": 4
|
|
},
|
|
"metrics": {
|
|
"keystroke_p50_ms": 2,
|
|
"keystroke_p95_ms": 8.2,
|
|
"keystroke_p99_ms": 17.4,
|
|
"keystroke_slow_16": 2
|
|
}
|
|
},
|
|
"transcript": {
|
|
"tolerance": {
|
|
"tolFrac": 0.75,
|
|
"tolAbs": 40
|
|
},
|
|
"metrics": {
|
|
"transcript_mount_ms": 280.1,
|
|
"transcript_longtask_ms": 431,
|
|
"transcript_longtask_max_ms": 221
|
|
}
|
|
}
|
|
}
|
|
}
|