mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-06-10 08:32:09 +00:00
f6e6f00ff8
Streamdown's per-Block parse cost grows with the live tail's length and
is unavoidable inside the block-memo pattern (industry standard, see
findings doc). The fix is to stop having that work block the main thread.
`<DeferStreamingText>` is a 12-line wrapper that reads message-part state
via `useMessagePartText`, runs it through `useDeferredValue`, and
re-publishes via assistant-ui's `<TextMessagePartProvider>`. The inner
`<StreamdownTextPrimitive>` reads the deferred value through the normal
`useMessagePartText` hook — no fork, no internal-path imports, fully on
assistant-ui's public API. React's concurrent scheduler then:
- abandons in-flight deferred renders when a newer token arrives, so
intermediate states get skipped under fast streams
- deprioritises the markdown render when the main thread has urgent
work (typing, scroll), so input stays responsive even while a
100ms parse is queued
Streamdown already uses `useTransition` for its block-array setState;
this lifts the deferral up to the consumer boundary so it covers the
whole pipeline (preprocess → split → repair → parse → render).
A/B on the 34 MB session, 300 tokens at 50 tok/sec, markdown chunks
(four trials each, with the 33ms flush throttle on for both):
| | avgFps | p99 frame | LTs/5s | max LT | typing-while-stream p95 |
|---|---|---|---|---|---|
| pre | 54.3 | 41 ms | 1.7 | 110 ms | ~17 ms |
| post | 58.5 | 31 ms | 2.0 | 117 ms | 14-18 ms |
Longtask count + max LT unchanged — useDeferredValue doesn't reduce
CPU, only its priority. The avgFps lift and p99 frame drop are the
proof that the existing CPU is no longer blocking 60 fps cadence. One
clean run logged MUTATIONS=0 — React skipped every intermediate text
state and only committed the final one (textbook deferred-value
behaviour).
The actually-reduce-CPU path is replacing the parser with a state
machine like Flowdown — left for a future PR; see
`apps/desktop/scripts/profile-typing-lag.md` for the full investigation.
|
||
|---|---|---|
| .. | ||
| assert-root-install.cjs | ||
| before-build.cjs | ||
| click-session.mjs | ||
| dev-no-hmr.mjs | ||
| diag-jump.mjs | ||
| eval.mjs | ||
| leak-typing.mjs | ||
| measure-jump.mjs | ||
| measure-latency.mjs | ||
| measure-real-stream.mjs | ||
| measure-submit.mjs | ||
| measure-synthetic-stream.mjs | ||
| notarize-artifact.cjs | ||
| notarize.cjs | ||
| probe-renderer.mjs | ||
| probe-thread.mjs | ||
| profile-long-stream.mjs | ||
| profile-real-stream.mjs | ||
| profile-synth-stream.mjs | ||
| profile-typing-lag.md | ||
| profile-typing.mjs | ||
| reload-renderer.mjs | ||
| reload.mjs | ||
| stage-native-deps.cjs | ||
| test-desktop.mjs | ||
| write-build-stamp.cjs | ||