mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-21 16:18:55 +00:00
bd4953b30d
Fourth profiling round (#66033/#66347/#66470). Streamdown's
parseMarkdownIntoBlocks is a full `marked` lex of the entire message,
and during streaming every flush is a new string — so the splitter paid
O(full-text) ~30x/s: benchmarked 3.4-9.6ms per call at 64-192KB, i.e.
15-30% of a core burned re-lexing settled text on long agent replies.
(The rest of the May-2026 "re-parse elephant" was already eaten by
tailBoundedRemend, block-memo, the KaTeX memo, and deferred shiki —
the splitter was the last O(full-text) pass besides preprocess, which
benches at only 1.5-4.6ms and is left alone.)
New src/lib/markdown-blocks.ts (extracted from markdown-text.tsx) wraps
the splitter with two caches:
- the existing exact-string LRU (remounts: virtualizer scroll, session
switch) — moved, unchanged;
- a streaming-append cache: when the new text startsWith a recently
parsed text, reuse that parse's blocks up to a settled boundary and
lex only the suffix. The boundary drops trailing whitespace-only
blocks plus the last content block — the only block appended text can
reinterpret (open fence, list/table continuation, setext underline,
lazy blockquote). Earlier blocks are separated by settled blank lines
and can't change. Cross-block reference links can't regress:
Streamdown already renders each block as an independent document.
Safety: the splitter's blocks.join('') === text property makes offsets
exact and is defensively re-checked; any mismatch or non-append rewrite
(edit, branch swap) falls back to the full lex — byte-for-byte the old
behavior.
Property tests: at every random streaming cut over a corpus covering
fences, loose lists, tables, setext headings, lazy blockquotes, HTML
blocks, and display math — and token-by-token through a fence boundary
— the cached splitter's output is asserted deep-equal to a fresh full
lex. Bench (128KB reply, 200 flushes): 990ms -> 72ms total splitter CPU
(4.95 -> 0.36ms/flush).
Verification: tsc clean; eslint/prettier clean; lib + assistant-ui
suites green (473 tests).
|
||
|---|---|---|
| .. | ||
| bootstrap-installer | ||
| desktop | ||
| shared | ||