mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-22 16:25:58 +00:00
e934ee440e
The streaming block splitter added in #67154 dropped only the previous parse's trailing whitespace blocks plus its LAST content block before re-lexing the appended suffix. That boundary is unsound: a trailing Setext underline (`-`/`=`) underlines the paragraph ABOVE it, so appending to it can retroactively merge the previous parse's last TWO blocks into one. Minimal repro: cached "…#e\n5\n-" lexes to [ …, "#e\n", "5\n-" ], but grown to "…#e\n5\n-p2=kj:c" collapses "#e"/"5\n-" into a single block. The reused settled prefix still contained a stale "#e\n" block. The `blocks.join('') === text` guard can't detect this because the wrong split reconstructs the same source string, so the divergence rendered as mis-split blocks with no fallback. Fix: drop the last TWO content blocks (skipping whitespace-only blocks around them) before re-lexing the suffix. The block before the last is the deepest an append can reach — a Setext underline consumes exactly one preceding block — and earlier blocks stay fenced off by settled blank lines, so re-lexing two is sufficient and safe. Tests: a deterministic regression for the exact prev→grown pair, and a character-level streaming property fuzz (12 seeds × 500 growing prefixes over the markdown control alphabet). Both fail on the pre-fix boundary and pass after. tsc/eslint/prettier clean; markdown-text suite green. |
||
|---|---|---|
| .. | ||
| bootstrap-installer | ||
| desktop | ||
| shared | ||