mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-18 04:41:56 +00:00
fix(cli,tui): align CJK / wide-char markdown tables (#23863)
CJK and emoji glyphs render as two terminal cells but JS String#length and the model's own padding count them as one, so any markdown table with Chinese / Japanese / Korean cells drifts right per row when a real terminal renders it. Both surfaces fix this with a display-cell width measurement (wcswidth on the Python side, stringWidth on the TUI side). Changes: - agent/markdown_tables.py: new helper. realign_markdown_tables(text) detects markdown table blocks (header + |---| divider) and rewrites the row padding using wcwidth.wcswidth so every pipe and dash lines up across rows. No-op on text without tables. - cli.py: hook the helper into _render_final_assistant_content for strip / render modes (raw passes through untouched), and into the streaming line emitter so live token-by-token rendering also produces aligned tables. A small two-buffer state machine in _emit_stream_text holds table rows until the block ends, then flushes them through the realigner so all rows pad to a single per-column width. - ui-tui/src/components/markdown.tsx: renderTable now uses stringWidth (Bun.stringWidth fast path + East-Asian-width-aware fallback, already memoised in @hermes/ink) instead of UTF-16 String#length for both column-width measurement and per-cell padding. Drops the comment that documented the bug as a deliberate limitation. Validation: - New tests/agent/test_markdown_tables.py (11): every rebuilt block shares pipe column offsets across rows for pure CJK, mixed CJK+emoji, ragged-row, and multi-table inputs. - Updated tests/cli/test_cli_markdown_rendering.py: the existing strip-mode test asserted exact whitespace; rewritten to assert the alignment contract (cell content survives + every rendered row shares pipe offsets). - New ui-tui markdown.test.ts case (1): rendered column-2 start offset is identical for the header + every body row, including the CJK row that drifted before the fix. - Live: hermes chat -q with the user-reported screenshot prompt now produces a perfectly aligned table on the wire (header, divider, 4 body rows including '通义千问', all pipes at identical columns).
This commit is contained in:
parent
657874460f
commit
1d00716754
6 changed files with 559 additions and 15 deletions
|
|
@ -118,14 +118,37 @@ def test_strip_mode_preserves_table_structure_while_cleaning_cell_markdown():
|
|||
)
|
||||
|
||||
output = _render_to_text(renderable)
|
||||
assert "| Syntax | Example |" in output
|
||||
assert "|---|---|" in output
|
||||
assert "| Bold | bold |" in output
|
||||
assert "| Strike | strike |" in output
|
||||
|
||||
# Inline cell markdown is stripped (the contract this test enforces).
|
||||
assert "**" not in output
|
||||
assert "~~" not in output
|
||||
assert "`" not in output
|
||||
|
||||
# Cell *content* survives, even if the surrounding whitespace was
|
||||
# rewritten by the wcwidth-aware re-aligner. Asserting on bare
|
||||
# cell text keeps this test focused on the strip behaviour rather
|
||||
# than snapshotting incidental column padding (which is what the
|
||||
# CJK-alignment fix changes).
|
||||
assert "Syntax" in output
|
||||
assert "Example" in output
|
||||
assert "Bold" in output and "bold" in output
|
||||
assert "Strike" in output and "strike" in output
|
||||
|
||||
# Structural sanity: the table still renders as pipe-bordered rows
|
||||
# (header + divider + 2 body rows).
|
||||
body_rows = [ln for ln in output.splitlines() if ln.strip().startswith("|")]
|
||||
assert len(body_rows) == 4
|
||||
|
||||
# Every rendered table row shares the same pipe column offsets — the
|
||||
# alignment guarantee from realign_markdown_tables.
|
||||
pipe_cols = [
|
||||
[i for i, ch in enumerate(row) if ch == "|"] for row in body_rows
|
||||
]
|
||||
assert all(p == pipe_cols[0] for p in pipe_cols), (
|
||||
"table rows misaligned after strip-mode rendering:\n"
|
||||
+ "\n".join(body_rows)
|
||||
)
|
||||
|
||||
|
||||
def test_final_assistant_content_can_leave_markdown_raw():
|
||||
renderable = _render_final_assistant_content("***Bold italic***", mode="raw")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue