perf(cli): TTFT round 2 — live reasoning by default, partial-line streaming, prompt-build cache, stale budget-warning docs (#59389)

Follow-up to #59332 targeting the remaining PERCEIVED first-token latency
(the wire streaming was already per-token; these fix what the user sees):

1. display.show_reasoning default ON. On thinking models the reasoning
   phase streams for tens of seconds; with the display off users stare
   at a spinner the whole time and read it as a stall. Flipped in
   DEFAULT_CONFIG, load_cli_config defaults, tui_gateway raw-YAML
   fallbacks, and the hermes setup status line (all four read sites kept
   in sync). Gateway per-platform defaults intentionally stay off —
   messaging chats shouldn't fill with thinking text. /reasoning hide
   still turns it off and persists.

2. Response box force-flushes long partial lines. _emit_stream_text only
   painted on newline, so a response opening with a long paragraph
   stayed invisible until the first \n — seconds of blank box. Now
   partial lines wrap at terminal width and paint as tokens arrive
   (mirrors the reasoning box's 80-char force-flush that existed since
   day one). Table blocks remain batch-aligned; no content loss at wrap
   boundaries (regression tests added).

3. hermes_time timezone resolution uses read_raw_config (mtime-cached +
   libyaml C loader) instead of a raw yaml.safe_load of config.yaml
   (~110-140ms measured) inside the FIRST system prompt build. First
   build drops 320ms -> ~155ms on a 200-skill install.

4. Stale docs: configuration.md (en+zh) still documented the 70%/90%
   [BUDGET WARNING] tool-result injections. Those were removed in April
   2026 (c8aff7463) precisely because they hurt task completion; current
   behavior is exhaustion-message + one grace call, no mid-loop
   injection, no cache impact. Docs now describe reality.

Verified: token-count compression decisions already use API-reported
last_prompt_tokens (rough estimators are preflight-only and cost ~1.7ms
even on 1.7MB histories — not worth touching).
This commit is contained in:
Teknium 2026-07-06 00:16:38 -07:00 committed by GitHub
parent 4976d3c38d
commit 0800af0b8a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 163 additions and 34 deletions

33
cli.py
View file

@ -455,7 +455,9 @@ def load_cli_config() -> Dict[str, Any]:
"resume_max_assistant_chars": 200,
"resume_max_assistant_lines": 3,
"resume_skip_tool_only": True,
"show_reasoning": False,
# Live reasoning display default ON — keep in sync with
# hermes_cli/config.py DEFAULT_CONFIG (display.show_reasoning).
"show_reasoning": True,
"reasoning_full": False,
"streaming": True,
"busy_input_mode": "interrupt",
@ -3708,7 +3710,7 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin):
# bell_on_complete: play terminal bell (\a) when agent finishes a response
self.bell_on_complete = CLI_CONFIG["display"].get("bell_on_complete", False)
# show_reasoning: display model thinking/reasoning before the response
self.show_reasoning = CLI_CONFIG["display"].get("show_reasoning", False)
self.show_reasoning = CLI_CONFIG["display"].get("show_reasoning", True)
# reasoning_full: when reasoning display is on, print the post-response
# recap box uncollapsed instead of clamping to the first 10 lines.
self.reasoning_full = CLI_CONFIG["display"].get("reasoning_full", False)
@ -5836,6 +5838,33 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin):
line = _strip_markdown_syntax(line)
_emit_one(line)
# Force-flush long partial lines so a response that opens with a
# long paragraph paints as tokens arrive instead of staying blank
# until the first newline (TTFT perception fix — the reasoning box
# has done this at 80 chars since day one; the response box never
# did). Wrap at the terminal's visible width so we only ever emit
# text that would have line-broken at that point anyway; the
# remainder stays buffered as the logical line's continuation.
# Table-shaped partials are exempt — they need the whole block for
# realignment (see the table side-buffer above).
if (
self._stream_buf
and not self._in_stream_table
and not self._stream_buf.lstrip().startswith("|")
):
wrap_w = max(40, _terminal_width_for_streaming())
while len(self._stream_buf) >= wrap_w:
cut = self._stream_buf.rfind(" ", 0, wrap_w)
if cut <= 0:
cut = wrap_w # single unbreakable run — hard wrap
chunk, self._stream_buf = (
self._stream_buf[:cut],
self._stream_buf[cut:].lstrip(" "),
)
if self.final_response_markdown == "strip":
chunk = _strip_markdown_syntax(chunk)
_emit_one(chunk)
def _flush_stream(self) -> None:
"""Emit any remaining partial line from the stream buffer and close the box."""
# If we're still inside a "reasoning block" at end-of-stream, it was