mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-07-18 14:52:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7

perf(cli): TTFT round 2 — live reasoning by default, partial-line streaming, prompt-build cache, stale budget-warning docs (#59389)

Browse source 
Follow-up to #59332 targeting the remaining PERCEIVED first-token latency
(the wire streaming was already per-token; these fix what the user sees):

1. display.show_reasoning default ON. On thinking models the reasoning
   phase streams for tens of seconds; with the display off users stare
   at a spinner the whole time and read it as a stall. Flipped in
   DEFAULT_CONFIG, load_cli_config defaults, tui_gateway raw-YAML
   fallbacks, and the hermes setup status line (all four read sites kept
   in sync). Gateway per-platform defaults intentionally stay off —
   messaging chats shouldn't fill with thinking text. /reasoning hide
   still turns it off and persists.

2. Response box force-flushes long partial lines. _emit_stream_text only
   painted on newline, so a response opening with a long paragraph
   stayed invisible until the first \n — seconds of blank box. Now
   partial lines wrap at terminal width and paint as tokens arrive
   (mirrors the reasoning box's 80-char force-flush that existed since
   day one). Table blocks remain batch-aligned; no content loss at wrap
   boundaries (regression tests added).

3. hermes_time timezone resolution uses read_raw_config (mtime-cached +
   libyaml C loader) instead of a raw yaml.safe_load of config.yaml
   (~110-140ms measured) inside the FIRST system prompt build. First
   build drops 320ms -> ~155ms on a 200-skill install.

4. Stale docs: configuration.md (en+zh) still documented the 70%/90%
   [BUDGET WARNING] tool-result injections. Those were removed in April
   2026 (c8aff7463) precisely because they hurt task completion; current
   behavior is exhaustion-message + one grace call, no mid-loop
   injection, no cache impact. Docs now describe reality.

Verified: token-count compression decisions already use API-reported
last_prompt_tokens (rough estimators are preflight-only and cost ~1.7ms
even on 1.7MB histories — not worth touching).
This commit is contained in:
Teknium 2026-07-06 00:16:38 -07:00 • committed by GitHub
parent 4976d3c38d
commit 0800af0b8a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 163 additions and 34 deletions
Download patch file Download diff file Expand all files Collapse all files

15
website/docs/user-guide/configuration.md
View file

@ -823,16 +823,11 @@ Plugin engines are **never auto-activated** — you must explicitly set `context
See [Memory Providers](/user-guide/features/memory-providers) for the analogous single-select system for memory plugins.
## Iteration Budget Pressure
## Iteration Budget
When the agent is working on a complex task with many tool calls, it can burn through its iteration budget (default: 90 turns) without realizing it's running low. Budget pressure automatically warns the model as it approaches the limit:
When the agent is working on a complex task with many tool calls, it can burn through its iteration budget (default: 90 turns). Hermes does **not** inject mid-task pressure warnings — earlier builds warned the model at 70%/90% budget, which caused models to abandon complex tasks prematurely and was removed in April 2026.
| Threshold | Level | What the model sees |
|-----------|-------|---------------------|
| **70%** | Caution | `[BUDGET: 63/90. 27 iterations left. Start consolidating.]` |
| **90%** | Warning | `[BUDGET WARNING: 81/90. Only 9 left. Respond NOW.]` |
Warnings are injected into the last tool result's JSON (as a `_budget_warning` field) rather than as separate messages — this preserves prompt caching and doesn't disrupt the conversation structure.
Instead, when the budget is actually exhausted (90/90), Hermes injects one message asking the model to wrap up and allows a single **grace call** so it can deliver a final response. If that grace call still doesn't produce text, the agent is asked to summarise what it accomplished.
```yaml
agent:
@ -840,9 +835,7 @@ agent:
api_max_retries: 3 # Retries per provider before fallback engages (default: 3)
```
Budget pressure is enabled by default. The agent sees warnings naturally as part of tool results, encouraging it to consolidate its work and deliver a response before running out of iterations.
When the iteration budget is fully exhausted, the CLI shows a notification to the user: `⚠ Iteration budget reached (90/90) — response may be incomplete`. If the budget runs out during active work, the agent generates a summary of what was accomplished before stopping.
When the iteration budget is fully exhausted, the CLI shows a notification to the user: `⚠ Iteration budget reached (90/90) — response may be incomplete`.
`agent.api_max_retries` controls how many times Hermes retries a provider API call on transient errors (rate limits, connection drops, 5xx) **before** fallback-provider switching engages. The default is `3` — four attempts total. If you have [fallback providers](/user-guide/features/fallback-providers) configured and want to fail over faster, drop this to `0` so the first transient error on your primary immediately hands off to the fallback instead of churning retries against the flaky endpoint.