Commit graph

6 commits

Author SHA1 Message Date
Teknium
0800af0b8a
perf(cli): TTFT round 2 — live reasoning by default, partial-line streaming, prompt-build cache, stale budget-warning docs (#59389)
Follow-up to #59332 targeting the remaining PERCEIVED first-token latency
(the wire streaming was already per-token; these fix what the user sees):

1. display.show_reasoning default ON. On thinking models the reasoning
   phase streams for tens of seconds; with the display off users stare
   at a spinner the whole time and read it as a stall. Flipped in
   DEFAULT_CONFIG, load_cli_config defaults, tui_gateway raw-YAML
   fallbacks, and the hermes setup status line (all four read sites kept
   in sync). Gateway per-platform defaults intentionally stay off —
   messaging chats shouldn't fill with thinking text. /reasoning hide
   still turns it off and persists.

2. Response box force-flushes long partial lines. _emit_stream_text only
   painted on newline, so a response opening with a long paragraph
   stayed invisible until the first \n — seconds of blank box. Now
   partial lines wrap at terminal width and paint as tokens arrive
   (mirrors the reasoning box's 80-char force-flush that existed since
   day one). Table blocks remain batch-aligned; no content loss at wrap
   boundaries (regression tests added).

3. hermes_time timezone resolution uses read_raw_config (mtime-cached +
   libyaml C loader) instead of a raw yaml.safe_load of config.yaml
   (~110-140ms measured) inside the FIRST system prompt build. First
   build drops 320ms -> ~155ms on a 200-skill install.

4. Stale docs: configuration.md (en+zh) still documented the 70%/90%
   [BUDGET WARNING] tool-result injections. Those were removed in April
   2026 (c8aff7463) precisely because they hurt task completion; current
   behavior is exhaustion-message + one grace call, no mid-loop
   injection, no cache impact. Docs now describe reality.

Verified: token-count compression decisions already use API-reported
last_prompt_tokens (rough estimators are preflight-only and cost ~1.7ms
even on 1.7MB histories — not worth touching).
2026-07-06 00:16:38 -07:00
kshitij
5fba236644
chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355)
Six days after #23937 (608 fixes) the codebase had accumulated 241 new
PLR6201 violations. Same mechanical `x in (...)` → `x in {...}` fix,
same zero-risk profile: set lookup is O(1) vs O(n) for tuple and the
two are semantically equivalent for hashable scalar membership tests.

All 241 instances fixed via `ruff check --select PLR6201 --fix
--unsafe-fixes`, zero remaining. Every changed value is a hashable
scalar (str/int/None/enum/signal); no risk of unhashable runtime
errors. No behavior change.

Test plan:
- 119 files changed, +244/-244 (net zero) — exactly one-line edits
- `ruff check` clean afterward
- Compile checks pass on the largest touched files (cli.py, run_agent.py,
  gateway/run.py, gateway/platforms/discord.py, model_tools.py)
- Subset broad test run on tests/gateway/ tests/hermes_cli/ tests/agent/
  tests/tools/: 18187 passed, 59 pre-existing failures (verified against
  origin/main with the same shape — identical failure count, identical
  category — all xdist test-order flakes unrelated to this change)

Follows the same template as PR #23937 ([tracker: #23972](https://github.com/NousResearch/hermes-agent/issues/23972)).
2026-05-17 02:29:41 -07:00
Teknium
9e0ef2a1bc test: pin per-turn reasoning extraction semantics
Covers four scenarios for the reasoning-box extraction loop:
 - simple turn with reasoning
 - simple turn with no reasoning
 - tool-calling turn where reasoning lives on the tool-call step
 - prior turn had reasoning, current turn does not (the stale-display
   bug the fix exists for)
 - tool-calling turn where reasoning lives on BOTH steps (latest wins)
 - empty-string reasoning treated as missing

Also updates the four inline replica loops in tests/cli/test_reasoning_command.py
to match the new turn-boundary shape so the test file reflects
production semantics.
2026-05-05 05:00:05 -07:00
Teknium
598cba62ad
test: update stale tests to match current code (#11963)
Seven test files were asserting against older function signatures and
behaviors. CI has been red on main because of accumulated test debt
from other PRs; this catches the tests up.

- tests/agent/test_subagent_progress.py: _build_child_progress_callback
  now takes (task_index, goal, parent_agent, task_count=1); update all
  call sites and rewrite tests that assumed the old 'batch-only' relay
  semantics (now relays per-tool AND flushes a summary at BATCH_SIZE).
  Renamed test_thinking_not_relayed_to_gateway → test_thinking_relayed_to_gateway
  since thinking IS now relayed as subagent.thinking.
- tests/tools/test_delegate.py: _build_child_agent now requires
  task_count; add task_count=1 to all 8 call sites.
- tests/cli/test_reasoning_command.py: AIAgent gained _stream_callback;
  stub it on the two test agent helpers that use spec=AIAgent / __new__.
- tests/hermes_cli/test_cmd_update.py: cmd_update now runs npm install
  in repo root + ui-tui/ + web/ and 'npm run build' in web/; assert
  all four subprocess calls in the expected order.
- tests/hermes_cli/test_model_validation.py: dissimilar unknown models
  now return accepted=False (previously True with warning); update
  both affected tests.
- tests/tools/test_registry.py: include feishu_doc_tool and
  feishu_drive_tool in the expected builtin tool set.
- tests/gateway/test_voice_command.py: missing-voice-deps message now
  suggests 'pip install PyNaCl' not 'hermes-agent[messaging]'.

411/411 pass locally across these 7 files.
2026-04-17 21:35:30 -07:00
alt-glitch
c6c769772f fix: clean up stale test references to removed attributes 2026-04-10 03:44:43 -07:00
Siddharth Balyan
f3006ebef9
refactor(tests): re-architect tests + fix CI failures (#5946)
* refactor: re-architect tests to mirror the codebase

* Update tests.yml

* fix: add missing tool_error imports after registry refactor

* fix(tests): replace patch.dict with monkeypatch to prevent env var leaks under xdist

patch.dict(os.environ) can leak TERMINAL_ENV across xdist workers,
causing test_code_execution tests to hit the Modal remote path.

* fix(tests): fix update_check and telegram xdist failures

- test_update_check: replace patch("hermes_cli.banner.os.getenv") with
  monkeypatch.setenv("HERMES_HOME") — banner.py no longer imports os
  directly, it uses get_hermes_home() from hermes_constants.

- test_telegram_conflict/approval_buttons: provide real exception classes
  for telegram.error mock (NetworkError, TimedOut, BadRequest) so the
  except clause in connect() doesn't fail with "catching classes that do
  not inherit from BaseException" when xdist pollutes sys.modules.

* fix(tests): accept unavailable_models kwarg in _prompt_model_selection mock
2026-04-07 17:19:07 -07:00
Renamed from tests/test_reasoning_command.py (Browse further)