hermes-agent/tests/agent
Teknium 31f70d1f2a
fix(ci): recover 38 failing tests on main (#17642)
CI Tests workflow has been red on main for 40+ consecutive runs. This
commit recovers every failure visible in run 25130722163 (most recent
completed run prior to this PR).

Root causes, by group:

Test-mock drift after product landed (fix: update mocks)
- test_mcp_structured_content / test_mcp_dynamic_discovery (6 tests):
  product added _rpc_lock (#02ae15222) and _schedule_tools_refresh
  (#1350d12b0) without updating sibling test files. Install a real
  asyncio.Lock inside the fake run-loop and patch at _schedule_tools_refresh.
- test_session.py: renamed normalize_whatsapp_identifier → canonical_
  whatsapp_identifier upstream; keep a local alias so the legacy tests
  keep working.
- test_run_progress_topics Slack DM test: PR #8006 made Slack default
  tool_progress=off; explicitly set it to 'all' in the test fixture so
  the progress-callback path still runs. Also read tool_progress_callback
  at call time rather than freezing it in FakeAgent.__init__ — production
  assigns it AFTER construction.
- test_tui_gateway_server session-create/close race: session.create now
  defers _start_agent_build behind a 50ms timer — wait for the build
  thread to enter _make_agent before closing, otherwise the orphan-
  cleanup path never runs.
- test_protocol session.resume: product get_messages_as_conversation now
  takes include_ancestors kwarg; accept **_kwargs in the test stub.
- test_copilot_acp_client redaction: redactor is OFF by default (snapshots
  HERMES_REDACT_SECRETS at import); patch agent.redact._REDACT_ENABLED=True
  for the duration of the test.
- test_minimax_provider: after #17171, dots in non-Anthropic model names
  stay dots even with preserve_dots=False. Assert the new invariant
  rather than the old 'broken for MiniMax' behavior.
- test_update_autostash: updater now scans `ps -A` for dashboard PIDs;
  the test's catch-all subprocess.run stub needed stdout/stderr fields.
- test_accretion_caps: read_timestamps dict is populated lazily when
  os.path.getmtime succeeds. Use .get("read_timestamps", {}) to tolerate
  CI filesystems where the stat races file creation.

Change-detector tests (fix: rewrite as structural invariants)
- test_credential_sources_registry_has_expected_steps: was a frozen set
  comparison that broke when minimax-oauth was added. Rewrite as an
  invariant check (every step has description, no dupes, core steps
  present) per AGENTS.md 'don't write change-detector tests'.

xdist ordering / test pollution (fix: reset state, use module-local patches)
- test_setup vercel: sibling test saved VERCEL_PROJECT_ID='project' to
  os.environ via save_env_value() and never cleared it. monkeypatch.delenv
  the VERCEL_* vars in the link-file test.
- test_clipboard TestIsWsl: GitHub Actions is on Azure VMs whose real
  /proc/version often contains 'microsoft'. Patching builtins.open with
  mock_open didn't reliably intercept hermes_constants.is_wsl's call in
  xdist workers that had already cached _wsl_detected=True from an
  earlier test. Patch hermes_constants.open directly and add
  teardown_method to reset the cache after each test.

Pytest-asyncio cancellation hangs (fix: bound product await with timeout)
- test_session_split_brain_11016 (3 params) + test_gateway_shutdown
  cancel-inflight: under pytest-asyncio 1.3.0, 'await task' and
  'asyncio.gather(cancelled_tasks)' can stall for 30s when the cancelled
  task's finally block awaits typing-task cleanup. Bound both with
  asyncio.wait_for(..., timeout=5.0) and asyncio.shield — the stragglers
  are released from adapter tracking and allowed to finish unwinding in
  the background. This is also a legitimate hardening: a wedged finally
  shouldn't stall the caller's dispatch or a gateway shutdown.

Orphan UI config (fix: merge tiny tab into messaging category)
- test_web_server test_no_single_field_categories: the telegram.reactions
  config field lived in its own 'telegram' schema category with no
  siblings. Fold it under 'discord' via _CATEGORY_MERGE so the dashboard
  doesn't render an orphan single-field tab.

Local verification: 38/38 originally-failing tests pass; 4044/4044
gateway tests pass; 684/684 targeted subset (all 16 touched test files)
passes.
2026-04-29 20:05:32 -07:00
..
transports fix(gemini): nest OpenAI-compat thinking config under google 2026-04-29 12:10:40 -07:00
__init__.py test: add unit tests for 8 modules (batch 2) 2026-02-26 13:54:20 +03:00
test_anthropic_adapter.py Revert "fix(anthropic): remove Claude Code fingerprinting from OAuth Messages API path (#16957)" (#17397) 2026-04-29 03:55:03 -07:00
test_anthropic_keychain.py fix: re-auth on stale OAuth token; read Claude Code credentials from macOS Keychain 2026-04-24 07:14:00 -07:00
test_auxiliary_client.py fix(aux): skip kimi-coding in vision auto-detect (closes #17076) (#17451) 2026-04-29 06:10:23 -07:00
test_auxiliary_client_anthropic_custom.py fix(anthropic): complete third-party Anthropic-compatible provider support (#12846) 2026-04-19 22:43:09 -07:00
test_auxiliary_config_bridge.py fix: remove legacy compression.summary_* config and env var fallbacks (#8992) 2026-04-13 04:59:26 -07:00
test_auxiliary_main_first.py fix(copilot): send vision header for Copilot vision requests 2026-04-27 08:35:50 -07:00
test_auxiliary_named_custom_providers.py refactor(memory): remove flush_memories entirely (#15696) 2026-04-25 08:21:14 -07:00
test_auxiliary_transport_autodetect.py fix(auxiliary): auto-detect Anthropic Messages transport for all aux clients (#17027) 2026-04-28 06:50:14 -07:00
test_bedrock_1m_context.py fix(bedrock): send context-1m-2025-08-07 beta so Opus 4.6/4.7 get 1M context (#16793) 2026-04-27 20:41:36 -07:00
test_bedrock_adapter.py test(bedrock): add model picker and region routing tests 2026-04-28 03:53:11 -07:00
test_bedrock_integration.py fix(agent): handle aws_sdk auth type in resolve_provider_client 2026-04-24 07:26:07 -07:00
test_codex_cloudflare_headers.py fix(codex): pin correct Cloudflare headers and extend to auxiliary client 2026-04-19 11:59:25 -07:00
test_compress_focus.py fix: resolve CI test failures — add missing functions, fix stale tests (#9483) 2026-04-14 01:43:45 -07:00
test_compressor_image_tokens.py feat(image-input): native multimodal routing based on model vision capability (#16506) 2026-04-27 06:27:59 -07:00
test_context_compressor.py fix(compression): notify users when configured aux model fails even if main-model fallback recovers (#16775) 2026-04-27 20:08:23 -07:00
test_context_engine.py feat: wire context engine plugin slot into agent and plugin system 2026-04-10 19:15:50 -07:00
test_context_references.py fix(agent): fall back when rg is blocked for @folder references 2026-04-20 01:56:41 -07:00
test_copilot_acp_client.py fix(ci): recover 38 failing tests on main (#17642) 2026-04-29 20:05:32 -07:00
test_credential_pool.py fix(codex): resync pool entry from auth.json after reauth (#17001) 2026-04-28 05:43:09 -07:00
test_credential_pool_routing.py refactor: remove smart_model_routing feature (#12732) 2026-04-19 18:12:55 -07:00
test_crossloop_client_cache.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_curator.py feat(curator): per-run reports — run.json + REPORT.md under logs/curator/ (#17307) 2026-04-28 23:23:11 -07:00
test_curator_reports.py feat(curator): per-run reports — run.json + REPORT.md under logs/curator/ (#17307) 2026-04-28 23:23:11 -07:00
test_deepseek_anthropic_thinking.py test(anthropic): regression guard for DeepSeek /anthropic thinking replay 2026-04-29 08:10:29 -07:00
test_direct_provider_url_detection.py fix: restrict provider URL detection to exact hostname matches 2026-04-20 22:14:29 -07:00
test_display.py fix(display): render <missing old_text> in memory previews instead of empty quotes (#12852) 2026-04-19 22:45:47 -07:00
test_display_emoji.py feat(tools): centralize tool emoji metadata in registry + skin integration 2026-03-15 20:21:21 -07:00
test_error_classifier.py feat(image-input): native multimodal routing based on model vision capability (#16506) 2026-04-27 06:27:59 -07:00
test_external_skills.py feat(skills): support external skill directories via config (#3678) 2026-03-29 00:33:30 -07:00
test_gemini_cloudcode.py fix(gemini): assign unique stream indices to parallel tool calls 2026-04-20 02:10:53 -07:00
test_gemini_free_tier_gate.py feat(gemini): block free-tier keys at setup + surface guidance on 429 (#15100) 2026-04-24 04:46:17 -07:00
test_gemini_native_adapter.py fix(gemini): fail fast on missing API key + surface it in hermes dump (#15133) 2026-04-24 05:35:17 -07:00
test_gemini_schema.py fix(gemini): drop integer/number/boolean enums from tool schemas (#15082) 2026-04-24 03:40:00 -07:00
test_image_gen_registry.py feat(plugins): pluggable image_gen backends + OpenAI provider (#13799) 2026-04-21 21:30:10 -07:00
test_image_routing.py feat(image-input): native multimodal routing based on model vision capability (#16506) 2026-04-27 06:27:59 -07:00
test_insights.py test: stop testing mutable data — convert change-detectors to invariants (#13363) 2026-04-20 23:20:33 -07:00
test_kimi_coding_anthropic_thinking.py fix(anthropic): broaden Kimi thinking-suppression to custom endpoints (#17455) 2026-04-29 06:35:42 -07:00
test_local_stream_timeout.py fix(agent): recognize Tailscale CGNAT (100.64.0.0/10) as local for Ollama timeouts 2026-04-22 14:46:10 -07:00
test_memory_provider.py fix(memory): add write origin metadata 2026-04-24 14:37:55 -07:00
test_memory_session_switch.py fix(hindsight): route flush-on-switch through writer queue, not raw thread 2026-04-29 08:09:03 -07:00
test_memory_user_id.py feat(hindsight): richer session-scoped retain metadata 2026-04-22 05:27:10 -07:00
test_minimax_auxiliary_url.py fix: provider/model resolution — salvage 4 PRs + MiniMax aux URL fix (#5983) 2026-04-07 22:23:28 -07:00
test_minimax_provider.py fix(ci): recover 38 failing tests on main (#17642) 2026-04-29 20:05:32 -07:00
test_model_metadata.py revert: computer-use cua-driver (PR #16919) (#16927) 2026-04-28 01:57:21 -07:00
test_model_metadata_local_ctx.py fix(tui): show correct context length 2026-04-28 12:27:36 -07:00
test_model_metadata_ssl.py fix(auth): honor SSL CA env vars across httpx + requests callsites 2026-04-24 03:00:33 -07:00
test_models_dev.py feat: add Step Plan provider support (salvage #6005) 2026-04-22 02:59:58 -07:00
test_moonshot_schema.py fix(kimi,mcp): Moonshot schema sanitizer + MCP schema robustness (#14805) 2026-04-23 16:11:57 -07:00
test_nous_rate_guard.py fix(nous): don't trip cross-session rate breaker on upstream-capacity 429s (#15898) 2026-04-26 04:53:42 -07:00
test_onboarding.py docs(onboarding): lead OpenClaw residue banner with migrate, warn that cleanup breaks OpenClaw (#17507) 2026-04-29 08:08:36 -07:00
test_prompt_builder.py feat(agent): add PLATFORM_HINTS for matrix, mattermost, and feishu (#14428) 2026-04-23 12:50:22 +05:30
test_prompt_caching.py fix(prompt-caching): skip top-level cache_control on role:tool for OpenRouter 2026-03-21 16:54:43 -07:00
test_proxy_and_url_validation.py fix(agent): normalize socks:// env proxies for httpx/anthropic 2026-04-21 05:52:46 -07:00
test_rate_limit_tracker.py feat: capture provider rate limit headers and show in /usage (#6541) 2026-04-09 03:43:14 -07:00
test_redact.py feat: replace kimi-k2.5 with kimi-k2.6 on OpenRouter and Nous Portal (#13148) 2026-04-20 11:49:54 -07:00
test_shell_hooks.py feat: shell hooks — wire shell scripts as Hermes hook callbacks 2026-04-20 20:53:51 -07:00
test_shell_hooks_consent.py fix(shell_hooks): parse hooks_auto_accept as strict bool/string, not bool() (#16322) 2026-04-26 20:48:35 -07:00
test_skill_commands.py refactor(commands): drop /provider, /plan handler, and clean up slash registry (#15047) 2026-04-24 03:10:52 -07:00
test_streaming_context_scrubber.py style: trim verbose comment blocks added by previous commit 2026-04-27 12:37:33 -07:00
test_subagent_progress.py feat(delegate): orchestrator role and configurable spawn depth (default flat) 2026-04-21 14:23:45 -07:00
test_subagent_stop_hook.py feat: shell hooks — wire shell scripts as Hermes hook callbacks 2026-04-20 20:53:51 -07:00
test_subdirectory_hints.py fix(agent): catch PermissionError in subdirectory hint discovery 2026-04-09 03:10:30 -07:00
test_title_generator.py fix(auxiliary): custom provider URL rewrite + main_runtime model for title gen 2026-04-28 01:47:25 -07:00
test_unsupported_parameter_retry.py fix(auxiliary): generalize unsupported-parameter detector and harden max_tokens retry (#15633) 2026-04-25 05:50:34 -07:00
test_unsupported_temperature_retry.py refactor(memory): remove flush_memories entirely (#15696) 2026-04-25 08:21:14 -07:00
test_usage_pricing.py fix(usage): read top-level Anthropic cache fields from OAI-compatible proxies 2026-04-22 17:40:49 -07:00
test_vision_resolved_args.py fix: pass resolved args to resolve_vision_provider_client() 2026-04-16 07:45:13 -07:00