hermes-agent/tests
Teknium da02a4e283
fix: auxiliary client payment fallback — retry with next provider on 402 (#5599)
When a user runs out of OpenRouter credits and switches to Codex (or any
other provider), auxiliary tasks (compression, vision, web_extract) would
still try OpenRouter first and fail with 402.  Two fixes:

1. Payment fallback in call_llm(): When a resolved provider returns HTTP 402
   or a credit-related error, automatically retry with the next available
   provider in the auto-detection chain.  Skips the depleted provider and
   tries Nous → Custom → Codex → API-key providers.

2. Remove hardcoded OpenRouter fallback: The old code fell back specifically
   to OpenRouter when auto/custom resolution returned no client.  Now falls
   back to the full auto-detection chain, which handles any available
   provider — not just OpenRouter.

Also extracts _get_provider_chain() as a shared function (replaces inline
tuple in _resolve_auto and the new fallback), built at call time so test
patches on _try_* functions remain visible.

Adds 16 tests covering _is_payment_error(), _get_provider_chain(),
_try_payment_fallback(), and call_llm() integration with 402 retry.
2026-04-06 12:41:40 -07:00
..
acp feat(api): structured run events via /v1/runs SSE endpoint 2026-04-05 12:05:13 -07:00
agent fix: auxiliary client payment fallback — retry with next provider on 402 (#5599) 2026-04-06 12:41:40 -07:00
cron fix: extract MEDIA: tags from cron delivery before sending (#5598) 2026-04-06 11:42:44 -07:00
e2e test(e2e): remove section separator comments 2026-04-01 15:23:52 -07:00
fakes
gateway fix(signal): implement send_image_file, send_voice, and send_video for MEDIA: tag delivery 2026-04-06 11:41:34 -07:00
hermes_cli feat(discord): register skills as native slash commands via shared gateway logic (#5603) 2026-04-06 12:09:36 -07:00
honcho_plugin fix(honcho): migration guard for observation mode default change 2026-04-05 12:34:11 -07:00
integration refactor: remove mini-swe-agent dependency — inline Docker/Modal backends (#2804) 2026-03-24 07:30:25 -07:00
plugins fix: follow-up improvements for salvaged PR #5456 2026-04-06 02:00:55 -07:00
skills fix: protect profile-scoped google workspace oauth tokens 2026-04-03 17:49:18 -07:00
tools feat: implement MCP OAuth 2.1 PKCE client support (#5420) 2026-04-05 22:08:00 -07:00
__init__.py
conftest.py
run_interrupt_test.py
test_413_compression.py fix: remove stale test skips, fix regex backtracking, file search bug, and test flakiness 2026-04-04 10:18:57 -07:00
test_860_dedup.py
test_1630_context_overflow_loop.py
test_agent_guardrails.py
test_agent_loop.py
test_agent_loop_tool_calling.py fix: remove stale test skips, fix regex backtracking, file search bug, and test flakiness 2026-04-04 10:18:57 -07:00
test_agent_loop_vllm.py
test_anthropic_adapter.py fix: preserve Anthropic thinking block signatures across tool-use turns 2026-04-02 10:30:32 -07:00
test_anthropic_error_handling.py fix(ci): pin acp <0.9 and update retry-exhaust test (#3320) 2026-03-26 19:21:34 -07:00
test_anthropic_oauth_flow.py
test_anthropic_provider_persistence.py
test_api_key_providers.py fix: remove hardcoded OpenRouter/opus defaults 2026-04-01 15:22:26 -07:00
test_async_httpx_del_neuter.py fix: eliminate 'Event loop is closed' / 'Press ENTER to continue' during idle (#3398) 2026-03-27 09:45:25 -07:00
test_atomic_json_write.py
test_atomic_yaml_write.py
test_auth_codex_provider.py
test_auth_commands.py fix: hermes auth remove now clears env-seeded credentials permanently (#5285) 2026-04-05 12:00:53 -07:00
test_auth_nous_provider.py
test_auxiliary_config_bridge.py
test_batch_runner_checkpoint.py
test_branch_command.py fix: clear ghost status-bar lines on terminal resize (#4960) 2026-04-03 22:43:45 -07:00
test_cli_approval_ui.py
test_cli_background_tui_refresh.py fix(cli): refresh TUI before background task output to prevent status bar overlap (#3048) 2026-03-25 15:00:33 -07:00
test_cli_context_warning.py fix: add missing provider attrs to cli_obj test fixture 2026-04-01 01:12:23 -07:00
test_cli_extension_hooks.py refactor(cli): add protected TUI extension hooks for wrapper CLIs 2026-03-21 09:42:07 -07:00
test_cli_file_drop.py refactor: extract _detect_file_drop() + add 28 tests 2026-04-02 00:40:27 -07:00
test_cli_init.py fix(cli): surface recent sessions inside /history and /resume 2026-04-03 00:50:49 -07:00
test_cli_interrupt_subagent.py
test_cli_loading_indicator.py
test_cli_mcp_config_watch.py
test_cli_new_session.py fix: complete session reset — missing compressor counters + test 2026-03-20 04:35:17 -07:00
test_cli_plan_command.py
test_cli_prefix_matching.py
test_cli_preloaded_skills.py fix: move activated skills line below welcome text 2026-03-23 06:20:19 -07:00
test_cli_provider_resolution.py feat: show model pricing for OpenRouter and Nous Portal providers 2026-04-05 22:02:21 -07:00
test_cli_retry.py
test_cli_save_config_value.py fix(cli): use atomic write in save_config_value to prevent config loss on interrupt 2026-03-31 12:21:55 -07:00
test_cli_secret_capture.py
test_cli_skin_integration.py
test_cli_status_bar.py fix(cli): prevent status bar wrapping into duplicate rows (#3883) 2026-03-29 23:59:07 -07:00
test_cli_tools_command.py fix: resolve 7 failing CI tests (#3936) 2026-03-30 08:10:14 -07:00
test_codex_execution_paths.py fix(tests): provide model name in Codex 401 refresh tests for CI (#4166) 2026-03-30 21:17:09 -07:00
test_codex_models.py fix: repair OpenCode model routing and selection (#4508) 2026-04-02 09:36:24 -07:00
test_compression_boundary.py fix(agent): prevent silent tool result loss during context compression (#1993) 2026-03-18 15:22:51 -07:00
test_compression_persistence.py fix: persist compressed context to gateway session after mid-run compression 2026-03-30 18:49:14 -07:00
test_compressor_fallback_update.py feat(providers): add ordered fallback provider chain (salvage #1761) (#3813) 2026-03-29 16:04:53 -07:00
test_config_env_expansion.py feat(config): support ${ENV_VAR} substitution in config.yaml (#2684) 2026-03-23 16:02:06 -07:00
test_context_pressure.py fix: cap context pressure percentage at 100% in display (#3480) 2026-03-27 21:42:09 -07:00
test_context_references.py fix(context): restrict @ references to safe workspace paths (#2601) 2026-03-23 06:40:05 -07:00
test_context_token_tracking.py fix(tests): resolve all consistently failing tests 2026-03-22 05:58:26 -07:00
test_credential_pool.py Honor provider reset windows in pooled credential failover 2026-04-05 00:20:53 -07:00
test_credential_pool_routing.py Honor provider reset windows in pooled credential failover 2026-04-05 00:20:53 -07:00
test_crossloop_client_cache.py fix(agent): prevent AsyncOpenAI/httpx cross-loop deadlock in gateway mode (#2701) 2026-03-25 17:31:56 -07:00
test_dict_tool_call_args.py
test_display.py feat: add inline diff previews for write actions 2026-04-01 02:13:57 -07:00
test_evidence_store.py
test_exit_cleanup_interrupt.py feat(memory): pluggable memory provider interface with profile isolation, review fixes, and honcho CLI restoration (#4623) 2026-04-02 15:33:51 -07:00
test_external_credential_detection.py
test_fallback_model.py
test_file_permissions.py
test_flush_memories_codex.py
test_gemini_provider.py fix: update Gemini model catalog + wire models.dev as live model source 2026-04-06 10:28:03 -07:00
test_hermes_logging.py feat: centralized logging, instrumentation, hermes logs CLI, gateway noise fix (#5430) 2026-04-06 00:08:20 -07:00
test_hermes_state.py fix: merge dotted+hyphenated FTS5 quoting into single pass 2026-04-02 00:49:11 -07:00
test_honcho_client_config.py feat(memory): pluggable memory provider interface with profile isolation, review fixes, and honcho CLI restoration (#4623) 2026-04-02 15:33:51 -07:00
test_insights.py
test_interactive_interrupt.py
test_interrupt_propagation.py
test_large_tool_result.py feat: save oversized tool results to file instead of destructive truncation (#5210) 2026-04-05 10:29:57 -07:00
test_long_context_tier_429.py fix: handle Anthropic Sonnet long-context tier 429 by reducing to 200k (#4747) 2026-04-03 02:05:02 -07:00
test_managed_server_tool_support.py
test_mcp_serve.py feat: add MCP server mode — hermes mcp serve (#3795) 2026-03-29 15:47:19 -07:00
test_minisweagent_path.py chore: remove all remaining mini-swe-agent references 2026-03-24 08:19:23 -07:00
test_model_metadata_local_ctx.py fix: prefer loaded instance context size over max for LM Studio 2026-03-19 21:24:53 +01:00
test_model_normalize.py Fix #5211: Preserve dots in OpenCode Go model names 2026-04-06 11:25:06 -07:00
test_model_provider_persistence.py fix: repair OpenCode model routing and selection (#4508) 2026-04-02 09:36:24 -07:00
test_model_tools.py Add request-scoped plugin lifecycle hooks 2026-04-05 23:31:29 -07:00
test_model_tools_async_bridge.py fix: use per-thread persistent event loops in worker threads 2026-03-20 15:41:06 -04:00
test_ollama_cloud_auth.py fix: Ollama Cloud auth, /model switch persistence, and alias tab completion 2026-04-05 11:06:06 -07:00
test_openai_client_lifecycle.py
test_packaging_metadata.py chore: prepare Hermes for Homebrew packaging (#4099) 2026-03-30 17:34:43 -07:00
test_percentage_clamp.py fix: cap percentage displays at 100% in stats, gateway, and memory tool (#3599) 2026-03-28 14:55:18 -07:00
test_personality_none.py
test_plugin_cli_registration.py fix(plugins): only register CLI commands for the active memory provider 2026-04-05 12:34:11 -07:00
test_plugins.py feat(plugins): pre_api_request/post_api_request with narrow payloads 2026-04-05 23:31:29 -07:00
test_plugins_cmd.py fix(plugins): reject plugin names that resolve to the plugins root 2026-04-05 18:40:45 -07:00
test_primary_runtime_restore.py feat: per-turn primary runtime restoration and transport recovery (#4624) 2026-04-02 10:52:01 -07:00
test_project_metadata.py fix: exclude matrix from [all] extras — python-olm is upstream-broken (#4615) 2026-04-02 09:21:37 -07:00
test_provider_fallback.py feat(providers): add ordered fallback provider chain (salvage #1761) (#3813) 2026-03-29 16:04:53 -07:00
test_provider_parity.py test: add test for _should_sanitize_tool_calls() 2026-04-05 00:13:25 -07:00
test_quick_commands.py
test_real_interrupt_subagent.py
test_reasoning_command.py fix: prevent reasoning box from rendering 3x during tool-calling loops (#3405) 2026-03-27 09:57:50 -07:00
test_redirect_stdout_issue.py
test_resume_display.py
test_run_agent.py feat(plugins): pre_api_request/post_api_request with narrow payloads 2026-04-05 23:31:29 -07:00
test_run_agent_codex_responses.py fix(codex): handle reasoning-only responses and replay path (#2070) 2026-03-19 10:34:44 -07:00
test_runtime_provider_resolution.py fix(opencode-go): strip trailing /v1 from base URL for Anthropic models (#4918) 2026-04-03 18:47:51 -07:00
test_session_meta_filtering.py fix: filter transcript-only roles from chat-completions payload (#4715) 2026-04-03 14:57:33 -07:00
test_session_reset_fix.py fix(session): clear compressor summary and turn counter on /clear and /new (#3102) 2026-03-25 18:22:21 -07:00
test_setup_model_selection.py fix: repair OpenCode model routing and selection (#4508) 2026-04-02 09:36:24 -07:00
test_sql_injection.py fix(security): eliminate SQL string formatting in execute() calls 2026-03-19 15:16:35 +01:00
test_streaming.py test: add codex transport drop regression 2026-03-31 12:05:06 -07:00
test_strict_api_validation.py test: add strict API validation tests for Fireworks compatibility 2026-04-05 00:13:25 -07:00
test_surrogate_sanitization.py fix: sanitize surrogate characters from clipboard paste to prevent UnicodeEncodeError (#3624) 2026-03-28 16:53:14 -07:00
test_timezone.py
test_token_persistence_non_cli.py fix(insights): persist token usage for non-CLI sessions 2026-04-02 10:47:13 -07:00
test_tool_arg_coercion.py feat: coerce tool call arguments to match JSON Schema types (#5265) 2026-04-05 10:57:34 -07:00
test_tool_call_parsers.py fix(mistral-parser): handle nested JSON in fallback extraction 2026-03-21 09:41:17 -07:00
test_toolset_distributions.py
test_toolsets.py
test_trajectory_compressor.py fix: URL-based auth for third-party Anthropic endpoints + CI test fixes (#4148) 2026-03-30 20:36:56 -07:00
test_trajectory_compressor_async.py fix: create AsyncOpenAI lazily in trajectory_compressor to avoid closed event loop (#4013) 2026-03-30 13:16:16 -07:00
test_utils_truthy_values.py Gate tool-gateway behind an env var, so it's not in users' faces until we're ready. Even if users enable it, it'll be blocked server-side for now, until we unlock for non-admin users on tool-gateway. 2026-03-30 13:28:10 +09:00
test_worktree.py
test_worktree_security.py