hermes-agent/tests
wysie dbe5d84972 fix(auxiliary): universal main-model fallback for aux tasks (#31845)
Aux callers (title generation, vision, session search, etc.) can reach
resolve_provider_client() without an explicit model when the user
picked their main provider via 'hermes model' and didn't bother
configuring a per-task auxiliary.<task>.model override.  The
expectation in that case is universal: 'use my main model for side
tasks too.'

Before, the OAuth providers (xai-oauth, openai-codex) silently
returned (None, None) on an empty model — both lack a catalog default
because their accepted-model lists drift on the backend.  That caused
_resolve_auto to drop to its Step-2 fallback chain (OpenRouter /
Nous / etc.), so aux tasks billed against the wrong subscription
without warning.

The fix is at the top of resolve_provider_client() — a single
3-step universal fallback that runs before any provider branch, so
no provider-specific empty-model guards are needed (now or for any
future provider we add):

    1. caller-passed model (caller knew what they wanted)
    2. provider's catalog default (cheap aux model, if registered)
    3. user's main model from config.yaml

Behaviour by provider class:

- OAuth providers (xai-oauth, openai-codex) — no catalog default, so
  step 3 applies.  Title gen runs on grok-4.3 / gpt-5.4 against the
  user's actual subscription instead of leaking to OpenRouter.
- API-key providers (anthropic, gemini, kimi-coding, etc.) — catalog
  default wins at step 2, preserving the original 'cheap aux model'
  behaviour.  Anthropic users still get claude-haiku-4-5 for titles,
  not opus.
- Explicit-model callers (auxiliary.<task>.model config, programmatic
  callers) — caller wins at step 1, no surprise switching.

Salvaged from @wysie's PR #31845 which fixed the xai-oauth branch
specifically.  The universal shape supersedes the per-branch fix
and covers openai-codex (same bug class) plus any future OAuth
providers.

4 new tests in TestResolveProviderClientUniversalModelFallback:

- empty_model_for_oauth_provider_falls_back_to_main_model
- empty_model_for_codex_also_uses_main_model
- empty_model_for_catalog_provider_uses_catalog_default
- explicit_model_takes_precedence_over_fallbacks

365/365 across tests/agent/test_auxiliary_*, tests/run_agent/test_codex_xai_oauth_recovery.py, tests/hermes_cli/test_auth_xai_oauth_provider.py, and tests/hermes_cli/test_plugin_auxiliary_tasks.py.

Co-authored-by: wysie <wysie@users.noreply.github.com>
2026-05-25 05:50:56 -07:00
..
acp test(acp): drop flaky runtime_calls[-1] tail-position assertion 2026-05-24 23:23:12 -07:00
acp_adapter feat(azure-foundry): add Microsoft Entra ID auth 2026-05-18 10:14:38 -07:00
agent fix(auxiliary): universal main-model fallback for aux tasks (#31845) 2026-05-25 05:50:56 -07:00
cli feat(cli): show live background terminal-process count in status bar (#32061) 2026-05-25 05:35:02 -07:00
cron fix: add cron API provenance logging (#24889) 2026-05-25 01:15:56 -07:00
docker test(docker): fix svstat 'want up' assertion in profile-gateway lifecycle test 2026-05-25 12:25:06 +10:00
e2e refactor(gateway): migrate Discord adapter to bundled plugin (full Teams parity) 2026-05-22 14:21:41 -07:00
fakes
gateway fix(gateway): allow native delivery of freshly-produced agent files (#32060) 2026-05-25 05:34:31 -07:00
hermes_cli fix(tests): four pre-existing flakes from the security cluster merge (#32072) 2026-05-25 05:50:29 -07:00
hermes_state feat(session_search): single-shape tool with discovery, scroll, browse — no LLM (#27590) 2026-05-17 23:28:45 -07:00
honcho_plugin chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
integration refactor(gateway): migrate Discord adapter to bundled plugin (full Teams parity) 2026-05-22 14:21:41 -07:00
openviking_plugin
plugins feat(stt): add stt.providers.<name> command-provider registry 2026-05-25 01:41:19 -07:00
providers fix(custom): pass custom provider extra body 2026-05-21 07:48:53 -07:00
run_agent fix(streaming): route mid-tool-call partial-stream-stub through length continuation (#31998) (#32012) 2026-05-25 17:43:10 +05:30
scripts feat(acp-registry): switch to uvx distribution, drop npm launcher 2026-05-14 22:27:09 -07:00
skills fix(skills): add timeout to Google OAuth urlopen calls 2026-05-19 00:11:44 -07:00
stress docs: align kanban readiness docs and smoke tests 2026-05-18 21:07:03 -07:00
tools fix(tests): four pre-existing flakes from the security cluster merge (#32072) 2026-05-25 05:50:29 -07:00
tui_gateway chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
website
__init__.py
conftest.py test: keep tirith checks hermetic 2026-05-23 02:20:14 -07:00
run_interrupt_test.py
test_account_usage.py
test_atomic_replace_symlinks.py
test_base_url_hostname.py
test_batch_runner_checkpoint.py
test_bitwarden_secrets.py perf(cli): cut hermes startup 63% — flip head-to-head vs codex (#31968) 2026-05-25 03:06:39 -07:00
test_cli_file_drop.py
test_cli_manual_compress.py fix(tests): catch up six stale tests after compression/aux/kanban changes (#28465) 2026-05-18 21:43:59 -07:00
test_cli_skin_integration.py
test_ctx_halving_fix.py fix(cache): kill long-lived prefix layout — system prompt is now byte-static within a session (#24778) 2026-05-12 20:46:04 -07:00
test_empty_model_fallback.py
test_env_loader_secret_sources.py feat(secrets): label detected credentials with their source (Bitwarden) (#30364) 2026-05-22 03:32:58 -07:00
test_evidence_store.py
test_gateway_streaming_nested_config.py fix(gateway): load streaming config from nested gateway.streaming key 2026-05-14 14:51:07 -07:00
test_get_tool_definitions_cache_isolation.py
test_hermes_bootstrap.py fix(entry-points): guard hermes_bootstrap import so partial updates don't brick hermes (#22091) 2026-05-08 14:43:13 -07:00
test_hermes_constants.py fix(security): guard os.chmod(parent) against / and top-level dirs 2026-05-20 22:56:55 -07:00
test_hermes_home_profile_warning.py
test_hermes_logging.py fix(tests): catch up 25 stale tests after recent merges (#28626) 2026-05-19 01:28:32 -07:00
test_hermes_state.py fix(gateway): separate observed Telegram group context 2026-05-23 01:33:42 -07:00
test_hermes_state_wal_fallback.py fix(sqlite): fall back to journal_mode=DELETE on NFS/SMB/FUSE (#22043) 2026-05-09 02:09:35 -07:00
test_honcho_client_config.py
test_install_sh_browser_install.py fix(install): support non-sudo service-user installs on apt distros (#25814) 2026-05-14 09:05:31 -07:00
test_install_sh_pythonpath_sanitization.py
test_install_sh_setup_wizard_tty_probe.py
test_install_sh_symlink_stomp.py fix(install): preserve pip entry point when re-running on symlinked install 2026-05-14 07:08:45 -07:00
test_install_sh_termux_network_prereqs.py fix: strengthen termux install network prerequisites 2026-05-07 13:04:08 -07:00
test_ipv4_preference.py
test_lazy_session_regressions.py
test_lint_config.py lint: enable PLW1514 as a blocking ruff rule 2026-05-08 14:27:40 -07:00
test_live_system_guard_self_test.py chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
test_mcp_serve.py fix(mcp): unwrap platforms key in channels_list 2026-05-07 13:41:16 -07:00
test_mini_swe_runner.py
test_minimax_model_validation.py
test_minimax_oauth.py fix(minimax-oauth): refresh short-lived access tokens per request (#30619) 2026-05-22 15:16:15 -07:00
test_minisweagent_path.py
test_model_picker_scroll.py
test_model_tools.py chore: remove Atropos RL environments and tinker-atropos integration (#26106) 2026-05-15 10:36:38 +05:30
test_model_tools_async_bridge.py
test_ollama_num_ctx.py
test_package_json_lazy_deps.py fix(update): make Camofox lazy-installed instead of eager (#27055) 2026-05-16 12:15:45 -07:00
test_packaging_metadata.py
test_plugin_skills.py
test_process_loop_event_loop_warning.py fix(cli): replace get_event_loop() with get_running_loop() to silence RuntimeWarning in process_loop thread (#19285) 2026-05-07 06:35:54 -07:00
test_project_metadata.py fix(packaging): ship dashboard plugin assets in wheel 2026-05-18 20:35:00 -07:00
test_retry_utils.py
test_run_tests_parallel.py test: use subprocesses for each test file (#29016) 2026-05-21 16:40:04 +05:30
test_sanitize_tool_error.py security: sanitize tool error strings before injecting into model context (#26823) 2026-05-16 00:57:39 -07:00
test_sql_injection.py
test_subprocess_home_isolation.py fix: avoid process-wide cron profile home mutation 2026-05-18 17:39:50 +00:00
test_termux_all_extra_compat.py fix: add termux-all install profile and safe fallbacks 2026-05-07 13:04:08 -07:00
test_timezone.py chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
test_toolset_distributions.py
test_toolsets.py test(toolsets): lock web search into default platform coverage 2026-05-14 08:03:33 -07:00
test_trajectory_compressor.py
test_trajectory_compressor_async.py
test_transform_llm_output_hook.py
test_transform_tool_result_hook.py
test_tui_gateway_server.py fix(tui): stop slash dropdown from chopping last char of /goal (#31311) 2026-05-23 22:12:55 -07:00
test_utils_truthy_values.py
test_yuanbao_integration.py
test_yuanbao_markdown.py
test_yuanbao_pipeline.py
test_yuanbao_proto.py