hermes-agent/tests/agent
Teknium 1a3ae6ac6e
feat: structured API error classification for smart failover (#6514)
Add agent/error_classifier.py with a priority-ordered classification
pipeline that replaces scattered inline string-matching in the retry
loop with structured error taxonomy and recovery hints.

FailoverReason enum (14 categories): auth, auth_permanent, billing,
rate_limit, overloaded, server_error, timeout, context_overflow,
payload_too_large, model_not_found, format_error, thinking_signature,
long_context_tier, unknown.

ClassifiedError dataclass carries reason + recovery action hints
(retryable, should_compress, should_rotate_credential, should_fallback).

Key improvements over inline matching:
- 402 disambiguation: 'insufficient credits' = billing (immediate rotate),
  'usage limit, try again' = rate_limit (backoff first)
- OpenRouter 403 'key limit exceeded' correctly classified as billing
- Error cause chain walking (walks __cause__/__context__ up to 5 levels)
- Body message included in pattern matching (SDK str() misses it)
- Server disconnect + large session check ordered before generic transport
  catch so RemoteProtocolError triggers compression when appropriate
- Chinese error message support for context overflow

run_agent.py: replaced 6 inline detection blocks with classifier calls,
net -55 lines. All recovery actions (pool rotation, fallback activation,
compression, transport recovery) unchanged.

65 new unit tests + 10 E2E tests + live tests with real SDK error objects.
Inspired by OpenClaw's failover error classification system.
2026-04-09 04:10:11 -07:00
..
__init__.py test: add unit tests for 8 modules (batch 2) 2026-02-26 13:54:20 +03:00
test_anthropic_adapter.py fix(anthropic): smart thinking block signature management (#6112) 2026-04-08 03:38:08 -07:00
test_auxiliary_client.py fix: restore codex fallback auth-store lookup 2026-04-09 01:56:10 -07:00
test_auxiliary_config_bridge.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_auxiliary_named_custom_providers.py fix(auxiliary): resolve named custom providers and 'main' alias in auxiliary routing (#5978) 2026-04-07 17:59:47 -07:00
test_context_compressor.py test: add coverage for token-budget tail protection 2026-04-08 23:54:23 -07:00
test_context_references.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_credential_pool.py fix: reduce credential exhaustion TTL from 24 hours to 1 hour (#6504) 2026-04-09 02:37:23 -07:00
test_credential_pool_routing.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_crossloop_client_cache.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_display.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_display_emoji.py feat(tools): centralize tool emoji metadata in registry + skin integration 2026-03-15 20:21:21 -07:00
test_error_classifier.py feat: structured API error classification for smart failover (#6514) 2026-04-09 04:10:11 -07:00
test_external_skills.py feat(skills): support external skill directories via config (#3678) 2026-03-29 00:33:30 -07:00
test_insights.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_memory_plugin_e2e.py feat(memory): pluggable memory provider interface with profile isolation, review fixes, and honcho CLI restoration (#4623) 2026-04-02 15:33:51 -07:00
test_memory_provider.py fix: mem0 API v2 compat, prefetch context fencing, secret redaction (#5423) 2026-04-05 22:43:33 -07:00
test_memory_user_id.py fix: thread gateway user_id to memory plugins for per-user scoping (#5895) 2026-04-07 11:14:12 -07:00
test_minimax_auxiliary_url.py fix: provider/model resolution — salvage 4 PRs + MiniMax aux URL fix (#5983) 2026-04-07 22:23:28 -07:00
test_minimax_provider.py fix(minimax): correct context lengths, model catalog, thinking guard, aux model, and config base_url 2026-04-08 02:20:46 -07:00
test_model_metadata.py feat: overhaul context length detection with models.dev and provider-aware resolution (#2158) 2026-03-20 06:04:33 -07:00
test_model_metadata_local_ctx.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_models_dev.py feat: overhaul context length detection with models.dev and provider-aware resolution (#2158) 2026-03-20 06:04:33 -07:00
test_prompt_builder.py feat: switch managed browser provider from Browserbase to Browser Use (#5750) 2026-04-07 08:40:22 -04:00
test_prompt_caching.py fix(prompt-caching): skip top-level cache_control on role:tool for OpenRouter 2026-03-21 16:54:43 -07:00
test_rate_limit_tracker.py feat: capture provider rate limit headers and show in /usage (#6541) 2026-04-09 03:43:14 -07:00
test_redact.py test(redact): add regression tests for lowercase variable redaction (#4367) (#5185) 2026-04-05 00:10:16 -07:00
test_skill_commands.py fix: sanitize Telegram command names to strip invalid characters 2026-04-06 11:27:28 -07:00
test_smart_model_routing.py fix: hermes update causes dual gateways on macOS (launchd) (#1567) 2026-03-16 12:36:29 -07:00
test_subagent_progress.py feat(api): structured run events via /v1/runs SSE endpoint 2026-04-05 12:05:13 -07:00
test_subdirectory_hints.py fix(agent): catch PermissionError in subdirectory hint discovery 2026-04-09 03:10:30 -07:00
test_title_generator.py feat: auto-generate session titles after first exchange 2026-03-17 04:14:40 -07:00
test_usage_pricing.py feat: use endpoint metadata for custom model context and pricing (#1906) 2026-03-18 03:04:07 -07:00