mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-25 00:51:20 +00:00
a5bd56eae3
Three targeted changes to close the gaps between retry layers that caused users to experience 'No response from provider for 580s' and 'No activity for 15 minutes' despite having 5 layers of retry: 1. Remove non-streaming fallback from streaming path Previously, when all 3 stream retries exhausted, the code fell back to _interruptible_api_call() which had no stale detection and no activity tracking — a black hole that could hang for up to 1800s. Now errors propagate to the main retry loop which has richer recovery (credential rotation, provider fallback, backoff). For 'stream not supported' errors, sets _disable_streaming flag so the main retry loop automatically switches to non-streaming on the next attempt. 2. Add _touch_activity to recovery dead zones The gateway inactivity monitor relies on _touch_activity() to know the agent is alive, but activity was never touched during: - Stale stream detection/kill cycles (180-300s gaps) - Stream retry connection rebuilds - Main retry backoff sleeps (up to 120s) - Error recovery classification Now all these paths touch activity every ~30s, keeping the gateway informed during recovery cycles. 3. Add stale-call detector to non-streaming path _interruptible_api_call() now has the same stale detection pattern as the streaming path: kills hung connections after 300s (default, configurable via HERMES_API_CALL_STALE_TIMEOUT), scaled for large contexts (450s for 50K+ tokens, 600s for 100K+ tokens), disabled for local providers. Also touches activity every ~30s during the wait so the gateway monitor stays informed. Env vars: - HERMES_API_CALL_STALE_TIMEOUT: non-streaming stale timeout (default 300s) - HERMES_STREAM_STALE_TIMEOUT: unchanged (default 180s) Before: worst case ~2+ hours of sequential retries with no feedback After: worst case bounded by gateway inactivity timeout (default 1800s) with continuous activity reporting |
||
|---|---|---|
| .. | ||
| acp | ||
| agent | ||
| cli | ||
| cron | ||
| e2e | ||
| environments/benchmarks | ||
| fakes | ||
| gateway | ||
| hermes_cli | ||
| honcho_plugin | ||
| integration | ||
| plugins | ||
| run_agent | ||
| skills | ||
| tools | ||
| __init__.py | ||
| conftest.py | ||
| run_interrupt_test.py | ||
| test_batch_runner_checkpoint.py | ||
| test_cli_file_drop.py | ||
| test_cli_skin_integration.py | ||
| test_ctx_halving_fix.py | ||
| test_empty_model_fallback.py | ||
| test_evidence_store.py | ||
| test_hermes_constants.py | ||
| test_hermes_logging.py | ||
| test_hermes_state.py | ||
| test_honcho_client_config.py | ||
| test_ipv4_preference.py | ||
| test_mcp_serve.py | ||
| test_minisweagent_path.py | ||
| test_model_picker_scroll.py | ||
| test_model_tools.py | ||
| test_model_tools_async_bridge.py | ||
| test_ollama_num_ctx.py | ||
| test_packaging_metadata.py | ||
| test_project_metadata.py | ||
| test_retry_utils.py | ||
| test_sql_injection.py | ||
| test_subprocess_home_isolation.py | ||
| test_timezone.py | ||
| test_toolset_distributions.py | ||
| test_toolsets.py | ||
| test_trajectory_compressor.py | ||
| test_trajectory_compressor_async.py | ||
| test_utils_truthy_values.py | ||