mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-04 12:33:08 +00:00
fix(vision): cap vision_analyze fan-out concurrency process-wide
A single agent turn can fan out N vision_analyze calls at once — the classic trigger is "analyze every frame of this video", where ffmpeg explodes a clip into dozens of frames and the model calls vision_analyze on each. Every call does a CPU-heavy base64-encode/resize burst AND holds a long-lived LLM stream open. The tool executor runs concurrent tool calls on a per-session ThreadPoolExecutor (_MAX_TOOL_WORKERS=8), and multiple agent sessions share one process (the dashboard runs the agent in-process), so there was no global ceiling. In prod (June 2026) a video-frame fan-out pinned a worker thread at ~100% CPU and starved the shared asyncio event loop that also serves the dashboard's /api/status liveness probe, flapping the instance to UNHEALTHY even though nothing had crashed. Add a process-global threading.BoundedSemaphore that bounds how many vision analyses run concurrently across the whole process, held across the entire analysis (image load + encode + LLM call) in the single _handle_vision_analyze chokepoint (covers both the native fast path and the legacy aux-LLM path). It is a threading semaphore, NOT asyncio: each vision call is dispatched through model_tools._run_async on a per-thread event loop, so an asyncio primitive bound to one loop cannot coordinate across them. The acquire is offloaded via run_in_executor so waiting for a slot never blocks the calling loop. Default: min(host CPUs, 4), floored at 1 — respect the host's concurrency, or lower. Override via auxiliary.vision.max_concurrency (config.yaml) or HERMES_VISION_MAX_CONCURRENCY (env). Values < 1 are ignored so the cap can never be disabled into an unbounded fan-out. Tests: bounded-fan-out regression guard + a control proving it would fail without the cap; resolver tests for host-cpu default, ceiling clamp, low-cpu host, env override, and sub-1 rejection. Pre-existing handler tests updated for the now-async _handle_vision_analyze. Verified via the real registry.dispatch -> _run_async per-thread-loop path (16 concurrent calls, peak bounded to cap).
This commit is contained in:
parent
115e78c377
commit
eddfecd2ce
5 changed files with 346 additions and 37 deletions
|
|
@ -1005,6 +1005,9 @@ auxiliary:
|
|||
api_key: "" # API key for base_url (falls back to OPENAI_API_KEY)
|
||||
timeout: 120 # seconds — LLM API call timeout; vision payloads need generous timeout
|
||||
download_timeout: 30 # seconds — image HTTP download; increase for slow connections
|
||||
max_concurrency: 4 # max vision analyses running at once across the whole process
|
||||
# (default: min(host CPUs, 4)) — bounds video-frame fan-out so it
|
||||
# can't saturate the event loop. Minimum 1; values < 1 are ignored.
|
||||
|
||||
# Web page summarization + browser page text extraction
|
||||
web_extract:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue