fix(gateway): deliver the first message when the deferred agent build outlives 30s (#63078)

Leg 2 of #63078: prompt.submit returns {"status":"streaming"} immediately and
runs _start_agent_build + _wait_agent(timeout=30s) behind it. The deferred
build (MCP discovery with per-server retry backoff, synchronous model-metadata
HTTP, skills scanning) routinely outlives 30s on cold starts; on timeout
run_after_agent_ready emitted an error EVENT and returned without ever calling
_run_prompt_submit — the user's first message was permanently discarded while
the build finished successfully in the background. The desktop's optimistic
row eventually cleared with no visible error: the blank first session.

New _wait_agent_for_prompt replaces the flat cliff for the deferred prompt
path only (_sess()'s RPC-blocking _wait_agent keeps its 30s contract):

- The pending prompt stays attached to the (already off-RPC) run thread and
  is delivered the moment the still-running build completes — a slow build is
  no longer message loss.
- The wait runs in 5s slices so a cancel (session.interrupt / churn) is
  honored promptly; the cancelled path returns None and defers to the
  caller's cancel branch (the #65567 emit) for user-visible messaging.
- Past 30s the client gets ONE keyed notification.show ('Still starting the
  agent…', key=agent-build-slow, desktop toast / TUI status bar), cleared on
  delivery — patient, never silent.
- Permanent failure only when the build itself fails: agent_error set at
  ready, the build thread died without signalling ready (fail fast via the
  new _agent_build_thread handle instead of sitting out the cap on a corpse),
  or the bounded cap expired on a genuinely hung build. The cap defaults to
  600s and is tunable via agent.build_wait_timeout in config.yaml (no new
  env vars); the error message states the message was not sent.

Tests: slow-build delivery with zero error events; the keyed progress notice
shown once and cleared; build-failure surfacing exactly one error event with
the real reason; dead-thread fail-fast; cancel honored mid-wait; config
override + fallback semantics; cap expiry message. The compute-host fallback
test stubs the new waiter alongside _wait_agent.
This commit is contained in:
teknium1 2026-07-24 19:05:29 -07:00 • committed by Teknium
parent b20b235206
commit 60c8fc6290
4 changed files with 471 additions and 2 deletions

View file

@ -799,6 +799,14 @@ agent:
# window on /restart, and keep it well under systemd's TimeoutStopSec.
# restart_drain_timeout: 0
# Upper bound (seconds) a submitted prompt waits for the deferred agent
# build (MCP discovery, model metadata, skills scan) before failing with a
# visible error. The wait is patient — the message is delivered as soon as
# the build completes, and a progress notice is shown past 30s — so this cap
# only fires on a genuinely hung build. Raise it for deployments with many
# slow or unreachable MCP servers. Default 600.
# build_wait_timeout: 600
# Max app-level retry attempts for API errors (connection drops, provider
# timeouts, 5xx, etc.) before the agent surfaces the failure. Lower this
# to 1 if you use fallback providers and want fast failover on flaky