fix(gateway): default restart_drain_timeout to 0 to kill systemd crash loop (#54066)

A restart now interrupts in-flight agents immediately rather than holding
the gateway open for a grace window. The previous 180s default coupled two
independently-set timers: the gateway's own drain timer and systemd's
TimeoutStopSec. On a stale unit where TimeoutStopSec < drain, systemd
SIGKILLed the gateway mid-cleanup, leaving a stale lock that made the next
startup exit immediately ('already running') — an infinite crash loop under
Restart=on-failure (#31981).

Setting drain to 0 makes the mismatch structurally impossible: with drain 0
the generated unit gets TimeoutStopSec=90 against a near-instant drain, so
systemd never kills mid-cleanup. Contract: restart the gateway, in-flight
work stops. A grace window large enough to 'save' a long agent turn would
have to outlast an unbounded task, which is impossible.

Also fixes the stale-unit warning's suggested command
(hermes gateway service install --replace -> hermes gateway install --force);
the former subcommand does not exist.

Closes #31981
This commit is contained in:
Teknium 2026-06-28 01:14:34 -07:00 committed by GitHub
parent 0800f1c28b
commit c9df4bc094
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 20 additions and 11 deletions

View file

@ -626,10 +626,10 @@ agent:
# gateway_timeout_warning: 900
# Graceful drain timeout for gateway stop/restart (seconds).
# The gateway stops accepting new work, waits for in-flight agents to
# finish, then interrupts anything still running after this timeout.
# 0 = no drain, interrupt immediately.
# restart_drain_timeout: 60
# Default 0 = no drain: a restart interrupts in-flight agents immediately,
# cleans up, and exits. Set a positive value only if you want a grace
# window on /restart, and keep it well under systemd's TimeoutStopSec.
# restart_drain_timeout: 0
# Max app-level retry attempts for API errors (connection drops, provider
# timeouts, 5xx, etc.) before the agent surfaces the failure. Lower this