mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
A shared state.db is legitimately held for multi-second stretches by
sibling Hermes processes: VACUUM after auto-prune, the TRUNCATE WAL
checkpoint at close on a large WAL, offline recovery, or an older
still-running process whose FTS maintenance predates the bounded-merge
protocol (every `hermes update` leaves mixed-version processes sharing
the DB until the old ones exit).
The old retry budget was attempt-counted: 15 attempts x 20-150ms jitter
gives up after ~1.3s of waiting. Any hold longer than that surfaced as:
- append_message failing -> the conversation loop aborts the turn as
session_persistence_failed ('No reply: the turn was stopped because
session storage could not be written') on a perfectly healthy store;
- SessionDB() open failing -> the CLI disables persistence for the
entire run ('Failed to initialize SessionDB ... database is locked').
Both observed in production logs on 2026-07-29 (10.8 GB state.db, 9
concurrent hermes processes, three of them pre-dating the bounded-merge
fix pull).
Changes:
- _execute_write patience is now TIME-based with two budgets: routine
writes wait up to 20s; transcript-critical writes (append_message,
session-row creation — the ones whose failure aborts a user turn)
wait up to 60s. Jitter stays 20-150ms for the first 2s, then backs
off to 250ms-1s so a long hold isn't hammered with BEGIN IMMEDIATE.
- Exhausted patience raises an error that names the actual cause
(another process held the write lock; the database is healthy)
instead of a bare 'database is locked' that reads like disk damage —
and the turn-abort explainer inherits that clarity.
- SessionDB open now applies the same jittered patience to the
locked/busy class around connect+schema-init, instead of failing the
whole open (and disabling persistence for the run) on the first 1s
timeout. Non-lock errors, including the malformed-schema repair
class, propagate immediately as before.
Fixes #74478
|
||
|---|---|---|
| .. | ||
| test_compression_lineage_guard.py | ||
| test_fts_runtime_rebuild.py | ||
| test_write_lock_patience.py | ||