mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-29 18:46:59 +00:00
248 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Teknium
|
76b0ea5118 |
fix(state): rebuild legacy gateway_routing PK; guard session_store in dispatch hook
Two log-spam bugs found in live gateway logs: 1. gateway_routing UNIQUE-constraint spam (261 warnings in one errors.log): early builds of the #59203 routing-index migration created gateway_routing with 'session_key TEXT PRIMARY KEY' and no scope column. _reconcile_columns() ADDs the missing scope column but SQLite cannot ALTER a primary key, so the shipped composite PRIMARY KEY (scope, session_key) never lands on those databases. Both write paths then fail on every save: - save_gateway_routing_entry: 'ON CONFLICT clause does not match any PRIMARY KEY or UNIQUE constraint' - replace_gateway_routing_entries: 'UNIQUE constraint failed: gateway_routing.session_key' whenever the same session_key exists under another scope (e.g. test-suite scopes leaked into a live DB). New _heal_gateway_routing_pk() rebuilds the table once with the composite key, preserving rows (newest wins on collisions, NULL scope coalesced to ''). Same one-time-heal pattern as the #51646 active- column repair. Verified E2E against a copy of a real affected state.db. 2. pre_gateway_dispatch warned ''GatewayRunner' object has no attribute 'session_store'' and silently dropped the hook for every message on partially-initialized runners (bare object.__new__ runners in tests, and any future init-order change). Pass getattr(self, 'session_store', None) so the hook always fires (pitfall #17 pattern). Both regression tests fail without their fixes (sabotage-verified). |
||
|
kshitij kapoor
|
f228e145ba |
fix: follow-up for PR #65541 — track read conns, convert remaining read paths, mark WAL tests
- Route _get_read_conn through _connect_tracked_db so per-thread read-only connections are registered with the POSIX lock-safety guard (connect_tracked), matching the writer and existing read-only paths. Without this, byte-level probes of state.db could close() an fd that cancels locks held by an untracked read connection. - Convert _search_unindexed_gap, _run_trigram_search, CJK-bigram FTS search, and get_meta to _read_ctx — these are pure SELECT queries called from search_messages that were still taking self._lock, defeating the PR's contention fix for those paths. - Add @pytest.mark.requires_wal to the 5 tests that assume WAL is active. Hermes disables WAL on SQLite < 3.51.3 (WAL-reset bug), so these tests fail on the venv's SQLite 3.46.0 without the marker. - Remove unused 'time' import. |
||
|
Soju06
|
6623ee9bb2 |
perf(state): read-path split — per-thread read-only connections for recall reads
The gateway shares ONE SessionDB across every agent, so every recall/browse read (session_search discover/scroll/browse, memory prefetch, title resolve) queued behind every writer flush on self._lock — one Python lock in front of a WAL database that natively supports concurrent readers. Measured convoy: a 0.23s FTS query stretched to 112s and a browse flush to 137s while 6-8 concurrent turns flushed hundreds of tool results. Fix: under WAL, read-only methods (get_session, resolve_session_by_title, list_sessions_rich, get_messages, get_messages_around, get_anchored_view, search_messages) run on a per-thread mode=ro connection via _read_ctx(), taking no lock at all. Fresh read transactions begin per statement, so read-your-committed-writes holds for flush-then-search patterns. Non-WAL (NFS DELETE fallback) or read-conn open failure keeps the legacy locked single-connection path, remembered per thread to avoid per-query retries. |
||
|
dsad
|
748c12b148 |
fix(session): skip display_kind timeline rows in undo/retry turn targets
list_recent_user_messages and the in-memory /retry + session.undo walkers treated every role=user row as a real user turn. Timeline bookkeeping (model_switch, async_delegation_complete, auto_continue, hidden) is stored that way, so /undo soft-deleted from a marker and /retry re-sent opaque bookkeeping text. Exclude display_kind the same way CLI resume counting and the prompt.submit ordinal path do. |
||
|
liuhao1024
|
9e2f07e704 |
perf(dashboard): use GROUP BY for session stats instead of fetching 10k rows
Replaces the O(N) list_sessions_rich histogram in /api/sessions/stats with a single GROUP BY query, reducing response time from ~575ms to <1ms on large databases. Original PR #48921 by @liuhao1024. Salvage fixes based on review feedback from teknium1 and @wernerhp: 1. Preserve try/except guard — a DB error still degrades to empty by_source instead of failing the whole stats response. 2. GROUP BY COALESCE(source, 'cli') — the original GROUP BY source could emit duplicate 'cli' keys (NULL group + literal 'cli' group) that the dict comprehension silently dropped. 3. Add exclude_children=True — list_sessions_rich excludes subagent runs, delegates, and compression continuations by default; the bare GROUP BY counted all rows, inflating source counts. Aggregate shape (exclude_children/include_archived/limit params) adapted from closed duplicate #61120 by @mijanx. Closes #48914 Co-authored-by: mijanx <mijanx@users.noreply.github.com> |
||
|
kshitijk4poor
|
927633272f |
fix: claim busy before clearing queue in _stop_token_writer drain
Follow-up to PR #64171. The writer loop and flush_token_counts' caller-drain both set _token_writer_busy BEFORE popping the queue — that ordering is what makes flush's lock-free fast path (reads queue then busy, no cond held) sound. _stop_token_writer's leftover drain did it backwards (clear queue, then set busy), leaving a few-bytecode window where a concurrent flush could observe 'empty and idle' and return True with the popped batch still unapplied. Shutdown-only staleness, no data loss — but the protocol now matches at all three drain sites. Test: concurrent flush during a stop-drain mid-apply must time out (False), never report drained. |
||
|
kshitijk4poor
|
e49705d6af |
fix: respawn dead token writer and never let coalescing kill it
Follow-up to PR #64171. Two writer-death hardening gaps: - queue_token_counts only spawned the writer when the thread object was None, so a writer that died from an unexpected exception could never be replaced: deltas piled up on the uncapped deque until a reader's flush drained them synchronously. Respawn on 'not thread.is_alive()' instead. (atexit re-registration on respawn is safe: unregister removes all equal bound-method registrations and the drain hook is idempotent.) - _coalesce_token_deltas ran outside the per-delta try/except in _apply_token_batch, so a merge bug (e.g. an unclassified future kwarg summing None + 0) would escape and kill the writer thread. Wrap it and fall back to applying the raw batch — coalescing is an optimization, never load-bearing. Tests: coalesce-failure fallback + dead-writer respawn. |
||
|
Soju06
|
174ad45939 |
perf(state): apply per-call token accounting on a background single-writer queue
Every API call in the tool loop persisted its token/cost delta by calling SessionDB.update_token_counts() synchronously on the turn thread — a BEGIN IMMEDIATE sessions UPDATE plus a session_model_usage upsert, measured in production at p50 3.3ms / p95 70.4ms per call and up to 299ms against a cold multi-GB state.db. The tool loop stalls for that long between calls, N times per multi-tool turn. SessionDB gains queue_token_counts(): same signature and semantics as update_token_counts(), but the critical path is a deque append plus a condvar notify. A lazily started daemon thread applies deltas in enqueue order through the existing update_token_counts -> _execute_write path, so the established self._lock / BEGIN IMMEDIATE / jitter-retry discipline is unchanged. When a backlog forms, adjacent same-route incremental deltas coalesce into one UPDATE: token and api-call fields sum, cost fields sum None-preservingly (an all-None run stays None so COALESCE keeps the stored value), and absolute=True deltas never merge and act as ordering barriers. Route equality is required for a merge because those fields feed COALESCE backfill, the last-non-None-wins status fields, and the per-model usage attribution key — a merged apply is row-equivalent to sequential applies. Correctness and durability: - flush_token_counts() gives read-your-writes to token/cost readers (get_session, list_sessions_rich, _get_session_rich_row, list_gateway_sessions, InsightsEngine.generate) — a plain attribute check when nothing is queued. The writer sets its busy flag before popping the queue so the lock-free fast path can never miss an in-flight batch. - update_session_model / update_session_billing_route / update_session_meta write the sessions row synchronously, bypassing the queue, so they flush it first: a still-queued first-of-session delta carries the pre-switch route, and applying it after the switch UPDATE would trip the first_accounted_route branch (api_call_count == 0 plus a route mismatch) and resurrect the old model/provider. - AIAgent._persist_session flushes at turn finalize and every error-exit persist point; close() stops and drains the writer before the WAL checkpoint; an atexit hook (registered on first enqueue, unregistered on close so closed instances are not pinned until interpreter exit) drains at shutdown. Worst-case crash loss is the in-flight call's delta — the same window as the old inline write. - A flush trusts a live stop-flagged writer (its loop drains before exiting) and only drains on the caller's thread when the writer is dead or never started, claiming the same busy flag so concurrent flushes wait instead of racing an in-flight batch. - After close() has stopped the writer, queue_token_counts applies the delta inline instead of parking it on a queue nothing will drain; a closed-connection failure then raises at the call site, which already guards for it, exactly like the old synchronous path. - Writer apply failures are logged and never raise into a turn; the writer thread survives and keeps applying. Call sites switched to the queue: the per-call site in agent/conversation_loop.py and both codex app-server sites in agent/codex_runtime.py. In-memory per-turn counters (agent.session_estimated_cost_usd etc.) stay synchronous, so live turn displays never see the queue. Tests: tests/agent/test_async_token_accounting.py (19 tests: enqueue ordering, absolute-as-barrier, backlog coalescing with exact sums, coalesced-vs-sequential row equivalence, merge unit rules, None-cost preservation, read-your-writes, flush vs stop-flagged/concurrent drains, inline apply after writer stop, close/atexit durability, _persist_session drain, writer failure isolation); tests/run_agent/test_token_persistence_non_cli.py updated to the queue_token_counts contract. |
||
|
Teknium
|
373632e338 |
fix(state): recover from read-only database files at startup
Port from Kilo-Org/kilocode#12508: a stray read-only state.db / -wal / -shm (sudo run, restored backup, copied dotfiles) previously killed SessionDB init with an opaque 'sqlite3.OperationalError: attempt to write a readonly database' raised from deep inside _init_schema — naming no file and no fix — and the obvious wrong 'fix' (deleting the -wal) silently loses committed transactions. New preflight_db_writability() runs before the first connection on both DB open paths (SessionDB.__init__ and hermes_cli.kanban_db.connect): - files inside the Hermes home tree are repaired with chmod u+rw (the safe scope: Hermes owns them, and the OS makes chmod fail on files the user doesn't own, which bounds the repair exactly); - anything else (root-owned files, read-only mounts, custom paths) fails fast with an error naming the exact file and the exact chmod command, plus an explicit 'do NOT delete the -wal' warning; - WAL sidecars are never deleted or truncated — once writable, the normal open path checkpoints committed frames into the DB. Proven live on main first: chmod 444 state.db -> SessionDB() raises the opaque readonly error. With the fix: in-home DBs self-heal; out-of-home DBs get the actionable message. Sabotage run confirms the integration tests fail without the wiring (2 failed / 10 passed). |
||
|
kshitijk4poor
|
2c1809e6ca |
revert: PR #72817 — session activity watchdog, stall notify, compress timeout
Reverting #72817 (salvage of #72424) pending further review. All 4 commits reverted: feat, refactor, chore (contributor map), CI fix. |
||
|
fangliquanflq
|
cfb206fe2e |
feat(gateway): session activity watchdog, stall notify, compress timeout (#72424)
Three mechanisms to detect and notify when gateway sessions stall silently: 1. Mid-turn activity heartbeats stamped to SessionDB so hermes sessions list and hermes status show progress during long turns without new message rows. 2. Stall watchdog: when a busy session has pending inbound and the shared activity clock is idle past agent.session_stall_timeout (default 300), log a WARNING and notify the user once to try /new. Notify-only; does not kill the turn. 3. Compaction timeout: fenceless compress_context callers get a progress-aware host budget (compression.context_timeout_seconds default 120 idle, compression.context_total_ceiling_seconds default 600 ceiling). On timeout, cancel via commit fence, skip compaction without dropping messages, and continue the turn. Closes #72016 (slices 1-3; slice 4 cumulative SSE stream-retry deadline remains a follow-up). Cherry-picked from PR #72424 by @fangliquanflq. |
||
|
Brooklyn Nicholson
|
28a87d6319 |
fix(cli): scope -c/--resume to the current workspace
`hermes -c`/`--resume` (continue last session) resolved the globally most-recently-used session, then cd'd into *its* recorded cwd. So running `hermes -c` from repo A could land you in repo B's session — the session you last touched anywhere, not the last one *here*. Now `_resolve_last_session` scopes to the current workspace first: the git repo root when CWD is inside a repo (so all sessions across its subdirs/worktrees group together), else the CWD itself — matching the `workspace_key` identity `hermes sessions list --workspace` already groups on. It falls back to the unscoped global MRU when no session matches the current workspace, preserving the old behaviour for fresh directories. Adds `workspace_key` param to `SessionDB.search_sessions` and a `_workspace_key_clause` SQL helper that mirrors `workspace_key()`: a row matches when its `git_repo_root` equals the key, or (legacy rows without git metadata) when its `cwd` is at or under it. |
||
|
Frowtek
|
a228b81501 | fix(sessions): preserve recently active sessions during pruning | ||
|
Brooklyn Nicholson
|
c3d199c248 |
fix(sessions): keep the skill body out of session previews
`preview` is the head of the first user message and the title fallback on every surface — sidebar rows, pickers, exports, the desktop's sessionTitle(). An untitled /skill session therefore read `[IMPORTANT: The user has invoked the "work" skill, indicatin...` wherever it appeared. A scaffolded row now selects a wide enough excerpt to reach the typed instruction (head + tail spliced for a long body) and shapes it through describe_skill_invocation(). Because previews are computed on read, existing sessions are corrected without a migration. The six copies of the preview subquery and four copies of its shaping collapse into one expression and one helper along the way, and the /rewind picker gets the same treatment. |
||
|
joaomarcos
|
11c487e409 |
fix(compression-lock): reclaim crashed holders instead of stranding the lease
The compression lock was gated so that any holder on a Windows host was assumed alive until the full TTL expired. A crashed holder therefore stranded every other agent on the session for the whole lease window. Probe liveness with psutil.pid_exists, which is safe on nt. Keep the os.kill(pid, 0) path strictly for POSIX: on Windows signal 0 maps to CTRL_C_EVENT (bpo-14484) and can kill the target's console group, so with psutil absent the only safe answer stays 'assume alive' and let the TTL run out. Also carry the commit fence through cancelled compressions so an aborted run leaves the lock reacquirable rather than half-held. |
||
|
teknium1
|
fe431651c5 |
fix(state): make the byte-probe guard atomic, path-correct, and fail-closed
Addresses review findings on the previous commits. Three of them were real defects I reproduced against my own head before fixing. 1. Check/use race (BLOCKING). read_header_bytes_preopen() checked has_live_connection() under _live_lock, released it, then did the raw open/read/close outside the lock; connect_tracked() opened before registering. A thread could pass the "nothing is live" check, another could open a connection and BEGIN IMMEDIATE, and the first thread's close() then cancelled its POSIX locks -- the exact bug this guard exists to prevent. Reproduced deterministically (BLOCKED -> ACQUIRED). _live_lock now spans all three lifecycle transitions: open+register, unregister+close, and check+open+read+close. 2. Read-only connections keyed by URI spelling (BLOCKING). SessionDB's read-only path opens file:/…/state.db?mode=ro; that string was fed to Path.resolve(), producing <cwd>/file:/…/state.db?mode=ro. No probe of the real Path could match, so read-only connections were invisible to the guard and their locks cancellable. Reproduced with no forced scheduling. Keys now come from PRAGMA database_list (canonical path), with an explicit tracking_path override. 3. Fail-open wrapper (HIGH). _connect_tracked_db() caught every exception and retried an untracked plain connect, so any error silently disabled the guard. Now only ImportError (scaffold installs without hermes_cli) falls back; real failures propagate. 4. Backup paths that warned and proceeded (MEDIUM). _backup_corrupt_db() and _backup_db_file() raw-read live databases; they now REFUSE when a connection is live rather than warning. Losing a forensic copy beats corrupting the database being rescued. Custom factories are no longer rejected (that broke legitimate callers) nor silently untracked -- the tracking close() is mixed into whatever factory is in play, including when an opener substitutes its own after the fact. WAL POLICY: #70055 is RESTORED, not reverted. My earlier justification was confounded -- the clean WAL result came from 3.53.1, which carries both the WAL-reset fix AND 3.51.0's broken-lock defenses, so it said nothing about the bundled 3.50.4. Re-measured on 3.50.4 with the lock fix in place: WAL 0/3 and DELETE 0/3, i.e. no evidence WAL is safer. Upstream still documents the WAL-reset bug through 3.51.2 as serious. Keeping new databases out of WAL until a fixed runtime ships is the conservative call, and the WAL policy does not belong in this root-cause fix. Six sabotage runs confirm each new test fails when its defect is reinstated (including two that initially did NOT -- the race test was rewritten to pause inside the byte read, and a separate test added for the opener-substituted factory path). 1124 targeted tests green. |
||
|
teknium1
|
95fb477856 |
fix(state): close the tracking leak and finish the audit of raw DB reads
Completeness pass over the previous commit. Registry leak (would have silently disabled the guard): the first version incremented on open but decremented only in SessionDB.close(). kanban's connect() hands raw connections to callers who close them directly (4 sites), so its counter only ever went up — after enough kanban operations every byte-probe on that path would be refused forever, disabling zeroed-file and header detection. Replaced manual track/untrack with a TrackedConnection subclass that untracks in close(), the one method every close path goes through. Verified across plain close, contextlib.closing, double close, nested lifetimes, and 100-cycle churn; `with conn:` (a transaction scope, not a close) correctly stays tracked. Also: a caller-supplied factory now wins instead of raising TypeError on a duplicate kwarg, since tracking is an optimisation for the probe guard, not a precondition for opening the database. Removed _apply_delete_for_wal_reset_bug, dead after the force-DELETE revert. Audit notes: - DELETE mode is still reachable via the NFS/SMB/FUSE fallback and remains correct there; those users are protected by the raw-read fix, not by the journal mode. - The remaining whole-file reads are on genuinely offline artifacts: _backup_db_file (DB won't open; bytes preserved for forensics), _backup_corrupt_db (quarantine path, now warns if a connection is live), and backup verification of snapshots. backup._safe_copy_db already uses the SQLite backup API rather than a byte copy. - The mechanism is POSIX-specific (Windows byte-range locks are handle-scoped, not process-scoped), so this is a Linux/macOS correctness fix; the change is platform-neutral and safe on Windows. Sibling tests updated: session recovery no longer expects a DELETE-mode recovered DB, and the kanban WAL-fallback test patches the connect site that actually runs now. |
||
|
teknium1
|
fbd5e5772b |
fix(state): stop cancelling our own POSIX locks on live SQLite databases
`hermes sessions optimize` could corrupt state.db. Root cause is Hermes, not the SQLite WAL-reset bug (#69784). close() on ANY file descriptor for a SQLite database cancels every POSIX advisory lock the process holds on that file, including a running VACUUM's EXCLUSIVE lock (sqlite.org/howtocorrupt.html section 2.2). Hermes byte-probed live databases in several hot paths: the zeroed-state.db detector runs on every SessionDB construction (and the gateway builds those constantly), and kanban's post-commit invariant check ran after every COMMIT. While VACUUM rewrote the file, those probes dropped its lock and let other processes write into it. A/B against the real code, only variable being the raw read: SQLite 3.50.4, VACUUM + concurrent writers, DELETE mode raw open/close during VACUUM 8 vacuums, 319 vacuum errors, 2/2 corrupt no raw read (control) 229 vacuums, 0 vacuum errors, 0/2 corrupt SQLite 3.53.1 (WAL-reset FIXED) reproduces identically: 2/2 corrupt. After this change: 0/4 corrupt, 0 vacuum errors. Because the upgraded runtime corrupts too, replacing the embedded SQLite does not fix this class; and because DELETE is where it reproduces, #70055's "force DELETE on vulnerable builds" mitigation steered users into the failing mode. That gate is reverted here: vulnerable builds get WAL again and still warn so operators can upgrade. - add hermes_cli/sqlite_safe_read.py: read page_count via PRAGMA over the existing connection instead of open()+seek(28); byte-level probes are restricted to before any connection exists and refused once one is live, with an explicit force= escape for offline artifacts (snapshots, archives) - track live connections in SessionDB and kanban's connect so that guard is enforced rather than merely documented - kanban's torn-extend check now only applies under a rollback journal; in WAL a committed page may still legitimately sit in the -wal file - revert the force-DELETE WAL gate and update the tests that pinned it Regression tests assert the behavioural contract (an external process stays locked out across Hermes' inspection calls) and were verified to fail when the old raw-open behaviour is restored. |
||
|
Jasmine Naderi
|
22e5dac4b6 |
fix(state): quarantine lock fails closed when unacquireable (#68805)
Reviewer egilewski identified that the 5s lock timeout fell open: quarantine_zeroed_state_db() logged 'proceeding without the cross- process lock' and continued to re-check + rename state.db. A slow or paused startup that still owns the lock can overlap this fallback and the two processes can again act on the same live file. Fix: fail closed — return None without moving the file when the lock cannot be acquired within 5s. Log an error with recovery guidance (restore from state-snapshots). Test: test_quarantine_fails_closed_when_lock_held holds the cross- process lock from a background thread, calls quarantine, and asserts it returns None without moving the zeroed file. |
||
|
Jasmine Naderi
|
6048696ed0 |
fix(state): cross-process quarantine lock + oversized DB pruning suppression (#68805)
Reviewer egilewski identified two recovery-loss paths: Path A — quarantine race (hermes_state.py): SessionDB checks state.db before quarantine_zeroed_state_db() without a shared cross-process lock, and Path.rename() may replace an existing destination. Two writable startups can therefore move the first instance's newly created database over the .zeroed-*.bak, erase the original damaged-file evidence, and replace the live database with another empty one. Fix: add a cross-process lock (msvcrt on Windows, fcntl on POSIX) around quarantine_zeroed_state_db() with a 5s bounded timeout. Under the lock: re-check is_zeroed_state_db (another process may have already quarantined it and created a fresh DB), use a PID-suffixed unique destination, and non-clobbering rename with counter fallback. Path B — size-cap pruning gap (hermes_cli/backup.py): _too_large() runs before failed-database tracking. With keep=1 and a size cap, an oversized state.db is omitted while failed_dbs stays empty, so automatic pruning deletes the older complete snapshot that may contain the only recoverable database. Fix: track oversized DB files in a new oversized_skipped list (both in the directory walk and top-level file loop). The manifest now records oversized_skipped. Pruning is suppressed when failed_dbs or oversized_skipped is non-empty, preserving the older complete snapshot as recovery source. Tests: - test_concurrent_quarantine_no_clobber: two threads racing on the same zeroed state.db — verifies quarantine backup survives with original bytes and live DB is valid. - test_oversized_db_suppresses_pruning: keep=1 + oversized state.db verifies the older complete snapshot is not pruned. All 33 tests pass (3 zeroed_state_db + 25 TestQuickSnapshot + 5 quarantine_forensic_logging). |
||
|
Jasmine Naderi
|
ed5e41ddd6 |
fix(state): loud failed state.db snapshot + zeroed-file quarantine
Hardening for the Windows zeroed-state.db class (#68474): - Surface critical stdout when pre-update/quick snapshot cannot copy a present *.db (was log-only; update still looked successful). - Detect all-NUL SQLite header on SessionDB open, quarantine the bytes, and open a fresh DB with recovery guidance to state-snapshots. Does not claim storage-stack root cause. |
||
|
teknium1
|
0b3a50f108 |
fix(sessions): measure reclaimed space with SQLite page accounting
Builds on @ms-alan's label fix: the negative figure had a second cause that relabelling alone leaves in place. `hermes sessions optimize-storage` reported "reclaimed -3820.1 MB" on a database that had in fact shrunk 60% (25069 MB -> 9975 MB). Both figures came from os.path.getsize(). In WAL mode a VACUUM's rewrite lands in the -wal file, and the checkpoint that folds it back is REFUSED (SQLITE_BUSY) while any other connection holds a read-mark — e.g. the live gateway. So the main file still carried its pre-VACUUM size AND kept growing while the command ran, making the after-figure larger than the before-figure. The TRUNCATE checkpoint in close() lands after the caller has already measured and printed. - hermes_state.py: add SessionDB.logical_size_bytes() — page_count * page_size, the size the file settles at once the WAL is folded back. Correct immediately, readers or not; returns None when the connection is gone so callers fall back to stat(). - hermes_state.py: best-effort TRUNCATE checkpoint after the optimize VACUUM so the file settles promptly when nothing else holds the DB. Documented as insufficient alone (busy under a live gateway) so the stat() approach is not reintroduced. - hermes_cli/main.py: both `optimize-storage` and `optimize` report via logical_size_bytes(); fixing only the reported command would have left the same bug class next door. - hermes_cli/main.py: lift the contributor's inline label to a module-level _size_delta_label() shared by both sites, so the "grew by" wording is consistent and unit-testable without reading source. The two halves are complementary: page accounting removes the phantom negative, and "grew by" still covers a genuine increase when concurrent session writes outweigh what the rebuild freed. Verified: sabotaging logical_size_bytes() back to stat() fails the new test with a 22 MB overstatement, reproducing the report. The test asserts its own precondition (stat() must actually be lagging) so it cannot silently stop exercising the bug. Tests: 464 passed (test_hermes_state.py + the new label tests). Co-authored-by: chenbin <h-chenbin@voyah.com.cn> |
||
|
kshitijk4poor
|
ca35663013 |
refactor: extract lineage_is_logical local + document TOCTOU re-query
Follow-up cleanup for PR #71123: - Extract getattr(args, 'lineage', 'single') == 'logical' to a local (appeared 3x in the export block) - Document that the double _collect_delegate_child_ids traversal in delete_session is an intentional TOCTOU guard inside the write txn |
||
|
Drexuxux
|
c1fb170449 | fix(sessions): export delegate cascade before deletion | ||
|
Brooklyn Nicholson
|
19dc35cf57 |
fix(state): stop double-encoding display_metadata on write
export_session() reads through get_messages(), so before the read fix an already-serialized string went straight back into _insert_message_rows() and got re-dumped — an export/import round trip permanently corrupted the row. Guard the three write paths the same way tool_calls already is: parse a string argument before storing it, and drop metadata that isn't an object rather than persisting something no reader can use. Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> Co-authored-by: aml1973 <aml1973@users.noreply.github.com> |
||
|
Brooklyn Nicholson
|
3399bf28a5 |
fix(state): decode display_metadata at every message read path
get_messages(), get_messages_around() and get_anchored_view() returned the raw display_metadata column instead of the dict every caller expects. The desktop paints a resumed transcript from the REST prefetch, which reads through get_messages(), so any session holding an async_delegation_complete event failed resume with "Cannot use 'in' operator to search for 'task_count'" — on every such session, not just corrupted ones. Route all four read paths through one shared codec that also unwraps rows carrying a second JSON layer, so sessions already broken on disk recover on read rather than needing a migration. Co-authored-by: Studio729 <Studio729@users.noreply.github.com> Co-authored-by: aml1973 <aml1973@users.noreply.github.com> Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> |
||
|
Anthony Ruiz
|
0ee8d41878 | fix(compression): recover rotated session lineage | ||
|
Justin Bennington
|
17bf3c8283 | fix(runtime): repair vulnerable managed SQLite builds (E-949) | ||
|
abundantbeing
|
7cd48733db |
feat(api): backend-acknowledged session model lock with runtime routing
Add a persisted, backend-confirmed provider/model lock for Hermes
Browser and other session API clients. A confirmed lock is an
execution contract rather than response metadata:
- POST /api/sessions/{session_id}/model validates and persists a
confirmed browser_model_lock (advertised in /v1/capabilities)
- session chat + chat/stream consume the persisted lock on body-only
follow-up turns; a confirmed lock wins over an older gateway session
/model override and the session-persisted model
- a later successful session /model switch explicitly clears and
replaces the lock while preserving lineage markers (_branched_from)
and invalidating cached system-prompt model/provider metadata
- ordinary one-off request overrides never replace a confirmed lock
- provider-resolution failure fails closed as a typed provider-auth
error (controlled response, never global-credential reuse)
- confirmed locks disable the global fallback model chain
- the completed agent's actual provider/model must match the locked
route or the turn fails with a runtime-mismatch error
- responses carry sanitized runtime metadata reporting actual vs
requested provider/model and lock state
Rebased onto the provider-aware request routing (#70853) and
session-model parity (#70931) that landed since the original branch;
the lock now slots into that precedence chain as the top rung.
Salvaged from PR #61236 by @abundantbeing.
|
||
|
Brooklyn Nicholson
|
f16b80362c |
feat(sessions): opt-in auto-archive of stale sessions + durable pin flag
New sessions.auto_archive / auto_archive_days config: soft-hide (never
delete) sessions with no activity for N days, aging on last activity
rather than creation so an old-but-active chat is spared. Sweeps are
throttled through state_meta and fire from CLI startup, gateway startup
+ hourly housekeeping, and the serve/dashboard backend (opportunistic
on session list + an hourly lifespan ticker), so every surface honours
one setting.
A new pinned column (declaratively migrated) exempts sessions from the
sweep; PATCH /api/sessions/{id} accepts pinned and flips the whole
compression lineage as a unit, mirroring set_session_archived.
|
||
|
Brooklyn Nicholson
|
e9a243ef78 |
fix(state): inherit and stamp profile_name across rotation and branch children
profile_name was only written on the agent's initial lazy create
(
|
||
|
Teknium
|
fdefb2d38c |
fix(compression): prefer psutil.pid_exists for lease liveness probe; add same-pid self-reclaim guard
Hardening on top of the salvaged dead-PID lease reclamation from PR #65775 (@the3asic): - Probe via psutil.pid_exists (hard dependency; CONTRIBUTING.md critical rule #1) with the contributor's os.kill(pid, 0) POSIX probe retained only as a scaffold-phase fallback when psutil is missing. - Same-process holders (pid == os.getpid()) are never probed and never self-reclaimed — another thread's live lease is owned by the lease refresher/release path. - Any probe doubt (exceptions, permission errors) conservatively keeps the lease until normal TTL expiry; Windows stays TTL-only. - Tests: psutil-first dead-pid reclaim (probe call pinned), os.kill fallback path, probe-doubt keeps lease, same-pid no self-reclaim, legacy holder + Windows paths assert NO probe via either API. |
||
|
3ASiC
|
6ab8428b88 | fix(compression): keep PID probing POSIX-only | ||
|
3ASiC
|
8cd49c496f | fix(compression): reclaim locks from dead processes | ||
|
ethernet
|
a4bc1ca502
|
fix(timeline): persist typed display events (#69771)
* fix(desktop): hide persisted agent-only history scaffolding Filter verification-stop nudges and context-compaction handoffs at the stored-history mapper boundary. Preserve a real reply when a compaction handoff shares its stored message. * test(desktop): build persisted E2E sessions through the real agent Drive tui_gateway.entry over its stdio JSON-RPC transport against the mock provider, wait for real completion events, and persist normal session history through AIAgent and SessionDB. Migrate resume and hidden-history coverage, including real compression and live verify-on-stop scaffolding, then remove the unused direct SessionDB import scripts. * fix(desktop): use the provisioned Python for real-session E2Es Run the stdio gateway through uv's synced project environment outside the Nix dev shell, while retaining the fully provisioned Nix Python when the shell advertises HERMES_PYTHON_SRC_ROOT. * fix(nix): expose the provisioned Python environment to uv Mark the Nix-built Python environment active in the dev shell so the shared E2E session builder can always run through `uv run --active --no-sync`. * fix(timeline): persist typed display events * fix(timeline): strip display-only fields from provider payloads, preserve through rewrites, fix /resume display history Three review findings from PR #69771: 1. Provider payload leak: display_kind and display_metadata were forwarded to the provider API as unknown message fields. Strict OpenAI-compatible backends can reject the next request after a model switch or resumed typed event. Strip both from the per-request api_msg copy in conversation_loop alongside the existing api_content pop. 2. Rewrite/import data loss: _insert_message_rows preserved display_kind but silently dropped display_metadata. After replace_messages, archive_and_compact, or session import, async-delegation completion events lost their task counts and fell back to generic display text. Add display_metadata to the INSERT columns and bind tuple. 3. CLI /resume stale recap: startup --resume A set _resume_display_history from A's lineage. A subsequent in-session /resume B loaded B only into conversation_history via get_messages_as_conversation, leaving the stale A display projection. _display_resumed_history preferentially read the stale attribute, showing A's recap for B. Switch /resume to get_resume_conversations and update _resume_display_history alongside conversation_history. Tests: 890 Python (5 files), 35 desktop TS — all green. * feat(tui): render typed display events as ◈ markers in the Ink TUI The TUI was not handling display_kind at all — model switch markers and async delegation completions rendered as opaque user messages with the full [System: ...] text, and hidden compaction handoffs were visible. Wire display_kind through the full TUI chain: - _history_to_messages (tui_gateway/server.py) forwards display_kind and display_metadata to the gateway transcript payload. - GatewayTranscriptMessage (gatewayTypes.ts) gains both fields. - Msg.kind (types.ts) gains 'event' value. - toTranscriptMessages (domain/messages.ts) maps: - hidden → skip entirely - model_switch → event "model changed" - async_delegation_complete → event "N background agents finished" (or "background agent work finished" without metadata) - messageGroup (blockLayout.ts) routes event to its own group, with SELF_SPACED + PAINTS_TRAILING_GAP so it owns its margins. - messageLine.tsx renders event-kind as a dim ◈ marker with no gutter, matching the CLI's ◈ event rendering. - 4 new TUI tests for hidden/model_switch/async_delegation mapping. TUI typecheck: clean. TUI lint: 0 errors (2 pre-existing warnings). TUI tests: 9 passed (1 pre-existing failure on main, unrelated). |
||
|
Teknium
|
ec5835ab8b
|
fix(compression): persist anti-thrash state across process restarts (#69872)
The anti-thrash guard (_ineffective_compression_count) was in-memory
only: a fresh compressor bound to a resumed, already-compacted session
started with compression_count=0 and a disarmed guard, so a
near-threshold session could legally re-compact once per process
restart, forever.
Persist the counter through the durable session-state channel,
mirroring the failure-cooldown (#54465) and fallback-streak (
|
||
|
kshitij
|
7419de6ac6 |
refactor: simplify WAL-reset gate warning dedup + tuple handling
Consolidate the two near-identical warning strings in _log_wal_reset_bug_once into a single logger.warning call with an action variable. Remove overengineered defensive tuple-length handling in is_sqlite_wal_reset_vulnerable (sqlite3.sqlite_version_info always returns a 3-tuple). Remove extra blank line. Follow-up cleanup for PR #69981. |
||
|
HexLab98
|
953cbc0300 |
fix(state): refuse WAL on SQLite builds with the WAL-reset bug
On vulnerable SQLite (e.g. 3.50.4), do not enable WAL for fresh/non-WAL shared databases — prefer DELETE instead. Leave existing on-disk WAL alone (no live downgrade under concurrent gateway/cron openers). Surface Python/SQLite version details as a doctor warning (#69784). |
||
|
gnanam1990
|
96560ee60f |
fix(agent): recover pure-Latin search matches embedded in CJK text (#54242)
A pure-Latin query (no CJK characters) routes to the unicode61
`messages_fts` table, whose tokenizer does not insert a boundary between
Latin letters and adjacent CJK characters. Content like "修改youer服务端" is
indexed as a single token, so `search_messages("youer")` returned zero
results even though the substring is present, and the Latin path had no
fallback.
Add a zero-result trigram fallback to the pure-Latin path: when the
unicode61 search misses, retry against the existing `messages_fts_trigram`
table, which matches substrings regardless of word boundaries. The fallback
is gated on `_trigram_available` and on every token being >=3 chars (the
trigram minimum), and only fires on a zero-result miss, so successful Latin
searches keep their unicode61 ranking unchanged.
The trigram query construction shared with the CJK path is extracted into a
`_run_trigram_search()` helper; the CJK branch is refactored to use it with
no behavior change.
Adds regression tests in tests/test_hermes_state.py::TestCJKSearchFallback.
|
||
|
Teknium
|
f13f845116 |
feat(state): messages_fts_cjk — CJK-bigram index on the v23 external-content layout
Integration layer for the cjk_unicode61 tokenizer, rebuilt on the v23 schema (the contributed integration in PR #65544 predated it): - messages_fts_cjk: external-content FTS5 over a tool-row-excluding view (same v23 storage discipline as the trigram index it supersedes — zero inline text copies). Serves EVERY CJK query shape the legacy routing split between trigram (>=3 chars/token) and LIKE full scans (1-2 char tokens). Lone 1-char CJK runs and role_filter=['tool'] queries keep their legacy routes. - Dedicated marker pair (fts_cjk_rebuild_high_water/progress) gates the id-scoped triggers, so a cjk-only backfill never gates the complete messages_fts/trigram triggers. - Transitions ride (the existing throttled/resumable chunk engine): fresh DBs are born with the index; legacy v22 DBs land on v23+cjk in one run; already-optimized v23 DBs gaining the tokenizer get a marker-gated backfill; live writes are indexed immediately in every case. - Tokenizer-loss self-heal: a process that can't load the extension drops the cjk triggers (writes keep working), leaves a stale breadcrumb, and the index is rebuilt from scratch on the next optimize run — triggers are never reinstalled over a gap (external-content 'delete' on an unindexed rowid is the FTS5 corruption hazard the marker gating exists to prevent). - Capability classification: 'no such tokenizer: cjk_unicode61' joins the degraded-runtime error class everywhere (read probe, write probe, repair) so tokenizer absence is never misclassified as corruption. - Config: sessions.cjk_fts (default on, inert without the .so) and sessions.search_slow_ms in config.yaml, bridged to env by CLI + gateway (startup + per-turn reload). build.sh falls back to vendored SQLite headers so no libsqlite3-dev is needed. Slow-query log path attribution updated: fts_cjk / fts5 / trigram / like_scan. Tests: 14 lifecycle tests (fresh/legacy/stale/backfill paths, tokenizer-loss round-trip) + 5 config-bridge tests + slow-log suite. |
||
|
Soju06
|
8364576e33 |
feat(state): slow-query log for session search with routing-path attribution
One INFO line per slow search naming the path taken (fts_cjk / fts5 / trigram / like_scan), elapsed time, row count, and the query. The 2026-07 session_search investigation needed turn-trace archaeology plus workload replay to discover that short-CJK queries were full-scanning the table — with this line the next routing regression is a journalctl grep. Threshold: sessions.search_slow_ms (default 1000ms; 0 logs every call), bridged to HERMES_SEARCH_SLOW_MS. Salvaged from PR #65544 (adapted to the v23 schema in follow-up commits). |
||
|
Teknium
|
75099ca0ef |
fix(state): inherit git_branch + gateway origin columns on compression children
Follow-ups on top of #64731's cwd/git_repo_root inheritance:
- git_branch joins the parent-row backfill (same NULL-only COALESCE hop):
the Desktop sidebar branch chip otherwise vanishes at every compaction
boundary even though the workspace didn't change.
- Belt-and-suspenders for #59527: compression forks (parent already ended
with end_reason='compression') also inherit the gateway origin columns
(user_id/session_key/chat_id/chat_type/thread_id/display_name/
origin_json) at DB-level child creation. The gateway re-records the peer
after rotation (
|
||
|
Simplicio, Wesley (ext)
|
3c74c12554 |
fix(state): inherit cwd/git_repo_root on parent_session_id children
_insert_session_row never copied cwd/git_repo_root from a parent row when parent_session_id was set, and git_repo_root wasn't even in the INSERT's column list. The compression-fork path (and delegate/subagent spawns, branch continuations) creates a child session without passing cwd/ git_repo_root at all, so the child's tip is born NULL — and since the Desktop project sidebar groups sessions by cwd, the whole project silently drops out of the sidebar every time a long conversation compresses. A lineage that compresses repeatedly compounds this across generations. Add git_repo_root to _insert_session_row's INSERT/COALESCE-on-conflict column set, and backfill both cwd and git_repo_root from the immediate parent row (single non-recursive hop, matching the existing COALESCE "never overwrite an explicit value" contract) inside the same write transaction whenever parent_session_id is set. A multi-generation chain resolves correctly because each generation's own create_session call already backfills from its (already-resolved) immediate parent. Fixes #64709 |
||
|
Teknium
|
9acc4b47f5
|
perf(state): external-content FTS + tool-row-free trigram index (schema v23) (#65798)
* fix(desktop): refresh repo status on session switch with unchanged cwd (#68208) fix(desktop): refresh repo status on session switch with unchanged cwd * fix(checkpoints): honor gateway config and task cwd (#68195) * fix(gateway): wire checkpoint config into agents * fix(checkpoints): resolve gateway file paths by task cwd * ci: live-updating PR review comment with structured job statuses Replace the static comment-pending + comment-results two-job pattern with a live-updating comment system that polls the GitHub Actions API every 15s, re-assembles the review comment from whatever results are available, and upserts it via the <!-- hermes-ci-review-bot --> marker. The comment updates in real time as each job finishes — no waiting for the full pipeline. Every CI job that wants to appear in the review comment emits a review_status output — a JSON array of objects, each with a source and a results array: [ { "source": "review-label-gate", "results": [ {"kind": "action_required", "title": "...", "summary": "...", "how_to_fix": "..."}, {"kind": "info", "title": "...", "summary": "..."} ] }, { "source": "ci timing", "results": [ {"kind": "warning", "title": "CI timings", "summary": "...", "detail": "...", "link": "..."} ] } ] One job can emit multiple results of different kinds. The source field is used to exclude the corresponding job from the synthesized error list (case-insensitive, hyphen-normalized matching against GitHub Actions job display names). | job | source | kind (on failure) | section | |----------------------------|--------------------------|---------------------------|----------------------| | review-labels | review label gate | action_required / info | Action required | | lockfile-diff | lockfile-diff | action_required | Action required | | ci-timings | ci timing | warning / info | Warnings | | supply-chain scan | supply chain | error / (none) | Job failures | | supply-chain dep-bounds | supply chain | action_required / (none) | Action required | | osv-scanner | osv scan | warning / (none) | Warnings | | uv-lockfile-check | uv.lock check | action_required / (none) | Action required | | history-check | unrelated histories | action_required | Action required | | contributor-check | contributor attribution | action_required | Action required | Jobs that find nothing emit [] (empty array) — no noise info items. A single comment-live job polls the GitHub Actions API every 15s, classifies jobs into (completed, pending), assembles the comment, and upserts it. Merges review_status outputs from all needs jobs via toJSON(needs.*.outputs.review_status), and downloads the ci-timings artifact when it becomes available. Shows commit SHA + message below the header. The assembler has ZERO job-specific knowledge. It just: 1. collect_from_statuses() — flattens all nested status objects into ReviewItems 2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status 3. _attach_job_urls() — fills in per-job log links for ALL items 4. render_comment() — groups by severity, renders with group headers Each item shows links inline next to the title: View report (job-emitted URL) and View job (auto-attached logs link). Each info item is its own collapsible <details> block. # ૮ >ﻌ< ა ci review running on abc1234 — commit message first line ## ❌ Job failures ### {title} · [View job](url) {summary} ## ⚠️ Action required ### {title} · [View job](url) {summary} **How to fix:** {how_to_fix} ## ⚠️ Warnings ### {title} · [View report](url) · [View job](url) {summary} {detail} <details><summary>{title}</summary> {content} </details> Still running 3 jobs: ci-timings, docker - test_assemble_review_comment.py (48 tests): collect_from_statuses, collect_failed_jobs with exclude_sources, _attach_job_urls, render_comment (group headers, inline links, commit info, per-item details, pending footer), assemble integration - test_live_comment.py (16 tests): classify_jobs pure function - test_timings_report.py (10 tests): generate_review_status nested format - test_lockfile_diff.py (6 tests) - test_classify_changes.py (32 tests, pre-existing) * ci: migrate AUTOFIX_BOT_PAT to GitHub App token Replace the long-lived fine-grained PAT (AUTOFIX_BOT_PAT) with short-lived (1-hour) installation access tokens minted via a new get-app-token composite action wrapping actions/create-github-app-token@v3.2.0. The PAT was used in 13 spots across 8 workflow files for gh CLI / GitHub API calls. The per-repo GITHUB_TOKEN (1,000 req/hr) was getting rate-limited when multiple workflows fire concurrently (deploy-site, skills-index, ci-timings, supply-chain-audit, js-autofix). App installation tokens get 5,000 req/hr per installation and are scoped to the App's permissions, not a user account. New composite action: .github/actions/get-app-token/ - Wraps actions/create-github-app-token@bcd2ba49 (v3.2.0, SHA-pinned) - Reads APP_ID + APP_PRIVATE_KEY repo secrets - Outputs a 1hr installation token via steps.app-token.outputs.token Requires two new repo secrets (set after creating the GitHub App): - APP_ID: the App's numeric ID - APP_PRIVATE_KEY: the PEM private key App installation permissions needed: contents: write (js-autofix push, pypi release upload) pull-requests: write (js-autofix PR create/merge, supply-chain comment) issues: write (skills-index-freshness issue creation) actions: write (skills-index workflow trigger) workflows: write (skills-index triggers deploy-site.yml) The AUTOFIX_BOT_PAT secret can be deleted once CI passes on this PR. The comment in js-autofix.yml noting that PAT pushes trigger downstream workflows is updated — App tokens have the same property (they are not GITHUB_TOKEN), so the concurrency-cancel loop logic is unchanged. * style(desktop): satisfy merged eslint/prettier config The SSH modules predate the stricter lint config that landed on main (curly, no-empty, perfectionist sorting, prettier). Mechanical lint:fix + fmt pass, empty catch blocks filled with the codebase's void-0 convention, and inline no-control-regex disables on the three deliberate control-char patterns (same pattern as lib/ansi.ts). * fix(ci): pass App secrets as inputs to composite action Composite actions cannot access the secrets context — the runner's template engine rejects secrets.* references at load time with 'Unrecognized named-value: secrets'. Move APP_ID and APP_PRIVATE_KEY from direct secrets.* references inside the composite action to inputs passed by each calling workflow. The fallback logic (GITHUB_TOKEN when APP_ID is empty, for fork PRs) stays in the composite action's check step. * fix(ci): add detect to all-checks-pass needs so its failure blocks merge If detect fails, all downstream sub-workflows get SKIPPED (they have needs: detect). all-checks-pass used if: always() and only checked the sub-workflows — which all showed as 'skipped' (= success) — so it passed even though the root cause (detect) failed. This made the PR mergeable despite a broken CI pipeline. Add detect to all-checks-pass needs so its failure propagates to the gate job and blocks the merge. * fix(desktop): bump skills test timeout to fix cold-start flake (#68235) Test 1 in skills/index.test.tsx pays the full cold-start cost (jsdom env init + module transform + the @/hermes/@/store/profile import graph), which pushed past vitest's 5000ms default under load — caught at 8871ms on one run, 6.6s pure test time on another. Tests 2-4 are ~30-130ms each because all that setup is already cached, so only test 1 was at risk of timing out. Bump the describe-level timeout to 15s. Verified with 10 consecutive runs, 4 of which took 5.5-6.6s of test time and would have hard-failed under the old 5s default. * feat(desktop): open multiple full app windows (electron) Add createInstanceWindow() — a full-chrome peer of the primary that renders the complete app (sidebar, routing, its own draft) against the shared backend, so several GUI windows can run at once. Mirrors the primary's window options + chatWindowWebPreferences (backgroundThrottling stays off so a streamed answer never stalls when blurred) but never overwrites the mainWindow global and doesn't respawn the backend — the renderer's getConnection() joins the running one. New windows cascade off their source via the pure, tested instanceWindowBounds(). Exposed via the hermes🪟openInstance IPC and a "New Window" File menu item. Per-window fullscreen state now targets the window itself, and titlebar/native-theme repaints reach every open chat window instead of only the primary. Retires the now-orphaned compact new-session pop-out (its only caller was ⌘⇧N, repointed in the follow-up commit): drops createNewSessionWindow, the hermes🪟openNewSession handler, and the newSession/new=1 URL flag. * feat(desktop): wire New Window to ⌘⇧N + command palette Repoint session.newWindow (⌘⇧N) from the compact new-session pop-out to openNewWindow(), which opens a full peer instance via the new openWindow bridge, and add a "New Window" entry to the ⌘K palette (shown with its hotkey hint, gated on canOpenNewWindow()). Relabel the action "New window". Drops the retired openNewSessionWindow bridge and the vestigial isNewSessionWindow()/new=1 flag; renames the shared opener helper. * fix(desktop): de-dupe cross-window cues so peers don't spam With multiple full windows, each renderer independently reacts to the same backend event, so one-shot cues fired N times: OS notifications (the per-renderer throttle can't see other windows), the turn-end sound (playCompletionSound runs on every message.complete, ungated by focus), and auto-spoken replies (double voice when a chat is open in two windows). Add a single race-free owner in the main process (electron/event-dedupe.ts): main handles IPC serially, so the first window to claim a key within a short window wins and peers stay quiet. Notifications collapse at the hermes:notify choke point; the sound and spoken replies claim via a new hermes:ambient:claim IPC (keyed by session / reply id). Off Electron the claim falls back to "emit", preserving single-window behavior. The sound's mute check runs before the claim so a muted window can't win the cue and silence an audible peer. * refactor(desktop): tidy the cross-window deduper Drop the unused DEDUPE_WINDOW_MS export and rename its interval so "window" isn't overloaded against BrowserWindow in a multi-window feature (windowMs → intervalMs). DRY the completion-sound play path. No behavior change. * nix: add cage to devDeps * fix(desktop): avoid false remote gateway reauthentication (#68250) * fix(desktop): avoid false remote gateway reauthentication Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): harden remote revalidation state --------- Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): keep composer draft across compression tip rotation (#68079) * fix(desktop): keep composer draft across compression tip rotation Auto-compression swaps the live stored session id while the user may still be typing. Scope the composer/queue key on the lineage root and migrate any tip-keyed draft/queue entries onto that durable key when the tip rotates so the in-progress prompt does not vanish when the response lands. * test(desktop): cover draft survival across compression tip rotation Add regression coverage for migrateSessionDraft, lineage-scoped composer keys, and the rotation path that previously wiped an in-progress draft. * fmt(js): `npm run fix` on merge (#68305) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(desktop): Stop parks the queue instead of firing the next queued prompt Interrupting a busy turn with the Stop button (or Esc) settles the session to idle, and the edge-independent auto-drain immediately submits the head of the composer queue. The user pressed Stop to halt the agent, but it looks like Stop skipped the current turn and kept going — and the queued text is hard to find, since its only surface is the collapsed 'N queued' pill above the composer. The old userInterruptedRef latch (a23728dcc) fixed this but was removed in #40221 because it also suppressed the drain that send-now-while-busy depends on. This reintroduces the halt with source awareness instead of a blanket latch: - Explicit halts (Stop button, composer Esc, chat-focus Esc, the streaming message's hover Stop, runtime cancel) park the session's queue before interrupting. Parked queues are skipped by both auto-drain paths (mounted ChatBar + background drainer). - Interrupts that exist to advance the queue (send-now-while-busy) unpark first, so the settle drain they rely on still flows. - The park lifts on any renewed intent: resume, a manual drain (Enter on empty composer or the per-row send arrow), queueing a new prompt, or emptying the queue. It migrates with entries on a runtime re-key and is deliberately not persisted (a fresh process starts unparked). - The queue panel expands on park, switches to 'N Queued — paused' with a pause icon, and grows a Resume action, so the held prompts are visible instead of reading as vanished. Store contract, hook wiring, and background-drain coverage included; docs updated. * fix(cli,tui): recall real paste content on up-arrow Large pastes collapse to a placeholder in the composer, but input history stored the placeholder — so up-arrow recall showed a truncated reference (CLI) or lost the content entirely (TUI, where the `[[…]]` label has no backing snip after submit). Store the expanded content in history instead: - CLI: `_inline_pastes()` expands `[Pasted text #N -> file]` into the buffer before `reset(append_to_history=True)`; also reused by the external editor (dedup). History nav suppresses re-collapse of recalled content. - TUI: `dispatchSubmission` pushes `expandSnips(pasteSnips)(full)`; idempotent on label-free text so re-submitting a recalled entry stays stable. * fix(cli): suppress CPR on POSIX local TTYs under load Delayed ESC[6n replies leak as ^[[row;colR into the classic CLI on SSH/slow PTYs (#13870) and on local POSIX TTYs under heavy subagent load. Suppress CPR on non-Windows platforms (layout hint only); keep native Windows on prompt_toolkit's default pending native coverage. Wire selection through _select_classic_cli_pt_output. * test(cli): prove local CPR leak and Application CPR-disabled wiring Add a delayed-CPR PTY harness (no SSH) plus selection/Application assertions for POSIX local and Windows preserve-default. Update the gating unit test to the new contract. * refactor: drop platform kwarg, fix PTY test cleanup - Remove redundant platform= test seam from _terminal_may_leak_cpr(); use monkeypatch.setattr(sys, 'platform', ...) consistently in both test files. - Wrap PTY tests in try/finally for fd cleanup on assertion failure. - Guard select.select() in terminal thread against OSError after fd close (fixes PytestUnhandledThreadExceptionWarning). - Trim PR-number reference from test module docstring. * docs(portal): remove retired Nous Chat references * fix(web/ddgs): isolate DuckDuckGo search in a disposable process ThreadPoolExecutor timeouts cannot fire when primp holds the GIL in native code (#68096). Run each search in a child process the parent can terminate/kill, and honor tools.interrupt between polls. * test(web/ddgs): cover GIL-hold timeout, interrupt, and worker reap Regression tests for #68096: native GIL-hold and sleep hooks must time out or interrupt promptly with no orphaned search workers. * fix: sanitize subprocess env for DDGS worker os.environ.copy() passes all Hermes secrets (gateway tokens, API keys, dashboard session tokens) into the DDGS child process. Use _sanitize_subprocess_env() to strip Hermes-managed secrets before spawning the worker. * fix(agent): pass persisted-prefix boundary when rotation flushes on cold resume (#68196) The legacy rotation branch in agent/conversation_compression.py flushes the current turn to the OLD session before ending it (#47202) via _flush_messages_to_session_db(messages) with no conversation_history boundary. On the first turn after a cold Desktop resume, the restored transcript rows live in the message list as plain dicts that have not yet been stamped with _DB_PERSISTED_MARKER — the normal turn flush that stamps them runs after preflight compression. With no boundary, _flush_messages_to_session_db builds an empty history_ids set and treats every restored row as new, durably re-appending the whole transcript to the parent session. Repeated restart/resume + threshold compression keeps growing the parent transcript. Pass messages[:_persist_user_message_idx] (the already-durable prefix that turn_context anchors before preflight runs, guarded for int/bounds) as conversation_history so the flush skips the persisted rows by identity and writes only the current turn's new messages. Adds a regression test that pre-populates SQLite, cold-loads the transcript, appends one current user row, and forces rotating compression: it fails before this change (parent grows to 5 rows) and passes after (parent holds the two originals plus the single new turn). * fix(desktop): prevent contentEditable composer input from visually collapsing to near-zero height Fix #68095 The composer input box (contentEditable div) randomly shrank to a tiny/pixelated size when typing character-by-character (paste worked fine). Root cause: during per-keystroke input, the normalizeComposerEditorDom cleanup could briefly leave the contentEditable with zero child nodes, and without intrinsic content the browser collapsed it visually despite the CSS min-height. Two-pronged fix: 1. Add min-h-[1.625rem] bracket syntax alongside the CSS variable min-height to ensure the minimum height is enforced even if the CSS variable resolution is delayed or overridden by browser defaults. 2. In normalizeComposerEditorDom, ensure the contentEditable always has at least one <br> child when empty, giving it intrinsic height that the browser cannot collapse. This is a belt-and-suspenders approach with the CSS min-height. Closes #68095 * fix(agent): circuit-break AttributeError from commit-splice and detect code skew Fix #68178 The git-install auto-updater rewrites source while the desktop backend is live. Because agent/conversation_loop.py is imported lazily on the first API call, a process can end up running two different commits spliced together — one commit's AIAgent against another commit's conversation_loop. When the interface differs, every turn fails permanently with an AttributeError, and the loop retries indefinitely, burning provider API calls (576 failures, 149 wasted API calls observed). Three-prong fix: 1. Circuit-break AttributeError on agent objects: the outer-loop error classifier now detects AttributeError targeting agent/run_agent modules and breaks immediately instead of continuing the retry loop. 2. Code skew detection for desktop/serve backend: run_agent.py now snapshots the checkout revision at import time and exposes a cheap per-iteration check that the conversation loop uses to refuse new work with a clear 'restart required' message before the lazy import can crash. 3. Informative error message: when code skew is detected, the user gets a clear explanation of the mismatch (boot revision vs current revision) and actionable guidance to restart the application. * fix(telegram): preserve fatal recovery handoff Release the current polling-recovery task's ownership before invoking the fatal-error handler. The runner bounds adapter cleanup in a child task; disconnect() cancels the tracked polling-recovery task, so retaining the current notifier in _polling_error_task would cancel the fatal callback before the runner can finish its reconnect-queue or shutdown decision. The new _handoff_polling_fatal_error() helper clears _polling_error_task only when it is the current notifier. Other recovery tasks remain tracked and are still cancelled and awaited during teardown. Covers both network retry exhaustion and polling-conflict exhaustion. Replaces the misleading "Restarting gateway" message with "Escalating to gateway recovery". Fixes #68406. * fix(telegram): widen fatal handoff to heartbeat watchdog path The wedged-recovery heartbeat watchdog (line 2526) calls _notify_fatal_error() directly from the heartbeat task. disconnect() cancels _polling_heartbeat_task unconditionally (no current_task guard, unlike _polling_error_task). Same bug class as #68406: the child disconnect cancels the heartbeat parent before the runner can queue reconnect. Widen _handoff_polling_fatal_error() to also clear _polling_heartbeat_task when it is the current task, and route the heartbeat watchdog call site through the handoff helper. Co-authored-by: Imgaojp <6065749+Imgaojp@users.noreply.github.com> * fix(tests): make the live-system-guard canary fail closed tests/test_live_system_guard_self_test.py executes real kill primitives (os.kill(-1, SIGTERM), os.killpg, pkill -f python) and depends entirely on the autouse _live_system_guard fixture in tests/conftest.py to intercept them. That makes the canary fail-OPEN: in any collection context where the file is present but its home conftest is not — a published sdist that ships tests/ but not tests/conftest.py, a tree assembled by copying test*.py (that glob does not match conftest.py), pytest --noconftest, or a foreign rootdir — the primitives fire for real, and os.kill(-1, SIGTERM) SIGTERMs every process the invoking user owns (a full desktop-session kill was reported in the field). Add an autouse fixture that refuses to run any canary test unless the guard is provably active. The one thing the canary can detect about its own safety is that the guard monkeypatches os.kill with a plain Python function, whereas the unguarded primitive is a C builtin — so the probe keys off that. Tests marked @pytest.mark.live_system_guard_bypass still opt out, matching the guard's own bypass contract (e.g. test_bypass_marker_disables_guard). With the guard loaded every canary test behaves exactly as before; without it each test refuses at setup with zero side effects. Fixes #68311 * fix(billing): rename user-facing "terminal billing" copy to Remote Spending (#68355) * fix(billing): rename user-facing "terminal billing" copy to Remote Spending The capability was renamed Remote Spending on the portal (consent CTA: "Allow Remote Spending"; per-terminal states Granted/Stopped), but the terminal, desktop, and docs still said "terminal billing" everywhere. - Feature name: Remote Spending in titles/labels, lowercase mid-sentence. - Step-up action verb is now "allow", matching the portal consent CTA. - Kill-switch-off recovery copy points at the actual control ("a billing admin can turn it on from the portal's Hermes Agent page") instead of the dead-end "manage it on the portal". - Per-terminal revoke copy uses the portal vocabulary ("stopped"). - Wire identifiers (cli_billing_enabled, cli_billing_disabled, ...) are unchanged; copy, comments, docs, and test expectations only. * fix(billing): correct the post-step-up denial diagnosis + finish the desktop rename Adversarial review findings: (1) a repeated insufficient_scope after a successful step-up is a per-terminal authorization failure, but the copy blamed the org kill-switch and pointed at the wrong recovery control — now: "Remote Spending still isn't active for this terminal — the authorization didn't take. Retry, or make this change on the portal." (2) the desktop step-up flow started in Remote Spending vocabulary but finished in "billing management access" — renamed both end states. (3) prettier formatting on the touched files (matches the post-merge fmt bot). * feat(tui): show the plan catalog in /subscription on Free (#68357) * feat(tui): show the plan catalog in /subscription on Free The server returns the tier list even with no subscription, but the overlay hid the picker behind can_change_plan && !isFree, so a Free account got only "Start a subscription" with no idea what the plans cost. Now: - Overview on Free offers "Choose a plan" whenever the catalog has enabled paid tiers. - The picker on Free lists each plan as name · price · monthly credits (no upgrade/downgrade hints — there is nothing to move from), and picking one opens the portal, where starting a subscription actually happens (card capture + checkout live there; the upgrade RPC requires an existing subscription). - Paid-plan behavior (preview → confirm → apply) is unchanged. * refactor(tui): compute the picker row suffix once Review feedback: the isFree fork duplicated the label template and run handler; only the suffix differs. * fix(tui): arm the busy guard before the Free portal handoff Adversarial review: the Free branch returned before setting busyRef, so a double-Enter could open the portal twice; and the picker narrated a handoff that openManageLink already narrates (duplicate on success, contradictory on failure). Guard first, let the helper do the talking. * fix(tui): monthly credits are dollars — label them as such The Free picker showed "1000 credits/mo" for what is $1,000 of monthly credit — render "$1,000 credits/mo" (grouped, dollar-signed). * feat(tui): render the Free-plan catalog inline in the /subscription overview Sid ruling: the upsell belongs where the user already is — no intermediate "Choose a plan" hop. On Free the overview lists each paid plan (name · $/mo · $credits/mo) as a pickable row; picking opens the portal (openManageLink narrates). The generic "Start a subscription" row survives only when the catalog is empty. The picker reverts to its original change-only form (Free never reaches it). * feat(desktop): tier catalog chips on the Subscription row Desktop parity with the TUI inline catalog (Sid ruling): accounts that can act see the plans where they already are — Free gets the upsell list (every chip opens the portal), a subscriber sees all tiers with the current one marked inert. Members and team contexts see no chips. Chips learn an optional url (portal handoff) in the shared row model. * chore(tui): fixture harness mirrors the live tier catalog The dev screenshot fixtures showed invented plans ($50 Super / $99 Ultra, "1,000 credits"); align with the real catalog ($20/$100/$200 with $22/$110/$220 monthly credits) so fixture renders cannot be mistaken for product truth. The overlay itself always reads tiers from the subscription API. * chore: trim narration comments * fmt(js): `npm run fix` on merge (#68462) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(relay): attach metadata.user_id on guild replies for egress fallback (#68320) The relay adapter re-attaches an egress discriminator on outbound replies so the connector can resolve the owning tenant. It captured scope_id for scoped (guild) messages and user_id for DMs, but as MUTUALLY EXCLUSIVE: a scoped inbound hit an early return, so the author's user_id was never recorded, and _with_scope only attached user_id when there was no scope_id. Guild replies therefore went out with scope_id only. That's fine while the guild has a provision-time route row. But a MANAGED Discord agent joins guilds dynamically (the shared bot is added to / removed from servers at runtime), and GATEWAY_RELAY_ROUTE_KEYS — the only thing that writes guild route rows — is a self-hosted, static field never stamped for managed agents. So their guild has no route row, the connector's guild-route lookup misses, and with no user_id on the frame there's nothing to fall back to → every guild reply is declined "discord egress declined: target not routed to an onboarded tenant" even though INBOUND resolved the same guild fine (via the author-first SharedSocketRouter.targets() fallback). Fix: capture the authentic author user_id for EVERY inbound (DM and scoped alike) and re-attach it on the outbound reply alongside scope_id. The connector consults it only on a route/scope miss, so carrying both never overrides routing-table resolution. This is the gateway half of the paired gateway-gateway change (makeDiscordTenantOf guild-route-miss author-binding fallback); together they make guild replies resolve the same observed-author way inbound already does. Tests (tests/gateway/relay/test_relay_adapter.py): a guild reply now carries both scope_id AND user_id; a scoped inbound with no author still yields scope_id only (never invents one). Verified fail-without / pass-with. * build: declare pywin32 as a direct win32 dependency hermes_cli/windows_ssh_runtime.py imports win32security/win32file/etc. directly but pywin32 only arrived transitively via concurrent-log-handler -> portalocker. Declare it with a sys_platform gate so the Windows SSH runtime doesn't depend on the logging dep chain. Review follow-up on PR #68130. * fix(desktop): preserve dragging with empty titlebar slots * Revert "fix(agent): circuit-break AttributeError from commit-splice and detect code skew" This reverts commit |
||
|
liuhao1024
|
505fb58751 |
fix(state): add REINDEX strategy to repair stale B-tree indexes (#63386)
When PRAGMA integrity_check reports 'wrong # of entries in index' for B-tree indexes (e.g. idx_sessions_handoff_state), the existing repair strategies (FTS rebuild, sqlite_master dedup, drop-FTS+VACUUM) don't address the mismatch. Add Strategy 0.5: run REINDEX to rewrite the index b-tree from canonical table rows before escalating to more destructive strategies. |
||
|
Teknium
|
373ec23e37 |
fix(state): extend search-path FTS self-heal to the CJK/trigram branch
The trigram MATCH branch in search_messages() had the same OperationalError-only catch that #66420 fixed on the main FTS5 branch: a corrupt messages_fts_trigram shadow table raises the malformed / 'fts5: corrupt structure record' class (sqlite3.DatabaseError, parent of OperationalError), which propagated straight out of search_messages and crashed CJK session/history search for read-only sessions. Route that class through the shared one-shot _try_runtime_fts_rebuild() and retry the trigram query (catch moved outside self._lock so rebuild_fts() can re-acquire it, mirroring the main branch). If the rebuild is refused (guard consumed / FTS disabled / different error) or the retry fails, fall through to the existing LIKE substring fallback — which reads only the canonical messages table — instead of raising, so CJK search degrades gracefully rather than crashing. Adds two regression tests: trigram search self-heals in place after shadow-table corruption (answers from the rebuilt trigram index, not the LIKE fallback), and degrades to LIKE without raising when the one-shot rebuild was already consumed. Follow-up to #66420; refs #66296 #66724 |
||
|
Frowtek
|
11710c51fc |
fix(state): self-heal FTS corruption on the SessionDB search path too
Complements #66296 (self-heal on the write path): search_messages()'s main FTS5 MATCH query caught only sqlite3.OperationalError (a query-syntax error → return empty). A corrupt FTS index raises the malformed / "fts5: corrupt structure record" class, which is a sqlite3.DatabaseError — the parent of OperationalError, so it was NOT caught and propagated straight out of search_messages, crashing session/history search. The write path now rebuilds and retries on that class, but a read-only session (cron/CLI history search, or a search issued before any write) never triggers a write, so its search stayed broken until the next process restart ran the offline repair. Catch the DatabaseError corruption class on the search MATCH read too and route it through the existing one-shot _try_runtime_fts_rebuild(), then retry the query. The catch is moved outside `with self._lock` so rebuild_fts() can re-acquire the lock (mirrors _execute_write). The one-shot guard is shared with the write path, so a single instance never loops on a genuinely unrecoverable index. OperationalError syntax handling is unchanged (caught first). Adds a regression test: with a corrupted messages_fts and no post-corruption write, search_messages() rebuilds in place and returns the match; without the fix it raises DatabaseError. |
||
|
Enough1122
|
96c1511495 |
fix(state): preserve degraded-runtime read probe + use canonical FTS5 classifier
Two follow-ups on top of
|
||
|
Enough1122
|
57b3c477c0 |
fix(state): also catch sqlite3.DatabaseError in FTS5 read probe (#66724)
The FTS5 read probe in _db_opens_cleanly() only caught sqlite3.OperationalError. But the corruption class #66724 actually wants caught — partial shadow-table damage where MATCH / snippet / rank queries raise DatabaseError("database disk image is malformed") — is a DatabaseError, not OperationalError. Without this catch the probe crashes the caller instead of returning a reason, which is exactly the silent-fail mode the issue describes. Move the try/except inside the for-loop so each FTS table is probed independently (one table corrupted should still surface as a reason), add a separate except clause for DatabaseError that surfaces the same reason format, and use continue instead of pass so the loop still walks both tables when only one is missing on a brand-new DB. Tested by hand: with a corrupted messages_fts_trigram shadow table the function now returns 'fts5 read probe failed on messages_fts_trigram: database disk image is malformed' instead of crashing out. Without this fix it would still crash. |
||
|
Enough1122
|
11e76f4e01 |
fix(state): probe FTS5 read path in _db_opens_cleanly so partial index corruption is detected (#66724)
`hermes sessions repair --check-only` opens cleanly on state.db files with partial FTS5 index corruption — base tables read fine, the rolled-back write probe from #50502 succeeds, and `PRAGMA integrity_check` returns "ok". But every session_search / /resume title resolution / feature backed by MATCH / snippet / rank queries errors out with `database disk image is malformed` because internal shadow-table segments are bad. The official repair tool then gives false confidence. Add a representative FTS5 read probe against both `messages_fts` and `messages_fts_trigram` (the latter backs title resolution). Empty MATCH strings are accepted by every FTS5 index without requiring populated content, so the probe is safe on a freshly-init'd DB; missing-table / missing-column errors fall through to the existing "not yet a populated DB" branch, matching the write-probe's behaviour. Any other OperationalError is surfaced as the check reason, which sends `hermes sessions repair` to its existing FTS 'rebuild' path (repair_state_db_schema, line 616). Single-file change in hermes_state.py::_db_opens_cleanly. No public API change. No new imports. Fixes #66724. |