- export now shares _add_session_filter_args / build_prune_filters with
prune/archive: AGE grammar (5h/2d/1w/ISO) on --older-than plus the full
filter set (--model, --provider, --min-messages, --min-cost, --branch,
--chat-id, ...) for both JSONL and md/qmd bulk exports; --dry-run works
on JSONL too; removes the one-off list_export_candidates helper.
- new --redact flag runs exported message content and tool output through
force-mode secret redaction (agent.redact) for jsonl, md, and qmd.
- docs EN + zh-Hans updated; new tests for AGE grammar, extended filters,
filtered JSONL, and redaction.
Replaces the parallel export-md subcommand from the salvaged commit with
a --format jsonl|md|qmd flag on the existing export subcommand, so all
session export formats share one surface. Adds AUTHOR_MAP entry.
Salvage follow-up for PR #59542 by @web3blind.
Review finding: EXPLAIN QUERY PLAN showed the unconditional repair
UPDATE doing SCAN messages (~75-135ms on 500k rows, every startup) —
the existing idx_messages_session_active can't serve an active-only
predicate. A partial index on (active) WHERE active IS NULL costs
near-zero storage on healthy DBs (no NULL rows) and drops the 0-match
repair to ~0.04ms. Lives in DEFERRED_INDEX_SQL because it references
the reconciler-added column.
The repair UPDATE ('SET active = 1 WHERE active IS NULL') was gated at
schema_version < 12, so already-v12+ databases — the exact population hit
by #51646, where the reconciler-added active column lacks its NOT NULL
DEFAULT 1 — never healed rows written as NULL by the pre-fix INSERTs.
Move the idempotent repair into unconditional startup so historical
gateway transcripts become visible again after upgrading.
Follow-up to the salvage of #59832 by @HexLab98.
append_message() and _insert_message_rows() relied on the schema DEFAULT
for messages.active. Legacy databases that gained the column via ALTER TABLE
without a working INSERT default can store active=NULL, which makes
get_messages_as_conversation()'s active=1 filter drop every gateway turn.
* feat(sessions): full filter surface for prune + new bulk archive subcommand
hermes sessions prune previously only supported --older-than N (integer
days) and --source — no way to target a window like 'the last 5 hours'
(e.g. a batch of CI smoke-test sessions), and no non-destructive option.
- SessionDB.prune_sessions gains keyword filters that AND together:
started_before/started_after epoch bounds, title_like, end_reason,
cwd_prefix, min/max_messages, archived tri-state. Default call is
byte-for-byte compatible (90-day cutoff, ended-only, source).
- New SessionDB.list_prune_candidates (backs --dry-run + confirmation
previews) and SessionDB.archive_sessions (bulk soft-hide via the
existing set_session_archived lineage-aware path; nothing deleted).
- CLI: prune gains --newer-than/--before/--after (durations like 5h/2d/1w,
bare days, or ISO timestamps), --title, --end-reason, --cwd,
--min/--max-messages, --include-archived, --dry-run. New
'hermes sessions archive' takes the same filters, requires at least one,
and is idempotent. Both show a preview before confirming.
- Dashboard /api/sessions/prune accepts the same filters + dry_run.
- Docs: sessions.md + cli-commands.md updated.
Filter parsing lives in hermes_cli/session_filters.py with unit tests;
DB filters covered in tests/test_hermes_state.py.
* feat(sessions): prune/archive filters for model, provider, user, chat, branch, tokens, cost, tool calls
Extends the prune/archive filter surface to everything identifiable in
the sessions table:
- --model (substring on model slug), --provider (exact on
billing_provider, case-insensitive), --user, --chat-id, --chat-type
(exact), --branch (substring on git_branch), --min/--max-tokens
(input+output), --min/--max-cost (USD, actual_cost_usd falling back to
estimated_cost_usd), --min/--max-tool-calls.
- SessionDB prune/archive/list_prune_candidates now share the filter
kwargs via **filters into _prune_filter_where (unknown names raise
TypeError); candidates listing + CLI preview now include the model.
- Any attribute filter (except legacy --source) suppresses the implicit
90-day default so 'prune --model X' matches all ages.
- Dashboard /api/sessions/prune passes the new fields through.
- Docs + tests updated (7 new DB tests, 3 new parser tests).
Follow-up to #9006/#58899. The gateway routing index (session_key ->
SessionEntry) now lives in a new gateway_routing table in state.db as the
primary store; sessions.json is demoted to an optional legacy mirror.
- hermes_state.py: schema v19 — gateway_routing table (scope + session_key
PK; scope = resolved sessions_dir so multiple stores sharing one state.db
never cross-contaminate) with save/replace/load/delete methods
- gateway/session.py: _save() writes the whole index atomically to the DB
(mirrors the old full-file JSON rewrite semantics) and only falls back to
JSON when the DB write fails; _ensure_loaded reads the DB first and folds
in legacy sessions.json entries for keys the DB lacks (pre-migration
import; DB entries win over stale JSON)
- gateway/config.py + hermes_cli/config.py: new write_sessions_json flag
(default true for compat/downgrade safety); gateway.write_sessions_json:
false stops producing the file entirely
- sessions.json _README updated to say it's a legacy mirror + how to
disable it
Rehydration is now lossless across restarts even with sessions.json deleted:
suspended/resume_pending/model_override/token state all round-trip through
the DB (the old sessions-table recovery only rebuilt the bare key mapping).
Moves gateway routing metadata (display_name, origin_json, expiry_finalized)
into state.db, making SQLite the single source of truth for gateway session
discovery. Eliminates the dual-file (sessions.json + state.db) polling
dependency that caused the mcp_serve new-conversation race (#8925).
- hermes_state.py: schema v18 (3 new sessions columns + sessions.json
backfill migration), record_gateway_session_peer gains
display_name/origin_json, new set_expiry_finalized(),
list_gateway_sessions(), find_session_by_origin()
- gateway/session.py: peer recorder persists display_name + full origin
JSON; new SessionStore.set_expiry_finalized() single write-path
- gateway/run.py: expiry watcher success + give-up paths use the store
helper so the flag lands in both sessions.json and state.db
- mcp_serve.py: routing index reads state.db first (sessions.json fallback
for pre-migration DBs); _poll_once collapses to a single state.db mtime
check — the #8925 race is structurally impossible now
- gateway/mirror.py, gateway/channel_directory.py, hermes_cli/status.py:
query state.db first, sessions.json fallback
Closes#9006
Gateway users can now search resumable sessions from messaging surfaces:
/sessions search <query> (alias: find) matches titles and session ids —
including every title/id in a row's forward compression chain, so a
compressed-away title still surfaces its live tip — plus a
punctuation-normalized variant so 'an94' matches 'AN-94'.
Implemented by generalizing the existing id_query chain-filter in
SessionDB.list_sessions_rich into a combined SQL-level filter (search
stays ORDER BY last-active + LIMIT at SQL level), threading a
search_query through the shared query_session_listing helper, and
teaching parse_session_listing_args to split off a search query.
Search results pass through the existing _resume_row_visible guard
unchanged: origin scoping, admin-only 'all', and the fail-closed
legacy-row posture from the July 1 hardening are preserved exactly.
Over-fetch (50) before the visibility cut so origin-invisible matches
can't starve the page.
Salvages the feature direction of PR #57595 by @GodsBoy with a minimal
implementation that keeps the resume authorization surface untouched.
Addresses the egilewski/CodeRabbit and teknium1 reviews on PR #52355.
1) Persisted-row chat scope (egilewski/CodeRabbit). The sessions table stored
only source + user_id, so an identity-bearing caller could resume/list an
INACTIVE persisted row that matched source+user_id but belonged to a
DIFFERENT chat (probe: same user moves `same_user_chat_b` into chat-a).
Persist the messaging origin and compare it:
- schema: sessions gains origin_chat_id / origin_thread_id (declarative
auto-migration via the existing column reconciler).
- SessionDB._insert_session_row accepts + writes the two columns.
- the gateway records them at every origin-bearing creation: both
SessionStore create paths (get_or_create_session + reset/switch) and the
/title path that materializes a store-only session into the DB.
- _resume_target_allowed's identity branch now also requires
origin_chat_id AND origin_thread_id to match the caller. Legacy rows with
NULL origin (created before this change) cannot prove chat origin and
fail closed — resume them via a live session or an admin --all override.
The /sessions listing inherits the fix (non-Matrix rows route through the
same helper).
2) DM key-contract mirror (teknium1). _same_origin_chat's DM branch only
compared user_id and allowed when either side was missing, diverging from
build_session_key (no-chat_id DM keys are built from user_id_alt or
user_id). It now: treats an equal non-blank chat_id as sufficient (the DM
key IS the chat_id when present), and otherwise compares the effective
participant id (user_id_alt or user_id), failing closed on a
missing/different participant so two no-chat_id DM origins are never
conflated.
Tests: add same-user/different-chat (e2e + unit) and chat-scope unit cases;
add DM no-chat_id / user_id_alt / no-identity / same-chat_id cases; update
existing fixtures to record origin_chat_id like the gateway does; make the
cross-room `/resume --all` listing test run as admin (cross-room listing is
admin-gated) and give the boundary-state resume runner a live same-origin so
its post-resume clearing assertions exercise an authorized resume.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up widening the archived-history fix to the sibling save paths the
original PR did not cover. Model switches (_cmd_model, set_session_model) and
_restore mint a fresh AIAgent with _session_db_created=False, so the
agent-owns-persistence guard evaluates False and the blind full-history
replace_messages() fired — DELETEing the durable active=0/compacted=1 rows on
any compressed ACP session (same data-loss class the PR fixes, different
trigger).
- hermes_state.replace_messages: add active_only=True to delete/reinsert only
the live (active=1) rows, leaving soft-archived rows untouched (idea adopted
from the competing PR #50306 by @mrparker0980, credited).
- hermes_state.has_archived_messages: cheap existence probe for active=0 rows.
- acp_adapter._persist: when the agent doesn't own persistence but the session
already has archived rows on disk, replace active-only; otherwise the
destructive full replace stays (fresh create/fork has nothing to lose).
- Regression test: model-switch save on a compacted session keeps the archived
turn discoverable via get_messages(include_inactive=True) + search_messages.
The forked skill/memory review agent shares the parent's session_id for
prompt-cache warmth. Without isolation it wrote its harness turn ('Review the
conversation above and update the skill library…') plus its curator-mode reply
straight into the user's REAL session in state.db; the next live turn re-read
that injected user message as a standing instruction and the agent 'became' the
curator, refusing the actual task.
Root fix: a _persist_disabled flag on the fork that hard-stops every DB write
and lazy-open path (_flush_messages_to_session_db, _ensure_db_session,
_get_session_db_for_recall) — the review writes only to the skill/memory stores
via its tools. Defense-in-depth: _strip_background_review_harness drops any
stray harness message (and the assistant reply that followed) at load time in
get_messages_as_conversation, so an already-polluted session resumes clean.
Salvaged from #50296.
Co-authored-by: arminanton <29869547+arminanton@users.noreply.github.com>
get_messages_as_conversation ordered rows by (timestamp, id). append_message
stamps each row with time.time(), which is not monotonic — on WSL2, after an
NTP step, or when a VM/laptop resumes from sleep the clock can jump backwards
mid-conversation. A later row then carries an earlier timestamp than its
predecessor, so ORDER BY timestamp sorts an assistant tool_calls row after its
tool response, orphaning the tool call and triggering an HTTP 400 on the next
completion. Order by the AUTOINCREMENT id (true insertion order) instead.
This is the sibling path to c03acca50, which already fixed get_messages but
missed get_messages_as_conversation.
Salvaged from #50356.
Co-authored-by: pprism13 <290877921+pprism13@users.noreply.github.com>
The index references the handoff_state column which is added by
_reconcile_columns() on legacy databases. Placing it in SCHEMA_SQL
causes 'no such column' errors during schema migration tests because
SCHEMA_SQL runs before reconciliation.
Move to DEFERRED_INDEX_SQL which runs after _reconcile_columns() —
matching the existing pattern used by idx_messages_session_active.
Refs: #43504, #40695
(cherry picked from commit 40ecd61d49)
The message triggers append one FTS5 segment per insert into both the
porter and trigram indexes. Nothing ever called the existing
optimize_fts() maintenance helper, so on a long-lived state.db these
segments accumulate without bound (observed: ~34k trigram segments for
~27k messages). Every MATCH then has to scan all segments, and every
insert pays a growing automerge cost that lengthens the WAL write-lock
hold time. Because the gateway and cron agents are separate processes
sharing one state.db, those longer holds exhaust the 1s-timeout x 15-retry
budget in _execute_write and surface as repeated:
Session DB creation failed (will retry next turn): database is locked
Session DB append_message failed: database is locked
Wire optimize_fts() into the write path on a coarse cadence
(_OPTIMIZE_EVERY_N_WRITES = 1000), alongside the existing every-50-writes
checkpoint. 'optimize' is effectively free once the index is already
merged, so steady-state cost is negligible; only the first merge of a
neglected index is expensive. The call is best-effort and never fails the
surrounding write.
Tests: cadence fires on the write path; a failing optimize never breaks
the write.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 583647b56e)
Salvaged from PR #35130 (the safe subset of jnibarger01's security pass):
- threat_patterns.py: replace unbounded (?:\w+\s+)* filler with bounded
{0,8} + cap scan input at MAX_SCAN_CHARS (64KiB), and bound the .*
runs in the exfil/config-mod patterns. Kills catastrophic backtracking
on adversarial near-misses.
- hermes_state.py: cap FTS5 query length (MAX_FTS5_QUERY_CHARS) and
extract quoted phrases with a linear scan instead of a regex so
pathological quote runs can't induce backtracking.
- acp_adapter/edit_approval.py + agent/tool_dispatch_helpers.py: recognize
'*** Move File: src -> dst' V4A headers so patch-mode edits are
permissioned/traversal-checked (previously only Update/Add/Delete), and
surface a proposal for mode=patch V4A calls (previously replace-only).
Tests: +ReDoS-bound + FTS5-cap + Move-File-target + V4A-approval cases.
The gateway's get_or_create_session() creates a bare session row (source +
user_id) before the agent exists. The agent's later create_session() carries
the real model/model_config/system_prompt, but _insert_session_row used
INSERT OR IGNORE — silently dropping that enrichment. Gateway sessions were
left with NULL model and NULL billing metadata.
Switch to INSERT ... ON CONFLICT(id) DO UPDATE with COALESCE so NULL columns
get backfilled while values an earlier writer already set are never
overwritten (a later bare write with source='unknown' can't clobber a real
source/model). Credit: original report and fix direction by @LucidPaths (#5048).
On Darwin, synchronous=FULL (the WAL default) only issues a plain
fsync(), which Apple documents does NOT guarantee writes reach stable
storage or stay ordered. SQLite's WAL corruption-safety guarantee
assumes the OS honors the fsync barrier; macOS does not unless the app
uses F_FULLFSYNC. During a launchd *system* shutdown the page cache is
dropped (effectively power-loss for in-flight pages), so a WAL
checkpoint whose fsync 'reported' durable may never hit the platter —
corrupting state.db with a malformed image. That is the trigger in
#30636 ('SIGTERM during launchd shutdown under high load').
Apply PRAGMA checkpoint_fullfsync=1 (macOS-guarded) in
apply_wal_with_fallback. It forces the F_FULLFSYNC barrier only at
checkpoint boundaries (where WAL frames land in the main DB), so cost
amortizes to ~+0.1ms/commit vs ~+4ms for the broader fullfsync=1.
No-op off Darwin (F_FULLFSYNC is macOS-only).
Root-cause analysis by @catapreta on #30636. Supersedes #30654, whose
synchronous=FULL is a no-op (already FULL in WAL mode) and whose
TRUNCATE-on-close is already on main.
Co-authored-by: catapreta <catapreta@users.noreply.github.com>
A readable state.db can still reject every message write through the
messages_fts* triggers when the FTS5 index is corrupt: base-table reads and
PRAGMA integrity_check pass, but INSERT INTO messages fails with 'database
disk image is malformed'. The gateway reloads conversation_history from disk
each turn, so a silently-failed write hands the next turn stale/empty history
even though the same cached AIAgent still holds the live transcript — causing
immediate same-session amnesia. (#50502)
- hermes_state.py: _db_opens_cleanly() now drives a rolled-back message write
through the FTS triggers, so write-only corruption (which the read-only
probe reported healthy) is detected. repair_state_db_schema() gains an
in-place FTS5 'rebuild' strategy (tier 0) before the dedup/drop tiers, plus
an already_healthy short-circuit. Both 'hermes sessions repair' and
'hermes doctor' route through these, so the fix covers the whole class.
- hermes_cli/doctor.py: the state.db check runs the write-health probe even on
the success (readable) path and repairs in place with --fix.
- gateway/run.py: _select_cached_agent_history() prefers the cached agent's
longer live _session_messages over a shorter persisted transcript, so an
FTS write failure can't wipe in-session context.
- tests: regressions for write-health detection, in-place repair preserving
rows + resuming writes, the already_healthy shortcut, and the gateway guard.
Combines the approaches from #50504 (@0-CYBERDYNE-SYSTEMS-0, issue author),
#52165 (@davidgut1982), and #50576 (@trevorgordon981).
Follow-up to the salvage of #45035 + #48682. The two PRs touched different
functions (resolve_resume_session_id vs get_compression_tip) but #45035's
descendant walk followed ANY parent_session_id child, so a delegate/subagent
child could hijack the resume target. Apply the same _branched_from /
_delegate_from / source!='tool' exclusion the rest of hermes_state.py uses,
so the resume walk only follows genuine compression continuations.
Also updates the unrealistic delegation test fixture to carry the real
_delegate_from marker, and updates 3 list_sessions_rich test mocks for the
order_by_last_active kwarg #48682 added.
AUTHOR_MAP: map PINKIIILQWQ + ailang323 salvage authors.
After context compression, the parent session holds pre-compression messages
and a child (or deeper descendant) holds the continuation.
resolve_resume_session_id() short-circuited when the input session already
had messages (row is not None -> return session_id), causing REST API
endpoints, gateway resume, and CLI resume to serve stale parent messages.
Remove the early-return. Walk the full descendant chain, record the
deepest node that has messages (best), and return best if not None
else the original session_id (preserving the empty-chain fallback).
Callers (api_server.py, web_server.py, cli_agent_setup_mixin.py,
cli_commands_mixin.py) all use the resolved != input -> redirect pattern
and are transparent to this change.
The session database records billing_provider and billing_base_url using
COALESCE(column, ?) in update_token_counts(), making them write-once.
When a user switches models mid-session via /model, the runtime (agent.provider,
agent.base_url) updates correctly, but the session row never reflects the new
provider. This causes the dashboard Models page to display a stale provider
badge and misattributes token usage / cost analytics.
Fix: add update_session_billing_route() that unconditionally sets
billing_provider, billing_base_url, and billing_mode (no COALESCE), and call
it from switch_model() in agent_runtime_helpers.py after the swap succeeds.
This follows the same pattern as update_session_model() which already
unconditionally updates the model column (added for the identical COALESCE
problem on the model field).
Closes#48248
When the gateway persists a user message after a transient provider
failure (429/timeout/auth error), subsequent retries of the same
Telegram message could stack duplicate user turns in the transcript,
causing the agent to fall behind by 1-2 messages.
Add has_platform_message_id() to SessionDB (using the existing
idx_messages_platform_msg_id partial index) and a SessionStore wrapper.
The gateway's transient-failure path checks this before
append_to_transcript -- if the platform_message_id is already
persisted, the duplicate write is skipped.
Salvaged from #47869 by @davidgut1982. Adapted to current main which
has additional append sites and an existing content-based dedupe in
the exception handler path.
Closes#47237
Follow-up to #31501. When the send-fallback prune removes a chat's
final telegram_dm_topic_bindings row, also flip
telegram_dm_topic_mode.enabled to 0 in the same transaction.
Without this, a user who turns topics off in the Telegram client
(rather than via /topic off) leaves enabled=1 with zero lanes:
_recover_telegram_topic_thread_id keeps treating the chat as
topic-enabled and lobby messages keep hunting for bindings that no
longer exist. Clearing the flag makes recovery fully stand down once
the dead topics are gone.
Adds 3 regression tests covering the last-binding clear, the
multi-binding no-op, and the unmatched-prune no-op.
Targeted ``(chat_id, thread_id)`` prune for the
``telegram_dm_topic_bindings`` table — the missing piece for
#31501, where the Telegram adapter detects a topic the user
deleted out-of-band but the binding row keeps living in
state.db. The recovery logic in
``gateway.run._recover_telegram_topic_thread_id`` then steers
every future inbound message back to the dead topic, dropping
tool progress, approvals and replies into the wrong place.
Returns the number of rows deleted; silently no-ops when the
topic-mode tables haven't been migrated yet (read-only / pristine
profile) so the helper is safe to call from a send-fallback
hot path before the schema has run.
_collect_delegate_child_ids() walks the _delegate_from marker chain to
gather delegate subagents for cascade deletion, but started its visited
set empty. When the chain loops back onto a parent — a delegation cycle,
or a parent that is also another parent's delegate child when several ids
are deleted together — that parent was collected as one of its own
descendants and then permanently deleted, along with all of its messages,
by _delete_delegate_children().
Seed the visited set with the parent ids so they can never be re-collected,
and exclude them from the returned child set. Callers (delete_session,
bulk delete) remove the parents separately, so this only prevents the
unintended parent deletion; legitimate child collection is unchanged.
Add regression tests (in-memory sqlite) covering single/multi-level
delegate chains, the parent_session_id+marker branch, untagged children
(orphan-don't-delete contract), and the cycle case that previously leaked
the parent into the deletion set.
Fixes#49148
Follow-up to the soft-archive durability fix. Reusing the rewind/undo active=0
flag for compaction-archived turns inherited the wrong search semantics: undo
rows are intentionally HIDDEN from session_search (the user took them back), but
compaction-archived turns must stay DISCOVERABLE — that is the whole point of
Teknium's "searchable / recoverable" requirement. As built, search_messages
defaulted to WHERE active=1, so after in-place compaction the pre-compaction
turns were in the FTS index but filtered out of the default search. (The earlier
"searchable" claim only held for a raw FTS query / include_inactive=True, not
the actual session_search tool.)
Empirically confirmed the gap: search 'HMAC' returned 2 hits before compaction,
1 after (only the summary's mention) — the originals were hidden.
Fix — a `compacted` flag distinct from `active`, giving a 3-way state:
- active=1, compacted=0 → live context (normal)
- active=0, compacted=1 → compaction-archived: OUT of live context, IN search
- active=0, compacted=0 → rewind/undo: OUT of live context, OUT of search
Changes:
- messages.compacted INTEGER NOT NULL DEFAULT 0 added to SCHEMA_SQL. Declarative
_reconcile_columns adds it on existing DBs — no version bump (plain column add).
- archive_and_compact: UPDATE … SET active=0, compacted=1 (was active=0 only).
- search_messages: default WHERE active=1 → (active=1 OR compacted=1), on BOTH
the main FTS5 path and the trigram CJK path. include_inactive=True still
returns everything. The short-CJK LIKE fallback already returns all rows
(no active filter) — unchanged.
- Docstrings on archive_and_compact + search_messages document the 3-way state.
Verified: after compaction, session_search default finds the archived originals
(ids 1 & 4); rewind/undo rows stay hidden by default (recoverable via
include_inactive); live context still excludes both. 322 in-place + hermes_state
tests and 46 session_search tests green; ruff clean. Mutation check: reverting
the search WHERE to active-only fails the new searchable test.
(Surfaced by the question "is search semantic or only FTS?" — answer: session
search is FTS5 keyword/BM25 only, no embeddings over the transcript; semantic
retrieval lives in the optional memory-provider layer. Tracing that confirmed
the active-only filter gap above.)
Teknium review: keeping one durable session id must NOT come at the cost of
destroying history. The prior in-place implementation used replace_messages,
which hard-DELETEs the pre-compaction turns (they also drop out of the FTS
index) — same id, but the original conversation is gone with no recovery path
and the summary becomes the only record. Rotation today is non-destructive
(the old session's full transcript survives under the old id); in-place must
match that durability contract, not weaken it.
Fix: compact in place by SOFT-ARCHIVING, reusing the existing messages.active
flag (the /undo soft-delete mechanic), instead of deleting:
- New SessionDB.archive_and_compact(session_id, compacted): in one atomic
write, UPDATE messages SET active=0 on the live turns, then insert the
compacted set as fresh active=1 rows. Nothing is deleted.
- The insert loop is extracted into a shared _insert_message_rows() helper so
archive_and_compact and replace_messages don't duplicate the 60-line
column/encoding block (extend-don't-duplicate).
- Agent in-place branch calls archive_and_compact instead of replace_messages.
Durability outcome (proven by test + E2E across repeated compactions):
- Live context load (get_messages_as_conversation / get_messages) filters
active=1, so a resume reloads ONLY the compacted set — compaction still
shrinks the live session.
- The pre-compaction turns stay on disk at active=0, recoverable via
get_messages(include_inactive=True) / restore_rewound.
- They remain FTS-searchable: the messages_fts* triggers index on INSERT and
remove on DELETE only — they do NOT key on active, and active=0 is a
content-preserving UPDATE. session_search still finds them.
- Verified across TWO successive compactions: the 1st compaction's originals
are still recoverable + searchable after the 2nd (answers the "no recovery
path after the next compaction" concern directly).
message_count now reflects the LIVE (active/compacted) count, matching the
live load. replace_messages keeps its DELETE semantics (still correct for
/retry, /undo) and gains a docstring note pointing compaction at the
non-destructive method.
Tests: test_in_place_keeps_same_session_id strengthened to assert the 8
seeded originals survive at active=0 alongside the 2 compacted rows AND stay
FTS-searchable. Mutation check: swapping archive_and_compact back to a hard
DELETE fails the test, so the non-destructive contract is bound. 285
hermes_state + in-place tests green; rotation/persistence/compress-command/cli
suites green; ruff clean.
_is_compression_ancestor walked parent links in a 100-hop Python loop
issuing two SELECTs per hop and hand-re-encoded the compression
continuation edge a fourth time. Collapse it into a single recursive CTE
that reuses the canonical _COMPRESSION_CHILD_SQL fragment (already shared
by _ephemeral_child_sql and set_session_archived), so the edge definition
lives in exactly one place. The UNION recursion also dedups visited nodes,
making it cycle-safe without the defensive hop cap. Behavior is unchanged
(all TestSessionTitleLineage + existing title-command tests pass).
When context compression rotates a session, the original is ended and the
continuation is auto-numbered (e.g. "name" -> "name #2"). The session list
projects the ended root behind its live tip, so the user never sees the
predecessor. But set_session_title's uniqueness check compared against ALL
sessions, so renaming the visible tip back to "name" dead-ended with
"Title 'name' is already in use by session <id the user can't find>".
When the conflicting title is held by a compression ancestor of the session
being renamed, transfer the title instead of raising: clear it from the
ended predecessor and apply it to the continuation. Uniqueness is preserved
(still exactly one session carries the title) and the parent-link lineage is
untouched, so resume-by-title and tip projection keep working. Genuine
conflicts with unrelated sessions, and with non-compression children
(delegate/branch), still raise as before.
- Scope 'no such tokenizer' matcher to trigram specifically (#779)
- Decouple base FTS and trigram backfill in v11 migration (#1195)
- CJK search falls back to LIKE when trigram unavailable (#3384/#3430)
- Add _trigram_available tracking across init, migration, and startup
- Add regression tests for migration backfill and CJK LIKE fallback
- Add _is_trigram_unavailable_error and _warn_trigram_unavailable helpers
_is_fts5_unavailable_error only matched 'no such module: fts5', but
SQLite builds that ship FTS5 without the optional trigram tokenizer
raise 'no such tokenizer: trigram' instead. This caused SessionDB init
to crash on those builds.
Additionally, the trigram failure path called _warn_fts5_unavailable
which set _fts_enabled = False, globally disabling full-text search
even though the base FTS5 table was created successfully.
Fix:
- Extend _is_fts5_unavailable_error to also match 'no such tokenizer'
- Add _is_tokenizer_unavailable_error to distinguish tokenizer-specific
failures from whole-module absence
- Only call _warn_fts5_unavailable for module-level failures; skip it
for tokenizer-specific failures so base FTS5 remains usable
Fixes#47002
Auto-compression ends the live session and forks a continuation child
(linked via parent_session_id). A long-lived parent keeps its own flushed
message rows, so resolve_resume_session_id()'s empty-head walk never
redirected it — resuming the parent id reloaded the pre-compression
transcript and dropped every turn generated after compression, including
the assistant's response. On the desktop this is the recurring "I sent a
message, came back, and the reply isn't there" report on large sessions:
the chat's routed id is the pre-rotation id, and both the gateway
session.resume RPC and the REST /messages read anchored on it.
Fix the resolver at the chokepoint: resolve_resume_session_id() now
follows the compression-continuation chain forward via get_compression_tip()
before its existing empty-head descendant walk. get_compression_tip() only
follows children whose parent ended with end_reason='compression' (created
after the parent was ended), so delegation/branch children never hijack a
resume. This fixes every resume caller at once (REST /messages, CLI
--resume, gateway /resume).
session.resume in tui_gateway was the one resume path that never called the
resolver — it used the raw target id directly. Route it through
resolve_resume_session_id() too (non-lazy only; lazy watch windows must
stay on their exact child branch). Resolving up front also re-anchors the
live-session fast path so a still-live rotated session is reused by its new
key instead of rebuilding a duplicate agent on the stale parent.
Tests:
- resolve_resume_session_id follows the tip even when the parent retains
messages, and is not confused by a delegation child.
- session.resume binds the agent to the continuation tip and returns the
post-compression reply.
Consolidates these related Amy fork patches:
- 429830f39 feat(gateway): inject message timestamps into user messages for LLM context
- 3c3d6fac0 fix: handle both ISO string and epoch float timestamps in history replay
- 2874f7725 feat: human-friendly timestamp format with weekday and timezone name
- 3735f4c8b fix: render gateway message timestamps once
* feat(desktop): session-scoped status stack + kill new-window theme flash
Stack subagents, background tasks, and the queue into one collapsible
"sink" above the composer, reusing the queue's chrome so every status
reads as one piece. Extracts shared StatusSection / StatusRow /
TerminalOutput primitives and a unified $statusItemsBySession store
(subagents mirrored, background owned here, merged + grouped for render).
Renames BrailleSpinner → GlyphSpinner now that it drives more than braille.
Separately, fix the white flash on every new/cmd-clicked window: macOS
`vibrancy` paints an NSVisualEffectView that follows the OS appearance and
ignores `backgroundColor`, so a dark app on a light-mode Mac flashed white
until the renderer painted over it. Pin `nativeTheme.themeSource` to the
app theme (persisted to userData so cold launches paint right before the
renderer loads), hold windows with `show:false` until `ready-to-show`, and
pre-paint the themed background via an inline script before the bundle runs.
* feat(desktop): dock the slash popover to the composer via one shared fill var
The slash·@ popover (and ? help) now docks onto the composer's edge with the
same chrome as the queue/status stack — rounded outer corners, fused borderless
edge, no shadow — but keeps its own narrow width.
Surface + drawer paint a single --composer-fill var; the state ladder
(rest / scrolled / focused / drawer-open) lives once in styles.css on
[data-slot='composer-root']. The :has() drawer-open rule is last and forces an
opaque fill, since translucent glass sampling different backdrops (thread vs
fade gradient) can never match. This replaces the focus-within !important
override that repainted the surface behind every previous matching attempt.
Also drop the chevron column from the project file tree — the folder open/closed
icon already carries the expand state.
* feat(desktop): base inset for file tree rows (post-chevron alignment)
* feat(desktop): wire the status stack's background tasks to the real process registry
The background group was UI-only (dev-mock seeded). Now it's live e2e:
- tui_gateway: new session-scoped `process.list` (registry snapshot filtered
by the session's session_key, plus a 4KB output tail for the inline
terminal viewer) and `process.kill` (single process, ownership-checked —
unlike process.stop's kill_all).
- Renderer: `reconcileBackgroundProcesses` syncs snapshots into the store
layout-stably — rows keep their position when state flips (never re-sort),
new processes append, unchanged rows keep object identity so memoised rows
skip re-rendering, and a dismissed-set stops the registry's retained
finished procs from resurrecting X-ed rows.
- Refresh triggers: session open, terminal/process tool.complete,
status.update(kind=process) from the gateway's notification poller, and a
5s poll armed only while a running row is visible (catches silent exits).
- Stop = real `process.kill` + optimistic dismiss; Dismiss = client-side
with resurrection guard.
- Re-keyed the stack to the RUNTIME session id: it was keyed by the stored
session id, where neither subagent events nor process.list would ever land.
- Deleted dev-status-mocks.ts (__hermesStatusMocks) — no more seed shit.
Reconcile invariants covered in store/composer-status.test.ts.
* feat(desktop): todos + openable subagents in the status stack, self-healing file tree
- todo lists move out of the inline chat panel into the composer status stack
(checklist icon, dashed ring = pending, spinner = in progress, check = done),
fed live from todo tool events and seeded from history on session open
- subagent rows carry the child's real session id end-to-end
(delegate_tool → gateway → renderer) so clicking one opens ITS session window
- status stack publishes its measured height so the thread's bottom clearance
grows with it; card paints the shared --composer-fill so focused/scrolled
states match the composer exactly
- file tree self-heals: ENOENT roots retry on a 3s cadence + Try again button,
and the main process expands ~ in IPC paths (gateway cwds arrive as ~/...)
- composer drag-drop of tree entries inserts inline refs instead of attachments
* fix(desktop): file tree falls back to the workspace dir when a session's cwd is gone
Sessions record their launch cwd; deleted worktrees leave that path dead,
so opening such a session swapped the tree from the default workspace to a
directory that ENOENTs forever — the 3s retry just spun on it. On a root
read error the tree now asks main to sanitize the cwd (prefers the
configured default project dir), displays that fallback, and quietly
re-probes the original path so it switches back if the dir reappears.
* feat(desktop): working restore-checkpoint button on past user prompts
The discard icon on hover of a past user bubble was decorative — clicking
did nothing. It's now a real control: a confirmation dialog explains that
everything after the prompt is removed, then the session rewinds to that
turn and reruns the same prompt (prompt.submit with
truncate_before_user_ordinal, the same mechanism the edit composer uses).
Failures rethrow into the dialog's inline error instead of toasting.
* fix(desktop): show the restore-checkpoint button on the latest user prompt too
Restoring the most recent prompt is just 'retry this turn' — no reason to
exclude it. Stop still takes the slot while the turn is running.
* fix(desktop): finished todo lists clear themselves out of the status stack
A list whose every item is completed/cancelled lingers ~4s so the final
checkmark is visible, then the todo group drops out of the stack. A fresh
active list arriving within the linger cancels the scheduled clear.
* chore(desktop): drop dead editableCheckpoint copy, terser restore confirm
* fix(desktop): rewind clears the abandoned timeline's todos + background
Restoring to (or editing) an earlier prompt rewinds the conversation, but
the todos and background processes spawned by the now-discarded turns kept
showing in the status stack — and the real background processes kept
running. Both rewind paths now clear the session's todo rows and kill +
drop its background processes before the fresh run repopulates them. Also
drops the click-to-edit clamp transition, which flashed a half-expanded
bubble on the way into the edit composer.
* feat(desktop): user messages are always editable; edit/restore revert mid-stream
The bubble is now always click-to-edit — even while a turn streams — instead
of going inert during a run. Sending an edit acts like restore: it rewinds to
that prompt and re-runs with the new text. Both edit and restore can fire
mid-stream now; the gateway refuses prompt.submit while a turn runs (4009
"session busy"), so they interrupt the live turn first and retry the submit
until the cooperative interrupt winds it down. Restore (re-run as-is) shows on
every prompt except the latest running one, which keeps the Stop button.
* fix(desktop): label preview-pane ⌘L selections with the filename, not "zsh"
The terminal owns a global ⌘/Ctrl+L "send selection to composer" shortcut, so
selecting text in the file preview pane and hitting it fell through to the
terminal handler — which imported the right text but labelled the composer ref
"zsh:N lines" off the shell name. When the selection isn't an xterm selection,
label it with the previewed file instead.
* fix(desktop): ⌘L on a preview line selection inserts the @line ref, like dragging
The source preview lets you select lines in the gutter and drag them into the
composer as an @line:path:start-end ref. ⌘/Ctrl+L now does the same when a line
selection is active — it drops the identical ref instead of falling through to
the terminal's global handler (which grabbed the native text selection and sent
a bogus terminal block). Capture-phase + stopPropagation so it wins; with a line
selection there's no native selection, so the terminal handler stays out of it.
* chore: gitignore apps/desktop/demo/ scratch output
The desktop demo prompt writes demo/*.txt during recorded walkthroughs; it's
throwaway, never part of the app. Ignore it so it stops cluttering git status.
* feat(desktop): subagent watch windows, hard stop, sidebar hygiene
Child-session mirror for live subagent windows, delegate sessions tagged
and excluded from the sidebar, composer focus/stop polish, and WS stall
resilience on the gateway transport.
* refactor: DRY delegate SQL + trim status-stack noise
Extract shared listable-child and delegate-delete helpers in hermes_state,
collapse cancelRun busy release, and cut comment bloat in resume/status paths.
* fix(desktop): hide orphaned subagent sessions in sidebar
Cascade-delete all ephemeral children on parent delete (not just tagged rows),
run v16 backfill to tag legacy orphans, and record new delegates as source=subagent.
* fix: restore orphan contract for untagged children + lazy session eviction
Cascade-delete only _delegate_from-tagged rows (v16 backfill covers legacy),
walk marker chains recursively with FK-safe orphaning, gate lazy watch
sessions out of the still-starting eviction exemption via an explicit flag,
pass session_id to _make_agent only when resuming, and hide source=subagent
from session search.
* fix(gateway): gate child mirror off upgraded sessions + age out stale run entries
Review findings: the mirror could interleave synthetic events with a real
native stream once a watch window upgrades (prompt.submit builds an agent),
and a lost subagent.complete left _active_child_runs pinning running=true
forever. Mirror now stops when the live session owns an agent; liveness
reads ignore entries older than an hour.
* fix(gateway): reject prompt.submit into a watch session while its child runs
A lazy watch session's running flag is False (the run lives in the parent
turn), so typing mid-run sailed past the busy guard and built a second agent
racing the in-flight child on the same stored session. Busy error until the
run completes; afterwards the submit upgrades into a normal conversation.
* refactor(gateway): DRY watch-resume payload + compose listable-child SQL
Fold the duplicated child-run busy overlay into one _reuse_live_payload
helper across both resume reuse paths, collapse the twin mirror early-returns,
and build _LISTABLE_CHILD_SQL from _BRANCH_CHILD_SQL instead of restating it.
* fix(desktop): clip horizontal overflow on sidebar scroll areas
Add overflow-x-hidden alongside overflow-y-auto on session list scrollers
and the shared SidebarContent primitive — vertical scroll unchanged.
Drop the hermes_state.py column + persistence plumbing from the salvaged
interleaved-thinking fix. The ordered-block channel covers the failure
window in-memory (turn replayed within the live conversation loop). A
session reloaded from disk after a crash falls back to reconstruction;
if that replay 400s, the thinking-signature recovery (#43667) strips
reasoning_details and retries — one degraded call in a rare resume path
instead of a schema column. Replaces the DB-roundtrip test with a
fallback-shape test.
Interleaved-thinking turns (adaptive thinking, Claude 4.6+/Opus 4.8) emit
content blocks like:
thinking_1(signed) tool_use_1 thinking_2(signed) tool_use_2
Anthropic signs each thinking block against the turn content preceding it
at its position. normalize_response split the turn into two parallel lists
(reasoning_details + tool_calls), discarding cross-type order, and
_convert_assistant_message rebuilt it as [all thinking][text][all tool_use].
That moved thinking_2 ahead of tool_use_1, invalidating its signature, so
Anthropic rejected the latest assistant message with HTTP 400:
messages.N.content.M: `thinking` or `redacted_thinking` blocks in the
latest assistant message cannot be modified.
Observed repeatedly in agent.conversation_loop against api.anthropic.com /
claude-opus-4-8, recurring across sessions on multi-thinking-block turns.
Fix: carry a verbatim, order-preserving copy of the turn's content blocks
(anthropic_content_blocks) end-to-end - capture in normalize_response,
persist/restore through state.db, and replay unchanged for the latest
assistant message. Gated to turns that actually interleave signed thinking
with tool_use, so normal turns are unaffected.
Adds 3 regression tests including a SQLite round-trip covering the
crash-recovery reload path.
Port from google-gemini/gemini-cli#27770: starting the CLI and
immediately quitting (or rotating with /new, /clear) left an empty
untitled session row behind. These ghost rows pile up in /resume,
`hermes sessions list`, and the in-chat recent-sessions browser.
- SessionDB.delete_session_if_empty(): transactional check-and-delete
that only removes rows with no messages, no title, and no child
sessions (delegate subagent parents are preserved). Also removes
on-disk transcript files via the existing _remove_session_files.
- HermesCLI._discard_session_if_empty(): thin wrapper, wired into the
cli_close shutdown path and the new_session() rotation path.
Skipped when /exit --delete already handles removal.
Unlike the one-shot prune_empty_ghost_sessions migration (TUI-only,
24h-old rows), this prevents new ghost rows from accumulating at the
moment they would be created.
