Adds schema v7 'api_call_count' column. run_agent.py increments it by 1
per LLM API call, web_server analytics SQL aggregates it, frontend uses
the real counter instead of summing sessions.
The 'API Calls' card on the analytics dashboard previously displayed
COUNT(*) from the sessions table — the number of conversations, not
LLM requests. Each session makes 10-90 API calls through the tool loop,
so the reported number was ~30x lower than real.
Salvaged from PR #10140 (@kshitijk4poor). The cache-token accuracy
portions of the original PR were deferred — per-provider analytics is
the better path there, since cache_write_tokens and actual_cost_usd
are only reliably available from a subset of providers (Anthropic
native, Codex Responses, OpenRouter with usage.include).
Tests:
- schema_version v7 assertion
- migration v2 -> v7 adds api_call_count column with default 0
- update_token_counts increments api_call_count by provided delta
- absolute=True sets api_call_count directly
- /api/analytics/usage exposes total_api_calls in totals
* feat(state): auto-prune old sessions + VACUUM state.db at startup
state.db accumulates every session, message, and FTS5 index entry forever.
A heavy user (gateway + cron) reported 384MB with 982 sessions / 68K messages
causing slowdown; manual 'hermes sessions prune --older-than 7' + VACUUM
brought it to 43MB. The prune command and VACUUM are not wired to run
automatically anywhere — sessions grew unbounded until users noticed.
Changes:
- hermes_state.py: new state_meta key/value table, vacuum() method, and
maybe_auto_prune_and_vacuum() — idempotent via last-run timestamp in
state_meta so it only actually executes once per min_interval_hours
across all Hermes processes for a given HERMES_HOME. Never raises.
- hermes_cli/config.py: new 'sessions:' block in DEFAULT_CONFIG
(auto_prune=True, retention_days=90, vacuum_after_prune=True,
min_interval_hours=24). Added to _KNOWN_ROOT_KEYS.
- cli.py: call maintenance once at HermesCLI init (shared helper
_run_state_db_auto_maintenance reads config and delegates to DB).
- gateway/run.py: call maintenance once at GatewayRunner init.
- Docs: user-guide/sessions.md rewrites 'Automatic Cleanup' section.
Why VACUUM matters: SQLite does NOT shrink the file on DELETE — freed
pages get reused on next INSERT. Without VACUUM, a delete-heavy DB stays
bloated forever. VACUUM only runs when the prune actually removed rows,
so tight DBs don't pay the I/O cost.
Tests: 10 new tests in tests/test_hermes_state.py covering state_meta,
vacuum, idempotency, interval skipping, VACUUM-only-when-needed,
corrupt-marker recovery. All 246 existing state/config/gateway tests
still pass.
Verified E2E with real imports + isolated HERMES_HOME: DEFAULT_CONFIG
exposes the new block, load_config() returns it for fresh installs,
first call prunes+vacuums, second call within min_interval_hours skips,
and the state_meta marker persists across connection close/reopen.
* sessions.auto_prune defaults to false (opt-in)
Session history powers session_search recall across past conversations,
so silently pruning on startup could surprise users. Ship the machinery
disabled and let users opt in when they notice state.db is hurting
performance.
- DEFAULT_CONFIG.sessions.auto_prune: True → False
- Call-site fallbacks in cli.py and gateway/run.py match the new default
(so unmigrated configs still see off)
- Docs: flip 'Enable in config.yaml' framing + tip explains the tradeoff
Replaces global id +/- 1 context lookup with CTE-based same-session
neighbor queries. When multiple sessions write concurrently, id adjacency
does not imply session adjacency — the old query missed real neighbors.
Co-authored-by: Junass1 <ysfalweshcan@gmail.com>
After a conversation gets compressed, run_agent's _compress_context ends
the parent session and creates a continuation child with the same logical
conversation. Every list affordance in the codebase (list_sessions_rich
with its default include_children=False, plus the CLI/TUI/gateway/ACP
surfaces on top of it) hid those children, and resume-by-ID on the old
root landed on a dead parent with no messages.
Fix: lineage-aware projection on the read path.
- hermes_state.py::get_compression_tip(session_id) — walk the chain
forward using parent.end_reason='compression' AND
child.started_at >= parent.ended_at. The timing guard separates
compression continuations from delegate subagents (which were created
while the parent was still live) without needing a schema migration.
- hermes_state.py::list_sessions_rich — new project_compression_tips
flag (default True). For each compressed root in the result, replace
surfaced fields (id, ended_at, end_reason, message_count,
tool_call_count, title, last_active, preview, model, system_prompt)
with the tip's values. Preserve the root's started_at so chronological
ordering stays stable. Projected rows carry _lineage_root_id for
downstream consumers. Pass False to get raw roots (admin/debug).
- hermes_cli/main.py::_resolve_session_by_name_or_id — project forward
after ID/title resolution, so users who remember an old root ID (from
notes, or from exit summaries produced before the sibling Bug 1 fix)
land on the live tip.
All downstream callers of list_sessions_rich benefit automatically:
- cli.py _list_recent_sessions (/resume, show_history affordance)
- hermes_cli/main.py sessions list / sessions browse
- tui_gateway session.list picker
- gateway/run.py /resume titled session listing
- tools/session_search_tool.py
- acp_adapter/session.py
Tests: 7 new in TestCompressionChainProjection covering full-chain walks,
delegate-child exclusion, tip surfacing with lineage tracking, raw-root
mode, chronological ordering, and broken-chain graceful fallback.
Verified live: ran a real _compress_context on a live Gemini-backed
session, confirmed the DB split, then verified
- db.list_sessions_rich surfaces tip with _lineage_root_id set
- hermes sessions list shows the tip, not the ended parent
- _resolve_session_by_name_or_id(old_root_id) -> tip_id
- _resolve_last_session -> tip_id
Addresses #10373.
After context compression (manual /compress or auto), run_agent's
_compress_context ends the current session and creates a new continuation
child session, mutating agent.session_id. The classic CLI held its own
self.session_id that never resynced, so /status showed the ended parent,
the exit-summary --resume hint pointed at a closed row, and any later
end_session() call (from /resume <other> or /branch) targeted the wrong
row AND overwrote the parent's 'compression' end_reason.
This only affected the classic prompt_toolkit CLI. The gateway path was
already fixed in PR #1160 (March 2026); --tui and ACP use different
session plumbing and were unaffected.
Changes:
- cli.py::_manual_compress — sync self.session_id from self.agent.session_id
after _compress_context, clear _pending_title
- cli.py chat loop — same sync post-run_conversation for auto-compression
- cli.py hermes -q single-query mode — same sync so stderr session_id
output points at the continuation
- hermes_state.py::end_session — guard UPDATE with 'ended_at IS NULL' so
the first end_reason wins; reopen_session() remains the explicit
escape hatch for re-ending a closed row
Tests:
- 3 new in tests/cli/test_manual_compress.py (split sync, no-op guard,
pending_title behavior)
- 2 new in tests/test_hermes_state.py (preserve compression end_reason
on double-end; reopen-then-re-end still works)
Closes#12483. Credits @steve5636 for the same-day bug report and
@dieutx for PR #3529 which proposed the CLI sync approach.
Twelve tests under TestCJKSearchFallback guarding:
- CJK detection across Chinese/Japanese/Korean/Hiragana/Katakana ranges
(including the full Hangul syllables block \uac00-\ud7af, to catch
the shorter-range typo from one of the duplicate PRs)
- Substring match for multi-char Chinese, Japanese, Korean queries
- Filter preservation (source_filter, exclude_sources, role_filter)
in the LIKE path — guards against the SQL-builder bug from another
duplicate PR where filter clauses landed after LIMIT/OFFSET
- Snippet centered on the matched term (instr-based substr window),
not the leading 200 chars of content
- English fast-path untouched
- Empty/no-match cases
- Mixed CJK+English queries
Also:
- hermes_state.py: LIKE-fallback snippet is now
`substr(content, max(1, instr(content, ?) - 40), 120)`, centered on
the match instead of the whole-content default. Credit goes to
@iamagenius00 for the snippet idea in PR #11517.
- scripts/release.py: add @iamagenius00 to AUTHOR_MAP so future
release attribution resolves cleanly.
Refs #11511, #11516, #11517, #11541.
Co-authored-by: iamagenius00 <iamagenius00@users.noreply.github.com>
prune_sessions and delete_session only handled direct children when
satisfying the parent_session_id FK constraint. Multi-level chains
(A -> B -> C) caused IntegrityError because deleting B while C still
referenced it was blocked by the FK.
Fix: NULL out parent_session_id for any session whose parent is about
to be deleted. This orphans children instead of cascade-deleting them,
which also respects the prune retention window — newer child sessions
are no longer deleted just because an ancestor is old.
Reported by Aaryan2304 in PR #6463.
The original PR applied dotted and hyphenated regex quoting in two
sequential steps. For terms with both dots and hyphens (e.g.
my-app.config.ts), step 2 would re-match inside already-quoted output,
producing malformed double-quoted FTS5 syntax.
Merged into a single regex pass: \w+(?:[.-]\w+)+ — handles dots,
hyphens, and mixed terms in one shot. Added test coverage for the
mixed case.
When third-party tools (Paperclip orchestrator, etc.) spawn hermes chat
as a subprocess, their sessions pollute user session history and search.
- hermes chat --source <tag> (also HERMES_SESSION_SOURCE env var)
- exclude_sources parameter on list_sessions_rich() and search_messages()
- Sessions with source=tool hidden from sessions list/browse/search
- Third-party adapters pass --source tool to isolate agent sessions
Cherry-picked from PR #3208 by HenkDz.
Co-authored-by: Henkey <noonou7@gmail.com>
* fix(session-db): survive CLI/gateway concurrent write contention
Closes#3139
Three layered fixes for the scenario where CLI and gateway write to
state.db concurrently, causing create_session() to fail with
'database is locked' and permanently disabling session_search on the
gateway side.
1. Increase SQLite connection timeout: 10s -> 30s
hermes_state.py: longer window for the WAL writer to finish a batch
flush before the other process gives up entirely.
2. INSERT OR IGNORE in create_session
hermes_state.py: prevents IntegrityError on duplicate session IDs
(e.g. gateway restarts while CLI session is still alive).
3. Don't null out _session_db on create_session failure (main fix)
run_agent.py: a transient lock at agent startup must not permanently
disable session_search for the lifetime of that agent instance.
_session_db now stays alive so subsequent flushes and searches work
once the lock clears.
4. New ensure_session() helper + call it during flush
hermes_state.py: INSERT OR IGNORE for a minimal session row.
run_agent.py _flush_messages_to_session_db: calls ensure_session()
before appending messages, so the FK constraint is satisfied even
when create_session() failed at startup. No-op when the row exists.
* fix(state): release lock between context queries in search_messages
The context-window queries (one per FTS5 match) were running inside
the same lock acquisition as the primary FTS5 query, holding the lock
for O(N) sequential SQLite round-trips. Move per-match context fetches
outside the outer lock block so each acquires the lock independently,
keeping critical sections short and allowing other threads to interleave.
* fix(session): prefer longer source in load_transcript to prevent legacy truncation
When a long-lived session pre-dates SQLite storage (e.g. sessions
created before the DB layer was introduced, or after a clean
deployment that reset the DB), _flush_messages_to_session_db only
writes the *new* messages from the current turn to SQLite — it skips
messages already present in conversation_history, assuming they are
already persisted.
That assumption fails for legacy JSONL-only sessions:
Turn N (first after DB migration):
load_transcript(id) → SQLite: 0 → falls back to JSONL: 994 ✓
_flush_messages_to_session_db: skip first 994, write 2 new → SQLite: 2
Turn N+1:
load_transcript(id) → SQLite: 2 → returns immediately ✗
Agent sees 2 messages of history instead of 996
The same pattern causes the reported symptom: session JSON truncated
to 4 messages (_save_session_log writes agent.messages which only has
2 history + 2 new = 4).
Fix: always load both sources and return whichever is longer. For a
fully-migrated session SQLite will always be ≥ JSONL, so there is no
regression. For a legacy session that hasn't been bootstrapped yet,
JSONL wins and the full history is restored.
Closes#3212
* test: add load_transcript source preference tests for #3212
Covers: JSONL longer returns JSONL, SQLite longer returns SQLite,
SQLite empty falls back to JSONL, both empty returns empty, equal
length prefers SQLite (richer reasoning fields).
---------
Co-authored-by: Mibayy <mibayy@hermes.ai>
Co-authored-by: kewe63 <kewe.3217@gmail.com>
Co-authored-by: Mibayy <mibayy@users.noreply.github.com>
feat: persist reasoning across gateway session turns (schema v6)
Tested against OpenAI Codex (direct), Anthropic (direct + OAI-compat), and OpenRouter → 6 backends. All reasoning field types (reasoning, reasoning_details, codex_reasoning_items) round-trip through the DB correctly.
* fix: include ACP sessions in default search sources
* fix: remove hardcoded source allowlist from session search
The default source_filter was a hardcoded list that silently excluded
any platform not explicitly listed. Instead of maintaining an ever-growing
allowlist, remove it entirely so all sources are searched by default.
Callers can still pass source_filter explicitly to narrow results.
Follow-up to cherry-picked PR #1817.
---------
Co-authored-by: someoneexistsontheinternet <154079416+someoneexistsontheinternet@users.noreply.github.com>
Co-authored-by: Test <test@test.com>
_sanitize_fts5_query() was stripping ALL double quotes (including
properly paired ones), breaking user-provided quoted phrases like
"exact phrase". Hyphenated terms like chat-send also silently
expanded to chat AND send, returning unexpected or zero results.
Fix:
1. Extract balanced quoted phrases into placeholders before
stripping FTS5-special characters, then restore them.
2. Wrap unquoted hyphenated terms (word-word) in double quotes so
FTS5 matches them as exact phrases instead of splitting on
the hyphen.
3. Unmatched quotes are still stripped as before.
Based on issue report by @bailob (#1770) and PR #1773 by @Jah-yee
(whose branch contained unrelated changes and couldn't be merged
directly).
Closes#1770Closes#1773
Co-authored-by: Jah-yee <Jah-yee@users.noreply.github.com>
Resolve session IDs by exact match or unique prefix for sessions delete/export/rename so IDs copied from Preview Last Active Src ID
──────────────────────────────────────────────────────────────────────────────────────────
Search for GitHub/GitLab source repositories for 11m ago cli 20260315_034720_8e1f
[SYSTEM: The user has invoked the "minecraft-atm 1m ago cli 20260315_034035_57b6
1h ago cron cron_job-1_20260315_
[SYSTEM: The user has invoked the "hermes-agent- 9m ago cli 20260315_014304_652a
4h ago cron cron_job-1_20260314_
[The user attached an image. Here's what it cont 4h ago cli 20260314_233806_c8f3
[SYSTEM: The user has invoked the "google-worksp 1h ago cli 20260314_233301_b04f
Inspect the opencode codebase for how it sends m 4h ago cli 20260314_232543_0601
Inspect the clawdbot codebase for how it sends m 4h ago cli 20260314_232543_8125
4h ago cron cron_job-1_20260314_
Reply with exactly: smoke-ok 4h ago cli 20260314_231730_aac9
4h ago cron cron_job-1_20260314_
[SYSTEM: The user has invoked the "hermes-agent- 4h ago cli 20260314_231111_3586
[SYSTEM: The user has invoked the "hermes-agent- 4h ago cli 20260314_225551_daff
5h ago cron cron_job-1_20260314_
[SYSTEM: The user has invoked the "google-worksp 4h ago cli 20260314_224629_a9c6
k_sze — 10:34 PM Just ran hermes update and I 5h ago cli 20260314_224243_544e
5h ago cron cron_job-1_20260314_
5h ago cron cron_job-1_20260314_
5h ago cron cron_job-1_20260314_ work even when the table view truncates them. Add SessionDB prefix-resolution coverage and a CLI regression test for deleting by listed prefix.
Add regression coverage for backfilling NULL gateway session models in SQLite, preserving existing models, and forwarding the resolved agent model through SessionStore updates.
Authored by 0xbyt4. Fixes #N/A (no linked issue).
- Sanitize user input before FTS5 MATCH to prevent OperationalError on
special characters (C++, unbalanced quotes, dangling operators, etc.)
- Close SessionDB connection in mirror._append_to_sqlite() via finally block
- Added tests for both fixes
- Empty string titles normalized to None (prevents uncaught IntegrityError
when two sessions both get empty-string titles via the unique index)
- Escape SQL LIKE wildcards (%, _) in resolve_session_by_title and
get_next_title_in_lineage to prevent false matches on titles like
'test_project' matching 'testXproject #2'
- Optimize list_sessions_rich from N+2 queries to a single query with
correlated subqueries (preview + last_active computed in SQL)
- Add /title slash command to gateway (Telegram, Discord, Slack, WhatsApp)
with set and show modes, uniqueness conflict handling
- Add /title to gateway /help text and _known_commands
- 12 new tests: empty string normalization, multi-empty-title safety,
SQL wildcard edge cases, gateway /title set/show/conflict/cross-platform
Two bugs fixed:
1. search_messages() crashes with OperationalError when user queries
contain FTS5 special characters (+, ", (, {, dangling AND/OR, etc).
Added _sanitize_fts5_query() to strip dangerous operators and a
fallback try-except for edge cases.
2. _append_to_sqlite() in mirror.py creates a new SessionDB per call
but never closes it, leaking SQLite connections. Added finally block
to ensure db.close() is always called.
tool_call_count was inaccurate in two ways:
1. Under-counting: an assistant message with N parallel tool calls
(e.g. "kill the light and shut off the fan" = 2 ha_call_service)
only incremented tool_call_count by 1 instead of N.
2. Over-counting: tool response messages (role=tool) also incremented
tool_call_count, double-counting every tool interaction.
Combined: 2 parallel tool calls produced tool_call_count=3 (1 from
assistant + 2 from tool responses) instead of the correct value of 2.
Fix: only count from assistant messages with tool_calls, incrementing
by len(tool_calls) to handle parallel calls correctly. Tool response
messages no longer affect tool_call_count.
This impacts /insights and /usage accuracy for sessions with tool use.
