Surface the session's first-class Project in both chat surfaces: the
Desktop status bar (project name as the workspace label, full cwd in the
tooltip) and the TUI status label + /status output.
One source of truth. The per-profile projects.db is the authority, read
in tui_gateway via _project_info_for_cwd (backed by
projects_db.project_for_path) and threaded through every session.info
emission path the TUI consumes. The Desktop already caches that truth in
$projectTree, so it DERIVES the label from it (projectNameForCwd) instead
of carrying a second per-session $currentProject atom fed from
session.info.
That drops the parallel state #64721 introduced and the entire
reset/reconciliation surface it required (resume, agentless cwd.set,
gateway-switch, fresh-draft): the label is purely derived, so it stays
correct whenever the cwd or the project tree changes. Only explicit,
named projects resolve on both surfaces, so an auto-discovered repo root
keeps the cwd-leaf label everywhere.
Excludes the unrelated markdown shell-fence change bundled in #64721.
The PR added api_content to get_messages_as_conversation's inline SELECT
but missed the shared _CONVERSATION_ROW_COLUMNS constant used by
get_resume_conversations — _rows_to_conversation references
row["api_content"] but the column wasn't in the SELECT, causing
IndexError on resume-session tests.
Co-authored-by: Soju06 <qlskssk@gmail.com>
Deduplicates the sidecar handling across 9 sites:
- substitute_api_content(): 2 API-bound pop+substitute sites
(chat_completion_helpers, transport — conversation_loop keeps its
inline pop because the current-turn compose fallback needs the value)
- drop_stale_api_content(): 4 rewrite-drop sites
(context_compressor x2, replay_cleanup, agent_runtime_helpers)
- extract_api_content_sidecar(): 3 gateway forwarding sites
(gateway/session, gateway/slash_commands, cli_commands_mixin)
Also: restore the eager _pending_cli_user_message clear on the early
row-creation try (was lost when the crash-persist moved after prefetch —
a crash in compression between the two tries would leave a stale staged
input), and fix a comment indentation nit in run_agent.py.
Co-authored-by: Soju06 <qlskssk@gmail.com>
The first LLM call of every gateway turn gets ~0% provider prompt-cache
hit rate (in-turn calls: 97-99%) because the bytes sent for a turn's
user message are not the bytes replayed next turn: memory-prefetch and
pre_llm_call context are injected into the API copy only, the #48677
persist override writes cleaned content to the DB row, and
get_messages_as_conversation sanitize/strips user and assistant content
on load. Any of these diverges the request prefix at that message and
re-prefills everything after it — measured 27.9s for the first call vs
2.4-5.8s cached at a median ~156k-token context.
Persist what you send: a nullable messages.api_content column stores the
exact content string sent to the API when it differs from the clean
stored content, and replay substitutes it verbatim (no sanitize, no
strip). The injection composition lives in one helper
(turn_context.compose_user_api_content); the turn prologue stamps its
output onto the live user message, the api_messages build sends the
stamped bytes, and every outgoing copy pops the field so it never
reaches a provider. The crash-resilience user-turn persist moves after
prefetch/pre_llm_call so the user row is written once with its final
sidecar; _ensure_db_session stays before preflight compression (session
rotation needs the parent row under PRAGMA foreign_keys=ON). The
current-turn index trackers are re-anchored after compaction rebuilds
the message list, in-place preflight compaction backfills the stamp onto
the already-inserted row, and gateway replay forwards the sidecar only
when the replay pipeline did not rewrite the content. Rewrite paths that
would leave stale bytes (historical image strip, merge-summary-into-
tail, consecutive-user repair merge, stale-confirmation redaction) drop
the sidecar; the chat-completions transport and the max-iterations
summary path strip it defensively. codex_app_server and MoA turns are
excluded from stamping because their wire bytes differ from the
composition. A missing or dropped sidecar degrades to today's behavior
(one cache-boundary miss), never to wrong content.
The busy/priority/monitor/backup×2/drain paths each had near-identical
try/except blocks for transcribe+echo with only log_context, adapter,
and metadata varying. Extract into _transcribe_and_echo_pending_voice
which handles the cache lookup, echo dedup, and exception logging in
one place.
Uses a _UNSET sentinel to distinguish 'caller did not pass metadata'
(use the rich _thread_metadata_for_source fallback) from 'caller
explicitly passed None' (monitor/backup/drain paths that use the
simpler thread_id-only dict).
~120 lines of copy-paste collapsed into one 30-line helper.
merge_pending_message_event extends the existing event's media_urls in
place when two media-bearing messages arrive in quick succession (photo
bursts, consecutive voice messages). The gateway runner caches STT
transcripts on the event via _gateway_pending_stt_text; if the cached
event gains new media after the cache was populated, the stale transcript
was returned instead of transcribing the merged attachments.
Add _invalidate_pending_stt_cache() and call it from both media-merge
branches in merge_pending_message_event so the next transcription call
re-runs against the full merged media list.
Closes the merge-race edge case identified during review of PR #61519.
The function was introduced in d55304c39 but never had a single caller —
verified via git log -S and search_files across the whole repo. The
drain path at _stream_confirmed_final_delivery inlines its logic directly.
Keeping a dead function around that duplicates live logic is a maintenance
hazard: future changes to the live path won't propagate to the dead one.
Extract the inline pure-tool-call tail check to a named helper using
flatten_message_text (canonical content extraction). Fix a SQLite
durability regression: the incremental tool-call persist
(conversation_loop.py:4990) stamps _DB_PERSISTED_MARKER on the assistant
row, so the next _persist_session flush skips it — the filled content
reaches the in-memory transcript but NOT the durable store, and /resume
reloads content="". Pop the marker so the next flush re-writes the row.
Tests pass, ruff clean.
`finalize_turn` guarantees the invariant "delivered final_response =>
assistant row in transcript" (#43849 / #44100) for recovery paths that
return a response without appending a closing assistant message. It
enforces it by checking only the tail's ROLE:
if _tail_role != "assistant":
messages.append({"role": "assistant", "content": final_response})
A tail that is a *pure tool-call turn* — `assistant(tool_calls=[...])`
with no text of its own — satisfies that check while carrying none of the
delivered answer. The append is skipped and the response is never
persisted, so the durable transcript ends at an assistant row the user
never saw as the reply. On the next turn the model replays the user
backlog and re-answers it: exactly the symptom the block was added to
prevent, just reached through a different tail shape.
Observed with the real finalizer: with messages ending
`user -> assistant(content="", tool_calls=[t1])` and
`final_response="Here is your answer."`, the persisted transcript keeps
`content=""` and the answer is absent.
Fill that row's empty content instead of appending. This keeps the
invariant without disturbing the tool-call structure and without creating
an assistant->assistant pair. A tail tool-call row that already carries
model text is left untouched, so no model output is ever overwritten.
Adds regression tests for both: the empty tool-call tail is filled, and a
tool-call tail with existing text is not clobbered.
session.resume built two projections of the same lineage with two separate
get_messages_as_conversation calls — the model-fed copy (tip rows, alternation-
repaired) and the display copy (full lineage, verbatim). The display fetch
already reads a superset of the model fetch (the tip rows are part of the
lineage), so the second call re-scanned the same messages.
Add SessionDB.get_resume_conversations(): one lineage SELECT, split into the tip
(model) and full (display) projections in memory via the extracted
_rows_to_conversation helper. Byte-identical to the two separate reads
(test_get_resume_conversations_matches_separate_reads covers a lineage with a
dangling tool-call tail so repair diverges the two lengths, a single session,
and replayed-user dedup). Both session.resume build paths (deferred + eager) now
use it; the subagent single-read path is unchanged.
From the Desktop performance audit (P1: "Remove repeated resume transcript
work") — the gateway half. The renderer's concurrent REST prefetch is left as-is
(the audit flags dropping it as measure-first).
The sidebar refresh fired three /api/profiles/sessions calls (recents, cron,
messaging), and each one reopened every selected profile's state.db and re-ran
list_sessions_rich + session_count — ~3N DB opens/counts per refresh, on every
turn/broadcast/reconnect.
Add GET /api/profiles/sessions/sidebar: one pass that opens each profile DB
once and runs the three source-scoped queries together (recents scoped to the
active profile; cron + messaging cross-profile), returning the three windows in
one payload. Same read-only projection, 300s active heuristic, and caller-
supplied source taxonomy (recents_exclude / messaging_exclude / source=cron) as
the per-slice endpoint.
Renderer refreshSessions now makes one listSidebarSessions call and distributes
recents/cron/messaging to their stores (cron *jobs* stay a separate getCronJobs
API). Electron splices remote profiles per slice via fetchProfilesSessionSlice
(reusing the proven per-slice merge) so remote correctness is preserved; the
no-remote common case gets the single-open fast path.
From the Desktop performance audit (P1: "Batch sidebar session slices").
The Telegram gateway could go silently deaf for hours: the reconnect ladder
stalled mid-way (e.g. "attempt 4/10, reconnecting in 40s" then nothing) while
the process stayed active(running), so Restart=always never fired.
Root class: every recovery path — the ladder's re-entry
(_schedule_polling_recovery), the pending-update probe (_probe_pending_updates),
and PTB's error callback — gates new recovery on _polling_error_task.done(). If
that single task wedges on any hung await, all recovery returns early forever
and nothing retries.
The heartbeat loop is a separate task, so make it an independent, cause-agnostic
watchdog: if the same recovery task stays in-flight past
_POLLING_ERROR_TASK_STUCK_TIMEOUT (300s — well beyond a healthy ladder attempt's
bounded stop+drain+start+backoff), force a retryable-fatal so the background
reconnector rebuilds the adapter instead of relying on the frozen ladder. This
guarantees progress regardless of *where* the stall is (issue direction #1),
tracked locally so no task-assignment site needs to change.
Also salvages @koduri-mahesh-bhushan-chowdary's #66492 (drain-await timeout),
which closes the one concrete wedge vector documented in the incident
(_drain_polling_connections' unbounded shutdown()/initialize() on a wedged
CLOSE-WAIT pool). The watchdog covers the rest of the class.
Co-authored-by: Koduri Mahesh Bhushan Chowdary <mkoduri73@gmail.com>
_drain_polling_connections() awaited polling_req.shutdown() and
.initialize() without a timeout. When the getUpdates httpx connection is
wedged on a stale CLOSE-WAIT socket, that close can block forever, hanging
_handle_polling_network_error (the tracked _polling_error_task). The task
never completes, so every escalation path — _schedule_polling_recovery,
_probe_pending_updates, the heartbeat verifier — stays gated behind its
in-flight guard, the ladder freezes mid-way, _set_fatal_error is never
reached, and Restart=always never fires: the gateway is alive but silently
dead.
Wrap both drain awaits in asyncio.wait_for with a new module-level
_DRAIN_TIMEOUT (15.0s, matching _UPDATER_STOP_TIMEOUT), mirroring the
existing bounded stop()/start_polling() sites. On timeout the drain logs and
continues, so the handler task completes and the ladder always advances
toward the fatal-restart escalation.
Adds test_reconnect_continues_if_drain_hangs, which wedges the drain and
asserts the handler still reaches start_polling within a hard bound.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Direct-Anthropic requests used a single shared _anthropic_client, and the
stale/interrupt watchdog closed + rebuilt it from the poll (stranger) thread
at four sites (non-streaming stale/interrupt, streaming stale/interrupt).
Closing a client whose TLS socket a worker thread was still reading released
the FD from a stranger thread; the kernel recycled it under a live SSL BIO,
which then wrote a 24-byte TLS record into an unrelated SQLite header
(cron/executions.db), bricking every cron on the profile. Same shape as the
OpenAI-only #29507 fix, but the Anthropic path never got the owner-thread
contract.
Extend the #29507 ownership contract to Anthropic: build a per-request client
(_create_request_anthropic_client), register it with the request-client
holder tagged by kind, and route _close_request_client_once by kind — a
stranger thread only shuts the request client's sockets down
(_abort_request_anthropic_client), while the owning worker performs the SDK
close (_close_request_anthropic_client). The shared _anthropic_client is now
never closed from inside a request (streaming or non-streaming), including the
worker retry-cleanup sites, since each attempt builds a fresh request client.
The #28161 no-hang guarantee is preserved: the poll-thread socket abort
unblocks the worker immediately.
Salvages the approach from #51688 (@raymondyan-zhijie), reimplemented onto
current main (non-streaming dispatch was refactored into
_dispatch_nonstreaming_api_request; streaming grew _cancel_current_stream_attempt
and worker retry-cleanup sites). Tests updated to the request-local mechanism
(incl. replacing a banned source-reading test with a behavior test) plus new
regression coverage proving the watchdog aborts the request client and never
touches the shared client.
Co-authored-by: raymondyan-zhijie <32435458+raymondyan-zhijie@users.noreply.github.com>
Two structural fixes from the Desktop performance audit (P2 tier):
1. Scope live tool-diff subscriptions. `ToolEntry` subscribed to the whole
`$toolDiffs` map via `useStore`, so one `recordToolDiff` re-rendered every
mounted tool row. Add a cached per-toolCallId derived atom
(`$toolInlineDiff(id)`, mirroring the existing `$toolDisclosureOpen` pattern);
computed() only notifies when that id's diff string changes, so a live patch
re-renders one row.
2. Narrow profile / gateway-switch query invalidation. Both the active-profile
subscription and `wipeSessionListsForGatewaySwitch` called keyless
`queryClient.invalidateQueries()`, refetching account/marketplace/onboarding
caches on every switch. Add `invalidateProfileScopedQueries()` with a
denylist of profile-independent roots (billing, marketplace-themes,
onboarding-model-options, contrib-logs-tail). A denylist is correctness-safe:
a root we forget just refetches (cheap), whereas an allowlist that misses a
profile-scoped key would paint the previous profile's data.
Tests: per-tool notify isolation, and real-QueryClient invalidation partition
(profile-scoped invalidated, global left intact, unknown keys invalidated).
The system prompt building code hardcoded '~/.hermes' paths instead of
using get_hermes_home(). When HERMES_HOME is set to a custom location,
the prompt text still referenced ~/.hermes, confusing the AI about
where files actually live.
Changed:
- Import get_hermes_home from hermes_constants
- Default profile hint: ~/.hermes/profiles/<name>/ → <home>/profiles/<name>/
- Non-default profile hint: ~/.hermes/... → <home>/... for all paths
Closes#66450
_resolve_task_provider_model() read api_key from the auxiliary task
config but never consulted key_env (or api_key_env). When a user
configured an auxiliary task with key_env instead of a plaintext
api_key, the resolved API key was None, causing 401 on every call.
Add the same key_env → os.getenv() resolution pattern already used in
_fallback_entry_api_key() and named custom provider resolution.
Closes#66641
Hot-reload and multi-entry load_hermes_dotenv can hit the same UTF-32
file repeatedly; gate the refuse-to-mangle warning on a module-level
seen-set (house style: _WARNED_KEYS sibling) so logs are not spammed.
Notepad "Unicode" saves write UTF-16 with a BOM. The sanitizer decoded
those bytes as utf-8-sig with errors=replace, glued U+FFFD onto the first
key name, stripped NULs, and rewrote the mangled content permanently.
Sniff leading BOMs before any text decode (UTF-32 before UTF-16, because
UTF-32-LE's BOM starts with UTF-16-LE's FF FE). Decode UTF-16 correctly
and rewrite as clean UTF-8. Refuse UTF-32 (leave untouched + warning).
After errors=replace, do not persist a first line that starts with U+FFFD.
Does not touch _load_dotenv_with_fallback (#65124's surface).
_quote_env_value previously left internal spaces unquoted (only #/"/'
and leading/trailing whitespace triggered). Spaced macOS paths written
via hermes setup SSH / Google Chat SA path / hermes config set produced
lines that python-dotenv still parsed but shell `set -a; . file` word-split.
Extend needs_quoting with any(c.isspace()); escaping dialect unchanged.
The char-level streaming fuzz runs 12 seeds × 500 growing prefixes
(~6000 full+cached lexes) and first trips the pre-fix boundary at
seed 11 / step 257, so the workload can't shrink without gutting the
guard. The work is bounded but exceeds Vitest's 5s per-test default on
CI workers under parallelism, so give this one test an explicit 30s
timeout instead of weakening coverage.
The salvaged unit tests hand-construct completed futures to lock the
_run_on_mcp_loop contract. Add a live-loop test that reproduces the actual
field trigger: an inner asyncio.wait_for expiry stores a real TimeoutError
on a real future scheduled on the MCP loop. Asserts the fixed loop surfaces
it once, promptly -- not spinning to the outer deadline (which both leaked
memory and masked the real error behind the generic wrapper message).
A commit added a bullet and an em-dash inside two Write-Host/Write-Info string
literals in scripts/install.ps1. The file has no UTF-8 BOM, so Windows
PowerShell 5.1 (which the bootstrap runs the cached script under) reads it in
the system ANSI code page (CP1252), not UTF-8. The em-dash's UTF-8 tail byte
decodes to a smart close-quote (U+201D) that the tokenizer treats as a string
delimiter, prematurely closing the string and desyncing the parser -- surfacing
as the reported cascade of syntax errors at lines 1619/1770 and aborting the
Windows GUI installer before it does anything.
Non-ASCII bytes in '#' comments are harmless (skipped to end-of-line), so the
file carried em-dashes in comments for months; only the two chars in code
broke it. Convert all non-ASCII to ASCII equivalents (em-dash -> '--', already
this file's own comment convention; bullet -> '-') and add a source-level test
locking the pure-ASCII invariant, since Linux CI cannot run the PS installer.
Fixes#66994Fixes#67000
The streaming block splitter added in #67154 dropped only the previous
parse's trailing whitespace blocks plus its LAST content block before
re-lexing the appended suffix. That boundary is unsound: a trailing
Setext underline (`-`/`=`) underlines the paragraph ABOVE it, so
appending to it can retroactively merge the previous parse's last TWO
blocks into one.
Minimal repro: cached "…#e\n5\n-" lexes to [ …, "#e\n", "5\n-" ], but
grown to "…#e\n5\n-p2=kj:c" collapses "#e"/"5\n-" into a single block.
The reused settled prefix still contained a stale "#e\n" block. The
`blocks.join('') === text` guard can't detect this because the wrong
split reconstructs the same source string, so the divergence rendered
as mis-split blocks with no fallback.
Fix: drop the last TWO content blocks (skipping whitespace-only blocks
around them) before re-lexing the suffix. The block before the last is
the deepest an append can reach — a Setext underline consumes exactly
one preceding block — and earlier blocks stay fenced off by settled
blank lines, so re-lexing two is sufficient and safe.
Tests: a deterministic regression for the exact prev→grown pair, and a
character-level streaming property fuzz (12 seeds × 500 growing prefixes
over the markdown control alphabet). Both fail on the pre-fix boundary
and pass after. tsc/eslint/prettier clean; markdown-text suite green.
Bug 1 of #55048: when the MCP connection dropped (driver crash / restart),
_lifecycle_coro exited but left _started=True, so the next list_apps/capture
passed _require_started() and then operated on a None session — hanging
forever instead of reconnecting.
- _lifecycle_coro's finally now resets _started=False on ANY exit, so a dead
session is re-enterable (idempotent no-op on the normal stop() path; atomic
bool write, safe from the bridge-loop thread without the lock stop() holds).
- call_tool() re-enters start() when the session isn't active, rebuilding it
before the call. The start_session/end_session handshake (driven by start()/
stop() themselves) is exempted so bootstrap doesn't recurse.
Tests: two cases in test_computer_use_delivery_ladder.py — finally resets
_started, and call_tool restarts a dead session exactly once. Full
computer_use suite green (233).
Refs #55048 (Bug 1). Bug 2 (expose foreground dispatch) is covered by the
delivery_mode work in #67123.
Fourth profiling round (#66033/#66347/#66470). Streamdown's
parseMarkdownIntoBlocks is a full `marked` lex of the entire message,
and during streaming every flush is a new string — so the splitter paid
O(full-text) ~30x/s: benchmarked 3.4-9.6ms per call at 64-192KB, i.e.
15-30% of a core burned re-lexing settled text on long agent replies.
(The rest of the May-2026 "re-parse elephant" was already eaten by
tailBoundedRemend, block-memo, the KaTeX memo, and deferred shiki —
the splitter was the last O(full-text) pass besides preprocess, which
benches at only 1.5-4.6ms and is left alone.)
New src/lib/markdown-blocks.ts (extracted from markdown-text.tsx) wraps
the splitter with two caches:
- the existing exact-string LRU (remounts: virtualizer scroll, session
switch) — moved, unchanged;
- a streaming-append cache: when the new text startsWith a recently
parsed text, reuse that parse's blocks up to a settled boundary and
lex only the suffix. The boundary drops trailing whitespace-only
blocks plus the last content block — the only block appended text can
reinterpret (open fence, list/table continuation, setext underline,
lazy blockquote). Earlier blocks are separated by settled blank lines
and can't change. Cross-block reference links can't regress:
Streamdown already renders each block as an independent document.
Safety: the splitter's blocks.join('') === text property makes offsets
exact and is defensively re-checked; any mismatch or non-append rewrite
(edit, branch swap) falls back to the full lex — byte-for-byte the old
behavior.
Property tests: at every random streaming cut over a corpus covering
fences, loose lists, tables, setext headings, lazy blockquotes, HTML
blocks, and display math — and token-by-token through a fence boundary
— the cached splitter's output is asserted deep-equal to a fresh full
lex. Bench (128KB reply, 200 flushes): 990ms -> 72ms total splitter CPU
(4.95 -> 0.36ms/flush).
Verification: tsc clean; eslint/prettier clean; lib + assistant-ui
suites green (473 tests).
_make_tui_argv() called _ensure_tui_workspace(tui_dir) unconditionally
before checking for a prebuilt bundle. That function sys.exit(1)s when
ui-tui/ doesn't exist, which it never does on a pip/pipx install — the
wheel ships hermes_cli/tui_dist/entry.js but never ships ui-tui/ at all
(that directory only exists in a git checkout).
Every dashboard Chat tab connection on a pip/pipx install therefore
hard-exited before ever reaching _find_bundled_tui(), surfacing as the
unhelpful "Chat unavailable: 1" banner despite having a fully valid
bundled entry.js on disk.
Move the bundled-wheel/HERMES_TUI_DIR shortcut ahead of the workspace
check. --dev is unaffected (it never uses the bundled path and still
requires the workspace), and the checkout-without-bundle path is
unaffected (bundled lookup returns None, falls through to the existing
git-restore/npm-install/build flow).
Adds a contributors/emails mapping for the original author.
Fixes#56665
Co-authored-by: lucaskvasirr <lucaskvasir@duck.com>
Fixes the two review defects that kept PR #29923 open, plus docs:
- gateway: exclude --once from the session-store write-through. The
once-override lived only in memory before, but the write-through
persisted it, so a gateway restart before the finally-restore
rehydrated a supposedly one-turn model permanently.
- TUI: skip _sync_agent_model_with_config while a one-turn restore is
pending. The once-model is deliberately not pinned as a session
model_override, so the config sync saw a model mismatch and clobbered
the once-override back to the config model before the turn ran.
- tests: real _handle_model_command drive asserting --once never
touches set_model_override while --session still does; restore-pop
idempotency.
- docs: /model --once in configuring-models.md with an honest
prompt-cache cost note (one-shot switch breaks the cached prefix
twice; wins for short sessions and cheap-to-expensive escalation).
Adds --once to /model across CLI, TUI, and gateway: switch model for the
next turn only, restoring the previous model in a finally block so
success, exception, and interrupt all revert. Parsing extends
parse_model_flags_detailed(); resolve_persist_behavior() treats --once
as a persistence opt-out; --global + --once is rejected.
Salvaged from PR #29923 (image-generation lane split to #59815 per
review; conflict resolution against current main by the maintainers).
