An agent told to work in a fresh git worktree does exactly that — creates
it, cds in, and runs every later command there — but the session stayed
pinned to the checkout it started in. The desktop kept labelling the chat
with the primary branch while all the work landed somewhere else.
The desktop half already existed: session.info carrying a moved cwd runs
followActiveSessionCwd, which refreshes the project tree and scopes the
sidebar into the new project. The backend just never reported the move.
Reconcile the session's cwd against terminal_tool's per-session record at
the end of a turn, when the agent has stopped moving and its recorded cwd
is a stable answer. A plain cd stays what it always was — not a workspace
move — so the reconcile only fires when the recorded cwd sits in a
different git working tree than the session's workspace.
CuaDriverBackend caches a long-lived cua-driver subprocess for the life of
the Hermes process, and stop() was never called from anywhere — the driver
outlived the session that spawned it. #69903 stopped the orphan from pegging
a core by disabling the cursor overlay, but left the process behind; this is
item 3 of #28152 ("Hermes does not keep the driver alive after tool
completion").
Register an atexit hook, mirroring browser_tool's
atexit.register(_emergency_cleanup_all_sessions). atexit only, no signal
handlers, for the prompt_toolkit reason documented there. reset_backend_for_tests
now reuses the same teardown instead of repeating it.
Fold the null fallback into chatSurfaceRoot so both callers share one
target resolution, hoist the var names next to it, and drop the
duplicated rationale from the status-stack comment.
The existing `stream` scenario measures streaming into an empty thread, which
is exactly the case where per-delta transcript work does not show up. This adds
`--historyTurns`, which mounts a settled transcript and lets it drain BEFORE
the recorders start, so the measurement window contains streaming work only —
and `stream-history`, a report-only preset that turns it on.
Report-only (tier: manual) rather than gated: how much history a host can mount
varies, so the absolute number is not comparable across machines. It is meant
for same-machine before/after runs.
Co-authored-by: Jakub Wolniewicz <frizikk@users.noreply.github.com>
Once the coalescing floor had already elapsed, the next delta was scheduled
with requestAnimationFrame. Chromium pauses rAF for a renderer it considers
hidden, and that is not something this code can verify: backgroundThrottling:
false and the process-level switches in electron/main.ts cover the blurred and
occluded cases, but not a minimized window, a fully off-screen one, or a
renderer the compositor has otherwise parked. In those states the callback is
accepted and never runs, so a finished answer sits in the queue until some
later focus or input event happens to wake a frame — the reply looks stalled,
then lands all at once on refocus.
Always use a timer. The coalescing cadence is unchanged (the floor above is
what enforces it), timers are clamped rather than suspended in background
renderers, and disable-background-timer-throttling already opts out of that
clamp. The teardown path loses its cancelAnimationFrame branch with it.
The regression test parks rAF the way an occluded renderer does and asserts
the delta still arrives. It has to send a delta, let it flush, then idle past
the floor before the delta under test, because the frame-gated branch was only
reachable on that second scheduling — a single-delta version of this test
passes on main and proves nothing.
Co-authored-by: NetRunner2037 <rerdi92@users.noreply.github.com>
The composer and the out-of-flow status stack published their measured
heights onto document.documentElement, but both components mount once per
chat surface. Session tiles render a full ChatView beside the workspace
pane, so N surfaces raced for one value: a background tab with a tall
status stack inflated the foreground thread's bottom clearance and pushed
its jump-to-bottom button into mid-screen. Whichever surface unmounted
last also cleared the var for everyone still showing.
Publish onto the surface's own root instead, and re-declare the clearance
calc there — `:root` substitutes the root measurements once, so scoping
only the inputs would leave every thread reading the same value.
The workspace pane read the global $messages/$busy atoms — a mirror of
whichever session was active — while every ⌘T tile rendered from its own
$sessionStates slice. With two turns in flight, navigating away from a
still-streaming session left it painting into the surface now showing a
different conversation: the wrong transcript under the right route.
Point the primary view at the active session's own slice, keeping the
global atoms as the draft surface for a chat that has no runtime id yet.
Previews correct themselves on read, but a title already written to the DB
stays wrong. This regenerates those titles from what the user actually typed,
dry-run by default.
Two guards, both hit on a real store: a candidate that isn't title-shaped is
rejected rather than replacing a serviceable title with command output, and a
unique-title collision dedupes through the lineage the way the live auto-titler
does instead of leaving the leaked title in place.
`preview` is the head of the first user message and the title fallback on
every surface — sidebar rows, pickers, exports, the desktop's sessionTitle().
An untitled /skill session therefore read `[IMPORTANT: The user has invoked
the "work" skill, indicatin...` wherever it appeared.
A scaffolded row now selects a wide enough excerpt to reach the typed
instruction (head + tail spliced for a long body) and shapes it through
describe_skill_invocation(). Because previews are computed on read, existing
sessions are corrected without a migration.
The six copies of the preview subquery and four copies of its shaping collapse
into one expression and one helper along the way, and the /rewind picker gets
the same treatment.
generate_title() sent the first 500 characters of the user turn to the
auxiliary model. On a /skill invocation those characters are the skill's own
opening prose, so the session got named after the skill instead of the request
— /work sessions came back as "Isolated Git Worktree Setup".
Route the turn through describe_skill_invocation() first, so the titler sees
what the user typed. Also keep only the first line of the response: a model
that ignores "return ONLY the title" and answers the prompt would otherwise
have a shell transcript stored as the title, truncated mid-command.
A /skill invocation expands into a message that embeds the whole skill body.
Anything that summarizes a user turn from its raw content reads the skill's
prose as if the user had written it.
describe_skill_invocation() sits next to the existing extractor and reuses its
markers, returning `/work — fix the title leak` for an invocation with an
instruction and `/work` for a bare one. It also exports the SQL LIKE pattern
and excerpt-joint sentinel that listing queries need to recognize scaffolding
before a row reaches Python.
syncRepositoryIncrementally rewrote the entire transcript on every adapter
snapshot: one addOrUpdateMessage per message, a second full export to find
deletions, and an unconditional resetHead. During streaming that fires ~30x a
second, so the per-delta cost scaled with how long the conversation already
was — the reported symptom.
Now that settled messages keep reference identity, an identity check is a
sound "did this change?" test. Write only the items whose message or parentId
moved, and skip resetHead when the head did not move (it prunes the head's
descendants, so calling it needlessly is not free). Anything the fast path
cannot prove safe — a disjoint session swap, a changed message count, an id
with no repository entry — falls through to the original full rebuild.
Tests cover the perf contract (one write for one delta, zero for a no-op) and
the correctness cases the fast path must not break: appends, authoritative
deletion, disjoint session replacement, branch re-parenting, and an explicit
headId rewind. The two perf assertions fail on main.
Co-authored-by: Jakub Wolniewicz <frizikk@users.noreply.github.com>
useRuntimeMessageRepository keeps a WeakMap of converted messages so a
settled turn converts once. Building the export with
ExportedMessageRepository.fromBranchableArray threw that away again: it maps
the whole array through fromThreadMessageLike on every call, so each streamed
delta re-normalized the entire settled history and handed the runtime a fresh
object for every message.
Normalize on the cache miss instead, using the same fallback status
fromBranchableArray applies, and build the export literal directly. Settled
turns now keep reference identity across deltas, which is what lets the
reconcile below tell that only the tail moved.
Co-authored-by: Jakub Wolniewicz <frizikk@users.noreply.github.com>
The nonzero-exit guard already knew a failing exit code with real output
usually isn't a failure worth painting red. It only looked at output /
stdout / stderr, but background-process polls report their text under
output_preview — so a poll of any process that exited nonzero rendered
destructive-red with no error to show, even when the output was routine
npm chatter. Half the red process rows in a real session history were
this case.
Sentinel-released processes exposed a second bare-sample assertion in the
same file. The unread-dot check ran a synchronous .count() 140ms after the
running dot cleared:
15.91s poll "dot should disappear" -> 2
16.05s ... -> 0 (running dot gone)
16.05s bare .count() for unread dot -> 0 FAIL
"Finished — unread" is an event-driven transition that lands just after the
running dot clears. The old fixed `sleep 5` happened to leave enough slack
between the two that a single sample usually caught it; releasing the
process deterministically removed that incidental slack and made the latent
race deterministic instead.
Poll for it, matching how sidebar-states.spec.ts already asserts this exact
dot. The split-tile assertion at line 234 stays a bare sample on purpose —
it asserts an absence (toBe(0)), where polling would only wait for something
that must never appear.
The cross-session sidebar specs asserted a state that could expire before
they looked at it, making them the flakiest tests in the suite — two reds
on unrelated PRs within three minutes on 2026-07-26.
Root cause, from the failing run's trace: the tests need a background
process that is still RUNNING after the agent turn finishes, but the
process was a fixed `sleep 5` racing two other clocks — the turn itself
(two model round trips plus a real subagent delegation) and the 4s
success linger before a finished task auto-dismisses. On a loaded runner
the "dot should appear" poll took 7.5s to see the dot; by then `sleep 5`
had already exited, `waitForFunction(finalText)` returned in 0.08s
because the turn was long done, and the next line — a bare synchronous
`.count()`, not a wait — sampled 0.
The process lifetime is now test-controlled: `createBackgroundReleaseHandle()`
mints a sentinel path, the scripted command blocks until that file
appears, and the test releases it exactly when it wants the dot to clear.
One clock instead of three, and the "turn done, process still running"
state is stable rather than a window to catch. The wait is bounded (60s)
so a forgotten release can't hang a worker, and `sleep 5` stays as the
default for callers that pass no handle.
No product code touched — E2E harness only.
Memory rows fell back to the auto-derived name, so they read as a bare
"Memory" / "Running memory" next to a generic icon. Add the tool to
TOOL_META with proper copy across all five locales, plus a solid brain
glyph in the existing Phosphor fill set.
A memory tool call rendered destructive-red with an alert glyph and, when
expanded, a raw dump of the whole args + result payload.
Two causes. The red came from toolStatus treating any error text as a
failure: the memory store rejects an over-budget batch so the agent can
retry a smaller one, which is routine bookkeeping, not something the user
acts on. Those now land on the existing amber warning tier.
The payload dump came from memory having no case in toolSubtitle or
toolDetailText, so both fell through to the generic stringify-everything
fallback. Both now surface just the human-readable line.
Reported in Discord by @spherohero: `sessions recover --allow-partial` copied
20,817 of 20,824 message rows, then orphan cleanup deleted every one of them
because no session row was salvageable. Final output: 0 sessions, 0 messages.
The salvage worked and then threw the result away -- the exact opposite of
what --allow-partial exists to do.
_cleanup_partial_orphans() removed dependent rows whose session_id had no
matching sessions row. That is correct when a few sessions are lost; it is
catastrophic when the sessions b-tree is damaged worse than messages, which
is the common shape (sessions is small and hot, messages is large).
Reproduced exactly: 500 readable messages, unreadable sessions -> 500
copied, 500 removed, empty output.
Now _reconstruct_missing_sessions() runs FIRST, inside the same transaction,
synthesizing a minimal row per orphaned session_id (only id/source/started_at
are NOT NULL). started_at comes from the earliest surviving message.
Placeholders carry source='recovered' and an explicit title so a fabricated
session can never be mistaken for an original. Same repro now retains all
500 messages under 1 reconstructed session.
Reconstruction is reported as LOSS, not a clean recovery: session metadata
(title, model, timestamps, cost) is genuinely gone even though the
conversation text survived. loss_detected=True, partial=True,
complete=False, with a warning naming the counts.
Also fixes total_removed_or_relinked, which summed every dict value and would
now have counted retained messages as removed.
Sabotage-verified: removing the reconstruction call restores the wipe and
fails the test. 942 targeted tests green.
A slash command in a ⌘T tab or split pane routes through the primary
chat's dispatcher, which read the FOREGROUND view's identity for two
decisions it had no business asking the foreground about.
Busy: the gate read `busyRef`, a mirror of whatever chat is on screen.
A brand-new tab with zero turns was told "session busy — message queued"
because an unrelated chat was mid-stream, and the converse let a
background send fire into a live turn.
Identity: the output writer bound to the foreground's stored session, so
a tab's transcript writes re-keyed its cache entry onto the primary's
stored id and its queued payload landed on the primary's queue — the
kickoff would then drain into the wrong conversation. `submitText` also
dropped an explicit target when it routed to a slash command, running a
queue drain's command against whatever was in front.
Read the target session's own published state for both. One shared
resolver so submit and slash cannot drift apart again.
A slash command runs against the session `resolveTargetSessionId` picks,
which is routinely not the session on screen — a tile, a route rebind, or
a session created by the call itself. Both prompt pipelines gated on
`busyRef`, the FOREGROUND view's busy flag, so one session's send was
gated on another session's turn: a stale foreground `true` (a warm resume
of a still-running chat leaves one behind) parked an idle session's
command on the composer queue and reported "session busy" about a session
doing nothing. The converse also leaked — a background send could fire
mid-turn while the foreground happened to be idle.
Read the published per-session state instead, falling back to the
foreground flag only when the target has no state yet (a just-minted
session whose first publish hasn't landed). One shared resolver so submit
and slash cannot drift apart again.
Two existing cases in tests/test_tui_gateway_server.py covered the row's cwd
contract and this change had to answer both.
`_persisted_session_cwd` reached through `_session_cwd`, which falls back to
the gateway-wide completion cwd when the session carries none. That belongs to
no session in particular, so a session that never had a directory was given
one. Read the session's own `cwd` instead.
The remaining case asserted that a terminal session's directory is discarded,
which is the behavior this branch deliberately changes. Split it in two: a
terminal session now records its workspace, and the desktop keeps the "No
workspace" default it was written to protect.
The preview window masked its top 28% to transparent while reasoning
streamed, so a gradient appeared over the thinking text and vanished the
moment the block finished. Remove the mask; the max-height window and
bottom-pinned scroll still keep the preview compact and following the
newest tokens.
Sessions started from the terminal were stored with no `cwd` and no
`git_repo_root`, so the sidebar had nothing to group them by and they never
appeared under their project — they fell into "No workspace" instead. On one
real profile this covered 690 of 1063 TUI sessions.
The row write persisted a cwd only when `explicit_cwd` was set, which happens
only if the user switches directory mid-session. The intent was to avoid
filing chats under whatever folder the app launched in, and that reasoning
holds for the desktop, whose launch directory is an artifact of how the bundle
was opened. It does not hold for a terminal session: the user cd'd into that
directory before running hermes, and it is where the agent's terminal runs.
Split the two cases behind one helper. An explicit pick is always persisted;
otherwise the launch directory is recorded for terminal-started sessions and
left unset for the desktop, preserving the existing "No workspace" default
there.
Existing rows are unaffected: they carry neither cwd nor git_repo_root, so
there is nothing to recover them from.
The sidebar's project overview put git checkouts with zero Hermes sessions
above the repos the user actually works in, and dragging a project into place
did not stick.
Two causes. The repo discovery payload folded `discovered_repos.last_seen`
into `last_active`, but `last_seen` is when the disk scan last saw the
directory, so every scanned checkout was stamped "just now" and outranked
real work. Activity is now session-derived only; a repo with no sessions
reports no activity.
The overview also applied the manual drag-order through `orderByIds`, which
floats every id missing from the saved order to the top. That is right for
sessions, where a new chat should not sink, but the overview keeps receiving
newly-scanned repos — so once the user dragged anything, each new discovery
jumped above their hand-picked list. Projects the user has not ordered now
keep their deterministic position: ones with real activity still surface on
top, zero-session discoveries sort below the ordered list.
probe-model-picker.mjs times pill-click → menu painted over N rounds;
profile-model-picker.mjs wraps one open in a CPU profile with a
top-self-time table. Both attach to the perf:serve instance.
Each kept-alive tab whose model hasn't resolved ticks a GlyphSpinner —
setInterval + setState + a React commit every ~80ms, per mounted tab,
for pixels behind the active one. Gate the interval on the pane's
visibility context; the visible tab keeps its spinner, hidden tabs
resume from frame 0 on reveal.
ModelMenuPanel mounts a ModelEditSubmenu per model row, and each body ran
its hooks and built its JSX eagerly on menu open — ~90 rows of switches,
radio groups, and preset lookups nobody hovered yet, the largest app-code
slice in the open-latency profile. Wrap the body in a child component under
SubContent so Radix's presence gate leaves it unrendered until the submenu
actually opens; open latency drops roughly in half on a 91-model catalog.
Closes the last false-pass window @helix4u flagged on #71779. He explicitly
said not to hold the PR for it; it is two lines, so worth doing rather than
leaving a known-soft assertion in a concurrency test.
connection_opened.wait(1.0) proved the connection had not opened, but not
that the connector thread had actually reached connect_tracked() -- an
unscheduled thread produces the same observation. The connector now sets
connect_attempted immediately before the blocking call, and the test waits
on that first, so "still blocked" means blocked at the lock rather than
not yet started.
15/15 stable at ~1.1s. Removed-lock sabotage still fails.
Second round of @helix4u review on #71779. Both findings reproduced before
fixing.
1. My previous fix turned a crash into SILENT DATA LOSS. Returning
status="missing" for a present-but-unusable state_meta looked like a safe
degrade, but _verify_recovered_database only escalates "failed"/"partial"
into a warning + loss_detected. Measured on the branch: a run that dropped
a real metadata table reported warnings=[], loss_detected=False,
partial=False, complete=True. Strictly worse than the ValueError it
replaced -- that at least failed loudly.
Now "failed" when the table exists but lacks key/value, "missing" only
when genuinely absent. The damaged case yields
warnings=['state_meta copy status is failed'], loss_detected=True,
partial=True, complete=False, while staying verified=True so the output
is still installable-with-review.
2. The race test I wrote had its own scheduling race: after the guard
released the lock, the racer could win before the main thread set the
release event, failing on a correct implementation. Rewritten per
helix4u's design -- copy runs in a worker parked inside the patched
copy, a second worker attempts connect_tracked(), assert it stays blocked,
release, assert it then opens. Deterministic and ~1.1s instead of 10s;
12/12 stable.
Sabotage-verified. Note the third scenario only failed after adding a
unit-level test: recover_session_database short-circuits on the inspection
result when state_meta is entirely absent, so the helper's absent-branch is
unreachable end-to-end and a regression there was invisible. Both statuses
are now pinned directly.
939 targeted tests green.
Post-merge follow-up to #71770. Both defects were found by @helix4u in review
and reproduced against merged main before fixing.
1. Check/use race in _copy_source_bundle (my bug, from the #71770 follow-up
commit). It called has_live_connection(), released the registry lock, and
only then ran shutil.copy2() over the bundle. A tracked connection could
open in that window; the copy's close() then cancels its POSIX advisory
locks -- the exact class #71724 closed. Measured on main: a racer thread
opened a connection mid-copy after blocking 0.000s.
Adds sqlite_safe_read.offline_file_access(), a context manager that holds
the connection-lifecycle lock across an entire multi-step raw access, and
routes the bundle copy through it. Same racer now blocks 10.0s until every
raw descriptor is closed. Any future raw read of a database file (hashing,
moving a bundle aside) should use this rather than a bare pre-check.
2. _copy_state_meta_salvage assumed a 'key' column. A damaged state_meta can
keep 'value' and lose 'key'; columns.index("key") then raised ValueError
and aborted the whole partial recovery. The mirror case (key without
value) would have copied key-only rows and reported the table complete.
Now requires both, matching the non-partial _copy_state_meta, so an
unusable optional table is recorded as missing/failed and --allow-partial
still recovers sessions and messages.
Both regression tests verified by sabotage: reinstating the bare pre-check
fails the race test, removing the key/value requirement fails the other.
937 targeted tests green.
The name-based sibling probe can't reach every dead worktree: a dir renamed away
from its repo's prefix (`hermes-salvage-drafts` next to `hermes-agent`) shares
nothing to trim back to, and a scratch dir under /tmp was never a worktree at
all. Those fall through to the path-only heuristic, which reports the cwd as its
own repo root, and Tier 2 promotes it to a top-level project — one that can't be
opened and can only be dismissed by hand.
Gate auto-project promotion on the directory still existing, threaded in as an
injected predicate to keep the builder pure. The guard keys on the directory,
not on git-ness, so a plain non-git folder that's still on disk keeps its
project; callers that can't stat (remote backends) omit it and keep every
candidate. A stale persisted `git_repo_root` gets the same treatment, so a
deleted repo can't resurrect from the recorded value alone.
`_probe_sibling_worktree` recovers a deleted `<repo>-<suffix>` worktree by
trimming its name back to a sibling that still resolves, but it only trimmed
the LEAF segment. A session's cwd is usually a subdir of the worktree
(`<repo>-<suffix>/apps/desktop`), whose basename shares nothing with the repo,
so the probe no-oped and the dead path fell through to the path-only heuristic
— which minted it as its own main repo root, and then as a top-level project.
Walk the ancestors, deepest first, with the probe budget shared across the whole
walk so a deeply nested cwd can't fan out into a probe storm.
