A turn routed to a model that takes pixels directly sends `content` as a
parts list, and the session store deliberately ignores a plain-string
persist override for a list payload — a text override must not erase a
turn's image summary. So the override was dropped for every user on a
vision-capable main model, and the durable row kept only the caption plus a
literal `[Image attached at: ...]` / `[screenshot]`, which the renderer
cannot turn back into an image. Only vision-preprocessed (text-mode) turns
were actually fixed.
Mirror the shape instead: swap the text part for the `@image:` ref form and
keep the image parts, so the model still has the pixels for the rest of the
session, and drop the `[screenshot]` stand-in on the way into the bubble
when a ref was lifted from the same message.
The desktop gateway passed the vision-enriched, model-only message text
(carrying an `image_url:<path>` hint) straight into run_conversation as
the persisted user turn. The renderer only parses `@image:<path>`, so it
could not rebuild the attachment from history: after a restart the image
was gone and only the caption survived, and on a live session switch the
warm cache disagreed with the authoritative text and the frontend
"rescued" the image by appending it after the caption.
run_conversation already supports persist_user_message for exactly this
"what the model sees" vs "what gets stored" split; it was simply never
wired up for the attachment path.
The desktop and the Hermes backend it talks to version independently — a
remote VM running an older build still serves display_metadata as JSON text.
Indexing into that string with `in` threw and failed the whole resume, so
narrow the type to admit a string and parse it before reading task_count.
Falling back to the generic label keeps a delegation event renderable even
when the metadata is unusable.
Co-authored-by: xxxigm <xxxigm@users.noreply.github.com>
Co-authored-by: Studio729 <Studio729@users.noreply.github.com>
Two React passive-effect timing bugs let a session switch land in the wrong
chat: activeSessionIdRef/selectedStoredSessionIdRef (use-session-state-cache)
and the composer's attachment-scope swap (use-composer-draft) both mirrored
their source props via useEffect, which fires one commit AFTER the new
session's view has already painted — a synchronous read/submit in that window
observed the outgoing session's ids/attachments.
- use-session-state-cache.ts: mirror the session refs synchronously during
render instead of a useEffect, guarded to fire only when the prop itself
changed (not unconditionally) so an imperative pin from submit.ts /
use-session-actions (e.g. a freshly resumed runtime id, intentionally not
synced to the source atom) survives an unrelated re-render.
- use-composer-draft.ts: the per-thread attachment-scope-swap effect is now a
useLayoutEffect, closing the window before paint.
- submit.ts / session-context-drift.ts: add a 3rd drift prong comparing the
composer's loaded scope (SubmitTextOptions.composerScope) against the
submit target, resolved into the same lineage-root domain
(resolveComposerSessionKey) the composer itself uses — comparing against
the raw tip id would false-positive-abort every submit into any session
that has ever auto-compressed.
- routes.ts / chat/index.tsx: the primary composer's durable scope key now
prefers the route over a possibly-stale store selection
(primaryRouteSelectedSessionId).
- use-composer-draft.ts: redacted [composer-rehydrate] diagnostic log
(counts/kinds/scope only, never raw refs) for future reports in this class.
- chat-runtime.ts: normalize attachment id values (url/path) before hashing
so a re-attach with a trailing slash or backslash path dedupes correctly.
16 files, 286 tests across the touched/dependent suites (17 files) green,
including new regression coverage for each fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds a first-run Desktop choice between installing Hermes locally and
connecting to an existing remote Hermes gateway. The choice appears after
backend resolution but before ensureRuntime(), so selecting remote cannot
accidentally trigger local bootstrap.
New modules:
- first-run-setup-gate: concurrent first-run decision gate and reset semantics
- primary-backend-startup: Electron-free orchestration seam (saved remote
resolution, gate decision, remote re-resolution, local continuation)
- primary-connection-rehome: prevents dual-owner race where both cold boot()
and renderer softSwitch() could connect simultaneously
- first-run-remote-form: extracted remote form with stale-result guards
Reuses existing connection-config IPC, encrypted token storage, OAuth
session partition, and primary backend resolution.
Fixes#38602Fixes#36970
Group the flat recents list and entered-project lanes by recency: an
unlabelled head of the newest run of sessions (cut at a real break in
activity, sized toward the most recent handful), then one divider per
coarse calendar range — Earlier today / Yesterday / Earlier this week /
Last week / Earlier this month / month / month + year. Empty ranges are
skipped, the first rendered group is never labelled, branch clusters
never split, and hand-ordered lists / pinned / project previews stay
divider-free.
Frontend consumers for the events added by PR #59646: the TUI shows a
replace-in-place 'MoA: refs k/n' activity line (swapped for 'MoA:
aggregating…' on the aggregator phase), and desktop streams '◇ MoA refs
k/n' lines into the reasoning disclosure, self-cleaned by the first
moa.reference block.
* fix(desktop): hide persisted agent-only history scaffolding
Filter verification-stop nudges and context-compaction handoffs at the
stored-history mapper boundary. Preserve a real reply when a compaction
handoff shares its stored message.
* test(desktop): build persisted E2E sessions through the real agent
Drive tui_gateway.entry over its stdio JSON-RPC transport against the mock
provider, wait for real completion events, and persist normal session history
through AIAgent and SessionDB. Migrate resume and hidden-history coverage,
including real compression and live verify-on-stop scaffolding, then remove
the unused direct SessionDB import scripts.
* fix(desktop): use the provisioned Python for real-session E2Es
Run the stdio gateway through uv's synced project environment outside the
Nix dev shell, while retaining the fully provisioned Nix Python when the
shell advertises HERMES_PYTHON_SRC_ROOT.
* fix(nix): expose the provisioned Python environment to uv
Mark the Nix-built Python environment active in the dev shell so the shared
E2E session builder can always run through `uv run --active --no-sync`.
* fix(timeline): persist typed display events
* fix(timeline): strip display-only fields from provider payloads, preserve through rewrites, fix /resume display history
Three review findings from PR #69771:
1. Provider payload leak: display_kind and display_metadata were forwarded
to the provider API as unknown message fields. Strict OpenAI-compatible
backends can reject the next request after a model switch or resumed
typed event. Strip both from the per-request api_msg copy in
conversation_loop alongside the existing api_content pop.
2. Rewrite/import data loss: _insert_message_rows preserved display_kind
but silently dropped display_metadata. After replace_messages,
archive_and_compact, or session import, async-delegation completion
events lost their task counts and fell back to generic display text.
Add display_metadata to the INSERT columns and bind tuple.
3. CLI /resume stale recap: startup --resume A set _resume_display_history
from A's lineage. A subsequent in-session /resume B loaded B only into
conversation_history via get_messages_as_conversation, leaving the stale
A display projection. _display_resumed_history preferentially read the
stale attribute, showing A's recap for B. Switch /resume to
get_resume_conversations and update _resume_display_history alongside
conversation_history.
Tests: 890 Python (5 files), 35 desktop TS — all green.
* feat(tui): render typed display events as ◈ markers in the Ink TUI
The TUI was not handling display_kind at all — model switch markers and
async delegation completions rendered as opaque user messages with the
full [System: ...] text, and hidden compaction handoffs were visible.
Wire display_kind through the full TUI chain:
- _history_to_messages (tui_gateway/server.py) forwards display_kind
and display_metadata to the gateway transcript payload.
- GatewayTranscriptMessage (gatewayTypes.ts) gains both fields.
- Msg.kind (types.ts) gains 'event' value.
- toTranscriptMessages (domain/messages.ts) maps:
- hidden → skip entirely
- model_switch → event "model changed"
- async_delegation_complete → event "N background agents finished"
(or "background agent work finished" without metadata)
- messageGroup (blockLayout.ts) routes event to its own group, with
SELF_SPACED + PAINTS_TRAILING_GAP so it owns its margins.
- messageLine.tsx renders event-kind as a dim ◈ marker with no gutter,
matching the CLI's ◈ event rendering.
- 4 new TUI tests for hidden/model_switch/async_delegation mapping.
TUI typecheck: clean. TUI lint: 0 errors (2 pre-existing warnings).
TUI tests: 9 passed (1 pre-existing failure on main, unrelated).
Two fixes for desktop hands-free voice:
- The live speech session bound to the first assistant bubble with text, so
a tool-calling turn spoke only the opening narration and silently dropped
every later interim AND the final answer. The conversation selector now
aggregates all unspoken assistant bubbles in order (turn-scoped speech);
auto-speak keeps its latest-reply-only behavior.
- The speak-stream WS producer blocked forever on the text queue, so a
narration line with no trailing whitespace ("Let me check.") sat in the
sentence chunker until end-of-turn — spoken long after the tool finished,
with the UI stuck on "Preparing audio…". Mirror the CLI speaker's idle
flush: sentence-terminated buffers flush after 0.5s of producer silence,
anything else after ~2s; open <think> blocks are never flushed.
* fix(desktop): improve fallback tool-call details
Show failed image-generation calls through the normal fallback row, remove duplicate normal-mode web-search JSON, and format Technical Mode payloads as readable JSON.
* feat(desktop): render terminal tool calls as transcripts
Show terminal commands with a prompt and exit status, then reveal ANSI-safe stdout and stderr in the expanded tool row.
* fix(desktop): reconcile tool calls by command
Match context-only tool starts with command-bearing completions when their IDs differ, preventing stale duplicate terminal rows. Show the web-search query above its result cards.
Narrow the setup-error matcher, match the server's empty-key contract,
and route gateway-event plus create/resume/branch through one
credential-warning policy. Preserve configured=true only for
non-authoritative transport fallback.
Co-authored-by: Yingliang Zhang <zhangyingliang@outlook.com>
Co-authored-by: Brandon R <kingdomwarrior23@gmail.com>
* fix(desktop): keep clarify lifecycle when tool progress is off
* fix(desktop): render clarify prompt from the request event
Re-authored onto the current use-message-stream/gateway-event.ts (the
original patched the pre-split use-message-stream.ts). When the tool.start
row that normally mounts the inline clarify UI is missed (stream reconnect
/ hydration race), upsert a stable pending clarify tool row from
clarify.request itself so the prompt stays answerable; a real
tool.start/complete with the same request id merges rather than duplicates.
Co-authored-by: 정수환 <centerid@naver.com>
* chore(contributors): map centerid@naver.com -> lidises
Attribution mapping for the salvaged #47544 commit.
* fix(desktop): correlate clarify rows by question so hydration can't duplicate
The hydrated row (from clarify.request's request_id) and the real tool.start
row (the model's tool_call_id) have different ids, so id-only matching appended
a second clarify card in the normal path (caught by the BLOCKING_CLARIFY e2e:
'question' resolved to 2 elements). Add 'question' to the tool match-value keys
so a clarify upsert merges into the existing pending clarify row regardless of
id (same request<->args correlation ClarifyToolPending already uses); when no
row exists yet (reconnect/hydration) it still creates one.
---------
Co-authored-by: 정수환 <centerid@naver.com>
Keep Enter as the Cursor-style live-turn steering gesture and assign
Ctrl/Cmd+Enter to queue the current draft. Align keybind settings,
tooltips, translations, and the Electron queue-boundary coverage.
On a billing wall, raise a sticky, billing-specific toast (never the generic
error toast) and a persistent in-composer banner for the active session — both
with one recovery action: Nous → in-app Settings → Billing, other providers →
their billing page (deep-linked). The banner reuses the shared StatusRow chrome
(no bordered alert, Codicon glyph, shared buttons), and the composer stays usable
so slash commands keep working.
markVoicePlaybackInterrupted() / takeVoicePlaybackInterrupted() mirror
the backend latch in the renderer (the barge happens client-side, where
the audio plays). VAD barges and typing over playback mark it; the next
prompt.submit carries interrupted:true, which the TUI gateway latches
into the model note.
/api/audio/speak-stream WebSocket: one socket + one Web Audio clock per
reply. The renderer feeds raw LLM deltas as they arrive; the server
cuts sentences with the shared chunker and streams int16 PCM back while
generation continues — speech overlaps generation with no per-sentence
connection or synthesis gaps. Falls back to the POST data-URL path for
old backends / non-chunked providers.
Barge-in runs a MediaRecorder on the monitor's stream the whole time
playback is live (rotated while quiet to bound pre-roll); talking over
the agent cuts playback and the complete utterance goes straight to
transcription and submit. /api/audio/transcribe returns 200/"" for
no-speech results so quiet turns re-listen instead of toasting a 400.
Post-merge review polish of #68722 + #68761:
- Usage: drop the manual refresh + 'Updated Xm ago'; billing queries now
refetchInterval-poll while the page is mounted, like every other data view.
- No card: lead the page with a warn notice naming the blocker + 'Add card ↗',
so the silently-disabled buy/auto-refill controls have an obvious cause.
- Layout: unify on a shared SettingsCard/SettingsSection primitive; collapse the
three floating one-row sections (Payment / One-time top-up / Automatic refill)
into a single divide-y 'Payment & credits' card.
- Dev fixture switcher: relabel as a wrench + 'preview' dashed control so it
reads as the DEV-only tool it is (compiled out of production).
Removed formatUsageUpdatedAgo/oldestUpdatedAt/UsageRefreshRow + their tests;
added no-card-notice and no-manual-refresh tests. vitest 107 green, tsc + eslint clean.
* fix(desktop): route /compress through session.compress RPC with transcript replacement
Salvages #44462, #53755, and #68218 into a single canonical fix for the
desktop /compress cluster.
The desktop routed /compress through slash.exec, which sends it to the
_SlashWorker subprocess. Compressing a large session outlives both the
desktop's 30s WS timeout and the worker's 45s pipe timeout — the client
gives up, runExec's blanket catch swallows the error, and command.dispatch
surfaces a misleading "not a quick/plugin/skill command: compress" (#44456).
Even when compression succeeded via the _mirror_slash_side_effects path,
the desktop never received the post-compress message list, so summarized
bubbles stayed on screen forever — /compress looked like a no-op.
This change routes /compress to the dedicated session.compress RPC (the TUI's
path), combining the best of all three PRs:
- 120s client timeout matching the TUI's HERMES_TUI_RPC_TIMEOUT_MS (#44462)
- Transcript replacement from the response `messages` via toChatMessages,
the same converter session.resume uses (#68218, teknium1 review on #44462)
- Session-isolation guard: updateSessionState only publishes for the active
runtime, so a late result after a session switch can't clobber the
foreground transcript (#53755, teknium1 review on #53755)
- Coalescing: dedup concurrent compress requests per session (#53755)
- Progress toast ("compressing context...") outside the transcript (#53755)
- Error unmasking in runExec: when slash.exec fails and command.dispatch only
adds "not a quick/plugin/skill command" routing noise, surface the original
worker error instead (#44462)
- /compact alias + focus_topic forwarding
Co-authored-by: AlliDev <AIalliAI@users.noreply.github.com>
Co-authored-by: PinkEVO <PINKIIILQWQ@users.noreply.github.com>
* feat(desktop): route slash commands with dedicated RPCs to those RPCs
Salvages #63513 — introduces a new `rpc` kind on DesktopCommandSurface so
commands with a first-class gateway @method handler bypass slash.exec /
command.dispatch entirely, and a `renderRpcResult` utility that shapes
each RPC's structured reply into readable transcript text.
Migrates 6 commands from exec() to rpc(...):
/agents → agents.list
/save → session.save
/status → session.status
/steer → session.steer
/stop → process.stop
/usage → session.usage
/compress stays as action('compress') — it needs transcript replacement
from the response `messages`, which the generic rpc path can't do (per
teknium1 review on #44462/#63513).
Also includes the json-rpc-gateway timeout message improvement: the error
now includes the configured timeout duration ("request timed out after 120s:
session.compress") so a user can tell whether the default 30s fired or a
per-call override.
Co-authored-by: Jelvin <SmallNew2003@users.noreply.github.com>
* fix(desktop): preserve provider choice during config initialization
* fix(desktop): preserve slash command and host compression semantics
Keep commands whose CLI behavior exceeds their current RPC contracts on slash.exec.
Propagate the full compression timeout through compute-host control, return structured
host compression outcomes with metadata, and retain successful compression feedback
in the desktop transcript.
Add regressions for timeout forwarding, host aborts and metadata sync, structured host
control responses, command routing parity, and numeric stop counts.
* fix(desktop): harden compression state handling
Preserve the invoking stored-session binding for delayed compression results,
normalize replacement histories, and serialize provider selection. Stabilize
gateway platform tests and guard the desktop Git facade during renderer teardown.
---------
Co-authored-by: AlliDev <AIalliAI@users.noreply.github.com>
Co-authored-by: PinkEVO <PINKIIILQWQ@users.noreply.github.com>
Co-authored-by: Jelvin <SmallNew2003@users.noreply.github.com>
A stray tsc run can emit foo.js next to foo.ts under apps/shared/src or
apps/desktop/src. .gitignore hides the artifact from git status, but Vite
resolves extensionless imports .js-before-.ts, so the renderer silently runs
the stale compiled copy.
tsc -b . --clean already knows the emit graph and deletes
matching outputs. Run it before vite in all dev scripts.
This bit for real: a Jul 16 artifact of websocket-url.js predated the #68250
getGatewayWsUrl contract change ({ ok, wsUrl } IPC result), so its old
'if (fresh) return fresh' handed the whole result object to new WebSocket(),
dialing ws://127.0.0.1:5174/[object%20Object] on every boot. The desktop app
could never connect, and the failure survived reboots and cache wipes because
the poison lived in src/.
JsonRpcGatewayClient.connect() now rejects non-ws:// URLs with a readable
error instead of letting new WebSocket() coerce an object into
[object%20Object], so any future contract skew fails diagnosably.
- Use the fork glyph for branch and a sine wave for read aloud (all one lib now)
- Extract compact "2h ago" into formatAgo() in lib/time.ts (+ ageDays locale string)
- Cover formatAgo with a unit test
Extract the open_preview emitter into a shared tools/desktop_ui bridge
(one gateway-injected sink, routed by HERMES_UI_SESSION_ID) and add a
second desktop-gated tool on top of it:
- focus_pane(chat|files|terminal|review|sessions) -> pane.reveal event.
The desktop runs each pane's own reveal path (revealDesktopPane table)
and only acts on the active window -- a background turn never moves the
user's focus (desktop AGENTS.md: offer, don't hijack).
open_preview now emits through the same bridge. Both tools are check_fn
on HERMES_DESKTOP (zero footprint elsewhere), sitting beside
read_terminal/close_terminal in _HERMES_CORE_TOOLS.
Deliberately not adding run_slash: letting the agent fire slash commands
mid-turn (/model, /new, /clear) fights prompt-cache + conversation
invariants.
Since #65919 the live view seals each chunk of mid-turn assistant
commentary (message.interim) as its own finalized bubble. Every bubble
with visible text renders the hover action footer, so a tool-heavy turn
grew a copy/refresh bar under almost every paragraph — and the live
render didn't match rehydration, which merges the turn into one bubble.
Mark sealed interim bubbles with ChatMessage.interim, carry the flag
into the runtime message metadata (custom.interim), and skip the
AssistantFooter for them. The turn's final reply keeps the footer; a
previewed final that settles onto an interim bubble clears the mark so
the settled reply regains its actions. interim joins COMPARED_FIELDS /
chatMessagesEquivalent so flipping it repaints.
Also fix an id-collision flake this surfaced: stream/interim bubble ids
were Date.now()-only, so an interim seal and the next segment's first
delta in the same millisecond reused the id and the new segment appended
into the sealed bubble. Ids now include a monotonic sequence.
- tui_gateway commands.catalog: skip _TUI_EXTRA entries that collide with a
registry command or alias (the /compact class of bug, #57133; also removes
the pre-existing /sessions duplicate) — registry entry is canonical.
- apps/desktop: /compact now dispatches as an alias of /compress (matching
the registry's canonical alias) instead of dead-ending; /density (the
renamed TUI display toggle) is marked terminal-only so the desktop palette
doesn't advertise a command its dispatcher can't run.
- ui-tui/README.md: document /density instead of the old /compact toggle.
- tests: commands.catalog regression test asserting no duplicate advertised
names and no command shadowing another command's alias; desktop routing test.
Drop the unused DEDUPE_WINDOW_MS export and rename its interval so
"window" isn't overloaded against BrowserWindow in a multi-window
feature (windowMs → intervalMs). DRY the completion-sound play path.
No behavior change.
With multiple full windows, each renderer independently reacts to the
same backend event, so one-shot cues fired N times: OS notifications
(the per-renderer throttle can't see other windows), the turn-end sound
(playCompletionSound runs on every message.complete, ungated by focus),
and auto-spoken replies (double voice when a chat is open in two windows).
Add a single race-free owner in the main process (electron/event-dedupe.ts):
main handles IPC serially, so the first window to claim a key within a
short window wins and peers stay quiet. Notifications collapse at the
hermes:notify choke point; the sound and spoken replies claim via a new
hermes:ambient:claim IPC (keyed by session / reply id). Off Electron the
claim falls back to "emit", preserving single-window behavior.
The sound's mute check runs before the claim so a muted window can't win
the cue and silence an audible peer.
Repoint session.newWindow (⌘⇧N) from the compact new-session pop-out to
openNewWindow(), which opens a full peer instance via the new openWindow
bridge, and add a "New Window" entry to the ⌘K palette (shown with its
hotkey hint, gated on canOpenNewWindow()). Relabel the action "New window".
Drops the retired openNewSessionWindow bridge and the vestigial
isNewSessionWindow()/new=1 flag; renames the shared opener helper.
The SSH modules predate the stricter lint config that landed on main (curly, no-empty, perfectionist sorting, prettier). Mechanical lint:fix + fmt pass, empty catch blocks filled with the codebase's void-0 convention, and inline no-control-regex disables on the three deliberate control-char patterns (same pattern as lib/ansi.ts).
Sync with main after the contribution-shell refactor (#60638) and the backendConnectionState extraction (#65885) landed. Seven conflicts, all resolved in favor of main's new architecture with the SSH lifecycle ported on top:
- electron/main.ts: adopt backendConnectionState (attempt tokens, attachProcess, clearPromiseForAttempt) as the sole owner of the primary backend; drop the branch's raw hermesProcess/connectionPromise globals and commitConnectionFailure call site. SSH liveness-streak classification, teardown, and the SSH quit path are layered onto the new ownership model; before-quit combines SSH transactional teardown with the Windows sandbox marker (#38216).
- desktop-controller.tsx: deleted on main; its SSH beforeConnectionSwitch cleanup (preserved-route fresh draft, overlay return-route reset, project-tree reset, close-all-terminals) moves to contrib/wiring.tsx's useGatewayBoot call.
- use-session-actions/index.ts: keep main's onFreshDraftRouteIntent (fires unconditionally); preserveRoute gates only the navigate.
- gateway-settings.tsx: keep main's acceptSavedConfig/connectedCloudUrl; every save/sign-in/sign-out success path routes through acceptSavedConfig with the branch's stale-async seq guards intact.
- boot-failure-reauth.ts: keep both sshFailureMessage and main's isRemoteReauthFailure formatting.
- Both test harnesses take the union of props.
Validation: tsc -b clean; desktop suite 1704 passed / 1 skipped; electron SSH/connection modules 225 passed; python SSH + web_server suites 508 passed.
