The gateway now queues a model switch made during a turn and applies it
at the next turn start (#74756), but the desktop still bounced the pill
back to the old model: the post-switch refetch answered with the model
still running and repainted over the pick.
Skip that refetch when the switch was deferred — the apply publishes
session.info when it lands, and that is what re-syncs every surface.
An older gateway that still refuses with 4009 keeps the pick too rather
than rolling back and toasting at a user who did nothing wrong; it is
what the next turn runs anyway. Real failures still roll back and report.
The 4009 predicate lives beside the other gateway-compat probes in
lib/gateway-rpc.
A reference had two styling systems: a Tailwind class string assembled in
TypeScript (`directiveChipClass`) and a separate `link-chip` for prose links,
each carrying its own color-mix(). Same concept, three appearances.
Now every inline reference — a composer chip, a sent message's mention, a
markdown link, a completion row's glyph — is `class="ref"` plus
`data-ref="<kind>"`, and styles.css owns the accent. No hex or color-mix()
ships from a component, so a skin restyles all of them at once.
Keying the accent on `[data-ref]` alone rather than `.ref[data-ref]` also lets
the popover's icon column take a kind's hue without inheriting its inline-text
layout.
The `/` path has had a completion cache since the skills-scan work; the `@`
path never got one. Every keystroke was an uncached round trip behind the 60ms
debounce, so walking a tree — Tab in, Backspace out, retype a segment — paid
full price for paths it had just listed.
Measured in-process against this repo (8,036 files): `git ls-files` ~38ms,
ranking ~12ms. The backend already caches the file list for 5s, so the fix
belongs in the renderer: reuse the existing cache module with a short 15s TTL
(a directory listing, unlike the command catalog, can change under the user)
keyed on cwd + session + query, and wire `isCached` so a warm query skips BOTH
the debounce and the loading state.
That last part is what makes it feel instant rather than merely fast — a
spinner over an answer already in hand reads as latency the user isn't paying.
YOLO had a status-bar zap and a slash command but was never registered as a
palette contribution, so ⌘K couldn't reach it.
Adding it exposed the wider gap: "Toggle status bar" doesn't say which way it
will go. Rows already carry a muted `detail` slot, so paletteToggle fills it
with on/off — no new chrome, and the verb stays. Status bar, logs, and layout
edit mode go through the same helper.
workspace.openFolder (default mod+o, the editor-standard open-folder chord)
runs openFolderAsProject: pick a folder, enter the project that already owns
it or create one named after the folder, scope the sidebar, and land on a
fresh session draft anchored there. A stale backend without the projects.*
RPC still gets the workspace session, with a warning.
StartWorkSessionRequest grows an openTab flag so these opens-from-nowhere
stack a tab instead of spending an occupied main, and goToProject/
resolveNewSessionCwd share one projectRootCwd resolver.
One slot, Slack-style: on assistant rows the picker trigger and the landed
reaction are the same far-right element, so reacting never shifts layout
(empty → ☺ following the action bar's hover fade; reacted → the emoji, always
visible, always full-strength). User bubbles react via right-click and show
the badge beneath, in the checkpoint row's register.
- Clicks paint instantly from a local nanostores overlay — no round-trip in
the loop; the RPC persists behind it and rolls back visibly on rejection
- Agent reactions land via the message.reaction event into an overlay keyed
by DURABLE row id, so the end-of-turn resume (which regenerates renderer
ids and rebuilds from in-memory history) can't clobber the paint
- Full picker is frimousse behind the six-emoji quick row, fed from bundled
emojibase-data served at ./emojibase by a small vite plugin (offline, no
CDN), with Slack-style alternating cell tints keyed off the codepoint
- Reaction picker opts out of the shared popover glass: solid surface so
15%-alpha hover tints stay readable
- react_to_message tool blocks are suppressed in the transcript (like todo):
the reaction appearing IS the UI; failures still render
- rowId reaches rehydrated messages from both transcript shapes (gateway
row_id, REST numeric id)
composer.modelPicker shipped unbound and opened the full-screen picker
dialog. It now defaults to ⌘⇧M — the chord LibreChat, Open WebUI, and
Cherry Studio independently converged on — and toggles the composer
pill's live dropdown instead, search field ready: ⌘⇧M → 'gr' → Enter
switches model in two keypresses.
Routing follows the tab-verb convention (#74447): the request lands on
the chat surface in the hovered zone first, then the active composer,
skipping hidden keep-alive tabs. With no chat surface on screen
(settings, profiles) the keybind falls back to the full dialog; with
the gateway closed the pill opens the dialog like a click would.
The voice-interruption fix (5081551f0) covered the Python surfaces (CLI +
TUI gateway) but the desktop app has its own mic path in voice-barge-in.ts,
which still had both bugs on Windows:
- HALF-DUPLEX GAP: the barge monitor only opened when TTS playback started;
during LLM generation no mic was listening at all.
- PLAYBACK DEAFNESS: the monitor calibrated its noise floor while the
speakers were already playing TTS, baking bleed into the floor. On
Windows, Chromium echoCancellation does not reliably cancel same-app
output (measured live: quiet floor ~35-50 RMS vs playback bleed
600-1700 RMS), making the trigger unreachable.
Changes mirror tools/voice_mode.full_duplex_listen:
- voice-barge-in.ts: phase-aware full-duplex monitor — quiet-only
calibration (floor held through playback, never recalibrated against
bleed), playback min-trigger clamp + ceiling, 500ms grace on playback
onset only, windowed-majority detection.
- use-voice-conversation.ts: monitor arms at turn submit and spans
generation + playback (ensureBargeMonitor, idempotent). Mid-generation
speech fires the new onInterrupt callback; spoken stop-word during a
barge ends the conversation; submit waits for the interrupt to settle.
- use-composer-voice.ts + composer/index.tsx: plumb onInterrupt: haltRun
(same seam as the Stop button).
Tests: use-voice-conversation.test.tsx (6 tests) covering monitor lifecycle
across generation + playback, mid-generation interrupt, stop-word handling.
npm run typecheck + eslint clean; remaining desktop vitest failures
reproduce on a clean tree (pre-existing Windows env failures).
Long thinking/tool stretches in a voice conversation are dead air — the
user cannot tell whether the agent is alive. New: quiet, repeating soft
bubble blips while the agent works and no speech audio is flowing.
- tools/voice_mode.py: numpy-synthesized blips (no binary assets) — two
alternating low pitches (G4/E4) with pitch glide + smooth attack/decay
envelopes, ~0.8-1.2s randomized spacing, volume = voice.beep_volume * 0.5.
start_thinking_sound(should_play=...) / stop_thinking_sound() daemon-loop
lifecycle; macOS-TCC-safe (sounddevice output gated there → silent skip,
no per-second afplay churn). New mark_audio_output_active()/
is_audio_output_active() ref-count wraps play_audio_file and the
streaming OutputStream sentence writes so "audio is flowing" is accurate.
- Config: voice.thinking_sound (default true) off-switch.
- cli.py: starts when a voice-mode turn begins, per-blip gate skips while
TTS speaks / mic records / barge capture owns the mic; stopped in the
chat() finally on every exit path.
- tui_gateway/server.py: same lifecycle around _run_prompt_submit turns
(voice mode on), gated on is_audio_output_active + continuous capture.
- Desktop: renderer owns voice-conversation audio, so a matching WebAudio
implementation (src/lib/thinking-sound.ts, same envelope/pitches) runs
while conversation status === "thinking"; honors voice.thinking_sound
(via config store) and the shared sound-mute toggle; stops instantly on
speaking/listening/end.
Saying OR typing a configured stop phrase (voice.stop_phrases, default
"stop") now ends the voice chat everywhere, not just classic CLI PTT:
- hermes_cli/voice.py: new explicit on_stop_phrase callback through
start_continuous/stop_continuous. The force-transcribe path previously
DISCARDED the stop phrase silently — with auto_restart=False the client
re-arms the next capture, so the conversation never ended. Both halt
paths now fire on_stop_phrase (fallback: on_silent_limit for legacy
callers) as user intent, distinct from the no-speech timeout.
- tui_gateway/server.py: voice.record wires on_stop_phrase and emits
voice.transcript {stop_phrase: true} after flipping HERMES_VOICE(_TTS)
off and stopping streaming TTS — same teardown as /voice off. The TTS
barge-in monitor stop-checks its transcript too. prompt.submit consumes
a TYPED bare stop phrase at the server-side choke point when voice mode
is on (returns {voice_stopped: true}, no turn starts).
- ui-tui: voice.transcript {stop_phrase} ends voice mode with a clear
'voice chat ended' notice (distinct from the no-speech-limit message);
submitPrompt releases the busy latch on a consumed voice_stopped reply.
- cli.py: _typed_voice_stop in process_loop — typing a bare stop phrase
while voice mode/continuous is active ends voice mode instead of
sending 'stop' to the agent; typed 'stop' outside voice mode is
unchanged. Voice transcripts skip the check (already stop-checked).
- desktop: interceptsTypedVoiceStop — the composer's onSubmit ends the
live voice conversation (same path as clicking end on the pill) when a
bare stop command is typed with no attachments; renderer-owned loop, so
handled client-side like the existing spoken isVoiceStopCommand.
- tools/voice_mode.py: transcribe_recording never lets the Whisper
hallucination filter swallow a configured stop phrase (e.g. 'bye'
configured as a stop phrase is both a hallucination-blocklist entry and
a stop phrase — stop-phrase check now wins).
Tests: continuous-loop signal (sabotage-verified), force-transcribe stop
signal + legacy fallback, hallucination-filter ordering, typed-stop CLI
unit tests (voice on/off/longer text), prompt.submit typed-stop gateway
tests, TUI vitest for stop_phrase event handling, desktop vitest for the
typed-stop interceptor.
The statusbar derived its client and backend version labels inline, in two
near-identical blocks. Move the wording into a pure resolver so every surface
that names an install agrees on the label, the commit diff, and the tooltip.
Assert isMessagingSource('photon') is true and that Photon/iMessage search aliases resolve, so a silent removal of the photon entry from MESSAGING_SESSION_SOURCE_IDS fails CI instead of regressing the sidebar section added in #47395.
Register Photon as a messaging platform in the Desktop sidebar:
- Add Photon brand icon (three-bar mark) to PlatformAvatar catalog
- Add photon to SOURCE_LABELS, SOURCE_ALIASES, and MESSAGING_SESSION_SOURCE_IDS
Closes#46761
xterm paints to a canvas, so its selection is not a DOM selection: the Edit
menu's Copy and the right-click Copy both call webContents.copy(), find
nothing, and copy nothing. On macOS the menu also swallows the Cmd+C
accelerator before the renderer sees it. That left Ctrl+C as the only key that
did anything, and xterm correctly forwards it to the PTY as SIGINT.
Mirror the selection into xterm's hidden helper textarea (the mechanism xterm
already uses for Linux middle-click paste) so the OS sees a real selection and
every platform copy path works, and add explicit chords on top: Cmd+C/Cmd+V on
macOS, Ctrl+Shift+C/V elsewhere, matching VS Code. Plain Ctrl+C copies only
when text is selected -- the behavior Windows Terminal and Tabby ship -- and
stays SIGINT otherwise, so interrupting a process never breaks.
Reads go through a new hermes:readClipboard IPC handler for the same reason
writes already do: the renderer's clipboard API throws whenever the document
isn't focused.
The `/` popover listed skills alphabetically, so on a 200-skill install
the ones invoked daily sat below a wall of skills that shipped with
Hermes and were never opened — /research-paper-writing (never used)
outranked /research (60 invocations).
Sort the Skills section most-used first, A-Z within a tie, and on a bare
`/` drop bundled skills with no recorded activity. Typing a query keeps
every match: a search that hides a result is broken, so a typed `/re`
only reorders.
Diagnosed from a Windows desktop's logs: the backend TTS was generating
and saving the reply mp3 every turn (provider: openai), so synthesis was
fine — the DESKTOP wasn't playing the first reply. Root cause is Chromium's
autoplay policy: audio (HTMLAudioElement.play() and AudioContext) is
suspended until the frame sees a user gesture. A voice conversation started
by the 'Hey Hermes' wake word has no preceding click, so the first reply's
play() was rejected with NotAllowedError (silently swallowed) and only
turn 2+ spoke. Manual voice-start worked because the button click WAS the
gesture — exactly the 'first message in a new voice session is silent,
manual start is fine' report.
Fix (defense in depth):
- chatWindowWebPreferences sets autoplayPolicy: 'no-user-gesture-required'
on every chat window (primary + secondary), so a deliberately-launched
native app never gates audio on a gesture. Primary fix.
- voice-playback.ts resumes a suspended AudioContext on stream 'start' and
retries HTMLAudioElement.play() once via an unlock context after a
NotAllowedError — covers the dashboard-embedded surface that doesn't get
the Electron policy.
Tests: 16 electron session-window tests (incl. new autoplay assertion),
tsc + eslint clean.
The /api/audio/speak-stream WebSocket URL was always minted for the primary
backend with no profile param, so streaming read-aloud used the default
profile's TTS provider even when another profile was active. Mint the
ticket for the active profile's connection and carry ?profile= (same seam
as /api/pty), via a read-only getApiRequestProfile() accessor.
Saying 'stop' in a voice chat did nothing — the transcript was just
submitted to the agent as a normal turn, so the conversation never ended.
The only way out was the mouse/hotkey. That's not how a hands-free voice
assistant should work.
useVoiceConversation now checks each finished utterance against a spoken
stop-command matcher BEFORE submitting: 'stop', 'stop listening', 'never
mind', 'goodbye', 'cancel', 'that's all', etc., optionally addressed
('hey hermes, stop'). A match ends the conversation (flips enabled=false,
which drives the existing end() teardown — mic close, playback stop, wake
re-arm) instead of sending a turn.
Deliberately conservative: only a WHOLE-utterance stop phrase matches, so
substantive requests that merely contain 'stop' ('stop the docker
container', 'how do I stop a process') still go through.
- apps/desktop/src/lib/voice-stop-word.ts: isVoiceStopCommand() matcher
- use-voice-conversation.ts: onStopWord option + intercept before submit
- use-composer-voice.ts: wire onStopWord -> end the conversation
- 6 matcher tests (bare/multi-word/addressed stop; substantive requests
with 'stop' pass through; bare address words don't match)
- docs: note the spoken-stop behavior
A short, bright, rising two-note ding (G5 -> C6) plays the moment 'Hey
Hermes' is detected, before voice capture starts, so it's obvious the
wake registered. Deliberately distinct from the turn-end completion cue
(that one settles; this one rises = 'listening'). Reuses the same
lightweight WebAudio synthesis as completion-sound.ts — no asset to ship
— and is gated by the shared sound-mute toggle ($hapticsMuted), so
muting turn-end sounds silences it too.
- apps/desktop/src/lib/wake-sound.ts: playWakeSound()
- wiring.tsx: fire it at the top of the wake.detected handler
- 3 tests (plays two-note chime, silent when muted, never throws with
no WebAudio)
Ear icon next to the voice controls: highlighted while listening, muted
when off, hidden when the backend reports the wake word unavailable.
Backed by a feature-owned $wakeWord nanostore synced by both the button
(wake.start/stop) and the gateway-ready auto-arm (status-then-arm), so
the UI always reflects the real listener state; start refusals surface
their reason as a tooltip notice. i18n en/zh/zh-hant/ja.
The bubble, the queue panel, and the queue editor all showed a queued or
sent /skill turn's expanded body. Thread the invocation through submit and
the queue entry, and chip a leading slash command the way a mid-prose one
already chips.
The composer re-asked the gateway for the command catalog on every open
and for complete.slash on every keystroke. Both scan the skills dir on
the backend, so opening the menu cost a round trip against data that
only moves when a skill is added, removed, or toggled.
Hold both behind a one-hour cache keyed per query, and invalidate it
where the skill set actually changes: hub install/uninstall/update, a
Capabilities toggle, bulk toggle, archive, the agent's own skill_manage
call, and a profile switch. A cached query also skips the debounce and
the spinner, so a warm menu opens in the same frame.
The bare-slash list showed no skills at all: the backend categorizes
registry commands but appends skills to the flat pairs list only, and
the popover prefers the categorized layout. Re-add the uncategorized
leftovers under a Skills header, so /clean and /work are visible on /
and not just after typing enough of the name to match.
Picking a model closed the menu and then flashed the pill's tooltip over
the fresh selection. The existing focus guard gated on `:focus-visible`
alone, which does not separate a mouse pick from a Tab: the dropdown
autofocuses its search field and keyboard-navigates its rows, so Chromium
is already in keyboard modality when the menu restores focus to the
trigger, and the guard never fired.
Track the device behind the last real interaction and use it to qualify
`:focus-visible`. A mouse pick reports `pointer` and stays silent; Tab
focus still opens the tip.
A repeated id in the transcript reached assistant-ui's MessageRepository,
which throws on the second link and takes the whole workspace pane down to
the contribution error boundary — the pane the user is actually working in.
Dedupe where the repository export is built, so no upstream transcript bug
can crash the pane, and close the journal merge path that produced one: a
resume that replays a still-journaled turn appended rows the base already
held by id.
A mid-stream steer persists an interrupted-turn checkpoint so the model knows
its reply was cut off. That scaffolding — "[This response was interrupted by a
user correction.]" and the "Visible response before the interruption:" header —
was written straight into message content, so every reload painted the raw
machinery as an assistant bubble (and merged it into the preceding tool-call
bubble). Steered transcripts became unreadable.
Reuse the existing display/replay split instead of inventing new surface:
- Carry the scaffolded form in the server-only api_content sidecar (the exact
bytes replayed to the provider), keep content the user's/agent's real words.
- When nothing reached the screen there is no clean form, so mark the row
display_kind=hidden — replayed to the model, dropped by every transcript
surface, exactly like compaction-reference rows.
- Honor display_kind=hidden in the gateway's _history_to_messages projection
(it only sniffed the [System: convention), so the checkpoint can't leak
through the live/resume path to the TUI/CLI either.
The model still receives the full interrupted context on the wire; the
transcript shows the partial reply and the user's correction.
displayContentForMessage re-derived every ref from the attached context
block and re-emitted it as a detached list above the text. Now that the
token survives expansion, that list duplicated the inline chip. Hoist
only the refs the prose is missing, so turns persisted by an older
backend still render their chips.
Empty Enter while busy was a hard no-op, so a queued turn could only be
sent early via the panel's send arrow or Cmd+Shift+K. With prompts
queued, a second Enter now promotes the head and interrupts, mirroring
the idle empty-Enter drain. Nothing queued keeps the old no-op.
The panel's send-now row action switches from a bare up-arrow to the
return glyph, so the row states the keybind the double-send uses.
The resting chip fill and the bold weight both fought the brand mark for
attention. Links now sit in the theme's primary color at body weight, and
the tint fades in on hover.
Weight is overridden on .link-chip itself: `@tailwindcss/typography` sets
`prose a { font-weight: 500 }`, which outranks a utility class on the anchor,
so the per-call-site classes could never win.
PrettyLink leads with the site's mark, so a GitHub PR link reads as a GitHub
link at a glance. The artifacts pane uses the same lookup in place of its
generic chain glyph; unknown hosts render as before.
A lookup table of ~170 hosts plus a resolver that walks the hostname's
suffixes, so subdomains inherit their parent's mark (gist.github.com) while
a more specific entry (docs.google.com) still wins over the general one.
Typing a real message instead of picking an option left the agent parked:
the clarify blocks inside its tool batch waiting on clarify.respond, so a
follow-up routed through steer/queue sat undelivered until the 5-minute
clarify timeout. skipClarifyRequest answers the parked question with the
same empty answer the card's own Skip button sends, so the tool returns
and the turn carries on with the user's words.
