The layout tree persists the active tab, but every reload landed on main
anyway: boot's route resume sets $selectedStoredSessionId, and the
selection listener in store/session-states.ts treats every selection
change as a navigation — noteActiveTreeGroup(null) +
revealTreePane('workspace') — fronting the workspace tab over the
persisted active tile and then persisting that clobber.
A cold-start restore is a re-attachment, not a navigation, so
use-route-resume now arms a one-shot (markSelectionRestore) before
dispatching the window's FIRST resume; the selection listener consumes
it and skips homing exactly once. Every later resume — sidebar click,
route change, reconnect — homes as before, and starting on /new
consumes boot status too so a subsequent session open still homes.
Backspace path-ascend keeping the leading command pill, folder picks
alongside a command pill, commits spanning Chromium-split text nodes,
slash-pill hydration boundaries (committed vs half-typed vs arg-taking),
and replaceBeforeCaret refusing across chip boundaries.
textBeforeCaret serialized chip labels into the string the trigger
regexes scan, so a leading /work pill made the anchored command regex
swallow the rest of the line as its argument — silencing the @ popover
for the whole message. Chips now contribute an object-replacement
placeholder and <br> a newline, so committed pills can't poison
detection.
Two halves of one bug class: the composer treated 'rebuild the editor
from serialized text' as routine, and serialized text couldn't express a
slash pill.
In-place commits: Chromium fragments text nodes around
contenteditable=false chips, so the old commit path — whole token in ONE
text node with the caret at its end — failed almost every keyboard pick
and fell into the rebuild fallback. rangeBeforeCaret now walks the token
backwards across sibling text nodes and refuses only at real boundaries
(chip, <br>, block), making in-place replacement the default for picks,
folder descends, Backspace path-ascends, and action items in both the
main composer and the user-edit composer.
Slash-pill hydration: renderComposerContents re-chipped @kind:value refs
but never /command, so any surviving rebuild (draft restore, undo, the
fallback above) demoted a committed command pill. A leading no-arg
/command now hydrates back to its pill; arg-taking commands stay text
since their tail may be uncommitted prose.
Also: popover picks bank an undo point in the main composer, and Tab is
swallowed while completions are in flight instead of moving focus out of
the composer.
The poll this page's pairing block rode was retired hours earlier by
f8e07a332, which moved Messaging onto platforms.changed. That signal is the
mtime of gateway_state.json — where the gateway persists connect/disconnect
health — and a new pairing request moves none of it. So on an event-capable
backend a pending row stayed invisible until something unrelated
reconnected, and the count badge with it.
Adds pairing.changed to the change watcher, signed off the pending/approved
ledgers across the global store and every profile's own. _rate_limits.json
is deliberately excluded: it moves on every unauthorized DM, including ones
that produce no new row, so signalling on it would refetch for nothing.
The page now refreshes platforms and pairing on their own signals rather
than one combined call, and the legacy visible-tab poll (older backends)
covers both.
Desktop had no pairing surface at all. Someone DMs the bot, gets a code,
and lands in pending — where only the dashboard or `hermes pairing approve`
could let them in. That gap is why the dashboard's broken approve button
went unnoticed for so long: desktop users never saw the flow.
This puts it where the decision already lives. The messaging page is
already master-detail by platform, already polls every 6s, and already owns
"who can talk to this bot" — the allowlist env vars in the same detail pane
are what an approval writes into. A pending block sits above the credential
fields (approving is the hot path); a count badge on the platform row is the
discovery mechanism, so you see "Telegram 2" whenever you open Messaging for
any reason. Neither renders when nobody is waiting: approvals are rare, and
a permanent empty state would be chrome on a page that is otherwise about
credentials.
Approval sends the row's request_id and never a code — the code is the
requester's proof that the channel is theirs and is never returned by the
API. Rows paint optimistically from a snapshot and roll back visibly on
failure, and a 429 gets its own message since the code path's lockout is a
condition the operator can only wait out.
Pairing rides the existing platform refresh via allSettled, so a backend
without the endpoint yields no rows instead of blanking the page.
Both surfaces passed the sidebar's in-place intent, which means "load it
into main when it isn't already on screen" — right for a row you clicked
in a list you were looking at, wrong for a chat opened from outside the
workspace. Neither had a surface of its own, so they took the one you
were using.
Add a stack intent for that case. It focuses the session when it's
already open, spends an unused draft tab when there is one, and only
falls back to main while main is itself a blank draft. Modifiers still
force a tab or window.
newSessionOpensTab answers a question that isn't specific to the sidebar
"+" — is there a conversation on main that must not be discarded — and the
palette needs the same answer. Fold it into open-session as
mainChatOccupied so both callers share one definition.
The thread's bottom clearance is composer + status-stack + 2rem, and both
inputs are measured by JS onto the owning [data-chat-surface]. The surface-var
helpers fell back to document.documentElement when they couldn't resolve one —
but :root is where every surface's DEFAULTS live, so a single stale write there
becomes a global floor under every thread's clearance until reload.
An unowned publisher has nowhere to publish to, so it now publishes nowhere.
The change watcher (#73673) missed one always-on-while-mounted timer: the
Messaging page polled /api/messaging/platforms every 6s for connection
status. The gateway already persists platform connect/disconnect/health to
gateway_state.json, so watch that file's mtime and broadcast
platforms.changed (floored to 5s — the gateway also rewrites the file for
in-flight-count bookkeeping), route it through live-sync like its
siblings, and refresh the page on the tick. Older backends keep the
legacy visible-tab poll verbatim.
Finishes the always-on poll sweep for #73618.
The voice-interruption fix (5081551f0) covered the Python surfaces (CLI +
TUI gateway) but the desktop app has its own mic path in voice-barge-in.ts,
which still had both bugs on Windows:
- HALF-DUPLEX GAP: the barge monitor only opened when TTS playback started;
during LLM generation no mic was listening at all.
- PLAYBACK DEAFNESS: the monitor calibrated its noise floor while the
speakers were already playing TTS, baking bleed into the floor. On
Windows, Chromium echoCancellation does not reliably cancel same-app
output (measured live: quiet floor ~35-50 RMS vs playback bleed
600-1700 RMS), making the trigger unreachable.
Changes mirror tools/voice_mode.full_duplex_listen:
- voice-barge-in.ts: phase-aware full-duplex monitor — quiet-only
calibration (floor held through playback, never recalibrated against
bleed), playback min-trigger clamp + ceiling, 500ms grace on playback
onset only, windowed-majority detection.
- use-voice-conversation.ts: monitor arms at turn submit and spans
generation + playback (ensureBargeMonitor, idempotent). Mid-generation
speech fires the new onInterrupt callback; spoken stop-word during a
barge ends the conversation; submit waits for the interrupt to settle.
- use-composer-voice.ts + composer/index.tsx: plumb onInterrupt: haltRun
(same seam as the Stop button).
Tests: use-voice-conversation.test.tsx (6 tests) covering monitor lifecycle
across generation + playback, mid-generation interrupt, stop-word handling.
npm run typecheck + eslint clean; remaining desktop vitest failures
reproduce on a clean tree (pre-existing Windows env failures).
`hermes.desktop.lastSessionId` is keyed by owning profile (143942d49), but
`hermes.desktop.lastRoute` stayed global -- and cold-start restore prefers the
route over the id. A session route embeds a session id in its path, so
relaunching under profile B navigated straight into a session owned by profile
A, bypassing the id scoping entirely (#67603 family).
Key the remembered route by the same owner the id already uses
(rememberedSessionProfile), read it back for the active profile on restore, and
keep the default profile on the original unsuffixed key so existing installs'
remembered route survives the upgrade.
Salvaged from the profile-hint work in #49619, which threaded an explicit
profile through every route/IPC/window surface to reach the same end. The server
now stamps session ownership unconditionally (#74033), so the resolver already
gets the right answer and only this persisted key was left crossing profiles.
Co-authored-by: d31tcjg <d31tcjg@users.noreply.github.com>
Pebrd's fix is right but the `useMemo` it edits didn't list
`messagingSessions` as a dependency, so the index would go stale as the
messaging slice refreshed. Rather than regex around logic buried in a
1500-line component, lift it into `buildSessionByAnyId` where the
contract can be tested directly — the extraction LeonSGP43 proposed in
sibling PR #49896.
The test asserts the invariant that actually matters: a pin resolves
from every slice the sidebar fetches. Reverting the function to its
two-slice form fails it on the messaging and lineage-root cases.
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
The sessionByAnyId map only indexed cronSessions and visibleSessions
(local CLI chats), so a pinned gateway session (Telegram, etc.) stored
its pinId in localStorage but could never be resolved back to a
SessionInfo object — the Pinned section rendered empty for those.
Include messagingSessions in the lookup so pinned gateway chats appear
correctly at the top of the sidebar.
Long thinking/tool stretches in a voice conversation are dead air — the
user cannot tell whether the agent is alive. New: quiet, repeating soft
bubble blips while the agent works and no speech audio is flowing.
- tools/voice_mode.py: numpy-synthesized blips (no binary assets) — two
alternating low pitches (G4/E4) with pitch glide + smooth attack/decay
envelopes, ~0.8-1.2s randomized spacing, volume = voice.beep_volume * 0.5.
start_thinking_sound(should_play=...) / stop_thinking_sound() daemon-loop
lifecycle; macOS-TCC-safe (sounddevice output gated there → silent skip,
no per-second afplay churn). New mark_audio_output_active()/
is_audio_output_active() ref-count wraps play_audio_file and the
streaming OutputStream sentence writes so "audio is flowing" is accurate.
- Config: voice.thinking_sound (default true) off-switch.
- cli.py: starts when a voice-mode turn begins, per-blip gate skips while
TTS speaks / mic records / barge capture owns the mic; stopped in the
chat() finally on every exit path.
- tui_gateway/server.py: same lifecycle around _run_prompt_submit turns
(voice mode on), gated on is_audio_output_active + continuous capture.
- Desktop: renderer owns voice-conversation audio, so a matching WebAudio
implementation (src/lib/thinking-sound.ts, same envelope/pitches) runs
while conversation status === "thinking"; honors voice.thinking_sound
(via config store) and the shared sound-mute toggle; stops instantly on
speaking/listening/end.
One owner for the wording: voice_stop_hint() in tools/voice_mode.py —
sources the phrase from voice.stop_phrases (first entry) so a custom
phrase renders correctly, and returns "" when the feature is disabled
(stop_phrases: []) so no surface shows a hint.
- CLI: printed in /voice on output (style-matched dim notice).
- TUI: voice.toggle action=on now carries stop_hint; the Ink client
renders it in the "Voice mode enabled" block (older gateways omit
the field — no hint, no crash).
- Desktop: the renderer voice loop never touches tools/voice_mode.py,
so the phrase is read from config (voice.stop_phrases → $voiceStopPhrase
store, seeded in use-hermes-config) and shown as an info toast when a
voice conversation starts. i18n: en/ja/zh/zh-hant/ar.
Session rows served without ?profile= carried no profile field, so in
multi-profile desktops the default profile's sessions circulated unowned:
resolveStoredSession cached them profile-less, resolveSessionProfile returned
undefined, and session.resume targeted whichever gateway was active -- opening
a default-profile session from a non-default window failed with
"session can't be found" while the reverse direction worked (#67603 family).
Server: GET /api/sessions/{id} and GET /api/sessions now stamp profile/
is_default_profile unconditionally -- the serving profile is always known
(_cron_default_profile() when the request is unscoped).
Renderer: resolveStoredSession treats a profile-less $sessions cache hit as
unresolved when >1 profile exists (falls through to the stamped by-id ladder)
and back-fills the active profile on bare by-id hits from older backends, so
unowned rows are never re-cached.
Verified via CDP against a live 4-profile renderer: bare by-id GET returned
hasProfileField:false and the stale cache rows matched; with the fix both
lookups return the owning profile and the resume routes to the right backend.
Saying OR typing a configured stop phrase (voice.stop_phrases, default
"stop") now ends the voice chat everywhere, not just classic CLI PTT:
- hermes_cli/voice.py: new explicit on_stop_phrase callback through
start_continuous/stop_continuous. The force-transcribe path previously
DISCARDED the stop phrase silently — with auto_restart=False the client
re-arms the next capture, so the conversation never ended. Both halt
paths now fire on_stop_phrase (fallback: on_silent_limit for legacy
callers) as user intent, distinct from the no-speech timeout.
- tui_gateway/server.py: voice.record wires on_stop_phrase and emits
voice.transcript {stop_phrase: true} after flipping HERMES_VOICE(_TTS)
off and stopping streaming TTS — same teardown as /voice off. The TTS
barge-in monitor stop-checks its transcript too. prompt.submit consumes
a TYPED bare stop phrase at the server-side choke point when voice mode
is on (returns {voice_stopped: true}, no turn starts).
- ui-tui: voice.transcript {stop_phrase} ends voice mode with a clear
'voice chat ended' notice (distinct from the no-speech-limit message);
submitPrompt releases the busy latch on a consumed voice_stopped reply.
- cli.py: _typed_voice_stop in process_loop — typing a bare stop phrase
while voice mode/continuous is active ends voice mode instead of
sending 'stop' to the agent; typed 'stop' outside voice mode is
unchanged. Voice transcripts skip the check (already stop-checked).
- desktop: interceptsTypedVoiceStop — the composer's onSubmit ends the
live voice conversation (same path as clicking end on the pill) when a
bare stop command is typed with no attachments; renderer-owned loop, so
handled client-side like the existing spoken isVoiceStopCommand.
- tools/voice_mode.py: transcribe_recording never lets the Whisper
hallucination filter swallow a configured stop phrase (e.g. 'bye'
configured as a stop phrase is both a hallucination-blocklist entry and
a stop phrase — stop-phrase check now wins).
Tests: continuous-loop signal (sabotage-verified), force-transcribe stop
signal + legacy fallback, hallucination-filter ordering, typed-stop CLI
unit tests (voice on/off/longer text), prompt.submit typed-stop gateway
tests, TUI vitest for stop_phrase event handling, desktop vitest for the
typed-stop interceptor.
The Settings "Open plugins folder" action and the runtime disk-plugin
scanner both derived the plugin directory from getStatus().hermes_home.
Against a remote backend that value is a path on the REMOTE box (or
undefined), producing `undefined/desktop-plugins` — the folder action
errors ("Could not open the plugins folder undefined") and disk-plugin
discovery silently finds nothing, even with a valid local plugin.js.
Add an Electron-owned IPC resolver (hermes:fs:desktopPluginsRoot) that
returns <HERMES_HOME>/desktop-plugins computed from the main-process
HERMES_HOME — the local Electron path, valid in every connection mode —
creating it on demand. Route both the Settings folder action and the
runtime scanner through it, so a remote backend never determines the
local filesystem location used for Desktop runtime plugins.
Fixes#66899
Three GUI Capabilities-tab defects reported on Windows:
1. Browser rows stuck on 'Setup required' after a successful setup run.
Root causes, all in the readiness probe (not the installer):
- _has_agent_browser() never searched the Hermes-managed Node dir
(%LOCALAPPDATA%/hermes/node / $HERMES_HOME/node/bin) where the
Windows install lands, and probed node_modules/.bin/agent-browser
as the extensionless POSIX shim, which fails exec on Windows
(WinError 193) — now resolved via PATHEXT-aware shutil.which
against both rungs, mirroring _find_agent_browser().
- Cloud rows (Nous Subscription Browser Use, Browserbase, Browser
Use, Firecrawl) declared post_setup: agent_browser, whose
readiness gate requires a LOCAL Chromium build the cloud never
uses — switched to the cloud-scoped 'browserbase' hook (CLI-only).
- _agent_browser_installed() could read browser_tool's stale cached
'Chromium missing' result from before the install ran in the
spawned post-setup process — cache now dropped before probing so
the pill flips to Ready right after a successful run.
2. No way to tell which backend is active, and clicking a row to read
its details silently rewrote config. Row click now only
expands/collapses; activation is an explicit 'Use this backend'
button, the active row carries an 'Active' pill, and the expanded
active row says 'This is your active backend'.
3. OpenAI TTS showed one model and one voice. The options were always
defined but rendered through a native <datalist>, which filters by
the field's current value — a field already set to a valid option
suggested only itself. Replaced with a real combobox (Input +
dropdown) that lists every option, and voice suggestions now track
the selected model per the OpenAI TTS docs: tts-1/tts-1-hd = 9
voices, gpt-4o-mini-tts = 13 (adds ballad, verse, marin, cedar).
Type-to-focus routes through requestComposerFocus('active'), which resolved
to a module-level activeTarget claim. Inactive tabs stay mounted under
data-pane-hidden, so typing in a session tile then clicking the main tab
left activeTarget on the buried tile: use-keybinds preventDefaults the
keystroke, the buried composer ignores the request (or is filtered out),
and the main composer never sees it. Same class of bug after the inline
edit composer unmounts with activeTarget still 'edit'.
Heal 'active' against the visible data-composer-target stamp (the same
visibility policy as every other document-wide surface lookup), release
the claim on real unmounts (useComposerDraft + user-edit-composer), and
keep getActiveComposer honest so Esc / soft / / voice agree with the
keyboard path.
The unmount release is salvaged from #72625 (@briandevans); this PR adds
the keep-alive tab heal his unmount-only fix couldn't cover.
Co-authored-by: briandevans <252620095+briandevans@users.noreply.github.com>
Adds gpt-transcribe (OpenAI's new file-transcription model, $0.0045/min)
to the OpenAI STT provider:
- OPENAI_MODELS set: gpt-transcribe is recognized so provider
auto-correction keeps it on OpenAI and rejects it on Groq
- Language hint wiring: gpt-transcribe replaces the singular
'language' field with a 'languages' list; the API rejects the legacy
field, so the hint is sent via extra_body {languages: [..]}
- Config comment (DEFAULT_CONFIG), cli-config.yaml.example, desktop
settings enum, and docs (en + zh-Hans) updated
- Tests: model pass-through, languages-list hint shape, legacy singular
hint preserved for gpt-4o-transcribe, Groq auto-correction
gpt-live-transcribe (realtime WebSocket, $0.017/min) is NOT wired here:
the file-based STT pipeline has no realtime session path; it belongs in
a future realtime/voice-mode integration.
The command palette named the action but not the install it acts on. Carry the
version and its commit diff on the row, resolved from the shared resolver so it
reads identically to the statusbar.
The statusbar derived its client and backend version labels inline, in two
near-identical blocks. Move the wording into a pure resolver so every surface
that names an install agrees on the label, the commit diff, and the tooltip.
The binding check landed unscoped, so it fired for every caller passing a
sessionId/storedSessionId pair — not just queue drains. A slash skill
dispatch into a fresh ⌘T tab passes exactly that shape (sessionId=tab
runtime, storedSessionId=tab stored) with no central binding recorded yet,
so the check nulled the target and the kickoff dropped instead of landing
in the tab.
Only a drain pairs identifiers from two different clocks; every other
explicit-target caller resolves both ids in the same tick and is
authoritative by construction. Gate on fromQueue and add the scoping
invariant as a test.
Also refresh the two drain tests for `queued: true`, which prompt.submit
started sending for queued drains in ab68c5efe after this work branched.
Co-authored-by: theone139344 <theone139344@users.noreply.github.com>
A queue drain pairs two identifiers from different clocks: the queue key
(flips with the route) and the explicit runtime id (lags a resume behind).
Mid-switch the composer can fire a drain with storedSessionId=B but
sessionId=A-runtime, and prompt.submit then lands B's queued prompt — and
its whole answer turn — inside session A.
Make the central runtime binding authoritative for queued sends: when the
explicit runtime id no longer matches the binding recorded for the target
stored session, adopt the binding (or drop to the stored-id resume path
when none exists yet). The identity pair (storedSessionId === sessionId)
is the fresh-chat fallback and stays untouched.
Tests: re-home-via-resume and rebind-to-central-runtime; existing
background-drain and sleep/wake cases declare central bindings explicitly.
Null-fallback guard already on main; this PR is the remaining half.
Sibling of #65254 (main-slot endpoint preservation): the auxiliary scope of
POST /api/model/set dropped the request's base_url/api_key on the floor, so
an aux slot pinned to a custom/local endpoint silently depended on
model.base_url — and broke the moment the main slot switched away and
cleared it. The aux resolver already reads auxiliary.<task>.base_url/api_key
(_resolve_task_provider_model); this persists them.
Desktop side: setAuxiliaryToMain / applyAuxiliaryDraft now carry the
user-defined provider's api_url as base_url, mirroring applyMainModel.
Two follow-ups from the voice PR (#70509).
1. macOS ARM64 onnx migration. Existing users who pinned
openwakeword.inference_framework=onnx before the tflite fix landed kept a
wake word that arms but never fires (ONNX's embedding model is broken on
Apple Silicon, upstream #336). New resolve_inference_framework() honors an
explicit framework everywhere ONNX actually works, but coerces the one
provably-dead combination (explicit onnx + macOS ARM64) to tflite with a
one-time warning. No config mutation; empty still falls back to the platform
default. Both read sites (engine init + requirements check) route through
the shared resolver.
2. Voice turn-timeout leak. Each listen cycle reassigned turnTimeoutRef
without clearing the prior 60s timer, so a stale timer from an earlier
cycle could fire handleTurn() mid-way through a later listen — after enough
idle re-listens this wedged the loop into a non-re-arming state (the
'voice chat deactivates after ~a minute' report). Clear before re-arm.
Tests: 64 wake tests (added onnx-coercion / intel-kept / tflite-kept /
empty-default cases; updated the stale 'explicit onnx kept on ARM64' test
that encoded the old broken behavior), 39 desktop voice/wake vitest, tsc +
eslint clean.
The sidebar's messaging section renders one PlatformAvatar (labelIcon) and
StatusDot per platform group. The sidebar re-renders on every streaming tick
($sessions/$workingSessionIds/$messagingSessions churn), and both leaves were
unmemoized — so every platform's avatar + dot re-rendered on each delta even
though their props (platformId/tone/className) never changed.
- PlatformAvatar: memo(forwardRef(...)) — pure fn of platformId/name/class/style
- StatusDot: memo() — pure fn of tone/class
Measured (Messaging open, settled, 2 sessions streaming, 3s window):
before: 96 wasted (PlatformAvatar 32, StatusDot 32, + brand icons)
after: 0 wasted
Both are shared primitives; the memo also helps every other consumer
(session rows, gateway menu, session tiles) that renders them under a hot
parent.
- command-center/command-palette/skills: import ordering + drop unused
HermesGateway type
- settings: wrap openSubView/openProviderView/openKeysView in useCallback so
the navGroups memo deps are honest and stable
- command-center: add setSection to navGroups memo deps
The sidebar stays mounted beneath every overlay/page, and it subscribes to
$sessions + $workingSessionIds — both tick on every streaming token. An
unmemoized SidebarSessionRow re-rendered the whole list (Codicon, labels,
status dots) on each delta, and that churn bled into every overlay opened on
top: Cron, Profiles, Agents, Starmap, Webhooks, Command Center, Settings.
- SidebarSessionRow: memo() with a custom comparator that ignores the pure
id-forwarding callbacks (fresh closures by design) and compares only the
data that changes what the row paints. Rows bail out while siblings stream.
- SessionStatusDot: the 5 $...SessionIds arrays now read via useStoreSelector
returning this session's boolean, so a dot repaints only when ITS OWN
membership flips, not on every array tick.
- Artifacts: stable cellCtx (useMemo) + memoized Primary/Location/Session
cells so a link-title fetch on one row stops re-rendering the whole table.
Measured before/after (2s idle, sessions streaming), sidebar-fed overlays:
Cron 407->~30 wasted, Profiles 732->~80, Agents 132->9, Starmap 188->56,
Webhooks 154->22.
Register Photon as a messaging platform in the Desktop sidebar:
- Add Photon brand icon (three-bar mark) to PlatformAvatar catalog
- Add photon to SOURCE_LABELS, SOURCE_ALIASES, and MESSAGING_SESSION_SOURCE_IDS
Closes#46761
