`submit` measures the scroll jump when a turn is appended; nothing measured
the jump when a session is opened, which is the prepend/settle path. Clicks
sidebar rows and tracks how far the bottom turn moves after first paint.
Opening a session painted a small first budget, then prepended the rest of
the turns above the viewport with nothing holding the scroll position, so the
view was stranded near the top of the transcript until use-stick-to-bottom's
ResizeObserver caught up a frame or two later. The settle loop couldn't cover
for it either: keyed on sessionKey alone it measured an EMPTY viewport on a
cold load, saw a stable height, and declared the load settled hundreds of ms
before the messages arrived.
Anchor before the backfill prepends (the mechanism "Show earlier" already
uses) and re-arm the settle loop when the transcript actually lands.
Measured over 12 real sidebar-click loads: 12,179 -> 544 px of post-paint
movement per load, worst single jump 13,369 -> 2,296 px. Render churn is
unchanged (fewer commits and total renders, same wasted count).
Pebrd's fix is right but the `useMemo` it edits didn't list
`messagingSessions` as a dependency, so the index would go stale as the
messaging slice refreshed. Rather than regex around logic buried in a
1500-line component, lift it into `buildSessionByAnyId` where the
contract can be tested directly — the extraction LeonSGP43 proposed in
sibling PR #49896.
The test asserts the invariant that actually matters: a pin resolves
from every slice the sidebar fetches. Reverting the function to its
two-slice form fails it on the messaging and lineage-root cases.
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
The sessionByAnyId map only indexed cronSessions and visibleSessions
(local CLI chats), so a pinned gateway session (Telegram, etc.) stored
its pinId in localStorage but could never be resolved back to a
SessionInfo object — the Pinned section rendered empty for those.
Include messagingSessions in the lookup so pinned gateway chats appear
correctly at the top of the sidebar.
The pin bridge only ever pushed: localStorage to the backend, never
back. Two apps on the same gateway each kept their own localStorage, so
a pin made on the Mac never appeared on the Windows app.
Now that a pinned row is guaranteed to be in the page, its absence says
nothing about its pin state — which makes the server row authoritative.
`pullRemotePins` adopts pins this app hasn't seen and drops local pins
the server says are gone, keying on the durable lineage root so a pin
survives compression tip rotation. Adopted pins are recorded as already
mirrored rather than echoed back as a redundant write.
A list request in flight when we PATCH still carries the old value, so
honouring it would silently undo the pin the user just made. Writes are
guarded for the lifetime of their own request — no timers, no wall-clock
windows. A runtime predating the flag sends no `pinned` at all; that's
treated as no opinion and leaves the local set alone.
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Co-authored-by: Kyzcreig <9063726+Kyzcreig@users.noreply.github.com>
Co-authored-by: aman-merchant <274313970+aman-merchant@users.noreply.github.com>
Co-authored-by: rerdi92 <76791321+rerdi92@users.noreply.github.com>
Long thinking/tool stretches in a voice conversation are dead air — the
user cannot tell whether the agent is alive. New: quiet, repeating soft
bubble blips while the agent works and no speech audio is flowing.
- tools/voice_mode.py: numpy-synthesized blips (no binary assets) — two
alternating low pitches (G4/E4) with pitch glide + smooth attack/decay
envelopes, ~0.8-1.2s randomized spacing, volume = voice.beep_volume * 0.5.
start_thinking_sound(should_play=...) / stop_thinking_sound() daemon-loop
lifecycle; macOS-TCC-safe (sounddevice output gated there → silent skip,
no per-second afplay churn). New mark_audio_output_active()/
is_audio_output_active() ref-count wraps play_audio_file and the
streaming OutputStream sentence writes so "audio is flowing" is accurate.
- Config: voice.thinking_sound (default true) off-switch.
- cli.py: starts when a voice-mode turn begins, per-blip gate skips while
TTS speaks / mic records / barge capture owns the mic; stopped in the
chat() finally on every exit path.
- tui_gateway/server.py: same lifecycle around _run_prompt_submit turns
(voice mode on), gated on is_audio_output_active + continuous capture.
- Desktop: renderer owns voice-conversation audio, so a matching WebAudio
implementation (src/lib/thinking-sound.ts, same envelope/pitches) runs
while conversation status === "thinking"; honors voice.thinking_sound
(via config store) and the shared sound-mute toggle; stops instantly on
speaking/listening/end.
One owner for the wording: voice_stop_hint() in tools/voice_mode.py —
sources the phrase from voice.stop_phrases (first entry) so a custom
phrase renders correctly, and returns "" when the feature is disabled
(stop_phrases: []) so no surface shows a hint.
- CLI: printed in /voice on output (style-matched dim notice).
- TUI: voice.toggle action=on now carries stop_hint; the Ink client
renders it in the "Voice mode enabled" block (older gateways omit
the field — no hint, no crash).
- Desktop: the renderer voice loop never touches tools/voice_mode.py,
so the phrase is read from config (voice.stop_phrases → $voiceStopPhrase
store, seeded in use-hermes-config) and shown as an info toast when a
voice conversation starts. i18n: en/ja/zh/zh-hant/ar.
Session rows served without ?profile= carried no profile field, so in
multi-profile desktops the default profile's sessions circulated unowned:
resolveStoredSession cached them profile-less, resolveSessionProfile returned
undefined, and session.resume targeted whichever gateway was active -- opening
a default-profile session from a non-default window failed with
"session can't be found" while the reverse direction worked (#67603 family).
Server: GET /api/sessions/{id} and GET /api/sessions now stamp profile/
is_default_profile unconditionally -- the serving profile is always known
(_cron_default_profile() when the request is unscoped).
Renderer: resolveStoredSession treats a profile-less $sessions cache hit as
unresolved when >1 profile exists (falls through to the stamped by-id ladder)
and back-fills the active profile on bare by-id hits from older backends, so
unowned rows are never re-cached.
Verified via CDP against a live 4-profile renderer: bare by-id GET returned
hasProfileField:false and the stale cache rows matched; with the fix both
lookups return the owning profile and the resume routes to the right backend.
Two residual gaps from the #72707/#72632 probe-hardening series:
- backendSupportsServe never forwarded backend.shell to the serve
--help probe. A .cmd/.bat shim backend (which carries shell: true in
its step-4 descriptor) makes execFileSync throw EINVAL on modern
Node; the bare catch then caches supported=false for the process
lifetime, permanently routing that backend through the legacy
dashboard form. Forward the flag.
- The Windows python-discovery probes ran with no timeout at all:
reg query (registry read) and py.exe -c 'import sys;...' (bare
interpreter startup) are both synchronous execs on the boot path;
a wedged reg.exe or python.exe would hang the resolver forever.
Bound reg query at 5s and the py.exe probe at the shared
PROBE_TIMEOUT_MS budget.
Follow-up to #73907 (probe timeout 15s + env override + timeout-only
retry), widening the same fix to the two sibling sites it missed:
- backendSupportsServe's serve --help probe kept a bare execFileSync
with its own 15s literal: same cold-Windows Python-startup class
(#72632 measured ~10.5s for --version cold), no retry, and a false
negative is cached for the process lifetime - silently routing a
modern runtime through the legacy dashboard form. Route it through
execProbeSync with the shared PROBE_TIMEOUT_MS (honours
HERMES_PROBE_TIMEOUT_MS) and the timeout-only retry.
- resolveHermesBackend step 4 called unwrapWindowsVenvHermesCommand
twice; the second call re-ran the same un-memoized import probe,
costing up to another full probe timeout on a hung interpreter for
an answer the first call already gave. Drop the redundant re-probe.
Part of the #72707 bug class (transient disconnect must not strand a
healthy install).
The old test asserted _warm_gateway_module was fire-and-forget (startup
completes in << SLOW_SECONDS). PR #73291 intentionally reversed this:
the import now runs synchronously before the lifespan yield because
run_in_executor didn't release the GIL on Windows + Python 3.11.
Updated the test to assert startup blocks for >= SLOW_SECONDS.
Saying OR typing a configured stop phrase (voice.stop_phrases, default
"stop") now ends the voice chat everywhere, not just classic CLI PTT:
- hermes_cli/voice.py: new explicit on_stop_phrase callback through
start_continuous/stop_continuous. The force-transcribe path previously
DISCARDED the stop phrase silently — with auto_restart=False the client
re-arms the next capture, so the conversation never ended. Both halt
paths now fire on_stop_phrase (fallback: on_silent_limit for legacy
callers) as user intent, distinct from the no-speech timeout.
- tui_gateway/server.py: voice.record wires on_stop_phrase and emits
voice.transcript {stop_phrase: true} after flipping HERMES_VOICE(_TTS)
off and stopping streaming TTS — same teardown as /voice off. The TTS
barge-in monitor stop-checks its transcript too. prompt.submit consumes
a TYPED bare stop phrase at the server-side choke point when voice mode
is on (returns {voice_stopped: true}, no turn starts).
- ui-tui: voice.transcript {stop_phrase} ends voice mode with a clear
'voice chat ended' notice (distinct from the no-speech-limit message);
submitPrompt releases the busy latch on a consumed voice_stopped reply.
- cli.py: _typed_voice_stop in process_loop — typing a bare stop phrase
while voice mode/continuous is active ends voice mode instead of
sending 'stop' to the agent; typed 'stop' outside voice mode is
unchanged. Voice transcripts skip the check (already stop-checked).
- desktop: interceptsTypedVoiceStop — the composer's onSubmit ends the
live voice conversation (same path as clicking end on the pill) when a
bare stop command is typed with no attachments; renderer-owned loop, so
handled client-side like the existing spoken isVoiceStopCommand.
- tools/voice_mode.py: transcribe_recording never lets the Whisper
hallucination filter swallow a configured stop phrase (e.g. 'bye'
configured as a stop phrase is both a hallucination-blocklist entry and
a stop phrase — stop-phrase check now wins).
Tests: continuous-loop signal (sabotage-verified), force-transcribe stop
signal + legacy fallback, hallucination-filter ordering, typed-stop CLI
unit tests (voice on/off/longer text), prompt.submit typed-stop gateway
tests, TUI vitest for stop_phrase event handling, desktop vitest for the
typed-stop interceptor.
On Windows, applyUpdates kills its own backend (releaseBackendLock)
BEFORE the venv-blocker preflight but only writes the on-disk update
marker AFTER the scan. Killing the backend drops the renderer's
WebSocket; the renderer reconnects within ~1s and the marker-only
waitForUpdateToFinish gate happily spawns a fresh 'hermes serve' inside
the update's own critical section. scanVenvBlockers then finds that
brand-new process and aborts with 'another Hermes process is using this
installation' — a different PID on every attempt, so Desktop self-update
can never succeed.
Fix: extract the gate into update-gate.ts (pure, DI-testable) and make
it consult BOTH signals — the on-disk marker AND the in-process
updateInFlight flag. The success path writes the marker before the flag
clears in applyUpdates' finally, so there is no instant where both are
false and a waiter can slip through. Also gate spawnPoolBackend, which
previously had no waitForLocalStart at all — a background profile window
could respawn a pool backend during the same window with the identical
abort.
Tests: update-gate.test.ts covers the open gate, the flag-only window
(the #73822 shape), the flag→marker handoff with no gap, and timeout.
Follow-ups on top of #66911's salvaged commit:
- hermes:fs:desktopPluginsRoot now resolves the ACTIVE desktop profile
(readActiveDesktopProfile) so named profiles keep their own
profiles/<name>/desktop-plugins root instead of sharing the global one
(profile-scope concern raised on the PR thread).
- startDirWatch in runtime-loader.ts was a third sibling site still
deriving the watch path from the backend's hermes_home (added by the
later fs-watch commit); routed through the same Electron-local
resolver, with regression coverage.
The Settings "Open plugins folder" action and the runtime disk-plugin
scanner both derived the plugin directory from getStatus().hermes_home.
Against a remote backend that value is a path on the REMOTE box (or
undefined), producing `undefined/desktop-plugins` — the folder action
errors ("Could not open the plugins folder undefined") and disk-plugin
discovery silently finds nothing, even with a valid local plugin.js.
Add an Electron-owned IPC resolver (hermes:fs:desktopPluginsRoot) that
returns <HERMES_HOME>/desktop-plugins computed from the main-process
HERMES_HOME — the local Electron path, valid in every connection mode —
creating it on demand. Route both the Settings folder action and the
runtime scanner through it, so a remote backend never determines the
local filesystem location used for Desktop runtime plugins.
Fixes#66899
Three GUI Capabilities-tab defects reported on Windows:
1. Browser rows stuck on 'Setup required' after a successful setup run.
Root causes, all in the readiness probe (not the installer):
- _has_agent_browser() never searched the Hermes-managed Node dir
(%LOCALAPPDATA%/hermes/node / $HERMES_HOME/node/bin) where the
Windows install lands, and probed node_modules/.bin/agent-browser
as the extensionless POSIX shim, which fails exec on Windows
(WinError 193) — now resolved via PATHEXT-aware shutil.which
against both rungs, mirroring _find_agent_browser().
- Cloud rows (Nous Subscription Browser Use, Browserbase, Browser
Use, Firecrawl) declared post_setup: agent_browser, whose
readiness gate requires a LOCAL Chromium build the cloud never
uses — switched to the cloud-scoped 'browserbase' hook (CLI-only).
- _agent_browser_installed() could read browser_tool's stale cached
'Chromium missing' result from before the install ran in the
spawned post-setup process — cache now dropped before probing so
the pill flips to Ready right after a successful run.
2. No way to tell which backend is active, and clicking a row to read
its details silently rewrote config. Row click now only
expands/collapses; activation is an explicit 'Use this backend'
button, the active row carries an 'Active' pill, and the expanded
active row says 'This is your active backend'.
3. OpenAI TTS showed one model and one voice. The options were always
defined but rendered through a native <datalist>, which filters by
the field's current value — a field already set to a valid option
suggested only itself. Replaced with a real combobox (Input +
dropdown) that lists every option, and voice suggestions now track
the selected model per the OpenAI TTS docs: tts-1/tts-1-hd = 9
voices, gpt-4o-mini-tts = 13 (adds ballad, verse, marin, cedar).
Type-to-focus routes through requestComposerFocus('active'), which resolved
to a module-level activeTarget claim. Inactive tabs stay mounted under
data-pane-hidden, so typing in a session tile then clicking the main tab
left activeTarget on the buried tile: use-keybinds preventDefaults the
keystroke, the buried composer ignores the request (or is filtered out),
and the main composer never sees it. Same class of bug after the inline
edit composer unmounts with activeTarget still 'edit'.
Heal 'active' against the visible data-composer-target stamp (the same
visibility policy as every other document-wide surface lookup), release
the claim on real unmounts (useComposerDraft + user-edit-composer), and
keep getActiveComposer honest so Esc / soft / / voice agree with the
keyboard path.
The unmount release is salvaged from #72625 (@briandevans); this PR adds
the keep-alive tab heal his unmount-only fix couldn't cover.
Co-authored-by: briandevans <252620095+briandevans@users.noreply.github.com>
Adds gpt-transcribe (OpenAI's new file-transcription model, $0.0045/min)
to the OpenAI STT provider:
- OPENAI_MODELS set: gpt-transcribe is recognized so provider
auto-correction keeps it on OpenAI and rejects it on Groq
- Language hint wiring: gpt-transcribe replaces the singular
'language' field with a 'languages' list; the API rejects the legacy
field, so the hint is sent via extra_body {languages: [..]}
- Config comment (DEFAULT_CONFIG), cli-config.yaml.example, desktop
settings enum, and docs (en + zh-Hans) updated
- Tests: model pass-through, languages-list hint shape, legacy singular
hint preserved for gpt-4o-transcribe, Groq auto-correction
gpt-live-transcribe (realtime WebSocket, $0.017/min) is NOT wired here:
the file-based STT pipeline has no realtime session path; it belongs in
a future realtime/voice-mode integration.
The command palette named the action but not the install it acts on. Carry the
version and its commit diff on the row, resolved from the shared resolver so it
reads identically to the statusbar.
The command palette's row called applyBackendUpdate() directly, so in local
mode it drove the backend checkout rather than the client, and nothing opened
the updates overlay to report either outcome.
Route it through the same target selection the statusbar and About panel use,
always surface the overlay, and re-check instead of applying when the active
target is already current.
The statusbar derived its client and backend version labels inline, in two
near-identical blocks. Move the wording into a pure resolver so every surface
that names an install agrees on the label, the commit diff, and the tooltip.
The binding check landed unscoped, so it fired for every caller passing a
sessionId/storedSessionId pair — not just queue drains. A slash skill
dispatch into a fresh ⌘T tab passes exactly that shape (sessionId=tab
runtime, storedSessionId=tab stored) with no central binding recorded yet,
so the check nulled the target and the kickoff dropped instead of landing
in the tab.
Only a drain pairs identifiers from two different clocks; every other
explicit-target caller resolves both ids in the same tick and is
authoritative by construction. Gate on fromQueue and add the scoping
invariant as a test.
Also refresh the two drain tests for `queued: true`, which prompt.submit
started sending for queued drains in ab68c5efe after this work branched.
Co-authored-by: theone139344 <theone139344@users.noreply.github.com>
A queue drain pairs two identifiers from different clocks: the queue key
(flips with the route) and the explicit runtime id (lags a resume behind).
Mid-switch the composer can fire a drain with storedSessionId=B but
sessionId=A-runtime, and prompt.submit then lands B's queued prompt — and
its whole answer turn — inside session A.
Make the central runtime binding authoritative for queued sends: when the
explicit runtime id no longer matches the binding recorded for the target
stored session, adopt the binding (or drop to the stored-id resume path
when none exists yet). The identity pair (storedSessionId === sessionId)
is the fresh-chat fallback and stays untouched.
Tests: re-home-via-resume and rebind-to-central-runtime; existing
background-drain and sleep/wake cases declare central bindings explicitly.
Null-fallback guard already on main; this PR is the remaining half.
Sibling of #65254 (main-slot endpoint preservation): the auxiliary scope of
POST /api/model/set dropped the request's base_url/api_key on the floor, so
an aux slot pinned to a custom/local endpoint silently depended on
model.base_url — and broke the moment the main slot switched away and
cleared it. The aux resolver already reads auxiliary.<task>.base_url/api_key
(_resolve_task_provider_model); this persists them.
Desktop side: setAuxiliaryToMain / applyAuxiliaryDraft now carry the
user-defined provider's api_url as base_url, mirroring applyMainModel.
Two follow-ups from the voice PR (#70509).
1. macOS ARM64 onnx migration. Existing users who pinned
openwakeword.inference_framework=onnx before the tflite fix landed kept a
wake word that arms but never fires (ONNX's embedding model is broken on
Apple Silicon, upstream #336). New resolve_inference_framework() honors an
explicit framework everywhere ONNX actually works, but coerces the one
provably-dead combination (explicit onnx + macOS ARM64) to tflite with a
one-time warning. No config mutation; empty still falls back to the platform
default. Both read sites (engine init + requirements check) route through
the shared resolver.
2. Voice turn-timeout leak. Each listen cycle reassigned turnTimeoutRef
without clearing the prior 60s timer, so a stale timer from an earlier
cycle could fire handleTurn() mid-way through a later listen — after enough
idle re-listens this wedged the loop into a non-re-arming state (the
'voice chat deactivates after ~a minute' report). Clear before re-arm.
Tests: 64 wake tests (added onnx-coercion / intel-kept / tflite-kept /
empty-default cases; updated the stale 'explicit onnx kept on ARM64' test
that encoded the old broken behavior), 39 desktop voice/wake vitest, tsc +
eslint clean.
The sidebar's messaging section renders one PlatformAvatar (labelIcon) and
StatusDot per platform group. The sidebar re-renders on every streaming tick
($sessions/$workingSessionIds/$messagingSessions churn), and both leaves were
unmemoized — so every platform's avatar + dot re-rendered on each delta even
though their props (platformId/tone/className) never changed.
- PlatformAvatar: memo(forwardRef(...)) — pure fn of platformId/name/class/style
- StatusDot: memo() — pure fn of tone/class
Measured (Messaging open, settled, 2 sessions streaming, 3s window):
before: 96 wasted (PlatformAvatar 32, StatusDot 32, + brand icons)
after: 0 wasted
Both are shared primitives; the memo also helps every other consumer
(session rows, gateway menu, session tiles) that renders them under a hot
parent.
- command-center/command-palette/skills: import ordering + drop unused
HermesGateway type
- settings: wrap openSubView/openProviderView/openKeysView in useCallback so
the navGroups memo deps are honest and stable
- command-center: add setSection to navGroups memo deps
The sidebar stays mounted beneath every overlay/page, and it subscribes to
$sessions + $workingSessionIds — both tick on every streaming token. An
unmemoized SidebarSessionRow re-rendered the whole list (Codicon, labels,
status dots) on each delta, and that churn bled into every overlay opened on
top: Cron, Profiles, Agents, Starmap, Webhooks, Command Center, Settings.
- SidebarSessionRow: memo() with a custom comparator that ignores the pure
id-forwarding callbacks (fresh closures by design) and compares only the
data that changes what the row paints. Rows bail out while siblings stream.
- SessionStatusDot: the 5 $...SessionIds arrays now read via useStoreSelector
returning this session's boolean, so a dot repaints only when ITS OWN
membership flips, not on every array tick.
- Artifacts: stable cellCtx (useMemo) + memoized Primary/Location/Session
cells so a link-title fetch on one row stops re-rendering the whole table.
Measured before/after (2s idle, sessions streaming), sidebar-fed overlays:
Cron 407->~30 wasted, Profiles 732->~80, Agents 132->9, Starmap 188->56,
Webhooks 154->22.
Assert isMessagingSource('photon') is true and that Photon/iMessage search aliases resolve, so a silent removal of the photon entry from MESSAGING_SESSION_SOURCE_IDS fails CI instead of regressing the sidebar section added in #47395.
Register Photon as a messaging platform in the Desktop sidebar:
- Add Photon brand icon (three-bar mark) to PlatformAvatar catalog
- Add photon to SOURCE_LABELS, SOURCE_ALIASES, and MESSAGING_SESSION_SOURCE_IDS
Closes#46761
