test_install_ps1_is_pure_ascii guards against PowerShell 5.1 ANSI
codepage misdecoding (issues #66994/#67000); the Install-DesktopVoiceDeps
comment had an em-dash.
Two fixes from live testing (Teknium):
1. Desktop installs now ship the wake/voice stacks up front.
install.sh + install.ps1 desktop stages run 'uv pip install
-e .[wake,voice]' (best-effort, lazy-install remains the fallback)
before building the app, so the first ear-click arms instantly
instead of sitting through a multi-minute onnxruntime download.
CLI-only installs keep the lazy path — [all] curation unchanged.
2. The vanished ear: the STT/TTS gate made wake.status call
check_tts_requirements(), whose edge path runs _import_edge_tts →
lazy_deps.ensure — a synchronous PIP INSTALL inside a status poll.
On a venv without edge-tts that blew the desktop's 30s RPC timeout,
armWakeWord caught the error, the atom never learned enabled=true,
and the ear unmounted. _tts_ready is now a pure probe: deps missing
+ lazy installs allowed counts as ready (installs at first speak)
WITHOUT touching pip; check_tts_requirements only runs once deps
are present. Regression test asserts the probe never calls it while
deps are missing.
The ear vanished whenever a voice conversation ran (the ConversationPill
replaces the whole controls row) and whenever a transient start refusal
marked the feature unavailable — so a persistent, config-backed setting
silently disappeared mid-session. The wake word is passive by design:
it should be visibly listening no matter what the GUI is doing, with
exactly one pause state — an active voice chat holding the mic.
- ConversationPill now renders the ear in paused form (disabled, EarOff,
'paused during voice chat' tooltip) so voice chat shows the listener
yielding the mic instead of the toggle vanishing.
- WakeWordButton hides only when the feature can't run AND isn't enabled
in config; $wakeWord gains 'enabled' (config truth from wake.status /
start/stop responses) so transient 'unavailable' refusals no longer
unmount the button.
- Busy agent turns never touched the listener (it keeps listening
through agent loops; wake.detected already opens a fresh session),
and now they can't hide the toggle either.
- New i18n key wakeWordPausedVoice across en/ja/zh/zh-hant.
Tests: ear mounted during busy turn, mounted through refusal when
config-enabled, hidden when unavailable+disabled, paused ear disabled
inside the pill. 29 vitest green across controls + wake-word store.
The wake loop is wake → record → STT → agent → TTS. Arming without
either end configured gives a mic that hears you and then does nothing
perceivable — a useless experience. check_wake_word_requirements now
probes both (same probes /voice uses: stt.enabled + provider != none;
check_tts_requirements) and refuses with a pointer to `hermes tools`
naming exactly which half is missing. The desktop ear hides (available:
false already hides the button), /wake on prints the hint on CLI/TUI.
wake.start also validates requirements BEFORE persisting
wake_word.enabled, so a refused gesture can't leave config claiming on
while nothing can ever arm.
Tests: per-half and both-missing hint assertions; existing requirements
tests pinned via _voice_loop_ready so they don't depend on the test
venv's installed voice stack. E2E: stt.enabled=false in a real config
-> unavailable with the speech-to-text hint.
Internal testing (macOS): clicking the ear froze the button for ~30s,
then it went blue but 'hey hermes' never fired even though STT worked.
Two distinct bugs:
1. Frozen-then-timeout button: first-use wake.start lazy-installs the
detection engine (onnxruntime is a large wheel), which blows the
desktop's default 30s WS request timeout — the RPC 'failed' client-
side while the backend kept installing and armed later on its own.
wake.start now gets a 180s budget and the pending state says
'arming — first use may take a minute while the engine installs'
instead of a silent disabled button.
2. Armed but deaf: macOS grants mic permission per PROCESS. The
renderer having mic access (STT working) does not grant the Python
backend anything — CoreAudio hands an unentitled process a 'working'
stream that delivers zeros forever, so the listener looks healthy
and can never hear the phrase. The detector now tracks frame peaks:
10s of consecutive near-zero frames sets audio_silent, logged with
the exact macOS Settings path, cleared automatically when audio
appears. Surfaced everywhere: wake.status (audio_silent + hint),
desktop ear tooltip (kept visible while listening), /wake status on
TUI (⚠ line) and classic CLI, plus a docs troubleshooting section.
Tests: detector silent-flag set/recover cycle (fake silent/loud
streams), desktop tooltip keeps the dead-mic hint while listening.
44 wake Python tests, 22 desktop + 14 TUI vitest green.
Ending a voice conversation left the wake word silently off even with
wake_word.enabled: true — the desktop fired one wake.resume and hoped;
if the mic was still held by the just-released WebRTC capture (or the
resume raced teardown), the listener stayed dead until the user
re-toggled it. The wake word is a persistent setting: on is on until
the user explicitly turns it off.
- Desktop: resumeWakeAfterVoice() replaces the fire-and-forget resume —
resume, then verify against wake.status (config 'enabled' is the
authority) and re-arm via wake.start, with spaced retries to ride out
mic-release latency. Passive path: never passes persist, never writes
config; respects an explicit off and another surface's mic lease.
- Backend: wake.status now reports 'enabled' (config truth) so clients
reconcile against the setting, not runtime listener state.
- Backend: _wake_resume_if_owner self-heals — a resume that throws
(mic still busy) retries in a background thread for up to 15s. A
False return (lease gone/moved) is final, never retried, so the
retry can't steal another surface's mic. Covers the TUI/gateway
voice.record path which had no recovery at all (CLI has its idle
watchdog; the gateway had nothing).
- 6 new vitest cases: re-arm on enabled+down, no persist on the passive
path, resume-alone success, disabled stays off, owned lease yields,
older-backend no-op.
The wake-pause effect assigned wakePausedRef.current inside useEffect,
tripping eslint's no-restricted-syntax guard against atom→ref mirroring.
The ref is actually a request token (did WE issue wake.pause?), not a
reactive mirror — moving the assignment into a pauseWakeForVoice
callback keeps the semantics and passes the rule without a disable.
Clicking the desktop ear button or running /wake on|off now writes
wake_word.enabled to config.yaml (live, saved for future sessions), so the
feature no longer requires hand-editing config before the UI toggle works.
- wake.start accepts persist:true (explicit gesture): flips
wake_word.enabled on in config before arming; response reports
enabled_persisted. Passive auto-arm paths (desktop gateway-ready,
TUI reconnect) never pass it, so a mic can't become persistently
enabled without a deliberate user action.
- wake.stop accepts persist:true: writes wake_word.enabled: false so
auto-arm stays off next session; reports disabled_persisted.
- Split the refusal reason: 'disabled' (feature off in config — a
persisted gesture turns it on) vs 'disabled_for_surface' (explicit
wake_word.surface scoping, which persist does NOT override).
- Classic CLI /wake on|off and bare-toggle persist the flag too
(skips the write when config already matches).
- Desktop tooltip now maps refusal codes to friendly text (mirrors
the TUI's START_REASON_TEXT) instead of showing raw codes like
disabled_for_surface.
- Docs: quick-start notes the toggle persists; ear-button mention.
check_wake_word_requirements() gated 'available' on the audio probe,
but the probe imports sounddevice + numpy — two of the packages the
lazy installer would install. On a fresh machine deps_ok was False, so
audio_ok was always False and /wake on printed the manual pip hint and
bailed before the engine constructors' lazy_deps.ensure() could run.
Now the audio probe only runs once deps are installed; with deps
missing and lazy installs allowed (the default), /wake on proceeds and
ensure() installs the pinned engine deps in-process — no restart. The
manual pip hint remains for security.allow_lazy_installs=false, and a
mic hint still blocks when deps are present but no audio device works.
The CLI announces the one-time engine install so the pause is explained.
numpy is a lazy voice-extra dep absent from CI's hermetic env; the two
engine process() tests imported it for real. Stub asarray/float32 in
the fixture (verified against a blocked-numpy import, matching CI).
96-utterance TTS matrix (6 enrolled profile phrases x 4 voices/accents +
4 negative phrases x 4 voices) through the real engine: the old mapping
(default threshold 0.35) missed 3/24 positives; remapping so sensitivity
0.5 lands on sherpa's recommended 0.25 recovers 2 of 3 while keeping
0/16 false fires. Detection 23/24, routing accuracy 23/23.
One sherpa listener now enrolls every wake-enabled profile's phrase
(defaulting to "hey <profile name>") and reports WHICH phrase fired.
wake.detected gains a profile field; the desktop live-switches to the
matching profile (same path as the profile rail), opens a fresh session
there, and starts hands-free voice. The single-profile CLI/TUI print the
hermes -p switch command for foreign-profile phrases instead of
answering as the wrong profile. Opt out per listener with
wake_word.profile_routing: false.
TUI-local handler over the wake.start/stop/status RPCs with friendly
transcript one-liners (phrase, provider, foreign-owner note, refusal
reasons, unavailability hints). /wake off sets a session-scoped opt-out
the gateway.ready auto-arm respects, so the listener stays off across
reconnects until /wake on. cli_only already means CLI+TUI (verified:
messaging menus exclude it); zero Python changes needed.
Ear icon next to the voice controls: highlighted while listening, muted
when off, hidden when the backend reports the wake word unavailable.
Backed by a feature-owned $wakeWord nanostore synced by both the button
(wake.start/stop) and the gateway-ready auto-arm (status-then-arm), so
the UI always reflects the real listener state; start refusals surface
their reason as a tooltip notice. i18n en/zh/zh-hant/ja.
New "sherpa" wake_word provider: the configured phrase is BPE-tokenized
at runtime against a small streaming zipformer KWS model (~13 MB English,
one-time download cached under HERMES_HOME), so ANY typed phrase works
with zero training — including per-profile phrases like "hey coder".
wake.sherpa lazy-dep group + [wake] extra grow sherpa-onnx/sentencepiece;
requirements probe routes per provider; sensitivity maps onto sherpa
keywords_threshold. E2E-verified on real audio (target phrase fires,
foreign phrase stays silent, reset drops buffered state).
From #53378: ships hey_hermes.onnx/.tflite (openWakeWord pipeline,
Apache-2.0) under tools/wakewords/, resolves the default (and hey_hermes
aliases) to the bundled file, ensures openWakeWord base feature models
are fetched for custom paths too, and updates config defaults + docs
from hey_jarvis to hey hermes.
Regenerate uv.lock for the [wake] extra (openwakeword, pvporcupine,
onnxruntime) so uv lock --check passes. Replace angle-bracket URLs in
wake-word.md with markdown links — MDX treats <https://...> as JSX.
start_new_session was respected only by the CLI; the TUI and desktop GUI
always opened a fresh session on wake, ignoring the config. The gateway
now carries the flag in the wake.detected payload and both clients honor
it (open a fresh session vs. continue the current one), matching the CLI.
Two bugs surfaced by the desktop wake conversation:
1. Runaway loop: wake -> voice -> resume -> wake fired again within
~200ms. openWakeWord keeps its rolling feature buffer across
pause/resume, so on resume it immediately re-scored the "hey jarvis"
captured before the pause and re-fired, reopening a session and
restarting voice in a tight cycle. Reset the engine buffer on every
detector (re)start so resume begins from clean audio.
2. Empty-transcript toast: a silent re-listen returns
success:false / "… STT returned empty transcript", which the desktop
transcribe endpoint turned into a 400 -> thrown error -> "Voice
transcription failed" notification on every silent gap. Treat an empty
transcript as no-speech: return {ok, transcript: ""} so the voice loop
quietly re-listens. Real failures still 4xx/5xx.
Wake opened a fresh session but voice didn't start: the start intent was
a fire-once window CustomEvent, and the fresh-session remount tore down /
recreated the composer's subscription, so the deferred dispatch landed in
the gap and was lost.
Replace it with a latched nanostore ($voiceConversationStartRequest +
takeVoiceConversationStart): the controller sets it on wake.detected, and
the composer claims it once on (re)mount when the gateway is open, waiting
out any transient `disabled`. Drop the now-unused composer voice-start
window event.
Ending a voice conversation manually left the wake detector paused for
good, so the wake word couldn't be used again. The composer paused the
detector on voice start but only resumed on the voiceConversationActive
-> false render; if ending voice tore the composer down first, that
render never landed and the resume was skipped.
Resume on unmount as well (latched on wakePausedRef so it fires exactly
once), and stop early-returning when the $gateway atom is momentarily
null. Add wake.pause/resume INFO logs for visibility.
write_json routes via the request-scoped transport ContextVar, but the
wake detector's callback runs on a background thread where that var is
unset — so wake.detected fell back to _stdio_transport and was dumped to
the backend's stdout (visible as raw [hermes] {...} frames in desktop
logs) instead of crossing the desktop/dashboard websocket. The TUI was
unaffected because it IS stdio.
Capture the arming request's transport at wake.start and bind it around
the emit in _wake_on_detect so the background thread routes to the right
peer. Re-armed on each wake.start, so reconnects pick up the new socket.
The GUI armed the detector (wake.start) and the gateway fired
wake.detected, but the desktop never reacted: detection was wired through
a side-registered gatewayRef.current.on('wake.detected', …) listener that
was instance/timing-fragile (and silently dead across reconnects/HMR),
even though the raw events were arriving on the socket.
Route wake.detected through handleGatewayEventWithWake — the same onEvent
pipeline every gateway socket already feeds via useGatewayBoot — and open
a fresh session + start back-and-forth voice there. Drop the separate
.on() listener; the open-effect now only arms wake.start.
The detector logged listen/detect/close at debug, invisible at the
default level. Promote listen-start, phrase-detected, stream-closed, and
the wake.start outcome (disabled / unavailable / listening) to INFO, and
log wake.detected emission, so a non-triggering setup is diagnosable from
gateway/gui.log without flipping global log levels.
On wake, the desktop GUI now opens a fresh session AND starts the
browser voice conversation (continuous, with TTS), matching the CLI/TUI
hands-free flow instead of just opening a session.
- Add an explicit requestVoiceStart() intent to the composer bus
(idempotent start; toggle could stop an active loop).
- Composer owns mic hand-off: pause the server-side wake detector while
the browser voice loop is live, resume after (server no-ops when the
wake word isn't armed) — via the $gateway store accessor.
- Controller fires startFreshSessionDraft() + requestVoiceStart() on
wake.detected.
Makes the wake word a tri-surface feature with one configurable owner.
- wake_word.surface ("auto" | "cli" | "tui" | "gui") + shared
wake_surface_enabled() gate consulted by every surface, so exactly one
place owns the listener and the new session it opens.
- tui_gateway: wake.start/stop/pause/resume/status RPCs + a wake.detected
event, sharing one server-side detector for both TUI and desktop. The
detector yields the mic to voice.record (pause on capture start, resume
on terminal) and to the desktop's browser mic (wake.pause/resume).
- TUI (Ink): arm wake.start on gateway.ready; on wake.detected open a
fresh session and start voice capture.
- Desktop (Electron): arm wake.start on connect; on wake.detected open a
fresh session.
- CLI now gates on wake_surface_enabled("cli"); /wake status shows surface.
- Tests for the surface gate; docs cover the surface knob + cross-surface.
Adds an opt-in, on-device hotword listener for the CLI. With
wake_word.enabled (or /wake on), Hermes listens in the background for a
wake phrase; on detection it starts a fresh session, captures one
utterance through the existing voice pipeline, and answers — the
"Hey Siri" pattern.
- tools/wake_word.py: provider-pluggable detector (openWakeWord, free
local default; Porcupine, premium) over the shared 16 kHz sounddevice
capture path. Background daemon thread with pause/resume so it yields
the mic during a voice turn.
- CLI wiring: startup listener (off-thread), on-wake flow, an idle
watchdog that resumes the detector after each turn, cleanup hook, and
a /wake [on|off|status] command.
- config.yaml wake_word section; PORCUPINE_ACCESS_KEY as an optional
secret. Engines lazy-install via the [wake] extra.
- Hands a transcript to the input queue exactly like voice mode, so no
system-prompt/cache mutation. No new core model tool.
- Tests (mocked, no live audio/network) + feature docs.
CI shard 4 caught 4 failures the local baseline diff missed (local env
pollution made them look pre-existing): _make_compressor() constructs
under a get_model_context_length mock but the lazy deferral (#32221)
pushed resolution past the with block, so the 96K window / 75% floor the
tests rely on never materialized. Resolve inside the mock.
Review-pass findings on the lazy-init deferral:
- No-op guard: the codex app-server usage callback assigns
compressor.context_length on EVERY response (same window each time).
The setter unconditionally invalidated the derived budgets, wiping
runtime corrections applied directly to threshold_tokens /
tail_token_budget (aux-context threshold sync) — those persisted on
main's eager init. Same-value assignment is now a no-op.
- Re-floor on genuinely new window: the setter invalidates budgets but
previously kept the stale threshold_percent, so a codex window switch
recomputed threshold_tokens from the new window with the old model's
floored percent. Re-apply the raise-only small-context floor from
_base_threshold_percent so percent and tokens derive from the same
window (guarded with getattr for object.__new__ test instances).
- Init log extracted to _emit_init_summary_once() and also fired from
the setter path, so a consumer assigning context_length before any
read no longer strands the startup line forever.
- threshold_tokens getter resolves the window into a local before
reading threshold_percent — correctness no longer depends on
left-to-right argument evaluation order.
- reasoning_timeouts: fix inaccurate 'tuples are immutable' comment
(the container is a list; safety comes from build-once-at-import),
document why the slug stays in the tuple.
Adds TestContextLengthSetterCoherence (3 tests): same-value assignment
preserves overrides; new-window assignment re-floors both directions.
With the selective prompt-cache copy (#57046), un-marked messages on the
decorated api_messages list share their nested content parts with the
persistent conversation history — the per-message copy in
conversation_loop is shallow and decoration now deep-copies only the
marked messages. try_shrink_image_parts_in_messages previously wrote the
re-encoded image INTO the aliased part/source dicts, so an
image-too-large retry on an Anthropic route would silently replace the
original image bytes in agent.messages (and persist the degraded copy).
Replace the in-place writes with copy-on-write: rebuild the content list
with fresh part/source/image_url dicts and reassign msg['content'] — a
top-level write on the per-call copy that never reaches history.
Adds two regression tests simulating the aliasing; both fail against the
old in-place implementation (mutation-verified).
Baseline diff vs clean upstream/main (182 pre-existing environmental
failures on both sides) showed exactly 6 PR-caused failures, all the same
mechanism: tests construct ContextCompressor under a
get_model_context_length mock and read threshold_percent/threshold_tokens
after the with block, or build via object.__new__ and assign
context_length AFTER the derived budgets (the setter now resets the
lazily-cached budgets).
- test_compression_small_ctx_threshold_floor: resolve inside _make()
- test_cjk_token_estimation: resolve inside mock
- test_per_model_compression_threshold: resolve inside mock (2 tests)
- test_pre_compress_memory_context: assign context_length before
threshold/tail/summary budgets; add summary_target_ratio
TestThresholdTokensCap and TestLazyContextResolution landed on main after
the #38991 lazy-init base; they construct ContextCompressor under a
get_model_context_length patch and read threshold_tokens after the with
block. With deferred resolution the probe now fires lazily, so resolve
inside the mock (same pattern as the rest of the suite) and give the
lazy-resolution mock a real return_value.
Self-review found the deepcopy->shallow-copy change in
apply_anthropic_cache_control (#57046) had no test pinning the
"prompt caching is sacred / never mutate the caller's list" invariant.
The old test_returns_deep_copy only exercised the single marked message,
never an un-marked shared reference, so a regression that deep-copied too
little would pass the whole suite.
Add two tests: (1) caller list + every element left byte-identical after
the call, un-marked middle messages returned as shared references, marked
messages fresh copies, and mutating a returned marked message does not leak
upstream; (2) structural byte-equivalence vs a reference full-deepcopy
implementation across both native_anthropic modes and two TTLs.
Mutation-verified: neutering the per-message deepcopy makes test (1) fail.
The lazy-init change in #32221 deferred get_model_context_length() out of
ContextCompressor.__init__, but the "Context compressor initialized" log
(emitted only when quiet_mode=False) reads self.context_length,
self.threshold_tokens and self.tail_token_budget. Those property reads
resolve the deferred value, so the synchronous model-metadata probe still
ran inside __init__ on the interactive-CLI path — the exact blocking the
PR removed, just narrowed to the non-quiet path.
Emit the informative line once, on first context-length resolution, so
construction stays non-blocking on every path. Add a regression test
asserting no probe fires in __init__ with quiet_mode=False and that the
init log is emitted exactly once on first access.
_match_any() was re-sorting _REASONING_STALE_TIMEOUT_FLOORS (21 elements)
on every call. This function runs per API turn via error_classifier,
chat_completion_helpers, and thinking_timeout_guidance.
Also fixes thread-safety: the old _PATTERN_CACHE was a mutable dict
accessed from multiple threads without locking. Pre-compiling all
patterns at module load time eliminates both the per-call sort and
the TOCTOU race condition. The resulting list is effectively
immutable after import, safe for free-threaded Python 3.13+.
(cherry picked from commit e8b006b853)
Replaces full deepcopy with selective shallow copy in apply_anthropic_cache_control.
Only the 4 messages that receive cache_control markers are deep-copied; the rest
stay as references.
Measured on a 100-message conversation (typical long session):
- Before: ~15ms per call
- After: ~2ms per call
- 7.5x faster, scales with conversation length
Memory impact is also significant — no need to duplicate dozens of unchanged
messages on every turn.
The contract is unchanged: callers still get an independent message list
they can mutate. Only messages we modify (by injecting cache markers) are
copied. The rest are shared references to immutable history entries, which
is safe since the agent never mutates past turns.
(cherry picked from commit 17892df6cc)
The structural AST walker in test_compression_session_id_persistence.py
only recursed into control-flow children found via iter_child_nodes(stmt).
When a session_entry.session_id assignment lives inside an `else` block
whose statements are all assigns (no nested If/Try/etc to trigger
_walk_node), the assignment was invisible to the walker and the test
florped with 'No assignments found'.
Walk the stmt itself when it is a control-flow node so its
body/orelse/finalbody (and Try handlers) are always expanded, regardless
of whether iter_child_nodes yields an inner control-flow child.
Manual /compress already persists the rotated child transcript first and
only then repoints the live session_entry; a False rewrite_transcript
return keeps the entry on the original session_id so the conversation
stays reachable. Session hygiene auto-compress did the opposite: it
rebound session_id (and lease/topic) first, then called rewrite_transcript
without checking the return value. On a failed write the live entry
already pointed at an empty child SID and the turn continued — permanent
silent conversation loss. Persist first; rebind only after success.
Phase 2 review found two sibling sites with the same bug class:
- truncate_before_user_ordinal in prompt.submit counted display_kind
timeline rows as user turns, shifting the truncation target
- rollback.restore used the old pop-loop pattern that would pop a
display_kind marker instead of the last real exchange
Both now use the same predicate (role==user and not display_kind)
matching list_recent_user_messages, /undo, /retry, and CLI resume.
Added tests for both paths.
list_recent_user_messages and the in-memory /retry + session.undo walkers
treated every role=user row as a real user turn. Timeline bookkeeping
(model_switch, async_delegation_complete, auto_continue, hidden) is stored
that way, so /undo soft-deleted from a marker and /retry re-sent opaque
bookkeeping text. Exclude display_kind the same way CLI resume counting and
the prompt.submit ordinal path do.
Replaces the O(N) list_sessions_rich histogram in /api/sessions/stats
with a single GROUP BY query, reducing response time from ~575ms to
<1ms on large databases.
Original PR #48921 by @liuhao1024. Salvage fixes based on review
feedback from teknium1 and @wernerhp:
1. Preserve try/except guard — a DB error still degrades to empty
by_source instead of failing the whole stats response.
2. GROUP BY COALESCE(source, 'cli') — the original GROUP BY source
could emit duplicate 'cli' keys (NULL group + literal 'cli' group)
that the dict comprehension silently dropped.
3. Add exclude_children=True — list_sessions_rich excludes subagent
runs, delegates, and compression continuations by default; the
bare GROUP BY counted all rows, inflating source counts.
Aggregate shape (exclude_children/include_archived/limit params)
adapted from closed duplicate #61120 by @mijanx.
Closes#48914
Co-authored-by: mijanx <mijanx@users.noreply.github.com>
