'Clicked voice chat but it never starts listening' — a mic-device
contention race. Starting a voice conversation fires wake.pause (to free
the mic from the wake-word listener) and opens the conversation's own mic
in two separate effects, both keyed on voiceConversationActive with no
ordering between them. wake.pause was fire-and-forget, so getUserMedia
often raced the wake listener's stream teardown (which joins a reader
thread + closes the device in a finally). On Windows the capture device is
effectively single-owner, so opening it while wake still held it failed and
the conversation never started listening.
Wake-word-initiated starts don't hit this: the backend's _on_detect calls
pause_listening synchronously before emitting wake.detected, so the mic is
already free by the time the frontend opens it. Only the button/hotkey path
raced — matching the report.
Fix: pauseWakeForVoice now returns an awaitable barrier for the in-flight
wake.pause round-trip, and useVoiceConversation awaits a new beforeMicOpen
hook (wired to that barrier) right before handle.start(), re-checking
enabled/muted/busy/idle after the wait. No behavior change when the wake
word isn't running (barrier is null → no wait).
tsc + eslint clean, 39 desktop voice/wake tests green.
Diagnosed from a Windows desktop's logs: the backend TTS was generating
and saving the reply mp3 every turn (provider: openai), so synthesis was
fine — the DESKTOP wasn't playing the first reply. Root cause is Chromium's
autoplay policy: audio (HTMLAudioElement.play() and AudioContext) is
suspended until the frame sees a user gesture. A voice conversation started
by the 'Hey Hermes' wake word has no preceding click, so the first reply's
play() was rejected with NotAllowedError (silently swallowed) and only
turn 2+ spoke. Manual voice-start worked because the button click WAS the
gesture — exactly the 'first message in a new voice session is silent,
manual start is fine' report.
Fix (defense in depth):
- chatWindowWebPreferences sets autoplayPolicy: 'no-user-gesture-required'
on every chat window (primary + secondary), so a deliberately-launched
native app never gates audio on a gesture. Primary fix.
- voice-playback.ts resumes a suspended AudioContext on stream 'start' and
retries HTMLAudioElement.play() once via an unlock context after a
NotAllowedError — covers the dashboard-embedded surface that doesn't get
the Electron policy.
Tests: 16 electron session-window tests (incl. new autoplay assertion),
tsc + eslint clean.
The prior commit made save_config_value resolve its target via
get_hermes_home() (live env) instead of the import-time cli._hermes_home
constant. Two _persist_model_switch tests patched only cli._hermes_home
and wrote/read tmp_path/config.yaml, so the write now landed in the real
HERMES_HOME and the readback saw stale values. Set HERMES_HOME=tmp_path
(kept the _hermes_home patch for belt-and-suspenders). No production
change.
The wake-word ear reverted to disabled after every restart even when
closed enabled. Root cause is general, not wake-specific: save_config_value
followed load_cli_config's precedence, which falls back to the repo's
checked-in cli-config.yaml when HERMES_HOME/config.yaml doesn't exist yet.
On such installs (managed/desktop first launch), the toggle's persist wrote
wake_word.enabled=true into cli-config.yaml and returned success — but
every config reader (load_config -> get_hermes_home()/config.yaml,
including load_wake_word_config) reads only HERMES_HOME/config.yaml, so the
setting was invisible on the next launch. Same silent loss hit any runtime
persist (model switch, /reasoning, /fast, skin) on a config-less install.
save_config_value now always targets get_hermes_home()/config.yaml,
creating it if absent, and never writes the shipped repo template. Also
resolves HERMES_HOME live instead of the import-time _hermes_home constant
(profile-safe).
E2E verified: persist -> fresh module reload -> load_wake_word_config sees
enabled=true and wake_surface_enabled('gui') is True, for both a fresh
(no config.yaml) and an existing-config install.
- cli.py save_config_value: target user config, create if absent
- tests: 2 regression tests (creates user config when absent; never writes
repo cli-config.yaml); fixture now sets HERMES_HOME. 11 file + 206
adjacent config/model-switch tests green
Saying 'stop' in a voice chat did nothing — the transcript was just
submitted to the agent as a normal turn, so the conversation never ended.
The only way out was the mouse/hotkey. That's not how a hands-free voice
assistant should work.
useVoiceConversation now checks each finished utterance against a spoken
stop-command matcher BEFORE submitting: 'stop', 'stop listening', 'never
mind', 'goodbye', 'cancel', 'that's all', etc., optionally addressed
('hey hermes, stop'). A match ends the conversation (flips enabled=false,
which drives the existing end() teardown — mic close, playback stop, wake
re-arm) instead of sending a turn.
Deliberately conservative: only a WHOLE-utterance stop phrase matches, so
substantive requests that merely contain 'stop' ('stop the docker
container', 'how do I stop a process') still go through.
- apps/desktop/src/lib/voice-stop-word.ts: isVoiceStopCommand() matcher
- use-voice-conversation.ts: onStopWord option + intercept before submit
- use-composer-voice.ts: wire onStopWord -> end the conversation
- 6 matcher tests (bare/multi-word/addressed stop; substantive requests
with 'stop' pass through; bare address words don't match)
- docs: note the spoken-stop behavior
'hey hor' triggered the wake word: the default sensitivity was 0.5, which
for openWakeWord IS the raw per-frame score threshold — openWakeWord's own
permissive baseline that near-misses clear. Raised the default to 0.6 so
phonetic near-misses fall short while real 'hey hermes' (typically 0.9+)
still fires easily.
Also fixed a real cross-engine inconsistency found while checking: the
sensitivity knob is documented 'higher = stricter' and behaves that way
for openWakeWord (threshold = sensitivity) and sherpa (0.05 + 0.4*s), but
Porcupine's own 'sensitivities' param runs the opposite way (higher = MORE
false alarms, per Picovoice). Turning sensitivity up made Porcupine looser
— backwards. Now inverted (1 - sensitivity) so 'higher = stricter' holds
for every engine.
- tools/wake_word.py: default 0.6; _sensitivity fallback uses _DEFAULTS;
Porcupine sensitivity inverted with rationale
- hermes_cli/config.py + docs: default + consistent-direction note
- tests: Porcupine inversion, default>=0.6 regression, fallback-to-default
A short, bright, rising two-note ding (G5 -> C6) plays the moment 'Hey
Hermes' is detected, before voice capture starts, so it's obvious the
wake registered. Deliberately distinct from the turn-end completion cue
(that one settles; this one rises = 'listening'). Reuses the same
lightweight WebAudio synthesis as completion-sound.ts — no asset to ship
— and is gated by the shared sound-mute toggle ($hapticsMuted), so
muting turn-end sounds silences it too.
- apps/desktop/src/lib/wake-sound.ts: playWakeSound()
- wiring.tsx: fire it at the top of the wake.detected handler
- 3 tests (plays two-note chime, silent when muted, never throws with
no WebAudio)
The rebase's lock was produced by uv 0.9.28 (runtime venv); CI runs uv
0.11.33, whose resolver rejected it (uv sync --locked failed). Relocked
with 0.11.33 — only wake-extra deps and their transitives added, no
unrelated version churn.
openWakeWord scores one ~80ms frame at a time and the detector fired the
instant a SINGLE frame crossed threshold — so a stray phoneme in background
conversation could trigger the wake word unintentionally (reported in
testing). A real utterance of the phrase holds a high score across several
consecutive frames; an ambient blip spikes just one.
_OpenWakeWordEngine now requires N consecutive over-threshold frames
(wake_word.confirmation_frames, default 3) before firing. The streak resets
on any sub-threshold frame and on engine reset() (pause/resume), so a
pre-pause frame can't count toward a post-resume fire. confirmation_frames=1
restores the old single-frame behavior; clamped 1..10.
Only openWakeWord is affected — sherpa (streaming transducer) and porcupine
decode the whole phrase internally and already reject single-frame spikes.
- tools/wake_word.py: _confirmation_frames() accessor, streak logic in
process()/reset(), config default
- hermes_cli/config.py: wake_word.confirmation_frames documented default
- tests: 5 new (spike rejected, sustained fires once, =1 legacy behavior,
reset clears streak, config clamp) — 58 wake tests green
- docs: 'Reducing false triggers on ambient speech' section
Follow-up to benbarclay's macOS ARM64 tflite fix:
- test_openwakeword_bundled_model_matches_framework stubs
ensure_tflite_runtime()=True so it exercises artifact selection, not
runtime availability — off-Darwin the bridge legitimately returns
False and the engine falls back to onnx (that path is covered by its
own tests). Was failing on Linux CI otherwise.
- check_wake_word_requirements now ANDs both this branch's STT/TTS gate
and Ben's tflite_ok into 'available' (cherry-pick conflict resolution).
openWakeWord's ONNX backend returns near-zero scores on Apple Silicon
(dscripka/openWakeWord#336), so "Hey Hermes" never crossed the 0.5
threshold: the listener armed, the microphone worked, and nothing fired.
Bisecting the pipeline puts the fault in exactly one stage — feeding the
same audio through both backends, the melspectrogram front-end is
bit-identical (maxdiff 0.00000) and the wake classifier agrees on
identical features, while the shared embedding model diverges by 45.44.
Cross-feeding confirms it: tflite features scored through the *onnx*
classifier give 0.9948 vs 0.000009 for onnx features. A telling
secondary symptom is that scores fall as input gets louder (0.5x ->
0.00031, 8x -> 0.000066), which is garbage inference rather than a weak
detection.
Selecting tflite in config alone does not fix it. openWakeWord hardcodes
`import tflite_runtime.interpreter` but declares tflite-runtime for
`platform_system == "Linux"` only; on macOS the equivalent wheel is
ai-edge-litert, so that import always fails and model.py silently
downgrades back to onnx. The result is a detector that reports itself
listening and can never fire.
- default the backend per platform (tflite on macOS ARM64, onnx
elsewhere) instead of hardcoding onnx, and pick the matching bundled
model artifact
- bridge tflite_runtime -> ai_edge_litert through sys.modules, in-process,
with no writes to site-packages
- refuse the silent onnx downgrade on macOS ARM64 and report the missing
runtime through check_wake_word_requirements() so the GUI surfaces an
actionable hint rather than arming a dead ear
- lazy-install ai-edge-litert via its own feature key, because lazy-dep
specs cannot carry PEP 508 markers (_spec_is_safe rejects ";")
An explicit `inference_framework` in config still wins, so anyone pinning
a backend keeps it.
Verified on macOS 26.5.2 / M-series: "hey hermes" scores 0.0005 on onnx
and 0.9423 on tflite from the same clip, with cross-phrase controls at
0.0003. Live over-the-air through the real microphone fires 4/4
utterances (peak 0.9532).
Teknium: the ear must always be visible so a user can click to enable
passive listening. If it can't start (missing STT/TTS, deps still
installing, no mic permission), the click surfaces the reason in the
tooltip and the toggle stays off — but the control never disappears.
Removes the 'if (!wake.available && !wake.enabled) return null' hide
branch that made the button vanish on machines where a requirements
probe returned false (the Windows report). The only non-idle state is
paused-for-voice (disabled, in the voice-chat pill), since an active
voice conversation genuinely holds the mic.
Tests updated: 'stays visible when unavailable and not enabled' and
'surfaces the refusal reason in the tooltip, still visible' replace the
old hide assertion. 30 vitest green, tsc + eslint clean.
test_install_ps1_is_pure_ascii guards against PowerShell 5.1 ANSI
codepage misdecoding (issues #66994/#67000); the Install-DesktopVoiceDeps
comment had an em-dash.
Two fixes from live testing (Teknium):
1. Desktop installs now ship the wake/voice stacks up front.
install.sh + install.ps1 desktop stages run 'uv pip install
-e .[wake,voice]' (best-effort, lazy-install remains the fallback)
before building the app, so the first ear-click arms instantly
instead of sitting through a multi-minute onnxruntime download.
CLI-only installs keep the lazy path — [all] curation unchanged.
2. The vanished ear: the STT/TTS gate made wake.status call
check_tts_requirements(), whose edge path runs _import_edge_tts →
lazy_deps.ensure — a synchronous PIP INSTALL inside a status poll.
On a venv without edge-tts that blew the desktop's 30s RPC timeout,
armWakeWord caught the error, the atom never learned enabled=true,
and the ear unmounted. _tts_ready is now a pure probe: deps missing
+ lazy installs allowed counts as ready (installs at first speak)
WITHOUT touching pip; check_tts_requirements only runs once deps
are present. Regression test asserts the probe never calls it while
deps are missing.
The ear vanished whenever a voice conversation ran (the ConversationPill
replaces the whole controls row) and whenever a transient start refusal
marked the feature unavailable — so a persistent, config-backed setting
silently disappeared mid-session. The wake word is passive by design:
it should be visibly listening no matter what the GUI is doing, with
exactly one pause state — an active voice chat holding the mic.
- ConversationPill now renders the ear in paused form (disabled, EarOff,
'paused during voice chat' tooltip) so voice chat shows the listener
yielding the mic instead of the toggle vanishing.
- WakeWordButton hides only when the feature can't run AND isn't enabled
in config; $wakeWord gains 'enabled' (config truth from wake.status /
start/stop responses) so transient 'unavailable' refusals no longer
unmount the button.
- Busy agent turns never touched the listener (it keeps listening
through agent loops; wake.detected already opens a fresh session),
and now they can't hide the toggle either.
- New i18n key wakeWordPausedVoice across en/ja/zh/zh-hant.
Tests: ear mounted during busy turn, mounted through refusal when
config-enabled, hidden when unavailable+disabled, paused ear disabled
inside the pill. 29 vitest green across controls + wake-word store.
The wake loop is wake → record → STT → agent → TTS. Arming without
either end configured gives a mic that hears you and then does nothing
perceivable — a useless experience. check_wake_word_requirements now
probes both (same probes /voice uses: stt.enabled + provider != none;
check_tts_requirements) and refuses with a pointer to `hermes tools`
naming exactly which half is missing. The desktop ear hides (available:
false already hides the button), /wake on prints the hint on CLI/TUI.
wake.start also validates requirements BEFORE persisting
wake_word.enabled, so a refused gesture can't leave config claiming on
while nothing can ever arm.
Tests: per-half and both-missing hint assertions; existing requirements
tests pinned via _voice_loop_ready so they don't depend on the test
venv's installed voice stack. E2E: stt.enabled=false in a real config
-> unavailable with the speech-to-text hint.
Internal testing (macOS): clicking the ear froze the button for ~30s,
then it went blue but 'hey hermes' never fired even though STT worked.
Two distinct bugs:
1. Frozen-then-timeout button: first-use wake.start lazy-installs the
detection engine (onnxruntime is a large wheel), which blows the
desktop's default 30s WS request timeout — the RPC 'failed' client-
side while the backend kept installing and armed later on its own.
wake.start now gets a 180s budget and the pending state says
'arming — first use may take a minute while the engine installs'
instead of a silent disabled button.
2. Armed but deaf: macOS grants mic permission per PROCESS. The
renderer having mic access (STT working) does not grant the Python
backend anything — CoreAudio hands an unentitled process a 'working'
stream that delivers zeros forever, so the listener looks healthy
and can never hear the phrase. The detector now tracks frame peaks:
10s of consecutive near-zero frames sets audio_silent, logged with
the exact macOS Settings path, cleared automatically when audio
appears. Surfaced everywhere: wake.status (audio_silent + hint),
desktop ear tooltip (kept visible while listening), /wake status on
TUI (⚠ line) and classic CLI, plus a docs troubleshooting section.
Tests: detector silent-flag set/recover cycle (fake silent/loud
streams), desktop tooltip keeps the dead-mic hint while listening.
44 wake Python tests, 22 desktop + 14 TUI vitest green.
Ending a voice conversation left the wake word silently off even with
wake_word.enabled: true — the desktop fired one wake.resume and hoped;
if the mic was still held by the just-released WebRTC capture (or the
resume raced teardown), the listener stayed dead until the user
re-toggled it. The wake word is a persistent setting: on is on until
the user explicitly turns it off.
- Desktop: resumeWakeAfterVoice() replaces the fire-and-forget resume —
resume, then verify against wake.status (config 'enabled' is the
authority) and re-arm via wake.start, with spaced retries to ride out
mic-release latency. Passive path: never passes persist, never writes
config; respects an explicit off and another surface's mic lease.
- Backend: wake.status now reports 'enabled' (config truth) so clients
reconcile against the setting, not runtime listener state.
- Backend: _wake_resume_if_owner self-heals — a resume that throws
(mic still busy) retries in a background thread for up to 15s. A
False return (lease gone/moved) is final, never retried, so the
retry can't steal another surface's mic. Covers the TUI/gateway
voice.record path which had no recovery at all (CLI has its idle
watchdog; the gateway had nothing).
- 6 new vitest cases: re-arm on enabled+down, no persist on the passive
path, resume-alone success, disabled stays off, owned lease yields,
older-backend no-op.
The wake-pause effect assigned wakePausedRef.current inside useEffect,
tripping eslint's no-restricted-syntax guard against atom→ref mirroring.
The ref is actually a request token (did WE issue wake.pause?), not a
reactive mirror — moving the assignment into a pauseWakeForVoice
callback keeps the semantics and passes the rule without a disable.
Clicking the desktop ear button or running /wake on|off now writes
wake_word.enabled to config.yaml (live, saved for future sessions), so the
feature no longer requires hand-editing config before the UI toggle works.
- wake.start accepts persist:true (explicit gesture): flips
wake_word.enabled on in config before arming; response reports
enabled_persisted. Passive auto-arm paths (desktop gateway-ready,
TUI reconnect) never pass it, so a mic can't become persistently
enabled without a deliberate user action.
- wake.stop accepts persist:true: writes wake_word.enabled: false so
auto-arm stays off next session; reports disabled_persisted.
- Split the refusal reason: 'disabled' (feature off in config — a
persisted gesture turns it on) vs 'disabled_for_surface' (explicit
wake_word.surface scoping, which persist does NOT override).
- Classic CLI /wake on|off and bare-toggle persist the flag too
(skips the write when config already matches).
- Desktop tooltip now maps refusal codes to friendly text (mirrors
the TUI's START_REASON_TEXT) instead of showing raw codes like
disabled_for_surface.
- Docs: quick-start notes the toggle persists; ear-button mention.
check_wake_word_requirements() gated 'available' on the audio probe,
but the probe imports sounddevice + numpy — two of the packages the
lazy installer would install. On a fresh machine deps_ok was False, so
audio_ok was always False and /wake on printed the manual pip hint and
bailed before the engine constructors' lazy_deps.ensure() could run.
Now the audio probe only runs once deps are installed; with deps
missing and lazy installs allowed (the default), /wake on proceeds and
ensure() installs the pinned engine deps in-process — no restart. The
manual pip hint remains for security.allow_lazy_installs=false, and a
mic hint still blocks when deps are present but no audio device works.
The CLI announces the one-time engine install so the pause is explained.
numpy is a lazy voice-extra dep absent from CI's hermetic env; the two
engine process() tests imported it for real. Stub asarray/float32 in
the fixture (verified against a blocked-numpy import, matching CI).
96-utterance TTS matrix (6 enrolled profile phrases x 4 voices/accents +
4 negative phrases x 4 voices) through the real engine: the old mapping
(default threshold 0.35) missed 3/24 positives; remapping so sensitivity
0.5 lands on sherpa's recommended 0.25 recovers 2 of 3 while keeping
0/16 false fires. Detection 23/24, routing accuracy 23/23.
One sherpa listener now enrolls every wake-enabled profile's phrase
(defaulting to "hey <profile name>") and reports WHICH phrase fired.
wake.detected gains a profile field; the desktop live-switches to the
matching profile (same path as the profile rail), opens a fresh session
there, and starts hands-free voice. The single-profile CLI/TUI print the
hermes -p switch command for foreign-profile phrases instead of
answering as the wrong profile. Opt out per listener with
wake_word.profile_routing: false.
TUI-local handler over the wake.start/stop/status RPCs with friendly
transcript one-liners (phrase, provider, foreign-owner note, refusal
reasons, unavailability hints). /wake off sets a session-scoped opt-out
the gateway.ready auto-arm respects, so the listener stays off across
reconnects until /wake on. cli_only already means CLI+TUI (verified:
messaging menus exclude it); zero Python changes needed.
Ear icon next to the voice controls: highlighted while listening, muted
when off, hidden when the backend reports the wake word unavailable.
Backed by a feature-owned $wakeWord nanostore synced by both the button
(wake.start/stop) and the gateway-ready auto-arm (status-then-arm), so
the UI always reflects the real listener state; start refusals surface
their reason as a tooltip notice. i18n en/zh/zh-hant/ja.
New "sherpa" wake_word provider: the configured phrase is BPE-tokenized
at runtime against a small streaming zipformer KWS model (~13 MB English,
one-time download cached under HERMES_HOME), so ANY typed phrase works
with zero training — including per-profile phrases like "hey coder".
wake.sherpa lazy-dep group + [wake] extra grow sherpa-onnx/sentencepiece;
requirements probe routes per provider; sensitivity maps onto sherpa
keywords_threshold. E2E-verified on real audio (target phrase fires,
foreign phrase stays silent, reset drops buffered state).
From #53378: ships hey_hermes.onnx/.tflite (openWakeWord pipeline,
Apache-2.0) under tools/wakewords/, resolves the default (and hey_hermes
aliases) to the bundled file, ensures openWakeWord base feature models
are fetched for custom paths too, and updates config defaults + docs
from hey_jarvis to hey hermes.
Regenerate uv.lock for the [wake] extra (openwakeword, pvporcupine,
onnxruntime) so uv lock --check passes. Replace angle-bracket URLs in
wake-word.md with markdown links — MDX treats <https://...> as JSX.
start_new_session was respected only by the CLI; the TUI and desktop GUI
always opened a fresh session on wake, ignoring the config. The gateway
now carries the flag in the wake.detected payload and both clients honor
it (open a fresh session vs. continue the current one), matching the CLI.
Two bugs surfaced by the desktop wake conversation:
1. Runaway loop: wake -> voice -> resume -> wake fired again within
~200ms. openWakeWord keeps its rolling feature buffer across
pause/resume, so on resume it immediately re-scored the "hey jarvis"
captured before the pause and re-fired, reopening a session and
restarting voice in a tight cycle. Reset the engine buffer on every
detector (re)start so resume begins from clean audio.
2. Empty-transcript toast: a silent re-listen returns
success:false / "… STT returned empty transcript", which the desktop
transcribe endpoint turned into a 400 -> thrown error -> "Voice
transcription failed" notification on every silent gap. Treat an empty
transcript as no-speech: return {ok, transcript: ""} so the voice loop
quietly re-listens. Real failures still 4xx/5xx.
Wake opened a fresh session but voice didn't start: the start intent was
a fire-once window CustomEvent, and the fresh-session remount tore down /
recreated the composer's subscription, so the deferred dispatch landed in
the gap and was lost.
Replace it with a latched nanostore ($voiceConversationStartRequest +
takeVoiceConversationStart): the controller sets it on wake.detected, and
the composer claims it once on (re)mount when the gateway is open, waiting
out any transient `disabled`. Drop the now-unused composer voice-start
window event.
Ending a voice conversation manually left the wake detector paused for
good, so the wake word couldn't be used again. The composer paused the
detector on voice start but only resumed on the voiceConversationActive
-> false render; if ending voice tore the composer down first, that
render never landed and the resume was skipped.
Resume on unmount as well (latched on wakePausedRef so it fires exactly
once), and stop early-returning when the $gateway atom is momentarily
null. Add wake.pause/resume INFO logs for visibility.
write_json routes via the request-scoped transport ContextVar, but the
wake detector's callback runs on a background thread where that var is
unset — so wake.detected fell back to _stdio_transport and was dumped to
the backend's stdout (visible as raw [hermes] {...} frames in desktop
logs) instead of crossing the desktop/dashboard websocket. The TUI was
unaffected because it IS stdio.
Capture the arming request's transport at wake.start and bind it around
the emit in _wake_on_detect so the background thread routes to the right
peer. Re-armed on each wake.start, so reconnects pick up the new socket.
The GUI armed the detector (wake.start) and the gateway fired
wake.detected, but the desktop never reacted: detection was wired through
a side-registered gatewayRef.current.on('wake.detected', …) listener that
was instance/timing-fragile (and silently dead across reconnects/HMR),
even though the raw events were arriving on the socket.
Route wake.detected through handleGatewayEventWithWake — the same onEvent
pipeline every gateway socket already feeds via useGatewayBoot — and open
a fresh session + start back-and-forth voice there. Drop the separate
.on() listener; the open-effect now only arms wake.start.
The detector logged listen/detect/close at debug, invisible at the
default level. Promote listen-start, phrase-detected, stream-closed, and
the wake.start outcome (disabled / unavailable / listening) to INFO, and
log wake.detected emission, so a non-triggering setup is diagnosable from
gateway/gui.log without flipping global log levels.
On wake, the desktop GUI now opens a fresh session AND starts the
browser voice conversation (continuous, with TTS), matching the CLI/TUI
hands-free flow instead of just opening a session.
- Add an explicit requestVoiceStart() intent to the composer bus
(idempotent start; toggle could stop an active loop).
- Composer owns mic hand-off: pause the server-side wake detector while
the browser voice loop is live, resume after (server no-ops when the
wake word isn't armed) — via the $gateway store accessor.
- Controller fires startFreshSessionDraft() + requestVoiceStart() on
wake.detected.
Makes the wake word a tri-surface feature with one configurable owner.
- wake_word.surface ("auto" | "cli" | "tui" | "gui") + shared
wake_surface_enabled() gate consulted by every surface, so exactly one
place owns the listener and the new session it opens.
- tui_gateway: wake.start/stop/pause/resume/status RPCs + a wake.detected
event, sharing one server-side detector for both TUI and desktop. The
detector yields the mic to voice.record (pause on capture start, resume
on terminal) and to the desktop's browser mic (wake.pause/resume).
- TUI (Ink): arm wake.start on gateway.ready; on wake.detected open a
fresh session and start voice capture.
- Desktop (Electron): arm wake.start on connect; on wake.detected open a
fresh session.
- CLI now gates on wake_surface_enabled("cli"); /wake status shows surface.
- Tests for the surface gate; docs cover the surface knob + cross-surface.
Adds an opt-in, on-device hotword listener for the CLI. With
wake_word.enabled (or /wake on), Hermes listens in the background for a
wake phrase; on detection it starts a fresh session, captures one
utterance through the existing voice pipeline, and answers — the
"Hey Siri" pattern.
- tools/wake_word.py: provider-pluggable detector (openWakeWord, free
local default; Porcupine, premium) over the shared 16 kHz sounddevice
capture path. Background daemon thread with pause/resume so it yields
the mic during a voice turn.
- CLI wiring: startup listener (off-thread), on-wake flow, an idle
watchdog that resumes the detector after each turn, cleanup hook, and
a /wake [on|off|status] command.
- config.yaml wake_word section; PORCUPINE_ACCESS_KEY as an optional
secret. Engines lazy-install via the [wake] extra.
- Hands a transcript to the input queue exactly like voice mode, so no
system-prompt/cache mutation. No new core model tool.
- Tests (mocked, no live audio/network) + feature docs.
CI shard 4 caught 4 failures the local baseline diff missed (local env
pollution made them look pre-existing): _make_compressor() constructs
under a get_model_context_length mock but the lazy deferral (#32221)
pushed resolution past the with block, so the 96K window / 75% floor the
tests rely on never materialized. Resolve inside the mock.
Review-pass findings on the lazy-init deferral:
- No-op guard: the codex app-server usage callback assigns
compressor.context_length on EVERY response (same window each time).
The setter unconditionally invalidated the derived budgets, wiping
runtime corrections applied directly to threshold_tokens /
tail_token_budget (aux-context threshold sync) — those persisted on
main's eager init. Same-value assignment is now a no-op.
- Re-floor on genuinely new window: the setter invalidates budgets but
previously kept the stale threshold_percent, so a codex window switch
recomputed threshold_tokens from the new window with the old model's
floored percent. Re-apply the raise-only small-context floor from
_base_threshold_percent so percent and tokens derive from the same
window (guarded with getattr for object.__new__ test instances).
- Init log extracted to _emit_init_summary_once() and also fired from
the setter path, so a consumer assigning context_length before any
read no longer strands the startup line forever.
- threshold_tokens getter resolves the window into a local before
reading threshold_percent — correctness no longer depends on
left-to-right argument evaluation order.
- reasoning_timeouts: fix inaccurate 'tuples are immutable' comment
(the container is a list; safety comes from build-once-at-import),
document why the slug stays in the tuple.
Adds TestContextLengthSetterCoherence (3 tests): same-value assignment
preserves overrides; new-window assignment re-floors both directions.
With the selective prompt-cache copy (#57046), un-marked messages on the
decorated api_messages list share their nested content parts with the
persistent conversation history — the per-message copy in
conversation_loop is shallow and decoration now deep-copies only the
marked messages. try_shrink_image_parts_in_messages previously wrote the
re-encoded image INTO the aliased part/source dicts, so an
image-too-large retry on an Anthropic route would silently replace the
original image bytes in agent.messages (and persist the degraded copy).
Replace the in-place writes with copy-on-write: rebuild the content list
with fresh part/source/image_url dicts and reassign msg['content'] — a
top-level write on the per-call copy that never reaches history.
Adds two regression tests simulating the aliasing; both fail against the
old in-place implementation (mutation-verified).
Baseline diff vs clean upstream/main (182 pre-existing environmental
failures on both sides) showed exactly 6 PR-caused failures, all the same
mechanism: tests construct ContextCompressor under a
get_model_context_length mock and read threshold_percent/threshold_tokens
after the with block, or build via object.__new__ and assign
context_length AFTER the derived budgets (the setter now resets the
lazily-cached budgets).
- test_compression_small_ctx_threshold_floor: resolve inside _make()
- test_cjk_token_estimation: resolve inside mock
- test_per_model_compression_threshold: resolve inside mock (2 tests)
- test_pre_compress_memory_context: assign context_length before
threshold/tail/summary budgets; add summary_target_ratio
