Replaces the half-duplex per-playback barge monitors with ONE listener
that runs for the entire agent turn in continuous voice mode: armed at
utterance-submit, disarmed when the turn is fully done (response + TTS
finished). Fixes Teknium's live report that voice interruption never
works: (a) not while the LLM is generating, (b) not while TTS plays.
Root causes:
- HALF-DUPLEX GAP: the barge monitor only spawned when TTS playback
STARTED (cli.py streaming/whole-file paths, gateway _tts_stream_begin).
During LLM generation there was NO microphone listener at all.
- PLAYBACK DEAFNESS: the monitor calibrated its VAD noise floor WHILE the
speaker was blasting TTS (speaker bleed baked into the floor), then
multiplied it by 8x with a 1s strictly-consecutive block requirement —
normal speech could rarely reach the trigger, and the 2s grace
swallowed early interjections.
New model — tools/voice_mode.full_duplex_listen():
- Pre-playback calibration: quiet-room noise floor established at turn
start and HELD through playback (never recalibrated against bleed).
- Phase-aware trigger: generation = floor x voice.barge_in_threshold_multiplier
(new config, default 3.0, justified by synthetic-frame tests);
playback = additionally clamped to a 1500-RMS minimum so bleed alone
can't trip; 4000-RMS ceiling keeps speech always reachable.
- Windowed-majority detection (>=80% of a 300ms window) instead of the
strictly-consecutive counter that reset on intra-word energy dips.
- Grace on playback ONSET only (voice.barge_in_grace_seconds, default
down 2.0 -> 0.5) — suppresses the onset transient, not the mic.
- Debug diagnostics at every decision point (calibrated floor, per-window
RMS above 50% of trigger, trip/no-trip, grace suppressions) — always
logger.debug, mirrored to stderr under HERMES_VOICE_DEBUG=1.
Phase behavior (CLI cli.py + tui_gateway/server.py, same model):
- generation: speech interrupts the in-flight turn via the SAME seam the
typed/Ctrl+C interrupt uses (agent.interrupt()), cuts any pending TTS
pipeline so the stale reply never plays, and submits the captured
interjection (pre-roll capture, first syllable kept) as the next turn.
- playback: cuts TTS (streaming pipeline stop + fallback speak stop
events + file player) and submits the capture.
- stop phrase honored in BOTH phases: mid-generation 'stop' interrupts
the turn AND ends the voice chat (stop everything).
- one listener instance spans generation -> playback (no re-arm race);
double-arm refused (CLI _voice_fd_active / gateway _fd_listener_active).
Gateway specifics: _arm_full_duplex_listener() at _run_prompt_submit turn
start and inside _tts_stream_begin; _speak_text_with_barge registers its
(stop, done) pair in _fd_speak_pipelines so fallback speaks are cut and
tracked; _tts_stream_barge_in_monitor kept as a shim that arms the new
listener. Desktop renderer owns its own mic path (voice-barge-in.ts) and
is unaffected; if desktop backend-mic mode is used it inherits via the
gateway.
Tests: full_duplex_listen synthetic-RMS suite (speech-over-bleed trips,
bleed alone doesn't, quiet floor held through playback, grace window,
multiplier math 3x vs 8x, windowed-majority dips), CLI listener phase
tests (generation interrupt seam, playback cut, lifecycle spans phases,
double-arm, config forwarding, stop-phrase-mid-generation), gateway
generation-phase interrupt + stop-phrase tests. Generation-interrupt
test sabotage-verified.
tools/computer_use/tool.py keeps the CLI approval flow in module-globals:
_approval_callback plus the per-session unlock stores _always_allow /
_session_auto_approve. Any test that installs a callback (or drives CLI
init far enough that the real one is registered) and does not reset it
poisons every later computer-use test in the same process:
* a leaked callback that raises — dead UI infra or a stale two-argument
signature (the contract is (action, args, summary)) — becomes
verdict='deny' in _request_approval, so dispatch tests fail with an
empty backend call list;
* a leaked callback that blocks (the real CLI one waits on an answer
queue) hangs a single-process run forever.
Both are order-dependent: tests/tools/test_computer_use.py passes 220/220
in isolation but shows dispatch failures in single-process full-suite runs
(and, with a blocking leak, a permanent hang observed via py-spy inside
_request_approval -> callback -> queue.get with no timeout).
Fix: an autouse teardown-only fixture resets callback + unlock stores
after every test; tests that install their own callback keep it for their
own duration. Regression pair included: a 'forgetful' test leaves a stale
two-arg callback behind, the next test asserts dispatch still routes to
the backend — red without the fixture (1 failed), green with it (225
passed together with the whole computer-use file).
_start_lifecycle_locked reassigns a fresh threading.Event() at line
786, so patching the pre-made instance's wait() is lost and the real
30s wait races pytest-timeout. Patch threading.Event itself with a
FakeEvent whose wait() returns False immediately (#69372).
Premise check on live main: barge-in machinery EXISTS for the per-turn
STREAMING pipeline only — cli.py chat() arms _voice_barge_in_monitor and
tui_gateway _tts_stream_begin arms _tts_stream_barge_in_monitor. What was
actually broken for spoken interruptions:
1. CLI whole-file fallback (_voice_speak_response_async — used whenever
streaming TTS cannot start: sounddevice missing, requirement probe
fails): NO monitor was ever armed, so talking over the reply did
nothing. Now arms _voice_barge_in_monitor in continuous voice mode.
2. Gateway fallback speak (tts_queue None → speak_text thread) and the
voice.tts RPC (desktop-triggered speech): speak_text ran bare, and
its internal streaming dispatch created a PRIVATE stop event nothing
could reach — uninterruptible even by stop_playback(). New
_speak_text_with_barge() runs the same barge monitor beside the speak
thread; hermes_cli.voice.speak_text/_speak_text_streaming accept an
external stop_event so a barge cuts the streaming pipeline too.
Stop-phrase handling and voice.transcript submission are inherited
from the shared monitor (merged #73933 behavior preserved).
3. False barge during TTS (the reason interruption "worked" then
self-cancelled or fired randomly): salvaged PR #71083 by @beardedeagle
(previous commit, kept authorship) — rolling-window VAD floor, 8x
multiplier, 4000-RMS trigger ceiling, barge_in_grace_seconds (2s)
before the mic opens, min-floor clamp. barge_in_grace_seconds is now
documented in DEFAULT_CONFIG.
Desktop spoken barge (renderer mic via voice-barge-in.ts) already covers
both its live-stream and fallback speech paths — verified, no change.
Long thinking/tool stretches in a voice conversation are dead air — the
user cannot tell whether the agent is alive. New: quiet, repeating soft
bubble blips while the agent works and no speech audio is flowing.
- tools/voice_mode.py: numpy-synthesized blips (no binary assets) — two
alternating low pitches (G4/E4) with pitch glide + smooth attack/decay
envelopes, ~0.8-1.2s randomized spacing, volume = voice.beep_volume * 0.5.
start_thinking_sound(should_play=...) / stop_thinking_sound() daemon-loop
lifecycle; macOS-TCC-safe (sounddevice output gated there → silent skip,
no per-second afplay churn). New mark_audio_output_active()/
is_audio_output_active() ref-count wraps play_audio_file and the
streaming OutputStream sentence writes so "audio is flowing" is accurate.
- Config: voice.thinking_sound (default true) off-switch.
- cli.py: starts when a voice-mode turn begins, per-blip gate skips while
TTS speaks / mic records / barge capture owns the mic; stopped in the
chat() finally on every exit path.
- tui_gateway/server.py: same lifecycle around _run_prompt_submit turns
(voice mode on), gated on is_audio_output_active + continuous capture.
- Desktop: renderer owns voice-conversation audio, so a matching WebAudio
implementation (src/lib/thinking-sound.ts, same envelope/pitches) runs
while conversation status === "thinking"; honors voice.thinking_sound
(via config store) and the shared sound-mute toggle; stops instantly on
speaking/listening/end.
One owner for the wording: voice_stop_hint() in tools/voice_mode.py —
sources the phrase from voice.stop_phrases (first entry) so a custom
phrase renders correctly, and returns "" when the feature is disabled
(stop_phrases: []) so no surface shows a hint.
- CLI: printed in /voice on output (style-matched dim notice).
- TUI: voice.toggle action=on now carries stop_hint; the Ink client
renders it in the "Voice mode enabled" block (older gateways omit
the field — no hint, no crash).
- Desktop: the renderer voice loop never touches tools/voice_mode.py,
so the phrase is read from config (voice.stop_phrases → $voiceStopPhrase
store, seeded in use-hermes-config) and shown as an info toast when a
voice conversation starts. i18n: en/ja/zh/zh-hant/ar.
Replace one-shot VAD calibration with a rolling deque window that
continuously recalibrates the noise floor throughout TTS playback,
preventing false barge-in triggers from stale calibration. Add a
grace period before VAD activates so TTS playback establishes first.
Suppress duplicate text rendering when token streaming is enabled.
Mirror the barge-in and TTS stream stop logic to the TUI gateway path
so both CLI and gateway use the same VAD semantics.
Rolling-window VAD:
- 90th percentile of rolling window (~3s) for noise floor
- 8x multiplier (was 5x) for TTS volume variation headroom
- 4000 RMS trigger ceiling so genuine speech can still trip
- min_floor clamped to SILENCE_RMS_THRESHOLD * 2
- sustained_ms=1000, calibration_ms=800
Barge-in grace period (barge_in_grace_seconds, default 2.0s):
Delays VAD activation so TTS playback establishes before the mic opens.
Duplicate render suppression:
When streaming_enabled, pass display_callback=None to
stream_tts_to_speaker so the token stream is the sole display path.
TUI gateway mirror:
Mirror _tts_stream_stop and _tts_stream_barge_in_monitor changes to
tui_gateway/server.py so both code paths use the same VAD parameters,
grace period, and TTS CUT diagnostic logging.
Profile-scoped session DB and MoA progress events in tui_gateway/server.py
were necessitated by the TTS pipeline changes affecting session state
and event routing.
Normal-exit flag and TTS CUT diagnostic logging at all cut paths.
Regression tests:
- test_quiet_then_loud_playback_does_not_trip
- test_8x_multiplier_absorbs_tts_volume_spikes
- test_trigger_ceiling_lets_genuine_speech_trip
- test_silence_calibration_does_not_false_trip_on_tts
- test_tts_stream_stop_latches_interruption_for_next_turn
- test_tts_stream_stop_after_natural_finish_does_not_latch
- Profile-scoped session DB tests (10 tests)
The exact kwargs snapshot broke when VAD hardening added keys — the
test's real contract is 'null stt.local: must not crash or force
language/prompt'. Baseline kwargs are pinned by the dedicated suite.
Local faster-whisper called model.transcribe with bare {'beam_size': 5}:
no VAD, cross-window conditioning on, no confidence filtering. Pure
silence produced hallucinated tokens (E2E: 5s anullsrc WAV -> 'You',
no_speech_prob=0.705) and noisy clips could produce runs of junk, often
in other languages.
Three-layer class fix, one shared owner for every local-whisper call
site (build_local_transcribe_kwargs):
1. Silero VAD filter (bundled with faster-whisper) on by default —
silence never reaches the model. stt.local.vad: false restores the
raw behavior for music/ambient transcription.
stt.local.vad_min_silence_ms tunes chunk splitting (default 500).
2. condition_on_previous_text=False — one hallucinated token can no
longer seed a self-reinforcing run; negligible cost for
voice-note-length audio.
3. Segment confidence gate (_join_confident_segments): drop a segment
only when no_speech_prob > 0.6 AND avg_logprob < -1.0 (openai-whisper's
own heuristic shape; both must hit so quiet-but-real speech survives).
Config: stt.local.no_speech_prob_threshold / logprob_threshold.
The WHISPER_HALLUCINATIONS blocklist in voice_mode.py stays as
last-resort defense but should now almost never fire.
E2E (real faster-whisper 'base', CPU int8):
silence.wav before 'You' -> after ''
noise.wav before '' -> after ''
speech.wav before/after 'Hello World, this is a test of the
transcription system.' (unchanged)
Docs (EN + zh-Hans), DEFAULT_CONFIG, cli-config.yaml.example updated;
19 unit tests (kwargs contract, off-switch, confidence gate incl.
quiet-speech survival, _transcribe_local wiring), sabotage-verified.
Saying OR typing a configured stop phrase (voice.stop_phrases, default
"stop") now ends the voice chat everywhere, not just classic CLI PTT:
- hermes_cli/voice.py: new explicit on_stop_phrase callback through
start_continuous/stop_continuous. The force-transcribe path previously
DISCARDED the stop phrase silently — with auto_restart=False the client
re-arms the next capture, so the conversation never ended. Both halt
paths now fire on_stop_phrase (fallback: on_silent_limit for legacy
callers) as user intent, distinct from the no-speech timeout.
- tui_gateway/server.py: voice.record wires on_stop_phrase and emits
voice.transcript {stop_phrase: true} after flipping HERMES_VOICE(_TTS)
off and stopping streaming TTS — same teardown as /voice off. The TTS
barge-in monitor stop-checks its transcript too. prompt.submit consumes
a TYPED bare stop phrase at the server-side choke point when voice mode
is on (returns {voice_stopped: true}, no turn starts).
- ui-tui: voice.transcript {stop_phrase} ends voice mode with a clear
'voice chat ended' notice (distinct from the no-speech-limit message);
submitPrompt releases the busy latch on a consumed voice_stopped reply.
- cli.py: _typed_voice_stop in process_loop — typing a bare stop phrase
while voice mode/continuous is active ends voice mode instead of
sending 'stop' to the agent; typed 'stop' outside voice mode is
unchanged. Voice transcripts skip the check (already stop-checked).
- desktop: interceptsTypedVoiceStop — the composer's onSubmit ends the
live voice conversation (same path as clicking end on the pill) when a
bare stop command is typed with no attachments; renderer-owned loop, so
handled client-side like the existing spoken isVoiceStopCommand.
- tools/voice_mode.py: transcribe_recording never lets the Whisper
hallucination filter swallow a configured stop phrase (e.g. 'bye'
configured as a stop phrase is both a hallucination-blocklist entry and
a stop phrase — stop-phrase check now wins).
Tests: continuous-loop signal (sabotage-verified), force-transcribe stop
signal + legacy fallback, hallucination-filter ordering, typed-stop CLI
unit tests (voice on/off/longer text), prompt.submit typed-stop gateway
tests, TUI vitest for stop_phrase event handling, desktop vitest for the
typed-stop interceptor.
The Photon iMessage sidecar needs node_modules under
plugins/platforms/photon/sidecar/, but hosted/managed images keep the
whole install tree under an immutable /opt/hermes — every install and
self-heal path (setup CLI, stale-deps reinstall, cold install) died on
EROFS, and hosted users have no shell to work around it.
Three-layer fix, mirroring the WhatsApp bridge resolver pattern:
1. Bake the deps into the image. The Dockerfile now runs npm ci for the
sidecar in the layer-cached dependency stage (deterministic installs
from the committed lockfile; the postinstall spectrum-ts patch runs
at build time). Hosted happy path needs no runtime install at all.
2. New sidecar_paths.resolve_sidecar_dir() decides where the sidecar
runs from: PHOTON_SIDECAR_DIR override > writable source dir (dev
installs, unchanged) > read-only dir with baked fresh deps (managed
image) > mirror to $HERMES_HOME/photon/sidecar (writable data
volume) when deps are missing or stale in a read-only tree. The
mirror refreshes changed source files on image updates while
keeping node_modules, so the existing lockfile-staleness self-heal
works there.
3. connect() can now cold-install: _start_sidecar() runs the bounded
npm ci bootstrap when node_modules is missing instead of raising
immediately, and check_requirements() reports available when a
self-install is possible (npm present + writable resolved dir) so
the gateway actually creates the adapter on hosted instances. A
failed bootstrap still raises the actionable error, which connect()
surfaces as the retryable SIDECAR_FAILED fatal state on the
dashboard.
Tests: resolver decision table (env override, in-place, mirror,
refresh, fail-open), cold-install lifecycle paths, and a Dockerfile
contract test guarding the baked-deps + no-chown invariants.
Fixes NS-606.
Adds gpt-transcribe (OpenAI's new file-transcription model, $0.0045/min)
to the OpenAI STT provider:
- OPENAI_MODELS set: gpt-transcribe is recognized so provider
auto-correction keeps it on OpenAI and rejects it on Groq
- Language hint wiring: gpt-transcribe replaces the singular
'language' field with a 'languages' list; the API rejects the legacy
field, so the hint is sent via extra_body {languages: [..]}
- Config comment (DEFAULT_CONFIG), cli-config.yaml.example, desktop
settings enum, and docs (en + zh-Hans) updated
- Tests: model pass-through, languages-list hint shape, legacy singular
hint preserved for gpt-4o-transcribe, Groq auto-correction
gpt-live-transcribe (realtime WebSocket, $0.017/min) is NOT wired here:
the file-based STT pipeline has no realtime session path; it belongs in
a future realtime/voice-mode integration.
Installing a memory provider (Honcho, mem0, hindsight, ...) from the
dashboard Plugins page failed on hosted deployments with a permission
error: the setup endpoint shelled out to
`uv pip install --python sys.executable`, which targets the sealed
read-only venv under /opt/hermes (immutable hosted image, NS-579/#49113).
The correct mechanism already exists: tools/lazy_deps.py redirects
installs to the writable durable target on the data volume
(HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages) when the venv is
sealed (HERMES_DISABLE_LAZY_INSTALLS=1), appends the target to the END
of sys.path (core venv always wins collisions), and constrains shared
deps to core-venv versions. The dashboard installer simply never used
it.
Fix:
- tools/lazy_deps.py: new public install_specs() — installs arbitrary
manifest-declared pip specs through the same environment routing as
ensure(): venv-scoped by default, durable-target on sealed images,
refused with an actionable reason when gated off (config kill switch
or sealed venv without a target — never surfaces raw EROFS/EACCES).
Specs are validated with _spec_is_safe(); post-install it invalidates
import/metadata caches so availability rechecks in the same process
see the new packages without a restart. Never raises.
- hermes_cli/web_server.py: _install_memory_provider_pip_dependencies
now calls install_specs() instead of building its own uv/pip
subprocess. Blocked installs surface the gate reason in the setup
results; the response's status block reflects post-install
availability (stale 'missing deps' state clears immediately).
- hermes_cli/memory_setup.py, plugins/memory/honcho/cli.py,
plugins/memory/mem0/_setup.py: CLI setup wizards routed through
install_specs() too — same sealed-venv failure mode, same fix.
No hosted setup path writes to /opt/hermes anymore; provider discovery
and installation now use the same environment (sys.path activation is
shared with the lazy-install bootstrap in hermes_bootstrap).
Tests:
- tests/tools/test_lazy_deps.py: TestInstallSpecs — gating matrix
(sealed+no-target blocked with immutable-deployment reason, config
kill switch, sealed+target proceeds), spec-safety rejection before
any subprocess, venv-scoped vs --target command display, failure
stderr passthrough, never-raises contract.
- tests/hermes_cli/test_web_server.py: setup endpoint routes pip
through lazy_deps (regression guard asserts no direct 'pip install'
subprocess), blocked-reason surfacing, same-response availability
recheck clears stale missing state.
Fixes NS-605 (Plain T-1111).
Follow-up integration for the #47588 salvage, aligning the new streamers
with the post-campaign invariants:
- All streaming key lookups go through _resolve_key -> tts_tool.
_resolve_provider_key -> resolve_provider_secret (config > env/.env >
credential pool, profile-scoped) — never bare get_env_value. xAI
resolves via resolve_xai_http_credentials so OAuth users stream too.
- _capped(): every provider's chunk iterator is bounded at 16 MiB per
sentence, mirroring _read_tts_response_bytes' bounded-upstream-body
invariant on the sync paths.
- Tests updated for the resolver contract + new coverage for credential
routing and the cap.
Salvaged from PR #47588 and rebased onto the post-campaign streaming core:
the StreamingTTSProvider ABC/registry and the ElevenLabs/OpenAI streamers
already live on main (tools/tts_streaming.py), so this ports the pieces
main lacked:
- GeminiStreamer: streamGenerateContent?alt=sse -> base64 PCM chunks
(24 kHz mono int16), reusing main's DEFAULT_GEMINI_TTS_* constants.
- XAIStreamer: WebSocket wss://api.x.ai/v1/tts -> binary PCM frames,
async->sync bridged via the _collect_async test seam.
- tts.streaming.provider config knob: pin one streamer, or 'auto' to
walk the priority list elevenlabs -> gemini -> openai -> xai. Unset
keeps the never-swap-the-user's-voice default.
- docs/streaming-tts.md: architecture, capability matrix, how to add
a provider.
- Unit tests for the knob, SSE parsing, and the WS bridge; key-gated
E2E tests (skipped without credentials).
Refs: #47588
Two follow-ups from the voice PR (#70509).
1. macOS ARM64 onnx migration. Existing users who pinned
openwakeword.inference_framework=onnx before the tflite fix landed kept a
wake word that arms but never fires (ONNX's embedding model is broken on
Apple Silicon, upstream #336). New resolve_inference_framework() honors an
explicit framework everywhere ONNX actually works, but coerces the one
provably-dead combination (explicit onnx + macOS ARM64) to tflite with a
one-time warning. No config mutation; empty still falls back to the platform
default. Both read sites (engine init + requirements check) route through
the shared resolver.
2. Voice turn-timeout leak. Each listen cycle reassigned turnTimeoutRef
without clearing the prior 60s timer, so a stale timer from an earlier
cycle could fire handleTurn() mid-way through a later listen — after enough
idle re-listens this wedged the loop into a non-re-arming state (the
'voice chat deactivates after ~a minute' report). Clear before re-arm.
Tests: 64 wake tests (added onnx-coercion / intel-kept / tflite-kept /
empty-default cases; updated the stale 'explicit onnx kept on ARM64' test
that encoded the old broken behavior), 39 desktop voice/wake vitest, tsc +
eslint clean.
'photon:any;-;+1555...' targets matched no parser pattern, so
_handle_send bounced them off the channel directory and failed
resolution even though the adapter accepts the GUID verbatim (the
react handler already passed them through). Recognize the DM chat
GUID shape (mirrors the adapter's _DM_CHAT_GUID_RE) in
_parse_target_ref for photon only.
Command providers legitimately reference their own API keys in shell
templates (curl one-liners). The #70342 scrub removes ALL provider keys,
which would break such setups. Add a per-provider env_passthrough list
(TTS + STT) that copies named variables back from the parent env, plus
docs and tests. Scrub stays the default; passthrough is explicit opt-in.
Port the progress-based idle-timeout pattern from _run_command_tts
(PR #50087, @CleanDev-Fix) to _run_command_stt: the timeout resets on
any stdout/stderr output, so a slow-but-alive STT provider survives
while a silently stalled one is killed. Stuck detection stays
progress-based, never wall-clock.
`transcribe_audio` reads a local file and hands it to the configured STT
provider — for the hosted providers (Groq, OpenAI, Mistral, xAI, ElevenLabs)
that ships the file's bytes to a third-party API. The same local-input read
guard was added to image-gen (587be5b5b) and xAI video-gen (104232979) to keep
the agent from feeding credential/secret stores to a provider, but STT was
missed.
Call `get_read_block_error(file_path)` at the top of `transcribe_audio`, before
validation/dispatch, so a `.env`, `auth.json`, `.anthropic_oauth.json`,
`mcp-tokens/`, etc. is refused up front instead of being transcribed (and, for
hosted providers, exfiltrated). This is defense-in-depth, not a security
boundary — the guard's own message says so — but it restores parity with the
image/video-gen tools.
Regression test: a `.env` file is refused with the shared read-guard message
before any provider dispatch (mutation-verified).
HERMES_LOCAL_STT_COMMAND rendered quoted placeholders into a
user-configured template and passed the result to shell=True. Shell
metacharacters in the template therefore remained executable syntax even
though the placeholder values themselves were quoted.
Tokenize the rendered template and invoke it as an argv list while
preserving the existing timeout, closed stdin, and Windows creation flags.
Lock the invocation contract with metacharacter regression coverage and
document explicit shell wrapping for trusted templates that need it.
Salvages #32694
Co-authored-by: Ernest Hysa <takis312@hotmail.com>
Command-type TTS providers validated output_format against a hardcoded
{mp3,wav,ogg,flac} set; any other value was silently coerced back to mp3,
which then mismatched the output path the post-run check expects. This
blocked common ffmpeg-producible containers/codecs — notably m4a (AAC),
the portable choice for WeChat/iOS/mobile voice files — with no
config-only path (only a local source patch, lost on every update).
Widen COMMAND_TTS_OUTPUT_FORMATS to add m4a, aac, amr, opus. This only
permits a command provider to declare these; the user's command still
produces the file (e.g. via ffmpeg). No built-in provider behavior
changes and no new required config.
Update the two tests that pinned the old set, and add a positive case
covering the new formats. Document the supported output_format values.
## Summary
- Spawn system audio players (`ffplay` / `afplay` / `aplay`) with `hermes_subprocess_env(inherit_credentials=False)`.
- Prevent gateway tokens and provider API keys from leaking into OS media helpers.
- Add a regression test asserting scrubbed env on `Popen`.
## Salvage / credit
Sibling of #70342 / incomplete #56332 (TTS/STT command scrub) on the voice-mode playback path.
Salvage incomplete #56332: route command TTS/STT through hermes_subprocess_env
while preserving delegated-child lineage, and close the sibling local-whisper
subprocess.run path that still inherited the full process environment.
Co-authored-by: Cursor <cursoragent@cursor.com>
CI (Linux) failed with ModuleNotFoundError: portalocker — it's a
win32-only dependency (concurrent-log-handler chain). Tests now lock
handles via a platform helper (fcntl on POSIX, portalocker on Windows),
mirroring production _try_acquire_mcp_discovery_lock.
Review follow-up: test_micro_pause_tolerance_during_speech was left on real
sleeps, so it kept the wall-clock dependency the rest of this PR removes.
Its three sleeps (50 ms, 50 ms, 60 ms) feed the same monotonic gate at
tools/voice_mode.py:561 that the other two tests were converted for.
It was measured rather than assumed before being left out: 0 failures in 40
runs, because on Windows sleep() rounds up to the timer tick, so sleep(0.05)
really costs ~62.5 ms and the three sleeps clear the 150 ms gate by ~37.5 ms
-- wider than the 15.625 ms GetTickCount64() quantum that made the other two
flake. That margin is an accident of sleep granularity rather than anything
the test guarantees, so it is not worth keeping the bet.
Driving it from fake_clock makes the timeline exact (0.05 + 0.05 + 0.06 =
0.16 against the 0.15 gate, dip 0.05 under the 0.1 tolerance) and drops the
real sleeping from the suite. Mutation-checked: widening the gate at :561
fails the test.
test_silence_callback_fires_after_speech_then_silence and
test_custom_threshold_and_duration space their audio frames with
time.sleep(0.06) and check the result against 50 ms thresholds. That
leaves a 10 ms margin, which only survives if the clock is finer-grained
than the margin — and on Windows it isn't. time.monotonic() there is
GetTickCount64() with 15.625 ms resolution until CPython 3.13 moved it to
QueryPerformanceCounter(), so a sleep that really lasts 62.5 ms measures
as 46.9 ms often enough to matter, landing under the threshold. Depending
on which sleep got clipped, either the speech-confirm gate misses or the
silence timer never matures — which is why the same flake shows up on two
different asserts.
Both tests now advance a hand-driven clock instead of sleeping, so the
arithmetic is exact on every platform and neither test waits on real time.
The fixture patches the `time` name inside tools.voice_mode rather than
time.monotonic itself: voice_mode.time IS the stdlib module, so patching
through it would swap the clock out from under every other importer for
the duration of the test.
Measured on Windows 11 / Python 3.11.9, 20 runs of each test: 6/20 and
3/20 failed before, 0/30 after. The same machine on Python 3.13, where
monotonic() is QueryPerformanceCounter(), never failed either test — that
comparison is what pinned the clock resolution as the cause rather than
the detection logic. Linux CI never sees this: clock_gettime is
nanosecond-resolution there.
Running the suite could take over the machine it ran on. Two real
effects, both now closed:
- **The suite spoke through the speakers.** Once any test drove the
`voice.toggle` RPC with `action="tts"`, the handler set
`HERMES_VOICE_TTS=1` in the *live process environment*, and the flag
outlived that test. Every later test that drove a turn to completion
then fed its final response text to `hermes_cli.voice.speak_text` on a
background thread - real synthesis, real playback, no API key needed
(the default `edge` provider is keyless). A developer heard the fixture
string "partial answer complete" out loud. Because the flag is set from
inside the process, `scripts/run_tests.sh`'s `env -i` never protected
against this.
`tests/conftest.py` now blanks `HERMES_VOICE`/`HERMES_VOICE_TTS` per
test, and a new autouse `_audio_playback_guard` stubs `speak_text` and
its playback binding outright, so the speakers stay shut even inside the
test that sets the flag itself. `@pytest.mark.real_audio_playback` opts
out.
- **The suite launched Chrome.** `tests/tools/test_browser_supervisor.py`
spawned a real browser on any machine with Chrome on `PATH`. Its
docstring promised a `HERMES_E2E_BROWSER=1` gate that existed nowhere in
the code. That gate is now real, and the file is marked `integration`
so the default marker filter excludes it. `scripts/run_tests.sh`
forwards `HERMES_E2E_BROWSER` so the documented manual run still works.
Miscellanea
- `tests/test_audio_playback_guard.py`: regression cover for both defences,
driving the real `voice.toggle` handler rather than a stand-in.
- Move U+FFFC placeholder detection before _record_last_inbound() so the
placeholder message id is not recorded as the reaction target
- Cancel pending U+FFFC tasks in disconnect() to prevent task leaks
- Check mimeType audio/x-caf in addition to filename for CAF→VOICE promotion,
fixing unnamed attachments that default to '(unnamed)'
- Add 7 Photon adapter tests: CAF named/unnamed promotion, U+FFFC no-dispatch,
U+FFFC not recorded as last inbound, U+FFFC+attachment cancel, timeout fire,
disconnect cleanup
- Add 6 transcription tests: ffmpeg conversion, afconvert fallback, all-fail,
CAF→WAV before Groq, conversion failure error, local provider skip
_generate_xai_tts passes stream=True to requests.post since the TTS
speaker pipeline moved to the streaming core. Four fake_post mocks in
test_tts_xai_speech_tags.py still had the pre-streaming signature and
raised TypeError: unexpected keyword argument 'stream'. The other 20+
mocks in the same file already accept stream=False; this aligns the
stragglers.
Fixes the 4 failures in Python tests slice 5/8 on main.
