Commit graph

1849 commits

Author SHA1 Message Date
Teknium
bff2206972 test: convert null-local-config assertion from exact-dict to invariant
The exact kwargs snapshot broke when VAD hardening added keys — the
test's real contract is 'null stt.local: must not crash or force
language/prompt'. Baseline kwargs are pinned by the dedicated suite.
2026-07-29 00:01:33 -07:00
Teknium
bf8004e3a8 fix(stt): kill faster-whisper silence hallucinations at the source
Local faster-whisper called model.transcribe with bare {'beam_size': 5}:
no VAD, cross-window conditioning on, no confidence filtering. Pure
silence produced hallucinated tokens (E2E: 5s anullsrc WAV -> 'You',
no_speech_prob=0.705) and noisy clips could produce runs of junk, often
in other languages.

Three-layer class fix, one shared owner for every local-whisper call
site (build_local_transcribe_kwargs):

1. Silero VAD filter (bundled with faster-whisper) on by default —
   silence never reaches the model. stt.local.vad: false restores the
   raw behavior for music/ambient transcription.
   stt.local.vad_min_silence_ms tunes chunk splitting (default 500).
2. condition_on_previous_text=False — one hallucinated token can no
   longer seed a self-reinforcing run; negligible cost for
   voice-note-length audio.
3. Segment confidence gate (_join_confident_segments): drop a segment
   only when no_speech_prob > 0.6 AND avg_logprob < -1.0 (openai-whisper's
   own heuristic shape; both must hit so quiet-but-real speech survives).
   Config: stt.local.no_speech_prob_threshold / logprob_threshold.

The WHISPER_HALLUCINATIONS blocklist in voice_mode.py stays as
last-resort defense but should now almost never fire.

E2E (real faster-whisper 'base', CPU int8):
  silence.wav  before 'You'                        -> after ''
  noise.wav    before ''                           -> after ''
  speech.wav   before/after 'Hello World, this is a test of the
               transcription system.' (unchanged)

Docs (EN + zh-Hans), DEFAULT_CONFIG, cli-config.yaml.example updated;
19 unit tests (kwargs contract, off-switch, confidence gate incl.
quiet-speech survival, _transcribe_local wiring), sabotage-verified.
2026-07-29 00:01:33 -07:00
Teknium
ba13132298 fix(voice): bare stop phrase ends the voice chat on every surface, spoken or typed
Saying OR typing a configured stop phrase (voice.stop_phrases, default
"stop") now ends the voice chat everywhere, not just classic CLI PTT:

- hermes_cli/voice.py: new explicit on_stop_phrase callback through
  start_continuous/stop_continuous. The force-transcribe path previously
  DISCARDED the stop phrase silently — with auto_restart=False the client
  re-arms the next capture, so the conversation never ended. Both halt
  paths now fire on_stop_phrase (fallback: on_silent_limit for legacy
  callers) as user intent, distinct from the no-speech timeout.
- tui_gateway/server.py: voice.record wires on_stop_phrase and emits
  voice.transcript {stop_phrase: true} after flipping HERMES_VOICE(_TTS)
  off and stopping streaming TTS — same teardown as /voice off. The TTS
  barge-in monitor stop-checks its transcript too. prompt.submit consumes
  a TYPED bare stop phrase at the server-side choke point when voice mode
  is on (returns {voice_stopped: true}, no turn starts).
- ui-tui: voice.transcript {stop_phrase} ends voice mode with a clear
  'voice chat ended' notice (distinct from the no-speech-limit message);
  submitPrompt releases the busy latch on a consumed voice_stopped reply.
- cli.py: _typed_voice_stop in process_loop — typing a bare stop phrase
  while voice mode/continuous is active ends voice mode instead of
  sending 'stop' to the agent; typed 'stop' outside voice mode is
  unchanged. Voice transcripts skip the check (already stop-checked).
- desktop: interceptsTypedVoiceStop — the composer's onSubmit ends the
  live voice conversation (same path as clicking end on the pill) when a
  bare stop command is typed with no attachments; renderer-owned loop, so
  handled client-side like the existing spoken isVoiceStopCommand.
- tools/voice_mode.py: transcribe_recording never lets the Whisper
  hallucination filter swallow a configured stop phrase (e.g. 'bye'
  configured as a stop phrase is both a hallucination-blocklist entry and
  a stop phrase — stop-phrase check now wins).

Tests: continuous-loop signal (sabotage-verified), force-transcribe stop
signal + legacy fallback, hallucination-filter ordering, typed-stop CLI
unit tests (voice on/off/longer text), prompt.submit typed-stop gateway
tests, TUI vitest for stop_phrase event handling, desktop vitest for the
typed-stop interceptor.
2026-07-29 00:01:06 -07:00
Shannon Sands
0dfd5546fc fix(photon): support immutable install trees for the sidecar (NS-606)
The Photon iMessage sidecar needs node_modules under
plugins/platforms/photon/sidecar/, but hosted/managed images keep the
whole install tree under an immutable /opt/hermes — every install and
self-heal path (setup CLI, stale-deps reinstall, cold install) died on
EROFS, and hosted users have no shell to work around it.

Three-layer fix, mirroring the WhatsApp bridge resolver pattern:

1. Bake the deps into the image. The Dockerfile now runs npm ci for the
   sidecar in the layer-cached dependency stage (deterministic installs
   from the committed lockfile; the postinstall spectrum-ts patch runs
   at build time). Hosted happy path needs no runtime install at all.

2. New sidecar_paths.resolve_sidecar_dir() decides where the sidecar
   runs from: PHOTON_SIDECAR_DIR override > writable source dir (dev
   installs, unchanged) > read-only dir with baked fresh deps (managed
   image) > mirror to $HERMES_HOME/photon/sidecar (writable data
   volume) when deps are missing or stale in a read-only tree. The
   mirror refreshes changed source files on image updates while
   keeping node_modules, so the existing lockfile-staleness self-heal
   works there.

3. connect() can now cold-install: _start_sidecar() runs the bounded
   npm ci bootstrap when node_modules is missing instead of raising
   immediately, and check_requirements() reports available when a
   self-install is possible (npm present + writable resolved dir) so
   the gateway actually creates the adapter on hosted instances. A
   failed bootstrap still raises the actionable error, which connect()
   surfaces as the retryable SIDECAR_FAILED fatal state on the
   dashboard.

Tests: resolver decision table (env override, in-place, mirror,
refresh, fail-open), cold-install lifecycle paths, and a Dockerfile
contract test guarding the baked-deps + no-chown invariants.

Fixes NS-606.
2026-07-28 22:41:32 -07:00
Teknium
805c1c340c feat(stt): support OpenAI gpt-transcribe transcription model
Adds gpt-transcribe (OpenAI's new file-transcription model, $0.0045/min)
to the OpenAI STT provider:

- OPENAI_MODELS set: gpt-transcribe is recognized so provider
  auto-correction keeps it on OpenAI and rejects it on Groq
- Language hint wiring: gpt-transcribe replaces the singular
  'language' field with a 'languages' list; the API rejects the legacy
  field, so the hint is sent via extra_body {languages: [..]}
- Config comment (DEFAULT_CONFIG), cli-config.yaml.example, desktop
  settings enum, and docs (en + zh-Hans) updated
- Tests: model pass-through, languages-list hint shape, legacy singular
  hint preserved for gpt-4o-transcribe, Groq auto-correction

gpt-live-transcribe (realtime WebSocket, $0.017/min) is NOT wired here:
the file-based STT pipeline has no realtime session path; it belongs in
a future realtime/voice-mode integration.
2026-07-28 22:40:45 -07:00
Shannon Sands
8bbd77f368 fix: route memory-provider dep installs through lazy_deps durable target
Installing a memory provider (Honcho, mem0, hindsight, ...) from the
dashboard Plugins page failed on hosted deployments with a permission
error: the setup endpoint shelled out to
`uv pip install --python sys.executable`, which targets the sealed
read-only venv under /opt/hermes (immutable hosted image, NS-579/#49113).

The correct mechanism already exists: tools/lazy_deps.py redirects
installs to the writable durable target on the data volume
(HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages) when the venv is
sealed (HERMES_DISABLE_LAZY_INSTALLS=1), appends the target to the END
of sys.path (core venv always wins collisions), and constrains shared
deps to core-venv versions. The dashboard installer simply never used
it.

Fix:
- tools/lazy_deps.py: new public install_specs() — installs arbitrary
  manifest-declared pip specs through the same environment routing as
  ensure(): venv-scoped by default, durable-target on sealed images,
  refused with an actionable reason when gated off (config kill switch
  or sealed venv without a target — never surfaces raw EROFS/EACCES).
  Specs are validated with _spec_is_safe(); post-install it invalidates
  import/metadata caches so availability rechecks in the same process
  see the new packages without a restart. Never raises.
- hermes_cli/web_server.py: _install_memory_provider_pip_dependencies
  now calls install_specs() instead of building its own uv/pip
  subprocess. Blocked installs surface the gate reason in the setup
  results; the response's status block reflects post-install
  availability (stale 'missing deps' state clears immediately).
- hermes_cli/memory_setup.py, plugins/memory/honcho/cli.py,
  plugins/memory/mem0/_setup.py: CLI setup wizards routed through
  install_specs() too — same sealed-venv failure mode, same fix.

No hosted setup path writes to /opt/hermes anymore; provider discovery
and installation now use the same environment (sys.path activation is
shared with the lazy-install bootstrap in hermes_bootstrap).

Tests:
- tests/tools/test_lazy_deps.py: TestInstallSpecs — gating matrix
  (sealed+no-target blocked with immutable-deployment reason, config
  kill switch, sealed+target proceeds), spec-safety rejection before
  any subprocess, venv-scoped vs --target command display, failure
  stderr passthrough, never-raises contract.
- tests/hermes_cli/test_web_server.py: setup endpoint routes pip
  through lazy_deps (regression guard asserts no direct 'pip install'
  subprocess), blocked-reason surfacing, same-response availability
  recheck clears stale missing state.

Fixes NS-605 (Plain T-1111).
2026-07-28 22:40:33 -07:00
Teknium
7800bb1a29 fix(tts): route streaming-provider secrets through resolve_provider_secret; bound per-sentence stream bodies at 16 MiB
Follow-up integration for the #47588 salvage, aligning the new streamers
with the post-campaign invariants:

- All streaming key lookups go through _resolve_key -> tts_tool.
  _resolve_provider_key -> resolve_provider_secret (config > env/.env >
  credential pool, profile-scoped) — never bare get_env_value. xAI
  resolves via resolve_xai_http_credentials so OAuth users stream too.
- _capped(): every provider's chunk iterator is bounded at 16 MiB per
  sentence, mirroring _read_tts_response_bytes' bounded-upstream-body
  invariant on the sync paths.
- Tests updated for the resolver contract + new coverage for credential
  routing and the cap.
2026-07-28 22:31:40 -07:00
Carlos Diosdado
bc4dcb1b02 feat(tts): Gemini SSE + xAI WebSocket streaming providers, tts.streaming.provider knob, docs + E2E tests
Salvaged from PR #47588 and rebased onto the post-campaign streaming core:
the StreamingTTSProvider ABC/registry and the ElevenLabs/OpenAI streamers
already live on main (tools/tts_streaming.py), so this ports the pieces
main lacked:

- GeminiStreamer: streamGenerateContent?alt=sse -> base64 PCM chunks
  (24 kHz mono int16), reusing main's DEFAULT_GEMINI_TTS_* constants.
- XAIStreamer: WebSocket wss://api.x.ai/v1/tts -> binary PCM frames,
  async->sync bridged via the _collect_async test seam.
- tts.streaming.provider config knob: pin one streamer, or 'auto' to
  walk the priority list elevenlabs -> gemini -> openai -> xai. Unset
  keeps the never-swap-the-user's-voice default.
- docs/streaming-tts.md: architecture, capability matrix, how to add
  a provider.
- Unit tests for the knob, SSE parsing, and the WS bridge; key-gated
  E2E tests (skipped without credentials).

Refs: #47588
2026-07-28 22:31:40 -07:00
Teknium
0f64557c06 fix(wake): coerce dead onnx->tflite on macOS ARM64; clear stale voice turn-timeout
Two follow-ups from the voice PR (#70509).

1. macOS ARM64 onnx migration. Existing users who pinned
   openwakeword.inference_framework=onnx before the tflite fix landed kept a
   wake word that arms but never fires (ONNX's embedding model is broken on
   Apple Silicon, upstream #336). New resolve_inference_framework() honors an
   explicit framework everywhere ONNX actually works, but coerces the one
   provably-dead combination (explicit onnx + macOS ARM64) to tflite with a
   one-time warning. No config mutation; empty still falls back to the platform
   default. Both read sites (engine init + requirements check) route through
   the shared resolver.

2. Voice turn-timeout leak. Each listen cycle reassigned turnTimeoutRef
   without clearing the prior 60s timer, so a stale timer from an earlier
   cycle could fire handleTurn() mid-way through a later listen — after enough
   idle re-listens this wedged the loop into a non-re-arming state (the
   'voice chat deactivates after ~a minute' report). Clear before re-arm.

Tests: 64 wake tests (added onnx-coercion / intel-kept / tflite-kept /
empty-default cases; updated the stale 'explicit onnx kept on ARM64' test
that encoded the old broken behavior), 39 desktop voice/wake vitest, tsc +
eslint clean.
2026-07-28 19:40:25 -07:00
Teknium
f041c95b7f fix(send_message): pass photon DM chat GUIDs through as explicit targets
'photon:any;-;+1555...' targets matched no parser pattern, so
_handle_send bounced them off the channel directory and failed
resolution even though the adapter accepts the GUID verbatim (the
react handler already passed them through). Recognize the DM chat
GUID shape (mirrors the adapter's _DM_CHAT_GUID_RE) in
_parse_target_ref for photon only.
2026-07-28 18:21:01 -07:00
Teknium
e807b7106c test: set returncode on fake Popen proc (main's player loop reads returncode, not wait()) 2026-07-28 18:12:26 -07:00
Teknium
e251e78df9 feat(tools): env_passthrough allowlist for command-provider secret scrub
Command providers legitimately reference their own API keys in shell
templates (curl one-liners). The #70342 scrub removes ALL provider keys,
which would break such setups. Add a per-provider env_passthrough list
(TTS + STT) that copies named variables back from the parent env, plus
docs and tests. Scrub stays the default; passthrough is explicit opt-in.
2026-07-28 18:12:26 -07:00
Teknium
59fc68e93f test: align env-scrub fixtures with idle-timeout runner; assert env passed in no-shell kwargs 2026-07-28 18:12:26 -07:00
Teknium
fc26e965bb fix(tools): apply idle timeout to command STT runner (class fix for #50081)
Port the progress-based idle-timeout pattern from _run_command_tts
(PR #50087, @CleanDev-Fix) to _run_command_stt: the timeout resets on
any stdout/stderr output, so a slow-but-alive STT provider survives
while a silently stalled one is killed. Stuck detection stays
progress-based, never wall-clock.
2026-07-28 18:12:26 -07:00
Frowtek
38bb193f38 fix(stt): route transcription inputs through the shared read guard
`transcribe_audio` reads a local file and hands it to the configured STT
provider — for the hosted providers (Groq, OpenAI, Mistral, xAI, ElevenLabs)
that ships the file's bytes to a third-party API. The same local-input read
guard was added to image-gen (587be5b5b) and xAI video-gen (104232979) to keep
the agent from feeding credential/secret stores to a provider, but STT was
missed.

Call `get_read_block_error(file_path)` at the top of `transcribe_audio`, before
validation/dispatch, so a `.env`, `auth.json`, `.anthropic_oauth.json`,
`mcp-tokens/`, etc. is refused up front instead of being transcribed (and, for
hosted providers, exfiltrated). This is defense-in-depth, not a security
boundary — the guard's own message says so — but it restores parity with the
image/video-gen tools.

Regression test: a `.env` file is refused with the shared read-guard message
before any provider dispatch (mutation-verified).
2026-07-28 18:12:26 -07:00
dsad
37d0b6c81a fix(tts): block output to protected paths 2026-07-28 18:12:26 -07:00
Eugeniusz Gilewski
b76acacbb9 fix(tools): execute local STT templates without a shell
HERMES_LOCAL_STT_COMMAND rendered quoted placeholders into a
user-configured template and passed the result to shell=True. Shell
metacharacters in the template therefore remained executable syntax even
though the placeholder values themselves were quoted.

Tokenize the rendered template and invoke it as an argv list while
preserving the existing timeout, closed stdin, and Windows creation flags.
Lock the invocation contract with metacharacter regression coverage and
document explicit shell wrapping for trusted templates that need it.

Salvages #32694

Co-authored-by: Ernest Hysa <takis312@hotmail.com>
2026-07-28 18:12:26 -07:00
Sherman
273b986fd9 feat(tools): expand command TTS output_format allowlist (m4a/aac/amr/opus)
Command-type TTS providers validated output_format against a hardcoded
{mp3,wav,ogg,flac} set; any other value was silently coerced back to mp3,
which then mismatched the output path the post-run check expects. This
blocked common ffmpeg-producible containers/codecs — notably m4a (AAC),
the portable choice for WeChat/iOS/mobile voice files — with no
config-only path (only a local source patch, lost on every update).

Widen COMMAND_TTS_OUTPUT_FORMATS to add m4a, aac, amr, opus. This only
permits a command provider to declare these; the user's command still
produces the file (e.g. via ffmpeg). No built-in provider behavior
changes and no new required config.

Update the two tests that pinned the old set, and add a positive case
covering the new formats. Document the supported output_format values.
2026-07-28 18:12:26 -07:00
峯岸 亮
3ae25e0fbd fix(security): scrub credentials from voice playback subprocesses
## Summary
- Spawn system audio players (`ffplay` / `afplay` / `aplay`) with `hermes_subprocess_env(inherit_credentials=False)`.
- Prevent gateway tokens and provider API keys from leaking into OS media helpers.
- Add a regression test asserting scrubbed env on `Popen`.

## Salvage / credit
Sibling of #70342 / incomplete #56332 (TTS/STT command scrub) on the voice-mode playback path.
2026-07-28 18:12:26 -07:00
峯岸 亮
24a6fb6448 fix(security): scrub Hermes secrets from voice command subprocess env
Salvage incomplete #56332: route command TTS/STT through hermes_subprocess_env
while preserving delegated-child lineage, and close the sibling local-whisper
subprocess.run path that still inherited the full process environment.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-28 18:12:26 -07:00
CleanDev-Fix
4e8a66dace fix(tools): keep command TTS deadline through exit 2026-07-28 18:12:26 -07:00
CleanDev-Fix
1b97e3efc5 fix(tools): chunk command TTS stream reads 2026-07-28 18:12:26 -07:00
CleanDev-Fix
3884f078bd fix(tools): use idle timeout for command TTS 2026-07-28 18:12:26 -07:00
Teknium
7e7f7d3059
Merge pull request #70509 from NousResearch/hermes/hermes-29661bf6
feat(voice): on-device wake words with open-vocabulary phrases and multi-profile voice routing
2026-07-28 17:58:33 -07:00
Atakan
20de37d409 fix: reject ambiguous MCP tool name collisions 2026-07-28 15:02:52 -07:00
Atakan
9dcb44a219 fix(schema): preserve dependentRequired property names 2026-07-28 14:37:19 -07:00
Teknium
4f001742fd test(mcp): make discovery-lock tests cross-platform
CI (Linux) failed with ModuleNotFoundError: portalocker — it's a
win32-only dependency (concurrent-log-handler chain). Tests now lock
handles via a platform helper (fcntl on POSIX, portalocker on Windows),
mirroring production _try_acquire_mcp_discovery_lock.
2026-07-28 14:31:24 -07:00
Atakan
362fc21577 fix(mcp): guard discovery with cross-process lock 2026-07-28 14:31:24 -07:00
Atakan
56bda4529b fix(computer_use): revive ended cua-driver sessions once 2026-07-28 14:24:30 -07:00
atakan g
d98ea211f9 test: cover both multiplex URL policy orderings 2026-07-28 14:17:45 -07:00
atakan g
7b18de5f40 fix: scope private URL policy per profile 2026-07-28 14:17:45 -07:00
Teknium
f71d2d854f test(tts): cover microsecond default-output timestamp (salvaged #43911) 2026-07-28 14:07:21 -07:00
3ASiC
45f7030786 fix(tts): bind MiniMax credentials to their region endpoint 2026-07-28 14:07:21 -07:00
Solitud1nem
562c8b418f test(voice): drive the micro-pause test from the explicit clock too
Review follow-up: test_micro_pause_tolerance_during_speech was left on real
sleeps, so it kept the wall-clock dependency the rest of this PR removes.
Its three sleeps (50 ms, 50 ms, 60 ms) feed the same monotonic gate at
tools/voice_mode.py:561 that the other two tests were converted for.

It was measured rather than assumed before being left out: 0 failures in 40
runs, because on Windows sleep() rounds up to the timer tick, so sleep(0.05)
really costs ~62.5 ms and the three sleeps clear the 150 ms gate by ~37.5 ms
-- wider than the 15.625 ms GetTickCount64() quantum that made the other two
flake. That margin is an accident of sleep granularity rather than anything
the test guarantees, so it is not worth keeping the bet.

Driving it from fake_clock makes the timeline exact (0.05 + 0.05 + 0.06 =
0.16 against the 0.15 gate, dip 0.05 under the 0.1 tolerance) and drops the
real sleeping from the suite. Mutation-checked: widening the gate at :561
fails the test.
2026-07-28 14:07:21 -07:00
Solitud1nem
309ed7ec65 test(voice): drive silence detection tests with an explicit clock
test_silence_callback_fires_after_speech_then_silence and
test_custom_threshold_and_duration space their audio frames with
time.sleep(0.06) and check the result against 50 ms thresholds. That
leaves a 10 ms margin, which only survives if the clock is finer-grained
than the margin — and on Windows it isn't. time.monotonic() there is
GetTickCount64() with 15.625 ms resolution until CPython 3.13 moved it to
QueryPerformanceCounter(), so a sleep that really lasts 62.5 ms measures
as 46.9 ms often enough to matter, landing under the threshold. Depending
on which sleep got clipped, either the speech-confirm gate misses or the
silence timer never matures — which is why the same flake shows up on two
different asserts.

Both tests now advance a hand-driven clock instead of sleeping, so the
arithmetic is exact on every platform and neither test waits on real time.
The fixture patches the `time` name inside tools.voice_mode rather than
time.monotonic itself: voice_mode.time IS the stdlib module, so patching
through it would swap the clock out from under every other importer for
the duration of the test.

Measured on Windows 11 / Python 3.11.9, 20 runs of each test: 6/20 and
3/20 failed before, 0/30 after. The same machine on Python 3.13, where
monotonic() is QueryPerformanceCounter(), never failed either test — that
comparison is what pinned the clock resolution as the cause rather than
the detection logic. Linux CI never sees this: clock_gettime is
nanosecond-resolution there.
2026-07-28 14:07:21 -07:00
obelisk-complex
63be8ce863 fix(tests): stop the test suite speaking aloud and launching a browser
Running the suite could take over the machine it ran on. Two real
effects, both now closed:

- **The suite spoke through the speakers.** Once any test drove the
  `voice.toggle` RPC with `action="tts"`, the handler set
  `HERMES_VOICE_TTS=1` in the *live process environment*, and the flag
  outlived that test. Every later test that drove a turn to completion
  then fed its final response text to `hermes_cli.voice.speak_text` on a
  background thread - real synthesis, real playback, no API key needed
  (the default `edge` provider is keyless). A developer heard the fixture
  string "partial answer complete" out loud. Because the flag is set from
  inside the process, `scripts/run_tests.sh`'s `env -i` never protected
  against this.

  `tests/conftest.py` now blanks `HERMES_VOICE`/`HERMES_VOICE_TTS` per
  test, and a new autouse `_audio_playback_guard` stubs `speak_text` and
  its playback binding outright, so the speakers stay shut even inside the
  test that sets the flag itself. `@pytest.mark.real_audio_playback` opts
  out.

- **The suite launched Chrome.** `tests/tools/test_browser_supervisor.py`
  spawned a real browser on any machine with Chrome on `PATH`. Its
  docstring promised a `HERMES_E2E_BROWSER=1` gate that existed nowhere in
  the code. That gate is now real, and the file is marked `integration`
  so the default marker filter excludes it. `scripts/run_tests.sh`
  forwards `HERMES_E2E_BROWSER` so the documented manual run still works.

Miscellanea

- `tests/test_audio_playback_guard.py`: regression cover for both defences,
  driving the real `voice.toggle` handler rather than a stand-in.
2026-07-28 14:07:21 -07:00
Vivaan Dhawan
1c30c57f11 fix(whatsapp): preserve voice notes when STT fails 2026-07-28 14:06:56 -07:00
Florian Valade
afab7ed46e fix(photon): address review — U+FFFC before _record_last_inbound, MIME-based CAF promotion, add tests
- Move U+FFFC placeholder detection before _record_last_inbound() so the
  placeholder message id is not recorded as the reaction target
- Cancel pending U+FFFC tasks in disconnect() to prevent task leaks
- Check mimeType audio/x-caf in addition to filename for CAF→VOICE promotion,
  fixing unnamed attachments that default to '(unnamed)'
- Add 7 Photon adapter tests: CAF named/unnamed promotion, U+FFFC no-dispatch,
  U+FFFC not recorded as last inbound, U+FFFC+attachment cancel, timeout fire,
  disconnect cleanup
- Add 6 transcription tests: ffmpeg conversion, afconvert fallback, all-fail,
  CAF→WAV before Groq, conversion failure error, local provider skip
2026-07-28 14:06:56 -07:00
Teknium
f5ff8fa4f8
Merge remote-tracking branch 'origin/main' into wake-toggle-config 2026-07-28 13:24:23 -07:00
kshitij
64469bd2c8
test(tts): accept stream kwarg in the 4 stale xAI fake_post mocks (#73619)
_generate_xai_tts passes stream=True to requests.post since the TTS
speaker pipeline moved to the streaming core. Four fake_post mocks in
test_tts_xai_speech_tags.py still had the pre-streaming signature and
raised TypeError: unexpected keyword argument 'stream'. The other 20+
mocks in the same file already accept stream=False; this aligns the
stragglers.

Fixes the 4 failures in Python tests slice 5/8 on main.
2026-07-28 12:49:23 -07:00
Teknium
0cf58de85e
Merge remote-tracking branch 'origin/main' into wake-toggle-config
# Conflicts:
#	tests/test_tui_gateway_server.py
#	tui_gateway/server.py
2026-07-28 12:37:35 -07:00
Teknium
e04c2a9ebd test: update voice-submission test for _VoiceInputMessage sentinel (#65827) 2026-07-28 11:57:37 -07:00
Teknium
e6b0344052 test: adapt salvaged voice tests to current main + lint fix
- _FakeProc gains returncode (main's player loop checks proc.returncode)
- WSL gate tests clear SSH_* env vars (main hard-warns over SSH without
  forwarded audio) and accept the merged #37346 forwarded-sound-server
  notice wording
- test_tts_macos_output stubs resolve_streaming_provider so the
  OutputStream setup path actually runs on main's chunked-streamer code
- voice CLI integration tests unwrap the _VoiceInputMessage sentinel
- _is_wsl: explicit encoding + drop unreachable return (ruff PLW1514)
2026-07-28 11:57:37 -07:00
Teknium
fe3dc29009 fix(voice): honor quoted beep_enabled strings + correct PortAudio OSError hint
Two in-house micro-fixes from issue triage:

- #49883: voice.beep_enabled gates in cli.py and hermes_cli/voice.py used
  bool() on the config value, so a quoted YAML string like "false" or
  "off" kept beeps on. Route through utils.is_truthy_value.
- #18432: AudioRecorder.start() collapsed OSError from _import_audio into
  the 'pip install sounddevice numpy' hint — but OSError means the
  PortAudio SHARED LIBRARY is missing, which pip cannot fix. Mirror
  detect_audio_environment's system-package hint (libportaudio2 /
  brew portaudio / Termux pkg install portaudio) on that path.

Fixes #49883
Fixes #18432
2026-07-28 11:57:37 -07:00
Dean Chen
3524b20728 fix(cli): surface local STT model preparation 2026-07-28 11:57:37 -07:00
simonmmafs
a1bc12f191 voice: one macOS output policy across WAV, beeps, streaming TTS + tests
Addresses review on #62601. Applies a single rule — no sounddevice for
audio OUTPUT on macOS (PortAudio/CoreAudio init triggers a
kTCCServiceMediaLibrary prompt) — consistently at all three output sites:

- play_audio_file: WAV playback (already routed to afplay) now uses the
  shared _sounddevice_output_allowed() helper.
- play_beep: synthesize the tone with numpy only, then on macOS play it
  via a temp WAV through afplay instead of sounddevice.
- stream_tts_to_speaker (tts_tool): on macOS, skip the sounddevice
  OutputStream so playback falls through to the existing tempfile/afplay path.

Audio INPUT (recording) is untouched — it legitimately needs mic permission.

Tests: TestMacOSAudioOutputPolicy (voice_mode) proves WAV + beep routing
does not import sounddevice on Darwin and still uses it off Darwin;
test_tts_macos_output proves streaming TTS skips the OutputStream on Darwin.
Existing test_play_wav_via_sounddevice pinned to non-Darwin for determinism.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 11:57:37 -07:00
xxxigm
0a5c5519fc test(voice-mode): pin Termux:API detection probe ladder + #31015 fallback
Two test classes lock in the new detection contract:

1. TestTermuxApiAppInstalledProbeLadder — drives _termux_api_app_installed
   with a fake subprocess.run dispatcher and walks each rung of the
   ladder:
     - non-Termux env returns False (no probes run),
     - pm-confirms returns True (back-compat),
     - pm-clean-miss → cmd-confirms returns True,
     - pm-FileNotFoundError → cmd-confirms returns True,
     - pm-TimeoutExpired → cmd-confirms returns True,
     - pm-nonzero-exit → cmd-confirms returns True,
     - both probes inconclusive + binary on PATH returns True (the
       core #31015 case),
     - both probes inconclusive + no binary returns False,
     - both probes clean-miss returns False (the genuine "CLI without
       app" case keeps the existing warning),
     - case-insensitive package match for ROMs that capitalise differently.

2. TestDetectAudioEnvironmentTermuxFallback — end-to-end through
   detect_audio_environment, asserting the misleading "Termux:API
   Android app is not installed" warning no longer fires when probes
   are inconclusive but the binary is on PATH (the user-reported #31015
   symptom), AND that the warning still fires when probes can
   conclusively report the app is missing.

Refs: NousResearch/hermes-agent#31015
2026-07-28 11:57:37 -07:00
ygd58
0560f52047 fix(voice): add WSL2 PowerShell audio fallback for TTS playback
Ports #63768 forward onto current main per teknium1's review.

On WSL2 without a PulseAudio bridge, ffplay and aplay have no audio
device and TTS playback silently fails (issue #17608). When
powershell.exe and ffmpeg are available, convert the audio to a
uniquely-named WAV in the Windows %TEMP% directory and play it via
Media.SoundPlayer.

Per review, this fixes two gaps in the original port:

1. Exit-status masking: the cleanup subshell was
   '( ffmpeg && powershell ); rm -f wav' -- the shell's exit status is
   the LAST command's (rm -f, which is always 0), so a real
   ffmpeg/PowerShell failure could never be detected by the rc-checking
   fallback logic added to the player loop. Now captures the real
   status before cleanup and re-exits with it:
   '( ffmpeg && powershell ); rc=0; rm -f wav; exit '.

2. The no-Pulse WSL gate in detect_audio_environment() still hard-blocked
   voice mode entirely (input AND output) even when the PowerShell
   fallback made TTS output viable. Added _wsl_powershell_tts_available()
   and use it to downgrade the WSL-without-Pulse case from a hard
   'warnings' block to a non-blocking 'notices' entry when the fallback
   is available -- the same PulseAudio-bridge recording guidance is still
   surfaced (mic capture genuinely still needs it), it just no longer
   blocks /voice on for TTS-only usage. cli.py's existing
   env_check['available'] gate needed no changes since it already
   respects this flag.

Also fixed the flaky uniqueness test (the original asserted
len(filenames) >= 2, which passed trivially on zero captured
filenames) and added a real fallback-triggering regression test for
the exit-status fix.

10 new/fixed tests pass in TestWSL2PowerShellFallback and the new
TestWSLAudioEnvironmentGate; 80/80 in the full tests/tools/test_voice_mode.py file.
2026-07-28 11:57:37 -07:00
Slippy87
ef686c3878 fix(voice): honor forwarded audio (PIPEWIRE_REMOTE) in WSL detection 2026-07-28 11:57:37 -07:00
Brice
2a75664c0c fix(voice): capture at the input device's native sample rate
AudioRecorder hard-codes SAMPLE_RATE (16 kHz) when opening the input
stream, but some capture devices (e.g. USB microphones exposed through
ALSA hw) reject 16 kHz outright — sd.InputStream fails with
PaErrorCode -9997 (Invalid sample rate) and voice recording is broken.

Query the default input device for its native default_samplerate at
recording start and open the stream / write the WAV at that rate,
falling back to the Whisper-friendly 16 kHz constant when the backend
does not expose a usable rate. STT providers accept standard WAV rates,
so downstream transcription is unaffected.
2026-07-28 11:57:37 -07:00