feat(voice): calm ambient "thinking" sound while the agent works in voice chat

Long thinking/tool stretches in a voice conversation are dead air — the
user cannot tell whether the agent is alive. New: quiet, repeating soft
bubble blips while the agent works and no speech audio is flowing.

- tools/voice_mode.py: numpy-synthesized blips (no binary assets) — two
  alternating low pitches (G4/E4) with pitch glide + smooth attack/decay
  envelopes, ~0.8-1.2s randomized spacing, volume = voice.beep_volume * 0.5.
  start_thinking_sound(should_play=...) / stop_thinking_sound() daemon-loop
  lifecycle; macOS-TCC-safe (sounddevice output gated there → silent skip,
  no per-second afplay churn). New mark_audio_output_active()/
  is_audio_output_active() ref-count wraps play_audio_file and the
  streaming OutputStream sentence writes so "audio is flowing" is accurate.
- Config: voice.thinking_sound (default true) off-switch.
- cli.py: starts when a voice-mode turn begins, per-blip gate skips while
  TTS speaks / mic records / barge capture owns the mic; stopped in the
  chat() finally on every exit path.
- tui_gateway/server.py: same lifecycle around _run_prompt_submit turns
  (voice mode on), gated on is_audio_output_active + continuous capture.
- Desktop: renderer owns voice-conversation audio, so a matching WebAudio
  implementation (src/lib/thinking-sound.ts, same envelope/pitches) runs
  while conversation status === "thinking"; honors voice.thinking_sound
  (via config store) and the shared sound-mute toggle; stops instantly on
  speaking/listening/end.
This commit is contained in:
Teknium 2026-07-29 00:50:44 -07:00
parent 6fdfdc1597
commit df093bf33c
12 changed files with 690 additions and 5 deletions

30
cli.py
View file

@ -13350,6 +13350,7 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
# chunks as they arrive, everything else synthesizes per sentence.
use_streaming_tts = False
_streaming_box_opened = False
_thinking_started = False
text_queue = None
tts_thread = None
stream_callback = None
@ -13554,6 +13555,27 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
agent_thread = threading.Thread(target=run_agent, daemon=True)
agent_thread.start()
# Ambient "thinking" sound: calm bubble blips while the agent
# works in voice mode with no audio flowing, so the user knows
# it's alive during long thinking/tool stretches. Skipped per-blip
# while TTS speaks, the mic records, or a barge capture is live;
# stopped outright as soon as the turn ends. voice.thinking_sound
# gates it (default on); macOS is handled inside (TCC-safe skip).
_thinking_started = False
if self._voice_mode:
try:
from tools.voice_mode import start_thinking_sound
_thinking_started = start_thinking_sound(
should_play=lambda: (
self._voice_tts_done.is_set()
and not self._voice_recording
and not self._voice_barge_capture.is_set()
)
)
except Exception:
_thinking_started = False
# Monitor the dedicated interrupt queue while the agent runs.
# _interrupt_queue is separate from _pending_input, so process_loop
# and chat() never compete for the same queue.
@ -13960,6 +13982,14 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
print(f"Error: {e}")
return None
finally:
# Stop the ambient thinking sound the moment the turn ends —
# every exit path (normal, error, interrupt) lands here.
if _thinking_started:
try:
from tools.voice_mode import stop_thinking_sound
stop_thinking_sound()
except Exception:
pass
# Ensure streaming TTS resources are cleaned up even on error.
# Normal path sends the sentinel at line ~3568; this is a safety
# net for exception paths that skip it. Duplicate sentinels are