From 83456d0bc6e6fb433b16b8b980e1b54df5d8063b Mon Sep 17 00:00:00 2001 From: Brooklyn Nicholson Date: Wed, 22 Jul 2026 17:47:25 -0500 Subject: [PATCH] docs(voice): streaming TTS + barge-in across surfaces --- website/docs/user-guide/features/voice-mode.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/website/docs/user-guide/features/voice-mode.md b/website/docs/user-guide/features/voice-mode.md index 14a4235e3e7c..d0945478dac6 100644 --- a/website/docs/user-guide/features/voice-mode.md +++ b/website/docs/user-guide/features/voice-mode.md @@ -159,11 +159,20 @@ Both `silence_threshold` and `silence_duration` are configurable in `config.yaml ### Streaming TTS -When TTS is enabled, the agent speaks its reply **sentence-by-sentence** as it generates text — you don't wait for the full response: +When TTS is enabled, the agent speaks its reply **sentence-by-sentence** as it generates text — you don't wait for the full response. This works with **every TTS provider**: 1. Buffers text deltas into complete sentences (min 20 chars) -2. Strips markdown formatting and `` blocks -3. Generates and plays audio per sentence in real-time +2. Strips markdown formatting, emoji, and `` blocks +3. Plays audio per sentence in real-time — providers with a chunked PCM API (ElevenLabs, OpenAI) stream raw audio for the lowest time-to-first-word; every other provider (including the default Edge) synthesizes and plays each sentence as it completes + +The same pipeline runs in the classic CLI, the TUI, and the desktop app. In a desktop voice conversation the reply text is fed **live** into a per-reply speech WebSocket as the model generates it, so speech overlaps generation — one socket and one audio clock per reply, no per-sentence connection gaps. + +### Barge-in + +You can interrupt the agent mid-speech: + +- **Talk over it** — in continuous voice mode, a voice-activity monitor listens while the agent speaks and cuts playback the moment you start talking, then goes straight back to recording. The detector calibrates its noise floor against the playback itself, so speaker bleed doesn't self-trigger. Disable with `voice.barge_in: false` in `config.yaml`. +- **Type or press the record key** — sending a new message or hitting the push-to-talk key stops playback instantly on every surface. ### Hallucination Filter