mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-23 16:36:23 +00:00
docs(voice): streaming TTS + barge-in across surfaces
This commit is contained in:
parent
93e9061f15
commit
83456d0bc6
1 changed files with 12 additions and 3 deletions
|
|
@ -159,11 +159,20 @@ Both `silence_threshold` and `silence_duration` are configurable in `config.yaml
|
|||
|
||||
### Streaming TTS
|
||||
|
||||
When TTS is enabled, the agent speaks its reply **sentence-by-sentence** as it generates text — you don't wait for the full response:
|
||||
When TTS is enabled, the agent speaks its reply **sentence-by-sentence** as it generates text — you don't wait for the full response. This works with **every TTS provider**:
|
||||
|
||||
1. Buffers text deltas into complete sentences (min 20 chars)
|
||||
2. Strips markdown formatting and `<think>` blocks
|
||||
3. Generates and plays audio per sentence in real-time
|
||||
2. Strips markdown formatting, emoji, and `<think>` blocks
|
||||
3. Plays audio per sentence in real-time — providers with a chunked PCM API (ElevenLabs, OpenAI) stream raw audio for the lowest time-to-first-word; every other provider (including the default Edge) synthesizes and plays each sentence as it completes
|
||||
|
||||
The same pipeline runs in the classic CLI, the TUI, and the desktop app. In a desktop voice conversation the reply text is fed **live** into a per-reply speech WebSocket as the model generates it, so speech overlaps generation — one socket and one audio clock per reply, no per-sentence connection gaps.
|
||||
|
||||
### Barge-in
|
||||
|
||||
You can interrupt the agent mid-speech:
|
||||
|
||||
- **Talk over it** — in continuous voice mode, a voice-activity monitor listens while the agent speaks and cuts playback the moment you start talking, then goes straight back to recording. The detector calibrates its noise floor against the playback itself, so speaker bleed doesn't self-trigger. Disable with `voice.barge_in: false` in `config.yaml`.
|
||||
- **Type or press the record key** — sending a new message or hitting the push-to-talk key stops playback instantly on every surface.
|
||||
|
||||
### Hallucination Filter
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue