Follow-up integration for the #47588 salvage, aligning the new streamers
with the post-campaign invariants:
- All streaming key lookups go through _resolve_key -> tts_tool.
_resolve_provider_key -> resolve_provider_secret (config > env/.env >
credential pool, profile-scoped) — never bare get_env_value. xAI
resolves via resolve_xai_http_credentials so OAuth users stream too.
- _capped(): every provider's chunk iterator is bounded at 16 MiB per
sentence, mirroring _read_tts_response_bytes' bounded-upstream-body
invariant on the sync paths.
- Tests updated for the resolver contract + new coverage for credential
routing and the cap.
Salvaged from PR #47588 and rebased onto the post-campaign streaming core:
the StreamingTTSProvider ABC/registry and the ElevenLabs/OpenAI streamers
already live on main (tools/tts_streaming.py), so this ports the pieces
main lacked:
- GeminiStreamer: streamGenerateContent?alt=sse -> base64 PCM chunks
(24 kHz mono int16), reusing main's DEFAULT_GEMINI_TTS_* constants.
- XAIStreamer: WebSocket wss://api.x.ai/v1/tts -> binary PCM frames,
async->sync bridged via the _collect_async test seam.
- tts.streaming.provider config knob: pin one streamer, or 'auto' to
walk the priority list elevenlabs -> gemini -> openai -> xai. Unset
keeps the never-swap-the-user's-voice default.
- docs/streaming-tts.md: architecture, capability matrix, how to add
a provider.
- Unit tests for the knob, SSE parsing, and the WS bridge; key-gated
E2E tests (skipped without credentials).
Refs: #47588
Salvaged from PR #66311 (@moeadham), rebased onto the current streaming
registry. tts.elevenlabs.base_url (+ optional wss_url, derived from
base_url when omitted) routes both the sync ElevenLabs path and the
chunked ElevenLabsStreamer through an ElevenLabsEnvironment, matching
the STT side's ELEVENLABS_STT_BASE_URL/config override pattern.
Follow-up to salvaged PR #70307 (@aml1973): OpenAIStreamer now checks
tts.openai.api_key (config.yaml) ahead of the env resolver in both
available() and stream(), completing config parity between the sync
and streaming OpenAI TTS paths.
Use the shared OpenAI audio key resolver and prefer tts.openai.base_url over the global environment fallback in the Desktop streaming path. Add focused regression coverage for credential and endpoint propagation.
mark_speech_interrupted() / take_speech_interrupted(): a one-shot,
TTL'd (120s) latch plus SPEECH_INTERRUPTED_NOTE. Barge-in paths mark
it when they cut live speech; the next turn's submit path pops it and
prepends the note to the model-bound message — API-call local, never
persisted, so history and prompt caching are untouched.
StreamingTTSProvider ABC + registry + resolve_streaming_provider() —
ElevenLabs (pcm_24000) and OpenAI (response_format=pcm) stream chunked
PCM; every other provider keeps its configured voice and falls back to
per-sentence sync synthesis. SentenceChunker is the one incremental
cutter every surface shares: sentence-boundary cuts on the delta
stream, <think> blocks stripped even when split across deltas, short
fragments merged forward so they never stall as tiny clips.
