Replace 20 lines of manual os.open/O_EXCL/fdopen/fsync/os.replace with the
existing atomic_json_write() from utils.py, which is already used by 6+
modules and handles temp-file creation, fsync, atomic replace, mode
control, and owner preservation. The only novel helper (_fsync_directory)
is retained — atomic_json_write does not do directory fsync.
Update test_flush_write_failure_leaves_no_recovery_file to monkeypatch
utils.os.replace (the new call path) instead of gateway.shutdown_flush.os.replace.
Same class of bug as the LINE adapter (NS-603): defaulting the webhook
bind to "0.0.0.0" (or hardcoding it) binds IPv4 ONLY, so the listener
is unreachable over IPv6-only private networks such as Fly.io 6PN.
- wecom callback_adapter: DEFAULT_HOST None; config.py env seed no
longer forces 0.0.0.0 when WECOM_CALLBACK_HOST is unset.
- msgraph_webhook: DEFAULT_HOST None; the allowed_source_cidrs
requirement still fires for the all-interfaces default (host=None is
treated as network-accessible).
- whatsapp_cloud: DEFAULT_WEBHOOK_HOST None.
- teams: hardcoded 0.0.0.0 TCPSite bind → _DEFAULT_HOST=None with new
TEAMS_HOST / extra.host override (mirrors LINE_HOST pattern).
- telegram: hardcoded listen="0.0.0.0" → default "" (tornado
bind_sockets opens one socket per address family; verified against
PTB 22.6/tornado) with new TELEGRAM_WEBHOOK_HOST / extra.webhook_host
override.
Explicit host overrides everywhere are preserved; empty/unset collapses
to the dual-stack default. "::" remains a bad substitute on
bindv6only=1 hosts (see LINE adapter comment).
The ownership guard let a sentinel with pid=None pass as self-owned, so
an exiting life could clobber evidence of unknown provenance with a
clean-exit claim. Tighten: only rewrite when the sentinel pid matches
os.getpid() exactly; pid=None (or malformed) is left untouched. Adds
tests for the pid=None no-op and the own-pid rewrite paths.
scripts/check-windows-footguns.py (blocking CI lint) rightly flagged the
os.kill(pid, 0) liveness probe: on Windows sig=0 collides with
CTRL_C_EVENT and GenerateConsoleCtrlEvent hard-kills the target's whole
console group (bpo-14484) — a forensics module must never be able to
kill the process it's checking on. Route through gateway.status._pid_exists,
the repo's canonical psutil-backed no-kill probe.
Hosted agents that die uncleanly (kernel OOM kill, SIGKILL, whole-VM
death) leave no trace: shutdown_forensics only covers graceful signals,
gateway-exit-diag.log only covers exit paths that actually run, and the
VM reboot wipes dmesg before anyone can capture it. NS-608 (BlueAtlas
hourly crash cycle, July 12-15) took days of manual log correlation to
classify because nothing recorded 'the previous life ended violently'.
Add gateway/lifecycle_ledger.py — a sentinel state machine persisted to
<HERMES_HOME>/state/gateway.lifecycle.json:
- start_gateway() claims the sentinel (phase=running) right after the
PID-file/runtime-lock claim, and reports any prior life that never
reached an exit path as gateway.previous_unclean_exit in
gateway-exit-diag.log + a WARNING log line.
- Every exit funnel marks the sentinel exited with a reason:
_exit_after_graceful_shutdown (graceful_shutdown), the shutdown
watchdog (shutdown_watchdog), and the loop-liveness watchdog
(loop_liveness_watchdog).
- Ownership-guarded for --replace takeovers: a live matching owner is
never reported dead, and the old life cannot clobber the
replacement's freshly claimed sentinel on its way out.
The 30s loop heartbeat now embeds a cheap /proc memory sample (own RSS,
MemAvailable, swap used) so every unclean-death report carries a
'memory N seconds before death' snapshot; the detector flags
suspected_oom when the last sample shows <64MiB or <5% available.
container-boot.log lines gain prior_exit=clean|unclean|unknown per
profile, stamping unclean container deaths into the volume-persisted
boot log where support can grep for them.
Tests: tests/gateway/test_lifecycle_ledger.py (16 cases) + 4 new
container-boot annotation cases. Existing watchdog/forensics/boot
suites all green; ruff clean.
Add an opt-in streaming-audio adapter seam to BasePlatformAdapter so
voice-capable gateway platforms (LiveKit, Discord voice, future adapters)
can consume LLM output as streaming PCM audio before the full response
completes, dropping perceived voice latency from ~2-3.5s to ~500-800ms.
Adapter contract (gateway/platforms/base.py):
- AudioFormat dataclass: declared sample_rate, channels, sample_width
- StreamingTTSHandle: opaque handle with audible/aborted flags
- supports_streaming_tts / begin_streaming_tts / write_streaming_tts
/ finish_streaming_tts / abort_streaming_tts
- All default to unsupported/no-op so existing adapters are source-compatible
- Per-turn _streaming_tts_completed_chats set suppresses duplicate whole-file
auto-TTS when streaming succeeded; cleared after turn completion
Gateway consumer (gateway/streaming_tts_consumer.py):
- StreamingTTSConsumer: bridges sync agent deltas to async adapter audio sink
- Uses existing SentenceChunker (no competing parser)
- Thread-safe bounded queue; on_delta never blocks the agent worker thread
- Resolves configured streaming provider via resolve_streaming_provider()
- Serialises clause playback in order; flushes tail on completion
- Pre-audio failure: completed=False (falls back to whole-file TTS)
- Post-audio failure: completed=True, partial=True (no replay from start)
- Abort is idempotent; late chunks silently dropped
- Per-turn state isolated across concurrent chats
Gateway integration (gateway/run.py):
- message_type parameter threaded through _run_agent -> _run_agent_inner
- StreamingTTSConsumer created when voice input + auto-TTS + provider active
- Delta callback teed to both text stream consumer and TTS consumer
- TTS-only delta callback installed when text streaming is off
- finish() called from executor; wait_complete() in async context after
- Barge-in aborts the consumer at all three interrupt detection points
- Runner-level _send_voice_reply suppressed when streaming TTS completed
Tests (tests/gateway/test_streaming_tts_consumer.py):
- 15 focused tests: adapter defaults, lifecycle, ordered chunks,
unsupported/No-streamer fallback, abort idempotency, late-chunk drop,
pre/post-audio failure, concurrent-turn isolation, think-block suppression,
queue backpressure
Does not touch desktop/TUI code or add config flags. Plugin TTS provider
stream() metadata gap (#47896) is explicitly out of scope — built-in
ElevenLabs/OpenAI PCM streamers are the first consumers.
Refs: #60671, #47896
Sibling of #64686: _run_agent's empty-final_response early return dropped
failure_reason (and #64686 only fixed the non-empty path), so downstream
consumers (TUI billing surface, transient-failure persistence) lost the
structured reason exactly when a failed run produced no text.
Also hardens the two BasePlatformAdapter identity checks (edit_message /
delete_message) with getattr so duck-typed adapters without the attribute
mean 'capability absent', not AttributeError — this was crashing the
send_progress_messages path for minimal adapters and test fakes.
Follow-up to salvaged PR #56473: dedupe the five cleanup_*_cache bodies
into a shared _cleanup_cache_dir() helper, add cleanup_video_cache() and
cleanup_screenshot_cache() (with get_screenshot_cache_dir()), and drive
all five from a single (name, fn) loop in _start_gateway_housekeeping()
instead of one copy-pasted try/except per cache. Covers the video/
screenshot half of #56427.
Adds cleanup_audio_cache() to gateway/platforms/base.py and wires it into
_start_gateway_housekeeping() in gateway/run.py, matching the existing
image/document cache cleanup pattern.
Re-implements the fix proposed in #16473, which targeted the now-deprecated
_start_cron_ticker() and has since stalled with a merge conflict against
main. _start_gateway_housekeeping()'s docstring already claimed audio
cleanup ("prunes the image/audio/document cache"); this makes the code
match it.
Co-authored-by: nftpoetrist <264138787+nftpoetrist@users.noreply.github.com>
When WeChat (Weixin) returns a voice_item.text (Tencent Cloud's STT),
Hermes previously trusted that text as the user-visible message body and
skipped downloading the raw audio. For non-Chinese audio that text is
garbage — the original report was a Russian voice message that came
back as English phonemes — and the user sees nonsense as their own
message. International users on the WeChat gateway effectively can't
use voice.
Two short-circuits in gateway/platforms/weixin.py caused this:
- _download_voice() returned None whenever voice_item.text was set,
so the raw SILK/Opus audio was never fetched.
- _extract_text() returned voice_item.text verbatim as the body,
so even if the audio had been downloaded, the central STT
pipeline in gateway/run.py never had a chance to replace it.
Fix both: always download the raw audio (the central pipeline picks
it up via _collect_media()), and skip voice items in _extract_text()
so the body comes from Hermes' own mlx-whisper / whisper.cpp /
faster-whisper transcription instead of Tencent's. Behavior is
unchanged when voice_item.text is absent (the original happy path
where audio was already being downloaded).
Tests:
- 5 new tests in TestWeixinVoiceAlwaysDownloaded covering both
functions, the _collect_media integration path, and a
regression guard for the text-item path.
- 74/74 in tests/gateway/test_weixin.py pass.
Minimal fix: add 'if ct == "file": return False' before extension
matching. The original fallback logic is preserved for empty/unknown
content_types. Only the bug case (file uploads with audio extensions)
is fixed.
Removed the over-engineered _looks_like_voice helper.
Refined the fix: instead of removing extension-based fallback entirely,
only skip it when content_type is explicitly 'file' (or image/video).
Empty or unknown content_types still fall back to extension matching
as a defensive measure.
- content_type='voice' or 'audio/*' → True (API signal)
- content_type='file' → False (file transfer, never voice)
- content_type='' → extension fallback (defensive)
- content_type=unknown → extension fallback (defensive)
Added _looks_like_voice() module-level helper and comprehensive tests.
The _is_voice_content_type() heuristic matched audio file extensions
(.wav, .mp3, .ogg, etc.) even when the QQ Bot API explicitly reported
content_type='file'. This caused files sent via QQ's file-transfer
feature to be routed through the speech-to-text pipeline instead of
being saved as regular attachments.
The QQ Bot API already distinguishes voice messages (content_type=
'voice') from file uploads (content_type='file'), so filename-based
extension sniffing is unnecessary and harmful.
Removed the _VOICE_EXTENSIONS fallback; now only content_type is
checked.
Closes #XXXX
Trimmed cherry-pick of PR #40592 (duration + dedup hunks only; the
voice-classification hunk duplicates #29235 and the send_voice Opus
rewrite is out of scope for this inbound-focused PR):
- adapter.py: ffprobe duration (off-loop) attached to Feishu voice
uploads via _build_file_upload_body(duration=...) and the audio
message payload (#16524, #8300)
- gateway/run.py: TTS dedup narrowed to the current turn;
_enrich_message_with_transcription dedups repeated audio paths
Refs #40592#16524#8300
Two log-spam bugs found in live gateway logs:
1. gateway_routing UNIQUE-constraint spam (261 warnings in one errors.log):
early builds of the #59203 routing-index migration created
gateway_routing with 'session_key TEXT PRIMARY KEY' and no scope
column. _reconcile_columns() ADDs the missing scope column but SQLite
cannot ALTER a primary key, so the shipped composite
PRIMARY KEY (scope, session_key) never lands on those databases. Both
write paths then fail on every save:
- save_gateway_routing_entry: 'ON CONFLICT clause does not match any
PRIMARY KEY or UNIQUE constraint'
- replace_gateway_routing_entries: 'UNIQUE constraint failed:
gateway_routing.session_key' whenever the same session_key exists
under another scope (e.g. test-suite scopes leaked into a live DB).
New _heal_gateway_routing_pk() rebuilds the table once with the
composite key, preserving rows (newest wins on collisions, NULL scope
coalesced to ''). Same one-time-heal pattern as the #51646 active-
column repair. Verified E2E against a copy of a real affected state.db.
2. pre_gateway_dispatch warned ''GatewayRunner' object has no attribute
'session_store'' and silently dropped the hook for every message on
partially-initialized runners (bare object.__new__ runners in tests,
and any future init-order change). Pass
getattr(self, 'session_store', None) so the hook always fires
(pitfall #17 pattern).
Both regression tests fail without their fixes (sabotage-verified).
Follow-up for salvaged #65023/#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from #67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of #58780. Adds extract_media gating tests for #44826 and the
contributor mapping for chefboyrdave21.
_invalidate_pending_stt_cache() clears the gateway-side transcription
cache when merge_pending_message_event() folds a follow-up message into a
still-pending event, so the next transcription picks up the merged text
and attachments. It also cleared _gateway_pending_stt_echo_sent, but that
flag is not derived state — it records that the transcript was already
delivered to the user.
Dropping it makes the re-run transcription echo the earlier notes a second
time. Both merge branches are affected, including the text-only follow-up
case where no new audio arrived at all: there the cache is invalidated,
the same voice note is transcribed again (a second paid STT call) and the
same line is echoed again.
Sequence:
1. voice note arrives, interrupt monitor transcribes it and echoes
'🎙️ "hello"'
2. user sends a follow-up while the turn is still pending, so it merges
3. drain path re-transcribes and echoes '🎙️ "hello"' a second time
Keep the ledger out of the invalidation set and track it as a count of
already-echoed transcripts instead of a single boolean. A count is what
the merge case actually needs: re-running transcription over the extended
media list returns the earlier transcripts as a prefix of the new one, so
echoing only the unsent tail suppresses the repeat while still surfacing a
newly merged voice note. A count rather than a set of seen values, so two
separate notes that transcribe identically stay two distinct deliveries —
covered by test_pending_stt_merge_echoes_two_identical_transcripts.
The guard stays within the 12-line window that
test_all_gateway_transcript_echo_sends_are_gated enforces over run.py.
[[audio_as_voice]] is message-global but was applied to every media file in a
message. A non-audio file flagged is_voice is excluded from the embedded-photo
batch and falls through to send_document, so an image in a message that also
carries a voice note arrives as a file attachment instead of an inline photo.
Gate the voice flag on the file extension so one message can carry an inline
image AND a voice bubble. Also wrap the path append in try/except so a crafted
~\x00 path is skipped rather than aborting extraction of all attachments.
Two hand-written fixes in the voice input path:
- _handle_voice_channel_input now resolves the bound text channel's
channel_prompt via the adapter's _resolve_channel_prompt so voice input
gets the same per-channel context as typed messages (fixes#50149).
- _enrich_message_with_transcription now guards success=True results whose
transcript is empty/whitespace-only (silence, cut-off, inaudible audio):
instead of emitting empty quotes the agent gets a clear sentinel note.
Reimplemented against the current plain-quoted note wording; original
concept and tests by @deacon-botdoctor in PR #41603 (fixes#41603).
- Wire adapter._voice_input_callback at connect and reconnect so voice
transcription is forwarded without requiring /voice join (#60623).
- Add optional text_channel_id and source params to DiscordAdapter
.join_voice_channel() so automatic/programmatic voice joins can
establish the text-channel binding needed by _handle_voice_channel_input.
- Add TestVoiceInputCallbackWiring: asserts callback wiring on startup
and reconnect for Discord adapters with voice attributes.
_voice_input_callback was only set in _handle_voice_channel_join, not at adapter connect or reconnect. Voice transcription was logged but never forwarded as an inbound message without explicit /voice join.
Wire the callback at both connect and reconnect paths.
Fixes#60623
Salvaged from PR #62040 (@giladbau), simplified per post-#73072 main: the
central _repair_ogg_container transcode makes an explicit .ogg output path
sufficient — no target_platform plumbing through the TTS tool needed.
Root cause (class-level): both gateway auto-TTS delivery call sites relied
on the TTS tool reading HERMES_SESSION_PLATFORM to pick Ogg/Opus vs MP3,
but that contextvar is cleared by _clear_session_env before the base
adapter's post-handler auto-TTS block runs, so want_opus was always False
on that path → MP3 → Telegram sent an audio attachment instead of a native
voice bubble (#57049, #36685). The runner's _send_voice_reply had the
sibling bug: it hardcoded .ogg for Telegram only, leaving Matrix and
Feishu runner voice replies as MP3 (#14841, #45557).
Fix: new build_auto_tts_output_path(platform) in gateway/platforms/base.py
hands an explicit .ogg temp path when the platform is in the TTS tool's
OPUS_VOICE_PLATFORMS set (single source of truth — telegram/matrix/feishu/
whatsapp/signal), .mp3 otherwise. Used by BOTH delivery call sites:
- BasePlatformAdapter auto-TTS block (also honors the tool's success flag
and cleans up requested + returned paths)
- GatewayRunner._send_voice_reply (replaces the telegram-only ternary)
Fixes#57049Fixes#36685
Refs #14841#45557
Salvaged from PR #51196 (@55nx954gn6-debug). _should_send_voice_reply only
consulted the runner's _voice_mode dict (/voice on|voice_only|all), so the
global voice.auto_tts config default — which is synced into each adapter's
_auto_tts_default on gateway connect — was invisible to the runner path.
Net effect: with streaming enabled and only global auto-TTS configured
(no per-chat /voice opt-in), the streamed reply consumed the text, the
base adapter's auto-TTS got text_content=None, and no voice reply was
ever sent (#51867/#23983 remainder).
The runner now also asks the adapter's _should_auto_tts_for_chat(chat_id)
(which encodes per-chat /voice on|off overrides over the global default);
an explicit /voice off chat mode remains a hard override.
Refs #51867#23983#51282#13126
When /reasoning show is enabled, the model's <think> blocks appear in
the final assistant message. TTS reads these aloud, which is unwanted —
users want to see reasoning but not hear it spoken.
- Add _THINK_BLOCK regex to _strip_markdown_for_tts() (tools/tts_tool.py)
— the general TTS text-preparation path used by all TTS providers
- Add <think> stripping to prepare_tts_text() (gateway/platforms/base.py)
— the gateway auto-TTS path
- Reuses the existing regex pattern from stream_tts_to_speaker()
Closes#34213
Review follow-up: the spoken script is for synthesis only. Caption
eligibility and payload stay on the original reply, so a long reply
whose normalized script fits the 1024 char limit is still delivered
in full as its own message. Adds the long-original/short-normalized
regression case to the auto-TTS caption tests.
Auto-TTS previously fed raw chat Markdown and compact symbols straight to the
speech provider, so units were read as stray letters and headings or bullets ran
together. This routes spoken text through a new normalizer,
tools/tts_text_normalize.prepare_spoken_text, that expands units (for example a
temperature written with the degree symbol becomes "degrees Celsius") and
flattens Markdown into a transcript-like script with sentence pauses.
The normalizer is best-effort: if it ever fails the code falls back to the
previous markdown-strip behavior, so auto-TTS keeps working. The Telegram voice
caption uses the same normalized text. Includes a unit test.
One sniffer owns magic-byte container detection (Teknium's one-concept-
one-owner rule): the new tools/audio_container.py is used by
- gateway/platforms/base.py _sniff_audio_ext (inbound cache — PR #36166's
central sniffer, now covering AAC/ADTS, MP4-brand disambiguation, webm)
- gateway/platforms/signal.py _guess_extension (audio/AV branches
delegated; RIFF/WAVE fix from PR #50690 and M4A-brand fix from
PR #72490 now live centrally)
- tools/tts_tool.py _sniff_audio_container (outbound repair, PR #73072)
cache_audio_from_url inherits the sniff via cache_audio_from_bytes.
Adds tests/tools/test_audio_container.py covering every magic-byte type,
wrong-extension repair on the inbound cache, unknown passthrough, the
URL path, and Signal's delegation.
iOS Signal delivers voice notes as MP4-container AAC carrying an audio
ftyp brand ("M4A "). `_guess_extension()` returned ".mp4" for every
`ftyp` file regardless of brand, so those attachments were cached as
documents instead of audio and STT rejected the upload:
API error: Error code: 400 - Invalid file format.
Supported formats: ['flac','m4a','mp3','mp4','mpeg','mpga','oga',
'ogg','wav','webm']
Read the 4-byte brand at offset 8 and return ".m4a" for audio brands
("M4A ", "M4B ") so they satisfy `_is_audio_ext()` and route to
`cache_audio_from_bytes()`. Video brands (isom/mp42/avc1/qt) still
return ".mp4".
This mirrors the existing brand/form-type disambiguation already used
in this function for RIFF (WEBP vs WAVE) and for ADTS AAC vs MP3.
Verified against a real iOS Signal voice note: pre-fix the upload was
rejected with the 400 above; post-fix the same bytes transcribe
successfully.
The cap is shared across CLI, desktop/TUI and the messaging gateway, so the
surface that gets rejected is rarely the one holding the slots. The rejection
read "Hermes is at the active session limit (5/5). Try again when another
session finishes." while every slot was an idle desktop tab, which took
filesystem access to work out.
Name the holders in the message, and show slot usage plus each holder in
`hermes status`. Both are inert when max_concurrent_sessions is unset, which
is the default. The gateway's duplicate copy of the message now reuses the
shared helper.
Manual /compress already persists the rotated child transcript first and
only then repoints the live session_entry; a False rewrite_transcript
return keeps the entry on the original session_id so the conversation
stays reachable. Session hygiene auto-compress did the opposite: it
rebound session_id (and lease/topic) first, then called rewrite_transcript
without checking the return value. On a failed write the live entry
already pointed at an empty child SID and the turn continued — permanent
silent conversation loss. Persist first; rebind only after success.
list_recent_user_messages and the in-memory /retry + session.undo walkers
treated every role=user row as a real user turn. Timeline bookkeeping
(model_switch, async_delegation_complete, auto_continue, hidden) is stored
that way, so /undo soft-deleted from a marker and /retry re-sent opaque
bookkeeping text. Exclude display_kind the same way CLI resume counting and
the prompt.submit ordinal path do.
