Companion to the cherry-picked #74158 fix (non-streaming path):
- gateway/run.py: remove the identical history-dedup filter from
_deliver_media_from_response — the post-stream rescan is explicit-only
by design (#20834), so every MEDIA tag it finds is a deliberate
attachment request; drop the now-unused history_media_paths parameter
and its call-site plumbing.
- gateway/platforms/base.py: log suppressed bare local file paths on the
surviving local_files history dedup (#73771 observability ask).
- tests: focused regression file covering explicit resend delivery on
both lanes, current-turn tool-echo poisoning, surviving bare-path
dedup + its log line, and the upstream auto-append dedup invariant.
Fixes#73771
Closes#73771
The session-wide MEDIA dedup in (base.py)
filtered ALL media paths against prior-turn history, including explicit
MEDIA: tags the model deliberately included in its response text. When a
user asked the agent to resend an image, the dedup silently swallowed it
because that path already existed in the session transcript.
The dedup is already handled correctly by the auto-append path in
(run.py), which scopes its scan to the current turn and
filters against via .
The base.py filter was redundant for auto-appended tags and harmful for
explicit ones.
Fix: remove the dedup filter on in base.py while preserving
the variable for the dedup (bare file
paths, which lack run.py protection).
Two relay-lane bugs from live Discord staging testing (2026-07-29):
1. TTS audio never attached over relay (any platform, any lane).
_history_media_paths_for_session excluded only the trailing assistant
entry from the persisted transcript when building the delivered-media
dedup set. The agent persists rows as it produces them, so THIS turn's
text_to_speech tool result (media_tag JSON) was already in the
transcript at delivery time — the fresh TTS path deduped against
itself and extract_media's attachment was silently stripped
(response_delivery_dropped for a MEDIA-tag-only reply; fly logs show
the exact signature). Fix: exclude everything from the last USER
message onward (the current turn); prior-turn dedup unchanged.
Affects every platform adapter (native + relay) on the non-streaming
delivery path — the streaming path passes explicit history and was
unaffected.
2. Connector-auto-created threads never got the LLM session-title
rename. The title fires on the FIRST exchange, whose source is the
PARENT channel event — the thread didn't exist at ingest, so the
Phase 4 auto-thread markers can't be present and
_is_discord_auto_thread_lane never matches on the relay title turn
(initial titles worked; semantic renames never happened; staging
telemetry shows zero thread_rename ops ever sent). Fix: consume the
connector's new send-result feedback (paired gateway-gateway PR —
contract §SendResult thread_id/auto_thread_name, additive):
RelayAdapter.send() caches (thread_id, initial_name) per chat
(bounded 256), run.py's title-callback registration + rename lane
read it back and pass initial_name as only_if_current_name so the
human-rename-wins guard holds on the relay lane too. Native marker
path unchanged; connectors that don't stamp the fields degrade to
exactly the old behavior.
Tests: 3 new (send-result feedback capture, absence, bound) in
test_relay_threads.py; 3 new in test_history_media_current_turn.py
(current-turn TTS not deduped, prior-turn still deduped, no-user-row
fallback). Relay suite 144 passed.
Four hot-path consumers paid a full config deepcopy per read:
- telemetry gate relay_shared_metrics.enabled() — runs 2-3x per agent
turn (2x per API call from lifecycle hooks + 1x per tool call) and
called read_raw_config(), which deepcopies the whole raw config every
call. New read_raw_config_readonly() serves the cached dict directly:
248 us -> 4.6 us per call (54x) on Teknium's real 77-key config.
- interruptible_streaming_api_call local-endpoint stale-timeout branch
called load_config() once per API call for every local-model user.
- gateway get_inbound_media_max_bytes() + _get_ephemeral_system_ttl_default()
called load_config() on per-message paths. All three switched to
load_config_readonly() (345 us -> 12 us; PR #28866 lineage).
Together these account for ~90% of the ~1,900 deepcopy primitives per
turn measured in the 26-call stubbed-LLM profile.
read_raw_config_readonly() keeps the (mtime_ns, size) freshness key so
config edits are picked up next call, and preserves the identity
invariant (cache-miss returns the same object later hits serve) —
regression-tested with 'is', per the PR #28866 identity-bug lesson.
The mutable read_raw_config() is unchanged for save-path callers.
581 targeted tests green (config, relay metrics x2, ephemeral reply,
platform base, new readonly suite).
- base.on_processing_complete implements the opt-in remove-ack/add-outcome
flow driven by _OK_EMOJI/_FAIL_EMOJI class attrs and the
_add_reaction(chat_id, message_id, emoji)/_remove_reaction(chat_id,
message_id) primitive shape; default stays a no-op.
- photon drops its override (exact behavioral match).
- slack/discord/feishu/matrix/telegram/google_chat keep overrides: divergent
primitive signatures (team_id routing, raw message objects, reaction-id
handles, replace-semantics setMessageReaction) or extra state protocols.
Add an opt-in streaming-audio adapter seam to BasePlatformAdapter so
voice-capable gateway platforms (LiveKit, Discord voice, future adapters)
can consume LLM output as streaming PCM audio before the full response
completes, dropping perceived voice latency from ~2-3.5s to ~500-800ms.
Adapter contract (gateway/platforms/base.py):
- AudioFormat dataclass: declared sample_rate, channels, sample_width
- StreamingTTSHandle: opaque handle with audible/aborted flags
- supports_streaming_tts / begin_streaming_tts / write_streaming_tts
/ finish_streaming_tts / abort_streaming_tts
- All default to unsupported/no-op so existing adapters are source-compatible
- Per-turn _streaming_tts_completed_chats set suppresses duplicate whole-file
auto-TTS when streaming succeeded; cleared after turn completion
Gateway consumer (gateway/streaming_tts_consumer.py):
- StreamingTTSConsumer: bridges sync agent deltas to async adapter audio sink
- Uses existing SentenceChunker (no competing parser)
- Thread-safe bounded queue; on_delta never blocks the agent worker thread
- Resolves configured streaming provider via resolve_streaming_provider()
- Serialises clause playback in order; flushes tail on completion
- Pre-audio failure: completed=False (falls back to whole-file TTS)
- Post-audio failure: completed=True, partial=True (no replay from start)
- Abort is idempotent; late chunks silently dropped
- Per-turn state isolated across concurrent chats
Gateway integration (gateway/run.py):
- message_type parameter threaded through _run_agent -> _run_agent_inner
- StreamingTTSConsumer created when voice input + auto-TTS + provider active
- Delta callback teed to both text stream consumer and TTS consumer
- TTS-only delta callback installed when text streaming is off
- finish() called from executor; wait_complete() in async context after
- Barge-in aborts the consumer at all three interrupt detection points
- Runner-level _send_voice_reply suppressed when streaming TTS completed
Tests (tests/gateway/test_streaming_tts_consumer.py):
- 15 focused tests: adapter defaults, lifecycle, ordered chunks,
unsupported/No-streamer fallback, abort idempotency, late-chunk drop,
pre/post-audio failure, concurrent-turn isolation, think-block suppression,
queue backpressure
Does not touch desktop/TUI code or add config flags. Plugin TTS provider
stream() metadata gap (#47896) is explicitly out of scope — built-in
ElevenLabs/OpenAI PCM streamers are the first consumers.
Refs: #60671, #47896
Follow-up to salvaged PR #56473: dedupe the five cleanup_*_cache bodies
into a shared _cleanup_cache_dir() helper, add cleanup_video_cache() and
cleanup_screenshot_cache() (with get_screenshot_cache_dir()), and drive
all five from a single (name, fn) loop in _start_gateway_housekeeping()
instead of one copy-pasted try/except per cache. Covers the video/
screenshot half of #56427.
Adds cleanup_audio_cache() to gateway/platforms/base.py and wires it into
_start_gateway_housekeeping() in gateway/run.py, matching the existing
image/document cache cleanup pattern.
Re-implements the fix proposed in #16473, which targeted the now-deprecated
_start_cron_ticker() and has since stalled with a merge conflict against
main. _start_gateway_housekeeping()'s docstring already claimed audio
cleanup ("prunes the image/audio/document cache"); this makes the code
match it.
Co-authored-by: nftpoetrist <264138787+nftpoetrist@users.noreply.github.com>
_invalidate_pending_stt_cache() clears the gateway-side transcription
cache when merge_pending_message_event() folds a follow-up message into a
still-pending event, so the next transcription picks up the merged text
and attachments. It also cleared _gateway_pending_stt_echo_sent, but that
flag is not derived state — it records that the transcript was already
delivered to the user.
Dropping it makes the re-run transcription echo the earlier notes a second
time. Both merge branches are affected, including the text-only follow-up
case where no new audio arrived at all: there the cache is invalidated,
the same voice note is transcribed again (a second paid STT call) and the
same line is echoed again.
Sequence:
1. voice note arrives, interrupt monitor transcribes it and echoes
'🎙️ "hello"'
2. user sends a follow-up while the turn is still pending, so it merges
3. drain path re-transcribes and echoes '🎙️ "hello"' a second time
Keep the ledger out of the invalidation set and track it as a count of
already-echoed transcripts instead of a single boolean. A count is what
the merge case actually needs: re-running transcription over the extended
media list returns the earlier transcripts as a prefix of the new one, so
echoing only the unsent tail suppresses the repeat while still surfacing a
newly merged voice note. A count rather than a set of seen values, so two
separate notes that transcribe identically stay two distinct deliveries —
covered by test_pending_stt_merge_echoes_two_identical_transcripts.
The guard stays within the 12-line window that
test_all_gateway_transcript_echo_sends_are_gated enforces over run.py.
[[audio_as_voice]] is message-global but was applied to every media file in a
message. A non-audio file flagged is_voice is excluded from the embedded-photo
batch and falls through to send_document, so an image in a message that also
carries a voice note arrives as a file attachment instead of an inline photo.
Gate the voice flag on the file extension so one message can carry an inline
image AND a voice bubble. Also wrap the path append in try/except so a crafted
~\x00 path is skipped rather than aborting extraction of all attachments.
Salvaged from PR #62040 (@giladbau), simplified per post-#73072 main: the
central _repair_ogg_container transcode makes an explicit .ogg output path
sufficient — no target_platform plumbing through the TTS tool needed.
Root cause (class-level): both gateway auto-TTS delivery call sites relied
on the TTS tool reading HERMES_SESSION_PLATFORM to pick Ogg/Opus vs MP3,
but that contextvar is cleared by _clear_session_env before the base
adapter's post-handler auto-TTS block runs, so want_opus was always False
on that path → MP3 → Telegram sent an audio attachment instead of a native
voice bubble (#57049, #36685). The runner's _send_voice_reply had the
sibling bug: it hardcoded .ogg for Telegram only, leaving Matrix and
Feishu runner voice replies as MP3 (#14841, #45557).
Fix: new build_auto_tts_output_path(platform) in gateway/platforms/base.py
hands an explicit .ogg temp path when the platform is in the TTS tool's
OPUS_VOICE_PLATFORMS set (single source of truth — telegram/matrix/feishu/
whatsapp/signal), .mp3 otherwise. Used by BOTH delivery call sites:
- BasePlatformAdapter auto-TTS block (also honors the tool's success flag
and cleans up requested + returned paths)
- GatewayRunner._send_voice_reply (replaces the telegram-only ternary)
Fixes#57049Fixes#36685
Refs #14841#45557
When /reasoning show is enabled, the model's <think> blocks appear in
the final assistant message. TTS reads these aloud, which is unwanted —
users want to see reasoning but not hear it spoken.
- Add _THINK_BLOCK regex to _strip_markdown_for_tts() (tools/tts_tool.py)
— the general TTS text-preparation path used by all TTS providers
- Add <think> stripping to prepare_tts_text() (gateway/platforms/base.py)
— the gateway auto-TTS path
- Reuses the existing regex pattern from stream_tts_to_speaker()
Closes#34213
Review follow-up: the spoken script is for synthesis only. Caption
eligibility and payload stay on the original reply, so a long reply
whose normalized script fits the 1024 char limit is still delivered
in full as its own message. Adds the long-original/short-normalized
regression case to the auto-TTS caption tests.
Auto-TTS previously fed raw chat Markdown and compact symbols straight to the
speech provider, so units were read as stray letters and headings or bullets ran
together. This routes spoken text through a new normalizer,
tools/tts_text_normalize.prepare_spoken_text, that expands units (for example a
temperature written with the degree symbol becomes "degrees Celsius") and
flattens Markdown into a transcript-like script with sentence pauses.
The normalizer is best-effort: if it ever fails the code falls back to the
previous markdown-strip behavior, so auto-TTS keeps working. The Telegram voice
caption uses the same normalized text. Includes a unit test.
One sniffer owns magic-byte container detection (Teknium's one-concept-
one-owner rule): the new tools/audio_container.py is used by
- gateway/platforms/base.py _sniff_audio_ext (inbound cache — PR #36166's
central sniffer, now covering AAC/ADTS, MP4-brand disambiguation, webm)
- gateway/platforms/signal.py _guess_extension (audio/AV branches
delegated; RIFF/WAVE fix from PR #50690 and M4A-brand fix from
PR #72490 now live centrally)
- tools/tts_tool.py _sniff_audio_container (outbound repair, PR #73072)
cache_audio_from_url inherits the sniff via cache_audio_from_bytes.
Adds tests/tools/test_audio_container.py covering every magic-byte type,
wrong-extension repair on the inbound cache, unknown passthrough, the
URL path, and Signal's delegation.
When FTS5 index corruption prevents INSERT INTO messages, the gateway
accumulates messages in _pending_messages (memory-only). On shutdown,
.clear() discards the only surviving copy — permanent user data loss.
Changes:
- Add gateway/shutdown_flush.py with flush_pending_to_file() and
recover_pending_to_db() for two-phase data preservation.
- Patch gateway/run.py: flush runner._pending_messages before clear()
in _stop_impl_body, and recover on startup after runner.start().
- Patch gateway/platforms/base.py: flush adapter._pending_messages
before clear() in the adapter shutdown path.
Recovery behavior:
- Reads pending JSON files from ~/.hermes/pending_messages/
- Inserts messages into state.db directly
- Per-session isolation: one corrupt session doesn't block others
- Successful recovery deletes the flush file
- Failed recovery re-saves for next startup retry
One relay adapter fronts N platforms on one WS, but the capability surface
(MAX_MESSAGE_LENGTH / message_len_fn) was a scalar from whichever descriptor
resolved the handshake — and the transport's read loop OVERWROTE it on every
descriptor frame (last-writer-wins). A Discord chat on a gateway whose
applied descriptor was Telegram's inherited the 4,096-char cap and over-sent
into Discord's 2,000-char API 400 (observed live: 2,543/2,641-char replies
silently lost while inbound kept working).
- ws_transport: accumulate one descriptor per platform in
_descriptors_by_platform (exposed via descriptor_for_platform); the FIRST
descriptor of a connection generation stays the session default instead of
last-writer-wins; the map resets on re-dial.
- BasePlatformAdapter: new max_message_length_for_chat /
message_len_fn_for_chat hooks defaulting to the scalar surface (native
single-platform adapters unchanged).
- RelayAdapter: overrides resolve the chat's platform from _platform_by_chat
(the same map per-frame egress uses) and look up that platform's negotiated
descriptor; falls back to the scalar for unknown chats/transports.
- stream_consumer (streaming budget, _raw_message_limit, fallback-continuation
chunking) + run.py tool-progress limit now resolve per-chat.
Tests: tests/gateway/relay/test_relay_per_platform_caps.py (7) — verified
fail-without/pass-with (all 7 fail with the fix stashed). Relay + stream
consumer suites green (213 + 223).
Cross-surface coverage #23768 missed (per review feedback):
- tools/clarify_gateway.py: _ClarifyEntry carries a multi_select flag
(register() accepts it; signature() exposes it to adapters).
_coerce_text_response now parses multi-select replies — comma- or
space-separated numbers ('1,3' / '1 3'), exact labels, dedup — into a
JSON array string that _parse_multi_select_response decodes into a
list. Out-of-range/unknown tokens reject the reply (native button UI)
or fall back to custom text (awaiting_text/'Other' mode).
- gateway/run.py: _clarify_callback_sync accepts multi_select and
registers it on the pending entry.
- gateway/platforms/base.py: default numbered-list text fallback tells
the user multiple selections are allowed and how to reply.
- tui_gateway/server.py: clarify_callback passes multi_select through
the clarify.request payload as a hint; renderers without checkbox
support ignore the field and remain single-select-compatible.
- tests: 13 new gateway tests (flag storage, comma/space/single-number
parsing, label matching, out-of-range rejection, dedup, end-to-end
resolve, single-select regressions).
Follow-up wave to #72170 resolving the remaining open MEDIA-delivery gaps:
- #24032: add .kmz/.kml/.geojson/.gpx to MEDIA_DELIVERY_EXTS, and recover
unknown-extension paths containing spaces via _match_extensionless_path —
validation-gated forward extension across single spaces (bounded at 8
tokens, stops at newline / next MEDIA: keyword). The regex itself stays
non-greedy and whitespace-bounded so the #68773 absorption bug class
cannot return; the on-disk file check is the oracle.
- #16434 (streaming half): _strip_media_tag_directives now uses the same
mask-as-locator pattern as extract_media, so MEDIA tags inside fenced
code blocks, inline-code examples, and JSON string values survive in
streamed display text instead of being mangled. Display and delivery
now agree on every protected-span rule.
- Updated three stream_consumer display expectations that pinned the old
inconsistent behavior (backtick/double-quote tags stripped from display
while delivery never attempted them).
The model sometimes echoes a previous MEDIA:path tag or bare file path in a later response. Previously both streaming delivery and non-streaming delivery would re-send those files, causing duplicate documents/images on later turns.
Changes:
- _collect_history_media_paths now also scans assistant messages for MEDIA: tags, not just tool results.
- _deliver_media_from_response accepts pre-computed history_media_paths and filters extracted media/local files.
- BasePlatformAdapter gains _history_media_paths_for_session for non-streaming dedup.
Two remaining formatting variants that silently killed file delivery:
- A trailing sentence period (MEDIA:/x/data.csv.) failed the boundary
lookahead, so the tag neither extracted nor stripped. The period is now
accepted as a boundary only when followed by whitespace/EOL, keeping
multi-part extensions (.tar.gz) intact.
- A whole tag wrapped in inline code (`MEDIA:/x/data.csv`) was masked as
a prose example (#35695). Models routinely format paths as inline code,
eating real deliveries. Inline-code tags now deliver when the path
validates on disk; non-existent example paths stay masked and fenced
code blocks remain fully masked.
Adds a regression matrix covering both plus the salvaged contributor
fixes (emphasis wrap, glued tags, glued [[as_document]], dedupe,
unknown-extension and extensionless delivery).
MEDIA_DELIVERY_EXTS in gateway/platforms/base.py omitted .3gp, causing
MEDIA: tags with .3gp files to leak as plain text instead of being
extracted for native video delivery. _VIDEO_EXTS in
tools/send_message_tool.py and _MIGRATION_VIDEO_EXTS in the Feishu
adapter omitted .webm, causing .webm files to be classified as
documents instead of video on Telegram and other platforms.
Both extensions are already present in every gateway-side _VIDEO_EXTS
definition (run.py, kanban_watchers.py, weixin.py, base.py local).
Closes#71621, Closes#71603
Surface a user-visible delivery notice when non-streaming media dispatch
gets success=False after MEDIA tags were stripped, and validate forum
starter-message attachments the same way as direct channel sends (#66797).
Outbound MEDIA video/document tags on the Discord non-streaming path
were extracted (stripped from the visible text) but never delivered:
no attachment, no error, and no dispatch log line (#66797). The
open-handle discord.File plus singular file= form could race Discord's
multipart encoder after an earlier image batch on the same channel and
return a successful message carrying zero attachments.
Fix _send_file_attachment (used by send_video/send_document/
send_image_file) to:
- use a path-based discord.File via the plural files=[...] kwarg, the
same pattern as the working send_multiple_images batch path;
- pre-flight os.path.isfile and return a File not found result instead
of raising deep inside discord.File;
- fail loud when Discord accepts the message but attaches nothing, so
the dispatch loop surfaces a warning rather than a silent drop;
- add INFO dispatch logs (base non-image MEDIA fan-out, video send, and
file attachment) so a MEDIA:.mp4 cannot vanish without a trace.
Tests (tests/gateway/test_discord_send.py):
- path-based files=[...] kwarg used, singular file= unset;
- fail-loud when the returned message has no attachments;
- missing file fails fast without resolving the channel;
- forum-parent delivery routes through create_thread with files=[...];
- end-to-end image+video response routes the mp4 to send_video while
images still batch.
Fixes#66797
When the same file is referenced multiple times in one message (common
when the agent emits MEDIA tags both inline and in a summary footer),
the platform adapter uploads/sends the same file twice — visible as
duplicate attachments in Telegram, duplicate posts in Slack, etc.
Set-based dedup on the expanded path, preserving first-occurrence order.
Two MEDIA: path tags emitted back-to-back without a separator merged
into a single invalid path and were silently dropped. The same happened
for extension-less tags.
Root cause: both regexes in gateway/platforms/base.py used greedy
quantifiers in their path class, causing adjacent tags to be absorbed.
Fix: make both regexes non-greedy (add ? to quantifiers) and add
MEDIA: to the trailing lookahead boundary set so the next MEDIA:
keyword stops the current match cleanly.
Models routinely present a file to the user with the delivery tag wrapped in
Markdown emphasis — `**MEDIA:/path.pptx**`, `*MEDIA:/path*`, `_MEDIA:/path_`.
MEDIA_TAG_CLEANUP_RE only tolerated a single leading/trailing quote or backtick
(`[`"']?`), and its closing lookahead set excluded `*` and `_`, so an
emphasis-wrapped tag never matched. The file was then silently never delivered
and the literal `MEDIA:/path` text leaked into the chat instead — the user sees
a path, not the attachment.
Allow a short run of emphasis/quote markers (`[`"'*_]{0,3}`) on both sides of
the tag and add `*`/`_` to the closing lookahead. Code-block, inline-code and
blockquote contexts are still neutralised earlier by `_mask_protected_spans`
(#35695), so documentation/example tags remain non-deliverable; the
absolute-path anchor still rejects relative paths; `_` inside a filename is
unaffected.
Adds regression coverage in TestExtractMedia for bold/italic/underscore
wrapping, mid-prose bold, emphasis-wrapped .html, underscore-in-filename, and
emphasis-wrapped relative-path rejection.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MEDIA_TAG_CLEANUP_RE failed to match when a directive like [[as_document]]
was concatenated directly to the file extension without whitespace
(e.g., MEDIA:/home/user/report.xlsx[[as_document]]). This caused the
file to be silently not delivered while the gateway reported success.
Root cause: the lookahead character class [\s`",;:)\}\]|$) did not
include [, so [[as_document]] immediately after the extension broke
the lookahead assertion.
Fix: add \[ to the lookahead class so directives can follow the path
without whitespace. This is safe because [ is already stripped elsewhere
in the same file via .replace("[[as_document]]", "").
Added regression tests covering:
- Directive glued to extension (issue case)
- Directive with whitespace (baseline)
- Tag at end of string ($ anchor)
- RelayAdapter.send_exec_approval / send_slash_confirm / send_clarify:
override the base text fallbacks with ONE platform-abstract `prompt` op
(connector renders Discord components / Telegram inline keyboards /
Slack Block Kit / WhatsApp buttons+lists). Option sets mirror the native
adapters exactly (once/session/always/deny with the same
allow_session/allow_permanent/smart_denied gating; once/always/cancel;
choices + Other). Clarify option ids are positional (c0..cN/other) —
choice text is arbitrary UTF-8, callback budgets are 64 bytes.
- Pending-prompt registry: gateway-minted 8-hex prompt ids →
{kind, session_key, extras}; one answer wins, lazy expiry, unanswered
prompts swept opportunistically. Wire timeout_s stays advisory.
- _consume_prompt_response (wired into _on_inbound AND the Discord
passthrough lane): routes answers to the SAME primitives the native
button handlers call — tools.approval.resolve_gateway_approval,
tools.slash_confirm.resolve, tools.clarify_gateway
resolve/mark_awaiting_text — then acks in-channel. Unknown/expired ids
fall through as command-shaped text (typed-reply degradation, the
relay's analog of the native 'approval expired' edit).
- Discord type-3 stub replaced: an hp1:<prompt>:<option> custom_id decodes
to a structured prompt_response (codec mirrored from the connector's
promptCodec); foreign custom_ids keep the legacy best-effort text shape.
- MessageEvent.prompt_response field + ws_transport wire parsing (additive).
- react ack lifecycle: on_processing_start/complete → `react` ops
(👀 → ✅/❌, remove-then-add), op-gated on supported_ops, best-effort by
contract (a react failure never touches the turn).
- Op gating throughout: a connector not advertising `prompt` gets
success=False from send_exec_approval/send_slash_confirm (run.py's text
fallback takes over — same contract as a failed native button send) and
the base numbered-text clarify; `react` silently no-ops.
- docs/relay-connector-contract.md §4: prompt / prompt_response / react
semantics (callback token, budgets, authorization-parity, foreign-id
behavior, per-platform react mappings).
- tests: tests/gateway/relay/test_relay_interactive.py (19) — option-set
rendering + gating matrices, registry consume-once/expiry, resolver
routing for all three kinds (monkeypatched primitives), fall-through
cases, Discord hp1 decode + foreign-id shape, react lifecycle
(success/failure/cancelled), op-gated/best-effort react.
Cross-repo pair: gateway-gateway 'Phase 3 interactive' PR (prompt/react
senders on all four lanes + interaction ingest).
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.
Fixes#53428 (master tracker for Windows GBK locale crash).
Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
Adds `ensure_closed_code_fences()` helper to detect text with an odd
count of triple-backtick markers (indicating an unclosed code block)
and append a closing fence.
Applies the fix to all four identified gap paths:
G1: truncate_message early-break path for final chunk
G2: _send_or_edit streaming edit path (most commonly hit)
G3: overflow split first chunk (covered by G2's fix)
G4: _send_fallback_final fallback send path
Closes: #TBD
Build the full reaction pipeline on top of the #29916 base:
- Opt-in gate: slack.reaction_triggers (default OFF — reaction events
stay acked-and-dropped so busy channels don't wake the agent on every
emoji). 'true' routes reactions on the bot's OWN messages; an explicit
emoji-name list routes those emojis from any message (handoff flows).
- reaction_removed events now route too, distinguished by the
cross-platform text convention reaction:added:<emoji> /
reaction:removed:<emoji> (matches the Feishu and Photon adapters, so
agents and skills see one shape everywhere).
- Authorization: the reactor becomes the synthesized message's user, so
the early _is_user_authorized gate and allowed_channels whitelist
apply exactly as for typed messages. _hermes_force_process only skips
the mention requirement (a reaction on the bot's own message is
definitionally addressed to the bot), mirroring Feishu/Photon.
- Gateway hooks (#33111 by @johnkattenhorn): every human reaction on a
message item fires reaction:added / reaction:removed through the new
BasePlatformAdapter.set_reaction_handler → GatewayRunner
._handle_reaction_event → HookRegistry.emit, independent of the
routing opt-in. Documented in hooks.md.
- Channel handoff (#45265 by @Kev-fs): slack.reaction_trigger_target
routes the reaction turn to a configured channel (top-level via
_hermes_no_thread_response + reply-anchor suppression in
gateway/platforms/base.py) or C123:<ts> thread.
- Manifest: reaction_removed event subscription added alongside
reaction_added/reactions:read.
- Docs: slack.md Reaction Triggers section; hooks.md event table rows.
Also credits #44508 by @harrisonmedmedmetrics (inbound reaction_added
handling — same plumbing class, superseded by this consolidated shape).
Co-authored-by: johnkattenhorn <john.kattenhorn.personal@gmail.com>
Co-authored-by: Kev-fs <kevin@fleetsmarts.net>
Co-authored-by: harrisonmedmedmetrics <harrison@medmetricsrx.com>
Install connect-time DNS validation for Hermes-owned direct httpx clients so SSRF-sensitive fetch paths dial a vetted IP instead of re-resolving after preflight. This preserves Host/SNI semantics for direct HTTP(S) connections and keeps proxy routing as an explicit trusted egress boundary.
Wire the guarded clients into media cache downloads, vision downloads, Skills Hub direct/raw fetches, and platform attachment fetch paths that already perform SSRF preflight and redirect validation.
Fixes#8033
Co-authored-by: Tom Qiao <zqiao@microsoft.com>
When --replace misses a cross-HERMES_HOME Telegram token holder, platform
connect used to retry forever. Terminate a verified gateway holder once
(with the takeover marker) and re-acquire the scoped lock (#65176).
Co-authored-by: Cursor <cursoragent@cursor.com>
A final response generated but not confirmed-delivered was the one
artifact the gateway could lose without a trace: crash or planned
restart between finalize and platform ACK dropped it silently, and the
resume path re-ran the whole turn at full cost (#58818 P1, #41696,
#63695's gateway half).
gateway/delivery_ledger.py records each outbound final response in
state.db (same conventions as the async-delegation ledger: WAL, owner
pid + process-start-time liveness, bounded retention):
pending -> attempting -> delivered | failed
startup sweep on dead-owner rows -> redeliver | abandoned
Contract (the lessons from the closed delivery-outbox attempt #61790):
- obligation recorded BEFORE the first send attempt; cleared only on
SendResult.success (destination acceptance, #51184)
- ambiguity is labeled, never silently retried: rows that were mid-send
when the process died redeliver with a visible '♻️ Recovered reply —
may be a duplicate' prefix (honest at-least-once)
- stable ids from session_key + inbound message id + content, so
distinct threads/topics can never collide
- poison rows bounded: 3 attempts / 24h freshness -> abandoned; claim
atomically re-stamps ownership so racing sweeps can't double-claim
- redelivery clears resume_pending for the session so the resume path
never re-runs a turn whose answer the ledger already holds
- best-effort everywhere: ledger failure can never block or delay a send
- slash-command/ephemeral/empty responses are not recorded; cron and
proactive delivery stay on DeliveryRouter (separate subsystem)
Config: gateway.delivery_ledger (default on; no version bump needed).
Validation: 30 ledger+producer tests; 352 blast-radius gateway tests
green; cross-process E2E (record in process A, kill it mid-send, claim
+ marker + redeliver in a fresh process B against the same state.db).
merge_pending_message_event extends the existing event's media_urls in
place when two media-bearing messages arrive in quick succession (photo
bursts, consecutive voice messages). The gateway runner caches STT
transcripts on the event via _gateway_pending_stt_text; if the cached
event gains new media after the cache was populated, the stale transcript
was returned instead of transcribing the merged attachments.
Add _invalidate_pending_stt_cache() and call it from both media-merge
branches in merge_pending_message_event so the next transcription call
re-runs against the full merged media list.
Closes the merge-race edge case identified during review of PR #61519.
Gateway tests build adapters via object.__new__() without __init__ (the
documented bare-instance pattern), so the new _status_text dict must be
accessed through getattr guards in set_status_text, the _keep_typing
finally cleanup, and the Slack send_typing read — same treatment as
other post-hoc __init__ attributes. Fixes CI shard 2/8
(test_active_session_text_merge).
Builds on the salvaged typing_status_text plumbing (PR #62007): instead
of a static 'is thinking...', Slack's assistant status line now updates
live as the agent works — 'is running pytest tests/…', 'is reading
docs/api.md…' — and reverts to the static text between tool calls.
Mechanics:
- agent/display.py: build_status_phrase() derives a <=49-char present-
tense phrase from the existing _TOOL_VERBS table (+ 'is using <name>'
for plugin/MCP tools; None for _thinking).
- base adapter: supports_status_text capability flag + set_status_text()
per-chat store, cleared when the typing loop winds down.
- Slack adapter: send_typing() renders the live phrase when set, falling
back to typing_status_text then 'is thinking...'.
- gateway/run.py: progress_callback stashes the phrase on tool.started
and clears on tool.completed. Rendering rides the existing
_keep_typing refresh cadence — zero additional Slack API calls, no
rate-limit exposure. Works with tool_progress: off (Slack default);
the callback is now armed whenever the adapter supports status text.
- display.live_status config (full|verb|off, default full): 'verb' hides
argument previews for shared/customer-facing channels.
Also fixes a latent crash in the cherry-picked from_dict: malformed
non-dict 'extra' sections broke typing_status_text resolution (uses the
already-coerced extra dict).
Design notes: status text is a side-effect display channel only — never
enters the transcript, no prompt-cache impact. Lifecycle guarantees from
the stuck-status fix family are preserved (per-thread tracking,
clear-on-finish via existing stop_typing paths). Related: #45109
(closed; same direction via lifecycle states), #59010/#51363 (native
task cards — complementary, larger scope).
Replace REST-based Discord liveness probe with local WebSocket/heartbeat
state detection. REST success doesn't prove Gateway event delivery — a
half-closed WebSocket can leave Bot.start() alive while REST returns 200.
Now samples ready/open/ACK state and heartbeat latency; consecutive
unhealthy samples emit one retryable fatal code so GatewayRunner rebuilds
the adapter through the existing reconnect path.
Also fixes three lifecycle gaps in the recovery path:
1. asyncio.wait_for() can remain blocked if adapter cleanup swallows
cancellation — now uses bounded asyncio.wait() with task detachment.
2. Multiplexed secondary-profile adapters had no profile-scoped reconnect
owner — now uses one runner-owned reconnect slot per profile.
3. An in-flight turn could send its final text through the disconnected
adapter after a replacement was registered — now resolves the live
same-profile replacement for unsent final responses only (message IDs
never migrate, edits/deletes stay on the old transport).
Adds an opt-in Linux/systemd event-loop watchdog (gateway.systemd_watchdog_seconds,
default 0) for the failure mode where the whole asyncio loop stops making
progress and no in-process liveness task can run. stdlib-only sd_notify,
Type=notify/WatchdogSec generation, READY/STOPPING lifecycle.
Co-authored-by: 王鑫 <wx.xw@bytedance.com>
Unifies the two gateway subsystems that were fighting each other: the
'never lose a session' recovery machinery (#54878 stale-route self-heal,
find_latest_gateway_session_for_peer reopening agent_close/ws_orphan_reap
rows) and the session reset/expiry machinery (expiry watcher, /new,
/resume, resume_pending freshness gate).
The unified contract:
- INTENTIONAL boundaries (expiry finalization, auto-reset, /new,
/resume switch) are recorded durably via promote_to_session_reset(),
which upgrades accidental recoverable end_reasons (agent_close,
ws_orphan_reap) to the explicit boundary while preserving other
explicit reasons (compression, etc.). Recovery then correctly refuses
to resurrect them.
- ACCIDENTAL ends (crash, cleanup bug, mistaken reaper) stay
recoverable — genuine crash recovery is untouched.
On top of the cherry-picked contributor commits:
- promote_to_session_reset widened to ws_orphan_reap + parameterized
reason so auto-reset paths stay auditable (idle/daily/suspended/
resume_pending_expired) (#61220, #61993, #63539)
- get_or_create_session auto-reset, reset_session (/new), and
switch_session (/resume) all write through the promote path — the
first-reason-wins end_session no-op could previously leave a reset
session resurrectable behind a stale agent_close row (#61993)
- resume_pending freshness gate now honors session_reset.mode=none:
explicit opt-out of automatic resets also opts out of the zombie
gate (#61052)
- resume recovery note extracted to build_resume_recovery_note() and
made adapter-aware via a new interactive_resume capability flag:
webhook/api_server auto-resume turns now CONTINUE the interrupted
task instead of emitting an unanswerable 'session restored'
acknowledgement that abandoned the work (#57056)
- tests updated to call the real note builder instead of mirroring it
E2E-validated against a real SessionDB + SessionStore in a temp
HERMES_HOME: expiry->agent_close->no-resurrection, /new promote,
crash recovery preserved, mode=none opt-out, routing-table flag sync.
Follow-up to the truncate_message split-loop floor. Two review points:
- CapabilityDescriptor.from_json trusted the wire max_message_length
verbatim, so a connector advertising 0 ('no limit') — or a buggy one
sending 0/negative — produced a descriptor whose bound flowed straight
into the adapter's MAX_MESSAGE_LENGTH and truncate_message. Normalize
it to the documented 4096 default (mirrors from_platform_entry's
'or 4096' and docs/relay-connector-contract.md), fixing the degenerate
budget at its source rather than only surviving it downstream.
- Document the truncate_message length contract for a budget too small
for one codepoint (max_length=1 with a 2-unit surrogate pair under
utf16_len): the chunk intentionally exceeds max_length by that one
indivisible codepoint, because emitting it whole preserves content
where the alternatives are data loss or an infinite loop.
Tests: from_json normalizes 0 and negative bounds to 4096 and passes a
real positive bound through unchanged; the sub-codepoint budget emits
whole codepoints with no data loss (all emojis preserved) and a chunk
that necessarily exceeds the 1-unit budget.
