Commit graph

2317 commits

Author SHA1 Message Date
teknium1
bf15259e33 refactor(gateway): shared media-cache mime dispatch for adapter downloads (per-adapter overrides preserve historical mappings) 2026-07-29 10:14:59 -07:00
victor-kyriazakos
1773752c8c Merge remote-tracking branch 'origin/main' into feat/gateway-health-diagnostics-monitoring
# Conflicts:
#	uv.lock
2026-07-29 15:37:14 +00:00
Enough1122
5cc5c58e01 fix(gateway): flush pending memory writes before session teardown (#73297)
The gateway's /reset cleanup path called shutdown_memory_provider without
first draining the memory manager's serialized background write worker.
shutdown_all only gives that worker a bounded (~5s) drain and abandons
whatever is still queued past it, so a /reset could silently drop writes
the session had already handed off -- the next session then loaded
stale MEMORY.md.

The CLI exit path already drains via MemoryManager.flush_pending before
shutdown; this PR pins the same contract on the gateway cleanup path.

Cleanup now calls agent._memory_manager.flush_pending(timeout=10) before
the existing shutdown_memory_provider step. The flush is best-effort:
a flush failure must never block teardown, so it is wrapped in
try/except and the existing shutdown path remains the fallback.

Closes #73297
2026-07-29 17:21:55 +05:30
StellarisW
8c50aaceb6 fix(gateway): keep models.dev refreshes off event loop 2026-07-29 17:13:48 +05:30
kshitijk4poor
23e44a2843 fix: consolidate agent-history preservation into shutdown_flush module
Follow-up for salvaged PRs #73400 + #73372:
- Move _preserve_agent_history_on_shutdown logic into
  gateway/shutdown_flush.py as flush_agent_history_to_file()
- Reuse the hardened _write_payload (atomic writes, fsync, private
  permissions, UUID filenames) from #73372 instead of the plain
  open()/write() from #73400
- Replace os.environ.get('HERMES_HOME') with get_hermes_home() for
  profile-safe path resolution
- Update tests to test the real function directly
- Net: removes 37 lines from gateway/run.py, unifies both fix paths
  in a single module with consistent atomic-write guarantees
2026-07-29 16:45:35 +05:30
Baophan00
40837e2dd0 Fix #72680 (retargeted): preserve agent._session_messages on shutdown flush failure
The previous attempt (#73171) snapshotted GatewayRunner._pending_messages,
which on current main has no writers (commit f6736ced8 removed its write
path; interrupt delivery uses adapter._pending_messages instead). The live
container is the per-agent agent._session_messages, flushed via
_flush_messages_to_session_db. When that flush raises (FTS/SQLite corruption,
the disk=0/memory=N state from #72680), the in-memory transcript is lost when
the process exits.

Retarget the preservation to the real path: in _finalize_shutdown_agents, wrap
the _flush call; on exception, dump agent._session_messages to an external JSON
recovery snapshot under $HERMES_HOME/shutdown-recovery/ (tagged issue=#72680)
so an operator can salvage it after repairing state.db. The dump is fully
guarded (non-fatal) so shutdown never blocks on a best-effort backup.

This directly addresses the reviewer note on #73171: retarget to the actual
cached-agent history (agent._session_messages) and prove a stale DB + shutdown
leaves a recoverable transcript.

Regression tests: tests/gateway/test_session_messages_shutdown_preserve.py
- flush raises -> recovery file written with session_id + messages
- healthy flush -> no recovery file
- write error -> non-fatal, no raise

Fixes #72680
2026-07-29 16:45:35 +05:30
kshitij
720cdd1d14 refactor: use atomic_json_write instead of hand-rolled _write_payload
Replace 20 lines of manual os.open/O_EXCL/fdopen/fsync/os.replace with the
existing atomic_json_write() from utils.py, which is already used by 6+
modules and handles temp-file creation, fsync, atomic replace, mode
control, and owner preservation. The only novel helper (_fsync_directory)
is retained — atomic_json_write does not do directory fsync.

Update test_flush_write_failure_leaves_no_recovery_file to monkeypatch
utils.os.replace (the new call path) instead of gateway.shutdown_flush.os.replace.
2026-07-29 11:16:09 +05:30
deacon-botdoctor
72024950cf fix(gateway): harden shutdown message flush 2026-07-29 11:16:09 +05:30
Teknium
cf1e3585b6 test(line): skip dual-stack both-families assertion on IPv4-only hosts
test_default_bind_serves_both_families asserts an IPv6 listening socket
exists, which is the environment's capability, not the adapter's — CI
runners with IPv6 disabled would flake. Probe an actual ::1 bind (not
just socket.has_ipv6, a compile-time constant) and skip cleanly when
the host has no usable IPv6 stack.
2026-07-28 22:42:41 -07:00
Shannon Sands
e24bb0b42f fix(line): dual-stack webhook bind — 0.0.0.0 default unreachable over IPv6-only networks
The LINE adapter's webhook server defaulted to host="0.0.0.0", which
binds IPv4 ONLY. On IPv6-only private networks — notably Fly.io 6PN,
where the hosted edge router reverse-proxies LINE ingest to
<app>.internal:8646 over an fdaa: IPv6 address — nothing is listening
on the dialed address: connection refused → customer-visible 502 when
LINE's console verifies the webhook (NS-603).

This is the same bug the generic webhook adapter fixed in d542894ad;
the LINE adapter was never updated to match. Fix mirrors that commit:

- DEFAULT_HOST = None → asyncio/aiohttp create_server binds one socket
  per address family (v4 + v6), regardless of the bindv6only sysctl.
  "::" is NOT a valid substitute — Fly machines set bindv6only=1, so
  it would yield an IPv6-only socket and break IPv4 loopback probes.
- Empty-string host collapses to None; LINE_HOST/extra.host still pin
  a specific bind address.
- reuse_address=False scoped to macOS only (BSD wildcard-socket
  traffic-splitting footgun), mirroring 9420ad946.
- The three outbound-media guards compared webhook_host == "0.0.0.0"
  by string equality; extracted to _missing_public_url() which treats
  None/0.0.0.0/::/"" as "no fetchable hostname" so the LINE_PUBLIC_URL
  requirement still fires under the new default. _media_url() falls
  back to 127.0.0.1 instead of interpolating 'None' into URLs.

Tests: dual-stack default decision table (default None, empty→None,
pinned preserved, LINE_HOST override), behavioural both-families bind
proof via runner.addresses, and the media public-URL guard matrix.
88 passed, ruff clean.

Companion router fix (hermes-agent-router) routes /webhooks/line and
/line/webhook|/line/media to :8646 over 6PN; both are needed for
end-to-end hosted LINE delivery.

Fixes NS-603
2026-07-28 22:42:41 -07:00
Teknium
a65494ed00 fix(gateway): treat pid=None lifecycle sentinel as unknown ownership in mark_exited
The ownership guard let a sentinel with pid=None pass as self-owned, so
an exiting life could clobber evidence of unknown provenance with a
clean-exit claim. Tighten: only rewrite when the sentinel pid matches
os.getpid() exactly; pid=None (or malformed) is left untouched. Adds
tests for the pid=None no-op and the own-pid rewrite paths.
2026-07-28 22:41:09 -07:00
Shannon Sands
9c76c133b7 fix(gateway): detect and report unclean shutdowns via lifecycle ledger (NS-608)
Hosted agents that die uncleanly (kernel OOM kill, SIGKILL, whole-VM
death) leave no trace: shutdown_forensics only covers graceful signals,
gateway-exit-diag.log only covers exit paths that actually run, and the
VM reboot wipes dmesg before anyone can capture it. NS-608 (BlueAtlas
hourly crash cycle, July 12-15) took days of manual log correlation to
classify because nothing recorded 'the previous life ended violently'.

Add gateway/lifecycle_ledger.py — a sentinel state machine persisted to
<HERMES_HOME>/state/gateway.lifecycle.json:

- start_gateway() claims the sentinel (phase=running) right after the
  PID-file/runtime-lock claim, and reports any prior life that never
  reached an exit path as gateway.previous_unclean_exit in
  gateway-exit-diag.log + a WARNING log line.
- Every exit funnel marks the sentinel exited with a reason:
  _exit_after_graceful_shutdown (graceful_shutdown), the shutdown
  watchdog (shutdown_watchdog), and the loop-liveness watchdog
  (loop_liveness_watchdog).
- Ownership-guarded for --replace takeovers: a live matching owner is
  never reported dead, and the old life cannot clobber the
  replacement's freshly claimed sentinel on its way out.

The 30s loop heartbeat now embeds a cheap /proc memory sample (own RSS,
MemAvailable, swap used) so every unclean-death report carries a
'memory N seconds before death' snapshot; the detector flags
suspected_oom when the last sample shows <64MiB or <5% available.

container-boot.log lines gain prior_exit=clean|unclean|unknown per
profile, stamping unclean container deaths into the volume-persisted
boot log where support can grep for them.

Tests: tests/gateway/test_lifecycle_ledger.py (16 cases) + 4 new
container-boot annotation cases. Existing watchdog/forensics/boot
suites all green; ruff clean.
2026-07-28 22:41:09 -07:00
Carl Taylor
3a4aa2f8e6 feat(gateway): streaming TTS adapter contract and consumer (#60671)
Add an opt-in streaming-audio adapter seam to BasePlatformAdapter so
voice-capable gateway platforms (LiveKit, Discord voice, future adapters)
can consume LLM output as streaming PCM audio before the full response
completes, dropping perceived voice latency from ~2-3.5s to ~500-800ms.

Adapter contract (gateway/platforms/base.py):
- AudioFormat dataclass: declared sample_rate, channels, sample_width
- StreamingTTSHandle: opaque handle with audible/aborted flags
- supports_streaming_tts / begin_streaming_tts / write_streaming_tts
  / finish_streaming_tts / abort_streaming_tts
- All default to unsupported/no-op so existing adapters are source-compatible
- Per-turn _streaming_tts_completed_chats set suppresses duplicate whole-file
  auto-TTS when streaming succeeded; cleared after turn completion

Gateway consumer (gateway/streaming_tts_consumer.py):
- StreamingTTSConsumer: bridges sync agent deltas to async adapter audio sink
- Uses existing SentenceChunker (no competing parser)
- Thread-safe bounded queue; on_delta never blocks the agent worker thread
- Resolves configured streaming provider via resolve_streaming_provider()
- Serialises clause playback in order; flushes tail on completion
- Pre-audio failure: completed=False (falls back to whole-file TTS)
- Post-audio failure: completed=True, partial=True (no replay from start)
- Abort is idempotent; late chunks silently dropped
- Per-turn state isolated across concurrent chats

Gateway integration (gateway/run.py):
- message_type parameter threaded through _run_agent -> _run_agent_inner
- StreamingTTSConsumer created when voice input + auto-TTS + provider active
- Delta callback teed to both text stream consumer and TTS consumer
- TTS-only delta callback installed when text streaming is off
- finish() called from executor; wait_complete() in async context after
- Barge-in aborts the consumer at all three interrupt detection points
- Runner-level _send_voice_reply suppressed when streaming TTS completed

Tests (tests/gateway/test_streaming_tts_consumer.py):
- 15 focused tests: adapter defaults, lifecycle, ordered chunks,
  unsupported/No-streamer fallback, abort idempotency, late-chunk drop,
  pre/post-audio failure, concurrent-turn isolation, think-block suppression,
  queue backpressure

Does not touch desktop/TUI code or add config flags. Plugin TTS provider
stream() metadata gap (#47896) is explicitly out of scope — built-in
ElevenLabs/OpenAI PCM streamers are the first consumers.

Refs: #60671, #47896
2026-07-28 22:31:40 -07:00
Nick Edson
201be3e546 fix(gateway): reserve retrying Photon listener ownership 2026-07-28 21:45:52 -07:00
Nick Edson
c608a6937a fix(gateway): release failed Photon listener claims 2026-07-28 21:45:52 -07:00
Nick Edson
a908c62d28 fix(gateway): guard Photon sidecar listener collisions 2026-07-28 21:45:52 -07:00
Nick Edson
cf19ac8ff3 fix(gateway): prevent duplicate Photon sidecar storms 2026-07-28 21:45:52 -07:00
Teknium
41233e19c6 fix(gateway): forward failure_reason through the empty-response return path
Sibling of #64686: _run_agent's empty-final_response early return dropped
failure_reason (and #64686 only fixed the non-empty path), so downstream
consumers (TUI billing surface, transient-failure persistence) lost the
structured reason exactly when a failed run produced no text.

Also hardens the two BasePlatformAdapter identity checks (edit_message /
delete_message) with getattr so duck-typed adapters without the attribute
mean 'capability absent', not AttributeError — this was crashing the
send_progress_messages path for minimal adapters and test fakes.
2026-07-28 21:45:44 -07:00
Teknium
07e931fcb4 feat(buzz): WebSocket inbound transport — NIP-42 auth, live DM discovery, poll fallback
Consolidates the native-transport half of PR #73636 by @ScaleLeanChris
onto the merged adapter: persistent NIP-42-authenticated Nostr WebSocket
subscription as the default inbound path (transport=auto|websocket|poll),
kind-44100 membership events for live DM discovery, since-timestamp
resume on reconnect with bounded exponential backoff, and automatic
fallback to CLI polling when the WS can't be established. Events route
through the same _handle_event() pipeline as the poll loop, so de-dupe,
mention gating, p-tag DM latching, and allow-lists behave identically on
both transports. Outbound stays on the CLI (one-shot sends never race a
WS auth handshake — his design).

E2E verified against a real in-process websockets relay: NIP-42
challenge -> signed kind-22242 AUTH (event id re-derived server-side) ->
REQ subscription -> EVENT dispatch -> clean disconnect.

Co-authored-by: ScaleLeanChris <chris@scalelean.com>
2026-07-28 21:45:34 -07:00
Teknium
1b9377b1fd fix(buzz): scoped identity lock, negative name caching, sidebar registration
Follow-up on the salvaged #71610 commits:
- acquire/release a scoped lock on relay_url+pubkey in connect/disconnect
  (IRC pattern) so two profiles can't drive one Buzz identity — duplicate
  replies and split de-dupe state; +2 tests
- negative-cache _resolve_user_name failures so a profile-less pubkey
  doesn't re-hit 'users get' every poll sweep (flagged by @jethac on the PR)
- register user-guide/messaging/buzz in website/sidebars.ts (page was
  unreachable — the #63359 trap)
2026-07-28 17:57:48 -07:00
Rob Zehner
7e1e3b92f8 fix(plugin): classify Buzz DMs by p-tag so un-mentioned DMs dispatch
On hosted relays `buzz dms list` reliably returns [] even when DM
conversations exist, so DMs leaked in via `channels list` (as entries
named "DM" with an empty description) and were seeded chat_type="group".
That put them behind the channel mention gate: "@Chip /whoami" worked
but an un-mentioned DM was silently dropped.

Classify from the Nostr tags of real traffic instead: a message another
user sends in a DM carries a structural ["p", <own pubkey>] tag even
when the text never mentions the agent, while in a real channel a
p-tag-to-self only ever accompanies a visible @mention (typed mention
or reply). A group conversation therefore latches to chat_type="dm" on
the first kind-9 event that is p-tagged to self WITHOUT a visible
mention in the content — guarded by channels-list metadata so a real
community channel (real name / non-empty description) is never
reclassified by a reply or mention that p-tags the agent.

- latch during history seeding too, so a leaked DM bypasses the
  mention gate from the very first poll after connect
- keep `dms list` as a best-effort source, and scan `channels list`
  as a fallback so DM conversations opened mid-run still get watched
- strip a leading @mention in DMs as well, so "@Chip /whoami" keeps
  firing as a slash command after the conversation reclassifies

Refs #68871

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 17:57:48 -07:00
Rob Zehner
66fc2e2a92 feat(plugin): add Buzz (Block/Nostr) platform adapter
Plugin-path adapter (zero core changes) connecting Hermes to a Buzz
community relay via the buzz CLI binary (JSON in/out, arg-list exec,
key passed via env only). Inbound uses a poll loop with per-channel
high-water marks seeded from newest (no history replay), event-id
de-dupe, self-echo suppression by pubkey, and mention gating in
channels (DMs always dispatch). Registers env_enablement, cron
home-channel delivery, and an out-of-process standalone sender,
mirroring the IRC plugin.

Verified against a live relay: connect -> send -> poll -> MessageEvent
round-trip, self-echo suppressed, clean disconnect.

Known limitation: polled inbound (default 4s); a websocket transport
(buzz-ws-client) is a future optimization.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 17:57:48 -07:00
Siddharth Balyan
3cedac00b7
fix(credits): reintroduce grant_spent, gated on an in-session crossing (#73634)
* Revert "fix(credits): remove the 'Grant spent · $X top-up left' notice"

This reverts commit 5dc6a14c14.

* fix(credits): reintroduce grant_spent behind an in-session crossing gate

The removed notice nagged because its condition is a steady state for
accounts living on top-up, the latch is per-session, and the cold-start seed
runs the policy at every session open — every session re-announced
'Grant spent · $X top-up left'. Reintroduce it gated so only a session that
WATCHES the grant run out announces:

- seen_grant_unspent crossing gate, mirroring seen_below_90: opens when the
  session observes the grant meaningfully unspent (>= GRANT_UNSPENT_MIN_MICROS,
  1 cent — portal-seeded states derive micros from float dollars and can carry
  sub-cent residue where headers report exactly spent). Seeds never prime it.
- The gate guards only the show branch and is consumed by the announcement —
  one announcement per crossing. Header flicker (used_fraction None and back)
  clears the sticky line but cannot re-announce; a renewal re-opens the gate.
- new_credits_latch() centralizes the latch shape (agent build, lazy re-init,
  and the policy test helper all build through it).
- Tests: steady-state-open stays silent (seed + policy seams), live crossing
  announces once, flicker/renewal/residual cases locked; the rendering test
  primes the gate and asserts its leg count so a gate regression cannot
  silently shrink coverage.
2026-07-28 21:36:23 +00:00
Atakan
8c5e846536 fix(gateway): bind HTTP auth to routed profiles 2026-07-28 14:22:18 -07:00
atakan g
5423e26a7d fix(gateway): preserve rate-limit failure metadata 2026-07-28 14:19:14 -07:00
Teknium
bb09e3eaac test(tts): adapt salvaged #57681 tempfile-fallback test to registry streamers
Main's streaming path now resolves providers via
tools.tts_streaming.resolve_streaming_provider rather than constructing an
ElevenLabs client inline; stub the registry resolver instead so the test is
provider-agnostic and hermetic.
2026-07-28 14:07:21 -07:00
Que0x
555d4e10ad fix(tts): close temp WAV handle before playback in streaming fallback
`_play_via_tempfile` passes the NamedTemporaryFile *object* to `wave.open()`.
`wave.open()` flushes but does not close a file it did not open itself (by
name), so the OS handle to the temp WAV stays open. On Windows that open write
handle blocks the system player from reading the file and blocks the
`os.unlink()` cleanup (WinError 32, silently swallowed by `except OSError:
pass`), leaving orphaned temp .wav files piling up. The fallback runs whenever
no sounddevice output stream is available (sounddevice not installed / no audio
device), which is common on headless and Windows setups.

Close the temp file handle before invoking the player and again in the finally
block (idempotent), so the player can read the file and `os.unlink()` can
remove it.

Regression test drives `stream_tts_to_speaker` with the sounddevice path forced
unavailable and asserts the temp handle is closed before `play_audio_file` is
called (mutation-verified: reverting the close leaves the handle open at play
time and the test fails). Cross-platform.
2026-07-28 14:07:21 -07:00
Teknium
4c7c51fcb2 refactor(gateway): one media-cache cleanup loop — extend pruning to video + screenshot caches
Follow-up to salvaged PR #56473: dedupe the five cleanup_*_cache bodies
into a shared _cleanup_cache_dir() helper, add cleanup_video_cache() and
cleanup_screenshot_cache() (with get_screenshot_cache_dir()), and drive
all five from a single (name, fn) loop in _start_gateway_housekeeping()
instead of one copy-pasted try/except per cache. Covers the video/
screenshot half of #56427.
2026-07-28 14:07:21 -07:00
vb3
a61edf952f fix(gateway): add cleanup_audio_cache() and wire it into gateway housekeeping
Adds cleanup_audio_cache() to gateway/platforms/base.py and wires it into
_start_gateway_housekeeping() in gateway/run.py, matching the existing
image/document cache cleanup pattern.

Re-implements the fix proposed in #16473, which targeted the now-deprecated
_start_cron_ticker() and has since stalled with a merge conflict against
main. _start_gateway_housekeeping()'s docstring already claimed audio
cleanup ("prunes the image/audio/document cache"); this makes the code
match it.

Co-authored-by: nftpoetrist <264138787+nftpoetrist@users.noreply.github.com>
2026-07-28 14:07:21 -07:00
Zavian
98fea1323b test(gateway): keep Discord document-handling tests hermetic against host DNS
These tests mock the actual download; stub is_safe_url so host DNS/proxy
mappings for cdn.discordapp.com can't decide whether document handling is
exercised.

Residual hunk salvaged from PR #25426 — the rest of that PR's hermeticity
fixes (bedrock botocore fakes, faster_whisper module fakes, kittentts
plain-sequence PCM, web-provider SSRF stubs) have since landed on main in
equivalent form.
2026-07-28 14:07:21 -07:00
轩辕
368d3488d2 fix(weixin): preserve voice transcript origin 2026-07-28 14:06:56 -07:00
Kewe63
cf6ff8cd30 test(gateway/weixin): add integration handoff regression for #27300 voice routing
Address teknium1 review on PR #47125: the existing adapter-level tests cover
the prerequisites (_download_voice / _collect_media / _extract_text) but not
the gateway-runner handoff. Add TestWeixinVoiceGatewayHandoff covering the
final routing contract:

- An inbound Weixin voice item carrying Tencent Cloud text is surfaced as a
  VOICE MessageEvent whose media is audio/silk (the shape the runner keys off
  to enter Hermes' STT pipeline), exercised through the real _process_message.
- That VOICE event's body does NOT leak Tencent's STT text, so the central
  transcript replaces it rather than being trusted.
- A VOICE/audio/silk event reaches the real GatewayRunner
  _enrich_message_with_transcription (patched as a spy), proving the runner
  handoff the adapter-only tests missed.
2026-07-28 14:06:56 -07:00
Kewe63
cb1798b43c fix(gateway/weixin): route voice messages through Hermes STT instead of trusting Tencent Cloud's text
When WeChat (Weixin) returns a voice_item.text (Tencent Cloud's STT),
Hermes previously trusted that text as the user-visible message body and
skipped downloading the raw audio. For non-Chinese audio that text is
garbage — the original report was a Russian voice message that came
back as English phonemes — and the user sees nonsense as their own
message. International users on the WeChat gateway effectively can't
use voice.

Two short-circuits in gateway/platforms/weixin.py caused this:

  - _download_voice() returned None whenever voice_item.text was set,
    so the raw SILK/Opus audio was never fetched.
  - _extract_text() returned voice_item.text verbatim as the body,
    so even if the audio had been downloaded, the central STT
    pipeline in gateway/run.py never had a chance to replace it.

Fix both: always download the raw audio (the central pipeline picks
it up via _collect_media()), and skip voice items in _extract_text()
so the body comes from Hermes' own mlx-whisper / whisper.cpp /
faster-whisper transcription instead of Tencent's. Behavior is
unchanged when voice_item.text is absent (the original happy path
where audio was already being downloaded).

Tests:
  - 5 new tests in TestWeixinVoiceAlwaysDownloaded covering both
    functions, the _collect_media integration path, and a
    regression guard for the text-item path.
  - 74/74 in tests/gateway/test_weixin.py pass.
2026-07-28 14:06:56 -07:00
Vivaan Dhawan
1c30c57f11 fix(whatsapp): preserve voice notes when STT fails 2026-07-28 14:06:56 -07:00
Zhekinmaksim
a2d8d3afc5 fix(qqbot): always clean up temp stt wav 2026-07-28 14:06:56 -07:00
Zioywishing
b5c2dc7cd2 fix(qqbot): skip voice detection for file uploads (content_type='file')
Minimal fix: add 'if ct == "file": return False' before extension
matching. The original fallback logic is preserved for empty/unknown
content_types. Only the bug case (file uploads with audio extensions)
is fixed.

Removed the over-engineered _looks_like_voice helper.
2026-07-28 14:06:56 -07:00
Zioywishing
e7cc10111a fix(qqbot): keep extension fallback for voice detection, skip only for explicit file uploads
Refined the fix: instead of removing extension-based fallback entirely,
only skip it when content_type is explicitly 'file' (or image/video).
Empty or unknown content_types still fall back to extension matching
as a defensive measure.

- content_type='voice' or 'audio/*' → True (API signal)
- content_type='file' → False (file transfer, never voice)
- content_type='' → extension fallback (defensive)
- content_type=unknown → extension fallback (defensive)

Added _looks_like_voice() module-level helper and comprehensive tests.
2026-07-28 14:06:56 -07:00
Zioywishing
3122dc1744 test(qqbot): update voice detection tests for content_type-only logic
Update TestIsVoiceContentType to match the new behavior:
- Empty content_type with audio extensions → False (no sniffing)
- File upload with audio extension → False
- Added test_file_upload_with_audio_extension for the reported bug case
2026-07-28 14:06:56 -07:00
Kronexoi
73e193c03d fix(line): normalize inbound media types and cache routing 2026-07-28 14:06:56 -07:00
Teknium
7ee111ff9b fix(dingtalk): don't let richText re-derivation clobber VOICE classification
The msg_type_str == "richText" branch reset msg_type to PHOTO/TEXT after
the rich-text item scan had already promoted it (e.g. a native voice item
→ VOICE), dropping voice notes from the auto-STT path. Only re-derive when
the scan left the type at TEXT.

Ports the root-cause analysis from PR #38276 (stale, targeted the deleted
gateway/platforms/dingtalk.py) onto the live plugin adapter, with
regression tests.

Refs #38211 #38219 #38276
2026-07-28 14:06:56 -07:00
raywu
c762314561 fix(dingtalk): add EXT_MAP constant, PHOTO classification for images, card/interactiveCard message handling
- Promote EXT_MAP to a module-level constant for reuse
- Classify DingTalk image messages and image file attachments as PHOTO
- Extract DingTalk card/interactiveCard document link content with defensive parsing
- Handle None, empty, JSON, and plain-string card content fields
- Add coverage: 8 card + 4 interactiveCard + 5 _extract_media = 17 test cases
- Remove a shadowed duplicate TestExtractText class (pytest collected only the later one)
2026-07-28 14:06:56 -07:00
wuli666
1ca1deb7f6 fix(feishu): classify native voice messages as VOICE for auto-transcription
Lark's native "audio" msg_type is an in-app voice recording — uploaded
audio files arrive as "file"/"media". But _resolve_normalized_message_type
resolved the "audio" preferred type to MessageType.AUDIO, which the gateway
treats as a non-transcribed file attachment (run.py: AUDIO -> audio_file_paths,
"never STT"; VOICE -> audio_paths, "always STT"). Result: a Feishu voice
note reached the agent as an untranscribable audio attachment and was
silently ignored — the user's spoken message never became text.

Every other platform that receives native voice notes (Telegram, Discord,
Slack, WhatsApp, Signal, Matrix, WeChat, WeCom, DingTalk, QQ, BlueBubbles,
Mattermost, Yuanbao) classifies them as MessageType.VOICE. Feishu was the
only one classifying them as AUDIO. This is the follow-up to #28993, which
added native voice-note transcription for Discord + DingTalk but did not
cover Feishu.

Return MessageType.VOICE for the "audio" branch. The branch is reached only
for Lark's top-level audio msg_type (set in the normalizer; file uploads map
to "document"), so VOICE is unconditionally correct here — no risk of
auto-transcribing an uploaded music/audio file.

- plugins/platforms/feishu/adapter.py: _resolve_normalized_message_type audio
  branch returns VOICE instead of resolving to AUDIO via mime.
- tests/gateway/test_feishu.py: test_extract_audio_message_downloads_and_caches
  asserted the old AUDIO behavior on a fixture literally named voice.ogg —
  updated to expect VOICE (the corrected classification).
- tests/gateway/test_feishu_voice_message_type.py: new focused regression
  tests (audio->VOICE with and without mime; photo/document/text unaffected).

Rebased onto current main: the Feishu platform moved from the single-file
gateway/platforms/feishu.py into the plugins/platforms/feishu/ package; the
fix applies to the same _resolve_normalized_message_type logic at its new home.

Verified the new voice tests fail when the branch resolves to AUDIO and pass
with VOICE, while the photo/document/text cases are unaffected either way.

Note: classification is mock-tested here; the downstream STT pipeline is the
shared, already-proven path (#28993). End-to-end verification on a live
Feishu account would be a welcome confirmation.

Refs #28993 (sibling-gap: Feishu was the platform left uncovered).
2026-07-28 14:06:56 -07:00
Gille
678916b427 fix(gateway): preserve memory prompt during manual compression 2026-07-29 02:13:41 +05:30
Teknium
76b0ea5118 fix(state): rebuild legacy gateway_routing PK; guard session_store in dispatch hook
Two log-spam bugs found in live gateway logs:

1. gateway_routing UNIQUE-constraint spam (261 warnings in one errors.log):
   early builds of the #59203 routing-index migration created
   gateway_routing with 'session_key TEXT PRIMARY KEY' and no scope
   column. _reconcile_columns() ADDs the missing scope column but SQLite
   cannot ALTER a primary key, so the shipped composite
   PRIMARY KEY (scope, session_key) never lands on those databases. Both
   write paths then fail on every save:
   - save_gateway_routing_entry: 'ON CONFLICT clause does not match any
     PRIMARY KEY or UNIQUE constraint'
   - replace_gateway_routing_entries: 'UNIQUE constraint failed:
     gateway_routing.session_key' whenever the same session_key exists
     under another scope (e.g. test-suite scopes leaked into a live DB).
   New _heal_gateway_routing_pk() rebuilds the table once with the
   composite key, preserving rows (newest wins on collisions, NULL scope
   coalesced to ''). Same one-time-heal pattern as the #51646 active-
   column repair. Verified E2E against a copy of a real affected state.db.

2. pre_gateway_dispatch warned ''GatewayRunner' object has no attribute
   'session_store'' and silently dropped the hook for every message on
   partially-initialized runners (bare object.__new__ runners in tests,
   and any future init-order change). Pass
   getattr(self, 'session_store', None) so the hook always fires
   (pitfall #17 pattern).

Both regression tests fail without their fixes (sabotage-verified).
2026-07-28 11:58:54 -07:00
Teknium
ea80b557ae fix: route busy-steer voice through the shared out-of-band STT choke point
Follow-up for salvaged #65023/#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from #67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of #58780. Adds extract_media gating tests for #44826 and the
contributor mapping for chefboyrdave21.
2026-07-28 11:57:11 -07:00
Frowtek
753d0d77e3 fix(gateway): keep the STT echo ledger across pending-media merges
_invalidate_pending_stt_cache() clears the gateway-side transcription
cache when merge_pending_message_event() folds a follow-up message into a
still-pending event, so the next transcription picks up the merged text
and attachments.  It also cleared _gateway_pending_stt_echo_sent, but that
flag is not derived state — it records that the transcript was already
delivered to the user.

Dropping it makes the re-run transcription echo the earlier notes a second
time.  Both merge branches are affected, including the text-only follow-up
case where no new audio arrived at all: there the cache is invalidated,
the same voice note is transcribed again (a second paid STT call) and the
same line is echoed again.

Sequence:

  1. voice note arrives, interrupt monitor transcribes it and echoes
     '🎙️ "hello"'
  2. user sends a follow-up while the turn is still pending, so it merges
  3. drain path re-transcribes and echoes '🎙️ "hello"' a second time

Keep the ledger out of the invalidation set and track it as a count of
already-echoed transcripts instead of a single boolean.  A count is what
the merge case actually needs: re-running transcription over the extended
media list returns the earlier transcripts as a prefix of the new one, so
echoing only the unsent tail suppresses the repeat while still surfacing a
newly merged voice note.  A count rather than a set of seen values, so two
separate notes that transcribe identically stay two distinct deliveries —
covered by test_pending_stt_merge_echoes_two_identical_transcripts.

The guard stays within the 12-line window that
test_all_gateway_transcript_echo_sends_are_gated enforces over run.py.
2026-07-28 11:57:11 -07:00
canorionen
0fd0161dfd fix(gateway): steer busy voice follow-ups after STT 2026-07-28 11:57:11 -07:00
izumi0uu
aa40f16d3e fix(gateway): transcribe clarify voice replies 2026-07-28 11:57:11 -07:00
Teknium
f76b2b47aa fix(gateway): pass channel_prompt into voice-channel STT events; guard empty transcripts
Two hand-written fixes in the voice input path:

- _handle_voice_channel_input now resolves the bound text channel's
  channel_prompt via the adapter's _resolve_channel_prompt so voice input
  gets the same per-channel context as typed messages (fixes #50149).
- _enrich_message_with_transcription now guards success=True results whose
  transcript is empty/whitespace-only (silence, cut-off, inaudible audio):
  instead of emitting empty quotes the agent gets a clear sentinel note.
  Reimplemented against the current plain-quoted note wording; original
  concept and tests by @deacon-botdoctor in PR #41603 (fixes #41603).
2026-07-28 11:56:37 -07:00
Teknium
a4c9994837 refactor(discord): delegate ffmpeg discovery to shared tools.transcription_tools helper
Keep one owner for PATH/local-prefix ffmpeg discovery: ffmpeg_utils now
delegates to tools.transcription_tools._find_ffmpeg_binary and only adds
the Discord-specific FFMPEG_PATH override and Windows winget fallback on
top (follow-up to PR #60627 by @LauraGPT, fixes #60624).
2026-07-28 11:56:37 -07:00