Commit graph

2896 commits

Author SHA1 Message Date
teknium1
2006cd5895 refactor(gateway): declarative busy_policy on CommandDef replaces hand-written mid-run command chain 2026-07-29 10:53:56 -07:00
teknium1
ed33ebca1d refactor: canonical config loaders for behavioral reads + guarded raw-read primitive (kills the managed-scope/env-expansion drift class)
The disease: ~15 scattered raw yaml.safe_load(config.yaml) reads that
silently miss managed-scope overlay, ${ENV_VAR} expansion, profile-aware
pathing, and root-model normalization. Every new config feature needed an
N-site sweep (incident chain 9cbcc0c9c8732293cf87b0e47a98f91928aa0443). This commit assigns every raw read to an owner and adds a
lint-guard test so the class cannot regrow.

New primitive (additive-only change to hermes_cli/config.py):
  read_user_config_raw(path=None) — reads the user file EXACTLY as
  written; docstring states it is ONLY legal for write-back round-trips
  and raw-file diagnostics. Behavioral reads must use
  load_config()/load_config_readonly().

BEHAVIOR FIXES (class-a sites migrated to a canonical loader — these
previously read values that could DIFFER from the effective config):

  gateway/run.py _try_resolve_fallback_provider → _load_gateway_runtime_config
    keys: fallback_providers/fallback_model (provider, model, base_url,
    api_key). Drift fixed: a managed-pinned fallback chain was ignored;
    an api_key of "${OPENROUTER_API_KEY}" reached the resolver unexpanded.
  gateway/run.py GatewayRunner._load_provider_routing → same loader
    key: provider_routing. Drift fixed: managed-pinned routing prefs and
    ${VAR} templates were ignored.
  gateway/run.py GatewayRunner._load_fallback_model → same loader
    keys: fallback chain. Same drift as above.
  gateway/run.py GatewayRunner._refresh_fallback_model
    keeps the raw primitive (its last-known-good-on-parse-failure contract
    forbids the fail-open loader, which returns {} on a torn write) but now
    applies managed overlay + env expansion inline. Drift fixed: chain
    edits under managed scope / env templates were previously frozen out.
  tui_gateway/server.py _load_cfg (72 behavioral call sites)
    now = raw read + managed overlay (pre-existing) + NEW ${VAR} expansion,
    split from a new _load_cfg_raw() write-back primitive. Drift fixed:
    e.g. custom_prompt: "hello ${VAR}", agent.system_prompt, model,
    api_key/base_url templates reached sessions unexpanded. DEFAULT_CONFIG
    is deliberately NOT merged (callers treat missing keys as unset;
    `_load_cfg() == {}` sentinels and _save_cfg round-trips depend on it).
  tui_gateway/server.py _profile_configured_cwd
    keys: terminal.cwd of a NON-launch profile. Drift fixed: managed
    overlay + ${VAR} expansion now apply (load_config() would resolve the
    wrong profile's home, so the raw primitive + inline pipeline is used).
  plugins/platforms/telegram/adapter.py _reload_dm_topics_from_config
    → load_config_readonly(). keys: platforms.telegram.extra.dm_topics.
    Drift fixed: managed overlay + profile-aware pathing + expansion.
  plugins/memory/holographic _load_plugin_config → load_config_readonly().
    keys: plugins.hermes-memory-store.*. Same drift class.

WRITE-BACK ROUND-TRIPS (class-b: stay raw BY DESIGN via read_user_config_raw;
merging defaults/overlay would pollute the saved user file):
  gateway/slash_commands.py: model persist x2, _save_gateway_config_key,
    memory/skills write_approval toggles
  gateway/platforms/yuanbao.py auto-sethome
  tui_gateway/server.py _write_config_key + all cfg→_save_cfg blocks
    (reasoning show/hide/full/clamp, details_mode[.section], prompt)
    → new _load_cfg_raw()
  plugins/memory/holographic save_config

RAW-FILE DIAGNOSTICS + presence-sensitive bridges (class-c: stay raw,
now via the shared primitive with an explanatory comment):
  hermes_cli/doctor.py x5 (model validation, stale-root-keys, .env drift,
    deprecation sweep, memory-provider probe — the latter two keep their
    inline managed overlay where they had one)
  gateway/run.py _bridge_max_turns_from_config and the module-level
    TERMINAL_*/HERMES_* env bridge (bridging merged defaults would export
    all of DEFAULT_CONFIG into the environment; both keep their inline
    overlay + expansion)
  hermes_cli/send_cmd.py env bridge (same presence-sensitivity)
  hermes_cli/gateway.py multiplex-conflict probe (reads the DEFAULT root's
    config, not the active profile's — load_config is the wrong owner)
  hermes_cli/profiles.py / hermes_cli/web_server.py / tools/wake_word.py
    multi-profile reads (load_config targets only the ACTIVE profile home)
  cron/jobs.py _resolve_default_model_snapshot and cron/scheduler.py
    run_job config read keep their existing inline overlay+expansion but
    now share the primitive (their fail-open + last-value semantics and
    the deliberate no-defaults merge are preserved exactly).

Failure-semantics audit: every migrated site preserves its exact previous
behavior on missing file ({} / early return) and parse failure (raise into
the caller's existing except, warn, last-known-good, or fail-open) —
read_user_config_raw intentionally mirrors bare open()+safe_load semantics
(raises on parse errors, {} only on FileNotFoundError/non-dict root).

Guard: tests/hermes_cli/test_config_read_guard.py scans the tree for
yaml.safe_load within 6 lines of a 'config.yaml' reference outside an
explicit ALLOWLIST (hermes_cli/config.py, gateway/config.py, gateway/run.py
fallback path, hermes_cli/managed_scope.py which reads the MANAGED file,
gateway/readiness.py parse-health probe) and fails on new offenders.

E2E: tests/hermes_cli/test_config_loader_e2e.py runs a subprocess with a
temp HERMES_HOME (config.yaml containing ${E2E_PROMPT_SUFFIX}) plus a
HERMES_MANAGED_DIR overlay pinning agent.reasoning_effort, asserting
tui _load_cfg resolves "hello world"/"high" while _load_cfg_raw +
_save_cfg round-trip the template and user value verbatim with no
managed/default leakage.
2026-07-29 10:53:29 -07:00
teknium1
bf15259e33 refactor(gateway): shared media-cache mime dispatch for adapter downloads (per-adapter overrides preserve historical mappings) 2026-07-29 10:14:59 -07:00
teknium1
7b6a67f82e refactor(gateway): extract duplicated StreamConsumerConfig setup into _build_stream_consumer_config (preserves both sites' divergent fallback semantics via parameter) 2026-07-29 10:14:45 -07:00
teknium1
3d48f893da refactor: single build_subprocess_env() factory for all child-process spawns (profile + secret-scrub single owner) 2026-07-29 10:14:11 -07:00
teknium1
6f7f7cd064 refactor: use shared strip_ansi for inline ANSI regexes 2026-07-29 10:13:50 -07:00
teknium1
7c198c5e44 refactor: single shared Retry-After parser 2026-07-29 10:13:50 -07:00
teknium1
19055492aa fix: route stray HERMES_HOME hardcodes through get_hermes_home() (profile + native-Windows safety) 2026-07-29 09:33:48 -07:00
victor-kyriazakos
1773752c8c Merge remote-tracking branch 'origin/main' into feat/gateway-health-diagnostics-monitoring
# Conflicts:
#	uv.lock
2026-07-29 15:37:14 +00:00
Enough1122
5cc5c58e01 fix(gateway): flush pending memory writes before session teardown (#73297)
The gateway's /reset cleanup path called shutdown_memory_provider without
first draining the memory manager's serialized background write worker.
shutdown_all only gives that worker a bounded (~5s) drain and abandons
whatever is still queued past it, so a /reset could silently drop writes
the session had already handed off -- the next session then loaded
stale MEMORY.md.

The CLI exit path already drains via MemoryManager.flush_pending before
shutdown; this PR pins the same contract on the gateway cleanup path.

Cleanup now calls agent._memory_manager.flush_pending(timeout=10) before
the existing shutdown_memory_provider step. The flush is best-effort:
a flush failure must never block teardown, so it is wrapped in
try/except and the existing shutdown path remains the fallback.

Closes #73297
2026-07-29 17:21:55 +05:30
kshitijk4poor
222ea2b6c9 refactor: fold simplify-code review findings
- extract _commit_registry/_note_refresh_failure shared by the background
  worker and foreground stage-4 (identical 4-step success + failure paths
  were duplicated); worker now commits under _models_dev_fetch_lock so a
  failing background refresh can never re-arm the backoff immediately
  after a successful force_refresh committed (unsynchronized-write race)
- add should_clear_context_pin_async to hermes_cli/route_identity.py
  (matching the get_model_context_length_async precedent) and use it at
  the 4 async gateway sites instead of inline asyncio.to_thread wraps;
  the sync _format_session_info site keeps the sync call (already
  off-loop via its callers' to_thread)
- test the background-refresh success path (the PR's primary new
  behavior): disk saved, mem cache swapped, backoff cleared, in_flight
  reset — mutation-checked
- replace the race-prone spin-wait on _models_dev_refresh_in_flight with
  a named-thread join in the backoff test
2026-07-29 17:13:48 +05:30
StellarisW
8c50aaceb6 fix(gateway): keep models.dev refreshes off event loop 2026-07-29 17:13:48 +05:30
kshitijk4poor
23e44a2843 fix: consolidate agent-history preservation into shutdown_flush module
Follow-up for salvaged PRs #73400 + #73372:
- Move _preserve_agent_history_on_shutdown logic into
  gateway/shutdown_flush.py as flush_agent_history_to_file()
- Reuse the hardened _write_payload (atomic writes, fsync, private
  permissions, UUID filenames) from #73372 instead of the plain
  open()/write() from #73400
- Replace os.environ.get('HERMES_HOME') with get_hermes_home() for
  profile-safe path resolution
- Update tests to test the real function directly
- Net: removes 37 lines from gateway/run.py, unifies both fix paths
  in a single module with consistent atomic-write guarantees
2026-07-29 16:45:35 +05:30
Baophan00
40837e2dd0 Fix #72680 (retargeted): preserve agent._session_messages on shutdown flush failure
The previous attempt (#73171) snapshotted GatewayRunner._pending_messages,
which on current main has no writers (commit f6736ced8 removed its write
path; interrupt delivery uses adapter._pending_messages instead). The live
container is the per-agent agent._session_messages, flushed via
_flush_messages_to_session_db. When that flush raises (FTS/SQLite corruption,
the disk=0/memory=N state from #72680), the in-memory transcript is lost when
the process exits.

Retarget the preservation to the real path: in _finalize_shutdown_agents, wrap
the _flush call; on exception, dump agent._session_messages to an external JSON
recovery snapshot under $HERMES_HOME/shutdown-recovery/ (tagged issue=#72680)
so an operator can salvage it after repairing state.db. The dump is fully
guarded (non-fatal) so shutdown never blocks on a best-effort backup.

This directly addresses the reviewer note on #73171: retarget to the actual
cached-agent history (agent._session_messages) and prove a stale DB + shutdown
leaves a recoverable transcript.

Regression tests: tests/gateway/test_session_messages_shutdown_preserve.py
- flush raises -> recovery file written with session_id + messages
- healthy flush -> no recovery file
- write error -> non-fatal, no raise

Fixes #72680
2026-07-29 16:45:35 +05:30
kshitij
720cdd1d14 refactor: use atomic_json_write instead of hand-rolled _write_payload
Replace 20 lines of manual os.open/O_EXCL/fdopen/fsync/os.replace with the
existing atomic_json_write() from utils.py, which is already used by 6+
modules and handles temp-file creation, fsync, atomic replace, mode
control, and owner preservation. The only novel helper (_fsync_directory)
is retained — atomic_json_write does not do directory fsync.

Update test_flush_write_failure_leaves_no_recovery_file to monkeypatch
utils.os.replace (the new call path) instead of gateway.shutdown_flush.os.replace.
2026-07-29 11:16:09 +05:30
deacon-botdoctor
72024950cf fix(gateway): harden shutdown message flush 2026-07-29 11:16:09 +05:30
Teknium
2c771be406 fix(gateway): dual-stack webhook bind for wecom/msgraph/whatsapp_cloud/teams/telegram siblings
Same class of bug as the LINE adapter (NS-603): defaulting the webhook
bind to "0.0.0.0" (or hardcoding it) binds IPv4 ONLY, so the listener
is unreachable over IPv6-only private networks such as Fly.io 6PN.

- wecom callback_adapter: DEFAULT_HOST None; config.py env seed no
  longer forces 0.0.0.0 when WECOM_CALLBACK_HOST is unset.
- msgraph_webhook: DEFAULT_HOST None; the allowed_source_cidrs
  requirement still fires for the all-interfaces default (host=None is
  treated as network-accessible).
- whatsapp_cloud: DEFAULT_WEBHOOK_HOST None.
- teams: hardcoded 0.0.0.0 TCPSite bind → _DEFAULT_HOST=None with new
  TEAMS_HOST / extra.host override (mirrors LINE_HOST pattern).
- telegram: hardcoded listen="0.0.0.0" → default "" (tornado
  bind_sockets opens one socket per address family; verified against
  PTB 22.6/tornado) with new TELEGRAM_WEBHOOK_HOST / extra.webhook_host
  override.

Explicit host overrides everywhere are preserved; empty/unset collapses
to the dual-stack default. "::" remains a bad substitute on
bindv6only=1 hosts (see LINE adapter comment).
2026-07-28 22:42:41 -07:00
Teknium
a65494ed00 fix(gateway): treat pid=None lifecycle sentinel as unknown ownership in mark_exited
The ownership guard let a sentinel with pid=None pass as self-owned, so
an exiting life could clobber evidence of unknown provenance with a
clean-exit claim. Tighten: only rewrite when the sentinel pid matches
os.getpid() exactly; pid=None (or malformed) is left untouched. Adds
tests for the pid=None no-op and the own-pid rewrite paths.
2026-07-28 22:41:09 -07:00
Shannon Sands
6459b8df76 fix(gateway): use no-kill _pid_exists probe in lifecycle ledger
scripts/check-windows-footguns.py (blocking CI lint) rightly flagged the
os.kill(pid, 0) liveness probe: on Windows sig=0 collides with
CTRL_C_EVENT and GenerateConsoleCtrlEvent hard-kills the target's whole
console group (bpo-14484) — a forensics module must never be able to
kill the process it's checking on. Route through gateway.status._pid_exists,
the repo's canonical psutil-backed no-kill probe.
2026-07-28 22:41:09 -07:00
Shannon Sands
9c76c133b7 fix(gateway): detect and report unclean shutdowns via lifecycle ledger (NS-608)
Hosted agents that die uncleanly (kernel OOM kill, SIGKILL, whole-VM
death) leave no trace: shutdown_forensics only covers graceful signals,
gateway-exit-diag.log only covers exit paths that actually run, and the
VM reboot wipes dmesg before anyone can capture it. NS-608 (BlueAtlas
hourly crash cycle, July 12-15) took days of manual log correlation to
classify because nothing recorded 'the previous life ended violently'.

Add gateway/lifecycle_ledger.py — a sentinel state machine persisted to
<HERMES_HOME>/state/gateway.lifecycle.json:

- start_gateway() claims the sentinel (phase=running) right after the
  PID-file/runtime-lock claim, and reports any prior life that never
  reached an exit path as gateway.previous_unclean_exit in
  gateway-exit-diag.log + a WARNING log line.
- Every exit funnel marks the sentinel exited with a reason:
  _exit_after_graceful_shutdown (graceful_shutdown), the shutdown
  watchdog (shutdown_watchdog), and the loop-liveness watchdog
  (loop_liveness_watchdog).
- Ownership-guarded for --replace takeovers: a live matching owner is
  never reported dead, and the old life cannot clobber the
  replacement's freshly claimed sentinel on its way out.

The 30s loop heartbeat now embeds a cheap /proc memory sample (own RSS,
MemAvailable, swap used) so every unclean-death report carries a
'memory N seconds before death' snapshot; the detector flags
suspected_oom when the last sample shows <64MiB or <5% available.

container-boot.log lines gain prior_exit=clean|unclean|unknown per
profile, stamping unclean container deaths into the volume-persisted
boot log where support can grep for them.

Tests: tests/gateway/test_lifecycle_ledger.py (16 cases) + 4 new
container-boot annotation cases. Existing watchdog/forensics/boot
suites all green; ruff clean.
2026-07-28 22:41:09 -07:00
Carl Taylor
3a4aa2f8e6 feat(gateway): streaming TTS adapter contract and consumer (#60671)
Add an opt-in streaming-audio adapter seam to BasePlatformAdapter so
voice-capable gateway platforms (LiveKit, Discord voice, future adapters)
can consume LLM output as streaming PCM audio before the full response
completes, dropping perceived voice latency from ~2-3.5s to ~500-800ms.

Adapter contract (gateway/platforms/base.py):
- AudioFormat dataclass: declared sample_rate, channels, sample_width
- StreamingTTSHandle: opaque handle with audible/aborted flags
- supports_streaming_tts / begin_streaming_tts / write_streaming_tts
  / finish_streaming_tts / abort_streaming_tts
- All default to unsupported/no-op so existing adapters are source-compatible
- Per-turn _streaming_tts_completed_chats set suppresses duplicate whole-file
  auto-TTS when streaming succeeded; cleared after turn completion

Gateway consumer (gateway/streaming_tts_consumer.py):
- StreamingTTSConsumer: bridges sync agent deltas to async adapter audio sink
- Uses existing SentenceChunker (no competing parser)
- Thread-safe bounded queue; on_delta never blocks the agent worker thread
- Resolves configured streaming provider via resolve_streaming_provider()
- Serialises clause playback in order; flushes tail on completion
- Pre-audio failure: completed=False (falls back to whole-file TTS)
- Post-audio failure: completed=True, partial=True (no replay from start)
- Abort is idempotent; late chunks silently dropped
- Per-turn state isolated across concurrent chats

Gateway integration (gateway/run.py):
- message_type parameter threaded through _run_agent -> _run_agent_inner
- StreamingTTSConsumer created when voice input + auto-TTS + provider active
- Delta callback teed to both text stream consumer and TTS consumer
- TTS-only delta callback installed when text streaming is off
- finish() called from executor; wait_complete() in async context after
- Barge-in aborts the consumer at all three interrupt detection points
- Runner-level _send_voice_reply suppressed when streaming TTS completed

Tests (tests/gateway/test_streaming_tts_consumer.py):
- 15 focused tests: adapter defaults, lifecycle, ordered chunks,
  unsupported/No-streamer fallback, abort idempotency, late-chunk drop,
  pre/post-audio failure, concurrent-turn isolation, think-block suppression,
  queue backpressure

Does not touch desktop/TUI code or add config flags. Plugin TTS provider
stream() metadata gap (#47896) is explicitly out of scope — built-in
ElevenLabs/OpenAI PCM streamers are the first consumers.

Refs: #60671, #47896
2026-07-28 22:31:40 -07:00
Nick Edson
201be3e546 fix(gateway): reserve retrying Photon listener ownership 2026-07-28 21:45:52 -07:00
Nick Edson
c608a6937a fix(gateway): release failed Photon listener claims 2026-07-28 21:45:52 -07:00
Nick Edson
a908c62d28 fix(gateway): guard Photon sidecar listener collisions 2026-07-28 21:45:52 -07:00
Nick Edson
cf19ac8ff3 fix(gateway): prevent duplicate Photon sidecar storms 2026-07-28 21:45:52 -07:00
Teknium
41233e19c6 fix(gateway): forward failure_reason through the empty-response return path
Sibling of #64686: _run_agent's empty-final_response early return dropped
failure_reason (and #64686 only fixed the non-empty path), so downstream
consumers (TUI billing surface, transient-failure persistence) lost the
structured reason exactly when a failed run produced no text.

Also hardens the two BasePlatformAdapter identity checks (edit_message /
delete_message) with getattr so duck-typed adapters without the attribute
mean 'capability absent', not AttributeError — this was crashing the
send_progress_messages path for minimal adapters and test fakes.
2026-07-28 21:45:44 -07:00
Atakan
8c5e846536 fix(gateway): bind HTTP auth to routed profiles 2026-07-28 14:22:18 -07:00
atakan g
5423e26a7d fix(gateway): preserve rate-limit failure metadata 2026-07-28 14:19:14 -07:00
Teknium
4c7c51fcb2 refactor(gateway): one media-cache cleanup loop — extend pruning to video + screenshot caches
Follow-up to salvaged PR #56473: dedupe the five cleanup_*_cache bodies
into a shared _cleanup_cache_dir() helper, add cleanup_video_cache() and
cleanup_screenshot_cache() (with get_screenshot_cache_dir()), and drive
all five from a single (name, fn) loop in _start_gateway_housekeeping()
instead of one copy-pasted try/except per cache. Covers the video/
screenshot half of #56427.
2026-07-28 14:07:21 -07:00
vb3
a61edf952f fix(gateway): add cleanup_audio_cache() and wire it into gateway housekeeping
Adds cleanup_audio_cache() to gateway/platforms/base.py and wires it into
_start_gateway_housekeeping() in gateway/run.py, matching the existing
image/document cache cleanup pattern.

Re-implements the fix proposed in #16473, which targeted the now-deprecated
_start_cron_ticker() and has since stalled with a merge conflict against
main. _start_gateway_housekeeping()'s docstring already claimed audio
cleanup ("prunes the image/audio/document cache"); this makes the code
match it.

Co-authored-by: nftpoetrist <264138787+nftpoetrist@users.noreply.github.com>
2026-07-28 14:07:21 -07:00
轩辕
368d3488d2 fix(weixin): preserve voice transcript origin 2026-07-28 14:06:56 -07:00
Kewe63
cb1798b43c fix(gateway/weixin): route voice messages through Hermes STT instead of trusting Tencent Cloud's text
When WeChat (Weixin) returns a voice_item.text (Tencent Cloud's STT),
Hermes previously trusted that text as the user-visible message body and
skipped downloading the raw audio. For non-Chinese audio that text is
garbage — the original report was a Russian voice message that came
back as English phonemes — and the user sees nonsense as their own
message. International users on the WeChat gateway effectively can't
use voice.

Two short-circuits in gateway/platforms/weixin.py caused this:

  - _download_voice() returned None whenever voice_item.text was set,
    so the raw SILK/Opus audio was never fetched.
  - _extract_text() returned voice_item.text verbatim as the body,
    so even if the audio had been downloaded, the central STT
    pipeline in gateway/run.py never had a chance to replace it.

Fix both: always download the raw audio (the central pipeline picks
it up via _collect_media()), and skip voice items in _extract_text()
so the body comes from Hermes' own mlx-whisper / whisper.cpp /
faster-whisper transcription instead of Tencent's. Behavior is
unchanged when voice_item.text is absent (the original happy path
where audio was already being downloaded).

Tests:
  - 5 new tests in TestWeixinVoiceAlwaysDownloaded covering both
    functions, the _collect_media integration path, and a
    regression guard for the text-item path.
  - 74/74 in tests/gateway/test_weixin.py pass.
2026-07-28 14:06:56 -07:00
Vivaan Dhawan
1c30c57f11 fix(whatsapp): preserve voice notes when STT fails 2026-07-28 14:06:56 -07:00
Zhekinmaksim
a2d8d3afc5 fix(qqbot): always clean up temp stt wav 2026-07-28 14:06:56 -07:00
Zioywishing
fef1b8cbbe docs(qqbot): clarify comment on file upload guard 2026-07-28 14:06:56 -07:00
Zioywishing
b5c2dc7cd2 fix(qqbot): skip voice detection for file uploads (content_type='file')
Minimal fix: add 'if ct == "file": return False' before extension
matching. The original fallback logic is preserved for empty/unknown
content_types. Only the bug case (file uploads with audio extensions)
is fixed.

Removed the over-engineered _looks_like_voice helper.
2026-07-28 14:06:56 -07:00
Zioywishing
e7cc10111a fix(qqbot): keep extension fallback for voice detection, skip only for explicit file uploads
Refined the fix: instead of removing extension-based fallback entirely,
only skip it when content_type is explicitly 'file' (or image/video).
Empty or unknown content_types still fall back to extension matching
as a defensive measure.

- content_type='voice' or 'audio/*' → True (API signal)
- content_type='file' → False (file transfer, never voice)
- content_type='' → extension fallback (defensive)
- content_type=unknown → extension fallback (defensive)

Added _looks_like_voice() module-level helper and comprehensive tests.
2026-07-28 14:06:56 -07:00
Zioywishing
b74c033ca1 fix(qqbot): stop routing file uploads through STT pipeline
The _is_voice_content_type() heuristic matched audio file extensions
(.wav, .mp3, .ogg, etc.) even when the QQ Bot API explicitly reported
content_type='file'.  This caused files sent via QQ's file-transfer
feature to be routed through the speech-to-text pipeline instead of
being saved as regular attachments.

The QQ Bot API already distinguishes voice messages (content_type=
'voice') from file uploads (content_type='file'), so filename-based
extension sniffing is unnecessary and harmful.

Removed the _VOICE_EXTENSIONS fallback; now only content_type is
checked.

Closes #XXXX
2026-07-28 14:06:56 -07:00
seamusmore
d0c6353999 fix(feishu): voice-note duration on upload, turn-scoped TTS dedup, audio path dedup
Trimmed cherry-pick of PR #40592 (duration + dedup hunks only; the
voice-classification hunk duplicates #29235 and the send_voice Opus
rewrite is out of scope for this inbound-focused PR):

- adapter.py: ffprobe duration (off-loop) attached to Feishu voice
  uploads via _build_file_upload_body(duration=...) and the audio
  message payload (#16524, #8300)
- gateway/run.py: TTS dedup narrowed to the current turn;
  _enrich_message_with_transcription dedups repeated audio paths

Refs #40592 #16524 #8300
2026-07-28 14:06:56 -07:00
Gille
678916b427 fix(gateway): preserve memory prompt during manual compression 2026-07-29 02:13:41 +05:30
Teknium
76b0ea5118 fix(state): rebuild legacy gateway_routing PK; guard session_store in dispatch hook
Two log-spam bugs found in live gateway logs:

1. gateway_routing UNIQUE-constraint spam (261 warnings in one errors.log):
   early builds of the #59203 routing-index migration created
   gateway_routing with 'session_key TEXT PRIMARY KEY' and no scope
   column. _reconcile_columns() ADDs the missing scope column but SQLite
   cannot ALTER a primary key, so the shipped composite
   PRIMARY KEY (scope, session_key) never lands on those databases. Both
   write paths then fail on every save:
   - save_gateway_routing_entry: 'ON CONFLICT clause does not match any
     PRIMARY KEY or UNIQUE constraint'
   - replace_gateway_routing_entries: 'UNIQUE constraint failed:
     gateway_routing.session_key' whenever the same session_key exists
     under another scope (e.g. test-suite scopes leaked into a live DB).
   New _heal_gateway_routing_pk() rebuilds the table once with the
   composite key, preserving rows (newest wins on collisions, NULL scope
   coalesced to ''). Same one-time-heal pattern as the #51646 active-
   column repair. Verified E2E against a copy of a real affected state.db.

2. pre_gateway_dispatch warned ''GatewayRunner' object has no attribute
   'session_store'' and silently dropped the hook for every message on
   partially-initialized runners (bare object.__new__ runners in tests,
   and any future init-order change). Pass
   getattr(self, 'session_store', None) so the hook always fires
   (pitfall #17 pattern).

Both regression tests fail without their fixes (sabotage-verified).
2026-07-28 11:58:54 -07:00
Teknium
ea80b557ae fix: route busy-steer voice through the shared out-of-band STT choke point
Follow-up for salvaged #65023/#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from #67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of #58780. Adds extract_media gating tests for #44826 and the
contributor mapping for chefboyrdave21.
2026-07-28 11:57:11 -07:00
Frowtek
753d0d77e3 fix(gateway): keep the STT echo ledger across pending-media merges
_invalidate_pending_stt_cache() clears the gateway-side transcription
cache when merge_pending_message_event() folds a follow-up message into a
still-pending event, so the next transcription picks up the merged text
and attachments.  It also cleared _gateway_pending_stt_echo_sent, but that
flag is not derived state — it records that the transcript was already
delivered to the user.

Dropping it makes the re-run transcription echo the earlier notes a second
time.  Both merge branches are affected, including the text-only follow-up
case where no new audio arrived at all: there the cache is invalidated,
the same voice note is transcribed again (a second paid STT call) and the
same line is echoed again.

Sequence:

  1. voice note arrives, interrupt monitor transcribes it and echoes
     '🎙️ "hello"'
  2. user sends a follow-up while the turn is still pending, so it merges
  3. drain path re-transcribes and echoes '🎙️ "hello"' a second time

Keep the ledger out of the invalidation set and track it as a count of
already-echoed transcripts instead of a single boolean.  A count is what
the merge case actually needs: re-running transcription over the extended
media list returns the earlier transcripts as a prefix of the new one, so
echoing only the unsent tail suppresses the repeat while still surfacing a
newly merged voice note.  A count rather than a set of seen values, so two
separate notes that transcribe identically stay two distinct deliveries —
covered by test_pending_stt_merge_echoes_two_identical_transcripts.

The guard stays within the 12-line window that
test_all_gateway_transcript_echo_sends_are_gated enforces over run.py.
2026-07-28 11:57:11 -07:00
Lumina
6710ce97c4 fix(gateway): gate [[audio_as_voice]] to audio files so images aren't sent as documents
[[audio_as_voice]] is message-global but was applied to every media file in a
message. A non-audio file flagged is_voice is excluded from the embedded-photo
batch and falls through to send_document, so an image in a message that also
carries a voice note arrives as a file attachment instead of an inline photo.
Gate the voice flag on the file extension so one message can carry an inline
image AND a voice bubble. Also wrap the path append in try/except so a crafted
~\x00 path is skipped rather than aborting extraction of all attachments.
2026-07-28 11:57:11 -07:00
canorionen
0fd0161dfd fix(gateway): steer busy voice follow-ups after STT 2026-07-28 11:57:11 -07:00
izumi0uu
aa40f16d3e fix(gateway): transcribe clarify voice replies 2026-07-28 11:57:11 -07:00
Teknium
f76b2b47aa fix(gateway): pass channel_prompt into voice-channel STT events; guard empty transcripts
Two hand-written fixes in the voice input path:

- _handle_voice_channel_input now resolves the bound text channel's
  channel_prompt via the adapter's _resolve_channel_prompt so voice input
  gets the same per-channel context as typed messages (fixes #50149).
- _enrich_message_with_transcription now guards success=True results whose
  transcript is empty/whitespace-only (silence, cut-off, inaudible audio):
  instead of emitting empty quotes the agent gets a clear sentinel note.
  Reimplemented against the current plain-quoted note wording; original
  concept and tests by @deacon-botdoctor in PR #41603 (fixes #41603).
2026-07-28 11:56:37 -07:00
isheng-eqi
31301c1af7 fix(discord): wire voice input callback at adapter connect time
- Wire adapter._voice_input_callback at connect and reconnect so voice
  transcription is forwarded without requiring /voice join (#60623).
- Add optional text_channel_id and source params to DiscordAdapter
  .join_voice_channel() so automatic/programmatic voice joins can
  establish the text-channel binding needed by _handle_voice_channel_input.
- Add TestVoiceInputCallbackWiring: asserts callback wiring on startup
  and reconnect for Discord adapters with voice attributes.
2026-07-28 11:56:37 -07:00
isheng
1b0836c7c7 fix(discord): wire voice input callback at adapter connect time
_voice_input_callback was only set in _handle_voice_channel_join, not at adapter connect or reconnect. Voice transcription was logged but never forwarded as an inbound message without explicit /voice join.

Wire the callback at both connect and reconnect paths.

Fixes #60623
2026-07-28 11:56:37 -07:00
Gilad Bauman
1753369f7c fix(gateway): platform-aware auto-TTS output path for native voice bubbles
Salvaged from PR #62040 (@giladbau), simplified per post-#73072 main: the
central _repair_ogg_container transcode makes an explicit .ogg output path
sufficient — no target_platform plumbing through the TTS tool needed.

Root cause (class-level): both gateway auto-TTS delivery call sites relied
on the TTS tool reading HERMES_SESSION_PLATFORM to pick Ogg/Opus vs MP3,
but that contextvar is cleared by _clear_session_env before the base
adapter's post-handler auto-TTS block runs, so want_opus was always False
on that path → MP3 → Telegram sent an audio attachment instead of a native
voice bubble (#57049, #36685). The runner's _send_voice_reply had the
sibling bug: it hardcoded .ogg for Telegram only, leaving Matrix and
Feishu runner voice replies as MP3 (#14841, #45557).

Fix: new build_auto_tts_output_path(platform) in gateway/platforms/base.py
hands an explicit .ogg temp path when the platform is in the TTS tool's
OPUS_VOICE_PLATFORMS set (single source of truth — telegram/matrix/feishu/
whatsapp/signal), .mp3 otherwise. Used by BOTH delivery call sites:
- BasePlatformAdapter auto-TTS block (also honors the tool's success flag
  and cleans up requested + returned paths)
- GatewayRunner._send_voice_reply (replaces the telegram-only ternary)

Fixes #57049
Fixes #36685
Refs #14841 #45557
2026-07-28 11:55:48 -07:00