Follow-up for salvaged #65023/#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from #67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of #58780. Adds extract_media gating tests for #44826 and the
contributor mapping for chefboyrdave21.
_invalidate_pending_stt_cache() clears the gateway-side transcription
cache when merge_pending_message_event() folds a follow-up message into a
still-pending event, so the next transcription picks up the merged text
and attachments. It also cleared _gateway_pending_stt_echo_sent, but that
flag is not derived state — it records that the transcript was already
delivered to the user.
Dropping it makes the re-run transcription echo the earlier notes a second
time. Both merge branches are affected, including the text-only follow-up
case where no new audio arrived at all: there the cache is invalidated,
the same voice note is transcribed again (a second paid STT call) and the
same line is echoed again.
Sequence:
1. voice note arrives, interrupt monitor transcribes it and echoes
'🎙️ "hello"'
2. user sends a follow-up while the turn is still pending, so it merges
3. drain path re-transcribes and echoes '🎙️ "hello"' a second time
Keep the ledger out of the invalidation set and track it as a count of
already-echoed transcripts instead of a single boolean. A count is what
the merge case actually needs: re-running transcription over the extended
media list returns the earlier transcripts as a prefix of the new one, so
echoing only the unsent tail suppresses the repeat while still surfacing a
newly merged voice note. A count rather than a set of seen values, so two
separate notes that transcribe identically stay two distinct deliveries —
covered by test_pending_stt_merge_echoes_two_identical_transcripts.
The guard stays within the 12-line window that
test_all_gateway_transcript_echo_sends_are_gated enforces over run.py.
Two hand-written fixes in the voice input path:
- _handle_voice_channel_input now resolves the bound text channel's
channel_prompt via the adapter's _resolve_channel_prompt so voice input
gets the same per-channel context as typed messages (fixes#50149).
- _enrich_message_with_transcription now guards success=True results whose
transcript is empty/whitespace-only (silence, cut-off, inaudible audio):
instead of emitting empty quotes the agent gets a clear sentinel note.
Reimplemented against the current plain-quoted note wording; original
concept and tests by @deacon-botdoctor in PR #41603 (fixes#41603).
Keep one owner for PATH/local-prefix ffmpeg discovery: ffmpeg_utils now
delegates to tools.transcription_tools._find_ffmpeg_binary and only adds
the Discord-specific FFMPEG_PATH override and Windows winget fallback on
top (follow-up to PR #60627 by @LauraGPT, fixes#60624).
Discord's voice socket needs a brief warm-up before receiving clients
actually hear audio; the first ~100-200ms is lost, clipping the first
word/syllable of TTS playback. Prepend a configurable lead of silence to
speech on both playback paths:
- Mixer path: new _lead_silence_bytes() helper prepends PCM silence
(BYTES_PER_MS constant added to voice_mixer.py) before play_speech, on
both the reply and the pre-tool ack.
- Legacy FFmpegPCMAudio path: apply -af adelay=<ms>:all=1.
Tunable via discord.voice_fx.lead_silence_ms (default 200, 0 disables).
Fixes#66827
- Wire adapter._voice_input_callback at connect and reconnect so voice
transcription is forwarded without requiring /voice join (#60623).
- Add optional text_channel_id and source params to DiscordAdapter
.join_voice_channel() so automatic/programmatic voice joins can
establish the text-channel binding needed by _handle_voice_channel_input.
- Add TestVoiceInputCallbackWiring: asserts callback wiring on startup
and reconnect for Discord adapters with voice attributes.
PR #61407 accesses self._handle_voice_channel_input in _platform_reconnect_watcher. The test mock runner created via _make_runner() must have this attribute.
- tests/gateway/test_base_auto_tts_output_format.py (new): the base
adapter auto-TTS block passes an explicit .ogg output path on every
OPUS_VOICE_PLATFORMS member (parametrized from the tts_tool set — the
single source of truth), keeps .mp3 on non-opus platforms, honors the
tool's success flag, and stays unique/uuid-based.
- tests/gateway/test_auto_voice_reply_format.py: runner _send_voice_reply
parametrized across Matrix/Feishu/WhatsApp/Signal (.ogg) alongside the
existing Telegram/Slack cases; streamed+global-auto-TTS gate regression.
- tests/gateway/test_telegram_voice_caption_markdown.py (new): caption
MarkdownV2 formatting, entity-rejection plain fallback, overflow skip,
and no-caption passthrough (#32029).
- test_voice_command.py filename-uuid contract test updated to point at
build_auto_tts_output_path (construction moved there).
Salvaged from PR #51196 (@55nx954gn6-debug). _should_send_voice_reply only
consulted the runner's _voice_mode dict (/voice on|voice_only|all), so the
global voice.auto_tts config default — which is synced into each adapter's
_auto_tts_default on gateway connect — was invisible to the runner path.
Net effect: with streaming enabled and only global auto-TTS configured
(no per-chat /voice opt-in), the streamed reply consumed the text, the
base adapter's auto-TTS got text_content=None, and no voice reply was
ever sent (#51867/#23983 remainder).
The runner now also asks the adapter's _should_auto_tts_for_chat(chat_id)
(which encodes per-chat /voice on|off overrides over the global default);
an explicit /voice off chat mode remains a hard override.
Refs #51867#23983#51282#13126
Salvaged from PR #68063 (@malaiwah). MatrixAdapter.send_voice now transcodes
any non-Ogg audio to Ogg/Opus at the adapter boundary (best-effort — the
original file is sent unchanged when ffmpeg is unavailable), so MSC3245
voice bubbles render even when a caller hands the adapter MP3/WAV audio.
_matrix_voice_metadata_for_file probing now runs via asyncio.to_thread so
ffprobe/ffmpeg subprocess timeouts can't stall the adapter event loop.
The PR's tools/tts_tool.py want_opus hunk was dropped: main's
OPUS_VOICE_PLATFORMS set (PR #73072) already includes matrix; the Matrix
opus-routing test is kept.
Refs #14841
Salvaged from PR #68063 (base commit, runner-path hunks superseded by the
platform-aware OPUS_VOICE_PLATFORMS fix in this branch). Element and other
Matrix clients render voice bubbles more reliably when m.audio events carry
duration and waveform metadata; probe both best-effort via ffprobe/ffmpeg.
Review follow-up: the spoken script is for synthesis only. Caption
eligibility and payload stay on the original reply, so a long reply
whose normalized script fits the 1024 char limit is still delivered
in full as its own message. Adds the long-original/short-normalized
regression case to the auto-TTS caption tests.
iOS Signal delivers voice notes as MP4-container AAC carrying an audio
ftyp brand ("M4A "). `_guess_extension()` returned ".mp4" for every
`ftyp` file regardless of brand, so those attachments were cached as
documents instead of audio and STT rejected the upload:
API error: Error code: 400 - Invalid file format.
Supported formats: ['flac','m4a','mp3','mp4','mpeg','mpga','oga',
'ogg','wav','webm']
Read the 4-byte brand at offset 8 and return ".m4a" for audio brands
("M4A ", "M4B ") so they satisfy `_is_audio_ext()` and route to
`cache_audio_from_bytes()`. Video brands (isom/mp42/avc1/qt) still
return ".mp4".
This mirrors the existing brand/form-type disambiguation already used
in this function for RIFF (WEBP vs WAVE) and for ADTS AAC vs MP3.
Verified against a real iOS Signal voice note: pre-fix the upload was
rejected with the 400 above; post-fix the same bytes transcribe
successfully.
The grant_spent notice fired for every subscription user with top-up
funds the moment their cap was reached and camped in the CLI/TUI status
bar and desktop toasts with no action to take — the account keeps
working off top-up. Remove it everywhere:
- agent/credits_tracker.py: drop grant_cond + the emit/clear block;
the dev fixture state now (correctly) produces no notice
- TUI: keep the turn-start clear of credits.grant_spent as back-compat
for older backends that still emit the key
- Desktop: drop the demo step and stale comment references
- Docs/config comments: remove grant-spent from credits_notices text
- Tests updated: policy/cold-start now assert the key never fires
Usage bands, depleted, and restored notices are unchanged; /usage still
reports the full balance breakdown.
The structural AST walker in test_compression_session_id_persistence.py
only recursed into control-flow children found via iter_child_nodes(stmt).
When a session_entry.session_id assignment lives inside an `else` block
whose statements are all assigns (no nested If/Try/etc to trigger
_walk_node), the assignment was invisible to the walker and the test
florped with 'No assignments found'.
Walk the stmt itself when it is a control-flow node so its
body/orelse/finalbody (and Try handlers) are always expanded, regardless
of whether iter_child_nodes yields an inner control-flow child.
Manual /compress already persists the rotated child transcript first and
only then repoints the live session_entry; a False rewrite_transcript
return keeps the entry on the original session_id so the conversation
stays reachable. Session hygiene auto-compress did the opposite: it
rebound session_id (and lease/topic) first, then called rewrite_transcript
without checking the return value. On a failed write the live entry
already pointed at an empty child SID and the turn continued — permanent
silent conversation loss. Persist first; rebind only after success.
Critical fixes to salvaged PR #73020:
- Use SessionDB.append_message instead of raw INSERT INTO messages.
The original used wrong column names (session_key/created_at vs
session_id/timestamp) and bypassed FTS indexing, session metadata
updates, display_kind, and all other columns append_message handles.
- Use get_hermes_home() instead of hardcoded Path.home()/'.hermes'.
Profile-aware path resolution under HERMES_HOME override and active
profile isolation.
- Add 11 tests covering flush, recovery, serialisation, edge cases.
- Change await self._send_read_receipt to asyncio.create_task to avoid
blocking message dispatch on slow bridge responses (matches BlueBubbles
pattern). Up to 5s per-message latency eliminated.
- Update tests: assert_called_once_with instead of assert_awaited_once_with
since the receipt is now scheduled, not directly awaited.
- Add send_read_receipts documentation to whatsapp.md following the
BlueBubbles docs pattern.
`@/Desktop` returned nothing while `@Desktop` worked. The leading slash
was always read literally, so the lookup went to the absolute `/Desktop`
— which doesn't exist — and dead-ended instead of finding the folder one
level down.
The `@` has already announced "this is a path", so the slash people type
next reads as a separator out of habit rather than a filesystem root.
Take the absolute meaning only when it resolves: the parent directory has
to exist, and a partially-typed segment has to match something in it.
Otherwise drop the slash and resolve from the cwd.
Real absolute paths are unaffected — `/usr`, `/etc/hos`, `/Users/...` all
pass the existence probe and keep their current behaviour. The guard
matters: stripping the slash unconditionally would let a repo-local
`etc/` shadow the real `/etc`.
The fuzzy branch of `complete.path` ranked basenames from
`_list_repo_files`, which lists files only, so a directory was only ever
reachable by typing a `/` — `@Desktop` returned nothing at all. Rank each
ancestor directory alongside the files, and break same-tier ties toward
the folder so `@docs` leads with `docs/` rather than `docs.md`.
Outside a git repo the fallback `os.walk` compounded this: it can spend
the whole `_FUZZY_CACHE_MAX_FILES` budget inside one deep subtree before
reaching a sibling, hiding top-level folders entirely. Seed the scan with
a `listdir` of the root so immediate children are always candidates.
Three mechanisms to detect and notify when gateway sessions stall silently:
1. Mid-turn activity heartbeats stamped to SessionDB so hermes sessions list
and hermes status show progress during long turns without new message rows.
2. Stall watchdog: when a busy session has pending inbound and the shared
activity clock is idle past agent.session_stall_timeout (default 300),
log a WARNING and notify the user once to try /new. Notify-only; does
not kill the turn.
3. Compaction timeout: fenceless compress_context callers get a progress-aware
host budget (compression.context_timeout_seconds default 120 idle,
compression.context_total_ceiling_seconds default 600 ceiling). On timeout,
cancel via commit fence, skip compaction without dropping messages, and
continue the turn.
Closes#72016 (slices 1-3; slice 4 cumulative SSE stream-retry deadline
remains a follow-up).
Cherry-picked from PR #72424 by @fangliquanflq.
One relay adapter fronts N platforms on one WS, but the capability surface
(MAX_MESSAGE_LENGTH / message_len_fn) was a scalar from whichever descriptor
resolved the handshake — and the transport's read loop OVERWROTE it on every
descriptor frame (last-writer-wins). A Discord chat on a gateway whose
applied descriptor was Telegram's inherited the 4,096-char cap and over-sent
into Discord's 2,000-char API 400 (observed live: 2,543/2,641-char replies
silently lost while inbound kept working).
- ws_transport: accumulate one descriptor per platform in
_descriptors_by_platform (exposed via descriptor_for_platform); the FIRST
descriptor of a connection generation stays the session default instead of
last-writer-wins; the map resets on re-dial.
- BasePlatformAdapter: new max_message_length_for_chat /
message_len_fn_for_chat hooks defaulting to the scalar surface (native
single-platform adapters unchanged).
- RelayAdapter: overrides resolve the chat's platform from _platform_by_chat
(the same map per-frame egress uses) and look up that platform's negotiated
descriptor; falls back to the scalar for unknown chats/transports.
- stream_consumer (streaming budget, _raw_message_limit, fallback-continuation
chunking) + run.py tool-progress limit now resolve per-chat.
Tests: tests/gateway/relay/test_relay_per_platform_caps.py (7) — verified
fail-without/pass-with (all 7 fail with the fix stashed). Relay + stream
consumer suites green (213 + 223).
Hardening on top of the salvaged #51642: free-text fields
(preview/goal/summary/output_tail) pass redact_sensitive_text(force=True)
before leaving on the public /v1/runs SSE stream — same treatment the API
already applies to error text — and child_session_id survives the
allowlist so clients can correlate the child's session. Both were flagged
in the sweeper review of #51642 and unaddressed.
Follow-up to the salvaged #71867 supervision fix. _spawn_supervised's own
backoff respawn created a new task without updating the external handle
self._reconnect_watcher_task, so after the reconnect watcher crashed and
self-restarted, _ensure_reconnect_watcher_running() saw the stale handle as
done() and spawned a SECOND concurrent watcher (double reconnect attempts).
Add an optional on_spawn callback to _spawn_supervised, fired with the live
task on every spawn INCLUDING internal respawns, and pass it at both reconnect-
watcher spawn sites so the tracked handle always advances. The two supervision
mechanisms (supervisor auto-restart + ensure-respawn) now compose instead of
racing. Regression test sabotage-verified.
Fixes#71758.
A platform adapter that dies on a transient upstream failure (marked
retryable=True, e.g. photon's sidecar exiting when its upstream
gRPC/CDN returns errors) is correctly queued into _failed_platforms
for background reconnection. But the reconnect watcher task itself
was spawned via a bare asyncio.create_task -- if an exception ever
escaped its OUTER while-loop (not just the per-platform inner
try/except), the watcher died silently: no log, no restart.
_ensure_reconnect_watcher_running() already existed to respawn a dead
watcher, but it's only called from _handle_adapter_fatal_error_impl()
when a NEW platform's fatal error arrives. If the watcher dies while a
platform is already sitting in the queue and no OTHER platform ever
fails afterward, nothing ever notices the watcher is dead -- exactly
matching the reported symptom: photon queued for reconnect, the
gateway itself healthy (other platforms kept working, so nothing
re-triggered the ensure-alive check), and the platform stayed dead for
17.5h until a manual restart, well after the transient upstream outage
had recovered.
Fix: spawn the reconnect watcher via the existing _spawn_supervised()
task-level supervisor (already used for kanban_dispatcher_watcher,
handoff_watcher, etc.) instead of a bare asyncio.create_task, at both
the initial startup spawn and the manual-respawn path in
_ensure_reconnect_watcher_running(). _spawn_supervised already
provides exactly what's missing here: catches and logs any exception
escaping the task, and auto-restarts with capped exponential backoff
(healthy-run counter resets so a daemon that crashes occasionally over
days is never permanently abandoned) -- self-healing independent of
any new fatal-error event.
Also hardened a related race: the watcher's per-platform loop looked
up self._failed_platforms[platform] via direct indexing after
snapshotting the keys with list(...). A platform removed concurrently
between the snapshot and the lookup (e.g. a manual /platform resume,
or a reconnect that succeeded via a different path) would raise an
uncaught KeyError -- exactly the class of bug this fix's supervision
now catches, but avoiding the crash-and-restart cycle entirely is
better than relying on it. Changed to .get() with a skip-if-missing
guard.
6 new tests pass (initial spawn uses _spawn_supervised, manual respawn
uses _spawn_supervised, the core regression -- watcher self-heals
after an uncaught exception with no new fatal-error event -- and the
race-guard scenario); 52/52 in the full
tests/gateway/test_platform_reconnect.py file (including the 4
pre-existing _ensure_reconnect_watcher_running tests, confirming no
regression to that respawn-when-dead-or-missing behavior).
The salvaged fix (#71593) rebuilds Telegram fallback pools lazily and
discards+aclose()s a pool on retryable connect failure (_reset_fallback),
bounding each at Limits(max_connections=8) as a setdefault default. The PR
shipped no test.
Add tests/gateway/test_telegram_fallback_pool_release_71593.py:
* failed fallback pool is aclose()d and dropped from _fallbacks (the
discard-on-failure path — reverting the _reset_fallback call fails it)
* a recovered pool is retained, only the failed one discarded
* _reset_fallback is a no-op when the pool was never built
* caller-supplied limits win over the _POOL_LIMITS setdefault default
* the max_connections=8 default applies when the caller omits limits
Update the eager-build assumptions in test_telegram_network.py to the new
lazy contract (fallbacks materialize via _get_fallback, not in __init__).
Widen the cherry-picked /diff base (#4839 by @SHL0MS) into one
cross-surface implementation, folding in the review feedback and the
best ideas from the two sibling PRs (#22703, #53527):
- tools/working_diff.py: shared git collection layer — unstaged
(default), staged, and all (vs HEAD) modes; untracked files folded in
via `git diff --no-index` so new files appear as additions (Codex
/diff parity); shlex-split arguments preserve quoted paths.
- CLI: handler moved to hermes_cli/cli_commands_mixin.py per the
current god-file decomposition (dispatch stays in cli.py), renders
through the rich console with a 400-line terminal-flood guard.
- Gateway: _handle_diff_command in gateway/slash_commands.py + dispatch
in gateway/run.py; fenced ```diff output truncated to 60 lines /
3000 chars before the platform senders apply their own per-platform
message clamps (tool-progress-style layered truncation). Localized
strings in all 17 locale catalogs.
- /diff session (from #53527): cumulative checkpoint-baseline diff of
everything Hermes changed, via new CheckpointManager.session_diff();
docstring records the retained-baseline approximation caveat from
review. Works on both surfaces; degrades with an actionable message
when checkpoints are off.
- Slack: /diff routed via /hermes diff (50-slash cap; keeps
telegram-parity test green and /version native).
- Registry: cross-surface CommandDef with staged|all|session
subcommands; docs: slash-commands reference (CLI + gateway tables +
both-surfaces list) and hermes-agent skill reference.
- Tests: tests/tools/test_working_diff.py (real git repos),
tests/hermes_cli/test_diff_command.py (real git + stubbed checkpoint
manager), tests/gateway/test_diff_command.py (end-to-end handler,
real checkpoint store), TestSessionDiff in
tests/tools/test_checkpoint_manager.py.
Salvaged from the /diff PR cluster #4839 + #22703 + #53527.
Co-authored-by: Ninso112 <ninso112@proton.me>
Co-authored-by: Harshkamdar67 <harshkamdar67@gmail.com>
Extends the cherry-picked /context command (PR #52184) and prompt-size
attribution helpers (PR #66656) into one visual context view across
surfaces, and absorbs the per-component budget-visibility goal of the
/tokens proposal (PR #48470):
- agent/context_breakdown.py: pure renderers over the existing payload —
a 5x20 glyph block grid (1 cell ~= 1% of the model window), an
'Estimated usage by category' table with free space, and expanded
per-skill / per-toolset listings via compute_context_details(), which
reuses the prompt-size attribution mechanism (skills index-line bytes +
registry tool->toolset map) converted to the same chars/4 heuristic.
- cli.py: /context [all] renders grid + category table (+ expanded
listings) from the live agent and in-memory conversation history.
- gateway/slash_commands.py: /context appends the plain-text category
table (no grid — monospace not guaranteed on messaging platforms);
/context all adds the expanded listings. Fail-open: breakdown errors
never break the gauge.
- hermes_cli/commands.py: /context gains the 'all' subcommand; /version
demoted to /hermes version on Slack to keep the 50-slash cap.
- tests: renderer unit tests against synthetic payloads, registry test,
gateway /context + /context all + failure-degradation handler tests.
- docs: slash-commands reference + CLI guide entries.
Read-only and locally computed: no provider calls, no prompt-cache impact.
Co-authored-by: RemyFevry <29257684+RemyFevry@users.noreply.github.com>
Co-authored-by: joelbrilliant <joelbrilliant1@gmail.com>
Co-authored-by: CharlesMcquade <6466275+CharlesMcquade@users.noreply.github.com>
A dedicated /context (alias /ctx) gateway slash command that gives a full
context-window view with:
- Usage gauge: visual bar + fraction + percentage + headroom
- Auto-compression threshold and how far away it is
- Compression count and how much the last one freed
- Cumulative session throughput (explicitly labelled as throughput,
NOT context size — each call re-sends the window)
- Cascading fallback: running agent → cached agent → SessionStore metadata
→ rough transcript estimate
Not included (per current-main design):
- Cache reporting removed: commit 446b8e239 intentionally removed cache
reporting from user-facing surfaces because providers that omit cached-token
details produce misleading values
- Sync DB calls replaced with async_session_store (current main requires
AsyncSessionStore with await)
Also rewords the /status tokens line from 'Cumulative API tokens (re-sent
each call)' to 'Lifetime tokens billed: ... (not your current context size;
use /context)' to reduce the recurring confusion that the cumulative figure
is the current context window.
Fixes salvation of PR #52184 (salvage commit replaces a 12K-commit-behind
fork branch with a fresh implementation against current main, incorporating
reviewer feedback from @whoislikemiha and the hermes-sweeper).
