Feishu adapter's disconnect() cancelled WSS-thread tasks but never
called the lark_oapi client's _disconnect() coroutine, so no
WebSocket CLOSE frame was sent. Feishu's server kept routing
messages to the stale endpoint for minutes (CLOSE-WAIT timeout),
silencing the channel across every shutdown path — systemd restart,
hermes update, hermes gateway restart, and the --replace takeover
during 'hermes dashboard' invocations.
Schedule ws_client._disconnect() on the WSS thread loop via
run_coroutine_threadsafe with a 5s timeout before the existing
task-cancel + loop-stop sequence. Defensive hasattr guard + broad
except keeps disconnect() resilient if lark_oapi's internals shift.
Fixes#10202
ruff check --fix --select F541 . on current main. Pure prefix removals;
adjacent-string concatenations keep the f only on interpolating fragments.
No string content or live placeholder altered.
Assert _await_thread_exit lets a coroutine scheduled onto the running loop by a
blocked worker thread complete (the #58818 deadlock a synchronous join caused),
returns False when the thread outlives the timeout, and handles None/dead
threads.
httpx ignores the client-level `limits` kwarg when a custom `transport`
is supplied. The #31599 keepalive fix injected limits via
`httpx_kwargs[limits]`, but the fallback-IP branch also passes a
custom `TelegramFallbackTransport` — so the limits were silently
discarded and the inner AsyncHTTPTransport instances ran with httpx
defaults (keepalive_expiry=5.0), leaking CLOSE_WAIT fds.
Pass the tuned limits directly into `TelegramFallbackTransport`
via `transport_kwargs` so its inner transports honour keepalive_expiry.
Only affects the fallback-IP branch; proxy and direct-DNS branches
continue to use `_with_limits()` as before.
Fixes#58790
Add an opt-in toggle (require_admin_for_exec_approval, default false) that
restricts who can click Approve/Deny on a dangerous-command prompt to admins
listed in allow_admin_from. Off by default, so the v0.16-restored user-scope
behavior is unchanged. When on, the clicker must pass the normal admission
check AND be an admin; fails closed (logged) when no admins are configured.
Only ExecApprovalView is gated — model picker / clarify / update-prompt stay
user-scope.
- bridge: only enqueue poll_update events for polls Hermes itself created
(tracked via recentlySentIds when /send-poll returns) so arbitrary human
polls in group chats don't inject agent-visible messages on every vote
- update test_already_whatsapp_italic for the new markdown-italic mapping
- AUTHOR_MAP entry for @devatnull (PR #58704 salvage)
* feat(approvals): /deny <reason> relays denial reason to the agent
Port from qwibitai/nanoclaw#2832 (reject with reason).
Gateway /deny now accepts an optional trailing reason (/deny <reason>
or /deny all <reason>). The reason rides on the per-session approval
entry through resolve_gateway_approval -> _await_gateway_decision and is
appended to the BLOCKED tool result the agent receives, so a declined
agent can adapt instead of only hearing 'denied'.
Adapted to hermes-agent's synchronous single-command /deny model: no DB
state, no second-message capture step, no migration. Reason is capped at
280 chars and threaded through both the terminal-command guard and the
execute_code guard. Plain /deny and the approve paths are unchanged.
- tools/approval.py: _ApprovalEntry.reason; resolve_gateway_approval gains
optional reason; _await_gateway_decision returns it; both gateway BLOCKED
messages include it
- gateway/slash_commands.py: parse leading 'all' + trailing reason
- locales/en.yaml: deny.denied_reason_{singular,plural}
- hermes_cli/commands.py: /deny args_hint '[all] [reason]'
- tests: 3 new (with-reason, all+reason, plain-deny regression)
* fix(ci): localize deny-reason keys across all locales + update interrupt-path assertions
CI surfaced two enforced invariants broken by the deny-with-reason change:
- test_i18n catalog-parity requires every locale to carry the same keys as
en.yaml with matching placeholders. Added deny.denied_reason_singular/plural
(with {count}/{reason}) to all 15 non-English locales.
- test_approval_interrupt asserts the exact dict from _await_gateway_decision,
which now carries a 'reason' key (None on the interrupt/timeout paths).
Both gateway compression entry points (session-hygiene auto-compress in
run.py; manual /compress in slash_commands.py) filtered the transcript
to user/assistant-only, content-bearing messages before calling
_compress_context. That starved the compressor:
- tool results are usually the bulk of the context, and
_prune_old_tool_results never saw them
- short filtered histories tripped the protect-first/last early-return,
so compression became a no-op even on huge sessions
- assistant tool_calls stubs (content=None) were dropped, so even the
summary lost the tool activity
Pass user/assistant/tool messages through intact, matching what the
agent loop itself feeds _compress_context.
Port of PR #3854 onto current main (the manual-compress handler moved
from run.py to slash_commands.py since the PR branched); regression test
asserts tool messages reach the compressor.
Authored-by: David Zhang <david.d.zhang@gmail.com> (@Git-on-my-level)
Co-authored-by: David Zhang <david.d.zhang@gmail.com>
Post-#48648, oversized mid-stream edits truncate to a 4096-char preview
instead of splitting. But when rich messages raise the consumer's overflow
budget to 32k, the consumer keeps accumulating past 4096 and keeps issuing
progressive edits every edit_interval — each one truncating to the SAME
preview text. Telegram counts every one of those no-op requests against the
flood budget: a long streamed reply fires ~1 identical edit per 0.8s for
the rest of the stream, trips flood control (200s+ penalties), and the
final delivery hangs behind inline flood sleeps. Users see the bot stuck
'streaming' and the chat unresponsive.
Fix at the chokepoint: track the last truncated preview per
(chat_id, message_id) and skip the API call when the new truncation is
identical. Previews still update when the visible prefix actually changes
(e.g. chunk-count marker 1/2 → 1/3). State clears on finalize and when
content shrinks back under the cap, so dedup can never mask a real edit.
Live repro: 19,956-char streamed reply, transport=edit, rich available —
4x flood-control hits within ~700ms, 250s penalties, hung final delivery.
E2E harness on the same stream: 14 edit calls on main vs 7 with the fix
(the delta is pure no-op duplicates; scales with stream length).
After a config change (e.g. switching model provider), the /new command
must clear the per-session _last_resolved_model cache so the next turn
resolves the model from the updated config instead of falling back to
the stale cached value.
Without this fix, if a transient config-cache miss occurs on the first
post-/new turn, the #35314 recovery path serves the old model from the
cache — the user sees the old model being used even though they changed
config.yaml and explicitly ran /new.
Fix applies to both call sites that reset session model state:
- GatewaySlashCommandsMixin._handle_reset_command (slash_commands.py)
- GatewayRunner compression-exhausted auto-reset (run.py)
Fixes#58403
Per-session /model overrides supplied api_key and provider but omitted
credential_pool, so billing rotation never ran on HTTP 402. Wire the pool
on fast override, rehydrate, and apply paths; backfill from provider for
legacy persisted overrides. Regression tests in tests/gateway/.
The webhook adapter enforced max_body_bytes only via the Content-Length
header; a Transfer-Encoding: chunked request (content_length=None) or a
spoofed small Content-Length bypassed the cap entirely and read the full
body (bounded only by aiohttp's implicit 1 MiB default, above any
operator-configured smaller limit).
- web.Application(client_max_size=max_body_bytes): aiohttp enforces the
cap on every read path, chunked included
- catch HTTPRequestEntityTooLarge -> 413 (was swallowed into generic 400)
- post-read length re-check as defense in depth
- chunked-upload regression test
Manual port of PR #3955 by @Gutslabs onto current main (handler had
been restructured since); authorship preserved.
Add unit tests for resolve_placeholder_terminal_cwd and extend the config
bridge simulation for docker mount-on vs mount-off vs local fallback.
Co-authored-by: Cursor <cursoragent@cursor.com>
The conflict-retry ladder schedules a background recovery task via
loop.create_task(self._handle_polling_conflict(...)) on each failed
start_polling. test_polling_conflict_becomes_fatal_after_retries never
cancelled the last one, so under a loaded scheduler a leaked task could
get a turn, re-drive the counter into the fatal branch, and fire
_notify_fatal_error a second time — breaking assert_awaited_once()
non-deterministically. The bounded updater.stop() guard added in this
salvage introduced an extra await/scheduling yield that surfaced the
latent leak in CI slice 3. Cancel the leaked task before the fatal
assertions so the test is deterministic regardless of scheduler timing.
The salvaged fix (#58272) guarded the primary network-error reconnect path.
Issue #58270's Scope section names three more unguarded await updater.stop()
sites that can hang identically on a CLOSE-WAIT socket:
- conflict handler (before the retry back-off sleep)
- conflict-retries-exhausted teardown (before the fatal notify)
- disconnect() teardown (would hang gateway shutdown/restart)
Each is now wrapped in asyncio.wait_for(..., _UPDATER_STOP_TIMEOUT) with a
warning on timeout, matching the primary path, so no reconnect/teardown ladder
can wedge on a dead socket.
Also hoist the shared 15.0s bound to a single module constant
_UPDATER_STOP_TIMEOUT (self-documenting + DRY across all 4 sites), and update
the CLOSE-WAIT regression test to patch that constant instead of monkeypatching
asyncio.wait_for process-wide.
Fatal-notify idempotency: bounding the conflict-exhausted teardown stop() adds
an await AFTER _set_fatal_error, which yields the loop and lets a concurrent
retry task (scheduled by an earlier conflict, already suspended past the entry
guard) reach the fatal branch too — double-firing the fatal handler (surfaced
as a Python 3.11 CI failure in test_polling_conflict_becomes_fatal_after_retries).
Snapshot the pre-transition fatal state and only notify on the first transition.
When the TCP connection enters CLOSE-WAIT the PTB polling task is blocked
on epoll on a dead socket and never wakes. updater.stop() awaits that task
and therefore hangs indefinitely.
Consequence: _polling_error_task stays alive-but-blocked forever; every
subsequent heartbeat probe sees it as "in-flight" and skips triggering a
new reconnect; the gateway silently drops messages for hours until a manual
restart. Field incident: 11-hour outage on 2026-07-04 UTC despite the
heartbeat loop firing a reconnect at 01:11 — stop() blocked the entire
ladder.
Fix: wrap the updater.stop() call inside asyncio.wait_for(timeout=15).
On TimeoutError log a warning and continue to _drain_polling_connections()
+ start_polling() — same recovery path, just unblocked.
The heartbeat loop (PR #48496) correctly detects the dead socket and fires
_handle_polling_network_error. This commit is the missing second half:
ensuring the reconnect itself always completes.
Test: test_handle_polling_network_error_updater_stop_timeout() simulates
a hang by making stop() sleep forever and verifies that drain + start_polling
are still reached after the timeout.
Fixes#58270
Follow-up to @msh01's wall-deadline init-timeout fix.
- Resource leak: on timeout the initialize() task is abandoned without
awaiting its (shielded, possibly-never-completing) cancellation, so the
half-built PTB app's httpx client / connection pool was never closed —
up to 8x across the retry ladder. Add an optional on_abandon cleanup to
_await_with_thread_deadline that best-effort app.shutdown()s the abandoned
app, run detached + exception-swallowed so it can never re-block or re-hang
the ladder (mirrors _close_client_on_timeout in agent/auxiliary_client.py).
- Cover the helper itself: the salvaged test monkeypatched out the real
_await_with_thread_deadline, so its abandonment/cleanup path was untested.
Add direct tests for happy-path return, prompt-timeout-with-cleanup, and
cleanup-error-swallowed; the wedged coroutines swallow cancellation for a
bounded window (proving the helper returns before cancellation completes,
the #58236 shielded-scope behavior) without leaving an immortal task that
would wedge pytest teardown. Widen the salvaged stub to accept on_abandon.
- Attribution: add yingwaizhiying@gmail.com -> msh01 to AUTHOR_MAP (bare
gmail does not auto-resolve the check-attribution gate).
Known follow-up (not addressed here): the retry ladder reuses the same
self._app across all 8 attempts; a fresh app per attempt would fully close
the coherence risk if an abandoned initialize() completes in the background.
That is a larger restructure of the ~130-line builder+handler setup, left
for a separate change.
Follow-up to the salvaged SSH-tilde-cwd fix. The predicate
"backend == ssh and (cwd == ~ or cwd.startswith(~/))" was inlined at
each expanduser guard site, which is how the test simulator drifted from
production (it grew an SSH guard on a top-level-alias branch that has no
production counterpart).
- Add tools/terminal_tool._is_ssh_remote_tilde_cwd(backend, cwd) as the
single source of truth (case/whitespace-tolerant).
- Use it in _get_env_config and the gateway config bridge.
- Test simulator imports the real helper instead of re-implementing the
predicate; revert the phantom SSH guard on the top-level-alias branch
(production maps top-level cwd: to a plain env var, not TERMINAL_CWD via
an SSH-guarded path — that branch tested nothing real).
Gateway users can now search resumable sessions from messaging surfaces:
/sessions search <query> (alias: find) matches titles and session ids —
including every title/id in a row's forward compression chain, so a
compressed-away title still surfaces its live tip — plus a
punctuation-normalized variant so 'an94' matches 'AN-94'.
Implemented by generalizing the existing id_query chain-filter in
SessionDB.list_sessions_rich into a combined SQL-level filter (search
stays ORDER BY last-active + LIMIT at SQL level), threading a
search_query through the shared query_session_listing helper, and
teaching parse_session_listing_args to split off a search query.
Search results pass through the existing _resume_row_visible guard
unchanged: origin scoping, admin-only 'all', and the fail-closed
legacy-row posture from the July 1 hardening are preserved exactly.
Over-fetch (50) before the visibility cut so origin-invisible matches
can't starve the page.
Salvages the feature direction of PR #57595 by @GodsBoy with a minimal
implementation that keeps the resume authorization surface untouched.
`_dispatch_sync` gathers the mautrix per-event handler tasks with a bare
`asyncio.gather(*tasks)`. Without `return_exceptions=True`, the first handler
that raises aborts the gather, so the sibling events in the same sync response
are dropped unprocessed — the exception propagates up to the sync loop, which
logs a single "sync error" and moves on. The invite/redaction gathers a few
lines above already use `return_exceptions=True`.
Use `return_exceptions=True` and log each failing handler, so one bad event no
longer takes out the rest of its batch and per-event failures stay visible.
Regression test: a batch with one failing and one succeeding handler no longer
raises, the good handler still runs, and the failure is logged (mutation-
verified — reverting re-raises RuntimeError out of _dispatch_sync).
_resolve_media_to_data_urls's ad-hoc _MEDIA_TAG_RE matched any bare
token after MEDIA: (no absolute-path anchor) and read the resolved
path directly with no denylist. A relative/traversal path like
MEDIA:../../../../etc/passwd.png slipped through, and any image-
suffixed file the process could read (including under ~/.ssh, ~/.aws,
etc.) was base64-inlined into the API response if its path merely
appeared in the model's own final reply text.
Every other platform adapter's MEDIA: handling already goes through
two shared primitives in gateway/platforms/base.py:
- MEDIA_TAG_CLEANUP_RE, which anchors the path to ~/, /, or a
Windows drive letter plus a known deliverable extension.
- validate_media_delivery_path, which resolves symlinks and rejects
paths under the credential/system-path denylist.
Reuse both here instead of the local unanchored pattern and naive
Path().expanduser() resolution.
