Deleting the matched user message breaks the strict role-alternation
invariant on the exact incident tail this fix targets — user(confirm) →
assistant('OK, restarting') becomes two consecutive assistant messages,
which strict providers reject and which the alternation-repair passes
upstream don't cover. Replace the message content with an explicit
'confirmation EXPIRED, re-confirm before any destructive action'
sentinel instead: the trigger text is still neutralized, the model gets
an affirmative instruction not to act, and the message sequence stays
valid. Adds an alternation-preservation regression test.
Follow-up to the salvage of #59640 by @knoal.
When a high-risk side effect (e.g. host restart via shutdown.exe) runs,
the user's plain-text confirmation phrase is persisted in the conversation
transcript. If the host restart killed the gateway process before the
assistant's tool result was written, the transcript tail ends on the
assistant's text response - and the dangerous confirmation text remains
in the user role.
On the next inbound message - possibly a casual 'are you there?' from
the user minutes later - the LLM sees the stale confirmation and may
interpret the new turn as a fresh re-confirmation, re-executing the
destructive action. This is the failure mode reported in #59607.
Fix:
- Add strip_stale_dangerous_confirmations() in agent/replay_cleanup.py
that removes user messages whose content matches a known dangerous
confirmation pattern AND whose timestamp is older than 60 seconds.
- Add is_dangerous_confirmation() helper with the matched patterns
(i18n-aware: covers 確認強制重開機 from the original incident).
- Wire the stripper into _build_gateway_agent_history() right after the
existing 75ed07ace strippers, so the strip chain is:
strip_interrupted_tool_tails -> strip_dangling_tool_call_tail ->
strip_stale_dangerous_confirmations.
- Update _build_replay_entry() to preserve the timestamp on user
messages (it was previously dropped), since the new stripper needs it.
Complements 75ed07ace (which strips the assistant side of the broken
tail) by handling the user side: a stale plain-text confirmation that
the assistant has not yet responded to in a way the resume logic
recognises.
Failing-test-first discipline: the bug-detection test
test_stale_confirmation_text_is_stripped_on_resume fails on unfixed
code (proves the test catches the bug) and passes after the fix.
Five additional safety tests confirm no regression on:
- fresh confirmations (within expiry) are preserved
- non-confirmation text is preserved
- non-matching histories are untouched
- dangerous-pattern detection works in all cases (case, i18n, None)
- direct unit test of the strip helper
Refs: #59607
* fix(gateway): use process-level HERMES_HOME for identity files
Gateway identity files (PID, lock, runtime status, takeover/stop markers)
were written via get_hermes_home() which honours the _HERMES_HOME_OVERRIDE
contextvar used for per-session profile dispatch. When a profile-context
task happened to be active at write time, files landed in the wrong profile
directory.
Add _get_process_hermes_home() that skips the contextvar and uses only the
HERMES_HOME env var or platform default, and route all gateway identity file
paths through it.
Fixes#56986
* chore(release): map liuhao1024 author email for PR #56993 salvage
---------
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Co-authored-by: Ben <ben@nousresearch.com>
* fix(gateway): per-profile pairing whitelist isolation for multiplex gateways
Pairing approvals are stored per profile (profiles/<name>/pairing/) and
authz routes pairing checks through the serving profile's store, so one
profile's approved users no longer authorize against every other
profile's whitelist in multiplex mode.
The global store remains for the hermes pairing CLI and single-profile
gateways; unregistered/unstamped sources fall back to it, preserving
existing behavior.
Salvaged from PR #53045 (pairing half). The SOUL.md half was dropped:
the agent turn already runs inside _profile_runtime_scope on main, so
load_soul_md() resolves per-profile without changes.
Original work by @soddy022.
* ci: redispatch after arm64 docker dashboard-slot flake (unrelated to this PR)
---------
Co-authored-by: soddy022 <290613374+soddy022@users.noreply.github.com>
* fix(gateway): scope reset banners' session info to the serving profile
The auto-reset notice and the manual /reset //new banner both appended
_format_session_info() outside any profile scope, so a multiplexed
gateway advertised the base config's model/provider/context while the
session actually ran on the profile's.
Route both call sites through a new _reset_notice_session_info(source),
which enters _profile_runtime_scope for the source's profile when
gateway.multiplex_profiles is on (mirroring _run_agent's gating), so
_load_gateway_config()/_resolve_gateway_model() resolve the profile's
config.yaml via the existing context-local home override. Single-profile
gateways never enter the scope — behavior unchanged.
Both call sites invoke the helper via asyncio.to_thread: under the
scope, resolution can do blocking work (credential refresh,
context-length HTTP probes) that previously failed fast unscoped and
must not run on the event loop.
Fixes#59003
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(release): map irresi author email for PR #59048 salvage
---------
Co-authored-by: irresi <blueirobin02@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The PR predates #31884, which changed the non-interrupted api_calls==0
empty path from silence to a retry hint. Flip the contributed test to
assert the current (correct) behavior.
A /stop sets _interrupt_requested on the session's cached agent, but the
flag is only cleared by the turn finalizer. When the stopped run is hung
or still draining, the flag survives the forced lock release and the
session's NEXT user message is killed at the top of the tool loop
(conversation_loop.py interrupt check): the run completes with
interrupted=True, api_calls=0 and an empty response, which
_normalize_empty_agent_response passed through as pure silence — the
user's message was swallowed with no trace except a
'response ready: ... api_calls=0 response=0 chars' log line.
Two-layer fix:
- _interrupt_and_clear_session now evicts the cached agent whenever it
releases the running state. The next message rebuilds the agent from
session history (mirroring the /new and /model paths), while the old
agent object keeps its interrupt flag so a hung drain still dies when
it unblocks. This intentionally does NOT clear the flag in place:
turn_context deliberately preserves a pending interrupt across turn
start (it carries interrupt-message delivery), and clearing it could
revive a hung run the user just stopped.
- _normalize_empty_agent_response distinguishes a drain from a swallowed
turn: an interrupted run that did work (api_calls > 0) stays silent as
before (deliberate stop/steer; queued messages are delivered by the
recursive drain inside _run_agent), but an interrupted run with ZERO
api_calls never processed the user's message at all and now surfaces a
'send it again' notice instead of nothing.
Same silent-delivery class as a1f76ba7e (#29346), which covered the
extract-stripped case; regression tests added next to that coverage.
Fixes#44212
The CWE-22 traversal guard in SessionEntry.from_dict rejects any
interior '/' in session_key, but session_key is a logical routing
key (never used as a filesystem path) and Google Chat resource names
legitimately contain '/' (spaces/<id>, spaces/<id>/threads/<id>).
All Google Chat sessions were silently dropped on gateway start.
Split the validation: session_id keeps the strict _is_path_unsafe
guard (it's the value used as a filename); session_key now uses a
relaxed _is_session_key_unsafe helper that only blocks genuine
traversal vectors (parent-dir '..', leading '/', leading '\', leading
Windows drive-letter prefix) and allows interior '/'.
load_gateway_config() only surfaced the top-level `multiplex_profiles`
key into gw_data before calling GatewayConfig.from_dict(). A config.yaml
that pinned the flag under the nested `gateway:` section -- the form
written by `hermes config set gateway.multiplex_profiles true` -- was
silently ignored, so the gateway loaded with multiplex_profiles=False.
from_dict() already honors the nested fallback, but load_gateway_config()
builds gw_data from top-level keys first, so the nested value never
reached it.
Read gateway.multiplex_profiles into gw_data when the top-level key is
absent, mirroring the existing nested fallback for max_concurrent_sessions.
Adds a load_gateway_config() regression test that writes a config.yaml
with `gateway.multiplex_profiles: true` and asserts the loaded config has
multiplex_profiles=True (fails without the fix).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes#50051 by preserving nested gateway.multiplex_profiles and routing gateway config env reads through the active profile secret scope when present.
This keeps secondary profile adapter startup from inheriting default-profile platform tokens or port-binding enables while preserving legacy single-profile behavior outside a scope.
Constraint: latest upstream main f57ff7aef1 still reproduced both nested-config loss and cross-profile env leakage
Rejected: special-casing API_SERVER_* only | left other profile-scoped tokens vulnerable to the same leak
Confidence: high
Scope-risk: moderate
Directive: keep future gateway/config env reads on the scoped helper path unless a variable is explicitly process-global
Tested: pytest -q tests/gateway/test_multiplex_phase0.py tests/gateway/test_multiplex_credential_isolation.py tests/gateway/test_config.py -k 'multiplex or scope or getenv or api_server or relay'
Not-tested: full gateway startup across live platform adapters
test_group_new_keeps_existing_reset_semantics_when_dm_topic_mode_enabled
asserts 'parallel work' not in the /new reply — but /new appends a
random tip from hermes_cli.tips (380 entries), and one tip's text
contains exactly that phrase (the delegate_task concurrency tip). CI
failed on PR #59331 slice 2 when the dice landed on it. Pin
get_random_tip in the test.
The routing sweep sends these paths through _adapter_for_source, which
reads source.profile. A bare MagicMock auto-attribute is truthy, so the
fixtures looked like stamped secondary profiles and hit the new
fail-closed branch. Real SessionSource.profile is None or str
(AGENTS.md pitfall #17).
Follow-up to #9006/#58899. The gateway routing index (session_key ->
SessionEntry) now lives in a new gateway_routing table in state.db as the
primary store; sessions.json is demoted to an optional legacy mirror.
- hermes_state.py: schema v19 — gateway_routing table (scope + session_key
PK; scope = resolved sessions_dir so multiple stores sharing one state.db
never cross-contaminate) with save/replace/load/delete methods
- gateway/session.py: _save() writes the whole index atomically to the DB
(mirrors the old full-file JSON rewrite semantics) and only falls back to
JSON when the DB write fails; _ensure_loaded reads the DB first and folds
in legacy sessions.json entries for keys the DB lacks (pre-migration
import; DB entries win over stale JSON)
- gateway/config.py + hermes_cli/config.py: new write_sessions_json flag
(default true for compat/downgrade safety); gateway.write_sessions_json:
false stops producing the file entirely
- sessions.json _README updated to say it's a legacy mirror + how to
disable it
Rehydration is now lossless across restarts even with sessions.json deleted:
suspended/resume_pending/model_override/token state all round-trip through
the DB (the old sessions-table recovery only rebuilt the bare key mapping).
Follow-up to the salvaged #54938: the bounded reader gives a proper 413 +
anomaly telemetry for oversized chunked bodies; client_max_size makes
aiohttp enforce the same 1 MiB cap on every other read path
(#58536/#58902/#59180 pattern). Test fixture's fake Application now
accepts kwargs.
The salvaged #25296 fixture's _FakeRequest.read() calls json.dumps but the
test module never imported json — the NameError was swallowed by the
handler's generic except → 400, failing 10 payload tests.
_handle_webhook() called request.read() with no size guard. Since the
endpoint is publicly reachable, an attacker can send an arbitrarily large
POST body to exhaust gateway memory.
Add _TWILIO_WEBHOOK_MAX_BODY_BYTES (64 KiB — well above any real Twilio
payload) and gate on both Content-Length and actual read size, returning
HTTP 413 with an empty TwiML Response on oversized requests. Mirrors the
guard already present in the Raft adapter.
* fix(docker): heal pairing-dir ownership after `docker exec` writes (#10270)
The official Docker image runs the gateway as the unprivileged `hermes`
user (uid 10000) via `gosu`, but `docker exec` defaults to root. Approval
files written by `docker exec <container> hermes pairing approve <code>`
end up as `-rw------- root:root`, and the post-gosu gateway process
cannot read them. The approval is silently ignored — the user keeps
hitting 'Unauthorized user' on every message.
The entrypoint's existing top-level chown is gated on the top-level
$HERMES_HOME being mis-owned, so on warm boots (where /opt/data is
already hermes:hermes) the recursive chown is skipped — meaning a
container restart does NOT self-heal the bug either.
Three-part fix:
1. docker/entrypoint.sh: chown the platforms/pairing/ (and legacy
pairing/) subtree on every container start, regardless of the
top-level decision. The directory is tiny (a few JSON files), so
the unconditional chown is effectively free. Container restart
now self-heals.
2. gateway/pairing.py: PairingStore._load_json was swallowing
PermissionError under its bare 'except OSError' branch, which is
what made this a silent failure. Split it out: log a WARNING that
names the file, the gateway's uid, the file's owner/mode, and the
exact docker exec -u hermes workaround. Still falls back to {} so
the gateway stays up.
3. website/docs/user-guide/security.md: add a Docker tip to the
pairing-CLI section pointing users at `docker exec -u hermes …`
up front.
Reproduced end-to-end in a containerized harness — before the fix
the gateway sees 0 approved users after `docker exec` + restart;
after the fix it sees the expected 1, and the file on disk goes
from `root:root 600` back to `hermes:hermes 600` on next start.
Fixes#10270
* fix(pairing): gate os.geteuid for Windows in PermissionError warning
Sibling sweep from the #58902 raft review found aiohttp servers still
running on the implicit 1 MiB default with no explicit body cap:
- bluebubbles webhook (127.0.0.1): 1 MiB explicit cap — events are small
JSON/form payloads; attachments arrive via the REST API
- teams Bot Framework listener (0.0.0.0 bind — most exposed): 1 MiB cap;
activities are JSON well under that
- hermes proxy server: 10 MB cap mirroring api_server's MAX_REQUEST_BYTES
(chat-completion payloads can be large, but must stay bounded)
client_max_size bounds every read path including chunked transfer-encoding
requests that carry no Content-Length (#58536/#58902 pattern).
Deliberately excluded: feishu, whatsapp_cloud, sms, line, wecom, msgraph —
open contributor PRs (#54938, #54944, #54620, #54931, #54934, #25296)
already cover those; reviewing them separately preserves their credit.
3 regression tests pin the wiring.
a0a3c716f fixed the exact same failure mode for Telegram (#58563):
post-#48648, oversized mid-stream edits truncate to a one-message preview
instead of splitting. Once a long streamed reply grows past that cap, every
subsequent progressive edit truncates to the SAME preview text — re-sending
an identical edit every tick still counts against the platform's edit rate
limit for the rest of the stream.
Discord's edit_message() has the identical architecture (mid-stream
truncate-in-place, both pre-flight and reactive-after-50035 truncation
paths) and this file's own docstring already calls out "the Telegram #48648
lesson" it's built on — but the saturated-preview dedup fix itself was never
ported over.
Fix: track the last truncated preview per (chat_id, message_id), mirroring
a0a3c716f exactly. Skip the edit call when the new truncation is identical;
still deliver when the visible content actually changes (e.g. the
chunk-count marker crosses (1/2) -> (1/3) as the stream grows). State
clears on finalize and when content shrinks back under the cap, so dedup
can never mask a real edit.
11b4a21a5 cleared the per-session _last_resolved_model cache on /new and
the compression-exhausted auto-reset, so a resumed/reset conversation
resolves the model from current config instead of a stale cached value
(#58403). Three other sites documented as the same "full conversation
boundary" treatment — pop _session_model_overrides, clear the reasoning
override, pop _pending_model_notes — still missed _last_resolved_model:
- _session_expiry_watcher's permanent finalization block (gateway/run.py):
a session that goes idle and is finalized, then resumed, could serve a
model cached before it went idle on a transient config-cache miss.
- The daily/idle/suspended auto-reset cleanup (_was_auto_reset handling,
gateway/run.py): same failure mode, different trigger.
- /resume (gateway/slash_commands.py), whose own comment already says
"conversation boundary just like /new" for the sibling dicts it clears.
Fix: pop the session's _last_resolved_model entry in all three, mirroring
the exact pattern 11b4a21a5 established.
_handle_wake() and _handle_activity() enforced max_body_bytes only via
the Content-Length header. A Transfer-Encoding: chunked request
(content_length=None) or a spoofed small Content-Length bypassed the
cap entirely, letting the actual read be bounded only by aiohttp's
implicit 1 MiB client_max_size default (64x the 16 KB default) — the
same pattern ec29590a0 just fixed for gateway/platforms/webhook.py.
Fix: web.Application(client_max_size=self._max_body_bytes) so aiohttp
enforces the cap on every read path including chunked bodies, catch
HTTPRequestEntityTooLarge -> 413 on both endpoints (was swallowed into
a generic 400), and re-check the actual bytes read as defense in depth.
Exposure here is narrower than the webhook adapter (binds to 127.0.0.1
by default and requires the bridge token), but the bypass is otherwise
identical.
_redact_telegram_error_text() strips bot tokens from api.telegram.org
URLs embedded in transport-error text, and is already applied across the
send/edit transient-error paths. Four sites still built their message
from the raw exception:
- connect()'s fatal-error handler is the most severe: the raw text is
passed to _set_fatal_error(), which persists it via
write_runtime_status() to a dashboard/admin-facing runtime status
file, not just a log line. A transient network error during startup
commonly embeds the request URL
(https://api.telegram.org/bot<TOKEN>/getMe), so this could leak the
live bot token into that surface.
- disconnect(), send_document(), send_video() build the same unredacted
pattern into a warning log line (lower blast radius, but the same
leak class).
Fix: route all four through the existing _redact_telegram_error_text()
helper before building the message/log line, mirroring the send/edit
paths exactly. Also drops exc_info=True from the two logger.error/
logger.warning calls that had it — exc_info prints the exception's own
traceback (including its unredacted message) separately from the format
string, which would otherwise defeat the redaction; the already-redacted
sibling call sites in this file follow the same convention.
_handle_message() re-checks a slash-skill command's per-platform disabled
status before dispatch, because get_skill_commands() only applies the
global disabled list at scan time. That check only covered the leading
skill: split_stacked_skill_commands() resolves additional /skill tokens
that follow it (stacked invocations, up to 5 skills, #57987), and
build_stacked_skill_invocation_message() loads every one of them via
_load_skill_payload() with no disabled-status check of any kind.
A message on a platform with skills.platform_disabled configured for a
given skill could still get that skill's full SKILL.md content injected
into the agent's context for the turn, as long as it was typed after an
allowed skill: `/allowed-skill /disabled-skill do X`.
Fix: after computing the stacked extra_keys, look up each one's skill
name and re-check it against the same get_disabled_skill_names(platform=)
set already used for the leading skill. If any stacked skill is disabled
for the platform, reject the whole invocation with the same style of
message the leading-skill check already returns, instead of partially
loading it.
Feishu adapter's disconnect() cancelled WSS-thread tasks but never
called the lark_oapi client's _disconnect() coroutine, so no
WebSocket CLOSE frame was sent. Feishu's server kept routing
messages to the stale endpoint for minutes (CLOSE-WAIT timeout),
silencing the channel across every shutdown path — systemd restart,
hermes update, hermes gateway restart, and the --replace takeover
during 'hermes dashboard' invocations.
Schedule ws_client._disconnect() on the WSS thread loop via
run_coroutine_threadsafe with a 5s timeout before the existing
task-cancel + loop-stop sequence. Defensive hasattr guard + broad
except keeps disconnect() resilient if lark_oapi's internals shift.
Fixes#10202
ruff check --fix --select F541 . on current main. Pure prefix removals;
adjacent-string concatenations keep the f only on interpolating fragments.
No string content or live placeholder altered.
Assert _await_thread_exit lets a coroutine scheduled onto the running loop by a
blocked worker thread complete (the #58818 deadlock a synchronous join caused),
returns False when the thread outlives the timeout, and handles None/dead
threads.
httpx ignores the client-level `limits` kwarg when a custom `transport`
is supplied. The #31599 keepalive fix injected limits via
`httpx_kwargs[limits]`, but the fallback-IP branch also passes a
custom `TelegramFallbackTransport` — so the limits were silently
discarded and the inner AsyncHTTPTransport instances ran with httpx
defaults (keepalive_expiry=5.0), leaking CLOSE_WAIT fds.
Pass the tuned limits directly into `TelegramFallbackTransport`
via `transport_kwargs` so its inner transports honour keepalive_expiry.
Only affects the fallback-IP branch; proxy and direct-DNS branches
continue to use `_with_limits()` as before.
Fixes#58790
Add an opt-in toggle (require_admin_for_exec_approval, default false) that
restricts who can click Approve/Deny on a dangerous-command prompt to admins
listed in allow_admin_from. Off by default, so the v0.16-restored user-scope
behavior is unchanged. When on, the clicker must pass the normal admission
check AND be an admin; fails closed (logged) when no admins are configured.
Only ExecApprovalView is gated — model picker / clarify / update-prompt stay
user-scope.
- bridge: only enqueue poll_update events for polls Hermes itself created
(tracked via recentlySentIds when /send-poll returns) so arbitrary human
polls in group chats don't inject agent-visible messages on every vote
- update test_already_whatsapp_italic for the new markdown-italic mapping
- AUTHOR_MAP entry for @devatnull (PR #58704 salvage)
