Port from openclaw/openclaw#95108: an unbounded response.read() on a
non-OK *streaming* response can balloon memory (huge body) or hang the
agent forever (body opens then stalls with no further bytes). The
diagnostic body is only ever shown truncated, so reading megabytes or
blocking indefinitely buys nothing.
Add agent/bounded_response.read_streaming_error_body() which caps the
read at a byte limit and enforces a hard wall-clock deadline (run on a
worker thread so it can interrupt a socket read that stalls mid-chunk,
which a between-chunk wall-clock check cannot). Wire it into all three
streaming error-body sites that previously did a bare response.read():
native Gemini, Gemini Cloud Code, and Antigravity Cloud Code. The
existing error builders now accept an optional pre-read body_text so
classification (status code, RESOURCE_EXHAUSTED, free-tier guidance,
Retry-After) is preserved unchanged.
Tests use a real in-process socket server (no mocks): oversize body is
capped, stalled body hits the deadline with partial text preserved,
normal error envelope reads intact and parses.
_redact_env already skips redaction when a KEY=value assignment's value is
a programmatic env lookup (os.getenv(...), os.environ[...], process.env.X)
per issue #2852 — masking it would corrupt a code snippet, not redact a
secret. _redact_json (JSON "key": "value" syntax) and _redact_yaml
(unquoted key: value syntax) are separate closures in the same function
and never got the same check, so the identical code-snippet-in-config-
syntax case still gets mangled:
{"apiKey": "os.getenv('OPENAI_API_KEY')"} -> {"apiKey": "os.get...EY')"}
api_key: os.getenv("OPENAI_API_KEY") -> api_key: os.get...EY")
Fix: apply the same _ENV_LOOKUP_VALUE_RE.match(value) check in both
closures before masking, mirroring _redact_env exactly. Real secret
values in JSON/YAML syntax are still redacted (verified live and via new
tests) — this only skips the case where the "value" already look like a
code snippet.
a0a3c716f fixed the exact same failure mode for Telegram (#58563):
post-#48648, oversized mid-stream edits truncate to a one-message preview
instead of splitting. Once a long streamed reply grows past that cap, every
subsequent progressive edit truncates to the SAME preview text — re-sending
an identical edit every tick still counts against the platform's edit rate
limit for the rest of the stream.
Discord's edit_message() has the identical architecture (mid-stream
truncate-in-place, both pre-flight and reactive-after-50035 truncation
paths) and this file's own docstring already calls out "the Telegram #48648
lesson" it's built on — but the saturated-preview dedup fix itself was never
ported over.
Fix: track the last truncated preview per (chat_id, message_id), mirroring
a0a3c716f exactly. Skip the edit call when the new truncation is identical;
still deliver when the visible content actually changes (e.g. the
chunk-count marker crosses (1/2) -> (1/3) as the stream grows). State
clears on finalize and when content shrinks back under the cap, so dedup
can never mask a real edit.
11b4a21a5 cleared the per-session _last_resolved_model cache on /new and
the compression-exhausted auto-reset, so a resumed/reset conversation
resolves the model from current config instead of a stale cached value
(#58403). Three other sites documented as the same "full conversation
boundary" treatment — pop _session_model_overrides, clear the reasoning
override, pop _pending_model_notes — still missed _last_resolved_model:
- _session_expiry_watcher's permanent finalization block (gateway/run.py):
a session that goes idle and is finalized, then resumed, could serve a
model cached before it went idle on a transient config-cache miss.
- The daily/idle/suspended auto-reset cleanup (_was_auto_reset handling,
gateway/run.py): same failure mode, different trigger.
- /resume (gateway/slash_commands.py), whose own comment already says
"conversation boundary just like /new" for the sibling dicts it clears.
Fix: pop the session's _last_resolved_model entry in all three, mirroring
the exact pattern 11b4a21a5 established.
_handle_wake() and _handle_activity() enforced max_body_bytes only via
the Content-Length header. A Transfer-Encoding: chunked request
(content_length=None) or a spoofed small Content-Length bypassed the
cap entirely, letting the actual read be bounded only by aiohttp's
implicit 1 MiB client_max_size default (64x the 16 KB default) — the
same pattern ec29590a0 just fixed for gateway/platforms/webhook.py.
Fix: web.Application(client_max_size=self._max_body_bytes) so aiohttp
enforces the cap on every read path including chunked bodies, catch
HTTPRequestEntityTooLarge -> 413 on both endpoints (was swallowed into
a generic 400), and re-check the actual bytes read as defense in depth.
Exposure here is narrower than the webhook adapter (binds to 127.0.0.1
by default and requires the bridge token), but the bypass is otherwise
identical.
_redact_telegram_error_text() strips bot tokens from api.telegram.org
URLs embedded in transport-error text, and is already applied across the
send/edit transient-error paths. Four sites still built their message
from the raw exception:
- connect()'s fatal-error handler is the most severe: the raw text is
passed to _set_fatal_error(), which persists it via
write_runtime_status() to a dashboard/admin-facing runtime status
file, not just a log line. A transient network error during startup
commonly embeds the request URL
(https://api.telegram.org/bot<TOKEN>/getMe), so this could leak the
live bot token into that surface.
- disconnect(), send_document(), send_video() build the same unredacted
pattern into a warning log line (lower blast radius, but the same
leak class).
Fix: route all four through the existing _redact_telegram_error_text()
helper before building the message/log line, mirroring the send/edit
paths exactly. Also drops exc_info=True from the two logger.error/
logger.warning calls that had it — exc_info prints the exception's own
traceback (including its unredacted message) separately from the format
string, which would otherwise defeat the redaction; the already-redacted
sibling call sites in this file follow the same convention.
_CuaDriverSession._call_tool_via_cli() (the EAGAIN/silent-empty MCP
fallback transport) invokes `cua-driver call <tool> <json>` via
subprocess.run() with no env= argument, so the third-party cua-driver
binary inherits the full, unsanitized parent environment. The primary
MCP spawn site (_lifecycle_coro) already applies
_sanitize_subprocess_env(cua_driver_child_env()) before opening the
stdio client, per the same policy #53503/#55709 established for other
subprocess spawn points — this fallback path, added alongside the
EAGAIN/silent-empty-capture hardening, missed it.
Fix: apply the same env=_sanitize_subprocess_env(cua_driver_child_env())
to the subprocess.run() call in _call_tool_via_cli(), mirroring the
sanctioned spawn site exactly (telemetry policy applied first, then
Hermes-managed secrets filtered).
_handle_message() re-checks a slash-skill command's per-platform disabled
status before dispatch, because get_skill_commands() only applies the
global disabled list at scan time. That check only covered the leading
skill: split_stacked_skill_commands() resolves additional /skill tokens
that follow it (stacked invocations, up to 5 skills, #57987), and
build_stacked_skill_invocation_message() loads every one of them via
_load_skill_payload() with no disabled-status check of any kind.
A message on a platform with skills.platform_disabled configured for a
given skill could still get that skill's full SKILL.md content injected
into the agent's context for the turn, as long as it was typed after an
allowed skill: `/allowed-skill /disabled-skill do X`.
Fix: after computing the stacked extra_keys, look up each one's skill
name and re-check it against the same get_disabled_skill_names(platform=)
set already used for the leading skill. If any stacked skill is disabled
for the platform, reject the whole invocation with the same style of
message the leading-skill check already returns, instead of partially
loading it.
Port from openai/codex#21069 ("Spill large hook outputs from context").
Both shell hooks and Python plugins can return {"context": "..."} from
pre_llm_call, which gets appended to the current turn's user message on
every subsequent API call. A plugin that emits a large blob inflates
every turn and blows out the prompt cache prefix.
- tools/hook_output_spill.py: shared helper that writes oversized
context to $HERMES_HOME/hook_outputs/<session_id>/<uuid>.txt and
returns a head/tail preview plus the saved path. Never raises.
- agent/turn_context.py: apply the cap at the pre_llm_call aggregation
site (moved here from run_agent.py since the original PR), covering
both Python plugins and shell hooks.
- agent/shell_hooks.py: reserve output_spill as a sub-key under hooks:
so the config block doesn't emit unknown-hook-event warnings.
- Docs: document the cap + config in build-a-hermes-plugin.md.
Config (behaviour-preserving when absent):
hooks.output_spill: enabled/max_chars/preview_head/preview_tail/directory
Tests: 14 unit tests; shell_hooks (56) and plugins (100) suites green.
E2E validated with isolated HERMES_HOME (spill, passthrough, traversal
sanitisation, reserved-key skip).
Feishu adapter's disconnect() cancelled WSS-thread tasks but never
called the lark_oapi client's _disconnect() coroutine, so no
WebSocket CLOSE frame was sent. Feishu's server kept routing
messages to the stale endpoint for minutes (CLOSE-WAIT timeout),
silencing the channel across every shutdown path — systemd restart,
hermes update, hermes gateway restart, and the --replace takeover
during 'hermes dashboard' invocations.
Schedule ws_client._disconnect() on the WSS thread loop via
run_coroutine_threadsafe with a 5s timeout before the existing
task-cancel + loop-stop sequence. Defensive hasattr guard + broad
except keeps disconnect() resilient if lark_oapi's internals shift.
Fixes#10202
Port from Kilo-Org/kilocode#11555: GLM-5.2 exposes a native
reasoning_effort knob with two enabled levels (high / max) on its
OpenAI-compatible endpoints. Previously the zai profile (direct Z.AI
/api/paas/v4) used the base ProviderProfile and emitted nothing, and the
OpenCode Go profile only handled Kimi K2 / DeepSeek — so a user's effort
preference for GLM-5.2 was silently dropped on both routes.
- zai: ZaiProfile maps effort onto high/max (xhigh/max -> max, lower -> high)
- opencode-go: same mapping for GLM-5.2, alongside existing Kimi/DeepSeek
- alias spellings recognized (glm-5.2 / glm-5-2 / glm-5p2, vendor-prefixed)
- disabled / no effort leaves the server default untouched
The google-antigravity and google-gemini-cli OAuth providers were removed
in #50492. They were the only producers of a cloudcode-pa:// base_url, so
the account-level-quota early-returns in _pool_may_recover_from_rate_limit
and _credential_pool_may_recover_rate_limit are now unreachable.
- Drop the dead cloudcode-pa:// checks and the now-unused provider/base_url
params on _pool_may_recover_from_rate_limit (only caller updated).
- Prune the obsolete CloudCode-specific regression tests; keep the live
single/multi-entry pool-rotation invariants (#11314).
ruff check --fix --select F541 . on current main. Pure prefix removals;
adjacent-string concatenations keep the f only on interpolating fragments.
No string content or live placeholder altered.
TestMcpParallelToolBatch seeded provenance under old-style
mcp_<server>_<tool> names, which no longer pass the
is_mcp_tool_parallel_safe() prefix gate after the naming change.
Port from anomalyco/opencode#33533. Native MCP tools now register as
mcp__<server>__<tool> (double-underscore delimiter) instead of
mcp_<server>_<tool>, aligning with the convention used by Claude Code,
Codex, and OpenCode.
The double-underscore delimiter disambiguates the server/tool boundary
even when either component contains underscores (the single-underscore
form was ambiguous, which is why is_mcp_tool_parallel_safe already had to
track provenance in a side-map). It also unifies native registration with
the Anthropic-OAuth wire form (_MCP_TOOL_PREFIX = 'mcp__'), so the
single->double promotion that path performed is now a no-op for native
tools while still handling legacy replayed names.
- tools/mcp_tool.py: add MCP_TOOL_NAME_PREFIX + mcp_prefixed_tool_name()
helper; route _convert_mcp_schema, utility schemas, refresh stale-set,
and the parallel-safe prefix gate through it
- agent/transports/codex_event_projector.py: mirror convention in the
deterministic call_id input for MCP server-executed tool calls
- tests: update produced-name assertions to the new convention
Defense-in-depth alongside the interpreter-shutdown guard: run_job closed
the cron agent's async resources (agent.close + cleanup_stale_async_clients)
in its finally block BEFORE run_one_job called _deliver_result, so a live
delivery could race a torn-down async client. run_job now accepts an optional
defer_agent_teardown holder; when set it hands the live agent back instead of
closing it, and run_one_job tears it down (via the extracted _teardown_cron_agent
helper) only AFTER delivery — in a finally so a failed run never leaks. Default
path (holder=None) is unchanged, so every existing caller keeps inline teardown.
Reorder approach based on #58777 by @LavyaTandel; reworked to keep a single
delivery site in run_one_job and add regression coverage.
Co-authored-by: LavyaTandel <lavya@loom.local>
Pins `_interpreter_shutting_down()` (finalizing flag + shutdown-error-text
fallback) and asserts the standalone delivery path skips gracefully without
scheduling a send when the interpreter is finalizing, while the normal
non-finalizing path still delivers. Source guardrails keep the guard wired
into both the dispatch (`_submit_with_guard`) and standalone-delivery sites.
A cron tick can fire while the gateway is tearing down (SIGTERM from
`hermes update` / `hermes gateway stop` / systemd restart, or an OOM-kill).
Once the interpreter is finalizing, `concurrent.futures` refuses new work
with `RuntimeError: cannot schedule new futures after interpreter shutdown`
and asyncio's default executor is gone, so the cron delivery and dispatch
paths crash the tick and spray a traceback into errors.log on every
restart-race. Telegram/live-adapter deliveries surface it as
"Telegram send failed: ... cannot schedule new futures after interpreter
shutdown".
Add `_interpreter_shutting_down()` and consult it at the scheduling sites:
- the standalone delivery path (`asyncio.run` + the fresh-pool fallback),
- the tick dispatch (`_submit_with_guard` `pool.submit`).
When finalizing, skip gracefully with a warning instead of raising; the job
stays due and fires on the next healthy tick. The helper also matches the
RuntimeError text as a fallback, since the concurrent.futures global flag can
be set a hair before `sys.is_finalizing()` flips.
Fixes#58720. Also addresses the cron paths in #55924.
Follow-up on the #58818 cron-drain fix. The housekeeping ticker uses the
same loop-scheduled-future pattern as cron — it refreshes the channel
directory via safe_schedule_threadsafe(build_channel_directory(...), loop)
and blocks on fut.result(timeout=30). The original fix swapped its
join(5) for _await_thread_exit(5), which is a strict improvement (the loop
stays alive so the future can run) but the 5s bound is shorter than the
30s future, so a refresh in flight at shutdown was still abandoned. Bound
the housekeeping drain at 35s (30s future + margin) via a dedicated
_HOUSEKEEPING_SHUTDOWN_DRAIN_TIMEOUT constant. Not user-facing (self-heals
next tick) but keeps the cooperative drain honest across both threads.
Assert _await_thread_exit lets a coroutine scheduled onto the running loop by a
blocked worker thread complete (the #58818 deadlock a synchronous join caused),
returns False when the thread outlives the timeout, and handles None/dead
threads.
A cron delivery uses the live adapter by scheduling the send coroutine onto the
gateway event loop (safe_schedule_threadsafe) and blocking the ticker thread on
future.result(). On shutdown/restart the cleanup ran a synchronous
cron_thread.join(timeout=5), which blocks the event loop — so the pending
delivery coroutine could never execute, the join always timed out, and the
message was silently dropped (#58818). The default agent.restart_drain_timeout
is 0, so this fired on every restart with an in-flight delivery.
Replace the blocking joins with _await_thread_exit(), which polls is_alive()
via await asyncio.sleep so the loop keeps running and finishes the queued
delivery before teardown. The cron wait is bounded by the delivery future's own
60s ceiling (plus margin); housekeeping keeps a short bound. When no delivery is
in flight the ticker exits on stop_event immediately, so shutdown stays snappy.
Phase-2 review follow-ups on the unreadable-config chokepoint work:
- hermes_cli/xai_retirement.py apply_migration() is a full-file config.yaml
rewriter (ruamel round-trip + plain open("w")) that lives outside the
atomic_yaml_write path, so the chokepoint didn't cover it. It reads the
file first (which already fails closed on an unreadable file), but add
require_readable_config_before_write() right before the write as a
backstop for the read-then-write window, and a regression test asserting
the original bytes survive an unreadable config.
- Drop the unnecessary "Path" string quotes on atomic_config_write's
annotation — Path is imported eagerly at module top, no forward ref needed.
auth.py _update_config_for_provider / _reset_config_provider intentionally
keep their standalone require_readable_config_before_write guard + bare
atomic_yaml_write: the guard must fire BEFORE the read (fail-fast) at those
read-then-write sites, and a test pins the atomic_yaml_write call. Both are
already fully guarded against the bug; routing them through the wrapper
would move the check to write time for no benefit.
The unreadable-config-overwrite bug (an existing config.yaml that reads as
{} on a permission/IO error gets replaced with only defaults or the edited
section) is not limited to save_config / config set / auth. The same
read-then-atomic_yaml_write pattern lives at ~7 other independent write
sites that don't route through those functions:
- gateway/slash_commands.py: _save_config_key, memory/skills write_approval
toggles, tool_progress toggle, runtime_footer toggle, personality set
- hermes_cli/doctor.py --fix (stale root-key migration)
- gateway/platforms/yuanbao.py auto-sethome
- plugins/platforms/telegram/adapter.py topic thread_id persistence
- tui_gateway/server.py _save_cfg
- agent/onboarding.py mark_seen
Rather than sprinkle require_readable_config_before_write() at each site,
add a single fail-closed chokepoint, atomic_config_write(), that runs the
guard then delegates to atomic_yaml_write, and route every config.yaml
write through it. Root cause remains that read_raw_config() can't tell an
absent file from an unreadable one (returns {} for both) — read-only
callers correctly stay fail-open, but any full-file replacement now fails
closed in one enforced place instead of relying on each caller to remember
the guard.
save_config / set_config_value / auth keep the contributor's original
guard calls (their commit); this commit widens the fix to the sibling
call paths and adds a regression test on the chokepoint (fails closed on
unreadable existing file + still creates a genuinely absent file).
Follow-up on the #58790 fallback-limits fix: tighten the now-stale
_with_limits docstring and note on the fallback branch why it injects
limits at the transport level (not via _with_limits) so a future editor
does not re-route it through the client-level helper httpx would discard.
httpx ignores the client-level `limits` kwarg when a custom `transport`
is supplied. The #31599 keepalive fix injected limits via
`httpx_kwargs[limits]`, but the fallback-IP branch also passes a
custom `TelegramFallbackTransport` — so the limits were silently
discarded and the inner AsyncHTTPTransport instances ran with httpx
defaults (keepalive_expiry=5.0), leaking CLOSE_WAIT fds.
Pass the tuned limits directly into `TelegramFallbackTransport`
via `transport_kwargs` so its inner transports honour keepalive_expiry.
Only affects the fallback-IP branch; proxy and direct-DNS branches
continue to use `_with_limits()` as before.
Fixes#58790
The two TestSourceGuardrail tests asserted the presence of literal
strings ("#58753", "_user_survives") in context_compressor.py. Those
are change-detector tests that break on any refactor without catching a
real regression. The four behavioral tests in
TestCompressAlwaysKeepsAUserTurn already exercise the real compress()
path and fully cover the invariant (user turn survives, summary pinned
to user, no consecutive user roles, surviving tail user untouched).
Regression coverage for the kanban-worker crash where compression left a
transcript with no user-role messages, triggering a non-retryable
`400 No user query found in messages` from vLLM/Qwen.
Exercises the real `compress()` path with the reporter's shape (no system
prompt in the list, a re-compaction with the only user turn in the
compressed middle) and asserts the output always keeps >=1 user turn,
never introduces consecutive user roles, and leaves a surviving tail user
message untouched. A source guardrail pins the guard so a future refactor
cannot silently drop it.
Compression could produce a transcript with ZERO user-role messages,
which OpenAI-compatible backends (vLLM/Qwen) reject with a non-retryable
`400 No user query found in messages`. This crashes `hermes kanban`
workers unrecoverably: every resume replays the same poisoned history and
fails on the very first request after a successful compaction.
The existing #52160 guard pins the handoff summary to role="user" only
when `last_head_role == "system"` — i.e. when the system prompt sits
inside `messages` (the gateway `/compress` path). The main
auto-compression path prepends the system prompt at request-build time,
so the list handed to `compress()` starts with a user/assistant turn,
`last_head_role` defaults to "user", and the summary is emitted as
role="assistant". A kanban worker seeded with a single short
`"work kanban task <id>"` prompt followed by nothing but assistant/tool
turns therefore ends up user-less once that early turn is summarised.
Generalise the guard: when no user-role message survives in the protected
head or the preserved tail, force the summary to carry role="user" so the
request always has at least one user turn. When a user does survive
(e.g. in the tail), the guard does not fire, so alternation is preserved.
Fixes#58753.
Add an opt-in toggle (require_admin_for_exec_approval, default false) that
restricts who can click Approve/Deny on a dangerous-command prompt to admins
listed in allow_admin_from. Off by default, so the v0.16-restored user-scope
behavior is unchanged. When on, the clicker must pass the normal admission
check AND be an admin; fails closed (logged) when no admins are configured.
Only ExecApprovalView is gated — model picker / clarify / update-prompt stay
user-scope.
coerce_tool_args only repaired the outermost value, so JSON-encoded
*elements* of array properties (and nested object sub-fields) were left
as strings. Three core tools have array<object> schemas — todo.todos,
delegate_task.tasks, memory.operations — so a model emitting
{"todos": ["{...}"]} would pass raw JSON strings into the tool and fail
downstream on item["id"]/item["goal"] access.
Adds a schema-guided recursive pass (_normalize_json_strings_for_schema)
that parses JSON-string array items and nested object fields only when
the matching schema position expects an array/object, preserving
legitimate JSON-looking string fields (type: string).
Adapted from cline/cline#11803 to hermes-agent's existing coercion layer.
Follow-up to the salvaged #58696 (devatnull) + #41343 (annguyenNous)
commits: instead of fully suppressing commentary/analysis-phase stream
deltas, fire on_reasoning_delta so the CLI/gateway display them like
thinking text. Matches Codex CLI semantics where commentary is never
the turn's final answer, while keeping the narration visible in the
reasoning display. Adds devatnull to AUTHOR_MAP.
GPT-5.x models on the Codex Responses API emit short pre-tool-call
"preamble" text as message items with phase="commentary". Previously,
_normalize_codex_response() added ALL message items to content_parts
regardless of phase, causing commentary text to leak as visible
assistant content on chat gateways.
Fix: when normalized_phase is "commentary" or "analysis", route the
message text to reasoning_parts instead of content_parts. This keeps
preamble/internal planning in the reasoning channel where it belongs.
FixesNousResearch/hermes-agent#41293
- bridge: only enqueue poll_update events for polls Hermes itself created
(tracked via recentlySentIds when /send-poll returns) so arbitrary human
polls in group chats don't inject agent-visible messages on every vote
- update test_already_whatsapp_italic for the new markdown-italic mapping
- AUTHOR_MAP entry for @devatnull (PR #58704 salvage)
