Moves gateway routing metadata (display_name, origin_json, expiry_finalized)
into state.db, making SQLite the single source of truth for gateway session
discovery. Eliminates the dual-file (sessions.json + state.db) polling
dependency that caused the mcp_serve new-conversation race (#8925).
- hermes_state.py: schema v18 (3 new sessions columns + sessions.json
backfill migration), record_gateway_session_peer gains
display_name/origin_json, new set_expiry_finalized(),
list_gateway_sessions(), find_session_by_origin()
- gateway/session.py: peer recorder persists display_name + full origin
JSON; new SessionStore.set_expiry_finalized() single write-path
- gateway/run.py: expiry watcher success + give-up paths use the store
helper so the flag lands in both sessions.json and state.db
- mcp_serve.py: routing index reads state.db first (sessions.json fallback
for pre-migration DBs); _poll_once collapses to a single state.db mtime
check — the #8925 race is structurally impossible now
- gateway/mirror.py, gateway/channel_directory.py, hermes_cli/status.py:
query state.db first, sessions.json fallback
Closes#9006
Port from openclaw/openclaw#95108: an unbounded response.read() on a
non-OK *streaming* response can balloon memory (huge body) or hang the
agent forever (body opens then stalls with no further bytes). The
diagnostic body is only ever shown truncated, so reading megabytes or
blocking indefinitely buys nothing.
Add agent/bounded_response.read_streaming_error_body() which caps the
read at a byte limit and enforces a hard wall-clock deadline (run on a
worker thread so it can interrupt a socket read that stalls mid-chunk,
which a between-chunk wall-clock check cannot). Wire it into all three
streaming error-body sites that previously did a bare response.read():
native Gemini, Gemini Cloud Code, and Antigravity Cloud Code. The
existing error builders now accept an optional pre-read body_text so
classification (status code, RESOURCE_EXHAUSTED, free-tier guidance,
Retry-After) is preserved unchanged.
Tests use a real in-process socket server (no mocks): oversize body is
capped, stalled body hits the deadline with partial text preserved,
normal error envelope reads intact and parses.
_redact_env already skips redaction when a KEY=value assignment's value is
a programmatic env lookup (os.getenv(...), os.environ[...], process.env.X)
per issue #2852 — masking it would corrupt a code snippet, not redact a
secret. _redact_json (JSON "key": "value" syntax) and _redact_yaml
(unquoted key: value syntax) are separate closures in the same function
and never got the same check, so the identical code-snippet-in-config-
syntax case still gets mangled:
{"apiKey": "os.getenv('OPENAI_API_KEY')"} -> {"apiKey": "os.get...EY')"}
api_key: os.getenv("OPENAI_API_KEY") -> api_key: os.get...EY")
Fix: apply the same _ENV_LOOKUP_VALUE_RE.match(value) check in both
closures before masking, mirroring _redact_env exactly. Real secret
values in JSON/YAML syntax are still redacted (verified live and via new
tests) — this only skips the case where the "value" already look like a
code snippet.
a0a3c716f fixed the exact same failure mode for Telegram (#58563):
post-#48648, oversized mid-stream edits truncate to a one-message preview
instead of splitting. Once a long streamed reply grows past that cap, every
subsequent progressive edit truncates to the SAME preview text — re-sending
an identical edit every tick still counts against the platform's edit rate
limit for the rest of the stream.
Discord's edit_message() has the identical architecture (mid-stream
truncate-in-place, both pre-flight and reactive-after-50035 truncation
paths) and this file's own docstring already calls out "the Telegram #48648
lesson" it's built on — but the saturated-preview dedup fix itself was never
ported over.
Fix: track the last truncated preview per (chat_id, message_id), mirroring
a0a3c716f exactly. Skip the edit call when the new truncation is identical;
still deliver when the visible content actually changes (e.g. the
chunk-count marker crosses (1/2) -> (1/3) as the stream grows). State
clears on finalize and when content shrinks back under the cap, so dedup
can never mask a real edit.
11b4a21a5 cleared the per-session _last_resolved_model cache on /new and
the compression-exhausted auto-reset, so a resumed/reset conversation
resolves the model from current config instead of a stale cached value
(#58403). Three other sites documented as the same "full conversation
boundary" treatment — pop _session_model_overrides, clear the reasoning
override, pop _pending_model_notes — still missed _last_resolved_model:
- _session_expiry_watcher's permanent finalization block (gateway/run.py):
a session that goes idle and is finalized, then resumed, could serve a
model cached before it went idle on a transient config-cache miss.
- The daily/idle/suspended auto-reset cleanup (_was_auto_reset handling,
gateway/run.py): same failure mode, different trigger.
- /resume (gateway/slash_commands.py), whose own comment already says
"conversation boundary just like /new" for the sibling dicts it clears.
Fix: pop the session's _last_resolved_model entry in all three, mirroring
the exact pattern 11b4a21a5 established.
_handle_wake() and _handle_activity() enforced max_body_bytes only via
the Content-Length header. A Transfer-Encoding: chunked request
(content_length=None) or a spoofed small Content-Length bypassed the
cap entirely, letting the actual read be bounded only by aiohttp's
implicit 1 MiB client_max_size default (64x the 16 KB default) — the
same pattern ec29590a0 just fixed for gateway/platforms/webhook.py.
Fix: web.Application(client_max_size=self._max_body_bytes) so aiohttp
enforces the cap on every read path including chunked bodies, catch
HTTPRequestEntityTooLarge -> 413 on both endpoints (was swallowed into
a generic 400), and re-check the actual bytes read as defense in depth.
Exposure here is narrower than the webhook adapter (binds to 127.0.0.1
by default and requires the bridge token), but the bypass is otherwise
identical.
_redact_telegram_error_text() strips bot tokens from api.telegram.org
URLs embedded in transport-error text, and is already applied across the
send/edit transient-error paths. Four sites still built their message
from the raw exception:
- connect()'s fatal-error handler is the most severe: the raw text is
passed to _set_fatal_error(), which persists it via
write_runtime_status() to a dashboard/admin-facing runtime status
file, not just a log line. A transient network error during startup
commonly embeds the request URL
(https://api.telegram.org/bot<TOKEN>/getMe), so this could leak the
live bot token into that surface.
- disconnect(), send_document(), send_video() build the same unredacted
pattern into a warning log line (lower blast radius, but the same
leak class).
Fix: route all four through the existing _redact_telegram_error_text()
helper before building the message/log line, mirroring the send/edit
paths exactly. Also drops exc_info=True from the two logger.error/
logger.warning calls that had it — exc_info prints the exception's own
traceback (including its unredacted message) separately from the format
string, which would otherwise defeat the redaction; the already-redacted
sibling call sites in this file follow the same convention.
_CuaDriverSession._call_tool_via_cli() (the EAGAIN/silent-empty MCP
fallback transport) invokes `cua-driver call <tool> <json>` via
subprocess.run() with no env= argument, so the third-party cua-driver
binary inherits the full, unsanitized parent environment. The primary
MCP spawn site (_lifecycle_coro) already applies
_sanitize_subprocess_env(cua_driver_child_env()) before opening the
stdio client, per the same policy #53503/#55709 established for other
subprocess spawn points — this fallback path, added alongside the
EAGAIN/silent-empty-capture hardening, missed it.
Fix: apply the same env=_sanitize_subprocess_env(cua_driver_child_env())
to the subprocess.run() call in _call_tool_via_cli(), mirroring the
sanctioned spawn site exactly (telemetry policy applied first, then
Hermes-managed secrets filtered).
_handle_message() re-checks a slash-skill command's per-platform disabled
status before dispatch, because get_skill_commands() only applies the
global disabled list at scan time. That check only covered the leading
skill: split_stacked_skill_commands() resolves additional /skill tokens
that follow it (stacked invocations, up to 5 skills, #57987), and
build_stacked_skill_invocation_message() loads every one of them via
_load_skill_payload() with no disabled-status check of any kind.
A message on a platform with skills.platform_disabled configured for a
given skill could still get that skill's full SKILL.md content injected
into the agent's context for the turn, as long as it was typed after an
allowed skill: `/allowed-skill /disabled-skill do X`.
Fix: after computing the stacked extra_keys, look up each one's skill
name and re-check it against the same get_disabled_skill_names(platform=)
set already used for the leading skill. If any stacked skill is disabled
for the platform, reject the whole invocation with the same style of
message the leading-skill check already returns, instead of partially
loading it.
Port from openai/codex#21069 ("Spill large hook outputs from context").
Both shell hooks and Python plugins can return {"context": "..."} from
pre_llm_call, which gets appended to the current turn's user message on
every subsequent API call. A plugin that emits a large blob inflates
every turn and blows out the prompt cache prefix.
- tools/hook_output_spill.py: shared helper that writes oversized
context to $HERMES_HOME/hook_outputs/<session_id>/<uuid>.txt and
returns a head/tail preview plus the saved path. Never raises.
- agent/turn_context.py: apply the cap at the pre_llm_call aggregation
site (moved here from run_agent.py since the original PR), covering
both Python plugins and shell hooks.
- agent/shell_hooks.py: reserve output_spill as a sub-key under hooks:
so the config block doesn't emit unknown-hook-event warnings.
- Docs: document the cap + config in build-a-hermes-plugin.md.
Config (behaviour-preserving when absent):
hooks.output_spill: enabled/max_chars/preview_head/preview_tail/directory
Tests: 14 unit tests; shell_hooks (56) and plugins (100) suites green.
E2E validated with isolated HERMES_HOME (spill, passthrough, traversal
sanitisation, reserved-key skip).
Feishu adapter's disconnect() cancelled WSS-thread tasks but never
called the lark_oapi client's _disconnect() coroutine, so no
WebSocket CLOSE frame was sent. Feishu's server kept routing
messages to the stale endpoint for minutes (CLOSE-WAIT timeout),
silencing the channel across every shutdown path — systemd restart,
hermes update, hermes gateway restart, and the --replace takeover
during 'hermes dashboard' invocations.
Schedule ws_client._disconnect() on the WSS thread loop via
run_coroutine_threadsafe with a 5s timeout before the existing
task-cancel + loop-stop sequence. Defensive hasattr guard + broad
except keeps disconnect() resilient if lark_oapi's internals shift.
Fixes#10202
Port from Kilo-Org/kilocode#11555: GLM-5.2 exposes a native
reasoning_effort knob with two enabled levels (high / max) on its
OpenAI-compatible endpoints. Previously the zai profile (direct Z.AI
/api/paas/v4) used the base ProviderProfile and emitted nothing, and the
OpenCode Go profile only handled Kimi K2 / DeepSeek — so a user's effort
preference for GLM-5.2 was silently dropped on both routes.
- zai: ZaiProfile maps effort onto high/max (xhigh/max -> max, lower -> high)
- opencode-go: same mapping for GLM-5.2, alongside existing Kimi/DeepSeek
- alias spellings recognized (glm-5.2 / glm-5-2 / glm-5p2, vendor-prefixed)
- disabled / no effort leaves the server default untouched
The google-antigravity and google-gemini-cli OAuth providers were removed
in #50492. They were the only producers of a cloudcode-pa:// base_url, so
the account-level-quota early-returns in _pool_may_recover_from_rate_limit
and _credential_pool_may_recover_rate_limit are now unreachable.
- Drop the dead cloudcode-pa:// checks and the now-unused provider/base_url
params on _pool_may_recover_from_rate_limit (only caller updated).
- Prune the obsolete CloudCode-specific regression tests; keep the live
single/multi-entry pool-rotation invariants (#11314).
ruff check --fix --select F541 . on current main. Pure prefix removals;
adjacent-string concatenations keep the f only on interpolating fragments.
No string content or live placeholder altered.
TestMcpParallelToolBatch seeded provenance under old-style
mcp_<server>_<tool> names, which no longer pass the
is_mcp_tool_parallel_safe() prefix gate after the naming change.
Port from anomalyco/opencode#33533. Native MCP tools now register as
mcp__<server>__<tool> (double-underscore delimiter) instead of
mcp_<server>_<tool>, aligning with the convention used by Claude Code,
Codex, and OpenCode.
The double-underscore delimiter disambiguates the server/tool boundary
even when either component contains underscores (the single-underscore
form was ambiguous, which is why is_mcp_tool_parallel_safe already had to
track provenance in a side-map). It also unifies native registration with
the Anthropic-OAuth wire form (_MCP_TOOL_PREFIX = 'mcp__'), so the
single->double promotion that path performed is now a no-op for native
tools while still handling legacy replayed names.
- tools/mcp_tool.py: add MCP_TOOL_NAME_PREFIX + mcp_prefixed_tool_name()
helper; route _convert_mcp_schema, utility schemas, refresh stale-set,
and the parallel-safe prefix gate through it
- agent/transports/codex_event_projector.py: mirror convention in the
deterministic call_id input for MCP server-executed tool calls
- tests: update produced-name assertions to the new convention
Defense-in-depth alongside the interpreter-shutdown guard: run_job closed
the cron agent's async resources (agent.close + cleanup_stale_async_clients)
in its finally block BEFORE run_one_job called _deliver_result, so a live
delivery could race a torn-down async client. run_job now accepts an optional
defer_agent_teardown holder; when set it hands the live agent back instead of
closing it, and run_one_job tears it down (via the extracted _teardown_cron_agent
helper) only AFTER delivery — in a finally so a failed run never leaks. Default
path (holder=None) is unchanged, so every existing caller keeps inline teardown.
Reorder approach based on #58777 by @LavyaTandel; reworked to keep a single
delivery site in run_one_job and add regression coverage.
Co-authored-by: LavyaTandel <lavya@loom.local>
Pins `_interpreter_shutting_down()` (finalizing flag + shutdown-error-text
fallback) and asserts the standalone delivery path skips gracefully without
scheduling a send when the interpreter is finalizing, while the normal
non-finalizing path still delivers. Source guardrails keep the guard wired
into both the dispatch (`_submit_with_guard`) and standalone-delivery sites.
Assert _await_thread_exit lets a coroutine scheduled onto the running loop by a
blocked worker thread complete (the #58818 deadlock a synchronous join caused),
returns False when the thread outlives the timeout, and handles None/dead
threads.
Phase-2 review follow-ups on the unreadable-config chokepoint work:
- hermes_cli/xai_retirement.py apply_migration() is a full-file config.yaml
rewriter (ruamel round-trip + plain open("w")) that lives outside the
atomic_yaml_write path, so the chokepoint didn't cover it. It reads the
file first (which already fails closed on an unreadable file), but add
require_readable_config_before_write() right before the write as a
backstop for the read-then-write window, and a regression test asserting
the original bytes survive an unreadable config.
- Drop the unnecessary "Path" string quotes on atomic_config_write's
annotation — Path is imported eagerly at module top, no forward ref needed.
auth.py _update_config_for_provider / _reset_config_provider intentionally
keep their standalone require_readable_config_before_write guard + bare
atomic_yaml_write: the guard must fire BEFORE the read (fail-fast) at those
read-then-write sites, and a test pins the atomic_yaml_write call. Both are
already fully guarded against the bug; routing them through the wrapper
would move the check to write time for no benefit.
The unreadable-config-overwrite bug (an existing config.yaml that reads as
{} on a permission/IO error gets replaced with only defaults or the edited
section) is not limited to save_config / config set / auth. The same
read-then-atomic_yaml_write pattern lives at ~7 other independent write
sites that don't route through those functions:
- gateway/slash_commands.py: _save_config_key, memory/skills write_approval
toggles, tool_progress toggle, runtime_footer toggle, personality set
- hermes_cli/doctor.py --fix (stale root-key migration)
- gateway/platforms/yuanbao.py auto-sethome
- plugins/platforms/telegram/adapter.py topic thread_id persistence
- tui_gateway/server.py _save_cfg
- agent/onboarding.py mark_seen
Rather than sprinkle require_readable_config_before_write() at each site,
add a single fail-closed chokepoint, atomic_config_write(), that runs the
guard then delegates to atomic_yaml_write, and route every config.yaml
write through it. Root cause remains that read_raw_config() can't tell an
absent file from an unreadable one (returns {} for both) — read-only
callers correctly stay fail-open, but any full-file replacement now fails
closed in one enforced place instead of relying on each caller to remember
the guard.
save_config / set_config_value / auth keep the contributor's original
guard calls (their commit); this commit widens the fix to the sibling
call paths and adds a regression test on the chokepoint (fails closed on
unreadable existing file + still creates a genuinely absent file).
httpx ignores the client-level `limits` kwarg when a custom `transport`
is supplied. The #31599 keepalive fix injected limits via
`httpx_kwargs[limits]`, but the fallback-IP branch also passes a
custom `TelegramFallbackTransport` — so the limits were silently
discarded and the inner AsyncHTTPTransport instances ran with httpx
defaults (keepalive_expiry=5.0), leaking CLOSE_WAIT fds.
Pass the tuned limits directly into `TelegramFallbackTransport`
via `transport_kwargs` so its inner transports honour keepalive_expiry.
Only affects the fallback-IP branch; proxy and direct-DNS branches
continue to use `_with_limits()` as before.
Fixes#58790
The two TestSourceGuardrail tests asserted the presence of literal
strings ("#58753", "_user_survives") in context_compressor.py. Those
are change-detector tests that break on any refactor without catching a
real regression. The four behavioral tests in
TestCompressAlwaysKeepsAUserTurn already exercise the real compress()
path and fully cover the invariant (user turn survives, summary pinned
to user, no consecutive user roles, surviving tail user untouched).
Regression coverage for the kanban-worker crash where compression left a
transcript with no user-role messages, triggering a non-retryable
`400 No user query found in messages` from vLLM/Qwen.
Exercises the real `compress()` path with the reporter's shape (no system
prompt in the list, a re-compaction with the only user turn in the
compressed middle) and asserts the output always keeps >=1 user turn,
never introduces consecutive user roles, and leaves a surviving tail user
message untouched. A source guardrail pins the guard so a future refactor
cannot silently drop it.
Add an opt-in toggle (require_admin_for_exec_approval, default false) that
restricts who can click Approve/Deny on a dangerous-command prompt to admins
listed in allow_admin_from. Off by default, so the v0.16-restored user-scope
behavior is unchanged. When on, the clicker must pass the normal admission
check AND be an admin; fails closed (logged) when no admins are configured.
Only ExecApprovalView is gated — model picker / clarify / update-prompt stay
user-scope.
coerce_tool_args only repaired the outermost value, so JSON-encoded
*elements* of array properties (and nested object sub-fields) were left
as strings. Three core tools have array<object> schemas — todo.todos,
delegate_task.tasks, memory.operations — so a model emitting
{"todos": ["{...}"]} would pass raw JSON strings into the tool and fail
downstream on item["id"]/item["goal"] access.
Adds a schema-guided recursive pass (_normalize_json_strings_for_schema)
that parses JSON-string array items and nested object fields only when
the matching schema position expects an array/object, preserving
legitimate JSON-looking string fields (type: string).
Adapted from cline/cline#11803 to hermes-agent's existing coercion layer.
Follow-up to the salvaged #58696 (devatnull) + #41343 (annguyenNous)
commits: instead of fully suppressing commentary/analysis-phase stream
deltas, fire on_reasoning_delta so the CLI/gateway display them like
thinking text. Matches Codex CLI semantics where commentary is never
the turn's final answer, while keeping the narration visible in the
reasoning display. Adds devatnull to AUTHOR_MAP.
- bridge: only enqueue poll_update events for polls Hermes itself created
(tracked via recentlySentIds when /send-poll returns) so arbitrary human
polls in group chats don't inject agent-visible messages on every vote
- update test_already_whatsapp_italic for the new markdown-italic mapping
- AUTHOR_MAP entry for @devatnull (PR #58704 salvage)
The SOM/AX element list dropped labels for two extremely common cua-driver
render forms, leaving the model unable to target elements by name:
- [79] AXButton (Dark) -> parenthesised label
- [4] AXStaticText = "Wi-Fi" -> = "value" form
- [92] AXPopUpButton = "Automatic" -> = "value" form
The old regex only matched quoted "label" and id=Label, so System Settings
buttons/text/popups all surfaced with empty labels. That's why selecting the
macOS Appearance 'Dark' button by element index required guessing — the
labels weren't available to aim with.
Fix: extend _ELEMENT_LINE_RE to capture all four label forms (= "value",
"quoted", (parenthesised), id=Label), skipping a pure-digit (N) order number
in favour of the id= label. Verified live against System Settings: the
Appearance buttons now surface as Auto/Light/Dark.
Adds a regression test covering all label forms. Full suite: 84 passed.
The first fix handled the EAGAIN McpError path. But the persistent MCP
session (long-running gateway/desktop worker) has a second failure mode:
list_windows or get_window_state 'succeed' over MCP yet return a
degenerate/empty payload (no windows, or no screenshot + blank tree)
WITHOUT raising — typically when the bridge reconnected mid-call and
dropped the heavy response. That surfaced to the model as a silent 0x0
capture with no error and no fallback firing (0.00s empty return).
Fix: detect empty results in capture() and re-fetch over the CLI
transport before giving up:
- empty list_windows -> CLI re-fetch the window list
- empty get_window_state (som/ax) -> CLI re-fetch the AX tree + screenshot
- empty screenshot (vision) -> CLI re-fetch get_window_state for the PNG
Adds 2 regression tests. Full suite: 83 passed.
The cua-driver MCP stdio bridge intermittently (and on some machines
persistently) fails to forward heavier calls like get_window_state to
the daemon with POSIX EAGAIN — 'daemon transport error forwarding
get_window_state: Resource temporarily unavailable (os error 35)'.
The wrapper surfaced this as an empty 0x0 capture, so computer_use
returned blank screenshots even though the display, permissions, and
the daemon were all healthy (the direct 'cua-driver call' CLI path
worked fine throughout).
Fix: when the MCP path raises the transient/transport error, fall back
to the 'cua-driver call' subprocess transport, which talks to the
daemon over a different socket. The CLI fallback routes get_window_state
screenshots to a temp file via screenshot_out_file (tiny JSON response
instead of a multi-MB base64 blob that congests the socket), reads the
PNG back, retries with backoff, and remaps the JSON into the same
{data, images, structuredContent, isError} shape the MCP path produces
so capture()/_action() are transport-agnostic.
Adds _is_transient_daemon_error() classifier and 3 regression tests.
Verified live: captures that returned 0x0 now return full
1567x905 screenshots with the AX element tree.
The 'never reached ready' error (issue #57025) was undiagnosable — doctor
and MCP test pass while the wrapper times out, with no hint where startup
stalled. Track a phase marker through _lifecycle_coro (binary-check →
manifest-discovery → mcp-initialize → capability-discovery → ready) and
include it in the timeout RuntimeError plus a pointer to doctor and the
agent.log phase timings.
Complements the 15s→30s bump + success-path phase timing log from #58760.
Replaces the POSIX `/bin/bash -c "$(curl …)"` invocation with a
download-then-exec flow: curl the upstream install.sh into a mkstemp
temp file (unpredictable name, 0600) and run it as a plain argv list.
No shell=True, no command substitution. The temp script is removed in
a finally block; download failures return cleanly without exec.
Salvages the intent of #34974 by @ErnestHysa. His original patch
targeted a fixed /tmp/cua-driver-install.sh path (symlink/TOCTOU-prone
on multi-user hosts) and predates Windows/Linux installer support;
this version uses mkstemp and keeps the powershell path untouched.
Co-authored-by: ErnestHysa <takis312@hotmail.com>
Follow-up to #57987: after /skill-a the completer previously went silent
for a second /skill token. Now, while the leading tokens form an unbroken
skill chain (each token a distinct installed skill, under the 5-cap) and
the word under the cursor starts with '/', the completer keeps offering
the remaining skill commands, and SlashCommandAutoSuggest ghost-suggests
the rest of the next skill name. Instruction text, path-like tokens, and
broken chains get no suggestions. The TUI's complete.slash RPC reuses
SlashCommandCompleter, so it inherits the behavior with no changes.
