Commit graph

9284 commits

Author SHA1 Message Date
Gille
cff60b205d fix(sessions): verify fully reconstructed recovery 2026-07-26 23:34:29 -07:00
brooklyn!
21dd2d4d41
Merge pull request #72507 from NousResearch/bb/cli-resume-cwd-scoped
fix(cli): scope -c/--resume to the current workspace
2026-07-27 01:32:19 -05:00
teknium1
98fe6d0a8e fix(delegation): integrate lifecycle refactor with tool-history + daemon pool
Follow-ups on the salvaged #63359:
- _finalize_child_results carries tool_call_history on subagent_stop
  (the #62011/#72403 field landed after the PR branched; the shared
  pipeline must emit it for both delegate_task and plugin-launched
  children). Lifecycle test updated for the new payload field.
- The lifecycle executor uses DaemonThreadPoolExecutor — a wedged or
  abandoned child must never block interpreter exit at atexit-join time
  (same rationale as _run_single_child's timeout executor and the
  async-delegation pool).
- delegate_task's batch path keeps live-transcript wiring while routing
  child construction through the shared
  _build_child_preserving_parent_tools helper.
2026-07-26 23:27:30 -07:00
Tony Simons
f60abd6e37 fix subagent lifecycle ownership invariants 2026-07-26 23:27:30 -07:00
Tony Simons
1865fb5fcd feat(plugins): add public subagent lifecycle API 2026-07-26 23:27:30 -07:00
Brooklyn Nicholson
28a87d6319 fix(cli): scope -c/--resume to the current workspace
`hermes -c`/`--resume` (continue last session) resolved the globally
most-recently-used session, then cd'd into *its* recorded cwd. So running
`hermes -c` from repo A could land you in repo B's session — the session
you last touched anywhere, not the last one *here*.

Now `_resolve_last_session` scopes to the current workspace first: the git
repo root when CWD is inside a repo (so all sessions across its
subdirs/worktrees group together), else the CWD itself — matching the
`workspace_key` identity `hermes sessions list --workspace` already groups
on. It falls back to the unscoped global MRU when no session matches the
current workspace, preserving the old behaviour for fresh directories.

Adds `workspace_key` param to `SessionDB.search_sessions` and a
`_workspace_key_clause` SQL helper that mirrors `workspace_key()`: a row
matches when its `git_repo_root` equals the key, or (legacy rows without
git metadata) when its `cwd` is at or under it.
2026-07-27 01:26:34 -05:00
teknium1
0a2c245cd6 fix(auxiliary): reach the main agent model when a sibling aux model fails on the same provider
Widen okalentiev's failed_model narrowing (#59561) to
_try_main_agent_model_fallback. The safety-net layer still skipped on a
provider-label match alone, so single-provider users whose aux compression
model and main model share one custom endpoint had ZERO fallbacks: the aux
model timing out exhausted the chain in one hop and compression aborted.

Real incident (0.19.0 debug dump): aux zai-org/glm-5.2 hung 324s and timed
out while main mindai/macaron-v1-venti on the SAME endpoint was serving
448K-token turns — the label-only skip discarded the one viable summarizer,
the session wedged over threshold, and the anti-thrash breaker tripped.

Same convention as the chain fix: model-specific failures (timeout,
connection, rate limit) pass failed_model so only the exact failed model is
skipped; provider-wide failures (auth 401 / payment 402) pass None and keep
the whole-provider skip. Both sync and async call_llm sites pass it.

Sabotage-verified: the new regression test fails on the provider-only skip.
2026-07-26 22:33:06 -07:00
Oleksii Kalentiev
83f20e07e0 fix(auxiliary): keep provider-wide skip for auth/payment failures in fallback chain
Follow-up to the same-provider fallback fix: narrowing the configured-chain
skip to the exact failed model is only correct for model-specific failures.
Auth (401) and payment (402) errors are provider-wide — every model on the
provider shares the same broken credentials/account — so trying a sibling
model can't recover and merely burns another doomed request before the
aux task fails. Worse, returning that sibling client bypasses the
main-agent-model safety net that a provider-wide skip would have reached.

Only forward failed_model to _try_configured_fallback_chain for
model-specific failures (timeout, connection, rate limit, model-incompatible,
invalid response). Auth/payment keep failed_model=None (whole-provider skip),
preserving the pre-existing safety-net behaviour for credential/billing
failures.

Adds an integration test that a timeout forwards the failed model, and
updates the payment-error test to assert failed_model=None.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 22:33:06 -07:00
Oleksii Kalentiev
e8f8b34b0c fix(auxiliary): don't skip sibling models when a configured fallback_chain reuses the same provider
_try_configured_fallback_chain skipped every fallback_chain entry whose
provider matched the one that just failed. A chain that intentionally lists
several models under the same provider (e.g. two more NVIDIA NIM models
after the primary NIM model times out) was therefore skipped wholesale,
falling straight through to the main-agent-model safety net instead of
trying the other configured models on that provider.

Add failed_model so the skip narrows to the exact (provider, model) pair
that failed. Callers that only know the provider (client-build failures,
where the whole provider is unreachable regardless of model) keep the old
provider-wide skip; the two runtime request-error call sites (call_llm,
async_call_llm) now pass the model that just failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-26 22:33:06 -07:00
teknium1
cb06017b1d refactor(guardrails): make runaway-loop caps per-turn, not session-total
Per Teknium: the caps should bound a single agent loop, not accumulate
over the whole session. Rename SessionCapConfig -> LoopCapConfig and the
config section session_caps -> loop_caps; move the counters into
reset_for_turn (invoked per turn via turn_context) so each turn starts
with a fresh budget; retune defaults 200 -> 50 (a single turn issuing 50
web searches / spawning 50 subagents is already pathological). Block
codes session_*_cap -> loop_*_cap and messages updated to drop the
/new-resets-the-budget guidance (irrelevant now that it resets per turn).
Tests flipped: the old persists-across-turn-resets assertion becomes
resets-each-turn.
2026-07-26 21:27:45 -07:00
teknium1
b68787ad25 Inspired by Claude Code: session-wide runaway-loop caps for web_search and delegate_task
Add per-session lifetime caps on web_search calls and subagent spawns
(defaults 200/200, matching Claude Code v2.1.212). Unlike the existing
per-turn tool-loop guardrails, these count over the whole session and
reset only when a fresh agent is built (/new, /clear). Hitting a cap
blocks the offending call and halts the turn cleanly.

- agent/tool_guardrails.py: SessionCapConfig + session counters on the
  controller (in __init__, not reset_for_turn, so they persist across
  turns). before_call() enforces caps first, independent of
  hard_stop_enabled. delegate_task batches count each task.
- hermes_cli/config.py: tool_loop_guardrails.session_caps defaults.
- docs + tests (unit + E2E validated against a real AIAgent).
2026-07-26 21:27:45 -07:00
Ben Barclay
96996a55bf
fix(relay): per-platform capability descriptors for multi-platform gateways (#70717)
One relay adapter fronts N platforms on one WS, but the capability surface
(MAX_MESSAGE_LENGTH / message_len_fn) was a scalar from whichever descriptor
resolved the handshake — and the transport's read loop OVERWROTE it on every
descriptor frame (last-writer-wins). A Discord chat on a gateway whose
applied descriptor was Telegram's inherited the 4,096-char cap and over-sent
into Discord's 2,000-char API 400 (observed live: 2,543/2,641-char replies
silently lost while inbound kept working).

- ws_transport: accumulate one descriptor per platform in
  _descriptors_by_platform (exposed via descriptor_for_platform); the FIRST
  descriptor of a connection generation stays the session default instead of
  last-writer-wins; the map resets on re-dial.
- BasePlatformAdapter: new max_message_length_for_chat /
  message_len_fn_for_chat hooks defaulting to the scalar surface (native
  single-platform adapters unchanged).
- RelayAdapter: overrides resolve the chat's platform from _platform_by_chat
  (the same map per-frame egress uses) and look up that platform's negotiated
  descriptor; falls back to the scalar for unknown chats/transports.
- stream_consumer (streaming budget, _raw_message_limit, fallback-continuation
  chunking) + run.py tool-progress limit now resolve per-chat.

Tests: tests/gateway/relay/test_relay_per_platform_caps.py (7) — verified
fail-without/pass-with (all 7 fail with the fix stashed). Relay + stream
consumer suites green (213 + 223).
2026-07-27 14:24:14 +10:00
Teknium
37e648128b fix(picker): fold live bare k3 wire id into curated kimi-k3 row
Follow-up to the salvaged #67409 search aliases: with kimi-k3 now in
the curated kimi-coding list (#68108), a Coding Plan key rendered TWO
rows for one model — curated 'kimi-k3' plus live-discovered bare 'k3'
(merge dedup was exact-string). Add model_alias_canonical() derived
from the same alias table and use it as the merge dedup key, so the
curated public slug wins and live-only models still surface.
2026-07-26 21:22:40 -07:00
HexLab98
39902f1888 test(models): cover kimi search alias for Kimi Coding k3
Assert the picker haystack keeps ordinary ids unchanged, surfaces wire
id k3 for "kimi"/"k3" queries, and accept search_labels in curses mocks.
2026-07-26 21:22:40 -07:00
Teknium
f9cd577915 feat(approvals): add cross-surface mode command
Co-authored-by: luxiaolu4827 <227715866+luxiaolu4827@users.noreply.github.com>
2026-07-26 21:13:03 -07:00
teknium1
8fbe2e388f feat(tool_search): probe-validate blind tool_call args against the deferred schema
Port from nearai/ironclaw#5149 (the describe-first live-hardening fix in
their progressive tool disclosure work): when a model invokes a deferred
tool through the tool_call bridge without the schema-required arguments,
return the tool's parameter schema instead of dispatching blind.

Pre-fix, a blind call produced an opaque downstream failure
("[TOOL_ERROR] Tool execution failed: KeyError: 'document_id'") that
teaches the model nothing about what the tool expects — IronClaw observed
cheap models looping ~30 identical invalid calls until the iteration
budget died. Post-fix, the model repairs the call in one round-trip.

- tools/tool_search.py: new validate_deferred_call_args() — key-absence
  check of schema 'required' fields only; no type checking (coerce_tool_args
  already repairs types downstream); fails open on any validator error so
  it can never block a legitimate dispatch.
- model_tools.py: probe after the scope gate in the bridge dispatch.
- agent/tool_executor.py: probe in both unwrap sites (concurrent +
  sequential) before the underlying tool replaces the bridge; sequential
  path flattens the payload to match its {"error": str} wrapping.
- tests: TestDeferredCallSchemaProbe — blind call returns schema (not
  KeyError), valid/optional calls dispatch, unvalidatable tools fail open,
  out-of-scope rejection unchanged.
2026-07-26 20:59:36 -07:00
Teknium
9b97dea1e6 fix(skills): parse stored GitHub credentials without scanner false positives
Co-authored-by: Syed Annas <28944679+AnnasMazhar@users.noreply.github.com>
Co-authored-by: Bryan Neva <13835061+bryanneva@users.noreply.github.com>
2026-07-26 20:59:26 -07:00
Teknium
4854961d74 test: update reasoning-only exhaustion siblings for the terminal excerpt
Two sibling tests asserted the #34452 'No reply:' explainer text for
reasoning-only exhaustion. That terminal now delivers the labeled
reasoning excerpt (strictly more informative — it carries the model's
reasoning, which may contain the answer); the explainer still covers
the truly-empty case. Update the assertions to pin the new contract:
excerpt present, reasoning text included, '(empty)' never delivered.
2026-07-26 20:59:21 -07:00
Teknium
214ae7b77c feat(agent): surface a labeled reasoning excerpt at the empty-response terminal
When the empty-response ladder is fully exhausted (thinking-prefill
continuation, empty-content retries, provider fallback) and the model
produced structured reasoning but never any visible text, deliver a
clearly labeled excerpt of that reasoning instead of a bare '(empty)' —
the reasoning frequently contains the actual answer.

Delivery-only by design: raw chain-of-thought is never promoted to a
normal answer earlier in the ladder (prefill continuation still gets
first crack, retries and fallback still run), transcript persistence
semantics are untouched (the '(empty)' sentinel scaffolding keeps its
replay-safety behavior), and a truly empty exhaustion still returns the
existing terminal.

Idea credit: PR #48795 (@ligl0325) proposed falling back to
reasoning_content on empty content; this lands the safe kernel of that
idea at the one point in the ladder where it is strictly an improvement.
2026-07-26 20:59:21 -07:00
Teknium
a751924c04 fix(gemini): preserve typed enum constraints as strings
Port from openclaw/openclaw#104567: Gemini requires enum metadata to be strings even when the declared tool parameter type is numeric or boolean.
2026-07-26 20:59:16 -07:00
Teknium
b41eee450b fix(redact): stop masking prose words that embed a secret keyword (Secretary, tokenizer, author=)
Port from nearai/ironclaw#6129: their sensitive-marker scrubber matched
markers as bare substrings, so tool results containing 'Secretary of the
Treasury' were scrubbed as 'secret' on replay, evicting legitimate content
and forcing the model into a re-fetch loop. Hermes' lowercase/dotted/YAML
config-key redaction patterns (_CFG_DOTTED_RE, _CFG_ANCHORED_RE,
_YAML_ASSIGN_RE) had the same false-positive class: their key classes allow
arbitrary alphanumeric affixes around the keyword, so ordinary document
text like 'Secretary: J.Smith', 'tokenizer: cl100k_base' (HF model cards),
and BibTeX 'author=Smith' got value-masked on the surfaces that run these
passes (browser snapshots, log lines, kanban summaries, CLI-echoed output).

Fix: post-match word-boundary validation of the keyword occurrence inside
the matched key. Boundaries: key edges, non-letters (_ - . digits),
camelCase transitions (clientSecret, secretKey, APIToken), plural 's'
(secrets:, tokens:). Concatenated real-world compounds keep matching via
explicit alternatives (authtoken, authkey, secretkey, accesstoken). ALL-CAPS
keys keep legacy embedded matching (MYTOKEN=...) — all-caps is almost never
prose, same rationale as _ENV_ASSIGN_RE. Same discipline the file already
applies to exact-match body/query keys (ported from ironclaw#2529) and the
deliberate 'auth' exclusion that keeps 'author:' from matching.
2026-07-26 20:59:11 -07:00
Teknium
474c84ed8d fix(agent): uniquify duplicate tool-call ids to keep call/result pairing lossless
Port from openclaw/openclaw#110518 / #110956: some models reuse one call id
for different tool calls in a single batch (native Kimi Responses replays,
Ollama-compatible endpoints, degraded models at long context). Hermes kept
both calls but the pre-API sanitizer then dropped the later call/result pair
per id (#58327), so the second call's output silently vanished from every
replayed payload — the model never saw it and confabulated.

_uniquify_tool_call_ids renames later collisions to a deterministic <id>_d<n>
suffix at ingestion, before validation/dispatch/history build, so both pairs
survive. Composite Responses ids collide on the call half and keep their
response-item half. Deterministic suffixes preserve prompt-cache prefix
stability (no random UUIDs).
2026-07-26 20:59:06 -07:00
Teknium
d4381f0e39 fix(gemini): explain legacy Standard-key 401 rejections with migration guidance
Port from Kilo-Org/kilocode#12162.

Google began rejecting unrestricted legacy 'Standard' Google Cloud API keys
on the Gemini API on June 19, 2026 (all Standard keys stop working in
September 2026). The rejection is a 401 whose message misleadingly tells the
user to supply an OAuth 2 access token. gemini_http_error() now appends
actionable guidance (check key type in AI Studio, mint a new Gemini API key,
temporary restriction bridge) on that narrow shape — matched via
google.rpc.ErrorInfo reason ACCESS_TOKEN_TYPE_UNSUPPORTED or the
'Expected OAuth 2 access token' signature. Plain invalid keys
(API_KEY_INVALID) keep their existing message.

Also fixes a latent sibling gap: _summarize_api_error() preferred re-extracting
the raw response body for errors carrying .response, which stripped adapter-
composed guidance (this one AND the existing free-tier 429 guidance) from the
user-facing summary. GeminiAPIError now surfaces its composed message.
2026-07-26 20:58:57 -07:00
Teknium
da26ff986b fix(approval): detect recursive rm when flags follow operands
Port from openai/codex#33464: GNU rm permutes options, so
`rm build/ -rf`, `rm build/ -r -f`, and `rm build/ --recursive
--force` are equivalent to the flags-first spellings — but every
existing rm pattern required the flag group BEFORE the path, so these
spellings ran with no approval prompt at all (proven live on main).

The hardline floor was NOT affected: protected paths (/, system dirs,
$HOME) match regardless of flag position because the hardline path
matcher does not require flags. The gap was the approval-prompt layer
for arbitrary paths.

New DANGEROUS_PATTERNS entry with a tempered operand run: cannot cross
command separators (; | & newline), quotes, or a bare -- end-of-options
separator (after --, -rf is a literal filename). Flag token must be
whitespace-anchored so the r inside long options like --registry does
not count.

9 positive + 7 negative shapes in tests; approval cluster (868 tests)
green.
2026-07-26 20:58:52 -07:00
Teknium
53bfe40a35 fix(errors): classify throttle messages before token-overflow patterns; add new overflow shapes
Port from anomalyco/opencode#37848 (+ dev-branch twin #37840): expand
context-overflow patterns and guard against rate-limit messages that
mention tokens.

- 'Throttling error: Too many tokens, please wait before trying again.'
  (AWS Bedrock / proxy shape) classified as context_overflow and routed a
  healthy session into compression on every throttle. Added 'throttling'
  to _RATE_LIMIT_PATTERNS, which the message-only path checks BEFORE the
  overflow list.
- 'Input length N exceeds the maximum allowed input length of M tokens.'
  (Together/Fireworks shape) fell through to unknown — no compression
  recovery. Added 'maximum allowed input length' to overflow patterns.
- 'request_too_large' / 'Request exceeds the maximum size' (Anthropic 413
  type re-wrapped without a status code by aggregators/proxies) fell
  through to unknown. Added to _PAYLOAD_TOO_LARGE_PATTERNS.

All three shapes proven live on main before the fix; 265 classifier +
bedrock tests and 238 sibling rate-guard/compression tests pass.
2026-07-26 20:58:48 -07:00
Teknium
c4d1913294 fix(tools): normalize Unicode space family and minus sign in patch fuzzy matching
Port from anomalyco/opencode#38133/#38134 (patch Unicode matching corpus):
extend UNICODE_MAP with the Zs space-separator family (en/em quad, en/em/
three-per-em/four-per-em/six-per-em/figure/punctuation/thin/hair spaces,
narrow NBSP, medium mathematical space, CJK ideographic space) and the
Unicode minus sign U+2212.

Before: a file containing typographic spacing (French narrow NBSP, CJK
ideographic spaces, math minus) never matched a model's ASCII old_string
via the precise strategies — the edit only succeeded through the
similarity-based context_aware fallback, which (a) can pick the wrong
region (#54572 family) and (b) silently flattens the file's Unicode to
ASCII on replacement. After: these match at unicode_normalized (strategy
7), whose _preserve_unicode_in_replacement keeps the file's typographic
characters in unchanged spans.

All additions are 1:1 mappings, so the existing position-mapping and
preservation logic apply unchanged. Proven live before/after with a
multi-line probe; 59 fuzzy-match + 188 file-tools/patch/skill-manager
tests pass.
2026-07-26 20:58:43 -07:00
teknium1
6437701228 feat(approval): require approval for docker/podman daemon-redirect commands
Inspired by Claude Code 2.1.214, which added permission prompts for
container-CLI commands (including the Podman shim) carrying
daemon-redirect flags (--url, --connection, --identity, remote mode)
that previously ran without one.

A daemon redirect makes a local-looking command operate on a different
(often remote) daemon, silently acting on production infrastructure.
Any container-CLI invocation carrying a redirect now requires approval
regardless of subcommand:

- -H/--host and --context global flags (value required, global-flag
  position only — bare -h help and run-level -h <hostname> stay allowed)
- context use (persistently switches the default daemon)
- podman --url/--connection/--identity and -r/--remote
- DOCKER_HOST=/DOCKER_CONTEXT=/CONTAINER_HOST=/CONTAINER_CONNECTION=
  environment prefixes

Sibling-site widening: the existing container lifecycle rules matched
only the verb directly adjacent to the binary name, so a global flag or
a compose -f file flag slipped past the guard, and the legacy hyphenated
compose binary was never covered. They now tolerate global flags — the
same treatment the 'hermes ... gateway' rule already has — and match the
hyphenated compose binary.

Validation: 33 new tests; 339 pass in test_approval.py + new file;
442 pass across the adjacent guard suites; E2E battery of 12 dangerous
+ 17 safe commands via real imports, hot path ~330us/call.
2026-07-26 20:58:39 -07:00
teknium1
ef60509a26 test(delegate): model socket-abort interrupt for the inline child API path
test_interrupt_child_during_api_call pinned the OLD worker-thread contract:
a bare time.sleep(5) fake request that only 'interrupts fast' because the
worker thread gets abandoned. Delegated children now run the request INLINE
(#60203) and interrupt responsiveness comes from the cross-thread socket
abort (_abort_request_openai_client) — which a sleep can't feel. Wire the
fake request to an abort event that raises ConnectionError when the child's
abort hook fires, exactly as a real httpx recv unblocks on socket shutdown.
Sabotage-verified: disabling the abort hook fails the test at the full 5s;
with it the interrupt lands in ~10ms.
2026-07-26 20:37:51 -07:00
teknium1
ece050ac30 fix(delegation): route delegated-child API calls inline to avoid nested-pool wedge (#60203)
Root cause: delegate_task children run through three nested daemon-thread
layers (async-delegation executor -> per-child timeout executor -> the
interrupt worker interruptible_api_call spawns). After multi-day gateway
uptime the deepest layer wedges BEFORE the socket opens — the same
fingerprint as the gateway-cron hang (#62151): zero stale-detector output
(the worker never reaches dispatch), all providers, foreground/restart
works. The cron fix (should_use_direct_api_call) explicitly excluded
delegation 'for lack of evidence' — #60203 is that evidence.

- should_use_direct_api_call: extend the inline gate to delegated
  children, detected via the delegation ContextVar set by
  _run_single_child (platform='subagent' stamp as fallback). Scope
  unchanged otherwise: chat_completions wire only; Codex/Anthropic/
  Bedrock/MoA keep their established workers. Interrupts still work —
  the inline path registers _active_request_abort, which interrupt()
  invokes cross-thread (same mechanism the #72227 stall monitor uses).
- _dump_subagent_timeout_diagnostic: dump ALL thread stacks (bounded,
  40), not just the conversation worker — a pre-HTTP wedge is
  indistinguishable from a slow provider without seeing where the
  nested helper threads sit.
2026-07-26 20:37:51 -07:00
teknium1
666076d137 fix(api): redact subagent stream fields + forward child_session_id
Hardening on top of the salvaged #51642: free-text fields
(preview/goal/summary/output_tail) pass redact_sensitive_text(force=True)
before leaving on the public /v1/runs SSE stream — same treatment the API
already applies to error text — and child_session_id survives the
allowlist so clients can correlate the child's session. Both were flagged
in the sweeper review of #51642 and unaddressed.
2026-07-26 20:37:04 -07:00
LeonSGP43
4fbb86d2b8 fix(api): forward subagent lifecycle on run stream 2026-07-26 20:37:04 -07:00
embwl0x
a8c9ad0bcc fix(delegate): strip URL userinfo from tool history 2026-07-26 20:36:47 -07:00
embwl0x
e369d6ea3f feat(delegate): expose redacted child tool history 2026-07-26 20:36:47 -07:00
teknium1
bd7938fa0c fix(gateway): keep _reconnect_watcher_task tracking the live task after a supervised respawn
Follow-up to the salvaged #71867 supervision fix. _spawn_supervised's own
backoff respawn created a new task without updating the external handle
self._reconnect_watcher_task, so after the reconnect watcher crashed and
self-restarted, _ensure_reconnect_watcher_running() saw the stale handle as
done() and spawned a SECOND concurrent watcher (double reconnect attempts).

Add an optional on_spawn callback to _spawn_supervised, fired with the live
task on every spawn INCLUDING internal respawns, and pass it at both reconnect-
watcher spawn sites so the tracked handle always advances. The two supervision
mechanisms (supervisor auto-restart + ensure-respawn) now compose instead of
racing. Regression test sabotage-verified.
2026-07-26 19:43:17 -07:00
ygd58
4b039e9543 fix(gateway): spawn platform reconnect watcher with task-level supervision
Fixes #71758.

A platform adapter that dies on a transient upstream failure (marked
retryable=True, e.g. photon's sidecar exiting when its upstream
gRPC/CDN returns errors) is correctly queued into _failed_platforms
for background reconnection. But the reconnect watcher task itself
was spawned via a bare asyncio.create_task -- if an exception ever
escaped its OUTER while-loop (not just the per-platform inner
try/except), the watcher died silently: no log, no restart.

_ensure_reconnect_watcher_running() already existed to respawn a dead
watcher, but it's only called from _handle_adapter_fatal_error_impl()
when a NEW platform's fatal error arrives. If the watcher dies while a
platform is already sitting in the queue and no OTHER platform ever
fails afterward, nothing ever notices the watcher is dead -- exactly
matching the reported symptom: photon queued for reconnect, the
gateway itself healthy (other platforms kept working, so nothing
re-triggered the ensure-alive check), and the platform stayed dead for
17.5h until a manual restart, well after the transient upstream outage
had recovered.

Fix: spawn the reconnect watcher via the existing _spawn_supervised()
task-level supervisor (already used for kanban_dispatcher_watcher,
handoff_watcher, etc.) instead of a bare asyncio.create_task, at both
the initial startup spawn and the manual-respawn path in
_ensure_reconnect_watcher_running(). _spawn_supervised already
provides exactly what's missing here: catches and logs any exception
escaping the task, and auto-restarts with capped exponential backoff
(healthy-run counter resets so a daemon that crashes occasionally over
days is never permanently abandoned) -- self-healing independent of
any new fatal-error event.

Also hardened a related race: the watcher's per-platform loop looked
up self._failed_platforms[platform] via direct indexing after
snapshotting the keys with list(...). A platform removed concurrently
between the snapshot and the lookup (e.g. a manual /platform resume,
or a reconnect that succeeded via a different path) would raise an
uncaught KeyError -- exactly the class of bug this fix's supervision
now catches, but avoiding the crash-and-restart cycle entirely is
better than relying on it. Changed to .get() with a skip-if-missing
guard.

6 new tests pass (initial spawn uses _spawn_supervised, manual respawn
uses _spawn_supervised, the core regression -- watcher self-heals
after an uncaught exception with no new fatal-error event -- and the
race-guard scenario); 52/52 in the full
tests/gateway/test_platform_reconnect.py file (including the 4
pre-existing _ensure_reconnect_watcher_running tests, confirming no
regression to that respawn-when-dead-or-missing behavior).
2026-07-26 19:43:17 -07:00
dsad
3c4220cd9c fix(agent): keep system cache breakpoints across provider failover
`apply_anthropic_cache_control` runs once per call block, before the retry
loop, and splits the system prompt into `[static prefix, volatile tail]` text
blocks carrying the cache_control breakpoints.

A failover fires *inside* that retry loop, and `_sync_failover_system_message`
assigns a bare string over `api_messages[0]["content"]` to refresh the
`Model:`/`Provider:` identity lines. That drops the block list and both
breakpoints. `convert_messages_to_anthropic` only emits system cache blocks for
list content (`isinstance(content, list)`), so the retried request ships
`system` as one plain string: zero breakpoints, nothing written to cache, and
the whole system prompt re-billed at full write price. The next call misses
too, so a failover costs two full-price prompts instead of one write + one
read.

Measured with the real functions, same history, native Anthropic layout:

  normal turn       system=BLOCK LIST(2)  system breakpoints=2
  after failover    system=plain string   system breakpoints=0

`_sync_failover_system_message`'s own docstring notes this fires on "every
gateway turn, since fallback re-activates per message while the primary is
down" -- so a gateway running on a degraded primary pays it on every message.

Fix: rewrite the decorated blocks in place instead of flattening them.
`rewrite_prompt_model_identity` only touches the LAST `Model:`/`Provider:`
lines, and those live in the volatile tail, so the static prefix stays
byte-identical and its cache entry keeps matching -- the failover retry keeps
both breakpoints AND the warm prefix. Shapes we cannot safely patch fall back
to the existing plain-string assignment.

This is the same class the retry loop already guards against eight lines below
the gap, for reasoning fields:

    # api_messages is built once, before this retry loop, while the primary
    # provider is active. [...] so the fallback request isn't sent with stale,
    # primary-shaped reasoning fields.
    agent._reapply_reasoning_echo_for_provider(api_messages)

There was no equivalent for cache decoration.
2026-07-26 19:42:43 -07:00
TheEpTic
c2ee5039ee fix(gateway): preserve media dedup after streamed replies 2026-07-26 19:31:08 -07:00
AIconcept Guru
47f5795046 fix(install): avoid realpath-dependent uv launcher on macOS 2026-07-26 19:30:32 -07:00
teknium1
71c9910ff5 test(telegram): regression for #71593 fallback-pool discard-on-failure
The salvaged fix (#71593) rebuilds Telegram fallback pools lazily and
discards+aclose()s a pool on retryable connect failure (_reset_fallback),
bounding each at Limits(max_connections=8) as a setdefault default. The PR
shipped no test.

Add tests/gateway/test_telegram_fallback_pool_release_71593.py:
  * failed fallback pool is aclose()d and dropped from _fallbacks (the
    discard-on-failure path — reverting the _reset_fallback call fails it)
  * a recovered pool is retained, only the failed one discarded
  * _reset_fallback is a no-op when the pool was never built
  * caller-supplied limits win over the _POOL_LIMITS setdefault default
  * the max_connections=8 default applies when the caller omits limits

Update the eager-build assumptions in test_telegram_network.py to the new
lazy contract (fallbacks materialize via _get_fallback, not in __init__).
2026-07-26 19:30:27 -07:00
Frowtek
a228b81501 fix(sessions): preserve recently active sessions during pruning 2026-07-26 19:30:21 -07:00
Kyzcreig
769dba1758 fix(gateway): bound the startup-restore inbound gate on a slow boot-resume turn 2026-07-26 19:30:14 -07:00
dsad
0fb46b8185 fix(agent): keep the compaction summary when the turn-end override rewrites the user row
`_apply_persist_user_message_override` replaces the anchored user message's
content with the clean transcript text. Its DB-write twin refuses to do that
for a compaction-merged row, and says why:

    # Preflight compaction can re-anchor the override index at a message
    # whose content was MERGED with the compaction summary
    # (merge-summary-into-tail). Overwriting that with the clean gateway text
    # would silently drop the summary from the durable transcript.
    ... and not msg.get(COMPRESSED_SUMMARY_METADATA_KEY)

The live mutation has no such guard, and `finalize_turn` runs it FIRST -- it
calls `_apply_persist_user_message_override(messages)` and only then
`_persist_session(...)`. So the row is already clobbered by the time the
DB-side guard looks at it, and the clobbered list is also what is returned as
the continuation history the next turn is built from.

The anchor lands on the merged row by design, not by accident:
`reanchor_current_turn_user_idx` prefers the last user row whose content
matches this turn's text and "fall[s] back to the last user message when no
exact match survives (merge-summary-into-tail rewrites the content ...)" --
after a merge the exact match cannot survive, so the fallback selects exactly
the merged row.

Effect: on the turn a compaction fires, the `[MERGED PRIOR CONTEXT]` summary --
the entire pre-compaction conversation -- is deleted from the history the
session continues from. It is silent, there is no error, and the archived
pre-compaction turns are already rotated away. The durable child row still has
the summary, so the live session and a later `/resume` now replay different
bytes for the same row.

Add the same guard to the live path. The paired timestamp override is
unrelated and still applies.

Both halves of this invariant are now covered:
`TestFlushCompressedSummaryOverrideGuard` already asserted it for the DB write;
the two new tests assert it for the in-memory path, plus a negative control
that an ordinary turn is still cleaned in place.
2026-07-26 19:30:08 -07:00
HexLab98
6290cd0d59 test(environments): cover multiline session env snapshot injection
Regression for #71296: dump+source must not execute continuation lines from
HERMES_SESSION_CHAT_NAME/USER_NAME, and the export snippet must unset by
name instead of grepping declare lines.
2026-07-26 19:30:02 -07:00
menhguin
59482ea800 fix(skills): never change a skill's source registry on update
`hermes skills update` could silently replace a same-named skill from a
DIFFERENT registry — deleting the user's files and rewriting the lockfile's
recorded provenance. Two defects on main:

- tools/skills_hub.py: check_for_skill_updates fell back to *all* sources
  (`... or sources`) when no adapter matched the recorded source, so any
  registry with a same-named skill could satisfy the fetch and be reported
  as an update — silently reassigning provenance. Now reports the entry as
  `unavailable` instead of cross-registry fallback.
- hermes_cli/skills_hub.py: do_update called do_install with a bare, slash-
  less identifier and no source constraint, letting _resolve_short_name fuzzy-
  match a same-named skill in another registry. do_install now takes an
  optional source_id pin that ABORTS (rather than falls back) when no adapter
  matches, and do_update forwards the lockfile's recorded source as that pin.

Includes regression tests reproducing the cross-registry hijack.

Salvaged from PR #72216 onto current main; re-attributed to the human
contributor.

Co-authored-by: menhguin <menhguin@users.noreply.github.com>
2026-07-26 19:29:56 -07:00
dsad
c3e99fce49 fix(anthropic): keep the assistant cache breakpoint on the ordered-replay path
`apply_anthropic_cache_control` marks an assistant turn with non-empty text by
writing `cache_control` INTO `content` -- `_apply_cache_marker`'s list branch
puts it on the last content block, not at the top level.

`_convert_assistant_message`'s ordered-replay branch rebuilds the message from
`anthropic_content_blocks` and returns early. Its only cache sources are
`_relocated_replay_cache_control` (markers rescued from blocks the replay
sanitizer dropped) and the top-level `m["cache_control"]`; it never reads
`m["content"]`. So for an assistant turn that interleaves signed thinking with
a tool_use AND has preamble text, the breakpoint is dropped.

It is burned, not relocated: `_can_carry_marker` returns True for this message
(non-empty content), so the breakpoint budget already counted it. Hermes'
accounting believes the marker landed.

Measured with the real functions, native layout, same history, only
`anthropic_content_blocks` differing:

    normal path   4 breakpoints   assistant text block carries cache_control
    replay path   3 breakpoints   assistant carries NONE

Under the static-prefix layout only two conversation-tier breakpoints exist, so
this halves them. Nothing is logged and the request succeeds with identical
model output, which is why it survives: it recurs on every request for the life
of any Claude thinking+tools session, since the flag rides the message in
`agent.messages`.

#56195 fixed the complementary shape -- blank assistant content, where the
marker lands top-level -- and its regression test pins `"content": ""`. The
non-empty case is the one a Claude 4.5/4.6 tool turn normally produces (a
sentence of preamble before the call) and was never covered.

Harvest an in-`content` marker next to the existing top-level lookup and hand
it to the same `_apply_assistant_cache_control_to_last_cacheable_block` helper.
2026-07-26 19:29:51 -07:00
teknium1
ebbcad26ce test(memory): cover mode-aware provider deps + force reinstall path 2026-07-26 19:29:18 -07:00
LeonSGP43
5645169c8f fix(update): refresh active memory provider deps after venv rebuild
Surgical reapply of #53505 by @LeonSGP43 onto current main (stale-base
cherry-pick conflicted in main.py):

- _refresh_active_memory_provider_dependencies() in hermes_cli/main.py,
  wired into BOTH the git-pull update path (after lazy refresh) and the
  ZIP update path — the provider's plugin.yaml bridge packages are not
  in extras or LAZY_DEPS, so the core reinstall could strip/downgrade
  them and the update flow never healed them (#53272 mem0ai).
- _install_dependencies(force=True) in memory_setup.py: hand every
  declared spec to the resolver on update so missing AND version-drifted
  packages are restored (no-op when satisfied).
- Skip guards: no provider / builtin store / memory.enabled=false.

Widened beyond the original PR for the #70636 half:
- _provider_pip_dependencies(): mode-aware expansion — Hindsight in
  local/local_embedded mode needs hindsight-all (daemon + embedder), not
  just the declared hindsight-client; setup installs it but plugin.yaml
  can't express it, so update-time healing previously missed
  hindsight-embed and the daemon stayed broken.
- Spec-aware import probing: version ranges in pip_dependencies
  (mem0ai>=2.0.10,<3) no longer break the pip-name -> import-name
  mapping.

Fixes #53272. Fixes the hindsight-embed half of #70636.
2026-07-26 19:29:18 -07:00
teknium1
623762f2f0 fix(deps): move CVE pins to current fixed versions so update stops downgrading patched envs
Adjusts the salvaged pin refresh (#60839 by @embwl0x) to the actually
mergeable versions and regenerates the lock:

- cryptography 46.0.7 -> 48.0.1 (GHSA-537c-gmf6-5ccf fixed in 48.0.1;
  49.x is NOT possible: msal caps <49 and alibabacloud-tea-openapi caps
  <49 — documented at the pin site). Also resolves the
  hindsight-api-slim>=48.0.1 conflict reported in Discord.
- starlette 1.0.1 -> 1.3.1 across core/web/mcp/computer-use/dev extras
  and LAZY_DEPS (fastapi accepts >=0.40, mcp >=0.27)
- python-multipart 0.0.27 -> 0.0.32 ([web] + tool.dashboard)
- uv.lock regenerated (tea-openapi 0.4.4->0.4.5 for the <49 crypto cap)

Keeps @embwl0x's anti-downgrade floor guard in test_packaging_metadata
with the corrected cryptography floor (48,0,1). Fixes #60685: a user env
already upgraded to these versions is no longer downgraded by
hermes update, because the pins now ARE those versions.
2026-07-26 19:29:07 -07:00
embwl0x
a48251c3dc fix(update): refresh cve dependency pins 2026-07-26 19:29:07 -07:00
teknium1
1959e2a6b6 test(deps): class invariant — every shared LAZY_DEPS exact pin must match uv.lock
Generalizes the huggingface-hub lockstep test (#72320) to the whole
LAZY_DEPS surface: any package exact-pinned in LAZY_DEPS that the core
lock also resolves must pin the SAME version, so hermes update's lazy
refresh can never churn or downgrade a shared package out from under
its other consumers (#60783 class, #31817 class).

Together with the anchor-based activation gate (previous commit,
salvaged from #27878 by @paralegalia), this closes both halves of
#44404: features no longer false-activate from shared transitives, and
even a feature that legitimately activates cannot move a shared package
away from the locked version.
2026-07-26 19:28:55 -07:00