Follow-up to the salvaged #71867 supervision fix. _spawn_supervised's own
backoff respawn created a new task without updating the external handle
self._reconnect_watcher_task, so after the reconnect watcher crashed and
self-restarted, _ensure_reconnect_watcher_running() saw the stale handle as
done() and spawned a SECOND concurrent watcher (double reconnect attempts).
Add an optional on_spawn callback to _spawn_supervised, fired with the live
task on every spawn INCLUDING internal respawns, and pass it at both reconnect-
watcher spawn sites so the tracked handle always advances. The two supervision
mechanisms (supervisor auto-restart + ensure-respawn) now compose instead of
racing. Regression test sabotage-verified.
Fixes#71758.
A platform adapter that dies on a transient upstream failure (marked
retryable=True, e.g. photon's sidecar exiting when its upstream
gRPC/CDN returns errors) is correctly queued into _failed_platforms
for background reconnection. But the reconnect watcher task itself
was spawned via a bare asyncio.create_task -- if an exception ever
escaped its OUTER while-loop (not just the per-platform inner
try/except), the watcher died silently: no log, no restart.
_ensure_reconnect_watcher_running() already existed to respawn a dead
watcher, but it's only called from _handle_adapter_fatal_error_impl()
when a NEW platform's fatal error arrives. If the watcher dies while a
platform is already sitting in the queue and no OTHER platform ever
fails afterward, nothing ever notices the watcher is dead -- exactly
matching the reported symptom: photon queued for reconnect, the
gateway itself healthy (other platforms kept working, so nothing
re-triggered the ensure-alive check), and the platform stayed dead for
17.5h until a manual restart, well after the transient upstream outage
had recovered.
Fix: spawn the reconnect watcher via the existing _spawn_supervised()
task-level supervisor (already used for kanban_dispatcher_watcher,
handoff_watcher, etc.) instead of a bare asyncio.create_task, at both
the initial startup spawn and the manual-respawn path in
_ensure_reconnect_watcher_running(). _spawn_supervised already
provides exactly what's missing here: catches and logs any exception
escaping the task, and auto-restarts with capped exponential backoff
(healthy-run counter resets so a daemon that crashes occasionally over
days is never permanently abandoned) -- self-healing independent of
any new fatal-error event.
Also hardened a related race: the watcher's per-platform loop looked
up self._failed_platforms[platform] via direct indexing after
snapshotting the keys with list(...). A platform removed concurrently
between the snapshot and the lookup (e.g. a manual /platform resume,
or a reconnect that succeeded via a different path) would raise an
uncaught KeyError -- exactly the class of bug this fix's supervision
now catches, but avoiding the crash-and-restart cycle entirely is
better than relying on it. Changed to .get() with a skip-if-missing
guard.
6 new tests pass (initial spawn uses _spawn_supervised, manual respawn
uses _spawn_supervised, the core regression -- watcher self-heals
after an uncaught exception with no new fatal-error event -- and the
race-guard scenario); 52/52 in the full
tests/gateway/test_platform_reconnect.py file (including the 4
pre-existing _ensure_reconnect_watcher_running tests, confirming no
regression to that respawn-when-dead-or-missing behavior).
`apply_anthropic_cache_control` runs once per call block, before the retry
loop, and splits the system prompt into `[static prefix, volatile tail]` text
blocks carrying the cache_control breakpoints.
A failover fires *inside* that retry loop, and `_sync_failover_system_message`
assigns a bare string over `api_messages[0]["content"]` to refresh the
`Model:`/`Provider:` identity lines. That drops the block list and both
breakpoints. `convert_messages_to_anthropic` only emits system cache blocks for
list content (`isinstance(content, list)`), so the retried request ships
`system` as one plain string: zero breakpoints, nothing written to cache, and
the whole system prompt re-billed at full write price. The next call misses
too, so a failover costs two full-price prompts instead of one write + one
read.
Measured with the real functions, same history, native Anthropic layout:
normal turn system=BLOCK LIST(2) system breakpoints=2
after failover system=plain string system breakpoints=0
`_sync_failover_system_message`'s own docstring notes this fires on "every
gateway turn, since fallback re-activates per message while the primary is
down" -- so a gateway running on a degraded primary pays it on every message.
Fix: rewrite the decorated blocks in place instead of flattening them.
`rewrite_prompt_model_identity` only touches the LAST `Model:`/`Provider:`
lines, and those live in the volatile tail, so the static prefix stays
byte-identical and its cache entry keeps matching -- the failover retry keeps
both breakpoints AND the warm prefix. Shapes we cannot safely patch fall back
to the existing plain-string assignment.
This is the same class the retry loop already guards against eight lines below
the gap, for reasoning fields:
# api_messages is built once, before this retry loop, while the primary
# provider is active. [...] so the fallback request isn't sent with stale,
# primary-shaped reasoning fields.
agent._reapply_reasoning_echo_for_provider(api_messages)
There was no equivalent for cache decoration.
The salvaged fix (#71593) rebuilds Telegram fallback pools lazily and
discards+aclose()s a pool on retryable connect failure (_reset_fallback),
bounding each at Limits(max_connections=8) as a setdefault default. The PR
shipped no test.
Add tests/gateway/test_telegram_fallback_pool_release_71593.py:
* failed fallback pool is aclose()d and dropped from _fallbacks (the
discard-on-failure path — reverting the _reset_fallback call fails it)
* a recovered pool is retained, only the failed one discarded
* _reset_fallback is a no-op when the pool was never built
* caller-supplied limits win over the _POOL_LIMITS setdefault default
* the max_connections=8 default applies when the caller omits limits
Update the eager-build assumptions in test_telegram_network.py to the new
lazy contract (fallbacks materialize via _get_fallback, not in __init__).
`_apply_persist_user_message_override` replaces the anchored user message's
content with the clean transcript text. Its DB-write twin refuses to do that
for a compaction-merged row, and says why:
# Preflight compaction can re-anchor the override index at a message
# whose content was MERGED with the compaction summary
# (merge-summary-into-tail). Overwriting that with the clean gateway text
# would silently drop the summary from the durable transcript.
... and not msg.get(COMPRESSED_SUMMARY_METADATA_KEY)
The live mutation has no such guard, and `finalize_turn` runs it FIRST -- it
calls `_apply_persist_user_message_override(messages)` and only then
`_persist_session(...)`. So the row is already clobbered by the time the
DB-side guard looks at it, and the clobbered list is also what is returned as
the continuation history the next turn is built from.
The anchor lands on the merged row by design, not by accident:
`reanchor_current_turn_user_idx` prefers the last user row whose content
matches this turn's text and "fall[s] back to the last user message when no
exact match survives (merge-summary-into-tail rewrites the content ...)" --
after a merge the exact match cannot survive, so the fallback selects exactly
the merged row.
Effect: on the turn a compaction fires, the `[MERGED PRIOR CONTEXT]` summary --
the entire pre-compaction conversation -- is deleted from the history the
session continues from. It is silent, there is no error, and the archived
pre-compaction turns are already rotated away. The durable child row still has
the summary, so the live session and a later `/resume` now replay different
bytes for the same row.
Add the same guard to the live path. The paired timestamp override is
unrelated and still applies.
Both halves of this invariant are now covered:
`TestFlushCompressedSummaryOverrideGuard` already asserted it for the DB write;
the two new tests assert it for the in-memory path, plus a negative control
that an ordinary turn is still cleaned in place.
Regression for #71296: dump+source must not execute continuation lines from
HERMES_SESSION_CHAT_NAME/USER_NAME, and the export snippet must unset by
name instead of grepping declare lines.
`hermes skills update` could silently replace a same-named skill from a
DIFFERENT registry — deleting the user's files and rewriting the lockfile's
recorded provenance. Two defects on main:
- tools/skills_hub.py: check_for_skill_updates fell back to *all* sources
(`... or sources`) when no adapter matched the recorded source, so any
registry with a same-named skill could satisfy the fetch and be reported
as an update — silently reassigning provenance. Now reports the entry as
`unavailable` instead of cross-registry fallback.
- hermes_cli/skills_hub.py: do_update called do_install with a bare, slash-
less identifier and no source constraint, letting _resolve_short_name fuzzy-
match a same-named skill in another registry. do_install now takes an
optional source_id pin that ABORTS (rather than falls back) when no adapter
matches, and do_update forwards the lockfile's recorded source as that pin.
Includes regression tests reproducing the cross-registry hijack.
Salvaged from PR #72216 onto current main; re-attributed to the human
contributor.
Co-authored-by: menhguin <menhguin@users.noreply.github.com>
`apply_anthropic_cache_control` marks an assistant turn with non-empty text by
writing `cache_control` INTO `content` -- `_apply_cache_marker`'s list branch
puts it on the last content block, not at the top level.
`_convert_assistant_message`'s ordered-replay branch rebuilds the message from
`anthropic_content_blocks` and returns early. Its only cache sources are
`_relocated_replay_cache_control` (markers rescued from blocks the replay
sanitizer dropped) and the top-level `m["cache_control"]`; it never reads
`m["content"]`. So for an assistant turn that interleaves signed thinking with
a tool_use AND has preamble text, the breakpoint is dropped.
It is burned, not relocated: `_can_carry_marker` returns True for this message
(non-empty content), so the breakpoint budget already counted it. Hermes'
accounting believes the marker landed.
Measured with the real functions, native layout, same history, only
`anthropic_content_blocks` differing:
normal path 4 breakpoints assistant text block carries cache_control
replay path 3 breakpoints assistant carries NONE
Under the static-prefix layout only two conversation-tier breakpoints exist, so
this halves them. Nothing is logged and the request succeeds with identical
model output, which is why it survives: it recurs on every request for the life
of any Claude thinking+tools session, since the flag rides the message in
`agent.messages`.
#56195 fixed the complementary shape -- blank assistant content, where the
marker lands top-level -- and its regression test pins `"content": ""`. The
non-empty case is the one a Claude 4.5/4.6 tool turn normally produces (a
sentence of preamble before the call) and was never covered.
Harvest an in-`content` marker next to the existing top-level lookup and hand
it to the same `_apply_assistant_cache_control_to_last_cacheable_block` helper.
Surgical reapply of #53505 by @LeonSGP43 onto current main (stale-base
cherry-pick conflicted in main.py):
- _refresh_active_memory_provider_dependencies() in hermes_cli/main.py,
wired into BOTH the git-pull update path (after lazy refresh) and the
ZIP update path — the provider's plugin.yaml bridge packages are not
in extras or LAZY_DEPS, so the core reinstall could strip/downgrade
them and the update flow never healed them (#53272 mem0ai).
- _install_dependencies(force=True) in memory_setup.py: hand every
declared spec to the resolver on update so missing AND version-drifted
packages are restored (no-op when satisfied).
- Skip guards: no provider / builtin store / memory.enabled=false.
Widened beyond the original PR for the #70636 half:
- _provider_pip_dependencies(): mode-aware expansion — Hindsight in
local/local_embedded mode needs hindsight-all (daemon + embedder), not
just the declared hindsight-client; setup installs it but plugin.yaml
can't express it, so update-time healing previously missed
hindsight-embed and the daemon stayed broken.
- Spec-aware import probing: version ranges in pip_dependencies
(mem0ai>=2.0.10,<3) no longer break the pip-name -> import-name
mapping.
Fixes#53272. Fixes the hindsight-embed half of #70636.
Adjusts the salvaged pin refresh (#60839 by @embwl0x) to the actually
mergeable versions and regenerates the lock:
- cryptography 46.0.7 -> 48.0.1 (GHSA-537c-gmf6-5ccf fixed in 48.0.1;
49.x is NOT possible: msal caps <49 and alibabacloud-tea-openapi caps
<49 — documented at the pin site). Also resolves the
hindsight-api-slim>=48.0.1 conflict reported in Discord.
- starlette 1.0.1 -> 1.3.1 across core/web/mcp/computer-use/dev extras
and LAZY_DEPS (fastapi accepts >=0.40, mcp >=0.27)
- python-multipart 0.0.27 -> 0.0.32 ([web] + tool.dashboard)
- uv.lock regenerated (tea-openapi 0.4.4->0.4.5 for the <49 crypto cap)
Keeps @embwl0x's anti-downgrade floor guard in test_packaging_metadata
with the corrected cryptography floor (48,0,1). Fixes#60685: a user env
already upgraded to these versions is no longer downgraded by
hermes update, because the pins now ARE those versions.
Generalizes the huggingface-hub lockstep test (#72320) to the whole
LAZY_DEPS surface: any package exact-pinned in LAZY_DEPS that the core
lock also resolves must pin the SAME version, so hermes update's lazy
refresh can never churn or downgrade a shared package out from under
its other consumers (#60783 class, #31817 class).
Together with the anchor-based activation gate (previous commit,
salvaged from #27878 by @paralegalia), this closes both halves of
#44404: features no longer false-activate from shared transitives, and
even a feature that legitimately activates cannot move a shared package
away from the locked version.
Widen the cherry-picked /diff base (#4839 by @SHL0MS) into one
cross-surface implementation, folding in the review feedback and the
best ideas from the two sibling PRs (#22703, #53527):
- tools/working_diff.py: shared git collection layer — unstaged
(default), staged, and all (vs HEAD) modes; untracked files folded in
via `git diff --no-index` so new files appear as additions (Codex
/diff parity); shlex-split arguments preserve quoted paths.
- CLI: handler moved to hermes_cli/cli_commands_mixin.py per the
current god-file decomposition (dispatch stays in cli.py), renders
through the rich console with a 400-line terminal-flood guard.
- Gateway: _handle_diff_command in gateway/slash_commands.py + dispatch
in gateway/run.py; fenced ```diff output truncated to 60 lines /
3000 chars before the platform senders apply their own per-platform
message clamps (tool-progress-style layered truncation). Localized
strings in all 17 locale catalogs.
- /diff session (from #53527): cumulative checkpoint-baseline diff of
everything Hermes changed, via new CheckpointManager.session_diff();
docstring records the retained-baseline approximation caveat from
review. Works on both surfaces; degrades with an actionable message
when checkpoints are off.
- Slack: /diff routed via /hermes diff (50-slash cap; keeps
telegram-parity test green and /version native).
- Registry: cross-surface CommandDef with staged|all|session
subcommands; docs: slash-commands reference (CLI + gateway tables +
both-surfaces list) and hermes-agent skill reference.
- Tests: tests/tools/test_working_diff.py (real git repos),
tests/hermes_cli/test_diff_command.py (real git + stubbed checkpoint
manager), tests/gateway/test_diff_command.py (end-to-end handler,
real checkpoint store), TestSessionDiff in
tests/tools/test_checkpoint_manager.py.
Salvaged from the /diff PR cluster #4839 + #22703 + #53527.
Co-authored-by: Ninso112 <ninso112@proton.me>
Co-authored-by: Harshkamdar67 <harshkamdar67@gmail.com>
Extends the cherry-picked /context command (PR #52184) and prompt-size
attribution helpers (PR #66656) into one visual context view across
surfaces, and absorbs the per-component budget-visibility goal of the
/tokens proposal (PR #48470):
- agent/context_breakdown.py: pure renderers over the existing payload —
a 5x20 glyph block grid (1 cell ~= 1% of the model window), an
'Estimated usage by category' table with free space, and expanded
per-skill / per-toolset listings via compute_context_details(), which
reuses the prompt-size attribution mechanism (skills index-line bytes +
registry tool->toolset map) converted to the same chars/4 heuristic.
- cli.py: /context [all] renders grid + category table (+ expanded
listings) from the live agent and in-memory conversation history.
- gateway/slash_commands.py: /context appends the plain-text category
table (no grid — monospace not guaranteed on messaging platforms);
/context all adds the expanded listings. Fail-open: breakdown errors
never break the gauge.
- hermes_cli/commands.py: /context gains the 'all' subcommand; /version
demoted to /hermes version on Slack to keep the 50-slash cap.
- tests: renderer unit tests against synthetic payloads, registry test,
gateway /context + /context all + failure-degradation handler tests.
- docs: slash-commands reference + CLI guide entries.
Read-only and locally computed: no provider calls, no prompt-cache impact.
Co-authored-by: RemyFevry <29257684+RemyFevry@users.noreply.github.com>
Co-authored-by: joelbrilliant <joelbrilliant1@gmail.com>
Co-authored-by: CharlesMcquade <6466275+CharlesMcquade@users.noreply.github.com>
`hermes prompt-size` reported skills as one <available_skills> block total
and tools as one json-bytes total, so there was no way to see which
installed skill or toolset actually dominates the fixed prompt budget.
Add two additive breakdowns to compute_prompt_breakdown (hermes_cli/
prompt_size.py):
- toolsets_breakdown: each resolved tool is attributed to its single
canonical registry toolset (registry.get_tool_to_toolset_map), summed by
group. Fully attributable — the grand total equals the existing
tools.json_bytes minus JSON array framing (2*count bytes).
- skills_breakdown: parsed from the rendered <available_skills> block, one
entry per skill with two honest, distinct numbers — index_line_bytes (the
always-on cost of listing the skill) and skill_md_bytes (on-disk SKILL.md
size, the real read cost paid only on skill_view). Sorted largest-first
by read cost.
render_breakdown prints both as sorted "Toolsets by size" / "Skills by
size" tables (skills capped at 20; --json carries them all). All existing
keys and output are unchanged.
Runs fully offline (dummy credentials, no network). Tests cover shapes,
largest-first ordering, per-tool attribution reconciling to the total,
namespaced-name parsing, and unmapped-skill handling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A dedicated /context (alias /ctx) gateway slash command that gives a full
context-window view with:
- Usage gauge: visual bar + fraction + percentage + headroom
- Auto-compression threshold and how far away it is
- Compression count and how much the last one freed
- Cumulative session throughput (explicitly labelled as throughput,
NOT context size — each call re-sends the window)
- Cascading fallback: running agent → cached agent → SessionStore metadata
→ rough transcript estimate
Not included (per current-main design):
- Cache reporting removed: commit 446b8e239 intentionally removed cache
reporting from user-facing surfaces because providers that omit cached-token
details produce misleading values
- Sync DB calls replaced with async_session_store (current main requires
AsyncSessionStore with await)
Also rewords the /status tokens line from 'Cumulative API tokens (re-sent
each call)' to 'Lifetime tokens billed: ... (not your current context size;
use /context)' to reduce the recurring confusion that the cumulative figure
is the current context window.
Fixes salvation of PR #52184 (salvage commit replaces a 12K-commit-behind
fork branch with a fresh implementation against current main, incorporating
reviewer feedback from @whoislikemiha and the hermes-sweeper).
After N consecutive guardian denials in a session the deny message escalates to a hard-stop instruction. Inspired by ChatGPT Work auto-review circuit breaker.
Two halves close the 'legacy pythonw gateways survive updates forever' gap:
1. hermes update now regenerates the installed Scheduled Task / Startup
launcher scripts (gateway.cmd + gateway.vbs) during the gateway resume
phase. They are persistence artifacts written once at install time;
updates never touched them, so pre-aa2ae36c3f installs kept launching
the gateway through pythonw.exe forever — every descendant spawn
flashed a conhost (#54220/#56747) and, since #70344, the console-less
gateway died at startup with RuntimeError: sys.stderr is None (#71671).
The task /TR points at a stable script path, so rewriting the files
retargets it with no schtasks call and no UAC. No-op for modern
installs; best-effort so a failed refresh never fails the update.
2. _resolve_detached_python() normalizes a legacy pythonw.exe interpreter
to its sibling console python.exe when it exists, so the update
pause/resume argv-replay path (and any other caller handed a legacy
command line) respawns on the current design instead of faithfully
resurrecting the old one. Keeps pythonw when no sibling exists — a
failed respawn is worse than a console-less gateway.
Append a "⊙ goal 3/20" segment (turns used / turn budget) to the CLI
status bar whenever a standing /goal is active. Mirrors the desktop
composer goal indicator: active-goal-only — paused/done goals stay out
of the bar since they already print their own glyph lines in-thread.
- Snapshot: goal_active / goal_turns_used / goal_max_turns from the
cached GoalManager (in-memory attribute read, no DB hit per repaint).
- Rendered in all three width tiers of both _build_status_bar_text and
_get_status_bar_fragments, and it respects the /statusbar toggle for
free (the toggle gates _get_status_bar_fragments as a whole).
- Tests: segment composition, active-only contract, all width tiers.
Status-bar goal indicator concept from #43020.
Co-authored-by: Akshan Krithick <akshankrithick305@gmail.com>
Assisted-by: Claude Fable 5 via Hermes Agent
Maps CLAUDE.md/AGENTS.md, permission allowlists, MCP servers, skills, and memories into their Hermes equivalents. Follows the openclaw migration pattern. Inspired by ChatGPT Work import-from-another-agent onboarding.
Cross-surface coverage #23768 missed (per review feedback):
- tools/clarify_gateway.py: _ClarifyEntry carries a multi_select flag
(register() accepts it; signature() exposes it to adapters).
_coerce_text_response now parses multi-select replies — comma- or
space-separated numbers ('1,3' / '1 3'), exact labels, dedup — into a
JSON array string that _parse_multi_select_response decodes into a
list. Out-of-range/unknown tokens reject the reply (native button UI)
or fall back to custom text (awaiting_text/'Other' mode).
- gateway/run.py: _clarify_callback_sync accepts multi_select and
registers it on the pending entry.
- gateway/platforms/base.py: default numbered-list text fallback tells
the user multiple selections are allowed and how to reply.
- tui_gateway/server.py: clarify_callback passes multi_select through
the clarify.request payload as a hint; renderers without checkbox
support ignore the field and remain single-select-compatible.
- tests: 13 new gateway tests (flag storage, comma/space/single-number
parsing, label matching, out-of-range rejection, dedup, end-to-end
resolve, single-select regressions).
Follow-ups to the salvaged #23768 commit, which targeted a pre-79559214
codebase:
- agent/tool_executor.py + agent/agent_runtime_helpers.py: pass
multi_select at both current clarify dispatch points (the PR's
run_agent.py edits landed on dead code paths).
- tools/clarify_tool.py: replace the broad TypeError-retry in
_invoke_callback with inspect.signature detection, so a compatible
callback that raises TypeError internally is not invoked twice
(addresses hermes-sweeper review feedback on #23768).
- tests: cover single-invocation on internal TypeError, legacy 2-arg
callbacks, **kwargs callbacks, and registry handler multi_select
pass-through (schema arg → handler → callback).
An accepted mid-turn redirect wrote its correction over inflight_turn["user"].
That field is the only user text session.resume can replay, so the prompt that
started the turn was gone the moment the user typed again while it ran. On the
next resume the client rebuilt the thread without it.
Record corrections in their own list instead, alongside the prompt. Renamed
_replace_inflight_user to _record_inflight_correction now that it appends.
_start_inflight_turn rebuilds the dict wholesale, so corrections cannot leak
into a later turn.
The sidebar labelled sections and workspace lanes `loaded/total`, which
read as a progress bar people expected to fill up rather than a count of
loaded rows. Pricing that label cost a COUNT(*) per profile database on
every sidebar refresh, purely so the numerator and denominator could
differ.
Pagination only needs to know whether another page exists, and that comes
free from the rows the query already returned: a window that comes back
full means more remain on disk. Sections now show the loaded count alone,
and the backend reports per-profile `profiles_truncated` flags in place of
`total` / `profile_totals`.
hermes update's lazy-refresh pass re-asserts LAZY_DEPS pins whenever the
package is present (active_features() is presence-based). The
tool.trace_upload pin huggingface-hub==1.2.3 sat below transformers'
>=1.5.0,<2 requirement, so every update force-downgraded the shared
package and broke Hindsight local embeddings on daemon startup (#60783).
Keep the exact-pin security posture — no ranges — but move the pin to
1.24.0 (current) and bump uv.lock in lockstep (uv lock --upgrade-package
huggingface-hub: hub 1.4.1->1.24.0, hf-xet 1.3.1->1.5.2, click
8.3.1->8.4.2, drops typer-slim), so the entire tree converges on ONE hub
version. The refresh pass now reports 'current' with zero churn.
Invariant tests (not snapshots): the lazy pin must equal the uv.lock
resolved version, and must sit inside transformers' accepted window.
HfApi surface used by trace upload (whoami/create_repo/upload_file)
verified present with identical kwargs on 1.24.0 in a live venv.
Compile and checksum-pin SQLite 3.53.4 in the published image, preserve Hermes' required SQLite features, and assert the final Python linkage plus FTS5 trigram behavior during image builds.\n\nMake doctor remediation install-aware so Docker users pull and recreate every Hermes container instead of running the inapplicable git updater.\n\nFixes #70480
The managed uv is installed with UV_UNMANAGED_INSTALL, which disables
'uv self update' by design — the swallowed failure left its embedded
python-build-standalone catalog frozen at bootstrap age forever.
python-build-standalone re-releases existing patch versions with fixed
SQLite (3.11.15 was re-cut with 3.53.1), so a stale catalog resolves
the same version number to the OLD vulnerable build, the probe rejects
it, and the patch-retry loop cannot recover because the fixed build
carries no newer number to try. Result: 'hermes update' printed a
guaranteed-failure provisioning warning on every run (issue #72093).
- When provisioning fails, re-bootstrap the Hermes-managed uv binary
via the official installer (the only supported refresh for unmanaged
installs) and retry provisioning once — only when the binary version
actually changed, so no wasted download cycles.
- Never touch a caller-supplied uv outside the managed path.
- Soften the failure report from alarming ⚠ to informational ℹ and say
why it is safe to wait: the WAL gate keeps databases out of WAL on
vulnerable builds, and the next update retries.
Verified: 56 unit tests green; sabotage run (retry block removed) fails
the 3 new retry tests; live E2E replaced a fake managed uv via the real
astral installer and the refreshed binary resolved the 3.11 catalog.
Fixes#72093
A providers: entry with only a default_model/model (no explicit models:
list) is un-narrowed — the singular field is just the active selection.
Section 3 derived has_explicit_models from the merged models list, so
the lone default_model entry counted as an explicit catalog and
suppressed the /v1/models probe for no-key endpoints, leaving a
one-line /model picker menu for local llama.cpp/Ollama/vLLM servers.
Track explicit models: declarations separately at group-build time
(mirrors section 4's declaration-tracking from #40542 / PR #61928) and
gate the probe on that instead.
Salvaged from PR #68984 by @vigilancetech-com (the probe_custom_providers
gate removal in that PR is not taken — the GUI no-probe gate is
intentional).
Completes #51690 on top of the salvaged #60378 timeout metadata:
- async_delegation: terminal 'stalled' events now carry structured
stall context (stalled_after_quiet_seconds, stall_threshold_seconds,
stall_phase idle|in_tool, stall_grace_seconds) on both single and
batch paths, persisted in the durable row so restart-restored events
keep it. Mirrors the sync path's timeout_seconds/timed_out_after_
seconds/timeout_phase from #60378.
- list_async_delegations(): exposes seconds_since_progress and live
children_activity (per-child api_calls, current_tool,
seconds_since_activity) sampled from the dispatch's progress_fn
outside the records lock; private monitor bookkeeping and callables
never leak.
- /agents (CLI + gateway): background delegations render per-child
activity rows, quiet-time hints, and the stalling state; gateway
section is new (previously async delegations were invisible there).
New locale key gateway.agents.background_delegations in all 17
catalogs.
Tests: stall-metadata event shape, live-listing projection, gateway
/agents rendering (real registry dispatch, sabotage-verified), sync
timeout metadata fields, non-timeout None contract.
Follow-up to the salvaged #71756: instead of webhook importing cron's
private _is_cron_silence_response, the loose autonomous-lane matcher now
lives in gateway/response_filters.py as is_autonomous_silence_response,
sharing LIVE_GATEWAY_SILENT_MARKERS with the interactive exact-marker
rule so the marker sets can never drift. Cron and webhook both delegate
to it. Interactive gateway behavior unchanged.
A webhook route that answered `[SILENT]` still delivered, whenever the model
added a sentence saying why it was staying quiet:
[SILENT]
The new inbound was the same email quoted back a second time, on a ticket
we already answered. Nothing new to reply to, so I closed it.
Webhook subscription prompts tell the agent to answer `[SILENT]` on a tick that
produced no story — a duplicate inbound, a stand-down because a sibling lane
already replied, a routine close. Nobody is waiting on the other end of a
webhook, so a "nothing happened" message has no reader.
Delivery went through the live gateway's `is_intentional_silence_response`,
which requires the response to be EXACTLY a marker. That rule is right for an
interactive chat: swallowing a real answer because it opens with a marker is
much worse than showing a stray marker. It is the wrong trade for an autonomous
lane, where a leaked non-story is a pointless notification on every tick and
models reliably append the explanation that flips the check back to "deliver".
Cron already resolved this the other way — `cron/scheduler.py` treats a marker
on its own first or last line as silence — so the two autonomous lanes
disagreed while the interactive path was fine.
Suppress in `WebhookAdapter.send`, before the deliver-type switch, so every
route (log, github_comment, cross-platform) behaves the same. Reuses cron's
`_is_cron_silence_response` rather than restating the rule, so the two lanes
cannot drift; prose that merely mentions a marker mid-sentence still delivers.
The interactive gateway path is untouched.
Tests: six cases in tests/gateway/test_webhook_adapter.py — bare marker,
marker + trailing prose (the reported shape), marker on the last line, a real
report, a report quoting a marker mid-sentence, and a `log` route. Verified
red-first: with the suppression removed the three silence cases fail
("Expected send to not have been awaited") while the three delivery cases still
pass, so the tests assert the fix rather than the framework.
