Removes Homebrew and PyPI wheel/sdist as Hermes distribution paths while
preserving the supported source, Docker, and Nix workflows.
Changes:
- Removes the Homebrew formula, PyPI publish workflow, sdist manifest
(MANIFEST.in), and wheel/sdist release-attachment logic from scripts/release.py.
- Keeps setuptools metadata and entry points required by editable installs
and Docker/Nix builds, but adds a setup.py guard that rejects wheel/sdist
builds outside a sealed Nix derivation (HERMES_NIX_BUILD=1).
- Removes pip/Homebrew install detection, PyPI update checks, the pip
self-update path, the deprecation-banner state, the postinstall subcommand,
wheel data-directory fallbacks in agent/i18n.py and hermes_constants.py,
and the ACP Registry manifest/version-lockstep release logic.
- Adds /nix/store/ path detection so `nix run` / `nix profile install`
installs (which don't set HERMES_MANAGED) are correctly identified as
"nix" rather than falling through to "git"/"unknown".
- Retired install-method values ("pip", "homebrew") in existing
.install_method stamps (both code-scoped and home-scoped) are ignored by
the allowlist reader and fall through to "unknown" instead of resurrecting
a retired enum value.
- Updates Nix packaging to ship bare runtime data (locales, optional-mcps)
through store symlinks and wrapper env vars instead of wheel data-files.
- Removes the ACP Registry manifest/icon and their version-lockstep tests.
- Deletes or rewrites packaging, pip-update, Homebrew, and ACP Registry
tests; adds parametrized coverage for the packaging build guard covering
BOTH sdist and wheel paths (the guards live in separate cmdclass entries
— a passing sdist test proves nothing about the wheel path).
- Updates installation/platform documentation and related user-facing copy.
- Adjusts the supply-chain scan so deleted install-hook files do not trigger
a finding, while additions or modifications still require the existing
ci-reviewed label gate.
Supported installation paths (unchanged):
- git installer (install.sh)
- Docker
- Nix/NixOS
- editable development installs (uv sync, uv pip install -e ., pip install -e .)
When Codex returns 429 usage_limit_reached, Hermes persists the provider's
reset_at on the pool entry and freezes the credential until it elapses --
which can be days out for weekly windows. But the upstream window can
reopen EARLY: the user redeems a banked rate-limit reset (Codex CLI /
ChatGPT UI), upgrades their plan, or OpenAI resets the window. Hermes
never re-checked, so it kept erroring with 'Codex provider quota
exhausted (429); retry after Ns' until a manual re-auth rewrote the
tokens (issue #43747, externally-reset variant).
- hermes_cli/auth.py: add _probe_codex_quota_restored() -- a throttled
(5 min/token) GET of the Codex /usage endpoint; quota counts as
restored when every reported window is <100% used. Add
clear_codex_pool_quota_cooldowns() to lift 429/quota-shaped cooldowns
from persisted pool entries (DEAD and auth-shaped entries untouched).
- resolve_codex_runtime_credentials(): before surfacing a pool-only
cooldown as 'quota exhausted', probe upstream; on a positive probe
clear the cooldown and return the pool credential.
- agent/credential_pool.py: _available_entries() probes frozen
openai-codex entries (clear_expired path only) and unfreezes them when
upstream confirms the reset.
- agent/account_usage.py: a successful /usage reset redemption now
clears persisted pool cooldowns immediately.
Negative paths preserved: probe 429/exhausted/indeterminate keeps the
cooldown; read-only enumeration never probes; non-JWT tokens never
probe (no network in hermetic tests).
The default `busy_input_mode: interrupt` now redirects the live turn instead
of hard-stopping it and re-queuing a fresh turn, wired consistently across
every first-party surface via the shared core primitive.
- CLI, gateway (busy + PRIORITY paths), TUI (`_handle_busy_submit`), desktop
(`session.redirect` RPC), and ACP call `redirect()` when the agent advertises
`_supports_active_turn_redirect`, and fall back to the proven interrupt +
next-turn queue for older runtimes.
- Redirect is gated to plain text with no attachments: captioned or
attachment-bearing events (including adapters that classify unknown media as
`TEXT`) stay queued so media is never dropped.
- ACP `cancel()` records the interrupted prompt, sets its cancel event, and
hard-stops the agent while holding `runtime_lock`, closing the
cancel-then-correct ordering gap; connection I/O happens after the lock is
released.
- Desktop appends the correction as a real user transcript message so the live
view matches the durable history after reload.
- `/busy` help, onboarding hints, and the new `session.redirect` RPC describe
the redirect behavior; `/stop` remains the hard stop.
The Codex app-server runtime bypasses the main conversation loop and drives
its own subprocess turn, so it needs first-class hooks rather than the
OpenAI-loop interrupt path.
- `AIAgent.interrupt()` now forwards a hard stop to
`CodexAppServerSession.request_interrupt()`, and `redirect()` uses Codex's
native `turn/steer` protocol instead of cancelling the subprocess.
- `run_turn()` no longer clears an interrupt that arrived during
`ensure_started()`: a stop landing mid-startup is honored before `turn/start`,
and the interrupt event is cleared on every exit path.
- `run_codex_app_server_turn()` mirrors the loop finalizer's interrupt handoff
(surface `interrupted` / `interrupt_message`, then `clear_interrupt()`) on
both the normal and exception early-return paths, so a hard stop can't leave
`_interrupt_requested` stale for the next turn.
A follow-up sent while the model is still generating previously ended the
turn: Hermes kept only the visible partial text (reasoning was display-only),
cleared the loop, and replayed the message as a fresh next turn. If the
correction referred to something that only appeared in the thinking stream,
the model no longer had that context.
Add `AIAgent.redirect(text)`: a corrective interrupt distinct from a hard
stop. It cancels only the in-flight model request (not tool workers or child
agents), stashes the correction under a lock shared with `interrupt()` so a
concurrent `/stop` always wins, and lets the loop rebuild the same logical
iteration. `_apply_active_turn_redirect()` checkpoints the reasoning that was
actually shown to the user plus any visible partial text as an ordinary
assistant message, then appends the correction as a real user turn — never
replaying incomplete signed/encrypted provider reasoning, and keeping strict
role alternation and prompt-cache stability intact. During tool execution it
degrades to `steer()` so a running tool finishes at a safe boundary.
`_fire_reasoning_delta` now only records reasoning that a display callback
actually consumed, so `show_reasoning: false` never leaks hidden provider
thinking into the persisted transcript.
Follow-up for the salvaged #55800 idle-compaction commit:
- turn_context.py: treat a skipped _compress_context (per-session
compression lock held by another path, failure cooldown, anti-thrash
breaker, codex-native routing) as a strict no-op — only re-baseline
conversation_history and re-anchor current_turn_user_idx after a REAL
compaction. Also re-anchor the user-message index after idle compaction
(the PR predates the reanchor helper).
- hermes_cli/config.py: add idle_compact_after_seconds: 0 to
DEFAULT_CONFIG's compression block (the PR only had
cli-config.yaml.example).
- gateway/run.py: add the idle-compaction status wording to
_TELEGRAM_NOISY_STATUS_RE so the new 💤 message stays out of
human-facing chat surfaces (routine compaction is silent by design);
pin it in tests/gateway/test_telegram_noise_filter.py.
- docs: idle_compact_after_seconds in user-guide/configuration.md and
developer-guide/context-compression-and-caching.md parameter table.
- tests/agent/test_idle_compaction_lock_and_guards.py: end-to-end
coverage with a real AIAgent + SessionDB proving the idle path honors
the per-session compression lock (added after the PR), the persisted
failure cooldown, and the anti-thrash breaker, and that the lock is
released after an idle-triggered compaction.
Salvaged from #55800 by @iso2kx. Implements #27579.
Long-lived sessions (e.g. a Telegram thread resumed over hours/days)
accumulate a large context that the existing size-based threshold only
trims once it crosses `threshold × context_window`. Until then every
turn re-reads the full history, which on large-context models can mean
hundreds of K of cache-read tokens per call even across long idle gaps.
Add a time-based trigger that complements (does not replace) the size
threshold: when a session resumes after `compression.idle_compact_after_seconds`
of inactivity, compact the accumulated history up front, before the first
reply. Disabled by default (0), so existing behaviour is unchanged.
The trigger reuses `_last_activity_ts` (the last time the turn loop did
work) to measure the idle gap at turn start, gates the token estimate
behind a cheap gap pre-check, and skips compaction when the context is
already at/below the post-compression target (threshold × target_ratio)
so a short idle thread never pays for a summarization that saves nothing.
It also defers to an active compression-failure cooldown.
The decision is factored into a pure predicate, `_should_idle_compact`,
which is unit-tested without a live agent.
Composing #57835's multi-fossil summary scan with #47274's merged-handoff
unwrap: when the restart-decay path pulls a merged handoff into the
compression window, its genuine prior-tail user content must enter the
summarizer input (folded into the fresh summary) rather than being
dropped with the summary row. Standalone handoffs still drop. The
continuity test now pins the composed contract: recovered verbatim OR
via summarizer input, never silently deleted, never duplicated.
protect_first_n decay state (compression_count / _previous_summary) is
in-memory only, so a gateway restart re-protected the persisted handoff
summary and head fossils, growing the head unboundedly across
restart+compaction cycles (#57814).
_effective_protect_first_n now probes a bounded resumed-head window for
a persisted handoff summary (by metadata or content prefix) and decays
protection when one is found, before compress_start is computed. The
first post-restart compaction self-heals: stacked summary fossils are
folded into the next summary prompt instead of preserved verbatim,
rehydration is rolled back on abort, deterministic fallback carries a
bounded redacted previous-summary snapshot, and forced user-leading
merged summaries keep the live tail request after the summary end
marker so they stay rehydratable.
Squashed reapply of the 12-commit series from PR #57835 onto current
main (branch was 1210 commits behind; single add/add conflict with the
task-snapshot grounding helpers resolved by keeping both).
Fixes#57814.
- aux summary call on main intentionally omits max_tokens; use .get() in the
telemetry hook (and widen the param type) so the hook never breaks the call
- update test expectation: aux_output_reservation is None on main
- record no_progress failure_class in the no-progress boundary branch
Follow-up for salvaged PR #60444.
The summary_idx head-copy skip (from #69302) dropped the entire merged
handoff message, deleting the genuine prior-tail user content that
#47274's _strip_context_summary_handoff_message correctly unwraps.
Strip handles both shapes: standalone handoffs drop, merged handoffs
keep their real content. Caught by
test_recompression_of_current_merged_handoff_preserves_prior_tail_once
when both PRs landed together.
Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.
This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.
Rework from PR #24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
(threshold_tokens_cap parameter on ContextCompressor.__init__),
not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
switches and fallback activations (the old approach was undone by
update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
already contains background-review memory isolation in 973f27e95).
Config example:
compression:
threshold: 0.50
threshold_tokens: 200000 # never compress later than 200K tokens
Compaction summaries persist across sessions and re-enter every subsequent
summarizer prompt, but every redact_sensitive_text() call in
context_compressor.py used default mode: a no-op under
security.redact_secrets:false, and opaque OAuth-callback / URL-userinfo
credentials passed through even when enabled. The stored _previous_summary
also re-entered the iterative-update prompt unredacted.
Add _redact_compaction_text() — redact_sensitive_text(force=True,
redact_url_credentials=True) — and thread it through all compaction text
boundaries: serializer input (content + tool args), deterministic fallback
summary, summarizer LLM output, manual + auto focus topics, the latest-user
task snapshot, and _previous_summary re-entry.
Note: force=True at this boundary intentionally overrides
security.redact_secrets:false — that opt-out targets live tool output, not
persisted summaries.
Salvages the compaction half of #49556 (the redact.py strict-URL half
landed independently via 75af6dc57/62a00a739). Addresses #43666 item 2.
Co-authored-by: AndrewMoryakov <topazd2@gmail.com>
Follow-up to the salvaged contributor commit, closing the three gaps
flagged in the sweeper review:
1. Init ordering: assign compression.model_thresholds to a selected
plugin context engine BEFORE the initial update_model() call in
agent_init.py, so the initial model's override applies from init
(previously it only took effect after the first /model switch).
Base-class ContextEngine.update_model() now snapshots the
pre-override percent once so repeated switches fall back to the
engine's configured threshold, not a previous model's override.
2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to
hermes_cli/config.py — additive key, no _config_version bump.
3. Docs: document the key in
website/docs/developer-guide/context-compression-and-caching.md
(yaml example, parameter table, dedicated section) and update the
plugin-boundary note in context-engine-plugin.md to state the
explicit context-engine contract for model_thresholds.
Adds tests/run_agent/test_per_model_threshold_init_ordering.py:
plugin-engine AIAgent init regression (override applies at init,
empty map unchanged), DEFAULT_CONFIG key presence, floor interaction
on the model-switch path (override below the small-context floor is
raised to the floor; above the floor wins), and base-class config
snapshot across repeated switches. Also maps @bennybuoy in
contributors/emails/.
Addresses teknium1 review feedback on PR #60781:
1. Gateway cache invalidation: added ('compression', 'model_thresholds')
to _CACHE_BUSTING_CONFIG_KEYS so a live config edit to the map
invalidates the cached compressor (previously kept stale thresholds).
2. Integrated resolver with small-context floor: per-model overrides are
resolved FIRST, then the existing 75% floor for <512K models is applied
on top. The floor is no longer replaced — it stacks. An override below
75% on a small-context model still gets floored to 75% (raise-only);
an override above 75% wins.
3. Clean rebase on upstream main — no unrelated deletions or anti-thrashing
changes. Only the per-model threshold feature is added.
Changes:
- resolve_model_threshold() module-level helper (longest substring match)
- ContextCompressor.__init__ accepts model_thresholds dict
- _base_threshold_percent stores the per-model resolved value
- _config_threshold_percent stores the raw config value (fallback base)
- update_model() re-resolves on /model switch, falls back to config value
- ContextEngine base class update_model() applies overrides for plugin engines
- agent_init.py reads compression.model_thresholds from config, passes to ctor
- gateway/run.py cache busting key added
- cli-config.yaml.example documents the feature
- 17 tests covering resolve helper, compressor init (large/small context,
override above/below floor), update_model (re-resolve, fallback), base class
Co-authored-by: Copilot <copilot@github.com>
skip_memory=True was meant to skip the external memory *provider* for flush/
background agents, but it also suppressed creation of the built-in file-backed
MemoryStore. When a caller still enables the "memory" toolset, the memory tool
dispatched with store=None and every call failed with "Memory is not available",
silently losing the main automatic memory-capture path.
Now the built-in store is created whenever memory is enabled in config OR the
memory toolset is explicitly enabled, while the external-provider block stays
gated on skip_memory (preserving flush-agent intent).
The salvaged guard from #52276 only fired when a system prompt was
present in messages[] (system is not None after extraction). The live
repro of #52160 is the auto path: the system prompt is passed outside
messages[], so after the second compaction messages[0] is the
assistant-role summary with system=None — and the guard never ran.
Make _ensure_leading_user_turn unconditional, exactly mirroring the
Bedrock Converse adapter ('Converse requires the first message to be
from the user' — convert_messages_to_converse). Add a regression test
building the exact post-double-compaction shape (no system in
messages, messages[0]=assistant summary) asserting the converted
payload leads with a user turn, and update existing fixtures that
started with a bare assistant message to locate roles instead of
indexing result[0].
Anthropic extracts the system prompt into a separate `system` field and
requires messages[0] to be role="user"; a leading assistant turn is rejected
with HTTP 400. After a second context compaction the only surviving leading
anchor is the system prompt, so a summary/handoff message emitted as
role="assistant" becomes the first messages entry once the system prompt is
extracted. Anthropic reports this with a misleading error —
`messages.N: tool_use ids were found without tool_result blocks immediately
after: toolu_...` — even when every tool_use/tool_result pair is adjacent and
matched; the real structural defect is the leading assistant role (#52160).
This is engine-agnostic: it fires for any producer of a leading-assistant
transcript (built-in compressor, the DAG/LCM context engine, session
truncation), unlike the compressor-scoped fix in #52167. The native Bedrock
Converse adapter already guards the same invariant
(convert_messages_to_converse); this mirrors it for the native Anthropic path.
Add _ensure_leading_user_turn() to the convert_messages_to_anthropic
post-processing chain, scoped to the system-extracted case (the production
trigger) so bare assistant-only unit fixtures are unaffected. Adds regression
tests (leading-assistant, leading-assistant-with-adjacent-tool_use, and a
no-op negative control).
Two compounding defects in the holographic memory provider (#57682):
1. The on_session_end gate used plain truthiness on auto_extract, but the
plugin's own config schema declares it as a string enum with default
"false" — and not "false" is False, so extraction ran for users who
had it configured off. Coerce with the shared utils.is_truthy_value
(same fix class as the merged byterover no-op fix).
2. _auto_extract_facts scanned every role=user message. Context-compaction
handoff summaries can be inserted as role=user messages and their prose
reliably matches the decision patterns (we decided/agreed, the project
uses), so the compactor's own output was persisted as a durable project
fact on every rollover following a compaction — recreated even after
manual deletion.
Adds agent.context_compressor.is_compaction_summary_message(), a public
helper that prefers the in-process COMPRESSED_SUMMARY_METADATA_KEY marker
and falls back to _is_context_summary_content() (covers merged-into-tail
and historical prefixes), since the metadata key is stripped by wire
sanitizers and doesn't survive all session-store round-trips. The plugin
skips summary messages before pattern matching.
Fixes#57682
Review follow-up on #67431 (hermes-sweeper):
- The viability check compared the floored percentage against the raw
context window, but the built-in trigger recomputation also applies the
output-token reservation, the 64K floor, and the degenerate-window
guard (_compute_threshold_tokens). Mirror that math exactly, so e.g. a
200K window with max_tokens=120K recomputes to max(0.75*80K, 64K)=64K
and the suggestion is correctly KEPT for an 80K aux model instead of
being suppressed by the raw-window percentage.
- Gate the built-in policy behind isinstance(ContextCompressor): external
context engines own compaction policy (#44439), so plugin engines keep
the plain suggestion untouched.
- The non-viable explanation now names the recomputed trigger instead of
hardcoding the 75%/512K wording, so it stays accurate when the
reservation (not the percentage floor) is what makes the value
unreachable.
Tests: reservation-viability regression and plugin-engine passthrough,
per the review; the floored-branch assertion updated to the recomputed
number.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The auxiliary-compression feasibility warning computes its
compression.threshold suggestion as aux_context / main_context,
independently of ContextCompressor._effective_threshold_percent()'s
raise-only small-context floor. For main windows under 512K the floor
raises any configured value below 75% back up, so a suggestion like
'threshold: 0.40' is silently ignored and the same warning returns every
session.
Derive the suggestion's viability through the compressor's own floor
logic: offer the 'lower the threshold' option only when the floored value
still fits the auxiliary model's context; otherwise recommend only a
larger compression model and explain the floor, so the guidance is always
actionable.
Tests: the updated auto-correct test pins the floored branch (no
threshold suggestion, floor explained); two new tests pin the surviving
suggestion at/above the floor on a small window and below 75% on a
512K+ window where no floor applies. The updated test fails against the
previous code.
Fixes#67422
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refresh the agent activity tracker while context compression is blocked in the auxiliary summarizer so gateway watchdogs do not report inactivity during long compactions.
Add regression coverage for successful heartbeats, exception cleanup, touch failures, and strict-signature compressor fallback.
(cherry picked from commit c09e58b7709cc60c5b454701f0ecf840e759222f)
The salvaged estimator ran a per-character Python loop on every
estimate_tokens_rough() call — a ~28,000,000x slowdown vs (len+3)//4 on a
1MB ASCII tool output (measured ~3.0s per call). Gate it:
- str.isascii() O(1) fast path keeps pure-ASCII text bit-identical to the
classic (len+3)//4 rule at ~1.3x baseline cost (0.23us vs 0.17us per
1MB call).
- Non-ASCII text counts dense CJK chars via a compiled character-class
regex in C (len(text) - len(re.sub(''))): ~352ms/1MB hangul vs ~2.1s
for the per-char loop.
- Non-ASCII-but-non-CJK text (accents, Cyrillic, emoji) keeps the classic
rule.
Also: parity tests against the per-char reference implementation, and
updated two stale expectations that encoded the old behavior (CJK now
counted ~1 token/char; short string content now ceil-divided instead of
floored to 0). The continuity test now detects merged-into-tail summaries
via _is_context_summary_content.
Follow-up to the salvaged #64010 (Kenmege) and #63870 (dombejar) commits,
making one resolved compression.max_attempts cap govern ALL per-turn
compression attempt sites:
- conversation_loop: resolve max_compression_attempts ONCE at turn start
(it was previously re-resolved inside the API-call loop) and route the
pre-API pressure gate through it — that gate still hardcoded
'compression_attempts < 3' and logged 'attempt=%s/3'.
- conversation_loop: the salvaged post-tool compaction gate now uses the
resolved cap instead of a hardcoded 3.
- turn_context: the preflight compaction loop was 'for _pass in range(3)';
it now sizes itself from the same resolved cap.
- agent_init: harden the max_attempts parser — reject booleans (bool
subclasses int; 'true' would coerce to 1), reject fractional floats
instead of truncating them, keep accepting integral floats and numeric
strings; anything else falls back to 3 (floor 1, ceiling 10 unchanged).
- tests: replace #63870's inspect.getsource source-shape test with
behavioral loop tests (post-tool compaction fires <= cap times per turn,
shares its budget with the pre-API gate, resets between turns); add an
e2e test proving a 4th preflight pass runs at config cap=6 while the
unset default still stops at 3; extend the #64010 config tests with the
bool/float parser semantics.
Salvages #64010 by @Kenmege and #63870 by @dombejar.
The conversation loop hardcodes max_compression_attempts = 3. Sessions
that legitimately need more rounds are stranded: on a restart history
reload, incompressible tool schemas can keep the per-request estimate
above the compressor threshold even though the message floor compresses
correctly, so three rounds cannot clear it and the turn dies with
"Context length exceeded: max compression attempts (3) reached" — the
same failure class as #62605, where the rough estimate similarly leaves
3 retries short.
Make the cap a config key, compression.max_attempts:
- default 3 = identical to today, so an unset key is behavior-neutral;
- parsed and validated in agent_init alongside the other compression.*
keys (>= 1, hard-capped at 10, non-integer values fall back to 3),
attached as agent.max_compression_attempts;
- the loop reads it via getattr(agent, "max_compression_attempts", 3),
so objects without the attribute keep the prior behavior;
- documented in the DEFAULT_CONFIG compression block.
Tests pin the parse/validate/attach seam: default preserved, custom
value honored, floor and ceiling enforced, garbage tolerated, and the
loop-side getattr degradation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reworks the salvaged command module into a CommandSource(SecretSource)
registered as the third bundled source, composing with Bitwarden and
1Password through the apply_all() orchestrator — enable any combination
simultaneously. The original PR's secrets.provider single-selector is
deliberately dropped: multi-source is first-class and a mutually
exclusive provider switch would regress that.
- fetch() only fetches; precedence/override/conflicts/environ writes stay
in the orchestrator. ErrorKind classification + remediation hints.
- apply_command_secrets() kept as a legacy shim (parser/security helpers
unchanged: HERMES_SECRET_KEY data-only key passing, cross-key misroute
guard, base64-padding disambiguation, timeout + output cap, structured-
fields-only failure logging, stderr discarded).
- Dispatch tests rewritten for the registry path incl. an explicit
two-sources-compose test; selector tests removed with the selector.
- cli-config.yaml.example + docs page (command.md), secrets index entry.
- contributors mapping for mvalentin@valensys.net -> 0xr00tf3rr3t.
Brings the agent's secret-source system to parity with the desktop app's
`command` secrets provider (hermes-desktop src/main/secrets/commandProvider.ts),
so a vault helper configured for the desktop also resolves on the gateway/CLI.
NEW agent/secret_sources/command.py — ports the TS provider's security model:
- Runs a user-configured helper via `/bin/sh -c`; the requested key travels
ONLY in the HERMES_SECRET_KEY env var, never interpolated into the command
string, so a hostile key name is inert data (not code).
- parse_secret_output mirrors the TS parser: exact dotenv-key match wins; >=2
env-shaped lines without the wanted key -> None; otherwise a bare value;
base64 '='-padding disambiguation; cross-key misroute guard (a single
OTHER_KEY=realvalue line never leaks into a different wanted key).
- Hard 3s timeout (kills the whole process group via killpg, so a forking
helper can't keep the pipe open), 1 MiB output cap, POSIX-only (Windows
degrades to an empty result + warning). Every failure degrades to "no value";
it never raises and never blocks startup.
- Logs ONLY structured fields (code=/signal=/errno=) to stderr; the helper's
stderr is piped and DISCARDED; the command string and secret values are
never logged. Reuses bitwarden.py's FetchResult so env_loader consumes both
sources identically.
hermes_cli/env_loader.py — _apply_external_secret_sources now reads a unified
`secrets.provider` selector ("env" | "command" | "bitwarden"):
- provider=command routes to apply_command_secrets, records the provenance as
"command" in _SECRET_SOURCES (so format_secret_source_suffix labels keys
"(from command)" — already generic, not duplicated), and re-runs the ASCII
credential sanitizer like the bitwarden path.
- provider=bitwarden keeps the existing behavior byte-for-byte.
- env / unset is a no-op (today's default — zero change for existing users).
- BACK-COMPAT: a config with only `secrets.bitwarden.enabled: true` and no
`provider` key is treated as provider=bitwarden, so existing Bitwarden users
are unaffected.
Config (the provider selector, command path, timeouts) lives in config.yaml
under `secrets:` per the project rubric — only resolved secret VALUES touch env.
Tests: NEW tests/test_command_secret_source.py — 27 cases, E2E against a real
temp HERMES_HOME with real chmod+x shell helpers (not mocks): bare/dotenv/
base64 round-trip, cross-key misroute, injection-inert key (canary not
created), timeout kill within bound, non-zero-exit degrade, no-secret-in-logs,
precedence/override, dispatch via config.yaml provider:command, idempotency,
and back-compat bitwarden routing. 27 new + 50 baseline green; wider
secrets/env_loader/config surface 229 passed / 5 skipped, no regression.
Snapshot values applied by external secret sources per resolved HERMES_HOME so a later profile cannot replace an earlier profile scope through shared os.environ.
Keep provider and credential-pool fallback reads on the active secret scope, and fail closed on unscoped multiplex reads.
Tests: scripts/run_tests.sh tests/test_env_loader_secret_sources.py tests/test_env_loader_op_bootstrap.py tests/agent/test_secret_scope.py tests/agent/test_credential_pool.py tests/tools/test_credential_pool_env_fallback.py tests/hermes_cli/test_xiaomi_provider.py tests/cron/test_run_one_job.py tests/hermes_cli/test_api_key_providers.py tests/gateway/test_multiplex_credential_isolation.py -q (395 passed)
Fixes the profile-clobber bug cluster at the apply_all() chokepoint so
every secret source — bundled and plugin — gets both behaviors for free:
- secrets.preserve_existing (#58073): env var names whose existing .env /
shell value always wins, even against a source with
override_existing: true. Escape hatch for per-profile platform
secrets while everything else rotates centrally.
- Profile aliasing (#51447): under a named profile, an applied
FOO_<PROFILE> var (credential-shaped suffixes only) also hydrates the
canonical FOO, so adapters/plugins that read fixed env names see the
profile's value. Direct supply beats alias; protected/claimed/
override guards all apply; secrets.profile_alias: false disables.
Reimplements the intent of PR #58085 (tianma-if, preserve_existing on the
legacy Bitwarden apply shim) and PR #51616 (LeonSGP43, profile aliasing
inside the Bitwarden backend) on the SecretSource orchestrator that
superseded those code paths.
Fixes#58073. Fixes#51447.
Co-authored-by: tianma-if <5895871+tianma-if@users.noreply.github.com>
Co-authored-by: LeonSGP43 <154585401+LeonSGP43@users.noreply.github.com>
fdab380a1 wraps every cron job in a <home>/.env secret scope regardless of
deployment mode. get_secret() treats any installed scope as authoritative,
so in single-profile deployments where provider keys live only in the
process environment (systemd Environment=, pass-cli/op run wrappers, shell
exports) every cron credential read returns empty, the OpenAI client is
built with the no-key-required placeholder, and each scheduled job 401s —
while interactive turns keep working. Scope-miss reads now fall through to
os.environ when multiplexing is off; multiplexed scopes stay authoritative.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 1Password secret source builds a minimal allowlisted environment for the
`op read` child process. The allowlist omits OP_LOAD_DESKTOP_APP_SETTINGS, so a
user who exports it (shell, .env, or service unit) sees it silently stripped
before it reaches `op`.
That var is `op`'s documented switch to skip the desktop-app integration probe.
When the 1Password desktop app is installed, `op` probes its settings/socket at
startup *before* evaluating service-account auth. If the desktop app's group
container is wedged (e.g. macOS 'Interrupted system call' on the 1Password group
container), that probe blocks with no timeout, so `op read` hangs indefinitely
even with a valid OP_SERVICE_ACCOUNT_TOKEN present. Setting
OP_LOAD_DESKTOP_APP_SETTINGS=false is the intended escape hatch — but stripping
it means it has no effect on exactly the headless boxes that need it.
Fix: add OP_LOAD_DESKTOP_APP_SETTINGS to _OP_ENV_ALLOWLIST so the documented
var reaches the child. No behavior change when it's unset. Adds a focused test
alongside the existing allowlist test.
Repro: on a machine with a wedged 1Password desktop container + a valid SA
token, `op read` hangs 600s+ without the var and returns in ~4s with it — but
only if it actually reaches the op process, which this allowlist entry ensures.
Co-authored-by: Minh Nguyen <menhguin@users.noreply.github.com>
_auth_fingerprint() built the 1Password secret cache-key from the
service-account token, OP_ACCOUNT, and OP_SESSION_* vars but omitted
OP_CONNECT_HOST/OP_CONNECT_TOKEN, which are in _OP_ENV_ALLOWLIST and are
forwarded to the op child (the Connect-server auth path). Rotating
OP_CONNECT_TOKEN or re-pointing OP_CONNECT_HOST at a different Connect
identity left the fingerprint unchanged, so both the in-process and disk
caches kept serving secrets resolved under the old Connect credentials for
the full TTL (default 300s, disk-persisted across invocations). This
contradicts the function's own docstring invariant that a value cached
under a previous identity is never served under a new one; it closes the
gap for the Connect path, matching the OP_SESSION_*/service-account paths
that are already protected.
