Follow-up to the salvaged #57634 commits:
- agent/manual_compression_feedback.py: new describe_compression_lock_skip()
— single source of truth for lock-skip wording. A descriptive holder
string means another compressor CONFIRMED holds the lock ('already in
progress (holder: ...)'); True/None means acquisition failed without a
confirmed holder (hermes_state.try_acquire_compression_lock catches
sqlite3.Error internally and returns False), so the message says
'could not acquire ... the lock check failed' instead of falsely
claiming a concurrent compression is running.
- cli.py, gateway/slash_commands.py, tui_gateway/server.py (all three
in-process consumers: session.compress RPC, command.dispatch compress
branch, slash.exec mirror) now route through the shared helper.
- tui_gateway/server.py command.dispatch compress branch: catch
CompressionLockHeld explicitly — it previously fell into the generic
'compress failed' error handler.
- Deferred-notify contract (#69324): lock-skip discards the pending
context-engine notification (committed=False) in _compress_session_history
and the CLI path before returning.
- tests: lock-skip wording pins per surface, VISIBLE_COMPRESSION_MESSAGES
noise-filter carve-outs for both wordings, MagicMock signal opt-outs for
sibling tests added on main after the original PR.
Advisor review found a critical stale-signal leak: if auto-compress
sets _compression_skipped_due_to_lock during a lock-skip, a subsequent
successful manual /compress will see the stale signal, falsely report
'Compression already in progress', and discard the compression results.
Fix:
- compress_context clears _compression_skipped_due_to_lock = None at
entry so each call's outcome alone determines the signal.
- Unified gateway 'holder: unknown' drift to match CLI/TUI pattern
(omit holder clause when not a descriptive string).
- Added MagicMock opt-outs in 3 sibling test files broken by the new
signal check (test_compress_here, test_compress_focus,
test_compress_plugin_engine).
- Added stale-signal-leak invariant test proving the fix.
The anti-thrash guard (_ineffective_compression_count) was in-memory
only: a fresh compressor bound to a resumed, already-compacted session
started with compression_count=0 and a disarmed guard, so a
near-threshold session could legally re-compact once per process
restart, forever.
Persist the counter through the durable session-state channel,
mirroring the failure-cooldown (#54465) and fallback-streak (af7dceaf7)
pattern:
- hermes_state.py: sessions.compression_ineffective_count column
(declarative reconciliation adds it on existing DBs) +
get/set_compression_ineffective_count accessors.
- context_compressor.py: every strike/clear verdict routes through
_record_ineffective_compression_verdict() which writes through to the
session row (no-change verdicts skip the DB write);
bind_session_state() loads the persisted value; the compression
rotation boundary carries the counter onto the child row;
update_model()'s reset also clears the durable copy; the
ineffective-only fast path in _automatic_compression_blocked() is
removed because the counter is now durable and another agent's clear
must unblock a stale local snapshot.
- conversation_compression.py: _refresh_persisted_compression_guards
re-reads the counter alongside cooldown + fallback streak.
Reset semantics are unchanged: any real provider reading below the
threshold still clears the counter — and now clears it durably too.
Resolves the residual gap identified in #54923 by @lanyusea (the
second-threshold mechanism was superseded by persisting the existing
guard state).
Co-authored-by: lanyusea <lanyusea@gmail.com>
Follow-up to srojk34's explicit-provider unwrap (PR #56691):
- Extract _resolve_moa_aggregator() as the single preset->aggregator
resolver shared by _resolve_auto(), _resolve_task_provider_model(),
and resolve_provider_client() so preset lookup/validation can't drift.
- When the main provider is moa, the aggregator model is now the default
for every UNSET auxiliary model: _read_main_model_for_aux() substitutes
the preset's acting (aggregator) model wherever fallback chains
pre-filled from _read_main_model() (router prefill, custom-endpoint
fallback, named-custom default, external-process default,
_try_main_agent_model_fallback).
- Unwrap moa at the resolve_provider_client() chokepoint so direct
callers (vision auto-detect, plugin code) can't dead-end in the
unknown-provider branch, and unwrap the vision auto-detect main
provider before capability probes run against the preset name.
- Real-config tests: temp HERMES_HOME + actual config.yaml exercising
the genuine load_config()/resolve_moa_preset() boundary.
_resolve_task_provider_model() returned an explicit provider="moa" override
(from a caller-passed arg, or auxiliary.<task>.provider: moa in config.yaml)
verbatim, with no MoA-preset unwrap. Only the *implicit* "main provider is
moa" path inside _resolve_auto() unwraps to the aggregator slot (#53827) —
this function never goes through _resolve_auto() at all, so the explicit
case was never covered.
MoA is a virtual provider with no real HTTP endpoint: resolve_provider_client()
looks "moa" up in PROVIDER_REGISTRY (no such entry), falls to the
unknown-provider dead end, and call_llm surfaces a nonsensical "Provider
'moa' is set in config.yaml but no API key was found. Set the MOA_API_KEY
environment variable..." error for a provider that was never meant to be
reached over the wire.
Fix mirrors #53827's aggregator-resolution approach exactly: when either the
explicit `provider` arg or the config-derived `cfg_provider` is "moa",
resolve the named (or default) MoA preset via resolve_moa_preset() and
continue with its aggregator's real provider+model, dropping any explicit
base_url/api_key (the moa:// virtual endpoint and placeholder key belong to
the facade, not the aggregator's real provider). If the preset can't be
resolved (renamed/deleted), degrades gracefully to the pre-fix behavior
instead of raising harder.
- agent/auxiliary_client.py: _unwrap_moa_provider() helper + call sites for
both the explicit-arg and config-derived provider="moa" cases in
_resolve_task_provider_model(). Also tightened base_url/api_key parameter
types to Optional[str] (matching their actual None-accepting behavior),
which incidentally resolved 5 pre-existing ty diagnostics at call sites.
- 5 new regression tests in tests/agent/test_auxiliary_client.py: explicit
arg unwrap, config-derived unwrap, default-preset fallback when no model
is configured, graceful degradation on preset-resolution failure, and a
non-moa regression guard.
Six spawn sites reachable from the desktop GUI / TUI gateway lacked CREATE_NO_WINDOW, so a windowless parent (pythonw/Electron) flashed a conhost per spawn: cli.exec RPC, quick-commands exec dispatch, and shell.exec RPC in tui_gateway/server.py; the CLI REPL quick-commands exec in cli.py; and the per-session provider transports in agent/copilot_acp_client.py and agent/transports/codex_app_server.py (Popen, hide-only so PIPE stdio stays intact).
All use hermes_cli._subprocess_compat.windows_hide_flags() (no-op on POSIX), matching the pattern already used at three other sites in tui_gateway/server.py. Deliberately hide-only — no detach flags, no Electron changes (per the #54220 revert history).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review follow-up: the auth path called pool.try_refresh_current() before
the hinted rotation, so a stale current() pointer could force-refresh a
different, healthy entry — consuming its single-use refresh token, or
(for non-OAuth entries, where a forced refresh marks the entry exhausted
outright) killing it entirely before api_key_hint was ever consulted.
Use try_refresh_matching(api_key_hint=...) to resolve and refresh the
entry that supplied the failing key under the pool lock, falling back to
the previous behavior when no key is known.
Adds a regression test with current() deliberately pointed at the
healthy entry: on the old code the healthy entry is exhausted by the
forced refresh and the pool ends up fully offline; with the fix the
failing entry is exhausted and recovery rotates to the healthy one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recover_with_credential_pool identified "which credential failed" via
pool.current(), a shared mutable pointer that is advanced by every
select() (round-robin rotation, concurrent turns, and other processes
reloading the pool reset it to None). By the time recovery ran, it
routinely pointed at a different, healthy entry — mark_exhausted_and_rotate
then stamped the failing request's error message and reset time onto that
innocent entry. With round_robin and one hard-capped key this
deterministically exhausted the healthy key too and took the entire pool
offline ("no available entries") from a single rate-limited credential.
mark_exhausted_and_rotate already supports api_key_hint for exactly this
(the auxiliary-client path passes it); the main conversation-loop path
never did. Pass agent.api_key — kept in sync with the entry in use by
_swap_credential — as the hint on all four rotation call sites, and make
the "already exhausted → rotate immediately" pre-check look up the failing
entry by key with the same fallback to current().
Adds regression tests that fail on the old attribution logic: a fresh
pool (current() is None) failing on key B must mark entry B, never
entry A.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-up for the salvaged #35191: the mid-turn pre-API pressure emit in
conversation_loop.py and the idle-resume emit in turn_context.py were not
routed through automatic_compaction_status_message, so an engine with
emit_automatic_compaction_status=False still leaked those lines. Both now
resolve through the hook (phases "pre_api" and "idle") while keeping the
#69550 template constants as the default wording. Suppression also skips the
#69546 structured 'compacted' terminal edge for compress-phase events that
opened no visible phase; failure warnings (_emit_warning) remain never
suppressible, pinned by test.
Follow-up hardening on the salvaged merge-into-trailing-turn fix:
- Merge only into REAL user tails (_is_real_user_message probe). Merging
into scaffolding tails (continuation marker, summary-as-user handoff)
would upgrade them to real-user evidence after SessionDB projection
strips the flags, breaking zero-user provenance (#69292 -
_is_synthetic_compression_user_turn keys on the TODO_INJECTION_HEADER
content marker, which merge-at-tail would bury mid-content).
- Strip a previously merged snapshot block before re-injection so
repeated boundaries refresh rather than accumulate todo state, and
refresh a bare stale snapshot row in place instead of stacking a
duplicate (empty/stale-skip semantics from #26981 by @YLChen-007).
- Scaffolding tails keep the flagged standalone append (pre-#53890
status quo; adjacent user rows are repaired downstream by
repair_message_sequence / _merge_consecutive_roles).
After context compression, the preserved todo list was unconditionally
appended as a standalone user message. When the compressed transcript
already ends with a user message (common case), this creates consecutive
user/user turns — a role-alternation violation some providers reject.
Fix: fold the snapshot into the trailing user message (blank-line separated)
when one exists with plain-string content. Falls back to append when the
tail is non-user, empty, or has structured (list) content.
Rebased on current upstream/main.
Closes#53890
After multiple in-place compactions, short tool-heavy sessions can leave
nearly every remaining message inside protect_last_n while those messages
are huge completed file/tool outputs. The middle compress window then
makes no material token progress and the turn dies with
"Cannot compress further" (#61932).
Cap the prune message floor at the same bound as tail-cut, and under
pressure demote bulky protected-tail tool bodies (keeping a short recent
floor) so preflight can reclaim headroom without wiping the active ask.
Follow-up for salvaged #58629: thread the previous flush baseline through
the idle-compaction caller in turn_context.py (the one caller the original
PR predates), and add regression coverage that every early-return path in
compress_context (breaker skip, no-progress, plus the completed-rotation
boundary) resets the per-attempt in-place outcome so a stale
_last_compaction_in_place from an earlier successful in-place compaction
can never baseline unflushed turns as persisted.
Relocates the #20424 wiring: the preflight region moved out of
run_agent.py into agent/turn_context.py (and through the
compression.max_attempts unification, #69315), so the contributor's
elif branch is reapplied at its current home as the else arm of the
threshold dispatch chain.
Integration contracts:
- Byte-identical default: the built-in ContextCompressor inherits
ContextEngine.should_compress_preflight() -> False, so the default
path performs no compression and touches no turn bookkeeping
(pinned by test_builtin_compressor_default_sub_threshold_path_unchanged).
- Attempt-cap: the engine gets exactly ONE compress() pass per turn,
mutually exclusive with the cap-bounded threshold multi-pass loop,
so turn-start passes stay within the resolved
compression.max_attempts budget in every case.
- No-op blocking (#64382 / 377244f7c): an engine pass that no-ops
(_compress_context returns the input list object) neither sets nor
clears preflight_compression_blocked and does not re-baseline the
flush history — a sub-threshold maintenance no-op proves nothing
about over-threshold compressibility.
- Engine exceptions are swallowed at debug level; cooldown/defer/
codex-native gates run before the hook is ever consulted.
Salvaged from #20424 by @Beandon13. Fixes#20316.
Invalid-tool exhaustion and truncated-tool early returns skipped finalize_turn, leaving role=tool transcripts that become tool→user on the next turn for strict providers. Call close_interrupted_tool_sequence before persist on those paths (same as interrupt aborts).
On vulnerable SQLite (e.g. 3.50.4), do not enable WAL for fresh/non-WAL
shared databases — prefer DELETE instead. Leave existing on-disk WAL
alone (no live downgrade under concurrent gateway/cron openers). Surface
Python/SQLite version details as a doctor warning (#69784).
The usage gauge is Nous subscription-cap-only (used_fraction requires a cap;
non-Nous providers emit no headers, so no notice fires). A bare percentage
implied a universal unit that doesn't exist, so report the absolute dollars
used of the cap instead: used = cap - remaining, from micros (money-safe),
clamped to [0, cap]. Still a snapshot at band-crossing (re-emits on band
change, not every turn) to keep the single escalating line and stay quiet on
append-only surfaces (messaging pushes one message per crossing).
recover_with_credential_pool() called mark_exhausted_and_rotate() without
api_key_hint, causing it to fall back to current() or _select_unlocked().
When a prior rotation left current() as None, _select_unlocked() returned
the NEXT (healthy) entry instead of the one that actually failed — marking
the wrong credential as exhausted (#43747).
Extract the current API key from agent.api_key (or pool.current().runtime_api_key
as fallback) and pass it as api_key_hint to all 4 call sites.
Post-sweep audit of every compression status emission (conversation_loop,
turn_context, conversation_compression): the buffered overflow/attempt-cap
retry chatter (🗜️ 'Context too large…', 'Compressed X → Y, retrying…',
'Context reduced to…'), the #69332-reworded auto-lower notice
("Auto-lowered this session's threshold…"), the aux-provider-unavailable
notice, and the concurrent-compression skip all leaked past
_TELEGRAM_NOISY_STATUS_RE on chat platforms. Add anchored alternatives for
each; the ', retrying'/'— compressing' anchors keep manual /compress
feedback ('Compressed: 30 → 12 messages') and failure/abort notices
visible per the deliberate carve-outs.
Also extract every routine compression status string into importable
template constants in agent/conversation_compression.py (single source of
truth shared by all emission sites), so tests can iterate the actual
emitted wording instead of hand-copied literals.
A context-compaction handoff is persisted as an ordinary history message
but is not a real turn. The ACP history replay streamed it as a bare
user/agent message chunk, dropping the in-process _compressed_summary
marker, so ACP frontends (editors, vscode-hermes) rendered the entire
handoff as a regular message.
Tag replayed summary chunks under _meta.hermes (ACP's extensibility
channel), covering all three persistence shapes the compressor emits:
- standalone role="user" handoff -> compactionSummary: true
- standalone role="assistant" handoff (alternation-driven role pick)
-> compactionSummary: true
- merge-into-tail message (preserved tail content + appended summary)
-> containsCompactionSummary: true, a distinct key so clients that
collapse standalone summaries cannot hide the preserved real content
Detection honors the in-process metadata flag and falls back to a new
ContextCompressor.classify_summary_content() content classifier
(standalone/merged/None), so it also works for a DB-reloaded session
that lost the in-memory flag. _is_context_summary_content is now a thin
wrapper over the classifier, keeping existing callers unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Detect a billing wall once (agent/error_classifier → FailoverReason.billing) and
map it to a recovery link + label in one place, then carry that structured
BillingBlock to every surface instead of re-parsing free-form error text per
surface.
- agent/billing_links.py: provider-agnostic slug/host → (label, billing URL)
table (single source of truth), Nous-aware (is_nous routes to the in-app flow);
unknown providers degrade to a readable label with no invented URL.
- conversation_loop: both billing exit paths return a billing_block through one
helper; the guidance message carries the derived URL for every provider.
- gateway forwards billing_block on message.complete (it was dropped).
- @hermes/shared: BillingBlock type shared by desktop + TUI.
Removes Homebrew and PyPI wheel/sdist as Hermes distribution paths while
preserving the supported source, Docker, and Nix workflows.
Changes:
- Removes the Homebrew formula, PyPI publish workflow, sdist manifest
(MANIFEST.in), and wheel/sdist release-attachment logic from scripts/release.py.
- Keeps setuptools metadata and entry points required by editable installs
and Docker/Nix builds, but adds a setup.py guard that rejects wheel/sdist
builds outside a sealed Nix derivation (HERMES_NIX_BUILD=1).
- Removes pip/Homebrew install detection, PyPI update checks, the pip
self-update path, the deprecation-banner state, the postinstall subcommand,
wheel data-directory fallbacks in agent/i18n.py and hermes_constants.py,
and the ACP Registry manifest/version-lockstep release logic.
- Adds /nix/store/ path detection so `nix run` / `nix profile install`
installs (which don't set HERMES_MANAGED) are correctly identified as
"nix" rather than falling through to "git"/"unknown".
- Retired install-method values ("pip", "homebrew") in existing
.install_method stamps (both code-scoped and home-scoped) are ignored by
the allowlist reader and fall through to "unknown" instead of resurrecting
a retired enum value.
- Updates Nix packaging to ship bare runtime data (locales, optional-mcps)
through store symlinks and wrapper env vars instead of wheel data-files.
- Removes the ACP Registry manifest/icon and their version-lockstep tests.
- Deletes or rewrites packaging, pip-update, Homebrew, and ACP Registry
tests; adds parametrized coverage for the packaging build guard covering
BOTH sdist and wheel paths (the guards live in separate cmdclass entries
— a passing sdist test proves nothing about the wheel path).
- Updates installation/platform documentation and related user-facing copy.
- Adjusts the supply-chain scan so deleted install-hook files do not trigger
a finding, while additions or modifications still require the existing
ci-reviewed label gate.
Supported installation paths (unchanged):
- git installer (install.sh)
- Docker
- Nix/NixOS
- editable development installs (uv sync, uv pip install -e ., pip install -e .)
When Codex returns 429 usage_limit_reached, Hermes persists the provider's
reset_at on the pool entry and freezes the credential until it elapses --
which can be days out for weekly windows. But the upstream window can
reopen EARLY: the user redeems a banked rate-limit reset (Codex CLI /
ChatGPT UI), upgrades their plan, or OpenAI resets the window. Hermes
never re-checked, so it kept erroring with 'Codex provider quota
exhausted (429); retry after Ns' until a manual re-auth rewrote the
tokens (issue #43747, externally-reset variant).
- hermes_cli/auth.py: add _probe_codex_quota_restored() -- a throttled
(5 min/token) GET of the Codex /usage endpoint; quota counts as
restored when every reported window is <100% used. Add
clear_codex_pool_quota_cooldowns() to lift 429/quota-shaped cooldowns
from persisted pool entries (DEAD and auth-shaped entries untouched).
- resolve_codex_runtime_credentials(): before surfacing a pool-only
cooldown as 'quota exhausted', probe upstream; on a positive probe
clear the cooldown and return the pool credential.
- agent/credential_pool.py: _available_entries() probes frozen
openai-codex entries (clear_expired path only) and unfreezes them when
upstream confirms the reset.
- agent/account_usage.py: a successful /usage reset redemption now
clears persisted pool cooldowns immediately.
Negative paths preserved: probe 429/exhausted/indeterminate keeps the
cooldown; read-only enumeration never probes; non-JWT tokens never
probe (no network in hermetic tests).
The default `busy_input_mode: interrupt` now redirects the live turn instead
of hard-stopping it and re-queuing a fresh turn, wired consistently across
every first-party surface via the shared core primitive.
- CLI, gateway (busy + PRIORITY paths), TUI (`_handle_busy_submit`), desktop
(`session.redirect` RPC), and ACP call `redirect()` when the agent advertises
`_supports_active_turn_redirect`, and fall back to the proven interrupt +
next-turn queue for older runtimes.
- Redirect is gated to plain text with no attachments: captioned or
attachment-bearing events (including adapters that classify unknown media as
`TEXT`) stay queued so media is never dropped.
- ACP `cancel()` records the interrupted prompt, sets its cancel event, and
hard-stops the agent while holding `runtime_lock`, closing the
cancel-then-correct ordering gap; connection I/O happens after the lock is
released.
- Desktop appends the correction as a real user transcript message so the live
view matches the durable history after reload.
- `/busy` help, onboarding hints, and the new `session.redirect` RPC describe
the redirect behavior; `/stop` remains the hard stop.
The Codex app-server runtime bypasses the main conversation loop and drives
its own subprocess turn, so it needs first-class hooks rather than the
OpenAI-loop interrupt path.
- `AIAgent.interrupt()` now forwards a hard stop to
`CodexAppServerSession.request_interrupt()`, and `redirect()` uses Codex's
native `turn/steer` protocol instead of cancelling the subprocess.
- `run_turn()` no longer clears an interrupt that arrived during
`ensure_started()`: a stop landing mid-startup is honored before `turn/start`,
and the interrupt event is cleared on every exit path.
- `run_codex_app_server_turn()` mirrors the loop finalizer's interrupt handoff
(surface `interrupted` / `interrupt_message`, then `clear_interrupt()`) on
both the normal and exception early-return paths, so a hard stop can't leave
`_interrupt_requested` stale for the next turn.
A follow-up sent while the model is still generating previously ended the
turn: Hermes kept only the visible partial text (reasoning was display-only),
cleared the loop, and replayed the message as a fresh next turn. If the
correction referred to something that only appeared in the thinking stream,
the model no longer had that context.
Add `AIAgent.redirect(text)`: a corrective interrupt distinct from a hard
stop. It cancels only the in-flight model request (not tool workers or child
agents), stashes the correction under a lock shared with `interrupt()` so a
concurrent `/stop` always wins, and lets the loop rebuild the same logical
iteration. `_apply_active_turn_redirect()` checkpoints the reasoning that was
actually shown to the user plus any visible partial text as an ordinary
assistant message, then appends the correction as a real user turn — never
replaying incomplete signed/encrypted provider reasoning, and keeping strict
role alternation and prompt-cache stability intact. During tool execution it
degrades to `steer()` so a running tool finishes at a safe boundary.
`_fire_reasoning_delta` now only records reasoning that a display callback
actually consumed, so `show_reasoning: false` never leaks hidden provider
thinking into the persisted transcript.
Follow-up for the salvaged #55800 idle-compaction commit:
- turn_context.py: treat a skipped _compress_context (per-session
compression lock held by another path, failure cooldown, anti-thrash
breaker, codex-native routing) as a strict no-op — only re-baseline
conversation_history and re-anchor current_turn_user_idx after a REAL
compaction. Also re-anchor the user-message index after idle compaction
(the PR predates the reanchor helper).
- hermes_cli/config.py: add idle_compact_after_seconds: 0 to
DEFAULT_CONFIG's compression block (the PR only had
cli-config.yaml.example).
- gateway/run.py: add the idle-compaction status wording to
_TELEGRAM_NOISY_STATUS_RE so the new 💤 message stays out of
human-facing chat surfaces (routine compaction is silent by design);
pin it in tests/gateway/test_telegram_noise_filter.py.
- docs: idle_compact_after_seconds in user-guide/configuration.md and
developer-guide/context-compression-and-caching.md parameter table.
- tests/agent/test_idle_compaction_lock_and_guards.py: end-to-end
coverage with a real AIAgent + SessionDB proving the idle path honors
the per-session compression lock (added after the PR), the persisted
failure cooldown, and the anti-thrash breaker, and that the lock is
released after an idle-triggered compaction.
Salvaged from #55800 by @iso2kx. Implements #27579.
Long-lived sessions (e.g. a Telegram thread resumed over hours/days)
accumulate a large context that the existing size-based threshold only
trims once it crosses `threshold × context_window`. Until then every
turn re-reads the full history, which on large-context models can mean
hundreds of K of cache-read tokens per call even across long idle gaps.
Add a time-based trigger that complements (does not replace) the size
threshold: when a session resumes after `compression.idle_compact_after_seconds`
of inactivity, compact the accumulated history up front, before the first
reply. Disabled by default (0), so existing behaviour is unchanged.
The trigger reuses `_last_activity_ts` (the last time the turn loop did
work) to measure the idle gap at turn start, gates the token estimate
behind a cheap gap pre-check, and skips compaction when the context is
already at/below the post-compression target (threshold × target_ratio)
so a short idle thread never pays for a summarization that saves nothing.
It also defers to an active compression-failure cooldown.
The decision is factored into a pure predicate, `_should_idle_compact`,
which is unit-tested without a live agent.
Composing #57835's multi-fossil summary scan with #47274's merged-handoff
unwrap: when the restart-decay path pulls a merged handoff into the
compression window, its genuine prior-tail user content must enter the
summarizer input (folded into the fresh summary) rather than being
dropped with the summary row. Standalone handoffs still drop. The
continuity test now pins the composed contract: recovered verbatim OR
via summarizer input, never silently deleted, never duplicated.
protect_first_n decay state (compression_count / _previous_summary) is
in-memory only, so a gateway restart re-protected the persisted handoff
summary and head fossils, growing the head unboundedly across
restart+compaction cycles (#57814).
_effective_protect_first_n now probes a bounded resumed-head window for
a persisted handoff summary (by metadata or content prefix) and decays
protection when one is found, before compress_start is computed. The
first post-restart compaction self-heals: stacked summary fossils are
folded into the next summary prompt instead of preserved verbatim,
rehydration is rolled back on abort, deterministic fallback carries a
bounded redacted previous-summary snapshot, and forced user-leading
merged summaries keep the live tail request after the summary end
marker so they stay rehydratable.
Squashed reapply of the 12-commit series from PR #57835 onto current
main (branch was 1210 commits behind; single add/add conflict with the
task-snapshot grounding helpers resolved by keeping both).
Fixes#57814.
- aux summary call on main intentionally omits max_tokens; use .get() in the
telemetry hook (and widen the param type) so the hook never breaks the call
- update test expectation: aux_output_reservation is None on main
- record no_progress failure_class in the no-progress boundary branch
Follow-up for salvaged PR #60444.
The summary_idx head-copy skip (from #69302) dropped the entire merged
handoff message, deleting the genuine prior-tail user content that
#47274's _strip_context_summary_handoff_message correctly unwraps.
Strip handles both shapes: standalone handoffs drop, merged handoffs
keep their real content. Caught by
test_recompression_of_current_merged_handoff_preserves_prior_tail_once
when both PRs landed together.
Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.
This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.
Rework from PR #24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
(threshold_tokens_cap parameter on ContextCompressor.__init__),
not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
switches and fallback activations (the old approach was undone by
update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
already contains background-review memory isolation in 973f27e95).
Config example:
compression:
threshold: 0.50
threshold_tokens: 200000 # never compress later than 200K tokens
Compaction summaries persist across sessions and re-enter every subsequent
summarizer prompt, but every redact_sensitive_text() call in
context_compressor.py used default mode: a no-op under
security.redact_secrets:false, and opaque OAuth-callback / URL-userinfo
credentials passed through even when enabled. The stored _previous_summary
also re-entered the iterative-update prompt unredacted.
Add _redact_compaction_text() — redact_sensitive_text(force=True,
redact_url_credentials=True) — and thread it through all compaction text
boundaries: serializer input (content + tool args), deterministic fallback
summary, summarizer LLM output, manual + auto focus topics, the latest-user
task snapshot, and _previous_summary re-entry.
Note: force=True at this boundary intentionally overrides
security.redact_secrets:false — that opt-out targets live tool output, not
persisted summaries.
Salvages the compaction half of #49556 (the redact.py strict-URL half
landed independently via 75af6dc57/62a00a739). Addresses #43666 item 2.
Co-authored-by: AndrewMoryakov <topazd2@gmail.com>
