Decompose children inherit the root's literal workspace_path (#37172),
so every sibling of a worktree-kind root points at the SAME checkout.
_resolve_worktree_workspace's existing-checkout shortcut then reuses
that directory on whatever branch is currently checked out, ignoring
the task's own branch_name. Net effect: sibling workers — which can be
promoted and dispatched concurrently — run in one directory on the
first sibling's branch, with no lock. Work lands on the wrong task's
branch (provenance corruption) and concurrent siblings trample each
other's index/tree.
Fix, two layers:
- decompose_triage_task: worktree-kind children no longer inherit the
root's literal path; each child materializes its own
<repo>/.worktrees/<child-id> at dispatch (dir/scratch inheritance
unchanged — children legitimately share those).
- _resolve_worktree_workspace: when the requested path is an existing
checkout of a DIFFERENT branch, fall back to a fresh
<repo>/.worktrees/<task-id> instead of silently reusing it (heals
rows that already carry a shared path). Same-branch reuse and the
no-repo/own-path degenerate cases keep the legacy behaviour.
Tests: tests/hermes_cli/test_kanban_worktree_isolation.py (5); full
test_kanban_db.py + test_kanban_decompose_db.py suites pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two related races in credential-pool cooldown state:
1. Lost update across processes: write_credential_pool merged only
entries missing from the caller's snapshot; for entries present on
both sides the caller's in-memory copy won wholesale. A process
holding a snapshot taken before another process marked a key
exhausted would, on its next persist (e.g. a round-robin rotation),
write the key back as healthy — erasing the cooldown so every
process resumes hammering a rate-limited key. Merge status fields by
last_status_at recency: adopt the on-disk status only when it is
strictly newer AND still binding (DEAD, or EXHAUSTED with an
unexpired cooldown), and never onto re-authed (token-changed)
entries, so legitimate expiry-clears and fresh logins are preserved.
2. Wrong-key quarantine: when mark_exhausted_and_rotate received an
api_key_hint that matched no entry, it fell through to
current()/_select_unlocked() — on a freshly loaded pool that selects
the NEXT healthy key and benches it for the full cooldown TTL,
punishing an innocent credential. When a hint is provided but
unmatched, rotate without marking anything instead of guessing.
Includes regression tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
git stash push --include-untracked exits non-zero when it saved
everything but could not DELETE some swept untracked files from the
working tree (e.g. a root-owned packaging/ directory left behind by a
sudo'd build: 'warning: failed to remove ...: Permission denied').
The updater ran the push with check=True, so this benign partial
failure raised CalledProcessError and aborted the whole update before
it even fetched — reliably, on every run, for any user with an
undeletable untracked path in the checkout.
Fix, both ends of the class:
- _stash_local_changes_if_needed: probe refs/stash before/after the
push. Non-zero push + fresh stash entry = changes are saved; warn,
reset the tracked-side leftovers (they're in the stash), and
continue the update. Non-zero push + NO stash entry = real failure;
keep aborting.
- _restore_stashed_changes: on restore, those same undeletable files
still sit in the tree, so 'git stash apply' exits 1 with 'already
exists, no checkout' even though every tracked change applied and
nothing was lost. Classify that stderr shape (strictly — any other
error line still routes to the conflict path) as restored instead
of resetting the tree and telling the user the restore failed.
Repro'd both halves with real git; behavioral E2E test covers
stash -> checkout -> restore round-trip with an undeletable dir.
Default kanban_create children now keep fresh scratch paths, while explicit dir sharing remains supported and project context resolves to a per-task worktree. Surface resolved workspace fields in create responses/events and cover scratch mutation, nesting, explicit sharing, and project inheritance.
Fixes#67567
The hermes console entry point is hermes_cli.main:main, and main.py imports
dotenv (via env_loader) and yaml (via config) at module level. In the #57828
failure state — a failed lazy backend refresh wiping a core package's import
files while metadata survives — a normal launch crashed while importing
main.py, before _recover_from_interrupted_install() and the recovery markers
from PR #58004 could act.
- hermes_cli/_early_recovery.py: stdlib-only bootstrap repair invoked at the
very top of main.py, before any third-party import. Probes the fragile
core packages via real imports, force-reinstalls broken ones using the
pyproject.toml pins, shares main.py's single-flight recovery lock, and
never clears markers (the confirmed lifecycle stays with the full recovery
path in main.py).
- Probe/repair tables now have one canonical home in _early_recovery, reused
by main.py so the two layers cannot drift.
- Manual --force-reinstall fallback commands now print pinned specs via
_lazy_refresh_repair_specs() instead of bare package names.
- tests: entry-point lifecycle coverage proving a broken dotenv import
crashes main.py without repair and imports cleanly with it, a stdlib-only
import guard for _early_recovery, and unit coverage for marker gating,
lock single-flight, pinned specs, and marker preservation.
Keep .update-incomplete for full .[all] recovery only. Lazy refresh uses
.lazy-refresh-incomplete and clears only after confirmed import probes;
unavailable probes are indeterminate, not healthy (#58004 review).
Keep .update-incomplete across normal hermes.exe launches, heal via
package-only import probes first, and only clear the marker after repair
succeeds (#57828 / #58004 review).
Upgrade pip before lazy refreshes, probe core imports when a lazy
install fails, force-reinstall corrupted packages with pyproject pins,
use package-only install (no shim quarantine) for repair, and keep the
.update-incomplete marker until refresh/repair succeeds (#57828).
The startup pruner only considered directories named hermes-* (the
hermes -w scratch trees), so salvage/review/port lanes created with raw
'git worktree add' accumulated forever — a real checkout reached 117
directories / 26 GB with trees dating back months. Two further leaks:
squash-merged branches' local commits stay unreachable from
refs/remotes/* forever, so the unpushed-commits guard preserved fully
merged scratch trees indefinitely; and preserved trees rotted silently
with no visibility.
- Pruner now covers every directory under .worktrees/ except kanban
task trees (t_<hex>, owned by 'hermes kanban gc'). Named (non
hermes-*) trees get a 3x timeline (72h soft / 9d hard) since they
were created deliberately.
- New _worktree_commits_all_merged_upstream(): git-cherry
patch-equivalence check against origin/HEAD|main|master, bounded at
20 commits ahead, fails safe toward preserve. Lets the pruner reap
trees whose every local-only commit already landed upstream via
squash-merge/cherry-pick.
- Dirty guard now applies at every tier (previously the 24-72h tier
skipped it — it only survived because the unpushed check usually
caught the same trees).
- Trees preserved for unpushed/dirty reasons older than 7 days are
listed in a single WARNING so in-flight work can't rot silently.
- tips.py text updated; 13 new behavior-contract tests.
The PR's inline toolset resolution (checking 'memory' in cli_toolsets
list) produced wrong results for composite toolsets like 'hermes-cli'
which expand to include the memory tool. Replace with the canonical
_get_platform_tools() from tools_config.py which correctly handles
composite toolsets and all edge cases.
Update tests to mock _get_platform_tools instead of raw config.
The 'Built-in: always active' label was a hardcoded string that never
reflected the user's actual configuration. It now shows three separate
indicators, each reading from the real source of truth:
- Memory injection: reads memory.memory_enabled from config.yaml
- User profile: reads memory.user_profile_enabled from config.yaml
- Memory tool: checks if 'memory' is in platform_toolsets.cli
(or defaults to enabled if no explicit list)
Before:
Built-in: always active
After:
Built-in (MEMORY.md / USER.md):
Memory injection: disabled ✗
User profile: disabled ✗
Memory tool: disabled ✗
Hoist the duplicated check_info(source_id) call out of both
if/else branches into a single call after the branch. Remove
trailing whitespace on the blank line after the except block.
Follow-up cleanup for PR #69981.
On vulnerable SQLite (e.g. 3.50.4), do not enable WAL for fresh/non-WAL
shared databases — prefer DELETE instead. Leave existing on-disk WAL
alone (no live downgrade under concurrent gateway/cron openers). Surface
Python/SQLite version details as a doctor warning (#69784).
Two fixes for desktop hands-free voice:
- The live speech session bound to the first assistant bubble with text, so
a tool-calling turn spoke only the opening narration and silently dropped
every later interim AND the final answer. The conversation selector now
aggregates all unspoken assistant bubbles in order (turn-scoped speech);
auto-speak keeps its latest-reply-only behavior.
- The speak-stream WS producer blocked forever on the text queue, so a
narration line with no trailing whitespace ("Let me check.") sat in the
sentence chunker until end-of-turn — spoken long after the tool finished,
with the UI stuck on "Preparing audio…". Mirror the CLI speaker's idle
flush: sentence-terminated buffers flush after 0.5s of producer silence,
anything else after ~2s; open <think> blocks are never flushed.
Cut steady-state Computer Use latency without changing default behavior
or waiting on cua-driver:
- Cap screenshots via set_config(max_image_dimension) on session start
(config: computer_use.max_image_dimension, default 1456)
- Cache aux-vision routing per (provider, model) so captures skip
repeated load_config()
- Add computer_use.capture_after_mode (default som) so users can opt
follow-ups down to ax (elements only) for speed
Auto-detect now disables the cursor overlay on darwin as well as
headless/WSL2 Linux. After start_session, also call
set_agent_cursor_enabled(false) when the policy is on so older drivers
without --no-overlay still tear the overlay down.
Co-authored-by: David Metcalfe <80915+DavidMetcalfe@users.noreply.github.com>
cua-driver's cursor overlay rendering loop can consume CPU indefinitely
when idle (#28152, #47032). On Linux/WSL2, the overlay serves no visual
purpose and the rendering path is the primary source of idle CPU usage.
Add computer_use.no_overlay config option (default: auto-detect) that
passes --no-overlay to cua-driver when enabled. Auto-detection disables
the overlay on Linux (covers WSL2, headless, containers) where it has no
benefit, and keeps it enabled on macOS/Windows where it is visually
useful.
Refs: #28152, #47032
Supersedes #52948 and #67402. Closes#52945.
Standalone hermes dashboard/serve was trusting HERMES_WEB_DIST and
HERMES_SERVE_HEADLESS inherited from a Desktop Electron parent, which
could serve the packaged desktop renderer ("Desktop IPC bridge is
unavailable") or disable the SPA. Drop only Electron-packaged WEB_DIST
paths (app.asar*) when HERMES_DESKTOP!=1, and clear inherited headless
for non-serve launches, while preserving caller-managed custom dist
overrides and the desktop-spawned backend path.
Co-authored-by: Bartok9 <danielrpike9@gmail.com>
Co-authored-by: Commander <commander@tianji.local>
Adds the missing write path for the per-task model_override column (which
was previously only settable via manual SQL) and pairs it with a
provider_override so cross-provider switches resolve correctly:
- kanban_db: provider_override column (+migration), set_model_override()
with model_override_set event, create_task(model_override=,
provider_override=), dispatcher spawns worker with -m <model>
[--provider <name>]
- dashboard: Model row in the task drawer — dropdown fed by a new
/model-options endpoint (build_models_payload substrate, provider-grouped,
free-text fallback), PATCH + bulk model override support
- CLI: kanban create --model/--provider, new kanban set-model subcommand,
show prints the provider
- agent tools: kanban_create accepts model/provider; show/list expose
provider_override
Rate-limit recovery flow: override is settable on running tasks and takes
effect on the next dispatch, without touching the worker profile's config.
* test(clarify-gateway): cover signature, timeout fallback, and notify paths for 100% coverage
Fixes#36531
(cherry picked from commit 5265dfe2f5)
* fix(clarify): one canonical timeout across CLI, TUI/desktop, and gateway
The clarify wait timeout was resolved three different (wrong) ways:
- CLI (`cli.py`, `hermes_cli/callbacks.py`) read a non-existent top-level
`clarify.timeout`, so it always fell through to a hardcoded 120s instead of
the canonical `agent.clarify_timeout` (default 3600) the gateway uses (#42969).
- The TUI/desktop bridge called `_block("clarify.request", …)` with no timeout,
so it used the hardcoded 300s `_block` default and ignored config (#51960).
- There was no way to disable the auto-skip: a user who wanted the agent to wait
indefinitely while they think couldn't get it.
Collapse all of this onto a single resolver:
- `tools.clarify_gateway.resolve_clarify_timeout(config)` is the one source of
truth. Order: explicit legacy `clarify.timeout` (back-compat) → canonical
`agent.clarify_timeout` → 3600. `<= 0` is preserved verbatim as "unlimited".
- CLI, callbacks, and the TUI bridge (`_clarify_timeout_seconds`) all route
through it, so the three surfaces can't drift.
- `<= 0` means unlimited everywhere: `wait_for_response` and `_block` drop the
deadline (heartbeat still fires), and the CLI hides its countdown.
Tests: resolver order / default / non-numeric / unlimited-sentinel; an
unlimited `wait_for_response` blocks until resolved rather than auto-skipping;
the TUI clarify bridge passes the configured timeout to `_block`.
Supersedes #42974 (CLI key), #51993 (TUI honors config), and #68986 (unlimited
wait); folds in #52031 (clarify_gateway coverage).
Co-authored-by: liuhao1024 <liuhao1024@users.noreply.github.com>
Co-authored-by: lkevincc0 <lkevincc0@users.noreply.github.com>
Co-authored-by: theone139344 <theone139344@users.noreply.github.com>
Co-authored-by: baauzi <baauzi@users.noreply.github.com>
---------
Co-authored-by: Christopher-Schulze <210261288+Christopher-Schulze@users.noreply.github.com>
Co-authored-by: liuhao1024 <liuhao1024@users.noreply.github.com>
Co-authored-by: lkevincc0 <lkevincc0@users.noreply.github.com>
Co-authored-by: theone139344 <theone139344@users.noreply.github.com>
Co-authored-by: baauzi <baauzi@users.noreply.github.com>
Classic /model confirmation already threaded custom_providers into the
context display, but the shrink-warning path did not. Probe-down then
matched the hardcoded "qwen" catalog (131072) and falsely warned that a
1M custom endpoint had shrunk — while the status bar still showed 1M.
Pass the same fresh inventory list used by switch_model/TUI (with
agent-snapshot fallback), and fall back to agent._custom_providers inside
merge_preflight_compression_warning when the kwarg is omitted.
Port of the Slack half of #13855 (by @kshitijk4poor), reimplemented against
the plugin adapter (the original PR targets the deleted
gateway/platforms/slack.py and six other legacy adapters).
Channels listed in slack.require_mention_channels (config.yaml) or
SLACK_REQUIRE_MENTION_CHANNELS ALWAYS require an explicit @mention, even
when require_mention is false globally or the channel is in
free_response_channels — the opposite direction of free_response_channels.
Instead of duplicating the PR's inline reply-to-bot-thread/mentioned-thread/
session checks, the forced channel falls through to the SAME decision chain
as normal mention gating, so all five wake checks in
_should_wake_on_unmentioned_message keep applying (single decision path).
Credit: adapted from #13855 by @kshitijk4poor (Slack half only; the other
platform halves target deleted legacy adapters and are out of scope for
this cluster).
Once the bot is @mentioned in a Slack thread it auto-follows and replies
to every later message in that thread, including ones a human addresses to
another human (e.g. "@rasha check this out"). The bot butts in.
Add slack.ignore_other_user_mentions (env SLACK_IGNORE_OTHER_USER_MENTIONS,
default off). When on, a channel/thread message whose first token @mentions
someone other than the bot is treated as addressed to that person and the
bot stays silent unless it is also mentioned. This is Slack parity for the
Discord option of the same name (#33501), adapted to Slack's thread model:
the trigger is a leading mention ("addressed to"), so a message that merely
references another user mid-sentence still reaches the bot.
The gate sits ahead of the free-response / require_mention ladder so it also
overrides the mentioned-thread auto-follow. DMs are never filtered.
/api/audio/speak-stream WebSocket: one socket + one Web Audio clock per
reply. The renderer feeds raw LLM deltas as they arrive; the server
cuts sentences with the shared chunker and streams int16 PCM back while
generation continues — speech overlaps generation with no per-sentence
connection or synthesis gaps. Falls back to the POST data-URL path for
old backends / non-chunked providers.
Barge-in runs a MediaRecorder on the monitor's stream the whole time
playback is live (rotated while quiet to bound pre-roll); talking over
the agent cuts playback and the complete utterance goes straight to
transcription and submit. /api/audio/transcribe returns 200/"" for
no-speech results so quiet turns re-listen instead of toasting a 400.
The streaming gate broadens from ElevenLabs-only to any provider that
passes check_tts_requirements(). In continuous voice mode a mic monitor
runs during playback: talking over the agent cuts TTS at detection
while the monitor keeps recording, then the captured interruption is
transcribed and queued as the next turn (process_loop's auto-restart
stands down while the capture owns the mic). New voice.barge_in config
key, default true.
Removes Homebrew and PyPI wheel/sdist as Hermes distribution paths while
preserving the supported source, Docker, and Nix workflows.
Changes:
- Removes the Homebrew formula, PyPI publish workflow, sdist manifest
(MANIFEST.in), and wheel/sdist release-attachment logic from scripts/release.py.
- Keeps setuptools metadata and entry points required by editable installs
and Docker/Nix builds, but adds a setup.py guard that rejects wheel/sdist
builds outside a sealed Nix derivation (HERMES_NIX_BUILD=1).
- Removes pip/Homebrew install detection, PyPI update checks, the pip
self-update path, the deprecation-banner state, the postinstall subcommand,
wheel data-directory fallbacks in agent/i18n.py and hermes_constants.py,
and the ACP Registry manifest/version-lockstep release logic.
- Adds /nix/store/ path detection so `nix run` / `nix profile install`
installs (which don't set HERMES_MANAGED) are correctly identified as
"nix" rather than falling through to "git"/"unknown".
- Retired install-method values ("pip", "homebrew") in existing
.install_method stamps (both code-scoped and home-scoped) are ignored by
the allowlist reader and fall through to "unknown" instead of resurrecting
a retired enum value.
- Updates Nix packaging to ship bare runtime data (locales, optional-mcps)
through store symlinks and wrapper env vars instead of wheel data-files.
- Removes the ACP Registry manifest/icon and their version-lockstep tests.
- Deletes or rewrites packaging, pip-update, Homebrew, and ACP Registry
tests; adds parametrized coverage for the packaging build guard covering
BOTH sdist and wheel paths (the guards live in separate cmdclass entries
— a passing sdist test proves nothing about the wheel path).
- Updates installation/platform documentation and related user-facing copy.
- Adjusts the supply-chain scan so deleted install-hook files do not trigger
a finding, while additions or modifications still require the existing
ci-reviewed label gate.
Supported installation paths (unchanged):
- git installer (install.sh)
- Docker
- Nix/NixOS
- editable development installs (uv sync, uv pip install -e ., pip install -e .)
_emit and _broadcast_global_event were each building the JSON-RPC event
envelope — extract _event_frame and use it from both. Type the registry as
set[Transport] (protocol already imported), and cut comment bloat at the
call sites. No behavior change; suites stay green.
Real-world failure from dogfooding the live-theme flow: display.skin was
already 'synthwave' in config, but the desktop never visibly applied it (the
activation event predated the WS transport fix / the connect). The desktop's
gateway.ready seed records the baseline WITHOUT painting (by design — never
stomp the persisted desktop theme on connect), so it believed it was synced.
Re-running 'hermes config set display.skin synthwave' then did nothing twice
over: the watcher signature (name, skin-file mtime) hadn't moved, so no
skin.changed fired; and even on an event, the desktop's name-equality guard
blocked the apply against the seeded baseline.
Two halves:
- hermes_cli: setting display.skin touches the named skin file so the
watcher signature always moves on an explicit set — a same-name re-affirm
now broadcasts skin.changed like any real move. Built-ins (no file) are
unaffected; a name switch already moves their signature.
- desktop: track whether the synced baseline was actually APPLIED vs merely
seeded at connect. A skin.changed matching a seed-only baseline is an
intentional apply and repaints; once applied, repeat same-name events stay
no-ops (protects a manual desktop-side theme switch from snap-back, incl.
across a reconnect re-seed).
When Codex returns 429 usage_limit_reached, Hermes persists the provider's
reset_at on the pool entry and freezes the credential until it elapses --
which can be days out for weekly windows. But the upstream window can
reopen EARLY: the user redeems a banked rate-limit reset (Codex CLI /
ChatGPT UI), upgrades their plan, or OpenAI resets the window. Hermes
never re-checked, so it kept erroring with 'Codex provider quota
exhausted (429); retry after Ns' until a manual re-auth rewrote the
tokens (issue #43747, externally-reset variant).
- hermes_cli/auth.py: add _probe_codex_quota_restored() -- a throttled
(5 min/token) GET of the Codex /usage endpoint; quota counts as
restored when every reported window is <100% used. Add
clear_codex_pool_quota_cooldowns() to lift 429/quota-shaped cooldowns
from persisted pool entries (DEAD and auth-shaped entries untouched).
- resolve_codex_runtime_credentials(): before surfacing a pool-only
cooldown as 'quota exhausted', probe upstream; on a positive probe
clear the cooldown and return the pool credential.
- agent/credential_pool.py: _available_entries() probes frozen
openai-codex entries (clear_expired path only) and unfreezes them when
upstream confirms the reset.
- agent/account_usage.py: a successful /usage reset redemption now
clears persisted pool cooldowns immediately.
Negative paths preserved: probe 429/exhausted/indeterminate keeps the
cooldown; read-only enumeration never probes; non-JWT tokens never
probe (no network in hermetic tests).
The default `busy_input_mode: interrupt` now redirects the live turn instead
of hard-stopping it and re-queuing a fresh turn, wired consistently across
every first-party surface via the shared core primitive.
- CLI, gateway (busy + PRIORITY paths), TUI (`_handle_busy_submit`), desktop
(`session.redirect` RPC), and ACP call `redirect()` when the agent advertises
`_supports_active_turn_redirect`, and fall back to the proven interrupt +
next-turn queue for older runtimes.
- Redirect is gated to plain text with no attachments: captioned or
attachment-bearing events (including adapters that classify unknown media as
`TEXT`) stay queued so media is never dropped.
- ACP `cancel()` records the interrupted prompt, sets its cancel event, and
hard-stops the agent while holding `runtime_lock`, closing the
cancel-then-correct ordering gap; connection I/O happens after the lock is
released.
- Desktop appends the correction as a real user transcript message so the live
view matches the durable history after reload.
- `/busy` help, onboarding hints, and the new `session.redirect` RPC describe
the redirect behavior; `/stop` remains the hard stop.
Follow-up for the salvaged #55800 idle-compaction commit:
- turn_context.py: treat a skipped _compress_context (per-session
compression lock held by another path, failure cooldown, anti-thrash
breaker, codex-native routing) as a strict no-op — only re-baseline
conversation_history and re-anchor current_turn_user_idx after a REAL
compaction. Also re-anchor the user-message index after idle compaction
(the PR predates the reanchor helper).
- hermes_cli/config.py: add idle_compact_after_seconds: 0 to
DEFAULT_CONFIG's compression block (the PR only had
cli-config.yaml.example).
- gateway/run.py: add the idle-compaction status wording to
_TELEGRAM_NOISY_STATUS_RE so the new 💤 message stays out of
human-facing chat surfaces (routine compaction is silent by design);
pin it in tests/gateway/test_telegram_noise_filter.py.
- docs: idle_compact_after_seconds in user-guide/configuration.md and
developer-guide/context-compression-and-caching.md parameter table.
- tests/agent/test_idle_compaction_lock_and_guards.py: end-to-end
coverage with a real AIAgent + SessionDB proving the idle path honors
the per-session compression lock (added after the PR), the persisted
failure cooldown, and the anti-thrash breaker, and that the lock is
released after an idle-triggered compaction.
Salvaged from #55800 by @iso2kx. Implements #27579.
Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.
This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.
Rework from PR #24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
(threshold_tokens_cap parameter on ContextCompressor.__init__),
not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
switches and fallback activations (the old approach was undone by
update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
already contains background-review memory isolation in 973f27e95).
Config example:
compression:
threshold: 0.50
threshold_tokens: 200000 # never compress later than 200K tokens
Integration layer for the cjk_unicode61 tokenizer, rebuilt on the v23
schema (the contributed integration in PR #65544 predated it):
- messages_fts_cjk: external-content FTS5 over a tool-row-excluding view
(same v23 storage discipline as the trigram index it supersedes — zero
inline text copies). Serves EVERY CJK query shape the legacy routing
split between trigram (>=3 chars/token) and LIKE full scans (1-2 char
tokens). Lone 1-char CJK runs and role_filter=['tool'] queries keep
their legacy routes.
- Dedicated marker pair (fts_cjk_rebuild_high_water/progress) gates the
id-scoped triggers, so a cjk-only backfill never gates the complete
messages_fts/trigram triggers.
- Transitions ride (the existing
throttled/resumable chunk engine): fresh DBs are born with the index;
legacy v22 DBs land on v23+cjk in one run; already-optimized v23 DBs
gaining the tokenizer get a marker-gated backfill; live writes are
indexed immediately in every case.
- Tokenizer-loss self-heal: a process that can't load the extension drops
the cjk triggers (writes keep working), leaves a stale breadcrumb, and
the index is rebuilt from scratch on the next optimize run — triggers
are never reinstalled over a gap (external-content 'delete' on an
unindexed rowid is the FTS5 corruption hazard the marker gating exists
to prevent).
- Capability classification: 'no such tokenizer: cjk_unicode61' joins the
degraded-runtime error class everywhere (read probe, write probe,
repair) so tokenizer absence is never misclassified as corruption.
- Config: sessions.cjk_fts (default on, inert without the .so) and
sessions.search_slow_ms in config.yaml, bridged to env by CLI + gateway
(startup + per-turn reload). build.sh falls back to vendored SQLite
headers so no libsqlite3-dev is needed.
Slow-query log path attribution updated: fts_cjk / fts5 / trigram /
like_scan. Tests: 14 lifecycle tests (fresh/legacy/stale/backfill paths,
tokenizer-loss round-trip) + 5 config-bridge tests + slow-log suite.
Follow-up to the salvaged contributor commit, closing the three gaps
flagged in the sweeper review:
1. Init ordering: assign compression.model_thresholds to a selected
plugin context engine BEFORE the initial update_model() call in
agent_init.py, so the initial model's override applies from init
(previously it only took effect after the first /model switch).
Base-class ContextEngine.update_model() now snapshots the
pre-override percent once so repeated switches fall back to the
engine's configured threshold, not a previous model's override.
2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to
hermes_cli/config.py — additive key, no _config_version bump.
3. Docs: document the key in
website/docs/developer-guide/context-compression-and-caching.md
(yaml example, parameter table, dedicated section) and update the
plugin-boundary note in context-engine-plugin.md to state the
explicit context-engine contract for model_thresholds.
Adds tests/run_agent/test_per_model_threshold_init_ordering.py:
plugin-engine AIAgent init regression (override applies at init,
empty map unchanged), DEFAULT_CONFIG key presence, floor interaction
on the model-switch path (override below the small-context floor is
raised to the floor; above the floor wins), and base-class config
snapshot across repeated switches. Also maps @bennybuoy in
contributors/emails/.
