Commit graph

16975 commits

Author SHA1 Message Date
brooklyn!
bb24364f25
Merge pull request #63104 from NousResearch/bb/active-turn-steering
feat: redirect active turns when users correct the agent
2026-07-22 13:54:32 -05:00
Bartok
c07973aa81
chore(gitignore): ignore installer .install_method stamp (salvage of #54855) (#67364)
* chore(gitignore): ignore installer .install_method stamp

Salvage of #54855 by @drissman — rebased onto current main with root-scoped
rule and sister-marker comments alongside .update-incomplete.

Closes #66189

Root cause: scripts/install.sh writes <install>/.install_method but git
did not ignore it, so managed checkouts show ?? .install_method and
hermes update may autostash the untracked marker.

Fix: add /.install_method to .gitignore (repo-root only).

Verification: git check-ignore -v .install_method

* test(update): assert .install_method survives update autostash (#66189)

Add hermetic regression mirroring the .hermes-bootstrap-complete test:
adopt the real .gitignore, drop the installer .install_method stamp, run
the exact 'git stash push --include-untracked' the updater uses, and assert
the marker is neither swept nor reported dirty. Requested by hermes-sweeper
review on #67364.
2026-07-22 14:40:44 -04:00
ethernet
5a40fd3777
Merge pull request #69453 from NousResearch/bb/stale-js-shadow-guard
fix(desktop): clean stale tsc emit + guard gateway WS URLs
2026-07-22 14:36:30 -04:00
ethernet
02fa447f8e fix(desktop): validate gateway WebSocket URLs 2026-07-22 14:31:53 -04:00
Teknium
fea838c9f2
fix(auth): detect upstream Codex quota resets and lift stale pool cooldowns (#69494)
When Codex returns 429 usage_limit_reached, Hermes persists the provider's
reset_at on the pool entry and freezes the credential until it elapses --
which can be days out for weekly windows. But the upstream window can
reopen EARLY: the user redeems a banked rate-limit reset (Codex CLI /
ChatGPT UI), upgrades their plan, or OpenAI resets the window. Hermes
never re-checked, so it kept erroring with 'Codex provider quota
exhausted (429); retry after Ns' until a manual re-auth rewrote the
tokens (issue #43747, externally-reset variant).

- hermes_cli/auth.py: add _probe_codex_quota_restored() -- a throttled
  (5 min/token) GET of the Codex /usage endpoint; quota counts as
  restored when every reported window is <100% used. Add
  clear_codex_pool_quota_cooldowns() to lift 429/quota-shaped cooldowns
  from persisted pool entries (DEAD and auth-shaped entries untouched).
- resolve_codex_runtime_credentials(): before surfacing a pool-only
  cooldown as 'quota exhausted', probe upstream; on a positive probe
  clear the cooldown and return the pool credential.
- agent/credential_pool.py: _available_entries() probes frozen
  openai-codex entries (clear_expired path only) and unfreezes them when
  upstream confirms the reset.
- agent/account_usage.py: a successful /usage reset redemption now
  clears persisted pool cooldowns immediately.

Negative paths preserved: probe 429/exhausted/indeterminate keeps the
cooldown; read-only enumeration never probes; non-JWT tokens never
probe (no network in hermetic tests).
2026-07-22 10:58:22 -07:00
Brooklyn Nicholson
2b27c171ca fix(redirect): cover the build-window and post-reconnect correction races
Two narrow timing windows (reported by null-runner) silently downgraded a
mid-turn correction to a plain next-turn message on the desktop client:

- Turn-build window: a fresh turn flips running=True and builds the agent
  asynchronously, so session["agent"] is briefly None. session.redirect
  answered 4010 "unsupported", which the renderer's catch swallowed into a
  lost follow-up. Queue the correction server-side instead and return
  status="queued" — lossless, and honest about what happened.

- Stale runtime id after reconnect: session.redirect 404s on a sid the
  gateway no longer maps. redirectPrompt now resumes the stored session and
  retries once, mirroring stopPrompt, so a correction fired right after a
  reconnect isn't dropped.

The desktop treats "queued" like "redirected": the correction reaches the
model either way, so it's recorded once as a real user message.
2026-07-22 12:48:14 -05:00
Teknium
b591afe1af test(state): cover pure-Latin embedded-in-CJK recovery via the cjk index
The zero-result fallback prefers messages_fts_cjk when built: exact
ranked token match for Latin runs unicode61 fused onto CJK, including
<3-char tokens the trigram leg can't recover.
2026-07-22 10:35:04 -07:00
gnanam1990
96560ee60f fix(agent): recover pure-Latin search matches embedded in CJK text (#54242)
A pure-Latin query (no CJK characters) routes to the unicode61
`messages_fts` table, whose tokenizer does not insert a boundary between
Latin letters and adjacent CJK characters. Content like "修改youer服务端" is
indexed as a single token, so `search_messages("youer")` returned zero
results even though the substring is present, and the Latin path had no
fallback.

Add a zero-result trigram fallback to the pure-Latin path: when the
unicode61 search misses, retry against the existing `messages_fts_trigram`
table, which matches substrings regardless of word boundaries. The fallback
is gated on `_trigram_available` and on every token being >=3 chars (the
trigram minimum), and only fires on a zero-result miss, so successful Latin
searches keep their unicode61 ranking unchanged.

The trigram query construction shared with the CJK path is extracted into a
`_run_trigram_search()` helper; the CJK branch is refactored to use it with
no behavior change.

Adds regression tests in tests/test_hermes_state.py::TestCJKSearchFallback.
2026-07-22 10:35:04 -07:00
brooklyn!
59614ef9af
Merge pull request #69505 from NousResearch/bb/inline-msg-actions
Flatten assistant message actions into an inline icon row
2026-07-22 12:24:20 -05:00
Brooklyn Nicholson
9160f10fc9 fix(desktop): tsc clean before dev / build
A stray tsc run can emit foo.js next to foo.ts under apps/shared/src or
apps/desktop/src. .gitignore hides the artifact from git status, but Vite
resolves extensionless imports .js-before-.ts, so the renderer silently runs
the stale compiled copy.

tsc -b . --clean already knows the emit graph and deletes
matching outputs. Run it before vite in all dev scripts.

This bit for real: a Jul 16 artifact of websocket-url.js predated the #68250
getGatewayWsUrl contract change ({ ok, wsUrl } IPC result), so its old
'if (fresh) return fresh' handed the whole result object to new WebSocket(),
dialing ws://127.0.0.1:5174/[object%20Object] on every boot. The desktop app
could never connect, and the failure survived reboots and cache wipes because
the poison lived in src/.

JsonRpcGatewayClient.connect() now rejects non-ws:// URLs with a readable
error instead of letting new WebSocket() coerce an object into
[object%20Object], so any future contract skew fails diagnosably.
2026-07-22 13:19:47 -04:00
Brooklyn Nicholson
ca244e2168 refine inline message actions: fork icon, sine-wave read-aloud, shared age util
- Use the fork glyph for branch and a sine wave for read aloud (all one lib now)
- Extract compact "2h ago" into formatAgo() in lib/time.ts (+ ageDays locale string)
- Cover formatAgo with a unit test
2026-07-22 12:18:59 -05:00
brooklyn!
4204e6d1cb
Merge pull request #69519 from NousResearch/bb/desktop-preview-open
feat(desktop): let the agent drive the shell — preview pane + pane focus
2026-07-22 12:18:52 -05:00
Brooklyn Nicholson
70ba3c4828 feat(desktop): agent can focus panes + shared desktop-UI event bridge
Extract the open_preview emitter into a shared tools/desktop_ui bridge
(one gateway-injected sink, routed by HERMES_UI_SESSION_ID) and add a
second desktop-gated tool on top of it:

- focus_pane(chat|files|terminal|review|sessions) -> pane.reveal event.
  The desktop runs each pane's own reveal path (revealDesktopPane table)
  and only acts on the active window -- a background turn never moves the
  user's focus (desktop AGENTS.md: offer, don't hijack).

open_preview now emits through the same bridge. Both tools are check_fn
on HERMES_DESKTOP (zero footprint elsewhere), sitting beside
read_terminal/close_terminal in _HERMES_CORE_TOOLS.

Deliberately not adding run_slash: letting the agent fire slash commands
mid-turn (/model, /new, /clear) fights prompt-cache + conversation
invariants.
2026-07-22 12:13:01 -05:00
Brooklyn Nicholson
3d40a1cbf2 feat(desktop): Cursor-style stop-and-correct on the composer
Plain Enter (and the primary send button) while a turn is running now redirects
the live turn with the typed correction instead of queueing it — matching
Cursor's stop-and-correct. Removes the now-redundant steering-wheel button
(redirect is the default gesture) and teaches the primary button the `steer`
action. Attachments still queue; slash commands still run inline.
2026-07-22 12:10:58 -05:00
Brooklyn Nicholson
79b047820f docs: describe active-turn redirect busy-input behavior
Update the CLI and messaging guides so the default `interrupt` mode reflects
the new behavior: a follow-up redirects the active turn (preserving displayed
reasoning and completed work, letting running tools finish at a safe boundary)
rather than hard-stopping it, with `/stop` still the explicit hard stop.
2026-07-22 12:10:58 -05:00
Brooklyn Nicholson
34d0de80e6 feat(surfaces): route busy-input corrections through active-turn redirect
The default `busy_input_mode: interrupt` now redirects the live turn instead
of hard-stopping it and re-queuing a fresh turn, wired consistently across
every first-party surface via the shared core primitive.

- CLI, gateway (busy + PRIORITY paths), TUI (`_handle_busy_submit`), desktop
  (`session.redirect` RPC), and ACP call `redirect()` when the agent advertises
  `_supports_active_turn_redirect`, and fall back to the proven interrupt +
  next-turn queue for older runtimes.
- Redirect is gated to plain text with no attachments: captioned or
  attachment-bearing events (including adapters that classify unknown media as
  `TEXT`) stay queued so media is never dropped.
- ACP `cancel()` records the interrupted prompt, sets its cancel event, and
  hard-stops the agent while holding `runtime_lock`, closing the
  cancel-then-correct ordering gap; connection I/O happens after the lock is
  released.
- Desktop appends the correction as a real user transcript message so the live
  view matches the durable history after reload.
- `/busy` help, onboarding hints, and the new `session.redirect` RPC describe
  the redirect behavior; `/stop` remains the hard stop.
2026-07-22 12:10:58 -05:00
Brooklyn Nicholson
e4877ba96e flatten assistant message actions into an inline icon row
Drop the kebab overflow so age, branch, copy, read aloud, and refresh are always one hover away.
2026-07-22 12:09:52 -05:00
Brooklyn Nicholson
f071f42244 feat(desktop): let the agent open the preview pane
Add a desktop-gated open_preview tool so 'open cnn.com in the preview
pane' works. The tool (check_fn on HERMES_DESKTOP, zero footprint
elsewhere) emits a preview.open event through a gateway-injected emitter,
mirroring the close_terminal -> terminal.close bridge. The desktop
handles it in usePreviewRouting, normalizing the target and opening the
pane for the active session only -- a background turn never hijacks it.

Bare domains and localhost are coaxed into fetchable URLs (www.cnn.com ->
https://, localhost:3000 -> http://); file paths and schemes pass through
to the renderer's normalizer.
2026-07-22 12:03:24 -05:00
Brooklyn Nicholson
f6d2ee4afc feat(codex): honor redirect and hard stop in the app-server runtime
The Codex app-server runtime bypasses the main conversation loop and drives
its own subprocess turn, so it needs first-class hooks rather than the
OpenAI-loop interrupt path.

- `AIAgent.interrupt()` now forwards a hard stop to
  `CodexAppServerSession.request_interrupt()`, and `redirect()` uses Codex's
  native `turn/steer` protocol instead of cancelling the subprocess.
- `run_turn()` no longer clears an interrupt that arrived during
  `ensure_started()`: a stop landing mid-startup is honored before `turn/start`,
  and the interrupt event is cleared on every exit path.
- `run_codex_app_server_turn()` mirrors the loop finalizer's interrupt handoff
  (surface `interrupted` / `interrupt_message`, then `clear_interrupt()`) on
  both the normal and exception early-return paths, so a hard stop can't leave
  `_interrupt_requested` stale for the next turn.
2026-07-22 12:02:40 -05:00
Brooklyn Nicholson
cbf5b05c70 feat(agent): add active-turn redirect core primitive
A follow-up sent while the model is still generating previously ended the
turn: Hermes kept only the visible partial text (reasoning was display-only),
cleared the loop, and replayed the message as a fresh next turn. If the
correction referred to something that only appeared in the thinking stream,
the model no longer had that context.

Add `AIAgent.redirect(text)`: a corrective interrupt distinct from a hard
stop. It cancels only the in-flight model request (not tool workers or child
agents), stashes the correction under a lock shared with `interrupt()` so a
concurrent `/stop` always wins, and lets the loop rebuild the same logical
iteration. `_apply_active_turn_redirect()` checkpoints the reasoning that was
actually shown to the user plus any visible partial text as an ordinary
assistant message, then appends the correction as a real user turn — never
replaying incomplete signed/encrypted provider reasoning, and keeping strict
role alternation and prompt-cache stability intact. During tool execution it
degrades to `steer()` so a running tool finishes at a safe boundary.

`_fire_reasoning_delta` now only records reasoning that a display callback
actually consumed, so `show_reasoning: false` never leaks hidden provider
thinking into the persisted transcript.
2026-07-22 12:02:40 -05:00
brooklyn!
9b1028f297
Merge pull request #69501 from NousResearch/bb/interim-message-actions
fix(desktop): no per-paragraph action bars on sealed interim messages
2026-07-22 11:37:33 -05:00
Brooklyn Nicholson
72dd01c553 fix(desktop): no per-paragraph action bars on sealed interim messages
Since #65919 the live view seals each chunk of mid-turn assistant
commentary (message.interim) as its own finalized bubble. Every bubble
with visible text renders the hover action footer, so a tool-heavy turn
grew a copy/refresh bar under almost every paragraph — and the live
render didn't match rehydration, which merges the turn into one bubble.

Mark sealed interim bubbles with ChatMessage.interim, carry the flag
into the runtime message metadata (custom.interim), and skip the
AssistantFooter for them. The turn's final reply keeps the footer; a
previewed final that settles onto an interim bubble clears the mark so
the settled reply regains its actions. interim joins COMPARED_FIELDS /
chatMessagesEquivalent so flipping it repaints.

Also fix an id-collision flake this surfaced: stream/interim bubble ids
were Date.now()-only, so an interim seal and the next segment's first
delta in the same millisecond reused the id and the new segment appended
into the sealed bubble. Ids now include a monotonic sequence.
2026-07-22 11:31:03 -05:00
hermes-seaeye[bot]
bcc3396b25
fmt(js): npm run fix on merge (#69503)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-07-22 16:30:14 +00:00
brooklyn!
163fab8d00
Merge pull request #69077 from NousResearch/bb/desktop-memory-dropdown-discovery
Some checks are pending
CI / Detect affected areas (push) Waiting to run
CI / Python tests (push) Blocked by required conditions
CI / Python lints (push) Blocked by required conditions
CI / JS & TS checks (push) Blocked by required conditions
CI / Desktop E2E (push) Blocked by required conditions
CI / Docs Site (push) Blocked by required conditions
CI / Deny unrelated histories (push) Blocked by required conditions
CI / Check contributors (push) Blocked by required conditions
CI / Check uv.lock (push) Blocked by required conditions
CI / package-lock.json diff (push) Blocked by required conditions
CI / Lint Docker scripts (push) Blocked by required conditions
CI / Build&Test Docker image (push) Blocked by required conditions
CI / Supply-chain scan (push) Blocked by required conditions
CI / Review label gate (push) Blocked by required conditions
CI / OSV scan (push) Waiting to run
CI / CI review comment (live) (push) Blocked by required conditions
CI / All required checks pass (push) Blocked by required conditions
CI / CI timing report (push) Blocked by required conditions
Deploy Site / deploy-vercel (push) Waiting to run
Deploy Site / deploy-docs (push) Waiting to run
auto-fix lint issues & formatting / Generate eslint --fix patch (push) Waiting to run
auto-fix lint issues & formatting / Apply patch (push) Blocked by required conditions
fix(desktop): feed memory.provider dropdown from live discovery
2026-07-22 11:18:37 -05:00
Teknium
1efe7094ad feat(compression): harden idle compaction — lock/guard interplay, config + docs
Follow-up for the salvaged #55800 idle-compaction commit:

- turn_context.py: treat a skipped _compress_context (per-session
  compression lock held by another path, failure cooldown, anti-thrash
  breaker, codex-native routing) as a strict no-op — only re-baseline
  conversation_history and re-anchor current_turn_user_idx after a REAL
  compaction. Also re-anchor the user-message index after idle compaction
  (the PR predates the reanchor helper).
- hermes_cli/config.py: add idle_compact_after_seconds: 0 to
  DEFAULT_CONFIG's compression block (the PR only had
  cli-config.yaml.example).
- gateway/run.py: add the idle-compaction status wording to
  _TELEGRAM_NOISY_STATUS_RE so the new 💤 message stays out of
  human-facing chat surfaces (routine compaction is silent by design);
  pin it in tests/gateway/test_telegram_noise_filter.py.
- docs: idle_compact_after_seconds in user-guide/configuration.md and
  developer-guide/context-compression-and-caching.md parameter table.
- tests/agent/test_idle_compaction_lock_and_guards.py: end-to-end
  coverage with a real AIAgent + SessionDB proving the idle path honors
  the per-session compression lock (added after the PR), the persisted
  failure cooldown, and the anti-thrash breaker, and that the lock is
  released after an idle-triggered compaction.

Salvaged from #55800 by @iso2kx. Implements #27579.
2026-07-22 09:14:11 -07:00
Muthuvel
72056faf8f feat(compression): add opt-in idle-triggered context compaction
Long-lived sessions (e.g. a Telegram thread resumed over hours/days)
accumulate a large context that the existing size-based threshold only
trims once it crosses `threshold × context_window`. Until then every
turn re-reads the full history, which on large-context models can mean
hundreds of K of cache-read tokens per call even across long idle gaps.

Add a time-based trigger that complements (does not replace) the size
threshold: when a session resumes after `compression.idle_compact_after_seconds`
of inactivity, compact the accumulated history up front, before the first
reply. Disabled by default (0), so existing behaviour is unchanged.

The trigger reuses `_last_activity_ts` (the last time the turn loop did
work) to measure the idle gap at turn start, gates the token estimate
behind a cheap gap pre-check, and skips compaction when the context is
already at/below the post-compression target (threshold × target_ratio)
so a short idle thread never pays for a summarization that saves nothing.
It also defers to an active compression-failure cooldown.

The decision is factored into a pure predicate, `_should_idle_compact`,
which is unit-tested without a live agent.
2026-07-22 09:14:11 -07:00
Teknium
76e17bc32d fix(compression): recover merged-handoff prior-tail content through the decay scan
Composing #57835's multi-fossil summary scan with #47274's merged-handoff
unwrap: when the restart-decay path pulls a merged handoff into the
compression window, its genuine prior-tail user content must enter the
summarizer input (folded into the fresh summary) rather than being
dropped with the summary row. Standalone handoffs still drop. The
continuity test now pins the composed contract: recovered verbatim OR
via summarizer input, never silently deleted, never duplicated.
2026-07-22 09:10:49 -07:00
Teknium
a4aedde3ae test(compression): adapt salvaged pins to task-snapshot grounding + add restart-simulation test
- Three '_previous_summary == "fresh summary"' exact pins and one
  transcript-wide fossil-absence pin predated main's deterministic
  task-snapshot grounding (761a0b124e), which prepends a
  '## Historical Task Snapshot' section to stored summaries and may
  quote a folded head turn inside the handoff. Re-pin the contracts
  (fresh body present, fossil absent from non-summary messages)
  instead of exact strings.
- Add test_restart_simulation_fresh_compressor_does_not_reprotect_head:
  a fresh ContextCompressor over a transcript containing a persisted
  handoff summary computes the same decayed protected-head boundary
  (compress_start base) as a live already-compacted process, and the
  first post-restart compaction does not preserve pre-restart head
  fossils (#57814).
2026-07-22 09:10:49 -07:00
harjoth
08ea88f4f7 fix(agent): decay protected summaries after restart
protect_first_n decay state (compression_count / _previous_summary) is
in-memory only, so a gateway restart re-protected the persisted handoff
summary and head fossils, growing the head unboundedly across
restart+compaction cycles (#57814).

_effective_protect_first_n now probes a bounded resumed-head window for
a persisted handoff summary (by metadata or content prefix) and decays
protection when one is found, before compress_start is computed. The
first post-restart compaction self-heals: stacked summary fossils are
folded into the next summary prompt instead of preserved verbatim,
rehydration is rolled back on abort, deterministic fallback carries a
bounded redacted previous-summary snapshot, and forced user-leading
merged summaries keep the live tail request after the summary end
marker so they stay rehydratable.

Squashed reapply of the 12-commit series from PR #57835 onto current
main (branch was 1210 commits behind; single add/add conflict with the
task-snapshot grounding helpers resolved by keeping both).

Fixes #57814.
2026-07-22 09:10:49 -07:00
Teknium
cbc1054e23 fix: adapt compression attempt logging to current main aux-call contract
- aux summary call on main intentionally omits max_tokens; use .get() in the
  telemetry hook (and widen the param type) so the hook never breaks the call
- update test expectation: aux_output_reservation is None on main
- record no_progress failure_class in the no-progress boundary branch

Follow-up for salvaged PR #60444.
2026-07-22 08:13:41 -07:00
Gabriele Di Gesù
356ff99030 feat: log compression attempt telemetry 2026-07-22 08:13:41 -07:00
Teknium
5a3ee3c537 fix(compression): let handoff-strip supersede the head-copy skip
The summary_idx head-copy skip (from #69302) dropped the entire merged
handoff message, deleting the genuine prior-tail user content that
#47274's _strip_context_summary_handoff_message correctly unwraps.
Strip handles both shapes: standalone handoffs drop, merged handoffs
keep their real content. Caught by
test_recompression_of_current_merged_handoff_preserves_prior_tail_once
when both PRs landed together.
2026-07-22 08:12:45 -07:00
WXBR
2b84ed921c fix: dedupe persisted compaction handoffs 2026-07-22 08:12:45 -07:00
Teknium
020bd1ba0a test(compression): behavioral + config wiring coverage for threshold_tokens
Follow-up for salvaged #24279:
- cli-config.yaml.example: document compression.threshold_tokens
  (commented-out, default null = disabled)
- contributors/emails: map maly.dan@gmail.com -> DanielMaly
- tests: should_compress() fires at the absolute cap below the pct
  threshold (first-fires-wins); DEFAULT_CONFIG ships None and 0/None
  are behavior-neutral incl. across update_model(); the small-context
  pct floor is unaffected by the cap and re-derives correctly on
  model switch
2026-07-22 08:12:14 -07:00
DanielMaly
e5078e3152 feat(compression): add absolute token threshold via compression.threshold_tokens
Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR #24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in 973f27e95).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
2026-07-22 08:12:14 -07:00
Teknium
0acdf1d8c8
fix(compression): apply strict redaction at every compaction text boundary (#69294)
Compaction summaries persist across sessions and re-enter every subsequent
summarizer prompt, but every redact_sensitive_text() call in
context_compressor.py used default mode: a no-op under
security.redact_secrets:false, and opaque OAuth-callback / URL-userinfo
credentials passed through even when enabled. The stored _previous_summary
also re-entered the iterative-update prompt unredacted.

Add _redact_compaction_text() — redact_sensitive_text(force=True,
redact_url_credentials=True) — and thread it through all compaction text
boundaries: serializer input (content + tool args), deterministic fallback
summary, summarizer LLM output, manual + auto focus topics, the latest-user
task snapshot, and _previous_summary re-entry.

Note: force=True at this boundary intentionally overrides
security.redact_secrets:false — that opt-out targets live tool output, not
persisted summaries.

Salvages the compaction half of #49556 (the redact.py strict-URL half
landed independently via 75af6dc57/62a00a739). Addresses #43666 item 2.

Co-authored-by: AndrewMoryakov <topazd2@gmail.com>
2026-07-22 08:11:40 -07:00
Teknium
2ee50c69d3 docs(compression): note blank-echo removal survives summary abort 2026-07-22 08:10:47 -07:00
John Lussier
97cd0d98f1 test(compression): cover leading and input-text blanks 2026-07-22 08:10:47 -07:00
John Lussier
bc4824167d fix(compression): preserve latest actionable user turn 2026-07-22 08:10:47 -07:00
Teknium
f13f845116 feat(state): messages_fts_cjk — CJK-bigram index on the v23 external-content layout
Integration layer for the cjk_unicode61 tokenizer, rebuilt on the v23
schema (the contributed integration in PR #65544 predated it):

- messages_fts_cjk: external-content FTS5 over a tool-row-excluding view
  (same v23 storage discipline as the trigram index it supersedes — zero
  inline text copies). Serves EVERY CJK query shape the legacy routing
  split between trigram (>=3 chars/token) and LIKE full scans (1-2 char
  tokens). Lone 1-char CJK runs and role_filter=['tool'] queries keep
  their legacy routes.
- Dedicated marker pair (fts_cjk_rebuild_high_water/progress) gates the
  id-scoped triggers, so a cjk-only backfill never gates the complete
  messages_fts/trigram triggers.
- Transitions ride  (the existing
  throttled/resumable chunk engine): fresh DBs are born with the index;
  legacy v22 DBs land on v23+cjk in one run; already-optimized v23 DBs
  gaining the tokenizer get a marker-gated backfill; live writes are
  indexed immediately in every case.
- Tokenizer-loss self-heal: a process that can't load the extension drops
  the cjk triggers (writes keep working), leaves a stale breadcrumb, and
  the index is rebuilt from scratch on the next optimize run — triggers
  are never reinstalled over a gap (external-content 'delete' on an
  unindexed rowid is the FTS5 corruption hazard the marker gating exists
  to prevent).
- Capability classification: 'no such tokenizer: cjk_unicode61' joins the
  degraded-runtime error class everywhere (read probe, write probe,
  repair) so tokenizer absence is never misclassified as corruption.
- Config: sessions.cjk_fts (default on, inert without the .so) and
  sessions.search_slow_ms in config.yaml, bridged to env by CLI + gateway
  (startup + per-turn reload). build.sh falls back to vendored SQLite
  headers so no libsqlite3-dev is needed.

Slow-query log path attribution updated: fts_cjk / fts5 / trigram /
like_scan. Tests: 14 lifecycle tests (fresh/legacy/stale/backfill paths,
tokenizer-loss round-trip) + 5 config-bridge tests + slow-log suite.
2026-07-22 07:56:47 -07:00
Soju06
8364576e33 feat(state): slow-query log for session search with routing-path attribution
One INFO line per slow search naming the path taken (fts_cjk / fts5 /
trigram / like_scan), elapsed time, row count, and the query. The 2026-07
session_search investigation needed turn-trace archaeology plus workload
replay to discover that short-CJK queries were full-scanning the table —
with this line the next routing regression is a journalctl grep.

Threshold: sessions.search_slow_ms (default 1000ms; 0 logs every call),
bridged to HERMES_SEARCH_SLOW_MS.

Salvaged from PR #65544 (adapted to the v23 schema in follow-up commits).
2026-07-22 07:56:47 -07:00
Soju06
b10952e9c6 feat(state): cjk_unicode61 FTS5 tokenizer — unicode61 + CJK bigrams (native extension)
unicode61 indexes a CJK run as ONE token, so 2-char Korean terms (일본,
구글, 우리, ...) can never match it and the trigram tokenizer needs >=3
chars per term — any query containing a 1-2 char CJK token falls through
to a LIKE full-table scan (measured 3-6.4s CPU per query on a 6.8GB
production state.db; the #1 base cost behind a 12.4s session_search
average on CJK workloads).

This ships a ~250-line loadable FTS5 tokenizer (no deps) that wraps
unicode61: maximal CJK runs inside its tokens are re-emitted as
overlapping character bigrams (Lucene CJKAnalyzer semantics), everything
else passes through unchanged. FTS5 phrase semantics turn consecutive
sub-tokens into exact substring matching down to 2-char terms at index
speed.

Build: native/fts5_cjk/build.sh -> ~/.hermes/lib/libfts5_cjk.so
(override: HERMES_FTS5_CJK_SO).

Salvaged from PR #65544; the schema integration lands separately on the
v23 external-content layout.
2026-07-22 07:56:47 -07:00
Teknium
17155e3ae0 chore(contributors): add email mappings for slack thread-lifecycle salvage
LevSky22 (#66069), vexclawx31 (#33215), knoal (#64067), kaiyisg (#24848).
2026-07-22 07:22:55 -07:00
Ben
5d747a91c4 fix(slack): humanize inbound user mentions + ground bot identity
Slack delivers user mentions as opaque IDs (<@U123>). The agent had no
way to tell one participant from another — or from itself — so it could
misread a mention of a human as a self-mention and answer messages
addressed to that person (the "bot thinks it's @someone-else" bug).

Two cooperating fixes:
- _humanize_user_mentions rewrites remaining <@UID> tokens (the bot's
  own mention is stripped earlier) to @DisplayName in the trigger text
  and reply_to_text — the Slack equivalent of Discord's clean_content.
  Handles the labelled <@UID|handle> form; unresolvable IDs fall back
  to the raw ID.
- _build_identity_prompt injects an ephemeral per-turn system-prompt
  line via the channel_prompt seam (applied at API-call time, never
  persisted — prompt caching preserved) naming the bot's own workspace
  handle (per-team in multi-workspace installs) so the agent has a
  positive "that's me" anchor.

Salvaged from #55340 by @benbarclay, rebased over the workspace-scoped
user-name cache (team_id-aware resolution) on main.
2026-07-22 07:22:55 -07:00
LeonSGP43
503c0c0e51 fix(slack): expose shared-thread author mention target
In shared Slack threads the model saw only [sender name] prefixes, with
no verifiable current-author Slack user ID — so 'mention me again'
requests could bind to a stale or unrelated <@U...> pulled from names,
memory, or prior history (#17916).

Two cooperating changes:
- The shared-session sender prefix on Slack now carries the current
  author's ID from the event envelope:
  '[Alice | Slack user <@U123>] ...' — per-turn data, so it does not
  touch the cached system prompt.
- The Slack platform notes gain a shared-thread instruction to use the
  current turn's sender prefix as the only verified mention target and
  never guess or reuse historical mentions.

Fixes #17916.

Salvaged from #18711 by @LeonSGP43 (asdigitos), rebased over the
sender-name neutralization added on main (the ID is appended after
neutralizing the display name; the ID itself comes from the Slack
event, not user-editable text).
2026-07-22 07:22:55 -07:00
Kai Yi
73d5c896ee fix(slack): route replies from mentioned thread parents
Two mention-tracking gaps around thread parents (#24848):

1. When a thread PARENT @-mentioned the bot (e.g. '<@bot> check this
   and ask me before running'), a later bare reply like 'run' fell
   through every wake check if the mention event predated this process
   (restart) — _mentioned_threads is in-memory only. Add a 5th wake
   check that fetches the parent text (with the bot mention preserved
   via strip_bot_mention=False) and wakes when the parent addressed the
   bot, registering the thread so later replies skip the fetch.

2. A TOP-LEVEL @mention starts a thread (session keying falls back to
   the message ts), but only the raw event thread_ts was registered in
   _mentioned_threads — so replies to a top-level mention did not
   auto-trigger. Register the session-scoped thread_ts instead.

_fetch_thread_parent_text reuses the shared thread-context cache (raw
payloads) so the parent check costs at most one conversations.replies
call per thread; _register_mentioned_thread centralizes the bounded-set
eviction.

Salvaged from #24848 by @kaiyisg, rebased onto the extracted
_should_wake_on_unmentioned_message helper.
2026-07-22 07:22:55 -07:00
knoal
fee392fee1 fix(slack): wake on human replies in threads whose root we authored
The un-mentioned wake decision relied on three checks: thread root in
_bot_message_ts (populated only by the adapter's own send() path),
_mentioned_threads (populated on @mention), and an existing session.
Two gaps (#63530):

- Gap A: bot messages posted OUTSIDE gateway send() — skills/scripts
  calling chat.postMessage directly, cron/API posts — never enter
  _bot_message_ts, so human replies in those threads were silently
  dropped.
- Gap B: _bot_message_ts is process memory; after a gateway restart the
  bot stopped waking on replies to threads it started before the
  restart.

Fix: add a 4th, API-derived check — _bot_authored_thread_root — which
resolves the thread root's author via conversations.replies (cached in
_thread_context_cache via the new parent_user_id field, TTL-bounded).
Root authorship comes from Slack itself, so it covers outside-send
posts and survives restarts, unlike in-memory ts tracking. The wake
decision is extracted into _should_wake_on_unmentioned_message for
direct unit testing; the legacy checks remain first (cheap, additive).

Fixes #63530.

Salvaged from #64067 by @knoal (author metadata normalized to their
GitHub identity), rebased over the thread-context formatter split and
extended with per-team bot-id resolution for multi-workspace installs.
2026-07-22 07:22:55 -07:00
vexclawx31
fc0009b9ba fix(slack): rehydrate thread context after gateway restart
Persistent sessions survive gateway restarts, but thread replies posted
while the gateway was DOWN never reached the session — and the adapter
had no way to notice, so the conversation silently resumed with a hole
in it.

On the first ordinary reply per thread after a restart (tracked by a
fresh-process _thread_rehydration_checked set), fetch the thread delta
past the persisted per-session watermark and inject any missed messages
as part of the new turn via channel_context. Exactly-once per thread
per process; when the watermark is empty (pre-feature sessions) the
check is a no-op. Steady-state replies keep advancing the watermark so
rehydration never re-injects messages the session already carries as
ordinary turns. Prior history is never rewritten (prompt caching safe).

Builds on the persisted watermark introduced for #23918.

Salvaged from #33215 by @vexclawx31, reworked from a repeated
full-thread injection guard into a watermark-delta injection so
rehydration adds only what the session actually missed.
2026-07-22 07:22:55 -07:00
heathley
ad4034711d fix(slack): refresh active thread context on explicit mention
Once a thread has an active session, a later reply that explicitly
@mentions the bot did not re-fetch Slack thread context, so the agent
missed messages added to the thread after the initial hydrate (e.g.
other bots/integrations replying in multi-agent workflows). The
explicit mention is a fresh intent signal and now triggers a refresh.

Mechanics:
- SessionEntry gains a small persisted metadata dict, with
  SessionStore.get/set_session_metadata accessors (survives gateway
  restarts via the routing index).
- The adapter stores a per-thread consumption watermark
  (slack_thread_watermark:<channel>:<thread>) recording the last
  thread ts the session consumed.
- On explicit mention in an active thread, _fetch_thread_context runs
  with force_refresh=True (bypassing the TTL cache) and after_ts=<the
  watermark>, so only NOT-yet-seen messages are injected — as part of
  the new turn via channel_context. Prior conversation history is
  never rewritten, preserving prompt caching.
- _fetch_thread_context caches raw conversations.replies payloads so
  watermark-scoped re-formatting needs no extra API call; formatting
  is split into _format_thread_context.
- Thread session keys are built once in _build_thread_session_key
  (shared by the wake gate and the watermark accessors), still via
  build_session_key().

Fixes #23918. Supersedes #62299 (keyword-triggered refresh limited to
'investigate' prompts — the mention signal is the general fix).

Salvaged from #23927 by @heathley, rebased onto the plugin adapter
layout and rerouted through channel_context instead of text-prepend.
2026-07-22 07:22:55 -07:00
LevSky22
fd433e046a fix(slack): preserve thread context for commands via channel_context
Prepending cold-start thread backfill directly onto the message text
moved a recognized command (e.g. a bang-normalized "!queue ...") away
from character zero, so downstream command routing misclassified it as
conversational text and the command silently didn't run.

Route the backfill through MessageEvent.channel_context instead —
gateway.run already prepends channel_context after command dispatch
("[New message]" framing), so commands keep their COMMAND type while
the recovered history stays available to the agent.

Supersedes #68020, which dropped the fetched context entirely for
commands instead of preserving it out-of-band.

Salvaged from #66069 by @LevSky22.
2026-07-22 07:22:55 -07:00