Follow-up to the salvaged contributor commit, closing the three gaps
flagged in the sweeper review:
1. Init ordering: assign compression.model_thresholds to a selected
plugin context engine BEFORE the initial update_model() call in
agent_init.py, so the initial model's override applies from init
(previously it only took effect after the first /model switch).
Base-class ContextEngine.update_model() now snapshots the
pre-override percent once so repeated switches fall back to the
engine's configured threshold, not a previous model's override.
2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to
hermes_cli/config.py — additive key, no _config_version bump.
3. Docs: document the key in
website/docs/developer-guide/context-compression-and-caching.md
(yaml example, parameter table, dedicated section) and update the
plugin-boundary note in context-engine-plugin.md to state the
explicit context-engine contract for model_thresholds.
Adds tests/run_agent/test_per_model_threshold_init_ordering.py:
plugin-engine AIAgent init regression (override applies at init,
empty map unchanged), DEFAULT_CONFIG key presence, floor interaction
on the model-switch path (override below the small-context floor is
raised to the floor; above the floor wins), and base-class config
snapshot across repeated switches. Also maps @bennybuoy in
contributors/emails/.
Addresses teknium1 review feedback on PR #60781:
1. Gateway cache invalidation: added ('compression', 'model_thresholds')
to _CACHE_BUSTING_CONFIG_KEYS so a live config edit to the map
invalidates the cached compressor (previously kept stale thresholds).
2. Integrated resolver with small-context floor: per-model overrides are
resolved FIRST, then the existing 75% floor for <512K models is applied
on top. The floor is no longer replaced — it stacks. An override below
75% on a small-context model still gets floored to 75% (raise-only);
an override above 75% wins.
3. Clean rebase on upstream main — no unrelated deletions or anti-thrashing
changes. Only the per-model threshold feature is added.
Changes:
- resolve_model_threshold() module-level helper (longest substring match)
- ContextCompressor.__init__ accepts model_thresholds dict
- _base_threshold_percent stores the per-model resolved value
- _config_threshold_percent stores the raw config value (fallback base)
- update_model() re-resolves on /model switch, falls back to config value
- ContextEngine base class update_model() applies overrides for plugin engines
- agent_init.py reads compression.model_thresholds from config, passes to ctor
- gateway/run.py cache busting key added
- cli-config.yaml.example documents the feature
- 17 tests covering resolve helper, compressor init (large/small context,
override above/below floor), update_model (re-resolve, fallback), base class
Co-authored-by: Copilot <copilot@github.com>
Review follow-up on #67431 (hermes-sweeper):
- The viability check compared the floored percentage against the raw
context window, but the built-in trigger recomputation also applies the
output-token reservation, the 64K floor, and the degenerate-window
guard (_compute_threshold_tokens). Mirror that math exactly, so e.g. a
200K window with max_tokens=120K recomputes to max(0.75*80K, 64K)=64K
and the suggestion is correctly KEPT for an 80K aux model instead of
being suppressed by the raw-window percentage.
- Gate the built-in policy behind isinstance(ContextCompressor): external
context engines own compaction policy (#44439), so plugin engines keep
the plain suggestion untouched.
- The non-viable explanation now names the recomputed trigger instead of
hardcoding the 75%/512K wording, so it stays accurate when the
reservation (not the percentage floor) is what makes the value
unreachable.
Tests: reservation-viability regression and plugin-engine passthrough,
per the review; the floored-branch assertion updated to the recomputed
number.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The auxiliary-compression feasibility warning computes its
compression.threshold suggestion as aux_context / main_context,
independently of ContextCompressor._effective_threshold_percent()'s
raise-only small-context floor. For main windows under 512K the floor
raises any configured value below 75% back up, so a suggestion like
'threshold: 0.40' is silently ignored and the same warning returns every
session.
Derive the suggestion's viability through the compressor's own floor
logic: offer the 'lower the threshold' option only when the floored value
still fits the auxiliary model's context; otherwise recommend only a
larger compression model and explain the floor, so the guidance is always
actionable.
Tests: the updated auto-correct test pins the floored branch (no
threshold suggestion, floor explained); two new tests pin the surviving
suggestion at/above the floor on a small window and below 75% on a
512K+ window where no floor applies. The updated test fails against the
previous code.
Fixes#67422
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refresh the agent activity tracker while context compression is blocked in the auxiliary summarizer so gateway watchdogs do not report inactivity during long compactions.
Add regression coverage for successful heartbeats, exception cleanup, touch failures, and strict-signature compressor fallback.
(cherry picked from commit c09e58b7709cc60c5b454701f0ecf840e759222f)
Follow-up to the salvaged #64010 (Kenmege) and #63870 (dombejar) commits,
making one resolved compression.max_attempts cap govern ALL per-turn
compression attempt sites:
- conversation_loop: resolve max_compression_attempts ONCE at turn start
(it was previously re-resolved inside the API-call loop) and route the
pre-API pressure gate through it — that gate still hardcoded
'compression_attempts < 3' and logged 'attempt=%s/3'.
- conversation_loop: the salvaged post-tool compaction gate now uses the
resolved cap instead of a hardcoded 3.
- turn_context: the preflight compaction loop was 'for _pass in range(3)';
it now sizes itself from the same resolved cap.
- agent_init: harden the max_attempts parser — reject booleans (bool
subclasses int; 'true' would coerce to 1), reject fractional floats
instead of truncating them, keep accepting integral floats and numeric
strings; anything else falls back to 3 (floor 1, ceiling 10 unchanged).
- tests: replace #63870's inspect.getsource source-shape test with
behavioral loop tests (post-tool compaction fires <= cap times per turn,
shares its budget with the pre-API gate, resets between turns); add an
e2e test proving a 4th preflight pass runs at config cap=6 while the
unset default still stops at 3; extend the #64010 config tests with the
bool/float parser semantics.
Salvages #64010 by @Kenmege and #63870 by @dombejar.
* feat(nous): send top-level session_id for provider sticky routing
The Nous Portal profile only embedded the session id inside portal tags,
so Claude traffic through the portal had no sticky-routing key. Multi-turn
sessions could reroute between upstream endpoints (Anthropic/Vertex/
Bedrock), cold-writing a fresh prompt cache on every reroute since each
provider's cache is instance-local.
Mirror the OpenRouter profile: emit extra_body.session_id whenever the
agent has one, pinning every turn of a session to the same endpoint so
explicit cache_control breakpoints stay warm.
* test: expect top-level session_id in Nous max-iterations summary body
Sibling site of the profile change — the max-iterations summary path
builds its request through the same NousProfile.build_extra_body(), so
its exact-shape assertion now includes the sticky-routing session_id
when the agent has a session.
Salvage follow-up for PR #68899:
- Restore .rstrip('/') on base_url in _swap_credential (both anthropic
and OpenAI paths) to match every other assignment site. The route
identity comparison still uses normalize_route_base_url which handles
trailing slash correctly.
- Extract should_clear_context_pin() into hermes_cli/route_identity.py,
consolidating 7 copy-pasted call sites across cli.py, gateway/run.py,
gateway/slash_commands.py, and hermes_cli/model_switch.py into a
single fail-closed helper.
C1 (anthropic path TLS re-application): pre-existing gap — the Anthropic
adapter (build_anthropic_client) has no TLS customization support at
all, so this is out of scope for this salvage.
* fix(desktop): preserve interim assistant text wiped at message.complete
When the agent emits interim text (commentary alongside tool calls, or the
attempted final answer before a verify-on-stop nudge), all UI surfaces
streamed it live but then wiped it at message.complete — keeping only the
final response. The user saw text appear during inference, then disappear.
This is the complete fix across all three layers: agent core, gateway
transport, and all UI surfaces (desktop + Ink TUI).
The verify-on-stop and pre_verify paths flagged the assistant's attempted
final answer as _verification_stop_synthetic, suppressing it from both
state.db and the UI. The user only saw the terse post-verification reply.
Now the assistant response is real content: it's persisted to state.db and
emitted as an interim message via _emit_interim_assistant_message(force_display=True)
before the verification loop runs. Only the synthetic nudge messages keep
the synthetic flags. The turn finalizer drops nudges from live history and
compares content (not just role) to avoid duplicating a published candidate.
Message sequence repair collapses verification candidates in the
consecutive-assistant merge.
Wire agent.interim_assistant_callback both at construction (_agent_cbs())
and per-turn (defense-in-depth), emitting a new message.interim event with
{text, already_streamed}. Gated on display.interim_assistant_messages
(default true). Cleared in the finally block so a stale closure can't
fire on a later turn.
Add message.interim to the GatewayEventName union (apps/shared) and a
typed payload to the TUI's GatewayEvent discriminated union.
The TUI already had the segment-anchoring machinery (flushStreamingSegment +
finalTail) but had no handler for message.interim. Added recordInterimMessage
+ interimBoundaryIndex to seal segments mid-turn, and updated
recordMessageComplete to only dedupe segments after the interim boundary.
Replaced the fragile sealed-set approach with a proper interimBoundaryPending
state flag on ClientSessionState. finalizeInterimAssistantMessage finalizes
the streaming bubble in place (or creates a standalone one), rotates the
stream ID so next deltas create a new bubble, and sets the flag. When the
final text equals an already-sealed interim, they stay as distinct messages.
Extracted mergeFinalAssistantText() as a pure function in chat-messages.ts,
used by both completeAssistantMessage and finalizeInterimAssistantMessage.
Split the bidirectional dedup predicate: reasoning is a restatement only when
the final FULLY covers it. A short final ("Done.") no longer swallows a
longer reasoning block that merely starts with it.
Honor display.interim_assistant_messages (default true) across all layers:
the tui_gateway gates the callback, the desktop wires it to a nanostores
atom via use-hermes-config. Updated hermes_cli/config.py and
cli-config.yaml.example comments to document the Desktop behavior.
_split_segment_tokens now accepts posix=False and _find_ad_hoc_match tries
both posix modes so ad-hoc verification scripts with Windows backslash
paths are matched correctly. (response_previewed forwarding from #53553
is not included — our emit-interim + persist approach makes it unnecessary
since the attempted answer is now surfaced before the verification loop.)
- tsc: clean (desktop + TUI + shared)
- vitest desktop: 73/73 pass (7 interim-sealing + 5 mergeFinalAssistantText + 4 config atom)
- vitest TUI: 83/83 pass (4 new message.interim tests)
- python: 390 tests pass (340 tui_gateway + 33 verification/finalizer + 6 config gating + 3 evidence + 8 continuation budget)
Co-authored-by: Liam Zhang <yingliang-zhang@users.noreply.github.com>
Co-authored-by: Lucas D'Alessandro <lucasfdale@users.noreply.github.com>
Co-authored-by: Eric Manganaro <superposition@users.noreply.github.com>
Co-authored-by: sweetcornna <sweetcornna@users.noreply.github.com>
Co-authored-by: DECK6 <DECK6@users.noreply.github.com>
Co-authored-by: matantsevs <matantsevs@users.noreply.github.com>
Co-authored-by: gitcommit90 <gitcommit90@users.noreply.github.com>
* fix: prefix-match interim streamed content to avoid benign duplicate bubbles
_interim_content_was_streamed used exact equality (streamed == visible_content),
so a final response that was the streamed text plus a trailing delta — or a
partial stream before the verify nudge fired — failed the match and left
_response_was_previewed false. The turn then showed two bubbles (interim +
identical final) instead of settling the interim in place.
Relax to a prefix check (visible_content.startswith(streamed)) in both the
core match and the desktop's settle-in-place gate. The TUI already used
prefix matching via finalTail. The reverse direction (streamed longer than
final) is intentionally not matched — that could suppress a needed resend
in the gateway path where already_streamed=True calls on_segment_break().
* test(desktop): add partial-stream-then-nudge dedup edge case
Third edge case for the interim-sealing dedup: model streams part of its
answer via message.delta, verify nudge fires, interim seals the streamed
prefix, then the final response is the same text plus a trailing delta.
Asserts one bubble (not two) containing the full final text.
Acceptance protocol #2 — covers all three dedup edges:
1. interim == final (existing)
2. interim = strict prefix of final (existing)
3. partial-stream-then-nudge (this commit)
---------
Co-authored-by: Liam Zhang <yingliang-zhang@users.noreply.github.com>
Co-authored-by: Lucas D'Alessandro <lucasfdale@users.noreply.github.com>
Co-authored-by: Eric Manganaro <superposition@users.noreply.github.com>
Co-authored-by: sweetcornna <sweetcornna@users.noreply.github.com>
Co-authored-by: DECK6 <DECK6@users.noreply.github.com>
Co-authored-by: matantsevs <matantsevs@users.noreply.github.com>
Co-authored-by: gitcommit90 <gitcommit90@users.noreply.github.com>
When a streaming response ends cleanly (HTTP 200) with no finish_reason
after delivering text but no tool calls, the chunk collector silently
stamps finish_reason='stop' and the conversation loop presents truncated
text as a complete response.
Three stream-drop paths now exist after chunk collection:
1. Zero-chunk → EmptyStreamError, retried (existing)
2. Tool-call in progress, no finish_reason → partial-stream-stub (existing)
3. Text-only, no finish_reason → partial-stream-stub (NEW — this fix)
Path 3 routes through the same PARTIAL_STREAM_STUB_ID + FINISH_REASON_LENGTH
machinery as path 2. The conversation loop shows 'Stream interrupted —
requesting continuation' and injects a continue prompt, giving the model
a chance to resume where the stream dropped.
Observed with DeepSeek provider where CloudFront drops SSE streams
mid-response after delivering partial text.
The salvaged retention check compared the built-in memory snapshot
before vs after the disk reload. That holds for a long-lived CLI agent,
but on fresh-agent surfaces (gateway per-turn agents, TUI) the cached
prompt is restored from the session DB and can predate mid-session
memory writes that the fresh MemoryStore already absorbed at init: the
snapshot is then identical on both sides of the reload while the prompt
itself is stale, so compression would retain (and re-persist via
update_system_prompt) a prompt missing the new memory for the life of
the session.
Replace the equality check with a containment check
(_cached_prompt_reflects_builtin_memory): retain the cached prompt only
when the freshly-reloaded rendered blocks appear verbatim inside it,
and rebuild when a leftover block header remains for a target whose
entries have since been emptied or disabled. Block headers are shared
via MEMORY_BLOCK_HEADERS in tools/memory_tool.py so the check stays in
lockstep with MemoryStore._render_block.
Adds regression guards for the gateway stale-restore path and the
emptied-memory leftover-block path; verified with a real-MemoryStore
E2E matrix (9 scenarios) against a temp HERMES_HOME.
_is_real_user_message no longer accepts a user-role compaction summary as
the human anchor, so _compress_context restores the original user turn
after the summary; update the lifecycle-status test's expectation.
Direct-Anthropic requests used a single shared _anthropic_client, and the
stale/interrupt watchdog closed + rebuilt it from the poll (stranger) thread
at four sites (non-streaming stale/interrupt, streaming stale/interrupt).
Closing a client whose TLS socket a worker thread was still reading released
the FD from a stranger thread; the kernel recycled it under a live SSL BIO,
which then wrote a 24-byte TLS record into an unrelated SQLite header
(cron/executions.db), bricking every cron on the profile. Same shape as the
OpenAI-only #29507 fix, but the Anthropic path never got the owner-thread
contract.
Extend the #29507 ownership contract to Anthropic: build a per-request client
(_create_request_anthropic_client), register it with the request-client
holder tagged by kind, and route _close_request_client_once by kind — a
stranger thread only shuts the request client's sockets down
(_abort_request_anthropic_client), while the owning worker performs the SDK
close (_close_request_anthropic_client). The shared _anthropic_client is now
never closed from inside a request (streaming or non-streaming), including the
worker retry-cleanup sites, since each attempt builds a fresh request client.
The #28161 no-hang guarantee is preserved: the poll-thread socket abort
unblocks the worker immediately.
Salvages the approach from #51688 (@raymondyan-zhijie), reimplemented onto
current main (non-streaming dispatch was refactored into
_dispatch_nonstreaming_api_request; streaming grew _cancel_current_stream_attempt
and worker retry-cleanup sites). Tests updated to the request-local mechanism
(incl. replacing a banned source-reading test with a behavior test) plus new
regression coverage proving the watchdog aborts the request client and never
touches the shared client.
Co-authored-by: raymondyan-zhijie <32435458+raymondyan-zhijie@users.noreply.github.com>
Second call site (non-streaming / gateway path) now flattens list-type
content with flatten_message_text before the inline <think> regex and the
surrogate sanitizer, matching the interim-text fix from the prior commit.
Adds regression tests (tests/run_agent/test_66267_multimodal_interim.py)
covering:
- build_assistant_message with list content does not raise TypeError
- inline <think> inside list content is extracted + stripped correctly
- _interim_assistant_visible_text is safe for tool messages (list content)
- duplicate_previous_interim dedup guards against tool messages
Verified the tests fail without the fix (TypeError: expected string... got
'list') and pass with it.
_bedrock_reasoning_stale_floor only matched dashed floor-table keys (opus
'claude-opus-4-6'), so sonnet reasoning models keyed with a dotted version in the
shared table ('claude-sonnet-4.5'/'4.6') got no reasoning floor from the dashed
Bedrock inference-profile id — premature stale-abort if the base timeout is set
below 180s. Generate both version-separator forms (digit-dash-digit <-> digit-dot-
digit, via lookbehind/lookahead so only version numbers flip) and try all
candidates; matches the table however each model is keyed, no edit to the shared
table. Verified: opus->240 (unchanged), sonnet-4.5/4.6->180, haiku->None. New
TestBedrockReasoningStaleFloor (8 cases).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-applied against v0.18.2. Upstream now covers the OpenAI-path stall watchdog
(cross-turn give-up breaker #58962) and the tool-batch deadline, so those are
dropped. Two gaps remain:
- Bedrock streaming had NO liveness watchdog and was excluded from the #58962
breaker (it returns before _check_stale_giveup). Add an on_event hook to
stream_converse_with_callbacks (fires per yielded event = true wire-level
liveness), drive a stale timer from it, and wire Bedrock INTO the existing
breaker: entry _check_stale_giveup, _bump_stale_streak on stall, raise to end
the call (invalidate_runtime_client can't abort the in-flight botocore stream,
so the streak escalates across turns like the OpenAI path), and reset the
streak on success.
- local providers still got float('inf') stale timeout (watchdog disabled) — give
a finite ceiling (HERMES_LOCAL_STREAM_STALE_TIMEOUT, default 900s).
- tests: on_event per-event + swallow; Bedrock stall bumps streak + aborts;
pre-elevated streak aborts at entry; success resets streak.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
strip_think_blocks() ran re.sub() directly on content that could be a
list of blocks (Anthropic via OpenRouter returns assistant content as
[{type:text,...},{type:thinking,...}]). A list reaching re.sub raised
'TypeError: expected string or bytes-like object, got list', which the
outer conversation loop swallowed and retried forever — the observed
infinite 'preparing terminal...' loop that re-emitted the same
assistant text every iteration.
The live-turn path normalized list content to a string, but
_interim_assistant_visible_text reads a *stored* history message whose
content was persisted as a list and passes it straight into the shared
strip_think_blocks helper. Fix at the shared choke point: coerce
list/dict content to visible text (dropping reasoning blocks, which is
the function's job) before any regex runs, so every caller is safe.
Fold #62349's broader provider-boundary handling into the header fix: bound top-level and xAI override keys again at preflight after middleware, preserve unrelated headers, and cover boundaries and collisions.
Co-authored-by: Nick Taylor <nicktaylor@TheWorldofNick-Lappy.local>
Widen the #65991 single-writer fence to run_codex_stream: each codex
attempt claims the delta sink before consuming events, and the consume
loop's interrupt_check now also stops the instant a newer attempt
supersedes this one. Parity with the chat_completions / anthropic /
bedrock paths from the salvaged fix.
Two regression tests: superseded codex stream is fenced mid-stream;
sole-writer codex stream delivers unchanged.
Assert that a superseded stream (older writer token, other thread) is fenced
from the delta sink, the active writer is never fenced, a non-claiming thread
is never treated as a writer, and the real consume loop stops the instant it is
superseded — so two streams can never interleave into one turn (#65991).
Port from anomalyco/opencode#36130: the Responses spec carries streaming
error details at the top level of the error frame, but the official OpenAI
SDK and several OpenAI-compatible proxies wrap them in an HTTP-style nested
envelope ({"type": "error", "error": {code, message, param}}).
_raise_stream_error only read top-level fields, so nested-envelope frames
collapsed to the generic 'stream emitted error event' placeholder with
code=None — the error classifier never saw the provider's real failure
reason, misrouting rate-limit / context-overflow / entitlement errors into
the generic retry path.
Top-level fields keep precedence; the envelope is a fallback. Null-tolerant
for spec-compliant frames with explicit nulls.
Follow-ups on top of @xxxigm's salvaged bridge (#33294):
- Remove the now-dead narrow item/started-only mapper from #38835
(_codex_note_to_tool_progress) — the full bridge supersedes it and
keeps the same tool-name contract; its tests are repointed at the
bridge helpers.
- Preserve main's request_routing/approval-bypass wiring on the
CodexAppServerSession constructor (landed after the PR was filed).
- Gate agentMessage interim delivery on display.show_commentary so the
app-server runtime honors the same toggle as the codex_responses
commentary channel (tool progress is unaffected).
- Add json import (bridge helpers use json.dumps) and modernize the
wiring test's stub agent for main's usage-accounting attributes.
Commentary delivery is on by default; users who find the extra mid-turn
narration noisy can set display.show_commentary: false to restore the
previous behavior (commentary routed to the reasoning channel, visible
only with show_reasoning).
- hermes_cli/config.py: display.show_commentary default true
- agent/agent_init.py: wire config -> agent.show_commentary
- run_agent.py: gate structured commentary extraction on the flag
- agent/codex_runtime.py: gate live-stream commentary callback (falls
back to legacy reasoning-channel routing when off)
- docs + 2 tests (interim path off, live stream fallback)
Also adds AUTHOR_MAP entries for davidrobertson and 100yenadmin.
Keep each xAI OAuth auth-add login as an independent manual device-code pool entry and recognize xAI personal-team spending-limit 403 responses as billing exhaustion. Preserve the structured top-level error message so the failed credential is quarantined and the next healthy account is selected without attempting a pointless token refresh.
Route direct xAI HTTP consumers through the credential pool as well. Proactive and 401-reactive refreshes update the exact issuing manual entry, preserve validated xAI base URL overrides, and serialize single-use refresh-token rotation across concurrent pool instances.
