Third correction, and the load-bearing one. The previous commit put the
"did compaction clear the threshold?" verdict inside should_compress(). But
conversation_loop calls should_compress() TWICE per turn with two different
measures (turn_context.py / conversation_loop.py:1033 and :4789):
* pre-API : request_pressure_tokens -- a rough estimate that can dip BELOW
the threshold
* post-API: real prompt tokens -- which stay above it
So the rough reading reset the strike every turn and the loop never stopped.
Reproduced: 8 compactions in 8 turns under the real two-call pattern, even
with the previous fix applied. (My earlier repro only called should_compress()
once per turn, which is why it looked contained.)
Move the verdict to update_from_response(), the one place that sees the
provider's real prompt_tokens for the just-compacted conversation, guarded by
the existing awaiting_real_usage_after_compression flag so it fires exactly
once per compaction. Real-vs-real: it cannot be fooled by a rough sub-threshold
reading, and (from the previous commit's lesson) never subtracts an estimate
from a real count. should_compress() goes back to a plain threshold test plus
the pre-existing cooldown and anti-thrash guards.
New test test_rough_preflight_reading_does_not_reopen_the_loop drives the real
two-call-per-turn pattern and fails on the prior should_compress()-based commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the previous commit, whose futility check was unsound:
incompressible_floor = max(0, display_tokens - pre_estimate)
`display_tokens` is the provider's real prompt count; `pre_estimate` is
`estimate_messages_tokens_rough(messages)`. Subtracting an estimate from a real
count folds the tokenizer skew into "floor" and misreads it as incompressible
overhead. With a 1.6x skew on a 200K window (threshold 150K, true floor 30K):
rough_msgs=253,804 real_prompt=436,086
computed floor = 182,282 <-- mostly skew; exceeds the threshold
after compaction: 401 -> 77 msgs, real prompt = 106,361 (CLEARS 150,000)
verdict: ineffective_count = 1 <-- false positive
Two such passes would permanently disable compaction on a healthy session --
worse than the loop this PR set out to fix.
Move the check into should_compress(), where both sides of the comparison are
the caller's own token count:
* prompt under the threshold -> not thrashing; reset the counter
* a compaction just ran and we are STILL over -> one strike
Real-vs-real, so tokenizer skew can never be mistaken for a floor, and nothing
subtracts an estimate from a real count. compress() now only ever increments the
counter; the reset lives with the one measure the trigger uses.
Adds `test_no_false_positive_under_tokenizer_skew` (the case above) and
`test_counter_resets_once_the_prompt_fits_again` (one failed pass must not
disable compaction forever). Against upstream, 5 of the 7 cases fail; the 2 that
pass are the regression guards, which is the intended shape.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`should_compress()` documents anti-thrashing protection ("if the last two
compressions each saved less than 10%, skip compression to avoid infinite
loops"). In practice `_ineffective_compression_count` reset on every pass,
so the guard was dead code and a mis-sized context window presented as a
hung CLI instead of a warning.
Two defects:
1. Mixed measurement bases. Effectiveness was
`(current_tokens - estimate(compressed)) / current_tokens`, where
`current_tokens` is the provider's FULL prompt (system prompt + tool
schemas + messages) but `estimate(compressed)` covers messages only.
Every compaction therefore reported ~96% savings and reset the counter.
Savings is now scored messages-vs-messages.
2. Message shrinkage is the wrong yardstick. `should_compress()` trips on
the full prompt, but compaction can only shrink messages -- the system
prompt and tool schemas are an incompressible floor. When that floor
alone meets the threshold, each pass shrinks messages by a healthy
margin, legitimately resets the counter, and still leaves the prompt
over the line; the next turn compacts again, forever. Observed in the
wild: 45+ consecutive compactions, one auxiliary-LLM call each, zero
progress. Effectiveness is now scored against the goal -- did the
projected prompt get under the threshold? -- and a futile pass warns
with the numbers that prove it.
Also record an ineffective pass on the "only N messages (need > M)" early
return, which previously returned the transcript unchanged without moving
any anti-thrash state -- the same class of bug the neighbouring
"no compressable window" branch was already fixed for.
Tests: 4 of the 5 new cases fail on main and pass here; the fifth pins
that effective compaction still resets the counter (121 -> 15 messages,
88.9% savings).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Preserve deletability, route identity, stored costs, aggregate reconciliation,
and zero-usage Codex route accounting on top of the salvaged per-model usage
work.
The `sessions` table records only the initial (model, billing_provider)
for a session, so when a user switches models mid-session (via `/model`
or programmatically) every token — including the switched model's — is
attributed to the first model. Insights/billing reports then hide the
cost of the new model entirely (e.g. a session that started on deepseek
and switched to opus shows $0 for opus).
Add a `session_model_usage` table keyed (session_id, model,
billing_provider) that accumulates each per-API-call delta under the
model active at the time of the call. `update_token_counts()` is the
single chokepoint every per-call delta flows through (CLI, gateway,
cron, delegated, codex), so recording there captures accurate
attribution on every platform. Only the incremental path records — the
gateway's `absolute=True` summary overwrite is skipped to avoid
double-counting cumulative totals that can't be split per model. When a
call omits the model, it falls back to the session's recorded model,
matching the existing COALESCE-from-session summary behaviour.
Insights `_compute_model_breakdown` now aggregates tokens and cost from
`session_model_usage`, so a switched session splits correctly across
models, with a defensive fallback to the per-session aggregate for any
session lacking usage rows. A v17 migration backfills one usage row per
existing token-bearing session from its aggregate totals (idempotent via
INSERT OR IGNORE), validated lossless against a 1.3 GB production DB.
Tests: per-model recording, mid-session split, model fallback, absolute
no-double-count, v17 backfill, and an insights-level switch breakdown.
Fixes#51607.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bundle Fireworks AI as a first-class BYOK provider across the CLI, web/TUI,
and desktop onboarding.
- New model-provider plugin with attribution headers (HTTP-Referer / X-Title)
so Fireworks can attribute Hermes traffic; PAYG-safe default aux + fallback
models (accounts/fireworks/models/...), IDs tracking fw-ai/fireconnect.
- Registered in CANONICAL_PROVIDERS so it appears in the CLI/web/TUI pickers.
- Alias wiring (fireworks-ai, fw) into both CLI resolvers.
- First-class wiring: OPTIONAL_ENV_VARS, HERMES_OVERLAYS (FIREWORKS_BASE_URL
override), doctor env hints. Live catalog + model_metadata are auto-derived.
- doctor: treat Fireworks' native slash-form IDs (accounts/fireworks/...) as
valid, not aggregator vendor prefixes, so it no longer tells Fireworks users
to switch to openrouter or drop the prefix.
- picker: plugin providers with no static curated list now lead with their
profile fallback_models, so the default is an agentic chat model instead of
whatever the live catalog returns first (Fireworks listed an image model,
flux-*, ahead of its chat models).
- Desktop onboarding: Fireworks as a RECOMMENDED hero card with the official
Fireworks logomark and a brand-purple badge, routing to the BYOK key form;
i18n in en/ja/zh/zh-hant.
- Tests: profile contract, first-class wiring (both resolvers, overlay, config,
doctor incl. the slash-form regression, aux headers, credentials), discovery
spot-check, and a live smoke test driven through the Hermes runtime.
Fire Pass (fpk_) support is coming soon; the future wiring is kept as a
commented-out scaffold in the plugin.
`hermes -t web chat` silently dropped the toolset filter (and the same
hold true for `-m`, `--provider`, `--tui`, `--dev` placed before
`chat`). Reported in #28780 for `-t/--toolsets`; the others are sibling
failures with the same root cause.
Root cause: the chat subparser re-declared these flags with `default=None`
(or `default=False` for store_true) on top of the matching top-level
parser flags. When argparse dispatches into the subparser it shares the
namespace via `dest`, so the subparser's default overwrites whatever the
top-level parser parsed before the subcommand. `-s/--skills`, `-r/-c/-w`,
`--yolo`, and `--pass-session-id` already use `default=argparse.SUPPRESS`
for exactly this reason — the chat-subparser action becomes a no-op
unless the user explicitly passes the flag after `chat`, and the parent
value survives.
Reproduction (origin/main, before fix):
>>> parser.parse_known_args(["-t", "web", "chat"]).toolsets
None
>>> parser.parse_known_args(["chat", "-t", "web"]).toolsets
'web'
After fix:
>>> parser.parse_known_args(["-t", "web", "chat"]).toolsets
'web'
>>> parser.parse_known_args(["chat", "-t", "web"]).toolsets
'web'
Sibling flags fixed in the same commit because they share the exact same
argparse pattern bug — verified via a new contract test that scans every
chat-subparser action whose `dest` is also on the top-level parser and
asserts `default is argparse.SUPPRESS`. The test fails on origin/main
listing all five offenders and passes after this fix.
Test additions in tests/hermes_cli/test_argparse_flag_propagation.py:
- TestChatSubparserInheritedValueFlags exercising real `_parser` build
(not the hand-rolled replica) so it catches future drift.
- Parametrized before-chat / after-chat cases for `-t`, `--toolsets`,
`-m`, `--model`, `--provider`.
- Negative case: passing none of the flags leaves attrs at the top-level
parser's `None` default (SUPPRESS does not remove existing attrs).
- Combined case: all three value flags before `chat` simultaneously.
- store_true cases for `--tui` / `--dev`.
- Contract test asserting every shared-`dest` flag on chat uses SUPPRESS.
Fixes#28780.
Review feedback: a same-host redirect to a different port can land on a
different service, which must not inherit the provider API key. Compare
(scheme, hostname, effective port) — with 80/443 defaults — instead of
hostname alone, and add a two-server regression test for the
same-host/different-port case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fetch_models() sends Authorization: Bearer <api_key> plus any
default_headers (x-api-key etc.) via urllib.request.urlopen, and
urllib's redirect handler forwards every header when following a
3xx — including to a different host. A catalog endpoint (or a
compromised/misconfigured proxy in front of it) answering with a
redirect to another origin therefore received the provider API key.
Install an HTTPRedirectHandler that drops authorization, x-api-key,
api-key, x-goog-api-key and cookie when the redirect target hostname
differs from the original request, mirroring the pattern already used
in skills/creative/comfyui/scripts/_common.py. Same-host redirects
keep credentials so legitimate path-level redirects still work.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_CodexCompletionsAdapter (agent/auxiliary_client.py) is a second,
independent producer of Codex Responses input — used by auxiliary
calls (context compression, flush_memories, MoA aggregation,
session_search) that route through CodexAuxiliaryClient instead of
the main agent's ResponsesApiTransport.build_kwargs. It calls
_chat_messages_to_responses_input() directly without is_github_responses,
so the previous commit's fix didn't cover it: an auxiliary call made
against a Copilot-backed session could still replay a connection-scoped
codex_message_items id and hit the same HTTP 401.
Detect the Copilot host from the adapter's own client.base_url (same
check the adapter already does further down for prompt_cache_key
opt-out) and pass is_github_responses through, closing the gap.
Still #32716.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Copilot (api.githubcopilot.com/responses) binds replayed assistant
codex_message_items ids to a specific backend "connection". Credential-
pool rotation, a gateway restart, or routine load-balancer churn between
turns all invalidate that binding, and Copilot rejects the stale id with
HTTP 401 "input item ID does not belong to this connection" — even for
short ids well under the #27038 64-char length cap, since this is a
connection-scope problem, not a length problem. Once a session captures
one of these ids it is persisted and replayed forever, permanently
bricking the session.
Thread an is_github_responses flag from build_kwargs/convert_messages
into _chat_messages_to_responses_input and drop the id unconditionally
on that path, mirroring how reasoning items already strip id on replay.
phase/status/content are still replayed so cache-relevant signal isn't
lost — only the connection-scoped id is unsafe to reuse.
Written to apply independently of the #27038 length-cap fix so the two
PRs don't block each other; they touch adjacent conditions in the same
block and merge cleanly in either order.
Fixes#32716
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Keep empty-tail recovery scoped to the current stream segment and bound fallback flood retries. Preserve Telegram's server retry hint without blocking final delivery through a long cooldown.
Codex assigns assistant message items server-side ids that can run
400+ chars (base64 encrypted blobs), but the Responses API caps
input[].id at 64 chars and rejects the whole request with a
non-retryable HTTP 400. Once a session captures one of these long
ids, every subsequent turn replays it and 400s forever, since the
history persists it in codex_message_items.
Add a 64-char length guard at both replay sites — the history-to-
input converter and the final preflight gate — so oversized ids are
dropped while short ids (msg_...) are kept for prefix-cache hits.
Mirrors the existing pattern for reasoning items, which already
strip their id before replay because store=False means the API
can't resolve ids server-side anyway.
Fixes#27038
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
finalize passed conversation_history=history aliasing the snapshot so flush
skipped every message and wrote nothing. now flush _session_messages via
marker dedup like gateway shutdown. add real db e2e tests.
force_close_tcp_sockets stayed shutdown-only after #29507 to avoid
cross-thread FD recycle. That left CLOSED sockets unreclaimed when
httpx.close() skipped already-shutdown sockets under long-lived
gateways (~1 CLOSED fd / 6 min via proxy).
Add release_fds= for the owning-thread dispose path only; abort still
defaults to shutdown-only.
get_due_jobs()'s one-shot stale-entry recovery (#38758) treated an
expired run_claim (#59229) as proof the claiming tick died, but a run
stalled on network I/O — or a laptop asleep mid-run — legitimately
outlives the TTL while very much alive. The recovery then deleted the
job record mid-flight: list showed the job gone, and when the run
finished mark_job_run() found nothing to update, so last_run_at /
last_status / last_delivery_error were never recorded.
Two guards, per the liveness signals available:
- Same process (the common single-gateway case): before removing a
dispatch-limit-reached one-shot, consult the scheduler's running set
via a lazy import; if the job is still running here it is slow, not
stale — keep the entry.
- Cross process: run_job's monitor loop now refreshes run_claim.at
every 60s while the run is alive (including under
HERMES_CRON_TIMEOUT=0, which previously blocked without polling), so
an expired claim really does mean the owner died and the TTL stays a
dead-owner detector.
Fixes#62002
