The first LLM call of every gateway turn gets ~0% provider prompt-cache
hit rate (in-turn calls: 97-99%) because the bytes sent for a turn's
user message are not the bytes replayed next turn: memory-prefetch and
pre_llm_call context are injected into the API copy only, the #48677
persist override writes cleaned content to the DB row, and
get_messages_as_conversation sanitize/strips user and assistant content
on load. Any of these diverges the request prefix at that message and
re-prefills everything after it — measured 27.9s for the first call vs
2.4-5.8s cached at a median ~156k-token context.
Persist what you send: a nullable messages.api_content column stores the
exact content string sent to the API when it differs from the clean
stored content, and replay substitutes it verbatim (no sanitize, no
strip). The injection composition lives in one helper
(turn_context.compose_user_api_content); the turn prologue stamps its
output onto the live user message, the api_messages build sends the
stamped bytes, and every outgoing copy pops the field so it never
reaches a provider. The crash-resilience user-turn persist moves after
prefetch/pre_llm_call so the user row is written once with its final
sidecar; _ensure_db_session stays before preflight compression (session
rotation needs the parent row under PRAGMA foreign_keys=ON). The
current-turn index trackers are re-anchored after compaction rebuilds
the message list, in-place preflight compaction backfills the stamp onto
the already-inserted row, and gateway replay forwards the sidecar only
when the replay pipeline did not rewrite the content. Rewrite paths that
would leave stale bytes (historical image strip, merge-summary-into-
tail, consecutive-user repair merge, stale-confirmation redaction) drop
the sidecar; the chat-completions transport and the max-iterations
summary path strip it defensively. codex_app_server and MoA turns are
excluded from stamping because their wire bytes differ from the
composition. A missing or dropped sidecar degrades to today's behavior
(one cache-boundary miss), never to wrong content.
1. Remove PR's redundant strip_think_blocks coercion (Teknium's fix
296494db0 already handles this at the same chokepoint with superior
logic that drops thinking/reasoning blocks).
2. Restore 'if not content: return ' guard at top of strip_think_blocks
that was lost during cherry-pick auto-merge. Without it, None content
hits str(None) → 'None' string instead of returning empty.
3. Fix error classifier design flaw: remove 'conversation_loop' and
'run_agent' from _local_processing_modules — these are the container
modules for the try/except, so every exception passes through them,
making _hit_local always True and misclassifying transient API/network
errors as non-retryable local bugs.
4. Move module sets to module-level frozenset constants (_LOCAL_PROCESSING_MODULES,
_API_CALL_MODULES) instead of rebuilding on every exception.
5. Replace traceback.extract_tb() with raw tb walk — avoids disk I/O for
source lines that are never used.
6. Remove unused 'import traceback'.
7. Fix docstring corruption: 3 lines where think tags were replaced
with Chinese characters during the PR's editing.
Degrading models (observed with gpt-5.6 past ~350K input) emit tool-call
batches like 6 valid named calls + 1 blank-name call. Previously the
whole turn was voided — every valid call got 'Skipped: another tool call
in this turn used an invalid name' — and three such batches tripped the
3-strike stop, killing sessions that were still making progress.
Now a mixed batch error-results ONLY the invalid call(s) (terse
anti-priming error for blank names per #47967, catalog dump for typos)
and dispatches the valid subset for execution. The assistant message
keeps every emitted call so provider-side tool_call/result pairing stays
intact. The 3-strike counter only advances when a turn contains NO valid
call, so a fully-degenerate model still stops while a mostly-coherent
one keeps working. Broken JSON args on a never-executing invalid call no
longer trigger the whole-turn JSON retry loop.
Field evidence: July 2026 debug bundle showed gpt-5.6-sol emitting
6-call batches with one blank-name rider at 559K/384K-token context in
two separate sessions; 13 valid tool calls were discarded before the
session stopped as partial.
Three fixes for the Bedrock Claude path:
1. Streaming fallback: When AnthropicBedrock SDK raises 'Unexpected event
order' (SDK misparses Bedrock error events as message_start), auto-switch
to native Converse API for the rest of the session instead of failing
after 3 retries.
2. Image base64 decode (#33317): data URL payloads were passed as base64
strings to source.bytes, but boto3 re-encodes at the wire layer. Now
decoded to raw bytes before passing to Converse API.
3. Bearer token routing (#28156): Users with AWS_BEARER_TOKEN_BEDROCK are
now routed through Converse API regardless of model, since the
AnthropicBedrock SDK only supports SigV4 signing.
3 new tests. 121 bedrock_adapter tests passing.
Two consecutive incomplete assistant interims with identical visible
content (content + reasoning) are collapsed even when opaque provider
state (encrypted reasoning item ids, message item phases) drifts per
continuation — previously that drift defeated dedup and caused message
storms (#52711). The latest opaque payload is written onto the existing
message in place, so continuation replay still uses fresh provider
state.
Salvage note: the original PR also made the 3-retry continuation cap
cumulative per turn (no reset on progress). That half is intentionally
dropped — a legitimate long turn alternating incomplete/progress would
hard-fail at 3 cumulative, changing semantics beyond the reported bug.
Map Codex Responses status=incomplete with incomplete_details.reason=content_filter to finish_reason=content_filter so the existing refusal/fallback path runs instead of burning incomplete continuation attempts.
Community reports of GPT 'infinitely thinking' are usually a slow or
overloaded provider plus silent retry machinery: the CLI/TUI/Desktop
spinner shows a generic 'cogitating...' verb for the whole wait and the
gateway heartbeat says only 'Working — N min'.
Add AIAgent._emit_wait_notice(): rewrites the live spinner/status line
(thinking_callback → CLI prompt_toolkit widget, thinking.delta → TUI +
Desktop) and updates the activity tracker (included in the gateway's
'⏳ Working — N min' heartbeat). Wired at the four wait points:
- non-streaming wait loop: after 30s with no response, the line becomes
'⏳ waiting on <model> — Ns with no response yet (provider may be slow
or overloaded; auto-reconnect at Ns)'
- streaming wait loop: same explanation after 30s with no chunks,
including the long-thinking case
- TTFB / stale-stream kills: '⚠ no response from provider in Ns —
reconnecting...'
- Codex continuation retries: '↻ model returned reasoning with no final
answer — asking it to continue (n/3)' instead of silence
Notices are fail-open (display errors never break the wait loop) and
gateway sessions without a display callback still get the improved
activity description.
Follow-up to the salvaged #63690: when the interim assistant message is
too empty to append (no content and no reasoning of any kind), the last
message in history is still the prior user/tool turn — appending the
user-role nudge there would create a user→user or tool→user sequence
that strict providers reject. Only append the nudge when the last
message is an assistant turn. Also add IpastorSan to AUTHOR_MAP.
grok-4.x on the xAI /v1/responses surface sometimes ends a turn with only
reasoning items — no message output item, no tool calls — and those
reasoning items carry no encrypted_content. Two compounding problems:
1. The model occasionally emits its final answer INSIDE the reasoning
channel, delimited by grok's internal "<response>" tag. The answer
exists but is classified reasoning-only → finish_reason=incomplete.
2. An interim assistant message holding only plain-text reasoning replays
as nothing in _chat_messages_to_responses_input, so every continuation
request is byte-identical to the one that just failed. The model
deterministically repeats the reasoning-only response until the retry
budget is exhausted and the turn dies with "Codex response remained
incomplete after 3 continuation attempts".
Fixes:
- _normalize_codex_response (xai_responses only): salvage the
<response>-delimited tail from the reasoning text and promote it to
assistant content; the untagged prefix stays as thinking text.
- Codex-incomplete continuation path: when the interim message has
nothing the input converter will replay (no content, no encrypted
reasoning items, no message items), append a user-role nudge so the
retry actually differs and explicitly asks for the final answer /
pending tool call. Mirrors the existing _get_continuation_prompt
pattern used for length truncation.
Observed live with grok-4.20 on xai-oauth (2026-07-13); sibling of the
grok-composer web_search incomplete-loop fix in transports/codex.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The upstream migration to inference.nousresearch.com broke routing for
inference-api.nousresearch.com. Restore the correct base_url and drop
the stale alias match in _is_nous_inference_route().
Fixes#60715
Add a bounded turn-end stop guard for kanban workers. When a worker
tries to exit with finish_reason=stop without having called
kanban_complete or kanban_block, inject up to two synthetic nudges
so the conversation loop continues instead of exiting cleanly (which
the dispatcher records as protocol_violation).
Mirrors the existing verify-on-stop pattern: same ephemeral scaffolding
flag (_kanban_stop_synthetic), same role-alternation contract, same
_pending_verification_response fallback for budget exhaustion.
Disabled by default (gated on HERMES_KANBAN_TASK env var set by the
dispatcher); kill switch via HERMES_KANBAN_STOP_NUDGE=0.
Salvaged from #62262 by @mdc2122. The original branch was 272 commits
behind main with ~538 files of stale-base reversions; this salvage
applies only the 4 substantive files (agent/kanban_stop.py,
conversation_loop.py insertion, run_agent.py _EPHEMERAL_SCAFFOLDING_FLAGS,
tests/agent/test_kanban_stop.py).
Salvage of #63888. The original fix clears stale _last_content_with_tools
on substantive tool-only turns but doesn't clear _mute_post_response, which
a prior housekeeping turn may have set. This suppresses tool progress
output via _vprint until the no-tool-call branch resets it at line ~4834
— after all tools have finished executing.
Fix: also reset _mute_post_response = False when clearing stale fallback.
Added test: verify pure housekeeping turns (content + only housekeeping
tools) still set the fallback correctly — the original use case the
fallback was designed for.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
A cached _last_content_with_tools response from a housekeeping-only turn
could survive a later substantive tool-only turn. When the model returned
an empty response, Hermes incorrectly finalized the older housekeeping
narration instead of invoking the post-tool empty-response nudge.
Production impact: scheduled cron jobs could return early without completing
their actual work (e.g., daily report job returning a housekeeping message
instead of producing the report artifact).
Root cause: The fallback state was only updated when a turn had both
content AND tool_calls. A turn with tool_calls but empty visible content
would skip state updates entirely, leaving stale fallback state intact.
Fix: Classify tools in every tool-call turn (regardless of visible content).
When any tool is substantive (non-housekeeping), clear the older fallback state
before processing later empty responses. This prevents two-turn-old housekeeping
narration from being treated as if it belonged to the immediately preceding
substantive tool turn.
Regression test added: tests/run_agent/test_conversation_fallback_state.py
Fixes#63860
Cache-decorated turns (apply_anthropic_cache_control converts string
content to [{type: text, ..., cache_control}] lists — applied BEFORE the
MoA facade since the #57675 cache-cold fix) and multimodal turns
(text + image_url parts) flattened to empty strings in
_reference_messages, which only read str content. On turn 1 of a
provider:moa session with a Claude aggregator the references received a
single EMPTY user message: Anthropic-side providers 400'd ('messages: at
least one message is required') while tolerant models answered 'no user
request is present' (live incident Jul 14 2026, preset 'closed').
Fixes, in totality:
- _reference_messages: extract visible text via
agent/message_content.flatten_message_text for user/assistant/tool
turns (skips image parts, so no base64 leaks into the advisory view);
decorated and undecorated transcripts now produce a byte-identical
advisory view (advisor cache prefix stays stable).
- image-only user turns get a placeholder instead of an empty message
(Anthropic rejects empty text blocks) or a silently dropped turn
(would break user/assistant alternation).
- degenerate-case fallback flattens structured content too.
- _attach_reference_guidance: a decorated/multimodal trailing user turn
now receives the guidance as a NEW text part appended AFTER the
cache_control-marked part (cached prefix byte-stable) instead of
falling through to a second consecutive user message (strict providers
reject user/user).
- conversation_loop MoA injection: multimodal user turns get the MoA
context appended as a trailing text part instead of being dropped;
user_prompt for the one-shot path flattens content lists instead of
str()-ing them (which leaked base64 payloads into the prompt).
Live-verified on the 'closed' preset (real OpenRouter wire, 2 user
turns, tool loop): all 4 reference calls carry the full document +
rendered tool state, end on user, zero tool-role/tool_calls; advisor
cache_write 7968 then cache_read 5909+; aggregator cache_read
14880-15237 on iterations 2+.
Co-authored-by: bo.fu <bo.fu@meituan.com>
Salvage of #63862. is_output_cap_error() returns False for vLLM/LM Studio
error messages that contain 'prompt contains ... input tokens' (treated as
input-overflow signal). But parse_available_output_tokens_from_error() CAN
extract a valid available_tokens from those same messages. The
compression-disabled guard only checked is_output_cap_error(), so vLLM/LM
Studio users with compression off still got a terminal failure instead of
the max-tokens retry.
Fix: also exempt when parse_available_output_tokens_from_error() returns a
value — that function determines whether the retry path can actually handle
the error, so it's the right predicate for the exemption.
Added test: verify vLLM-format error with compression_disabled=False still
triggers the max-tokens retry path.
Co-authored-by: dmabry <dmabry@users.noreply.github.com>
The branch computed safe_out from estimate_messages_tokens_rough(messages),
but the provider rejected the larger api_messages request (system prompt,
injected context, tool schemas). When API-only content is large, safe_out
could far exceed the provider's available_tokens.
Compute safe_out from estimate_request_tokens_rough(api_messages, tools=...)
and keep provider available_out as an upper bound. Do not alter context_length
or trigger compression for output-cap errors.
Add production-path run_conversation tests that assert the retry API call's
max_tokens, including a case where a large system prompt makes messages-only
estimation undercount the real request.
Fixes#55546
The retry loop computed safe_out from the error's available_tokens,
which reflected the *previous* request. Between retries the agent
appends tool results and error text, so the real input token count
grows. Deriving safe_out from the stale budget meant every retry
still exceeded the context ceiling by 1+ tokens, burning through the
3-attempt limit.
Compute safe_out from estimate_messages_tokens_rough(messages) so
the cap tracks the growing input on each retry attempt.
Cron jobs in the gateway process wedged before HTTP on later non-streaming
API calls because interruptible_api_call spawned a daemon worker inside
nested cron thread pools. Route cron platform turns through direct_api_call
on the conversation thread instead.
Treat intent-ack continuation text as non-final so last-turn exhaustion requests a real summary instead of surfacing a premature promise. Keep iteration-limit fallback text free of the abnormal-fragment explainer.
Name the continuation fallback for its actual verification-only provenance so unrelated continuation paths cannot accidentally inherit its cron-delivery semantics.
Track held-back verification responses explicitly so budget exhaustion returns the composed report without a second model call. Keep unrelated error and recovery exit reasons intact, preserve Kanban timeout accounting, and cover the real run_conversation paths.
Clear stale final_response before verify-on-stop/pre_verify loop
continues, and normalize iteration-limit exit reasons in turn_finalizer
when a composed answer survives with unknown/budget_exhausted.
Fixes#61631
Follow-up on the salvage of #59523. Two low-risk cleanups surfaced by review:
- Extract _ZAI_CODING_OVERLOAD_SHORT_ATTEMPTS as a module constant so
adaptive_rate_limit_backoff() and zai_coding_overload_retry_ceiling()
share one source of truth. Previously both hardcoded short_attempts=3
independently; tuning one without the other would silently desync the
retry ceiling from the backoff schedule.
- Replace the tautological formula-mirroring assert in
test_zai_overload_retry_ceiling_exceeds_short_attempts with a behavior
invariant (ceiling leaves headroom for every long-backoff entry), per the
repo's contracts-over-snapshots testing rule.
Z.AI Coding Plan GLM-5.2 reports server overload as HTTP 429 code 1305
("temporarily overloaded"). classify_api_error routes that to
FailoverReason.overloaded (so a valid credential pool isn't burned), but
the adaptive Z.AI backoff was gated on is_rate_limited — which excludes
overloaded — so it never ran (policy=default) and the request failed after
a few quick short retries.
Two compounding causes, both fixed here:
1. Detect the Z.AI overload 429 directly and let its adaptive backoff run
on the overloaded path, not only the rate_limit path.
2. Raise the retry ceiling for this narrow case via
zai_coding_overload_retry_ceiling(). The long-backoff tier
(30/60/90/120s) starts after short_attempts (3) retries, but the default
api_max_retries is also 3, so the loop always gave up before the long
tier could run — leaving the whole long-backoff schedule as dead code.
Scope is limited to the existing narrow is_zai_coding_overload_error match,
so other providers' 429/503/529 handling is unchanged.
The google-antigravity and google-gemini-cli OAuth providers were removed
in #50492. They were the only producers of a cloudcode-pa:// base_url, so
the account-level-quota early-returns in _pool_may_recover_from_rate_limit
and _credential_pool_may_recover_rate_limit are now unreachable.
- Drop the dead cloudcode-pa:// checks and the now-unused provider/base_url
params on _pool_may_recover_from_rate_limit (only caller updated).
- Prune the obsolete CloudCode-specific regression tests; keep the live
single/multi-entry pool-rotation invariants (#11314).
ruff check --fix --select F541 . on current main. Pure prefix removals;
adjacent-string concatenations keep the f only on interpolating fragments.
No string content or live placeholder altered.
Inspired by Claude Code v2.1.199 (July 2, 2026): SSL certificate errors
(TLS-inspecting proxies, missing CA bundles, expired certs) no longer
burn retries before showing actionable guidance — they fail immediately
with the fix hint.
- agent/error_classifier.py: new FailoverReason.ssl_cert_verification +
_SSL_CERT_VERIFY_PATTERNS, checked BEFORE the transient-SSL patterns
(cert-verify messages also contain '[SSL:' and previously retried
forever as timeout). Non-retryable, no compression, no fallback churn.
- agent/conversation_loop.py: dedicated status line + per-cause fix
hints (corporate proxy CA bundle, certifi refresh, self-signed local
endpoints) on the non-retryable abort path.
- 7 new tests incl. regression guards (transient alerts still retry,
large-session cert failure doesn't trigger compression).
* fix(cli): set correct x-initiator header per Copilot turn
copilot_default_headers() always hardcoded x-initiator: agent, but
GitHub Copilot billing requires "user" for user-initiated prompts and
"agent" for tool/follow-up calls. This caused premium requests to never
be consumed correctly, risking billing issues or account bans.
Adds is_agent_turn param to copilot_default_headers() and injects
extra_headers={"x-initiator": "user"} on the first API call of each
user turn when targeting Copilot URLs. The flag flips to False after
injection so subsequent calls (tool use, streaming fallback) default
back to "agent".
Fixes#3040
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore(release): add AUTHOR_MAP entry for @tjp2021 (PR #4097 salvage)
---------
Co-authored-by: Tim <tim@iteachyouai.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Self-review (hermes-pr-review Phase 2) flagged the mid-turn pre-API
compaction for reuse/duplication (W1/W2); fixing that surfaced a
regression against a deliberate existing feature, now also fixed.
The block now mirrors the turn-prologue preflight's guard chain exactly
(agent/turn_context.py) instead of a hand-rolled pressure limit:
1. should_defer_preflight_to_real_usage(rough) — defer when the rough
estimate is known-noisy vs a recent real provider prompt that fit
under threshold (schema overhead / post-compaction over-count, #36718).
2. get_active_compression_failure_cooldown() — skip during a same-session
compression-failure cooldown.
3. should_compress(rough) — reuses the canonical threshold_tokens (output
room already reserved by _compute_threshold_tokens) plus its summary-LLM
cooldown + anti-thrash guards (#11529).
Dropped the seven inline _reserve/_output_pressure locals (W2: they
re-derived _compute_threshold_tokens and omitted its 85% degenerate-window
fallback). compression_attempts stays as the hard per-turn backstop.
Without guard (1) the block fired a compaction the preflight deliberately
defers, breaking test_413_compression::test_preflight_defers_when_recent_
real_usage_fit (ValueError from the mocked _compress_context). Verified:
test_413_compression 26/26, codex 82/82, and anti-thrash engaged
(_ineffective_compression_count=2) still suppresses the block (0 calls).
The salvaged max-output/pressure fix set conversation_history=None after
the new pre-API compaction. That is only correct for legacy session-
rotation. Under the default in-place compaction (compression.in_place:
True), archive_and_compact inserts the compacted rows into the session DB
directly without stamping them with the intrinsic persisted-marker, so a
subsequent flush with conversation_history=None re-appends them — doubling
the active context and retriggering compression (the early-persist
duplicate-row trap).
Use conversation_history_after_compression(agent, messages), matching the
two existing compaction sites (post-response should_compress and the
turn-prologue preflight), which returns None for rotation and
list(messages) for in-place so the compacted dicts are skipped by identity.
Adds a regression test with a real SessionDB + real archive_and_compact
that asserts the compacted summary row is persisted exactly once (fails
with 2 copies on the None variant).
A single-model Hermes agent never sends temperature; the provider default
applies. MoA hardcoded reference_temperature=0.6 / aggregator_temperature=0.4,
and the coercion float(preset.get(key, 0.6) or 0.6) made unset IMPOSSIBLE to
express: absent, null, empty, and even an explicit 0 all collapsed to the
baked-in default. Every MoA advisor and aggregator therefore ran at 0.6/0.4
while the same model running solo used the provider default — silently
skewing solo-vs-MoA comparisons and overriding provider-tuned defaults.
- moa_config normalization: temperatures coerce to None when absent/blank/
invalid (new _coerce_float_or_none); explicit values incl. 0 honored.
- moa_loop: _preset_temperature() resolves preset values; None flows to
call_llm, which already omits the parameter when None (same contract as
max_tokens). Aggregator still inherits the acting agent's own configured
temperature when the preset doesn't pin one.
- conversation_loop (context-mode MoA): same resolution, no more hardcoded
0.6/0.4 at the call site.
- DEFAULT_CONFIG preset + web_server payload models + docs updated: unset
is the default, pinning stays available.
MoA per-turn latency is dominated by advisor GENERATION: turn wall time
correlates ~0.88 with output tokens and ~-0.03 with input tokens (measured over
52 turns). Each turn waits for the slowest advisor to finish writing, and
advisors were uncapped — writing multi-thousand-token essays the aggregator
only needs the gist of.
Add an opt-in per-preset reference_max_tokens knob (mirrors reference_temperature)
that caps ADVISOR output only; the acting aggregator is never capped. Default
None = uncapped, so existing presets are byte-for-byte unchanged (no regression).
Wired through both MoA execution paths (MoAChatCompletions.create and
aggregate_moa_context).
E2E: same task, closed preset uncapped vs reference_max_tokens=600 -> 59s to 33s
(~44% faster), final answer identical/correct.
- hermes_cli/moa_config.py: _coerce_int_or_none helper + reference_max_tokens
in _normalize_preset/_default_preset/flattened view
- agent/moa_loop.py: read preset.reference_max_tokens, pass to reference fan-out
- agent/conversation_loop.py: pass reference_max_tokens on the per-turn path
- tests + docs
On the MoA path agent.model/provider are the virtual preset name (e.g.
"closed") and "moa", which have no pricing entry. estimate_usage_cost()
returned None for the aggregator turn, so the `if amount_usd is not None`
guard skipped it and the session's estimated_cost_usd reflected only the
advisor fan-out — a ~50% undercount when the aggregator does the full acting
loop (verified: $0.91 advisor-only vs $1.96 true, aggregator = 54%).
MoAChatCompletions.create() now stashes the resolved aggregator slot as
last_aggregator_slot (exposed via MoAClient); conversation_loop reads it to
price the aggregator turn at its real model/provider. cost_source flips from
'none' to 'provider_models_api'.
Adds Vertex AI as a first-class provider for Gemini models via Vertex's
OpenAI-compatible endpoint. Vertex authenticates with short-lived OAuth2
access tokens (service-account JSON or ADC), not a static API key — the
missing piece behind the recurring requests (#13484, #12639, #56259).
- agent/vertex_adapter.py: OAuth2 token minting + refresh-on-expiry
(5-min margin), ADC->service-account fallback, global vs regional
endpoint URLs. Config precedence: env var > config.yaml > default.
- plugins/model-providers/vertex/: provider profile (auth_type=vertex),
reuses Gemini's extra_body.google.thinking_config translation.
- runtime_provider: vertex short-circuit BEFORE the credential pool so a
credentials-file path is never mistaken for a static API key; mints a
fresh token + computes base_url per resolve.
- run_agent + conversation_loop: _try_refresh_vertex_client_credentials()
re-mints the token and rebuilds the client on a mid-session 401, so a
long-lived gateway agent survives token expiry (~1h).
- auxiliary_client: vertex auth_type branch for side-LLM tasks.
- config.yaml: vertex.project_id / vertex.region (non-secret, bridged to
env); credential path stays in .env (VERTEX_CREDENTIALS_PATH).
- setup wizard + model picker: dedicated _model_flow_vertex; curated
google/gemini-* model list; --provider choices.
- pricing/metadata: Vertex prices off the gemini docs snapshot; endpoint
host auto-maps to the vertex provider (no probe spam).
- lazy_deps + pyproject [vertex] extra: google-auth, opt-in only.
- docs: guides/google-vertex.md + providers page; tests for adapter +
runtime resolution.
Salvages and modernizes #8427 by @slawt onto current main: rewired from
the legacy PROVIDER_REGISTRY path to the provider-profile architecture,
moved non-secret config out of .env into config.yaml, and added the
per-turn 401 token-refresh the original lacked.
MoA full-turn traces (moa.save_traces) recorded the aggregator's acting
output only on the non-streaming path, where it's captured inline at
call time. On the streaming path — which every hermes chat --query run
and every live gateway/CLI turn takes — the aggregator's raw token
stream is handed to the live consumer, so the trace left output=null and
only pointed at the session-db assistant row. An offline audit of a
benchmark run (HermesBench drives --query) then couldn't see what the
aggregator produced without hand-joining to state.db.
Capture the resolved streamed acting text at trace-flush time (the agent
already holds it in _current_streamed_assistant_text) and fold it into
the trace, so the record is self-contained in both modes. New
output_location value inline_from_stream marks a streamed turn whose text
was captured this way; a genuinely empty acting turn (pure tool call)
still points at the session db, matching state.db exactly.
Touches only the trace side-channel — no change to the acting path,
message history, role alternation, or prompt cache.
- agent/moa_loop.py: consume_and_save_trace(..., aggregator_output_fallback)
on both the facade and the MoAClient wrapper; prefer inline capture,
fall back to the resolved streamed text.
- agent/moa_trace.py: embed the fallback; add inline_from_stream location.
- agent/conversation_loop.py: pass _current_streamed_assistant_text at flush.
- tests: 5 cases across streaming / non-streaming / empty-fallback / no-double-write.
When text compression can't reduce a 413 request further, evict base64
image parts from tool messages and retry once instead of dead-ending
with 'Payload too large and cannot compress further.'
A 413 is a request-body byte-size limit, not a token limit. browser_vision
screenshots (2-5MB base64 each) keep the HTTP body oversized even after
aggressive summarization. The strip pass passes remember_model=False so a
413 does not poison _no_list_tool_content_models — that set is for providers
that reject list-type tool content, a distinct failure mode.
Cherry-picked from #47397 by Tranquil-Flow; placed onto main's current
token-aware 413 recovery else branch.
verify_on_stop / pre_verify append a synthetic assistant "done" plus a
synthetic user nudge to keep the agent going one more turn before it can
claim completion. Both were flagged (_verification_stop_synthetic on the
nudge only), but the flags were never registered in
_EPHEMERAL_SCAFFOLDING_FLAGS, so the central _is_ephemeral_scaffolding()
filter that guards both persistence sinks (SQLite flush + JSON snapshot)
let them through. The resumed transcript then inherited loop-only
scaffolding, invalidating the prompt-prefix cache on later turns.
- add _verification_stop_synthetic and _pre_verify_synthetic to
_EPHEMERAL_SCAFFOLDING_FLAGS (the single chokepoint both sinks use)
- flag the blocked attempt assistant message too, not just the nudge, so
the whole synthetic pair drops together and persistence does not keep a
premature done with the nudge stripped (assistant to assistant adjacency)
The API-payload leak claimed in the report is already handled: the
chat_completions transport strips every underscore-prefixed message key
before the wire, so the marker never reaches strict providers.
Reported by patppham.
