Claude-on-Bedrock already gets prompt caching via the AnthropicBedrock
SDK path. This adds it to the raw Converse API path used for non-Claude
models (Amazon Nova, and Claude when bearer-token auth forces Converse
routing, #28156) — a conservative model allowlist inserts cachePoint
blocks after tools, system, and the message before the newest turn, and
extracts cacheReadInputTokens/cacheWriteInputTokens into usage so cost
accounting picks them up through the existing Anthropic-style fallback.
Ref: relatorio-cache-performance-provedores-ia.md, P0 item 1.
- The #44861 stale-cache guard invalidated any cached value that differed
from the static table, which would have discarded legitimate
probe-derived windows larger than the table. Treat the table as a
FLOOR: only drop under-reporting cache entries.
- Update probe test fixtures that predated the 4.6+ 1M table flip
(opus-4-6 fallback expectations 200K -> 1M).
Bedrock models resolved their context window from a hardcoded table
(BEDROCK_CONTEXT_LENGTHS) keyed by longest-substring match. AWS ships
new model versions faster than the table tracks, so a new model like
claude-opus-4-8 (1M-token window) silently matched the older
"anthropic.claude-opus-4" entry and got capped at 200K — wasting 80%
of the available context.
Bedrock exposes the real window nowhere in metadata: get-foundation-model
omits it, Converse usage metrics omit it, CountTokens is unsupported on
several models. The only authoritative source is the ValidationException
raised when a prompt exceeds the window:
"prompt is too long: 1300032 tokens > 1000000 maximum"
Length validation runs before inference, so an oversized request is
rejected immediately and cheaply (no tokens generated, no input
processed). This adds probe_bedrock_context_length(): pad a request just
past a tier, parse the reported maximum, return it. get_bedrock_context_length()
now probes first and falls back to the static table only when the probe
can't run (missing creds, network error, unparseable error). The static
table stays as a safety net.
get_model_context_length() caches the probe result per model+region, so
the network cost is paid once, not every turn. probe=False / empty region
disables probing for offline/display paths — backward compatible with the
single-arg callers.
Verified E2E against live Bedrock (eu-central-1): claude-opus-4-8 resolves
to 1000000. Unit tests cover error parsing, unparseable errors, missing
client, probe-beats-table, and table fallback.
BEDROCK_CONTEXT_LENGTHS was missing entries for current 1M-context Claude
models, and the resolution path in get_model_context_length() short-circuits
to that table (step 1b) before DEFAULT_CONTEXT_LENGTHS is ever consulted, so
the catalog's correct values could never apply on Bedrock:
- claude-fable-5 (no entry at all) fell through to
BEDROCK_DEFAULT_CONTEXT_LENGTH and reported 128K for a 1M model.
- opus-4-7 / opus-4-8 substring-matched the generic 'anthropic.claude-opus-4'
key and reported 200K.
- opus-4-6 / sonnet-4-6 had explicit 200K entries predating their 1M windows.
The practical symptom: the agent compresses context prematurely (at ~128K or
~200K of a 1M window) on every Bedrock-hosted current Claude model.
Fixing the table alone is not enough for existing installs: a previously
persisted 128K/200K value in the context-length cache wins at step 1 and
masks the corrected table forever. Step 1 now reconciles Bedrock-context
cache hits against the static table (the table is authoritative for Bedrock
— there is no live probe to reconcile against), invalidating stale entries
so existing users converge to the right window without manual cache surgery.
Tests cover the new table entries (incl. inference-profile and versioned ID
forms), the 128K-default regression for Fable, the stale-cache invalidation
path, and that pre-4.6 models keep their 200K entries.
is_anthropic_bedrock_model() strips a regional prefix before checking for
"anthropic.claude" to route Claude through the AnthropicBedrock SDK path
(prompt caching, thinking budgets) instead of the Converse path. The prefix
list was missing "au." and "ap." does not match "apac.", so AU/APAC Claude
inference profiles silently lost prompt caching. Add "apac." and "au.".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses hermes-sweeper review on PR #66167:
1. _convert_content_to_converse() still emitted {"text": part} directly
for plain-string items inside a content list (as opposed to
{"type": "text"} dicts), bypassing _safe_text() entirely. A
whitespace-only string item (e.g. [" "]) could still reach Bedrock
as a blank block. Now routed through _safe_text().
2. tests/agent/test_bedrock_adapter.py::TestEmptyTextBlockFix asserted
the pre-fix behavior (whitespace -> literal space " "), contradicting
the new _safe_text()/_EMPTY_TEXT_PLACEHOLDER behavior added in
4618095a. Updated assertions to expect the non-whitespace placeholder,
plus added a regression test for the list-string-item case above.
The Bedrock static context table only had entries up to the 4-6
generation and capped all Claude models at 200K. Newer model IDs
(opus/sonnet 4-7, 4-8) silently inherited 200K via the generic
"anthropic.claude-opus-4" / "...-sonnet-4" substring fallback,
contradicting the native Anthropic table in model_metadata.py which
already maps these to 1M.
Map opus/sonnet 4-6/4-7/4-8 to 1_000_000 on Bedrock. Haiku 4.5,
sonnet 4.5, legacy Claude 4 and 3.x stay at 200K (no 1M window).
Add opus-4-8 coverage and haiku/sonnet-4-5 200K guards.
Update TestBedrockContextLength to assert the corrected values from
BEDROCK_CONTEXT_LENGTHS:
- test_claude_opus_4_6: 200_000 -> 1_000_000 (1M GA for Opus 4.6)
- test_claude_sonnet_versioned: 200_000 -> 1_000_000 (1M GA for Sonnet 4.6)
- test_inference_profile_resolves: 200_000 -> 1_000_000
(us.anthropic.claude-sonnet-4-6 resolves to Sonnet 4.6's 1M value)
Also adds three new test cases that document Anthropic's published
context windows explicitly and guard against future regressions:
- test_claude_opus_4_7: asserts 1_000_000 and cites the models overview
- test_claude_sonnet_4_5_is_200k: asserts 200_000 and cites the
April 30, 2026 release note retiring the 1M beta for Sonnet 4.5
- test_claude_haiku_4_5_is_200k: asserts 200_000 for Haiku 4.5
All 178 tests in test_bedrock_adapter.py pass after the change.
Rewrites the cherry-picked test to import os locally and monkeypatch the
resolver seams instead of patching bedrock_adapter internals; adds the
inverse assertion (no bearer -> AnthropicBedrock SDK path preserved).
Three fixes for the Bedrock Claude path:
1. Streaming fallback: When AnthropicBedrock SDK raises 'Unexpected event
order' (SDK misparses Bedrock error events as message_start), auto-switch
to native Converse API for the rest of the session instead of failing
after 3 retries.
2. Image base64 decode (#33317): data URL payloads were passed as base64
strings to source.bytes, but boto3 re-encodes at the wire layer. Now
decoded to raw bytes before passing to Converse API.
3. Bearer token routing (#28156): Users with AWS_BEARER_TOKEN_BEDROCK are
now routed through Converse API regardless of model, since the
AnthropicBedrock SDK only supports SigV4 signing.
3 new tests. 121 bedrock_adapter tests passing.
converse() and converse_stream() were added in boto3 1.34.59. When Hermes
is installed editable into system Python (e.g. Ubuntu 24.04 ships 1.34.46),
the system boto3 takes precedence and calls to converse_stream fail with
AttributeError. Add an early version check in _require_boto3() that raises
a clear RuntimeError with upgrade instructions.
Bedrock Converse rejects non-default sampling parameters for Opus 4.7 and 4.8 with a ValidationException. Reuse the Anthropic-native sampling-param guard in the Bedrock kwargs builder so those models omit temperature/topP while older Claude and non-Claude models keep existing behavior.
Includes the stop-sequence regression from the parallel fix to ensure stopSequences still pass through for restricted Opus models.
Co-authored-by: Tranquil-Flow <tranquil_flow@protonmail.com>
IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.
- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
detector (ClientError code check + wrapped-SDK message match);
call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
retries inline via converse() and sets _disable_streaming so later
turns skip the doomed stream attempt; the chat-completions retry
block also recognizes the denial for the AnthropicBedrock SDK path
(message pre-check avoids importing bedrock_adapter — and its lazy
boto3 install — for unrelated providers).
Both paths print a one-line notice telling the user which IAM action
restores streaming.
Remove unused imports (F401) and duplicate/shadowed import
redefinitions (F811) across the codebase using ruff's safe
autofixes. No behavioral changes -- imports only.
- ~1400 safe autofixes applied across 644 files (net -1072 lines)
- __init__.py re-exports preserved (excluded from F401 removal so
public re-export surfaces stay intact)
- Re-exports that are imported or monkeypatched by tests but look
unused in their defining module are kept with explicit # noqa:
F401 (gateway/run.py load_dotenv; run_agent re-exports from
agent.message_sanitization, agent.context_compressor,
agent.retry_utils, agent.prompt_builder, agent.process_bootstrap,
agent.codex_responses_adapter)
- Unsafe F841 (unused-variable) fixes deliberately skipped -- those
can change behavior when the RHS has side effects
- ruff lints remain disabled in pyproject.toml (only PLW1514 is
selected); this is a one-time cleanup, not a config change
Verification:
- python -m compileall: clean
- pytest --collect-only: all 27161 tests collect (zero import errors)
- core entry points import clean (run_agent, model_tools, cli,
toolsets, hermes_state, batch_runner, gateway)
- static scan: every name any test imports directly from an edited
module still resolves
Six tests in test_bedrock_adapter.py import botocore.exceptions
directly (ConnectionClosedError, EndpointConnectionError,
ReadTimeoutError, ClientError) without guarding the import. When
botocore is not installed (it's an optional dependency), these tests
fail with ModuleNotFoundError instead of being gracefully skipped.
Added pytest.importorskip('botocore') to each affected test function,
following the same pattern used elsewhere in the test suite (e.g.
test_voice_mode.py for numpy, test_mcp_oauth.py for mcp).
Tests affected:
- TestIsStaleConnectionError: 3 tests
- TestCallConverseInvalidatesOnStaleError: 3 tests
Before: 6 FAIL with ModuleNotFoundError
After: 6 SKIP with reason message
25 new tests (all Bedrock API calls mocked, no real AWS creds needed):
tests/hermes_cli/test_bedrock_model_picker.py (20 tests):
- provider_model_ids("bedrock") uses live discovery, returns regional
model IDs, falls back gracefully on empty/exception, resolves all
bedrock aliases (aws, aws-bedrock, amazon-bedrock) to live discovery
- list_authenticated_providers() section 2: bedrock appears with AWS
creds, model list from discover_bedrock_models(), total_models
matches, is_current flag works, absent creds hides bedrock, discovery
failure does not crash, no duplicate entries
- Region routing: botocore profile eu-central-1 yields eu.* model IDs
end-to-end; env var takes priority over botocore profile
- providers.py overlay: exists with correct transport/auth_type, label
is non-empty, all aliases normalize to bedrock
tests/agent/test_bedrock_adapter.py (5 tests):
- resolve_bedrock_region() botocore profile fallback, botocore failure
fallback, us-east-1 hard fallback (with botocore mocked)
## Problem
When a pooled HTTPS connection to the Bedrock runtime goes stale (NAT
timeout, VPN flap, server-side TCP RST, proxy idle cull), the next
Converse call surfaces as one of:
* botocore.exceptions.ConnectionClosedError / ReadTimeoutError /
EndpointConnectionError / ConnectTimeoutError
* urllib3.exceptions.ProtocolError
* A bare AssertionError raised from inside urllib3 or botocore
(internal connection-pool invariant check)
The agent loop retries the request 3x, but the cached boto3 client in
_bedrock_runtime_client_cache is reused across retries — so every
attempt hits the same dead connection pool and fails identically.
Only a process restart clears the cache and lets the user keep working.
The bare-AssertionError variant is particularly user-hostile because
str(AssertionError()) is an empty string, so the retry banner shows:
⚠️ API call failed: AssertionError
📝 Error:
with no hint of what went wrong.
## Fix
Add two helpers to agent/bedrock_adapter.py:
* is_stale_connection_error(exc) — classifies exceptions that
indicate dead-client/dead-socket state. Matches botocore
ConnectionError + HTTPClientError subtrees, urllib3
ProtocolError / NewConnectionError, and AssertionError
raised from a frame whose module name starts with urllib3.,
botocore., or boto3.. Application-level AssertionErrors are
intentionally excluded.
* invalidate_runtime_client(region) — per-region counterpart to
the existing reset_client_cache(). Evicts a single cached
client so the next call rebuilds it (and its connection pool).
Wire both into the Converse call sites:
* call_converse() / call_converse_stream() in
bedrock_adapter.py (defense-in-depth for any future caller)
* The two direct client.converse(**kwargs) /
client.converse_stream(**kwargs) call sites in run_agent.py
(the paths the agent loop actually uses)
On a stale-connection exception, the client is evicted and the
exception re-raised unchanged. The agent's existing retry loop then
builds a fresh client on the next attempt and recovers without
requiring a process restart.
## Tests
tests/agent/test_bedrock_adapter.py gets three new classes (14 tests):
* TestInvalidateRuntimeClient — per-region eviction correctness;
non-cached region returns False.
* TestIsStaleConnectionError — classifies botocore
ConnectionClosedError / EndpointConnectionError /
ReadTimeoutError, urllib3 ProtocolError, library-internal
AssertionError (both urllib3.* and botocore.* frames), and
correctly ignores application-level AssertionError and
unrelated exceptions (ValueError, KeyError).
* TestCallConverseInvalidatesOnStaleError — end-to-end: stale
error evicts the cached client, non-stale error (validation)
leaves it alone, successful call leaves it cached.
All 116 tests in test_bedrock_adapter.py pass.
Signed-off-by: Andre Kurait <andrekurait@gmail.com>
