mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-06-09 08:21:50 +00:00
fix(run_agent): preserve dotted Bedrock inference-profile model IDs (#11976)
Bedrock rejects ``global-anthropic-claude-opus-4-7`` with ``HTTP 400:
The provided model identifier is invalid`` because its inference
profile IDs embed structural dots
(``global.anthropic.claude-opus-4-7``) that ``normalize_model_name``
was converting to hyphens. ``AIAgent._anthropic_preserve_dots`` did
not include ``bedrock`` in its provider allowlist, so every Claude-on-
Bedrock request through the AnthropicBedrock SDK path shipped with
the mangled model ID and failed.
Root cause
----------
``run_agent.py:_anthropic_preserve_dots`` (previously line 6589)
controls whether ``agent.anthropic_adapter.normalize_model_name``
converts dots to hyphens. The function listed Alibaba, MiniMax,
OpenCode Go/Zen and ZAI but not Bedrock, so when a user set
``provider: bedrock`` with a dotted inference-profile model the flag
returned False and ``normalize_model_name`` mangled every dot in the
ID. All four call sites in run_agent.py
(``build_anthropic_kwargs`` + three fallback / review / summary paths
at lines 6707, 7343, 8408, 8440) read from this same helper.
The bug shape matches #5211 for opencode-go, which was fixed in commit
f77be22c by extending this same allowlist.
Fix
---
* Add ``"bedrock"`` to the provider allowlist.
* Add ``"bedrock-runtime."`` to the base-URL heuristic as
defense-in-depth, so a custom-provider-shaped config with
``base_url: https://bedrock-runtime.<region>.amazonaws.com`` also
takes the preserve-dots path even if ``provider`` isn't explicitly
set to ``"bedrock"``. This mirrors how the code downstream at
run_agent.py:759 already treats either signal as "this is Bedrock".
Bedrock model ID shapes covered
-------------------------------
| Shape | Preserved |
| --- | --- |
| ``global.anthropic.claude-opus-4-7`` (reporter's exact ID) | ✓ |
| ``us.anthropic.claude-sonnet-4-5-20250929-v1:0`` | ✓ |
| ``apac.anthropic.claude-haiku-4-5`` | ✓ |
| ``anthropic.claude-3-5-sonnet-20241022-v2:0`` (foundation) | ✓ |
| ``eu.anthropic.claude-3-5-sonnet`` (regional inference profile) | ✓ |
Non-Claude Bedrock models (Nova, Llama, DeepSeek) take the
``bedrock_converse`` / boto3 path which does not call
``normalize_model_name``, so they were never affected by this bug
and remain unaffected by the fix.
Narrow scope — explicitly not changed
-------------------------------------
* ``bedrock_converse`` path (non-Claude Bedrock models) — already
correct; no ``normalize_model_name`` in that pipeline.
* Provider aliases (``aws``, ``aws-bedrock``, ``amazon``,
``amazon-bedrock``) — if a user bypasses the alias-normalization
pipeline and passes ``provider="aws"`` directly, the base-URL
heuristic still catches it because Bedrock always uses a
``bedrock-runtime.`` endpoint. Adding the aliases themselves to the
provider set is cheap but would be scope creep for this fix.
* No other places in ``agent/anthropic_adapter.py`` mangle dots, so
the fix is confined to ``_anthropic_preserve_dots``.
Regression coverage
-------------------
``tests/agent/test_bedrock_integration.py`` gains three new classes:
* ``TestBedrockPreserveDotsFlag`` (5 tests): flag returns True for
``provider="bedrock"`` and for Bedrock runtime URLs (us-east-1 and
ap-northeast-2 — the reporter's region); returns False for non-
Bedrock AWS URLs like ``s3.us-east-1.amazonaws.com``; canary that
Anthropic-native still returns False.
* ``TestBedrockModelNameNormalization`` (5 tests): every documented
Bedrock model-ID shape survives ``normalize_model_name`` with the
flag on; inverse canary pins that ``preserve_dots=False`` still
mangles (so a future refactor can't decouple the flag from its
effect).
* ``TestBedrockBuildAnthropicKwargsEndToEnd`` (2 tests): integration
through ``build_anthropic_kwargs`` shows the reporter's exact model
ID ends up unmangled in the outgoing kwargs.
Three of the new flag tests fail on unpatched ``origin/main`` with
``assert False is True`` (preserve-dots returning False for Bedrock),
confirming the regression is caught.
Validation
----------
``source venv/bin/activate && python -m pytest
tests/agent/test_bedrock_integration.py tests/agent/test_minimax_provider.py
-q`` -> 84 passed (40 new bedrock tests + 44 pre-existing, including
the minimax canaries that pin the pattern this fix mirrors).
CI-aligned broad suite: 12827 passed, 39 skipped, 19 pre-existing
baseline failures (all reproduce on clean ``origin/main``; none in
the touched code path).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Brian D. Evans
Teknium
parent
323e827f4a
commit
1cf1016e72
2 changed files with 194 additions and 3 deletions
|
|
@ -267,3 +267,174 @@ class TestPackaging:
|
|||
from pathlib import Path
|
||||
content = (Path(__file__).parent.parent.parent / "pyproject.toml").read_text()
|
||||
assert '"hermes-agent[bedrock]"' in content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Model ID dot preservation — regression for #11976
|
||||
# ---------------------------------------------------------------------------
|
||||
# AWS Bedrock inference-profile model IDs embed structural dots:
|
||||
#
|
||||
# global.anthropic.claude-opus-4-7
|
||||
# us.anthropic.claude-sonnet-4-5-20250929-v1:0
|
||||
# apac.anthropic.claude-haiku-4-5
|
||||
#
|
||||
# ``agent.anthropic_adapter.normalize_model_name`` converts dots to hyphens
|
||||
# unless the caller opts in via ``preserve_dots=True``. Before this fix,
|
||||
# ``AIAgent._anthropic_preserve_dots`` returned False for the ``bedrock``
|
||||
# provider, so Claude-on-Bedrock requests went out with
|
||||
# ``global-anthropic-claude-opus-4-7`` (all dots mangled to hyphens) and
|
||||
# Bedrock rejected them with:
|
||||
#
|
||||
# HTTP 400: The provided model identifier is invalid.
|
||||
#
|
||||
# The fix adds ``bedrock`` to the preserve-dots provider allowlist and
|
||||
# ``bedrock-runtime.`` to the base-URL heuristic, mirroring the shape of
|
||||
# the opencode-go fix for #5211 (commit f77be22c), which extended this
|
||||
# same allowlist.
|
||||
|
||||
|
||||
class TestBedrockPreserveDotsFlag:
|
||||
"""``AIAgent._anthropic_preserve_dots`` must return True on Bedrock so
|
||||
inference-profile IDs survive the normalize step intact."""
|
||||
|
||||
def test_bedrock_provider_preserves_dots(self):
|
||||
from types import SimpleNamespace
|
||||
agent = SimpleNamespace(provider="bedrock", base_url="")
|
||||
from run_agent import AIAgent
|
||||
assert AIAgent._anthropic_preserve_dots(agent) is True
|
||||
|
||||
def test_bedrock_runtime_us_east_1_url_preserves_dots(self):
|
||||
"""Defense-in-depth: even without an explicit ``provider="bedrock"``,
|
||||
a ``bedrock-runtime.us-east-1.amazonaws.com`` base URL must not
|
||||
mangle dots."""
|
||||
from types import SimpleNamespace
|
||||
agent = SimpleNamespace(
|
||||
provider="custom",
|
||||
base_url="https://bedrock-runtime.us-east-1.amazonaws.com",
|
||||
)
|
||||
from run_agent import AIAgent
|
||||
assert AIAgent._anthropic_preserve_dots(agent) is True
|
||||
|
||||
def test_bedrock_runtime_ap_northeast_2_url_preserves_dots(self):
|
||||
"""Reporter-reported region (ap-northeast-2) exercises the same
|
||||
base-URL heuristic."""
|
||||
from types import SimpleNamespace
|
||||
agent = SimpleNamespace(
|
||||
provider="custom",
|
||||
base_url="https://bedrock-runtime.ap-northeast-2.amazonaws.com",
|
||||
)
|
||||
from run_agent import AIAgent
|
||||
assert AIAgent._anthropic_preserve_dots(agent) is True
|
||||
|
||||
def test_non_bedrock_aws_url_does_not_preserve_dots(self):
|
||||
"""Unrelated AWS endpoints (e.g. ``s3.us-east-1.amazonaws.com``)
|
||||
must not accidentally activate the dot-preservation heuristic —
|
||||
the heuristic is scoped to the ``bedrock-runtime.`` substring
|
||||
specifically."""
|
||||
from types import SimpleNamespace
|
||||
agent = SimpleNamespace(
|
||||
provider="custom",
|
||||
base_url="https://s3.us-east-1.amazonaws.com",
|
||||
)
|
||||
from run_agent import AIAgent
|
||||
assert AIAgent._anthropic_preserve_dots(agent) is False
|
||||
|
||||
def test_anthropic_native_still_does_not_preserve_dots(self):
|
||||
"""Canary: adding Bedrock to the allowlist must not weaken the
|
||||
existing Anthropic native behaviour — ``claude-sonnet-4.6`` still
|
||||
becomes ``claude-sonnet-4-6`` for the Anthropic API."""
|
||||
from types import SimpleNamespace
|
||||
agent = SimpleNamespace(provider="anthropic", base_url="https://api.anthropic.com")
|
||||
from run_agent import AIAgent
|
||||
assert AIAgent._anthropic_preserve_dots(agent) is False
|
||||
|
||||
|
||||
class TestBedrockModelNameNormalization:
|
||||
"""End-to-end: ``normalize_model_name`` + the preserve-dots flag
|
||||
reproduce the exact production request shape for each Bedrock model
|
||||
family, confirming the fix resolves the reporter's HTTP 400."""
|
||||
|
||||
def test_global_anthropic_inference_profile_preserved(self):
|
||||
"""The reporter's exact model ID."""
|
||||
from agent.anthropic_adapter import normalize_model_name
|
||||
assert normalize_model_name(
|
||||
"global.anthropic.claude-opus-4-7", preserve_dots=True
|
||||
) == "global.anthropic.claude-opus-4-7"
|
||||
|
||||
def test_us_anthropic_dated_inference_profile_preserved(self):
|
||||
"""Regional + dated Sonnet inference profile."""
|
||||
from agent.anthropic_adapter import normalize_model_name
|
||||
assert normalize_model_name(
|
||||
"us.anthropic.claude-sonnet-4-5-20250929-v1:0",
|
||||
preserve_dots=True,
|
||||
) == "us.anthropic.claude-sonnet-4-5-20250929-v1:0"
|
||||
|
||||
def test_apac_anthropic_haiku_inference_profile_preserved(self):
|
||||
"""APAC inference profile — same structural-dot shape."""
|
||||
from agent.anthropic_adapter import normalize_model_name
|
||||
assert normalize_model_name(
|
||||
"apac.anthropic.claude-haiku-4-5", preserve_dots=True
|
||||
) == "apac.anthropic.claude-haiku-4-5"
|
||||
|
||||
def test_preserve_false_mangles_as_documented(self):
|
||||
"""Canary: with ``preserve_dots=False`` the function still
|
||||
produces the broken all-hyphen form — this is the shape that
|
||||
Bedrock rejected and that the fix avoids. Keeping this test
|
||||
locks in the existing behaviour of ``normalize_model_name`` so a
|
||||
future refactor doesn't accidentally decouple the knob from its
|
||||
effect."""
|
||||
from agent.anthropic_adapter import normalize_model_name
|
||||
assert normalize_model_name(
|
||||
"global.anthropic.claude-opus-4-7", preserve_dots=False
|
||||
) == "global-anthropic-claude-opus-4-7"
|
||||
|
||||
def test_bare_foundation_model_id_preserved(self):
|
||||
"""Non-inference-profile Bedrock IDs
|
||||
(e.g. ``anthropic.claude-3-5-sonnet-20241022-v2:0``) use dots as
|
||||
vendor separators and must also survive intact under
|
||||
``preserve_dots=True``."""
|
||||
from agent.anthropic_adapter import normalize_model_name
|
||||
assert normalize_model_name(
|
||||
"anthropic.claude-3-5-sonnet-20241022-v2:0",
|
||||
preserve_dots=True,
|
||||
) == "anthropic.claude-3-5-sonnet-20241022-v2:0"
|
||||
|
||||
|
||||
class TestBedrockBuildAnthropicKwargsEndToEnd:
|
||||
"""Integration: calling ``build_anthropic_kwargs`` with a Bedrock-
|
||||
shaped model ID and ``preserve_dots=True`` produces the unmangled
|
||||
model string in the outgoing kwargs — the exact body sent to the
|
||||
``bedrock-runtime.`` endpoint. This is the integration-level
|
||||
regression for the reporter's HTTP 400."""
|
||||
|
||||
def test_bedrock_inference_profile_survives_build_kwargs(self):
|
||||
from agent.anthropic_adapter import build_anthropic_kwargs
|
||||
kwargs = build_anthropic_kwargs(
|
||||
model="global.anthropic.claude-opus-4-7",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
tools=None,
|
||||
max_tokens=1024,
|
||||
reasoning_config=None,
|
||||
preserve_dots=True,
|
||||
)
|
||||
assert kwargs["model"] == "global.anthropic.claude-opus-4-7", (
|
||||
"Bedrock inference-profile ID was mangled in build_anthropic_kwargs: "
|
||||
f"{kwargs['model']!r}"
|
||||
)
|
||||
|
||||
def test_bedrock_model_mangled_without_preserve_dots(self):
|
||||
"""Inverse canary: without the flag, ``build_anthropic_kwargs``
|
||||
still produces the broken form — so the fix in
|
||||
``_anthropic_preserve_dots`` is the load-bearing piece that
|
||||
wires ``preserve_dots=True`` through to this builder for the
|
||||
Bedrock case."""
|
||||
from agent.anthropic_adapter import build_anthropic_kwargs
|
||||
kwargs = build_anthropic_kwargs(
|
||||
model="global.anthropic.claude-opus-4-7",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
tools=None,
|
||||
max_tokens=1024,
|
||||
reasoning_config=None,
|
||||
preserve_dots=False,
|
||||
)
|
||||
assert kwargs["model"] == "global-anthropic-claude-opus-4-7"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue