fix(anthropic): keep the assistant cache breakpoint on the ordered-replay path

`apply_anthropic_cache_control` marks an assistant turn with non-empty text by
writing `cache_control` INTO `content` -- `_apply_cache_marker`'s list branch
puts it on the last content block, not at the top level.

`_convert_assistant_message`'s ordered-replay branch rebuilds the message from
`anthropic_content_blocks` and returns early. Its only cache sources are
`_relocated_replay_cache_control` (markers rescued from blocks the replay
sanitizer dropped) and the top-level `m["cache_control"]`; it never reads
`m["content"]`. So for an assistant turn that interleaves signed thinking with
a tool_use AND has preamble text, the breakpoint is dropped.

It is burned, not relocated: `_can_carry_marker` returns True for this message
(non-empty content), so the breakpoint budget already counted it. Hermes'
accounting believes the marker landed.

Measured with the real functions, native layout, same history, only
`anthropic_content_blocks` differing:

    normal path   4 breakpoints   assistant text block carries cache_control
    replay path   3 breakpoints   assistant carries NONE

Under the static-prefix layout only two conversation-tier breakpoints exist, so
this halves them. Nothing is logged and the request succeeds with identical
model output, which is why it survives: it recurs on every request for the life
of any Claude thinking+tools session, since the flag rides the message in
`agent.messages`.

#56195 fixed the complementary shape -- blank assistant content, where the
marker lands top-level -- and its regression test pins `"content": ""`. The
non-empty case is the one a Claude 4.5/4.6 tool turn normally produces (a
sentence of preamble before the call) and was never covered.

Harvest an in-`content` marker next to the existing top-level lookup and hand
it to the same `_apply_assistant_cache_control_to_last_cacheable_block` helper.
This commit is contained in:
dsad 2026-07-26 21:08:16 +00:00 committed by Teknium
parent 139282f241
commit c3e99fce49
2 changed files with 70 additions and 0 deletions

View file

@ -2060,6 +2060,28 @@ def _convert_assistant_message(m: Dict[str, Any]) -> Dict[str, Any]:
_apply_assistant_cache_control_to_last_cacheable_block(
replayed, m.get("cache_control")
)
# apply_anthropic_cache_control marks an assistant turn with
# non-empty text by writing cache_control INTO ``content`` (see
# _apply_cache_marker's list branch), not at the top level. This
# branch rebuilds the message from ordered_blocks and never reads
# ``content``, so that marker would be dropped -- and because
# _can_carry_marker already counted this message as a carrier, the
# breakpoint is burned rather than relocated. #56195 covered the
# complementary shape (blank content -> top-level marker); this is
# the interleaved thinking + preamble-text + tool_use shape.
_inline_cc = None
_msg_content = m.get("content")
if isinstance(_msg_content, list):
for _blk in _msg_content:
if isinstance(_blk, dict) and isinstance(
_blk.get("cache_control"), dict
):
_inline_cc = _blk["cache_control"]
break
if _inline_cc is not None:
_apply_assistant_cache_control_to_last_cacheable_block(
replayed, _inline_cc
)
return {"role": "assistant", "content": replayed}
blocks = _extract_preserved_thinking_blocks(m)

View file

@ -1190,6 +1190,54 @@ class TestConvertMessages:
assert tool_use["id"] == "tc_1"
assert tool_use["cache_control"] == {"type": "ephemeral"}
def test_ordered_replay_keeps_cache_control_from_nonempty_content(self):
"""An assistant turn that interleaves signed thinking with a tool_use
AND has preamble text carries its cache_control INSIDE ``content``
(apply_anthropic_cache_control marks the last content block, not the
top level). The ordered-replay branch rebuilds the message from
``anthropic_content_blocks`` alone, so without harvesting that marker
the breakpoint is dropped -- and it is *burned*, because
_can_carry_marker already spent a budget slot on this message.
#56195 covers the blank-content shape; this is the non-empty one, which
is what a Claude thinking+tools turn normally looks like.
"""
preamble = "I will read a.py now."
messages = apply_anthropic_cache_control([
{"role": "system", "content": "System prompt"},
{"role": "user", "content": "Read a.py"},
{
"role": "assistant",
"content": preamble,
"anthropic_content_blocks": [
{"type": "thinking", "thinking": "Need a tool.", "signature": "sig_1"},
{"type": "text", "text": preamble},
{"type": "tool_use", "id": "tc_1", "name": "test_tool", "input": {}},
],
"tool_calls": [
{
"id": "tc_1",
"type": "function",
"function": {"name": "test_tool", "arguments": "{}"},
}
],
},
{"role": "tool", "tool_call_id": "tc_1", "content": "contents"},
])
_system, converted = convert_messages_to_anthropic(messages)
assistant = next(m for m in converted if m.get("role") == "assistant")
marked = [
b for b in assistant["content"]
if isinstance(b, dict) and b.get("cache_control")
]
assert marked, (
"the assistant cache breakpoint was dropped by the ordered-replay "
"path and the budget slot is burned"
)
# The signed thinking block must still lead the replayed message.
assert assistant["content"][0]["type"] == "thinking"
def test_ordered_replay_tool_use_cache_control_is_preserved(self):
messages = apply_anthropic_cache_control([
{"role": "system", "content": "System prompt"},