docs(agent): state that user turns are never micro-compacted

`_find_one_exchange`'s docstring described an exchange as "(optional) user
message + assistant message + its tool results", but the walk skips past
user messages and starts at the assistant, so user turns are never absorbed
into the rolling summary.

The code is right and the docstring was wrong. Assistant output is largely
an account of what was done and survives summarising with little loss. The
user's messages are the intent everything else is derived from and cannot be
reconstructed from the work that followed — paraphrasing "use the existing
helper, don't add a new one" into a summary is how an agent ends up doing
the opposite six turns later. They are also cheap: a prompt is normally a
tiny fraction of what one tool result costs.

Correct the docstring, document the property (and its cost — a floor on how
small the middle can get, since user turns accumulate), and add a test so it
stays deliberate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Michael Jordan 2026-07-29 15:10:54 -04:00 committed by kshitij
parent cd9d9d03b3
commit 214b5d8612
3 changed files with 60 additions and 5 deletions

View file

@ -5043,14 +5043,24 @@ This compaction should PRIORITISE preserving all information related to the focu
) -> Optional[tuple[int, int]]:
"""Find the next complete exchange starting at *start*.
An exchange is: (optional) user message + assistant message + its tool
results. Returns ``(exchange_start, exchange_end)`` indices into
*messages*, or ``None`` if no complete exchange is available before
*tail_start*.
An exchange is an assistant message plus its tool results. Returns
``(exchange_start, exchange_end)`` indices into *messages*, or ``None``
if no complete exchange is available before *tail_start*.
Tool results are consumed as a group following the assistant message
(consecutive ``tool``-role messages). The assistant message itself must
exist; without one there is nothing to summarise.
User messages are deliberately NOT part of an exchange. The walk skips
past them to reach the assistant message, and ``exchange_start`` is that
assistant index, so user turns are never absorbed into the rolling
summary and stay verbatim for the life of the session. This is the
intended behaviour, not an oversight: what the assistant emits is
largely an account of what it did, which survives summarising, while the
user's own words are the instructions everything else is derived from
and are the one thing that cannot be reconstructed from context. They
are also cheap a prompt is normally a tiny fraction of the tokens a
single tool result costs.
"""
idx = start
n = len(messages)

View file

@ -44,9 +44,33 @@ followed it. In tool-heavy work that's where the bulk of the tokens live — a
file read or a command's output dwarfs the surrounding prose — which is why
absorbing one exchange at a time is worth doing at all.
## Your messages are never compacted
An exchange deliberately starts at the *assistant* message. Micro-compaction
walks straight past user messages to get there, so **what you typed is never
summarized** — your prompts stay verbatim for the entire session, no matter how
long it runs or how many times compaction fires.
This is the most useful property of the whole design, and it's worth being
explicit about why. What the assistant produces is largely an account of what it
did: it read this file, it ran that command, it got this result. That kind of
narration survives summarising with very little loss — "it did it this way" is
about as informative compressed as it was in full. Your instructions are a
different kind of thing. They're the intent everything else is derived from, and
they cannot be reconstructed from the work that followed. Paraphrasing "use the
existing retry helper, don't add a new one" into a summary is exactly how an
agent ends up confidently doing the thing you told it not to, six turns later.
So the asymmetry is on purpose: compact the derived material, keep the source of
truth. The cost is a floor on how small the middle can get, since user turns
accumulate and are never absorbed. In practice that floor is low — a prompt is
normally a tiny fraction of what a single tool result costs — but it is a real
floor. If you routinely paste 1020K-token prompts, that weight stays in context
by design.
## What it never touches
Two regions are protected and stay verbatim:
Two more regions are protected and stay verbatim:
- **The head** — the system prompt and the opening messages, so the session's
founding instructions are never paraphrased.

View file

@ -101,6 +101,27 @@ class TestMicroCompaction:
assert result[0] == messages[0], "system prompt must be preserved"
assert result[-1] == messages[-1], "most recent turn must be preserved"
def test_user_messages_are_never_absorbed(self):
"""User turns stay verbatim for the life of the session — by design.
Assistant output is largely an account of what was done and survives
summarising; the user's own words are the intent everything else is
derived from and can't be reconstructed from it. So an exchange starts
at the assistant message and the walk skips past user turns.
"""
cc = _compressor()
messages = _conversation(exchanges=10)
originals = [m["content"] for m in messages if m["role"] == "user"]
for _ in range(5):
messages = cc._micro_compact(messages)
surviving = [
m["content"] for m in messages
if m.get("role") == "user" and not m.get(COMPRESSED_SUMMARY_METADATA_KEY)
]
assert surviving == originals, "user turns must survive verbatim"
def test_short_conversation_is_untouched(self):
cc = _compressor()
messages = [