mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
feat(agent): report context occupancy, not just tokens saved
Tokens saved is the wrong headline for this feature. Micro-compaction is not an efficiency optimisation — the same summarization work happens either way. What it buys is (a) that work amortized across turns instead of one stall, and (b) a window kept low enough that a session runs much further before needing a hard compaction at all. Neither shows up in "net tokens saved". A session can save nothing on paper and still be a clear win on both counts. So the telemetry now carries occupancy: tokens_after as a share of the compaction threshold, plus the threshold and resolved window it was computed from. That is the number that says whether a session has headroom left. The report leads with it, and cross-references the batch `compression_attempt` lines already in the log so it can show how often the long pause actually fired — ideally never. Occupancy is read from the cached threshold only. The public `threshold_tokens` property resolves lazily and can issue a synchronous /models probe (#32221); telemetry must never be the thing that blocks a turn, so an unresolved window reports null. In practice a pass has already resolved it via the tail calculation, so the field is populated. A test pins the no-forcing behaviour directly against the emitter. The report is pure ASCII: `scripts/check_subprocess_stdin.py` currently dies on a cp1252 console before printing its results, and a diagnostic tool that crashes on the platform it is diagnosing is worse than no tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
cac9526d2a
commit
ac48add3a7
4 changed files with 201 additions and 57 deletions
|
|
@ -162,16 +162,40 @@ compaction. Everything else about compression is unchanged.
|
|||
|
||||
## Measuring it
|
||||
|
||||
Micro-compaction is not primarily a token-saving or time-saving optimisation,
|
||||
and judging it on tokens saved will undersell it. The two things it actually
|
||||
buys you are:
|
||||
|
||||
1. **The long pause is amortized.** The same summarization work happens, but as
|
||||
small increments after turns instead of one stall in the middle of a session.
|
||||
2. **Your context lasts longer.** Because the middle is continuously reclaimed,
|
||||
occupancy stays low instead of sawtoothing up to the threshold. A session
|
||||
runs much further — often indefinitely — before it needs a hard compaction
|
||||
at all.
|
||||
|
||||
So the number that matters is **occupancy**: how full the window is being kept,
|
||||
as a percentage of the compaction threshold. A session that holds steady around
|
||||
40% has headroom to keep going; one climbing through 90% is about to stall. The
|
||||
second number is **how many batch compactions actually fired** — ideally none.
|
||||
|
||||
A session can save nothing on paper and still be a clear win on both counts.
|
||||
|
||||
Every pass emits one content-free JSON line, in the same style as the batch
|
||||
compaction telemetry:
|
||||
|
||||
```
|
||||
micro compaction telemetry: {"event":"micro_compaction","outcome":"absorbed",
|
||||
"tokens_before":12739,"tokens_after":12060,"tokens_delta":-679,
|
||||
"occupancy_pct":38.4,"threshold_tokens":34816,"context_limit":40960,
|
||||
"exchange_tokens":868,"rolling_summary_tokens":31,"passes_total":1,
|
||||
"tokens_saved_total":679,"duration_ms":14,...}
|
||||
```
|
||||
|
||||
`occupancy_pct` is `tokens_after` as a share of the compaction threshold -- the
|
||||
headroom figure. It is null when the model's window has not been resolved yet:
|
||||
the telemetry reads only the cached value, because resolving it can issue a
|
||||
synchronous `/models` probe and telemetry must never be what blocks a turn.
|
||||
|
||||
`tokens_delta` is negative when the pass shrank the transcript.
|
||||
`tokens_saved_total` and `passes_total` accumulate across the session, so a whole
|
||||
run can be summarised from its last line. No transcript content appears in the
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue