fix(compression): stop compaction thrash — 75% trigger floor under 512K, no summary output cap, reasoning-trace exclusion (#60989)

Sessions on sub-512K-context models were spending most of their wall-clock
re-summarizing: the 50% trigger left too little post-compaction headroom
(the incompressible floor — system prompt, tool schemas, protected tail,
rolling summary — ate most of the reclaimed space), so compaction re-fired
every 1-2 turns. Three compounding defects fixed:

- Threshold floor: models with context windows below 512K now trigger at
  >=75% of the window (raise-only — a higher configured value or per-model
  autoraise like Codex gpt-5.5's 85% always wins). Re-derived on
  update_model() in both directions.
- No max_tokens on the summary call: the summary budget is prompt guidance
  only ("Target ~N tokens"). The wire cap truncated summaries mid-section
  on the Anthropic Messages / NVIDIA NIM paths (thinking models burn the
  cap on reasoning first), yielding truncated or thinking-only summaries
  and compaction loops. Summary token ceiling lowered 12K -> 10K to keep
  the guidance within the intended 1K-10K envelope.
- Reasoning traces excluded end-to-end: inline <think>/<reasoning> blocks
  are now stripped from assistant content before serialization to the
  summarizer, and from the summarizer's own output before the summary is
  stored (previously a thinking summarizer model's trace was persisted in
  _previous_summary and re-fed into every iterative update, compounding
  bloat). Native reasoning fields were already excluded.

Verified E2E with real imports against a temp HERMES_HOME: threshold table
across 64K-1M windows, override interactions (user 0.85 wins, spark 0.70
raised, gpt-5.5 0.85 kept), full compress() round-trip with a thinking
summarizer, and wire-kwargs capture proving no max_tokens is sent.
This commit is contained in:
Teknium 2026-07-08 11:56:17 -07:00 committed by GitHub
parent 8e734810df
commit 76381e2a8e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 321 additions and 40 deletions

View file

@ -409,6 +409,8 @@ compression:
# Trigger compression at this % of model's context limit (default: 0.50 = 50%)
# Lower values = more aggressive compression, higher values = compress later
# Models with context windows below 512K are floored at 0.75 (raise-only) so
# compaction doesn't fire with half the window still free; set above 0.75 to override.
threshold: 0.50
# Existing Codex gpt-5.5 behavior: raise Hermes' compaction trigger to 85%