mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-04-26 01:01:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

docs: add warning about summary model context length requirement

Browse source 
The summary model used for context compaction must have a context window
at least as large as the main agent model. If it's smaller, the
summarization API call fails and middle turns are dropped without a
summary, silently losing conversation context.

Promoted the existing note in configuration.md to a visible warning
admonition, and added a matching warning in the developer guide's
context compression page.
This commit is contained in:
Teknium 2026-04-11 11:03:10 -07:00
parent 50bb4fe010
commit 7fe1dc4e1d
No known key found for this signature in database
2 changed files with 7 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
website/docs/developer-guide/context-compression-and-caching.md
View file

@ -143,6 +143,10 @@ to find the parent assistant message, keeping groups intact.
### Phase 3: Generate Structured Summary
:::warning Summary model context length
The summary model must have a context window **at least as large** as the main agent model's. The entire middle section is sent to the summary model in a single `call_llm(task="compression")` call. If the summary model's context is smaller, the API returns a context-length error — `_generate_summary()` catches it, logs a warning, and returns `None`. The compressor then drops the middle turns **without a summary**, silently losing conversation context. This is the most common cause of degraded compaction quality.
:::
The middle turns are summarized using the auxiliary LLM with a structured
template: