mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-04-25 00:51:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: add warning about summary model context length requirement (#7879)

Browse source 
The summary model used for context compaction must have a context window
at least as large as the main agent model. If it's smaller, the
summarization API call fails and middle turns are dropped without a
summary, silently losing conversation context.

Promoted the existing note in configuration.md to a visible warning
admonition, and added a matching warning in the developer guide's
context compression page.
This commit is contained in:
Teknium 2026-04-11 11:13:48 -07:00 • committed by GitHub
parent 50bb4fe010
commit 55fac8a386
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 7 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
website/docs/developer-guide/context-compression-and-caching.md
View file

@ -143,6 +143,10 @@ to find the parent assistant message, keeping groups intact.
### Phase 3: Generate Structured Summary
:::warning Summary model context length
The summary model must have a context window **at least as large** as the main agent model's. The entire middle section is sent to the summary model in a single `call_llm(task="compression")` call. If the summary model's context is smaller, the API returns a context-length error — `_generate_summary()` catches it, logs a warning, and returns `None`. The compressor then drops the middle turns **without a summary**, silently losing conversation context. This is the most common cause of degraded compaction quality.
:::
The middle turns are summarized using the auxiliary LLM with a structured
template:

4
website/docs/user-guide/configuration.md
View file

@ -480,7 +480,9 @@ Points at a custom OpenAI-compatible endpoint. Uses `OPENAI_API_KEY` for auth.
| `nous` / `openrouter` / etc. | not set | Force that provider, use its auth |
| any | set | Use the custom endpoint directly (provider ignored) |
The `summary_model` must support a context length at least as large as your main model's, since it receives the full middle section of the conversation for compression.
:::warning Summary model context length requirement
The `summary_model` **must** have a context window at least as large as your main agent model's. The compressor sends the full middle section of the conversation to the summary model — if that model's context window is smaller than the main model's, the summarization call will fail with a context length error. When this happens, the middle turns are **dropped without a summary**, losing conversation context silently. If you override `summary_model`, verify its context length meets or exceeds your main model's.
:::
## Context Engine