docs: warn that mid-session model switches break prompt caching (#58747)

/model switches, primary-model fallback, and credential-pool key
rotation all change the prompt-cache key (model and/or account), so
the next turn re-reads the entire conversation at full input price.
Add cost warnings everywhere docs recommend or describe these paths:

- reference/slash-commands.md: cost note on both /model rows
- user-guide/features/fallback-providers.md: warning admonition
- user-guide/features/credential-pools.md: warning admonition
- user-guide/configuring-models.md: mid-session switch warning
- guides/tips.md: expand cache tip + /model tip
- reference/faq.md: warning on the switch-back-and-forth example
- user-guide/desktop.md: composer picker bullet
- developer-guide/context-compression-and-caching.md: new
  cache-aware design pattern (model identity is part of the key)
This commit is contained in:
Teknium 2026-07-05 04:34:05 -07:00 committed by GitHub
parent 7fde19afcc
commit 1c156736dc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 31 additions and 4 deletions

View file

@ -361,6 +361,16 @@ The marker is applied differently based on content type:
4. **TTL selection**: Default is `5m` (5 minutes). Use `1h` for long-running
sessions where the user takes breaks between turns.
5. **Model identity is part of the cache key**: Provider-side caches are scoped
to the model (and account/API key) serving the request. Any mid-conversation
model change — an explicit `/model` switch, primary-model fallback, or a
credential-pool rotation onto a different account — means the next request
gets zero cache hits and re-reads the full conversation at undiscounted
input price. This is inherent to how provider caches work, not something
Hermes can avoid; user-facing docs for `/model`, fallback providers, and
credential pools carry cost warnings for this reason. Don't add features
that silently swap the model or credentials mid-session.
### Enabling Prompt Caching
Prompt caching is automatically enabled when: