mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-22 16:25:58 +00:00
feat(nous): send top-level session_id for provider sticky routing (#69253)
* feat(nous): send top-level session_id for provider sticky routing The Nous Portal profile only embedded the session id inside portal tags, so Claude traffic through the portal had no sticky-routing key. Multi-turn sessions could reroute between upstream endpoints (Anthropic/Vertex/ Bedrock), cold-writing a fresh prompt cache on every reroute since each provider's cache is instance-local. Mirror the OpenRouter profile: emit extra_body.session_id whenever the agent has one, pinning every turn of a session to the same endpoint so explicit cache_control breakpoints stay warm. * test: expect top-level session_id in Nous max-iterations summary body Sibling site of the profile change — the max-iterations summary path builds its request through the same NousProfile.build_extra_body(), so its exact-shape assertion now includes the sticky-routing session_id when the agent has a session.
This commit is contained in:
parent
0c4cab56b4
commit
4c64ff3aa0
3 changed files with 26 additions and 3 deletions
|
|
@ -14,6 +14,15 @@ class NousProfile(ProviderProfile):
|
|||
self, *, session_id: str | None = None, **context
|
||||
) -> dict[str, Any]:
|
||||
body: dict[str, Any] = {"tags": nous_portal_tags(session_id=session_id)}
|
||||
if session_id:
|
||||
# Top-level session_id → provider sticky routing key. Pins every
|
||||
# turn of a session to the same upstream endpoint so explicit
|
||||
# Anthropic cache_control breakpoints stay warm instead of
|
||||
# cold-writing a fresh cache on each reroute (Anthropic/Vertex/
|
||||
# Bedrock caches are instance-local). Mirrors the OpenRouter
|
||||
# profile; without it the portal falls back to hashing the opening
|
||||
# messages, which breaks pinning whenever those shift.
|
||||
body["session_id"] = session_id
|
||||
provider_preferences = context.get("provider_preferences")
|
||||
if provider_preferences:
|
||||
body["provider"] = provider_preferences
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue