mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-06-10 08:32:09 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 11

feat: add xAI prompt caching via x-grok-conv-id header

Browse source 
When using xAI's API directly (base_url contains x.ai), send the
x-grok-conv-id header set to the Hermes session_id. This routes
consecutive requests to the same server, maximizing automatic
prompt cache hits.

Ref: https://docs.x.ai/developers/advanced-api-usage/prompt-caching
This commit is contained in:
Julien Talbot 2026-04-06 19:06:36 +04:00 • committed by Teknium
parent 0afa3a87d4
commit 92c19924a9
1 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
run_agent.py
View file

@ -5438,6 +5438,12 @@ class AIAgent:
if extra_body:
api_kwargs["extra_body"] = extra_body
# xAI prompt caching: send x-grok-conv-id header to route requests
# to the same server, maximizing automatic cache hits.
# https://docs.x.ai/developers/advanced-api-usage/prompt-caching
if "x.ai" in self._base_url_lower and hasattr(self, "session_id") and self.session_id:
api_kwargs["extra_headers"] = {"x-grok-conv-id": self.session_id}
return api_kwargs
def _supports_reasoning_extra_body(self) -> bool: