mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-29 06:31:32 +00:00
perf(prompt-cache): date-only timestamp + loud gateway-DB roundtrip logging
The system prompt's 'Conversation started:' line carried minute precision
(%I:%M %p), making it byte-unstable across every rebuild path. Within a
CLI session the in-memory cache held, but on the gateway path (fresh
AIAgent per turn → restore from session DB), any silent failure in the
read or write path dropped the cache stem and forced a full re-prefill
on every subsequent turn. Local prefix-caching backends (llama.cpp /
vLLM) saw this as KV-cache invalidation; remote prefix-caching providers
saw it as an Anthropic-style cache miss.
Three changes:
1. Date-only timestamp ('Sunday, May 17, 2026' instead of '... 03:42 PM').
System prompt now byte-stable for the full day. The model can still
query exact time via tools when it actually needs it. Credit:
@iamfoz (PR #20451).
2. Loud logging on session DB write failures. The update_system_prompt
call used to log at DEBUG, hiding disk-full / locked-database / schema
drift behind a silent fall-through that forced fresh rebuilds on
every subsequent turn. Now WARN with the session id and exception so
persistent issues show up in agent.log without verbose mode.
3. Three-way stored-state distinction on read. The previous
'session_row.get("system_prompt") or None' collapsed three states
into one (missing row / null column / empty string). Now we tell them
apart and WARN when a continuing session lands on null/empty (which
means the previous turn's write never persisted — every subsequent
turn rebuilds and the prefix cache misses every time).
The restore block is extracted into _restore_or_build_system_prompt()
so the prefix-cache path can be unit-tested in isolation.
E2E proof: fresh AIAgent constructed for turn 2 across a minute-boundary
sleep restores byte-identical bytes from the session DB. NULL stored
prompt fires the new warning. Date-only timestamp survives the rebuild
path. All on real SessionDB, no mocks.
Tests:
- tests/agent/test_system_prompt_restore.py (10 new tests)
- tests/run_agent/test_run_agent.py::TestBuildSystemPrompt::
test_datetime_is_date_only_not_minute_precision
Closes #20451 (date-only), #18547 (prefix stabilization),
#8689 (stabilize timestamp across compression), #15866 (timestamp
caching question), #8687 (compression timestamp), #27339
(claim #3: live timestamp in cached system prompt).
Co-authored-by: Martyn Forryan <9133432+iamfoz@users.noreply.github.com>
This commit is contained in:
parent
9b91377bec
commit
4a3f13b47b
4 changed files with 355 additions and 38 deletions
|
|
@ -989,6 +989,28 @@ class TestBuildSystemPrompt:
|
|||
# Should contain current date info like "Conversation started:"
|
||||
assert "Conversation started:" in prompt
|
||||
|
||||
def test_datetime_is_date_only_not_minute_precision(self, agent):
|
||||
"""Timestamp must be date-only (no HH:MM) so the system prompt
|
||||
stays byte-stable for the full day. Minute precision invalidates
|
||||
prefix-cache KV on every rebuild path (compression, fresh-agent
|
||||
gateway turns, session resume without a stored prompt)."""
|
||||
prompt = agent._build_system_prompt()
|
||||
# Find the line and strip it for inspection
|
||||
for line in prompt.splitlines():
|
||||
if line.startswith("Conversation started:"):
|
||||
# Must NOT contain AM/PM indicator (minute precision had %I:%M %p)
|
||||
assert " AM" not in line and " PM" not in line, (
|
||||
f"Timestamp line has time-of-day, breaks daily cache stability: {line!r}"
|
||||
)
|
||||
# Must NOT contain a colon followed by two digits (HH:MM pattern)
|
||||
import re as _re
|
||||
assert not _re.search(r":\d{2}", line), (
|
||||
f"Timestamp line has HH:MM, breaks daily cache stability: {line!r}"
|
||||
)
|
||||
break
|
||||
else:
|
||||
assert False, "Expected a 'Conversation started:' line in the system prompt"
|
||||
|
||||
def test_includes_nous_subscription_prompt(self, agent, monkeypatch):
|
||||
monkeypatch.setattr(run_agent, "build_nous_subscription_prompt", lambda tool_names: "NOUS SUBSCRIPTION BLOCK")
|
||||
prompt = agent._build_system_prompt()
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue