follow-up: integrate agent nudge + dispatcher retry docs and tests

- Nudge text now warns that repeated protocol violations will block the
  task and require manual intervention, so the model understands the
  consequence of ignoring the nudge.
- Kanban docs restructured to clearly separate the two defense layers:
  agent-side prevention (nudge, from #64350) and dispatcher-side
  recovery (bounded retry, from this PR).
- Two new integration tests verifying the nudge mentions blocking and
  that the agent-side and dispatcher-side budgets are independent.
This commit is contained in:
kshitijk4poor 2026-07-14 16:36:12 +05:30 committed by kshitij
parent 3cd8feb63c
commit 52cafa6f8e
3 changed files with 55 additions and 7 deletions

View file

@ -96,7 +96,8 @@ def build_kanban_stop_nudge(
"1. Finish any remaining deliverable (write the required file(s) now).\n"
"2. Call `kanban_complete(summary=..., artifacts=[...])` if the work "
"is done, OR `kanban_block(reason=...)` if you are blocked.\n\n"
"Never end a turn with only a promise of future action.]"
"Never end a turn with only a promise of future action. Repeated "
"protocol violations will block this task and require manual intervention.]"
)

View file

@ -93,3 +93,41 @@ def test_nudge_budget_exhausted(clear_kanban_env):
assert build_kanban_stop_nudge(messages=[], attempts=2) is None
assert build_kanban_stop_nudge(messages=[], attempts=1, max_attempts=1) is None
assert build_kanban_stop_nudge(messages=[], attempts=0, max_attempts=1) is not None
# ── Integration: agent nudge + dispatcher bounded retry ──────────────
# These tests verify the two layers compose correctly: the agent-side
# nudge fires first (up to 2 attempts), and if the worker still exits
# without a terminal call, the dispatcher's bounded retry (streak of 3)
# handles it. See also tests/hermes_cli/test_kanban_core_functionality.py
# for the dispatcher-side streak tests.
def test_nudge_text_warns_about_blocking(clear_kanban_env):
"""The nudge should warn that repeated violations will block the task."""
clear_kanban_env.setenv("HERMES_KANBAN_TASK", "t_abc")
nudge = build_kanban_stop_nudge(messages=[], attempts=0)
assert nudge is not None
assert "block" in nudge.lower(), (
"nudge should warn that repeated violations will block the task"
)
def test_nudge_and_dispatcher_budgets_are_independent(clear_kanban_env):
"""Agent-side nudge budget (2) and dispatcher-side streak (3) are
separate budgets the nudge counter does not affect the dispatcher's
violation streak, and vice versa.
This is a source-level invariant check: the nudge counter
(``_kanban_stop_nudges``) lives on the AIAgent instance and resets
per session, while the dispatcher streak lives in the task_runs DB
table and persists across worker respawns.
"""
clear_kanban_env.setenv("HERMES_KANBAN_TASK", "t_abc")
# Agent-side: 2 nudge attempts per session
assert build_kanban_stop_nudge(messages=[], attempts=0) is not None
assert build_kanban_stop_nudge(messages=[], attempts=1) is not None
assert build_kanban_stop_nudge(messages=[], attempts=2) is None
# Dispatcher-side streak is tracked in the DB, not in the nudge module —
# the nudge module has no knowledge of the streak counter.
assert not hasattr(build_kanban_stop_nudge, "_streak")

View file

@ -370,12 +370,21 @@ Every profile that works kanban tasks automatically gets the worker lifecycle
That final `kanban_complete` / `kanban_block` call is part of the worker
protocol. If the worker process exits with status 0 while the task is still
`running`, the dispatcher treats that as a protocol violation and emits a
`protocol_violation` event. Because a protocol violation is **not
deterministic** — the model may emit the missing tool call on a later run —
the dispatcher gives it a **bounded retry** (up to
`_PROTOCOL_VIOLATION_FAILURE_LIMIT` consecutive violations, default 3) before
auto-blocking the task instead of respawning it into the same loop. The budget
counts only *consecutive* clean-exit protocol violations — interleaved
`protocol_violation` event.
**Agent-side prevention:** Before the worker exits, Hermes injects up to two
synthetic nudges when it detects the model is about to stop without a terminal
board tool call. This catches the common case where the model narrates the next
step ("Let me write the report") and stops with `finish_reason=stop`. The nudge
reminds the model to call `kanban_complete` or `kanban_block` immediately. This
guard is active only for dispatcher-spawned workers (`HERMES_KANBAN_TASK` is
set) and can be disabled with `HERMES_KANBAN_STOP_NUDGE=0`.
**Dispatcher-side recovery:** If the nudges are exhausted or the worker crashes
before reaching the nudge, the dispatcher gives the violation a **bounded retry**
(up to `_PROTOCOL_VIOLATION_FAILURE_LIMIT` consecutive violations, default 3)
before auto-blocking the task instead of respawning it into the same loop. The
budget counts only *consecutive* clean-exit protocol violations — interleaved
rate-limited requeues are neutral, and any other failure kind resets the
streak — and a per-task `max_retries` overrides the bound. This usually means
the model wrote a plain-text answer and exited without using the Kanban tool