hermes-agent/tests/tui_gateway/test_interim_assistant_callback.py
ethernet c363db81e0
fix(desktop, ink): don't wipe messages before final message (#65919)
* fix(desktop): preserve interim assistant text wiped at message.complete

When the agent emits interim text (commentary alongside tool calls, or the
attempted final answer before a verify-on-stop nudge), all UI surfaces
streamed it live but then wiped it at message.complete — keeping only the
final response. The user saw text appear during inference, then disappear.

This is the complete fix across all three layers: agent core, gateway
transport, and all UI surfaces (desktop + Ink TUI).

The verify-on-stop and pre_verify paths flagged the assistant's attempted
final answer as _verification_stop_synthetic, suppressing it from both
state.db and the UI. The user only saw the terse post-verification reply.

Now the assistant response is real content: it's persisted to state.db and
emitted as an interim message via _emit_interim_assistant_message(force_display=True)
before the verification loop runs. Only the synthetic nudge messages keep
the synthetic flags. The turn finalizer drops nudges from live history and
compares content (not just role) to avoid duplicating a published candidate.
Message sequence repair collapses verification candidates in the
consecutive-assistant merge.

Wire agent.interim_assistant_callback both at construction (_agent_cbs())
and per-turn (defense-in-depth), emitting a new message.interim event with
{text, already_streamed}. Gated on display.interim_assistant_messages
(default true). Cleared in the finally block so a stale closure can't
fire on a later turn.

Add message.interim to the GatewayEventName union (apps/shared) and a
typed payload to the TUI's GatewayEvent discriminated union.

The TUI already had the segment-anchoring machinery (flushStreamingSegment +
finalTail) but had no handler for message.interim. Added recordInterimMessage
+ interimBoundaryIndex to seal segments mid-turn, and updated
recordMessageComplete to only dedupe segments after the interim boundary.

Replaced the fragile sealed-set approach with a proper interimBoundaryPending
state flag on ClientSessionState. finalizeInterimAssistantMessage finalizes
the streaming bubble in place (or creates a standalone one), rotates the
stream ID so next deltas create a new bubble, and sets the flag. When the
final text equals an already-sealed interim, they stay as distinct messages.

Extracted mergeFinalAssistantText() as a pure function in chat-messages.ts,
used by both completeAssistantMessage and finalizeInterimAssistantMessage.
Split the bidirectional dedup predicate: reasoning is a restatement only when
the final FULLY covers it. A short final ("Done.") no longer swallows a
longer reasoning block that merely starts with it.

Honor display.interim_assistant_messages (default true) across all layers:
the tui_gateway gates the callback, the desktop wires it to a nanostores
atom via use-hermes-config. Updated hermes_cli/config.py and
cli-config.yaml.example comments to document the Desktop behavior.

_split_segment_tokens now accepts posix=False and _find_ad_hoc_match tries
both posix modes so ad-hoc verification scripts with Windows backslash
paths are matched correctly. (response_previewed forwarding from #53553
is not included — our emit-interim + persist approach makes it unnecessary
since the attempted answer is now surfaced before the verification loop.)

- tsc: clean (desktop + TUI + shared)
- vitest desktop: 73/73 pass (7 interim-sealing + 5 mergeFinalAssistantText + 4 config atom)
- vitest TUI: 83/83 pass (4 new message.interim tests)
- python: 390 tests pass (340 tui_gateway + 33 verification/finalizer + 6 config gating + 3 evidence + 8 continuation budget)

Co-authored-by: Liam Zhang <yingliang-zhang@users.noreply.github.com>
Co-authored-by: Lucas D'Alessandro <lucasfdale@users.noreply.github.com>
Co-authored-by: Eric Manganaro <superposition@users.noreply.github.com>
Co-authored-by: sweetcornna <sweetcornna@users.noreply.github.com>
Co-authored-by: DECK6 <DECK6@users.noreply.github.com>
Co-authored-by: matantsevs <matantsevs@users.noreply.github.com>
Co-authored-by: gitcommit90 <gitcommit90@users.noreply.github.com>

* fix: prefix-match interim streamed content to avoid benign duplicate bubbles

_interim_content_was_streamed used exact equality (streamed == visible_content),
so a final response that was the streamed text plus a trailing delta — or a
partial stream before the verify nudge fired — failed the match and left
_response_was_previewed false. The turn then showed two bubbles (interim +
identical final) instead of settling the interim in place.

Relax to a prefix check (visible_content.startswith(streamed)) in both the
core match and the desktop's settle-in-place gate. The TUI already used
prefix matching via finalTail. The reverse direction (streamed longer than
final) is intentionally not matched — that could suppress a needed resend
in the gateway path where already_streamed=True calls on_segment_break().

* test(desktop): add partial-stream-then-nudge dedup edge case

Third edge case for the interim-sealing dedup: model streams part of its
answer via message.delta, verify nudge fires, interim seals the streamed
prefix, then the final response is the same text plus a trailing delta.
Asserts one bubble (not two) containing the full final text.

Acceptance protocol #2 — covers all three dedup edges:
  1. interim == final (existing)
  2. interim = strict prefix of final (existing)
  3. partial-stream-then-nudge (this commit)

---------

Co-authored-by: Liam Zhang <yingliang-zhang@users.noreply.github.com>
Co-authored-by: Lucas D'Alessandro <lucasfdale@users.noreply.github.com>
Co-authored-by: Eric Manganaro <superposition@users.noreply.github.com>
Co-authored-by: sweetcornna <sweetcornna@users.noreply.github.com>
Co-authored-by: DECK6 <DECK6@users.noreply.github.com>
Co-authored-by: matantsevs <matantsevs@users.noreply.github.com>
Co-authored-by: gitcommit90 <gitcommit90@users.noreply.github.com>
2026-07-20 11:42:29 -04:00

105 lines
3.9 KiB
Python

"""Tests for the interim_assistant_callback config gating in tui_gateway.
These tests exercise the real _agent_cbs() wiring rather than a local
imitation, so a break in the production callback registration is caught.
"""
from __future__ import annotations
from unittest.mock import patch
def test_load_interim_assistant_messages_defaults_true():
from tui_gateway.server import _load_interim_assistant_messages
with patch("tui_gateway.server._load_cfg", return_value={}):
assert _load_interim_assistant_messages() is True
def test_load_interim_assistant_messages_explicit_true():
from tui_gateway.server import _load_interim_assistant_messages
with patch("tui_gateway.server._load_cfg", return_value={"display": {"interim_assistant_messages": True}}):
assert _load_interim_assistant_messages() is True
def test_load_interim_assistant_messages_explicit_false():
from tui_gateway.server import _load_interim_assistant_messages
with patch("tui_gateway.server._load_cfg", return_value={"display": {"interim_assistant_messages": False}}):
assert _load_interim_assistant_messages() is False
def test_load_interim_assistant_messages_string_off():
from tui_gateway.server import _load_interim_assistant_messages
with patch("tui_gateway.server._load_cfg", return_value={"display": {"interim_assistant_messages": "off"}}):
assert _load_interim_assistant_messages() is False
def test_agent_cbs_includes_interim_callback_when_enabled():
"""_agent_cbs() includes interim_assistant_callback when the config is on.
Exercises the real _agent_cbs() wiring: the callback must be present in
the returned dict and, when invoked, must emit a message.interim event
with the text and already_streamed flag passed through.
"""
from tui_gateway.server import _agent_cbs
emitted: list[tuple] = []
def fake_emit(event_type, sid, payload=None):
emitted.append((event_type, sid, payload))
with patch("tui_gateway.server._load_cfg", return_value={}), \
patch("tui_gateway.server._emit", side_effect=fake_emit):
cbs = _agent_cbs("test-session")
assert "interim_assistant_callback" in cbs
cb = cbs["interim_assistant_callback"]
assert callable(cb)
# Invoke the real callback inside the patch context — the lambda
# resolves _emit by name at call time, so it must be called while
# the patch is active.
cb("hello world", already_streamed=True)
assert len(emitted) == 1
assert emitted[0][0] == "message.interim"
assert emitted[0][1] == "test-session"
assert emitted[0][2]["text"] == "hello world"
assert emitted[0][2]["already_streamed"] is True
def test_agent_cbs_omits_interim_callback_when_disabled():
"""_agent_cbs() omits interim_assistant_callback when the config is off.
Exercises the real _agent_cbs() wiring: the callback must NOT be present
in the returned dict when display.interim_assistant_messages is false.
"""
from tui_gateway.server import _agent_cbs
with patch("tui_gateway.server._load_cfg", return_value={"display": {"interim_assistant_messages": False}}):
cbs = _agent_cbs("test-session")
assert "interim_assistant_callback" not in cbs
def test_agent_cbs_interim_callback_passes_already_streamed_false():
"""The real callback passes already_streamed=False by default."""
from tui_gateway.server import _agent_cbs
emitted: list[tuple] = []
def fake_emit(event_type, sid, payload=None):
emitted.append((event_type, sid, payload))
with patch("tui_gateway.server._load_cfg", return_value={}), \
patch("tui_gateway.server._emit", side_effect=fake_emit):
cbs = _agent_cbs("test-session")
cb = cbs["interim_assistant_callback"]
cb("interim text")
assert emitted[0][2]["already_streamed"] is False
assert emitted[0][2]["text"] == "interim text"