test(cli): fix order-dependent test_resume_quiet_stderr flake at the source

test_session_not_found_goes_to_stdout_in_full_mode passes in isolation but
fails in a full tests/cli run. Two independent leaks from the same neighbor
test conspire:

1. test_cli_init.py's _make_cli() reloads cli.py while prompt_toolkit is
   stubbed with MagicMocks and never reloads it back, so sys.modules['cli']
   is left with a mock _pt_print/_PT_ANSI and cli._cprint silently no-ops
   for every later test. Fixed by reloading cli once more with the real
   modules visible (try/finally).

2. prompt_toolkit's print_formatted_text caches its Output on the
   process-global default AppSession the first time it renders without an
   explicit output=. Under capsys (which swaps sys.stdout per test), the
   first CLI test to emit through _cprint locks that cache onto its own
   captured stdout, so later capsys tests read an empty buffer. Fixed with
   an autouse fixture in a new tests/cli/conftest.py that resets the cached
   output around each test.

Neither change touches production code or the flaky test's own assertion.

Related to #59358 (which addresses the same flaky test by mocking _cprint in
the assertion instead; this fixes the two underlying leaks at the source and
does not modify test_resume_quiet_stderr.py).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
mehmetkr-31 2026-07-21 22:13:24 +03:00 committed by Teknium
parent 7ac63975f7
commit dda289933d
3 changed files with 73 additions and 7 deletions

View file

@ -0,0 +1 @@
mehmetkr-31

50
tests/cli/conftest.py Normal file
View file

@ -0,0 +1,50 @@
"""Shared fixtures for CLI tests.
prompt_toolkit / capsys isolation
---------------------------------
``cli._cprint`` renders through ``prompt_toolkit.print_formatted_text``,
which when called with no explicit ``output=`` lazily creates an
``Output`` from ``sys.stdout`` **and caches it on the process-global default
``AppSession``** (``prompt_toolkit.application.current._current_app_session``,
a ``ContextVar`` with a module-level default). The cache is keyed to nothing
and never re-reads ``sys.stdout``.
Under pytest, ``capsys`` swaps ``sys.stdout`` for a fresh buffer per test.
So the first CLI test that emits through ``_cprint`` (e.g. one exercising
``/queue``, which prints a "Queued: …" line) locks prompt_toolkit's cached
output onto *its* captured stdout. Every later ``capsys`` test that asserts
on ``_cprint`` output then reads an empty buffer, because the render went to
the first test's now-dead capture target. That is the mechanism behind the
order-dependent ``test_resume_quiet_stderr`` failure: it passes in isolation
and in its own file, but fails in a full ``tests/cli`` run.
Reset the cached output before every CLI test so each one re-creates a fresh
prompt_toolkit ``Output`` bound to its own ``sys.stdout`` on first use. This
is a no-op when prompt_toolkit isn't importable and cheap otherwise (the
property re-creates lazily).
"""
import pytest
@pytest.fixture(autouse=True)
def _reset_prompt_toolkit_output_cache():
"""Clear prompt_toolkit's cached AppSession output around each CLI test.
See the module docstring for the capsys/prompt_toolkit interaction this
guards against.
"""
def _clear() -> None:
try:
from prompt_toolkit.application.current import get_app_session
get_app_session()._output = None
except Exception:
# prompt_toolkit not importable / internal shape changed — the
# tests that rely on this simply keep their prior behavior.
pass
_clear()
yield
_clear()

View file

@ -44,13 +44,28 @@ def _make_cli(env_overrides=None, config_overrides=None, **kwargs):
"prompt_toolkit.formatted_text": MagicMock(),
"prompt_toolkit.auto_suggest": MagicMock(),
}
with patch.dict(sys.modules, prompt_toolkit_stubs), \
patch.dict("os.environ", clean_env, clear=False):
import cli as _cli_mod
_cli_mod = importlib.reload(_cli_mod)
with patch.object(_cli_mod, "get_tool_definitions", return_value=[]), \
patch.dict(_cli_mod.__dict__, {"CLI_CONFIG": _clean_config}):
return _cli_mod.HermesCLI(**kwargs)
try:
with patch.dict(sys.modules, prompt_toolkit_stubs), \
patch.dict("os.environ", clean_env, clear=False):
import cli as _cli_mod
_cli_mod = importlib.reload(_cli_mod)
with patch.object(_cli_mod, "get_tool_definitions", return_value=[]), \
patch.dict(_cli_mod.__dict__, {"CLI_CONFIG": _clean_config}):
return _cli_mod.HermesCLI(**kwargs)
finally:
# The reload above re-executed cli.py while prompt_toolkit was stubbed
# with MagicMocks, permanently rebinding cli's module globals
# (``_pt_print``, ``_PT_ANSI``, …) to those mocks. ``patch.dict``
# restores ``sys.modules`` on exit, but NOT the names the reloaded
# module already bound — so ``sys.modules["cli"]`` is left with a
# mock ``_pt_print``, and ``cli._cprint`` then silently no-ops for
# every later test (one half of the order-dependent
# ``test_resume_quiet_stderr`` full-suite failure; the other half is
# the prompt_toolkit output cache reset in this dir's conftest).
# Reload once more with the real modules visible so cli's globals
# rebind cleanly.
import cli as _cli_restore
importlib.reload(_cli_restore)
class TestMaxTurnsResolution: