mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-30 06:41:51 +00:00
* ci(tests): install ripgrep from prebuilt tarball instead of apt
apt-get update + install of ripgrep takes ~4 min on the GHA Ubuntu
runners (the apt-get update against archive.ubuntu.com is the slow
part; ripgrep itself is small). Switching to the upstream musl
binary tarball cuts the step to a few seconds.
- Pinned to ripgrep 15.1.0 with sha256 verification (same hash as
published in the releases sha256 sidecar file).
- Drops the `rg` binary into /usr/local/bin so it is on PATH for
every subsequent step without GITHUB_PATH manipulation.
- Applied to both the test and e2e jobs in tests.yml.
* fix(cli): compile syntax check to tempdir, not source __pycache__
`_validate_critical_files_syntax` runs `py_compile.compile()` on each
critical bootstrap file after a successful `git pull`. The default
`py_compile` writes the resulting `.pyc` next to the source under
`__pycache__/`, which causes two real problems:
1. Parallel test workers walking the same source tree (e.g. running
the suite under per-file process isolation) can race against each
other on the `__pycache__` write — manifests as flaky 'directory
not empty' errors during teardown.
2. In production, the post-pull syntax check leaves a `.pyc` behind
that the next interpreter run might pick up — fine when the
interpreter version matches, sketchy if it doesn't.
Fix: write the compiled output to a `tempfile.TemporaryDirectory()`
that's discarded on function exit. We only care about the compile-or-not
signal, not the artifact.
* test(runner): per-file process isolation, drop manual state reset + xdist
Replace fragile manual _reset_module_state test fixtures with robust
per-file subprocess isolation. Each test file runs in a fresh
`python -m pytest <file>` subprocess via ThreadPoolExecutor. No xdist,
no custom pytest plugin, no shared worker state.
Key changes:
* scripts/run_tests_parallel.py — new runner: discovers test files,
runs N in parallel via ThreadPoolExecutor, captures stdout per file,
treats exit code 5 (no tests collected) as pass, kills all children
on exit. Change from cpu_count to cpu_count*2. The runner is
I/O-bound (waiting on subprocess.communicate() from pytest children)
The parent process does almost no CPU work, so 2x oversubscription
keeps more pipes full. When a file fails, immediately show the last
30 lines of pytest output (stack traces + FAILED summary) plus a
ready-to-copy repro command:
python -m pytest tests/agent/test_auxiliary_client.py
* scripts/run_tests.sh — delegates to run_tests_parallel.py
* .github/workflows/tests.yml — test step: python
scripts/run_tests_parallel.py
* pyproject.toml — drop pytest-xdist, pytest-split; simplify addopts
* tests/conftest.py — remove ~200 lines of manual state-reset fixtures
* AGENTS.md — update Testing section for per-file design
* test(runner): speed gateway test antipattern scan up
* fix(test): web search provider plugin test missing xai
* fix(tests): make 14 test files pass under per-file subprocess isolation
Tests that relied on cross-file state pollution from xdist workers
fail when run in isolation (per-file subprocess model). Root causes
and fixes:
Tool registry not populated:
- test_video_generation_tool_surface_matrix: add discover_builtin_tools()
- test_web_providers_brave_free/ddgs/searxng/general: autouse fixtures
registering all 8 bundled web providers, reset after each test
- test_website_policy: same provider registration pattern
- test_web_tools_tavily: same pattern across 3 dispatch test classes
- Also add is_safe_url/check_website_access mocks where SSRF check
blocks example.com (DNS resolution fails in isolated envs)
Stale check_fn cache:
- test_kanban_tools: invalidate_check_fn_cache() + _clear_tool_defs_cache()
in both kanban guidance tests (prior test cached False for kanban_show)
- test_discord_tool: cache invalidation in setup/teardown
- test_homeassistant_tool: invalidate_check_fn_cache() before registry queries
Module-level state pollution:
- test_auxiliary_client: autouse fixture clearing _aux_unhealthy_until cache
- test_skill_commands: set_session_vars() instead of patch.dict(os.environ)
(ContextVar takes precedence over os.environ)
- test_dm_topics: overwrite sys.modules + separate telegram.constants mock
+ force-reimport of gateway.platforms.telegram
- test_terminal_tool_requirements: removed duplicate class declaration,
autouse _clear_caches fixture
* change(tests): run_tests.sh explicitly includes env vars
instead of manually dropping some vars, now we just only include some
* fix(tests): 5 more isolation/NixOS fixes
- test_approval_plugin_hooks: isolate HERMES_HOME so real user's
command_allowlist doesn't short-circuit the approval path
- test_google_chat: skipif when Platform.GOOGLE_CHAT not in enum
(feature not merged on this branch)
- test_write_deny: test systemd prefix against tmp_path instead of
/etc/systemd which resolves to /nix/store on NixOS
- test_pty_bridge: use shutil.which('cat') instead of /bin/cat
(doesn't exist on NixOS)
- profiles.py: rmtree onexc handler chmod's parent dirs too, fixing
profile deletion when copytree preserved read-only modes from
nix store
* fix(tests): clear unhealthy cache in autouse fixture for auxiliary_client
* fix(tests): skip send_message when telegram not installed; handle missing worker_id in browser_supervisor
* fix: py3.11 rmtree onexc compat + belt-and-suspenders unhealthy cache clear for expired codex test
* fix: address PR #29016 review feedback
- Remove tracked .pytest-cache/ artifact and add to .gitignore
- Fix stale 'xdist worker' comment in conftest.py
- Deduplicate web provider registration into tests/tools/conftest.py
shared helper (register_all_web_providers), replacing 8 copy-pasted
blocks across 6 test files
- Update PR description: remove stale recovered-test-files claim,
fix worker count to match code (cpu_count*2)
* fix: eliminate race in stale-cache achievements test
The background scan thread could complete and overwrite _SNAPSHOT_CACHE
before evaluate_all() returned the stale data — only 10 fake sessions
made the scan finish instantly. Added scan_delay param to _FakeSessionDB
and set it to 2s in the stale-cache test so the background thread can't
win the race.
180 lines
5.8 KiB
Python
180 lines
5.8 KiB
Python
"""Unit tests for hermes_cli.pty_bridge — PTY spawning + byte forwarding.
|
|
|
|
These tests drive the bridge with minimal POSIX processes (echo, env, sleep,
|
|
printf) to verify it behaves like a PTY you can read/write/resize/close.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
import shutil
|
|
import sys
|
|
import time
|
|
|
|
import pytest
|
|
|
|
pytest.importorskip("ptyprocess", reason="ptyprocess not installed")
|
|
|
|
from hermes_cli.pty_bridge import PtyBridge, PtyUnavailableError
|
|
|
|
|
|
skip_on_windows = pytest.mark.skipif(
|
|
sys.platform.startswith("win"), reason="PTY bridge is POSIX-only"
|
|
)
|
|
|
|
|
|
def _read_until(bridge: PtyBridge, needle: bytes, timeout: float = 5.0) -> bytes:
|
|
"""Accumulate PTY output until we see `needle` or time out."""
|
|
deadline = time.monotonic() + timeout
|
|
buf = bytearray()
|
|
while time.monotonic() < deadline:
|
|
chunk = bridge.read(timeout=0.2)
|
|
if chunk is None:
|
|
break
|
|
buf.extend(chunk)
|
|
if needle in buf:
|
|
return bytes(buf)
|
|
return bytes(buf)
|
|
|
|
|
|
@skip_on_windows
|
|
class TestPtyBridgeSpawn:
|
|
def test_is_available_on_posix(self):
|
|
assert PtyBridge.is_available() is True
|
|
|
|
def test_spawn_returns_bridge_with_pid(self):
|
|
bridge = PtyBridge.spawn(["true"])
|
|
try:
|
|
assert bridge.pid > 0
|
|
finally:
|
|
bridge.close()
|
|
|
|
def test_spawn_raises_on_missing_argv0(self, tmp_path):
|
|
with pytest.raises((FileNotFoundError, OSError)):
|
|
PtyBridge.spawn([str(tmp_path / "definitely-not-a-real-binary")])
|
|
|
|
|
|
@skip_on_windows
|
|
class TestPtyBridgeIO:
|
|
def test_reads_child_stdout(self):
|
|
bridge = PtyBridge.spawn(["/bin/sh", "-c", "printf hermes-ok"])
|
|
try:
|
|
output = _read_until(bridge, b"hermes-ok")
|
|
assert b"hermes-ok" in output
|
|
finally:
|
|
bridge.close()
|
|
|
|
def test_write_sends_to_child_stdin(self):
|
|
# `cat` with no args echoes stdin back to stdout. We write a line,
|
|
# read it back, then signal EOF to let cat exit cleanly.
|
|
bridge = PtyBridge.spawn([shutil.which("cat") or "cat"])
|
|
try:
|
|
bridge.write(b"hello-pty\n")
|
|
output = _read_until(bridge, b"hello-pty")
|
|
assert b"hello-pty" in output
|
|
finally:
|
|
bridge.close()
|
|
|
|
def test_read_returns_none_after_child_exits(self):
|
|
bridge = PtyBridge.spawn(["/bin/sh", "-c", "printf done"])
|
|
try:
|
|
_read_until(bridge, b"done")
|
|
# Give the child a beat to exit cleanly, then drain until EOF.
|
|
deadline = time.monotonic() + 3.0
|
|
while bridge.is_alive() and time.monotonic() < deadline:
|
|
bridge.read(timeout=0.1)
|
|
# Next reads after exit should return None (EOF), not raise.
|
|
got_none = False
|
|
for _ in range(10):
|
|
if bridge.read(timeout=0.1) is None:
|
|
got_none = True
|
|
break
|
|
assert got_none, "PtyBridge.read did not return None after child EOF"
|
|
finally:
|
|
bridge.close()
|
|
|
|
|
|
@skip_on_windows
|
|
class TestPtyBridgeResize:
|
|
def test_resize_updates_child_winsize(self):
|
|
# Query the TTY ioctl directly instead of using tput, which requires
|
|
# TERM and fails in GitHub Actions' non-interactive environment.
|
|
winsize_script = (
|
|
"import fcntl, struct, termios, time; "
|
|
"time.sleep(0.1); "
|
|
"rows, cols, *_ = struct.unpack('HHHH', "
|
|
"fcntl.ioctl(0, termios.TIOCGWINSZ, b'\\0' * 8)); "
|
|
"print(cols); print(rows)"
|
|
)
|
|
bridge = PtyBridge.spawn(
|
|
[sys.executable, "-c", winsize_script],
|
|
cols=80,
|
|
rows=24,
|
|
)
|
|
try:
|
|
bridge.resize(cols=123, rows=45)
|
|
output = _read_until(bridge, b"45", timeout=5.0)
|
|
# tput prints just the numbers, one per line
|
|
assert b"123" in output
|
|
assert b"45" in output
|
|
finally:
|
|
bridge.close()
|
|
|
|
|
|
@skip_on_windows
|
|
class TestPtyBridgeClose:
|
|
def test_close_is_idempotent(self):
|
|
bridge = PtyBridge.spawn(["/bin/sh", "-c", "sleep 30"])
|
|
bridge.close()
|
|
bridge.close() # must not raise
|
|
assert not bridge.is_alive()
|
|
|
|
def test_close_terminates_long_running_child(self):
|
|
bridge = PtyBridge.spawn(["/bin/sh", "-c", "sleep 30"])
|
|
pid = bridge.pid
|
|
bridge.close()
|
|
# Give the kernel a moment to reap
|
|
deadline = time.monotonic() + 3.0
|
|
reaped = False
|
|
while time.monotonic() < deadline:
|
|
try:
|
|
os.kill(pid, 0)
|
|
time.sleep(0.05)
|
|
except ProcessLookupError:
|
|
reaped = True
|
|
break
|
|
assert reaped, f"pid {pid} still running after close()"
|
|
|
|
|
|
@skip_on_windows
|
|
class TestPtyBridgeEnv:
|
|
def test_cwd_is_respected(self, tmp_path):
|
|
bridge = PtyBridge.spawn(
|
|
["/bin/sh", "-c", "pwd"],
|
|
cwd=str(tmp_path),
|
|
)
|
|
try:
|
|
output = _read_until(bridge, str(tmp_path).encode())
|
|
assert str(tmp_path).encode() in output
|
|
finally:
|
|
bridge.close()
|
|
|
|
def test_env_is_forwarded(self):
|
|
bridge = PtyBridge.spawn(
|
|
["/bin/sh", "-c", "printf %s \"$HERMES_PTY_TEST\""],
|
|
env={**os.environ, "HERMES_PTY_TEST": "pty-env-works"},
|
|
)
|
|
try:
|
|
output = _read_until(bridge, b"pty-env-works")
|
|
assert b"pty-env-works" in output
|
|
finally:
|
|
bridge.close()
|
|
|
|
|
|
class TestPtyBridgeUnavailable:
|
|
"""Platform fallback semantics — PtyUnavailableError is importable and
|
|
carries a user-readable message."""
|
|
|
|
def test_error_carries_user_message(self):
|
|
err = PtyUnavailableError("platform not supported")
|
|
assert "platform" in str(err)
|