hermes-agent/tests/gateway/test_stale_platform_lock_retryable.py
Teknium 39975613b1
test: prune wave 2 + speed fixes — 28,106 → 19,757 test functions, suite wall 315s → 294s
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.

Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
  (DEFAULT_CONFIG smart-approval leaked in) — pinned approval
  mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
  silently probed live provider catalogs (~2s/test) — stubbed
  cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
  shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
  MagicMock agents — interrupt.side_effect now clears _running_agents
  (22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
  (24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
  patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
  5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
  with rationale comment — the watchdog-dump assertion needs the loop
  blocked well past deadline+grace under parallel load (flaked once in
  the 40-worker verification run at a 0.2s margin).

Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
2026-07-29 13:39:40 -07:00

108 lines
3.7 KiB
Python

"""Regression tests for platform-lock acquire behavior.
#54167 — stale platform lock must be retryable.
When a gateway process is killed (SIGKILL, crash) during Telegram
initialization, the scoped lock file survives. On next startup,
``acquire_scoped_lock()`` detects the stale lock and deletes it, but may
still return ``(False, existing_dict)`` to the caller (e.g. if the
unlink fails due to permissions, or a race condition lets another
process grab the lock first).
``_acquire_platform_lock()`` must mark such failures as **retryable**
so the reconnect watcher can retry after a delay — not permanently kill
the platform.
#65176 — a live gateway token conflict may attempt one-shot takeover only
during the initial connect of an explicit ``gateway run --replace`` startup.
``gateway run --replace`` only kills same-HERMES_HOME PID-file holders.
A normal start or reconnect must retain the retryable conflict behavior and
must never evict the active holder.
"""
from typing import Any, Dict
from unittest.mock import MagicMock, patch
import pytest
from gateway.platforms.base import BasePlatformAdapter
from gateway.run import GatewayRunner
class _StubAdapter(BasePlatformAdapter):
"""Minimal concrete subclass for testing _acquire_platform_lock."""
platform = MagicMock(value="telegram")
async def connect(self, *, is_reconnect: bool = False) -> bool:
return True
async def disconnect(self) -> None:
pass
async def send(self, *args: Any, **kwargs: Any) -> None:
pass
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
return {}
@pytest.fixture()
def adapter():
"""Create a stub adapter with __init__ bypassed."""
obj = _StubAdapter.__new__(_StubAdapter)
obj._running = True
obj._fatal_error_code = None
obj._fatal_error_message = None
obj._fatal_error_retryable = True
obj._fatal_error_handler = None
obj._platform_lock_scope = None
obj._platform_lock_identity = None
obj._platform_lock_takeover_allowed = False
obj._platform_lock_takeover_attempted = False
obj._status_write_logged = None
return obj
def test_stale_lock_failure_is_retryable(adapter):
"""Lock failure must be retryable, not permanently fatal (#54167)."""
with patch(
"gateway.status.acquire_scoped_lock",
return_value=(False, {"pid": 99999, "start_time": "2026-01-01T00:00:00Z"}),
), patch.object(adapter, "_write_runtime_status_safe"):
result = adapter._acquire_platform_lock(
"telegram-bot-token", "test-token", "Telegram bot token"
)
assert result is False
assert adapter._fatal_error_retryable is True
assert adapter._fatal_error_code == "telegram-bot-token_lock"
def test_explicit_replace_takeover_reacquires_lock_once(adapter):
"""Initial explicit --replace may hand off and re-acquire once (#65176)."""
existing = {
"pid": 4242,
"kind": "hermes-gateway",
"argv": ["hermes", "gateway", "run"],
"start_time": 123,
}
acquire = MagicMock(side_effect=[(False, existing), (True, None)])
adapter._platform_lock_takeover_allowed = True
with patch("gateway.status.acquire_scoped_lock", acquire), patch(
"gateway.status.take_over_scoped_lock_holder",
return_value=4242,
) as takeover, patch.object(
adapter, "_write_runtime_status_safe"
):
result = adapter._acquire_platform_lock(
"telegram-bot-token", "test-token", "Telegram bot token"
)
assert result is True
assert adapter._platform_lock_takeover_allowed is False
assert adapter._platform_lock_takeover_attempted is True
takeover.assert_called_once_with(existing)
assert acquire.call_count == 2