test(mcp): stamp breaker-open time on the monotonic clock, not a literal (#69003)

test_session_expired_retry_waits_for_new_session hardcoded
_server_breaker_opened_at["hindsight"] = 123.0 to simulate a circuit breaker
whose cooldown has already elapsed. But the breaker in tools/mcp_tool.py
compares that stamp against time.monotonic() (age = monotonic() - opened_at,
elapsed when age >= _CIRCUIT_BREAKER_COOLDOWN_SEC). time.monotonic()'s origin
is arbitrary and small on a freshly-booted CI container, so age worked out to
only a few seconds there (< the 60s cooldown) — the breaker stayed open, the
half-open probe never fired, and the retry returned the "unreachable" error
instead of "bank ok". It passed on long-uptime dev boxes (large monotonic)
and failed under CI, with the reported "Auto-retry available in ~Ns" drifting
run to run as the container's monotonic clock varied.

Stamp opened_at relative to the same clock the code reads
(time.monotonic() - _CIRCUIT_BREAKER_COOLDOWN_SEC - 1.0) so the cooldown is
provably elapsed regardless of the monotonic origin, exercising the intended
half-open transition deterministically.
This commit is contained in:
brooklyn! 2026-07-21 19:54:40 -05:00 committed by GitHub
parent 8208fc5270
commit 75be8fb463
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -13,6 +13,7 @@ affected MCP server failed until the gateway was manually restarted.
import asyncio
import json
import threading
import time
from unittest.mock import MagicMock
import pytest
@ -507,7 +508,14 @@ def test_session_expired_retry_waits_for_new_session(monkeypatch, tmp_path):
server._reconnect_event = _ReconnectAdapter()
mcp_tool._servers["hindsight"] = server
mcp_tool._server_error_counts["hindsight"] = 7
mcp_tool._server_breaker_opened_at["hindsight"] = 123.0
# Stamp the breaker "open" far enough in the past that the cooldown has
# provably elapsed, so this call is a half-open probe. The breaker compares
# against time.monotonic() (tools/mcp_tool.py), whose origin is arbitrary and
# small on a freshly-booted CI container — a hardcoded literal like 123.0
# only looked "elapsed" on a long-uptime dev box and flaked under CI.
mcp_tool._server_breaker_opened_at["hindsight"] = (
time.monotonic() - mcp_tool._CIRCUIT_BREAKER_COOLDOWN_SEC - 1.0
)
try:
handler = _make_tool_handler("hindsight", "get_bank", 10.0)