Commit graph

3 commits

Author SHA1 Message Date
Jaret Bottoms
e16743b0d5 fix(telegram): diagnose blocked-loop init hangs, unbind DoH from system DNS
The #63309 hang class — gateway stuck at 'Connecting to Telegram
(attempt 1/8)' with no retry, no timeout, for minutes — can only occur
when the event loop thread itself is blocked in a synchronous call:
_await_with_thread_deadline's timer fires off-loop, but its expiry
hand-off (call_soon_threadsafe) still needs the loop to run, and the
gateway's outer wait_for is a pure loop timer. When the loop is pinned,
every layer goes silent simultaneously and the process wedges with no
evidence of where.

Two changes:

1. Loop-blocked watchdog in _await_with_thread_deadline: a second
   daemon timer fires one grace period (5s) after the deadline; if the
   loop still hasn't processed the expiry, it logs a WARNING from the
   timer thread and faulthandler-dumps all thread stacks to stderr —
   converting the silent hang into a trace that names the exact
   blocking frame. A threading.Event set by the expiry callback (and on
   normal exit) keeps completed awaits from ever being misreported.

2. discover_fallback_ips: the system-resolver leg runs
   socket.getaddrinfo in a worker thread with no timeout, and
   asyncio.gather waited on it unboundedly — a wedged OS resolver
   stalled discovery for minutes between the two startup log lines. Its
   result only feeds a log message, so it no longer gates discovery:
   DoH legs (already client-bounded) are gathered alone and the system
   leg is awaited with a _DOH_TIMEOUT cap, best-effort.

Refs #63309

Tests: 3 watchdog regressions (blocked-loop dump fires; responsive-loop
timeout does not; completed await does not) + 2 hung-resolver
regressions (DoH results returned promptly; worst-case seed fallback
stays bounded).
2026-07-14 17:09:50 +05:30
kshitijk4poor
a37fd66dec fix(telegram): shut down abandoned init app + AUTHOR_MAP + cover the deadline helper
Follow-up to @msh01's wall-deadline init-timeout fix.

- Resource leak: on timeout the initialize() task is abandoned without
  awaiting its (shielded, possibly-never-completing) cancellation, so the
  half-built PTB app's httpx client / connection pool was never closed —
  up to 8x across the retry ladder. Add an optional on_abandon cleanup to
  _await_with_thread_deadline that best-effort app.shutdown()s the abandoned
  app, run detached + exception-swallowed so it can never re-block or re-hang
  the ladder (mirrors _close_client_on_timeout in agent/auxiliary_client.py).
- Cover the helper itself: the salvaged test monkeypatched out the real
  _await_with_thread_deadline, so its abandonment/cleanup path was untested.
  Add direct tests for happy-path return, prompt-timeout-with-cleanup, and
  cleanup-error-swallowed; the wedged coroutines swallow cancellation for a
  bounded window (proving the helper returns before cancellation completes,
  the #58236 shielded-scope behavior) without leaving an immortal task that
  would wedge pytest teardown. Widen the salvaged stub to accept on_abandon.
- Attribution: add yingwaizhiying@gmail.com -> msh01 to AUTHOR_MAP (bare
  gmail does not auto-resolve the check-attribution gate).

Known follow-up (not addressed here): the retry ladder reuses the same
self._app across all 8 attempts; a fresh app per attempt would fully close
the coherence risk if an abandoned initialize() completes in the background.
That is a larger restructure of the ~130-line builder+handler setup, left
for a separate change.
2026-07-04 20:45:48 +05:30
yoma
d50aae0e35 fix(telegram): use wall deadline for init timeout 2026-07-04 19:06:44 +05:30