Commit graph

3 commits

Author SHA1 Message Date
Burke Autrey
ff56251555 fix(streaming): Bedrock liveness watchdog (into #58962 breaker) + finite local timeout
Re-applied against v0.18.2. Upstream now covers the OpenAI-path stall watchdog
(cross-turn give-up breaker #58962) and the tool-batch deadline, so those are
dropped. Two gaps remain:

- Bedrock streaming had NO liveness watchdog and was excluded from the #58962
  breaker (it returns before _check_stale_giveup). Add an on_event hook to
  stream_converse_with_callbacks (fires per yielded event = true wire-level
  liveness), drive a stale timer from it, and wire Bedrock INTO the existing
  breaker: entry _check_stale_giveup, _bump_stale_streak on stall, raise to end
  the call (invalidate_runtime_client can't abort the in-flight botocore stream,
  so the streak escalates across turns like the OpenAI path), and reset the
  streak on success.
- local providers still got float('inf') stale timeout (watchdog disabled) — give
  a finite ceiling (HERMES_LOCAL_STREAM_STALE_TIMEOUT, default 900s).
- tests: on_event per-event + swallow; Bedrock stall bumps streak + aborts;
  pre-elevated streak aborts at entry; success resets streak.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 19:38:21 +05:30
HexLab98
35cbffd5c8 test(streaming): cover the single-writer invariant for superseded streams
Assert that a superseded stream (older writer token, other thread) is fenced
from the delta sink, the active writer is never fenced, a non-claiming thread
is never treated as a writer, and the real consume loop stops the instant it is
superseded — so two streams can never interleave into one turn (#65991).
2026-07-17 06:49:23 -07:00
kshitijk4poor
144457d801 fix(interrupt): extend post-worker /stop guard to Bedrock streaming path
The salvaged fix added a post-worker _interrupt_requested re-check to the
main OpenAI/Anthropic streaming poll loop. The Bedrock Converse poll loop
(interruptible_streaming_api_call, api_mode='bedrock_converse') has the same
bug class: its worker calls stream_converse_with_callbacks(on_interrupt_check=
...), which breaks out of the event loop on interrupt and returns a PARTIAL
response WITHOUT raising (bedrock_adapter.py). The worker sets result[
'response'] and exits with _interrupt_requested still True, so the in-loop
raise never fires and the poll loop returns the partial — silently swallowing
/stop on Bedrock exactly as it was on the paths the salvaged commit fixed.

Add the identical post-worker re-check before the Bedrock loop's return.
The non-streaming loop (interruptible_api_call) is structurally immune: its
worker's only early return fires off _request_cancelled, which is set by the
main loop immediately before it raises in-loop, so no swallow window exists.

Guard test flips _interrupt_requested True mid-stream (after the pre-flight
check) and asserts InterruptedError is raised; verified RED without the fix
(DID NOT RAISE) and GREEN with it.
2026-07-07 14:54:50 +05:30