fix(error-classifier): retry HTTP 408 as timeout instead of aborting as format_error

_classify_by_status() routes every other transient HTTP status to a retryable
reason (500/502 -> server_error, 503/529 -> overloaded, 429 -> rate_limit,
413 -> payload_too_large), but 408 Request Timeout fell through to the generic
`400 <= status < 500` branch and was classified as a non-retryable
format_error -- the same bucket as a 400 Bad Request.

A 408 is a transient timing failure the server itself flags as safe to retry
(RFC 9110 15.5.9), not a malformed request, so the retry loop aborted the turn
when a simple retry would recover. Common trigger: a reverse proxy in front of
a self-hosted backend (llama.cpp / Ollama / vLLM) returns 408 when a long
generation outruns the proxy's request-read window.

Route 408 to the existing FailoverReason.timeout (rebuild client + retry).
Add a regression test plus a boundary test asserting 400 stays non-retryable.
This commit is contained in:
Frowtek 2026-07-02 11:09:40 +03:00 committed by Teknium
parent a88e0fd2ab
commit 8457752a3b
2 changed files with 31 additions and 0 deletions

View file

@ -1043,6 +1043,17 @@ def _classify_by_status(
)
return result_fn(FailoverReason.overloaded, retryable=True)
# 408 Request Timeout — a transient timing failure the server itself flags
# as safe to retry (RFC 9110 §15.5.9), not a malformed request. Commonly
# emitted by reverse proxies sitting in front of self-hosted backends
# (llama.cpp / Ollama / vLLM) when a long generation outruns the proxy's
# request-read window. Route to the dedicated ``timeout`` reason (rebuild
# client + retry) instead of falling through to the generic 4xx bucket
# below, which would abort the turn on a retry-safe error the same way it
# aborts a 400 Bad Request.
if status_code == 408:
return result_fn(FailoverReason.timeout, retryable=True)
# Other 4xx — non-retryable
if 400 <= status_code < 500:
return result_fn(

View file

@ -376,6 +376,26 @@ class TestClassifyApiError:
result = classify_api_error(e)
assert result.reason == FailoverReason.overloaded
def test_408_request_timeout_is_retryable_timeout(self):
"""HTTP 408 Request Timeout is a transient timing failure the server
itself flags as safe to retry (RFC 9110 §15.5.9) commonly emitted by
reverse proxies in front of self-hosted backends (llama.cpp / Ollama /
vLLM) when a long generation outruns the proxy's request-read window.
It must NOT fall into the generic 4xx bucket as a non-retryable
format_error, which would abort the turn on a retry-safe error."""
e = MockAPIError("Request Timeout", status_code=408)
result = classify_api_error(e, provider="vllm")
assert result.reason == FailoverReason.timeout
assert result.retryable is True
def test_400_bad_request_still_non_retryable_format_error(self):
"""Guard the boundary: a genuine 400 Bad Request must remain a
non-retryable format_error and must not be swept up by the 408 branch."""
e = MockAPIError("Bad Request", status_code=400)
result = classify_api_error(e)
assert result.reason == FailoverReason.format_error
assert result.retryable is False
def test_message_only_overloaded_without_status_is_overloaded(self):
"""Some Anthropic-compatible proxies surface 'overloaded' in the
message with no 503/529 status_code. It must classify as overloaded