fix(errors): classify throttle messages before token-overflow patterns; add new overflow shapes

Port from anomalyco/opencode#37848 (+ dev-branch twin #37840): expand
context-overflow patterns and guard against rate-limit messages that
mention tokens.

- 'Throttling error: Too many tokens, please wait before trying again.'
  (AWS Bedrock / proxy shape) classified as context_overflow and routed a
  healthy session into compression on every throttle. Added 'throttling'
  to _RATE_LIMIT_PATTERNS, which the message-only path checks BEFORE the
  overflow list.
- 'Input length N exceeds the maximum allowed input length of M tokens.'
  (Together/Fireworks shape) fell through to unknown — no compression
  recovery. Added 'maximum allowed input length' to overflow patterns.
- 'request_too_large' / 'Request exceeds the maximum size' (Anthropic 413
  type re-wrapped without a status code by aggregators/proxies) fell
  through to unknown. Added to _PAYLOAD_TOO_LARGE_PATTERNS.

All three shapes proven live on main before the fix; 265 classifier +
bedrock tests and 238 sibling rate-guard/compression tests pass.
This commit is contained in:
Teknium 2026-07-23 17:10:32 -07:00
parent c4d1913294
commit 53bfe40a35
2 changed files with 90 additions and 0 deletions

View file

@ -159,6 +159,14 @@ _RATE_LIMIT_PATTERNS = [
"throttlingexception",
"too many concurrent requests",
"servicequotaexceededexception",
# Generic throttle prefix — Bedrock (and some proxies) surface throttling
# as "Throttling error: Too many tokens, please wait before trying
# again." Without this entry the message falls through to the
# context-overflow list (which contains "too many tokens") and the retry
# loop compresses a healthy session instead of backing off. Matched
# BEFORE _CONTEXT_OVERFLOW_PATTERNS in the message-only path, so the
# throttle wins. (port of anomalyco/opencode#37848's exclusion guard)
"throttling",
]
# Patterns that indicate provider-side overload, NOT a per-credential rate
@ -212,6 +220,12 @@ _PAYLOAD_TOO_LARGE_PATTERNS = [
"request entity too large",
"payload too large",
"error code: 413",
# Anthropic's structured 413 error type. Normally arrives with an HTTP
# 413 status (handled by the status path), but aggregators/proxies can
# re-wrap it into a plain message with no status attribute — route it to
# the same compression recovery. (port of anomalyco/opencode#37848)
"request_too_large",
"request exceeds the maximum size",
]
# Image-size patterns. Matched against 400 bodies (not 413) because most
@ -298,6 +312,10 @@ _CONTEXT_OVERFLOW_PATTERNS = [
"max input token",
"input token",
"exceeds the maximum number of input tokens",
# Together/Fireworks-style: "Input length 131393 exceeds the maximum
# allowed input length of 131040 tokens." No other pattern in this list
# matches that wording. (port of anomalyco/opencode#37848)
"maximum allowed input length",
]
# Model not found patterns