mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-13 14:02:16 +00:00
fix(error-classifier): route 5xx context-overflow into compression
Local inference servers (llama.cpp/llama-server, vLLM/Ollama behind a Cloudflare/Tailscale hop) report context overflow with HTTP 500/502/503/529 instead of 400/413. _classify_by_status returned server_error/overloaded and retried blindly, then dropped the turn with no compaction. Route explicit _CONTEXT_OVERFLOW_PATTERNS matches on those 5xx codes to context_overflow (should_compress=True); plain 500 stays server_error, plain 503 overloaded.
This commit is contained in:
parent
74809b4e94
commit
a04b7024ff
2 changed files with 67 additions and 0 deletions
|
|
@ -964,9 +964,31 @@ def _classify_by_status(
|
|||
retryable=False,
|
||||
should_fallback=True,
|
||||
)
|
||||
# Some local inference servers (notably llama.cpp / llama-server)
|
||||
# report context overflow with an HTTP 500 instead of the standard
|
||||
# 400/413. The request-validation guard above already ran, so any
|
||||
# remaining explicit context-overflow signal routes into the
|
||||
# compression-and-retry path (mirroring _classify_400) instead of
|
||||
# blind server_error retries that exhaust and drop the turn.
|
||||
if any(p in error_msg for p in _CONTEXT_OVERFLOW_PATTERNS):
|
||||
return result_fn(
|
||||
FailoverReason.context_overflow,
|
||||
retryable=True,
|
||||
should_compress=True,
|
||||
)
|
||||
return result_fn(FailoverReason.server_error, retryable=True)
|
||||
|
||||
if status_code in {503, 529}:
|
||||
# Same overflow-as-5xx variant (server busy / model-load OOM, or a
|
||||
# Cloudflare/Tailscale hop relabeling the status). Route explicit
|
||||
# overflow bodies into compression; otherwise treat as transient
|
||||
# overload and retry.
|
||||
if any(p in error_msg for p in _CONTEXT_OVERFLOW_PATTERNS):
|
||||
return result_fn(
|
||||
FailoverReason.context_overflow,
|
||||
retryable=True,
|
||||
should_compress=True,
|
||||
)
|
||||
return result_fn(FailoverReason.overloaded, retryable=True)
|
||||
|
||||
# Other 4xx — non-retryable
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue