fix(xai): drop stale X Premium+ hint from entitlement 403 surfacing (#27110)

xAI announced on 2026-05-16 (https://x.ai/news/grok-hermes) that X Premium
subscriptions now work in Hermes Agent. The hint we shipped in PR #26644
asserted the opposite ("X Premium+ does NOT include xAI API access — only
standalone SuperGrok subscribers can use this provider"), which would now
misdirect Premium+ users who hit any other 403 (no Grok sub at all, wrong
tier, exhausted quota) into thinking they need to switch subscriptions
when their sub is in fact valid.

Remove _decorate_xai_entitlement_error and its two call sites in
_summarize_api_error. xAI's own body text already says "Manage subscriptions
at https://grok.com/?_s=usage" — surface that verbatim and let xAI's wording
do the diagnosis.

The _is_entitlement_failure guard (which prevents credential-pool refresh
loops on entitlement 403s) and the reasoning-replay gating for xai-oauth
are unrelated and untouched.

Update tests to assert the body still surfaces verbatim and that no
Hermes-side editorializing is appended.
This commit is contained in:
Teknium 2026-05-16 16:00:01 -07:00 committed by GitHub
parent fb05f5d4b5
commit dffb602f37
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 24 additions and 118 deletions

View file

@ -158,19 +158,22 @@ def test_codex_stream_postlude_error_still_falls_back():
# ---------------------------------------------------------------------------
# Fix B: friendly entitlement message
# Fix B: surface xAI's entitlement body verbatim (no editorializing)
#
# The original PR #26644 appended a hint that led with "X Premium+ does NOT
# include xAI API access — only standalone SuperGrok subscribers can use this
# provider." xAI announced on 2026-05-16 that X Premium subs now work in
# Hermes (https://x.ai/news/grok-hermes), making that hint actively wrong:
# a Premium+ user hitting a real entitlement issue (no Grok sub, wrong tier,
# exhausted quota) would be misdirected to switch subscriptions when their
# Premium sub is in fact valid. We now surface xAI's own body text verbatim
# (which already says "Manage subscriptions at https://grok.com/?_s=usage")
# and leave the diagnosis to xAI's wording.
# ---------------------------------------------------------------------------
def test_summarize_api_error_decorates_xai_entitlement_403():
"""xAI's OAuth 403 must surface the X Premium+ gotcha + neutral causes.
Wording deliberately leads with the X Premium+ gotcha because that's
the #1 confusing case: people see Grok in their X app, assume it
works here too, and hit this 403 with no idea API access is a
separate SKU. Other causes (no subscription, wrong tier, exhausted
quota) follow.
"""
def test_summarize_api_error_surfaces_xai_entitlement_body_verbatim():
"""xAI's OAuth 403 body must surface as-is, with no Hermes-side hint."""
from run_agent import AIAgent
error = RuntimeError(
@ -180,45 +183,15 @@ def test_summarize_api_error_decorates_xai_entitlement_403():
"subscriptions at https://grok.com'}"
)
summary = AIAgent._summarize_api_error(error)
# The original xAI text must survive — it's still useful diagnostic info.
# xAI's own body text must reach the user — they need it to diagnose.
assert "do not have an active Grok subscription" in summary
# The hint MUST lead with the X Premium+ gotcha (most likely cause
# for users who think they're subscribed).
assert "X Premium+ does NOT include" in summary
assert "standalone SuperGrok subscribers" in summary
# Other causes still listed.
assert "no Grok subscription" in summary
assert "tier doesn't include this model" in summary
assert "quota is exhausted" in summary
# The hint must point at the usage page where the user can verify.
assert "https://grok.com/?_s=usage" in summary
# Switching providers is still a valid escape hatch.
assert "/model" in summary
# No stale claim that X Premium is incompatible with Hermes.
assert "X Premium+ does NOT include" not in summary
assert "standalone SuperGrok subscribers" not in summary
def test_summarize_api_error_does_not_accuse_subscribers():
"""Hint must not confidently say the user has no subscription.
Don Piedro reported his subscription is active. The hint must not
contradict him leading with the X Premium+ gotcha gives subscribers
a plausible reason ("oh, I'm on Premium+ not pure SuperGrok") instead
of accusing them of lying about having a subscription.
"""
from run_agent import AIAgent
error = RuntimeError(
"HTTP 403: do not have an active Grok subscription"
)
summary = AIAgent._summarize_api_error(error)
# MUST NOT contain language that flatly assumes the user is unsubscribed.
assert "lacks SuperGrok" not in summary
assert "you are not subscribed" not in summary.lower()
# MUST lead with the most-likely-but-non-accusatory cause.
assert "X Premium+ does NOT include" in summary
def test_summarize_api_error_decorates_xai_body_message():
"""SDK-style error with structured body must also get the hint."""
def test_summarize_api_error_xai_body_message_unwrapped():
"""SDK-style error with structured body surfaces the message cleanly."""
from run_agent import AIAgent
class _XaiErr(Exception):
@ -235,19 +208,9 @@ def test_summarize_api_error_decorates_xai_body_message():
summary = AIAgent._summarize_api_error(_XaiErr("403"))
assert "HTTP 403" in summary
assert "X Premium+ does NOT include" in summary
def test_summarize_api_error_idempotent_for_entitlement_hint():
"""Decorating twice must not double up the hint."""
from run_agent import AIAgent
raw = "HTTP 403: do not have an active Grok subscription"
once = AIAgent._decorate_xai_entitlement_error(raw)
twice = AIAgent._decorate_xai_entitlement_error(once)
assert once == twice
# Sanity: the hint did fire on the first pass.
assert "X Premium+ does NOT include" in once
assert "do not have an active Grok subscription" in summary
# No editorializing on top of xAI's own wording.
assert "X Premium+ does NOT include" not in summary
def test_summarize_api_error_passes_through_unrelated_errors():