fix(xai-oauth): lead entitlement-403 hint with X Premium+ gotcha (#26672)

The #1 confusing cause of the xAI 403 (per Teknium): X Premium+
subscribers see Grok inside the X app and assume API access is
included.  It is NOT — only standalone SuperGrok subscribers can use
xai-oauth with Hermes today.  Without calling this out, every Premium+
user hits the 403 with no idea why.

PR #26666's neutral 4-cause list was correct but buried the most
common cause.  Lead with the Premium+ gotcha, then list the other
possibilities (no subscription, wrong tier, exhausted quota) as
fallbacks.  Same neutral framing — does not accuse anyone of being
unsubscribed.
This commit is contained in:
Teknium 2026-05-15 17:23:33 -07:00 committed by GitHub
parent 9818b9a1ac
commit 6784c80794
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 44 additions and 38 deletions

View file

@ -5018,20 +5018,21 @@ class AIAgent:
Manage subscriptions at https://grok.com/?_s=usage or subscribe Manage subscriptions at https://grok.com/?_s=usage or subscribe
at https://grok.com/supergrok"} at https://grok.com/supergrok"}
That body covers at least four real causes we cannot distinguish That body covers several real causes we cannot distinguish without
without more info from xAI: more info from xAI. The most common (and least obvious) one is
that **X Premium+ does NOT include API access** only standalone
SuperGrok subscribers can use Hermes against xai-oauth. Lots of
users see Grok in their X app, assume it works here too, and hit
this 403 with no idea why. Lead the hint with that.
* Account has no Grok subscription at all Other possible causes:
* Account has SuperGrok but the tier doesn't include the requested * No Grok subscription at all
model (e.g. grok-4.3 needs SuperGrok Heavy) * SuperGrok tier doesn't include the requested model (e.g.
* Monthly quota for the subscribed tier is exhausted (the grok-4.3 may need a higher tier)
``?_s=usage`` URL hints at this) * Monthly quota exhausted (the ``?_s=usage`` URL hints at this)
* SuperGrok is active but the API access add-on isn't enabled
Picking one ("you're not subscribed") is wrong for the other three Surface the raw xAI text verbatim and point at
and reads as insulting to subscribers. Surface the raw xAI text https://grok.com/?_s=usage where the user can see WHICH applies.
verbatim and point at https://grok.com/?_s=usage where the user
can see WHICH of those four it is.
Matched once per detail string won't double-decorate if the Matched once per detail string won't double-decorate if the
upstream already concatenated the same text. upstream already concatenated the same text.
@ -5047,15 +5048,16 @@ class AIAgent:
if not is_entitlement: if not is_entitlement:
return detail return detail
hint = ( hint = (
" — xAI rejected the request on this OAuth account. Could be a " " — xAI rejected this OAuth account. NOTE: X Premium+ does NOT "
"missing subscription, a tier that doesn't include this model, an " "include xAI API access — only standalone SuperGrok subscribers "
"exhausted quota, or API access not enabled. Check " "can use this provider. Other possible causes: no Grok "
"https://grok.com/?_s=usage to see which, or run `/model` to " "subscription, your tier doesn't include this model, or your "
"switch providers." "quota is exhausted. Check https://grok.com/?_s=usage to see "
"which, or run `/model` to switch providers."
) )
# Idempotency: detect prior decoration by a substring unique to the # Idempotency: detect prior decoration by a substring unique to the
# hint (not present in xAI's own body text). # hint (not present in xAI's own body text).
if "Could be a missing subscription" in detail: if "X Premium+ does NOT include" in detail:
return detail return detail
return f"{detail}{hint}" return f"{detail}{hint}"

View file

@ -163,11 +163,13 @@ def test_codex_stream_postlude_error_still_falls_back():
def test_summarize_api_error_decorates_xai_entitlement_403(): def test_summarize_api_error_decorates_xai_entitlement_403():
"""xAI's OAuth 403 must end with the neutral 4-cause hint. """xAI's OAuth 403 must surface the X Premium+ gotcha + neutral causes.
Wording is deliberately ambiguous because xAI returns the SAME body for: Wording deliberately leads with the X Premium+ gotcha because that's
no subscription, wrong tier, exhausted quota, or API access not enabled. the #1 confusing case: people see Grok in their X app, assume it
Picking one (e.g. "you're not subscribed") would insult subscribers. works here too, and hit this 403 with no idea API access is a
separate SKU. Other causes (no subscription, wrong tier, exhausted
quota) follow.
""" """
from run_agent import AIAgent from run_agent import AIAgent
@ -180,13 +182,15 @@ def test_summarize_api_error_decorates_xai_entitlement_403():
summary = AIAgent._summarize_api_error(error) summary = AIAgent._summarize_api_error(error)
# The original xAI text must survive — it's still useful diagnostic info. # The original xAI text must survive — it's still useful diagnostic info.
assert "do not have an active Grok subscription" in summary assert "do not have an active Grok subscription" in summary
# The hint must NOT confidently assert "lacks subscription"; it must # The hint MUST lead with the X Premium+ gotcha (most likely cause
# acknowledge the 4 possible causes. # for users who think they're subscribed).
assert "Could be a missing subscription" in summary assert "X Premium+ does NOT include" in summary
assert "tier that doesn't include this model" in summary assert "standalone SuperGrok subscribers" in summary
assert "exhausted quota" in summary # Other causes still listed.
assert "API access not enabled" in summary assert "no Grok subscription" in summary
# The hint must point at the usage page where the user can verify which. assert "tier doesn't include this model" in summary
assert "quota is exhausted" in summary
# The hint must point at the usage page where the user can verify.
assert "https://grok.com/?_s=usage" in summary assert "https://grok.com/?_s=usage" in summary
# Switching providers is still a valid escape hatch. # Switching providers is still a valid escape hatch.
assert "/model" in summary assert "/model" in summary
@ -196,8 +200,9 @@ def test_summarize_api_error_does_not_accuse_subscribers():
"""Hint must not confidently say the user has no subscription. """Hint must not confidently say the user has no subscription.
Don Piedro reported his subscription is active. The hint must not Don Piedro reported his subscription is active. The hint must not
contradict him it must list all 4 possible causes and let him contradict him leading with the X Premium+ gotcha gives subscribers
check which one applies. a plausible reason ("oh, I'm on Premium+ not pure SuperGrok") instead
of accusing them of lying about having a subscription.
""" """
from run_agent import AIAgent from run_agent import AIAgent
@ -205,12 +210,11 @@ def test_summarize_api_error_does_not_accuse_subscribers():
"HTTP 403: do not have an active Grok subscription" "HTTP 403: do not have an active Grok subscription"
) )
summary = AIAgent._summarize_api_error(error) summary = AIAgent._summarize_api_error(error)
# MUST NOT contain language that assumes the user is unsubscribed. # MUST NOT contain language that flatly assumes the user is unsubscribed.
assert "lacks SuperGrok" not in summary assert "lacks SuperGrok" not in summary
assert "lacks subscription" not in summary assert "you are not subscribed" not in summary.lower()
assert "your account doesn't have" not in summary.lower() # MUST lead with the most-likely-but-non-accusatory cause.
# MUST contain the neutral framing. assert "X Premium+ does NOT include" in summary
assert "Could be" in summary or "could be" in summary
def test_summarize_api_error_decorates_xai_body_message(): def test_summarize_api_error_decorates_xai_body_message():
@ -231,7 +235,7 @@ def test_summarize_api_error_decorates_xai_body_message():
summary = AIAgent._summarize_api_error(_XaiErr("403")) summary = AIAgent._summarize_api_error(_XaiErr("403"))
assert "HTTP 403" in summary assert "HTTP 403" in summary
assert "Could be a missing subscription" in summary assert "X Premium+ does NOT include" in summary
def test_summarize_api_error_idempotent_for_entitlement_hint(): def test_summarize_api_error_idempotent_for_entitlement_hint():
@ -243,7 +247,7 @@ def test_summarize_api_error_idempotent_for_entitlement_hint():
twice = AIAgent._decorate_xai_entitlement_error(once) twice = AIAgent._decorate_xai_entitlement_error(once)
assert once == twice assert once == twice
# Sanity: the hint did fire on the first pass. # Sanity: the hint did fire on the first pass.
assert "Could be a missing subscription" in once assert "X Premium+ does NOT include" in once
def test_summarize_api_error_passes_through_unrelated_errors(): def test_summarize_api_error_passes_through_unrelated_errors():