hermes-agent/tests/hermes_cli/test_billing_scope_stepup.py
Siddharth Balyan 73cd8622f9
feat(billing): /billing terminal billing — interactive TUI + CLI client (#45449)
* feat(billing): nous_billing http client + BillingState core (phase 2b)

Phase 2b terminal-billing client foundation:
- hermes_cli/nous_billing.py: typed client for the 4 /api/billing/* endpoints
  (state/charge/poll/auto-top-up). Raises typed errors (BillingScopeRequired,
  BillingRateLimited, BillingAuthError) mapped from the live-verified contract;
  fail-open is the caller's job. Idempotency-Key enforced client-side.
- agent/billing_view.py: surface-agnostic BillingState core + Decimal money
  parsing (server emits decimal strings, not 2dp), fail-open builder,
  idempotency-key gen, custom-amount validation.
- 51 unit tests (decimal parse/format, payload tiering, error->exception
  matrix, fail-open, amount validation).

Plan: docs/plans/2026-06-13-001-phase-2b-terminal-billing-tui-plan.md

* feat(billing): billing:manage scope + lazy step-up re-auth (phase 2b)

- NOUS_BILLING_MANAGE_SCOPE constant.
- nous_token_has_billing_scope(): split-based scope check (no false-positive
  substring match).
- step_up_nous_billing_scope(): re-runs the device flow requesting
  billing:manage, reusing the held credential's portal/inference URLs + client_id
  (so a preview stays a preview), persists like _login_nous but WITHOUT the model
  picker. Returns True iff the minted token carries the scope (False when NAS
  silently downscopes a non-admin / unticked grant).

Lazy step-up (plan D-A): normal login path unchanged; 403 insufficient_scope
from a billing call triggers this. 7 unit tests.

* feat(billing): billing JSON-RPC methods for the TUI (phase 2b)

billing.state / charge / charge_status / auto_reload / step_up in
tui_gateway/server.py. Return STRUCTURED success envelopes (result.ok +
result.error=<code>) rather than JSON-RPC-level errors, so the Ink rpc() promise
always resolves and the TUI branches on the typed billing error code
(insufficient_scope, rate_limited, no_payment_method, …) to render the right
affordance. Money serialized as decimal STRINGS + display strings. charge mints
+ echoes an idempotency_key for retry reuse. 16 unit tests.

* feat(billing): /billing CLI handler + command registry (phase 2b)

- CommandDef("billing", subcommands=buy|auto-reload|limit), added to
  _SLACK_VIA_HERMES_ONLY so it routes via /hermes on Slack (keeps the 50-cap
  parity test green, same as /credits).
- cli.py::_show_billing + screen helpers: all 5 screens (overview, buy→confirm→
  poll, auto-reload, monthly-limit read-only). Reuses _prompt_text_input_modal /
  _prompt_text_input (D-C). Non-interactive (_app is None) renders text + portal
  deep-link, never prompts (R7). Decimal money end-to-end. 2s/5-min cancellable
  poll loop; 429/503 = retry not failure; settled = ledger truth. Lazy step-up on
  403 insufficient_scope. no_payment_method treated as mainline funnel-to-portal.
- 6 CLI tests; 156 command tests (incl. Slack/Telegram parity) green.

* feat(billing): /billing Ink TUI screens + tests (phase 2b)

- ui-tui/src/app/slash/commands/billing.ts: /billing TUI command covering all 5
  screens — overview (text), buy <amt> → ConfirmReq → charge → non-blocking 2s/
  5-min poll loop → settled/failed/timeout branches, auto-reload <below> <to> →
  ConfirmReq → PATCH, limit (read-only). Reuses the existing ConfirmReq overlay
  (D-C) — no bespoke component. Typed-error envelope branching: insufficient_scope
  arms the lazy step-up confirm; no_payment_method/rate_limited/cap funnel to
  portal. Client-side amount validation mirrors the server (bounds + 2dp).
- gatewayTypes.ts: Billing* response interfaces.
- registry.ts: register billingCommands.
- billingCommand.test.ts: 12 vitest cases (overview/gating/buy-confirm-poll-
  settled/no_payment_method/step-up/limit/auto-reload/validation).

TUI build green; 12/12 vitest pass; slash tests pass once @hermes/ink is built.

* docs(billing): scrub private cross-repo references

NAS is a private repo — remove all references to it from the public PR:
- drop the cross-repo planning doc (planning scaffolding, not a deliverable;
  the PR description documents the design)
- replace 'NAS' / 'PR #412 preview' mentions in code + test comments with
  generic 'the server' / 'a preview deployment'

* docs(billing): scrub final NAS reference in step-up docstring

* docs(billing): drop dangling plan-doc refs

The phase-2b plan doc was removed in the cross-repo scrub (300afcc0b)
but two module docstrings still pointed at it. Drop the dead refs.

* feat(billing): interactive /billing overlay + step-up UX, portal-URL & token fixes

Adds the interactive /billing TUI overlay and hardens the terminal-billing
client across CLI and TUI.

- TUI: full /billing overlay state machine (overview to buy to confirm,
  auto-reload, read-only monthly limit) reusing the existing confirm overlay.
- Step-up: surface the verification link in-transcript and open the browser
  via the TUI's own opener (the device flow runs in the headless gateway, so a
  printed URL was being dropped); run the step-up handler off the main loop and
  emit the link as an out-of-band event so the gateway stays responsive.
- Step-up copy is scope-accurate ("Billing permission granted") and re-checks
  /state so it never claims "enabled" when the org kill-switch is still off.
- Portal deep-links resolve to absolute URLs against the active portal base
  (the server emits them relative) - fixes a bare "/billing?topup=open" link.
- Billing calls refresh an expired access token via the stored refresh token
  instead of reporting a false "not logged in".
- Optimistic funnel: advise "set up a saved card on the portal" up front when
  no card is on file (advisory, not a hard gate).
- Token resolution is cached briefly so the 2s charge poll loop stops
  re-locking + re-reading the auth store on every tick; 401 re-resolves fresh.
- Remove the temporary demo-mode shims.

Validation: 87 Python billing tests, 88 TS tests (billing command + gateway
event handler), tsc clean, ink + ui-tui builds green.

* docs(billing): add /billing TUI screenshots for PR

* fix(cli): guard _last_invalidate on bare instances; update stale prompt-fallback test

The UI-invalidate throttle read self._last_invalidate unconditionally, which
raised AttributeError on HermesCLI instances built without __init__ (the
thread-safety test's object.__new__ shell). Guard the read with getattr.

The off-main-thread branch of _prompt_text_input was changed (#23185) to cancel
cleanly to None instead of falling back to a bare input() that would hang on the
slash-worker thread; the test still asserted the old direct-input fallback.
Update it to assert the current intended behavior: returns None, calls neither
run_in_terminal nor input(), and does not hang.
2026-06-19 01:53:32 +05:30

193 lines
6.7 KiB
Python

"""Tests for the Phase 2b billing:manage scope step-up (auth.py)."""
from __future__ import annotations
import pytest
import hermes_cli.auth as auth
from hermes_cli.auth import (
NOUS_BILLING_MANAGE_SCOPE,
nous_token_has_billing_scope,
step_up_nous_billing_scope,
)
# ---------------------------------------------------------------------------
# nous_token_has_billing_scope
# ---------------------------------------------------------------------------
def test_has_scope_true_when_present(monkeypatch):
monkeypatch.setattr(
auth,
"get_provider_auth_state",
lambda p: {"scope": "inference:invoke tool:invoke billing:manage"},
)
assert nous_token_has_billing_scope() is True
def test_has_scope_false_when_absent(monkeypatch):
monkeypatch.setattr(
auth, "get_provider_auth_state", lambda p: {"scope": "inference:invoke tool:invoke"}
)
assert nous_token_has_billing_scope() is False
def test_has_scope_false_when_no_state(monkeypatch):
monkeypatch.setattr(auth, "get_provider_auth_state", lambda p: None)
assert nous_token_has_billing_scope() is False
def test_has_scope_no_substring_false_positive(monkeypatch):
# "billing:manage-lite" must NOT match billing:manage (split-based, not substring).
monkeypatch.setattr(
auth, "get_provider_auth_state", lambda p: {"scope": "billing:manage-lite"}
)
assert nous_token_has_billing_scope() is False
# ---------------------------------------------------------------------------
# step_up_nous_billing_scope
# ---------------------------------------------------------------------------
@pytest.fixture
def _stub_persist(monkeypatch):
"""Neutralize the persistence side-effects so step-up tests are pure."""
monkeypatch.setattr(auth, "_auth_store_lock", lambda: _NullCtx())
monkeypatch.setattr(auth, "_load_auth_store", lambda: {})
monkeypatch.setattr(auth, "_save_provider_state", lambda *a, **kw: None)
monkeypatch.setattr(auth, "_save_auth_store", lambda *a, **kw: "auth.json")
monkeypatch.setattr(auth, "_write_shared_nous_state", lambda *a, **kw: None)
monkeypatch.setattr(auth, "_sync_nous_pool_from_auth_store", lambda: None)
class _NullCtx:
def __enter__(self):
return self
def __exit__(self, *a):
return False
def test_step_up_requests_billing_scope_and_reuses_prior_urls(monkeypatch, _stub_persist):
monkeypatch.setattr(
auth,
"get_provider_auth_state",
lambda p: {
"scope": "inference:invoke tool:invoke",
"portal_base_url": "https://preview.example.com",
"inference_base_url": "https://inf.example.com",
"client_id": "hermes-cli",
},
)
captured = {}
def _fake_login(**kw):
captured.update(kw)
# Simulate the admin ticking the box → token comes back WITH the scope.
return {"scope": "inference:invoke tool:invoke billing:manage", "access_token": "t"}
monkeypatch.setattr(auth, "_nous_device_code_login", _fake_login)
granted = step_up_nous_billing_scope()
assert granted is True
# Requested scope must include billing:manage, preserving prior scopes.
assert NOUS_BILLING_MANAGE_SCOPE in captured["scope"].split()
assert "inference:invoke" in captured["scope"].split()
# Reuses the prior credential's deployment URLs (so a preview stays a preview).
assert captured["portal_base_url"] == "https://preview.example.com"
assert captured["client_id"] == "hermes-cli"
def test_step_up_returns_false_when_downscoped(monkeypatch, _stub_persist):
# Non-admin / unticked → the server silently downscopes; token comes back WITHOUT scope.
monkeypatch.setattr(auth, "get_provider_auth_state", lambda p: {"scope": "inference:invoke"})
monkeypatch.setattr(
auth,
"_nous_device_code_login",
lambda **kw: {"scope": "inference:invoke", "access_token": "t"},
)
assert step_up_nous_billing_scope() is False
def test_step_up_falls_back_to_standard_scope_when_no_prior(monkeypatch, _stub_persist):
monkeypatch.setattr(auth, "get_provider_auth_state", lambda p: {})
captured = {}
def _fake_login(**kw):
captured.update(kw)
return {"scope": "inference:invoke tool:invoke billing:manage"}
monkeypatch.setattr(auth, "_nous_device_code_login", _fake_login)
step_up_nous_billing_scope()
requested = captured["scope"].split()
assert "inference:invoke" in requested
assert "tool:invoke" in requested
assert NOUS_BILLING_MANAGE_SCOPE in requested
# ---------------------------------------------------------------------------
# on_verification callback plumbing (TUI surfaces the device-flow URL via this)
# ---------------------------------------------------------------------------
def test_step_up_forwards_on_verification_callback(monkeypatch, _stub_persist):
monkeypatch.setattr(auth, "get_provider_auth_state", lambda p: {})
captured = {}
def _fake_login(**kw):
captured.update(kw)
return {"scope": "inference:invoke tool:invoke billing:manage"}
monkeypatch.setattr(auth, "_nous_device_code_login", _fake_login)
def _cb(url, code):
pass
step_up_nous_billing_scope(on_verification=_cb)
# The callback must be threaded straight through to the device-code login.
assert captured["on_verification"] is _cb
def test_device_login_fires_on_verification_before_polling(monkeypatch):
"""on_verification(url, code) must fire BEFORE _poll_for_token (so the TUI
can render the link while the flow blocks waiting for approval)."""
order: list[str] = []
monkeypatch.setattr(
auth,
"_request_device_code",
lambda **kw: {
"verification_uri_complete": "https://portal.example/device?code=ABCD",
"user_code": "ABCD-1234",
"device_code": "dev",
"expires_in": 600,
"interval": 5,
},
)
def _fake_poll(**kw):
order.append("poll")
return {"access_token": "t", "scope": "inference:invoke", "expires_in": 3600}
monkeypatch.setattr(auth, "_poll_for_token", _fake_poll)
seen = {}
def _cb(url, code):
order.append("verify")
seen["url"] = url
seen["code"] = code
# We only assert the callback fires before polling. Post-poll token
# validation (JWT usability checks) is out of scope and may raise on the
# synthetic token — swallow it; the ordering assertion is what matters.
try:
auth._nous_device_code_login(open_browser=False, on_verification=_cb)
except Exception:
pass
assert order[:2] == ["verify", "poll"], "callback must fire before polling"
assert seen["url"] == "https://portal.example/device?code=ABCD"
assert seen["code"] == "ABCD-1234"