* fix(gateway): serialize concurrent turns per resolved session_id with a turn lease
Closes the serialization half of #64934. The busy guards are keyed by
routing key, but the durable transcript is owned by session_id — and
switch_session() makes the key→id mapping many-to-one (/resume from a
second chat/topic, CLI-continuity rebinding, async-delegation pinning,
topic-binding tip-walks). Two routing keys mapped to one session_id ran
concurrent turns on two different agent objects, invisible to every
per-key guard: flushes persisted in completion order, the identity-marker
dedup swallowed rows, and the second turn ran on a stale history base —
leaving a permanent user;user alternation wedge.
The fix: an asyncio lease keyed by RESOLVED session_id (gateway/turn_lease.py),
acquired in _handle_message_with_agent after session resolution is final
(post switch_session/tip-walk), immediately before the transcript load, and
released in _handle_message's finally on every exit path. Tokens are granted
per (routing key, run generation) so a stale unwind can never release a newer
turn's lease (#28686 ownership lesson). Same-key messages never reach the
acquisition point mid-turn (both routing-key guards hold them), so the lock
is uncontended outside the alias-key route — where the second turn now waits
for the first turn's flush and logs one WARNING naming the session and both
routing keys (pairs with the #67371 tripwire).
Fail-open: a stuck holder degrades to today's unserialized behavior with a
loud ERROR after agent.gateway_timeout — never a wedged session; a degraded
token holds nothing and can't steal the lease. Registry is size-capped and
never evicts a live lease. Persist-disabled review forks never dispatch
through _handle_message, so they cannot contend.
Known limits (tracked on #64934): CLI-continuity cross-process pairs need a
DB-level lease; mid-turn compression rotation leaves a small alias window
for a follow-up at the binding-sync sites.
Validation: 8 behavior tests (alias-key wait + flush order, no cross-session
contention, generation-scoped idempotent release, timeout fail-open without
lease theft, bounded registry, bare-runner-safe release wiring) + E2E against
a real SessionStore reproducing the issue's switch_session alias route —
strict alternation and arrival order preserved.
* refactor(gateway): conversation-scope funnel + mid-turn lease rebind
Completes the #64934 system beyond the point fix. Two structural changes,
both eliminating whole bug classes rather than instances:
1. _clear_conversation_scope — THE single conversation-boundary funnel.
/new, /resume, auto-reset, expiry finalization, and the
compression-exhausted reset each carried a hand-copied pop-list of the
per-session dicts, and the lists drifted every time a new dict was
added (#48031, #58403, #10702, #35809 were all 'boundary X forgot
dict Y' bugs). All five sites now make one funnel call driven by the
_CONVERSATION_SCOPED_STATE registry; adding a new conversation-scoped
dict means adding one name to the registry, and every boundary picks
it up automatically. Scope rules documented at the registry: turn-scoped
state, the monotonic generation counter, and the agent cache are
deliberately excluded (different lifecycles).
2. SessionTurnLeaseRegistry.rebind — the held turn lease now FOLLOWS
mid-turn compression rotation. Both rotation sites (session-hygiene
pre-compression, agent-result session_id swap) alias the same
_SessionLease object under the new id, so an alias routing key
resolving the fresh child (topic tip-walk) still serializes against
the in-flight turn. Closes the rotation-alias window flagged as a
known limit on #64934. Ownership-checked like release; when the
target id already has a live lease the rebind fails open with a loud
WARNING (never a mid-turn deadlock).
Tests: 3 new rebind behavior tests + 5 funnel behavior tests (including
a real-setter drift guard); the two AST change-detector pins in
test_10710/test_48031 were re-pointed at the funnel and the #58403 pin
converted to a behavioral test. E2E: rotation-alias scenario against a
real SessionStore + SessionDB — turn B on the fresh child waits behind
the rotated holder, sees its rows, alternation intact.
The per-message ephemeral context prompt re-renders every turn, and
any byte change (Discord auto-thread rename, reset notes, voice
channel state) both breaks the provider prompt-cache prefix at the
head of every request and changes the gateway agent-cache signature,
forcing a full agent rebuild per message. Pin the rendered block per
session keyed by a hash of exactly the fields it renders, so only a
real input change (rename, topic edit, /sethome, redact_pii flip)
re-renders; deliver one-shot per-turn facts (auto-reset note,
first-contact intro, voice-channel changes) on the current user
message via the api_content sidecar instead of the system prompt; sort
get_connected_platforms for byte-stable ordering.
A final response generated but not confirmed-delivered was the one
artifact the gateway could lose without a trace: crash or planned
restart between finalize and platform ACK dropped it silently, and the
resume path re-ran the whole turn at full cost (#58818 P1, #41696,
#63695's gateway half).
gateway/delivery_ledger.py records each outbound final response in
state.db (same conventions as the async-delegation ledger: WAL, owner
pid + process-start-time liveness, bounded retention):
pending -> attempting -> delivered | failed
startup sweep on dead-owner rows -> redeliver | abandoned
Contract (the lessons from the closed delivery-outbox attempt #61790):
- obligation recorded BEFORE the first send attempt; cleared only on
SendResult.success (destination acceptance, #51184)
- ambiguity is labeled, never silently retried: rows that were mid-send
when the process died redeliver with a visible '♻️ Recovered reply —
may be a duplicate' prefix (honest at-least-once)
- stable ids from session_key + inbound message id + content, so
distinct threads/topics can never collide
- poison rows bounded: 3 attempts / 24h freshness -> abandoned; claim
atomically re-stamps ownership so racing sweeps can't double-claim
- redelivery clears resume_pending for the session so the resume path
never re-runs a turn whose answer the ledger already holds
- best-effort everywhere: ledger failure can never block or delay a send
- slash-command/ephemeral/empty responses are not recorded; cron and
proactive delivery stay on DeliveryRouter (separate subsystem)
Config: gateway.delivery_ledger (default on; no version bump needed).
Validation: 30 ledger+producer tests; 352 blast-radius gateway tests
green; cross-process E2E (record in process A, kill it mid-send, claim
+ marker + redeliver in a fresh process B against the same state.db).
Rebased onto current main, where _ensure_session_db grew a per-profile
cache (get_hermes_home()-keyed) for /p/<profile>/ multiplex — after this
PR's base. The PR's async rewrite assumed the old single-self._session_db
model, so on current main `session_db=self._session_db` in _create_agent
would pass None in production (the real DB lives in the per-home cache).
Split the concern: keep a SYNC _ensure_session_db (per-profile, used by
_create_agent + the many sync-patching create_agent tests) and add an
async _ensure_session_db_async that captures the profile home on the loop
thread then offloads only the SQLite open via to_thread (single-flight).
Both share _open_and_cache_session_db. Request handlers use the async
variant; _create_agent reverts to the sync call. Updated the first-request
test's FakeDB to accept db_path to match main's SessionDB(db_path=...).
Co-authored-by: necoweb3 <sswdarius@gmail.com>
- Make create sequence (check + insert + title) atomic via single
_execute_write call with BEGIN IMMEDIATE, closing the TOCTOU window
where two concurrent same-ID creates could both return 201.
- Offload _ensure_session_db() to asyncio.to_thread with single-flight
lock so first-request SQLite init doesn't block the event loop.
- Add concurrent same-ID create test (one 201, one 409) and
first-request path test covering the initialization.
Deduplicates the sidecar handling across 9 sites:
- substitute_api_content(): 2 API-bound pop+substitute sites
(chat_completion_helpers, transport — conversation_loop keeps its
inline pop because the current-turn compose fallback needs the value)
- drop_stale_api_content(): 4 rewrite-drop sites
(context_compressor x2, replay_cleanup, agent_runtime_helpers)
- extract_api_content_sidecar(): 3 gateway forwarding sites
(gateway/session, gateway/slash_commands, cli_commands_mixin)
Also: restore the eager _pending_cli_user_message clear on the early
row-creation try (was lost when the crash-persist moved after prefetch —
a crash in compression between the two tries would leave a stale staged
input), and fix a comment indentation nit in run_agent.py.
Co-authored-by: Soju06 <qlskssk@gmail.com>
The first LLM call of every gateway turn gets ~0% provider prompt-cache
hit rate (in-turn calls: 97-99%) because the bytes sent for a turn's
user message are not the bytes replayed next turn: memory-prefetch and
pre_llm_call context are injected into the API copy only, the #48677
persist override writes cleaned content to the DB row, and
get_messages_as_conversation sanitize/strips user and assistant content
on load. Any of these diverges the request prefix at that message and
re-prefills everything after it — measured 27.9s for the first call vs
2.4-5.8s cached at a median ~156k-token context.
Persist what you send: a nullable messages.api_content column stores the
exact content string sent to the API when it differs from the clean
stored content, and replay substitutes it verbatim (no sanitize, no
strip). The injection composition lives in one helper
(turn_context.compose_user_api_content); the turn prologue stamps its
output onto the live user message, the api_messages build sends the
stamped bytes, and every outgoing copy pops the field so it never
reaches a provider. The crash-resilience user-turn persist moves after
prefetch/pre_llm_call so the user row is written once with its final
sidecar; _ensure_db_session stays before preflight compression (session
rotation needs the parent row under PRAGMA foreign_keys=ON). The
current-turn index trackers are re-anchored after compaction rebuilds
the message list, in-place preflight compaction backfills the stamp onto
the already-inserted row, and gateway replay forwards the sidecar only
when the replay pipeline did not rewrite the content. Rewrite paths that
would leave stale bytes (historical image strip, merge-summary-into-
tail, consecutive-user repair merge, stale-confirmation redaction) drop
the sidecar; the chat-completions transport and the max-iterations
summary path strip it defensively. codex_app_server and MoA turns are
excluded from stamping because their wire bytes differ from the
composition. A missing or dropped sidecar degrades to today's behavior
(one cache-boundary miss), never to wrong content.
The busy/priority/monitor/backup×2/drain paths each had near-identical
try/except blocks for transcribe+echo with only log_context, adapter,
and metadata varying. Extract into _transcribe_and_echo_pending_voice
which handles the cache lookup, echo dedup, and exception logging in
one place.
Uses a _UNSET sentinel to distinguish 'caller did not pass metadata'
(use the rich _thread_metadata_for_source fallback) from 'caller
explicitly passed None' (monitor/backup/drain paths that use the
simpler thread_id-only dict).
~120 lines of copy-paste collapsed into one 30-line helper.
merge_pending_message_event extends the existing event's media_urls in
place when two media-bearing messages arrive in quick succession (photo
bursts, consecutive voice messages). The gateway runner caches STT
transcripts on the event via _gateway_pending_stt_text; if the cached
event gains new media after the cache was populated, the stale transcript
was returned instead of transcribing the merged attachments.
Add _invalidate_pending_stt_cache() and call it from both media-merge
branches in merge_pending_message_event so the next transcription call
re-runs against the full merged media list.
Closes the merge-race edge case identified during review of PR #61519.
The function was introduced in d55304c39 but never had a single caller —
verified via git log -S and search_files across the whole repo. The
drain path at _stream_confirmed_final_delivery inlines its logic directly.
Keeping a dead function around that duplicates live logic is a maintenance
hazard: future changes to the live path won't propagate to the dead one.
Fixes the two review defects that kept PR #29923 open, plus docs:
- gateway: exclude --once from the session-store write-through. The
once-override lived only in memory before, but the write-through
persisted it, so a gateway restart before the finally-restore
rehydrated a supposedly one-turn model permanently.
- TUI: skip _sync_agent_model_with_config while a one-turn restore is
pending. The once-model is deliberately not pinned as a session
model_override, so the config sync saw a model mismatch and clobbered
the once-override back to the config model before the turn ran.
- tests: real _handle_model_command drive asserting --once never
touches set_model_override while --session still does; restore-pop
idempotency.
- docs: /model --once in configuring-models.md with an honest
prompt-cache cost note (one-shot switch breaks the cached prefix
twice; wins for short sessions and cheap-to-expensive escalation).
Adds --once to /model across CLI, TUI, and gateway: switch model for the
next turn only, restoring the previous model in a finally block so
success, exception, and interrupt all revert. Parsing extends
parse_model_flags_detailed(); resolve_persist_behavior() treats --once
as a persistence opt-out; --global + --once is rejected.
Salvaged from PR #29923 (image-generation lane split to #59815 per
review; conflict resolution against current main by the maintainers).
Preserve monotonic final-delivery completion, isolate recovery config and storage per adapter/profile, coalesce reconnect scans, release cancelled claims, bypass split-message debounce for historical events, and move bounded ledger setup into a short-timeout state module.
Route recovered messages through the live Discord ingress policy, preserve dedup and completion invariants, bound and retain the recovery ledger, and expose the opt-in config with docs and backup coverage.
Gateway tests build adapters via object.__new__() without __init__ (the
documented bare-instance pattern), so the new _status_text dict must be
accessed through getattr guards in set_status_text, the _keep_typing
finally cleanup, and the Slack send_typing read — same treatment as
other post-hoc __init__ attributes. Fixes CI shard 2/8
(test_active_session_text_merge).
Builds on the salvaged typing_status_text plumbing (PR #62007): instead
of a static 'is thinking...', Slack's assistant status line now updates
live as the agent works — 'is running pytest tests/…', 'is reading
docs/api.md…' — and reverts to the static text between tool calls.
Mechanics:
- agent/display.py: build_status_phrase() derives a <=49-char present-
tense phrase from the existing _TOOL_VERBS table (+ 'is using <name>'
for plugin/MCP tools; None for _thinking).
- base adapter: supports_status_text capability flag + set_status_text()
per-chat store, cleared when the typing loop winds down.
- Slack adapter: send_typing() renders the live phrase when set, falling
back to typing_status_text then 'is thinking...'.
- gateway/run.py: progress_callback stashes the phrase on tool.started
and clears on tool.completed. Rendering rides the existing
_keep_typing refresh cadence — zero additional Slack API calls, no
rate-limit exposure. Works with tool_progress: off (Slack default);
the callback is now armed whenever the adapter supports status text.
- display.live_status config (full|verb|off, default full): 'verb' hides
argument previews for shared/customer-facing channels.
Also fixes a latent crash in the cherry-picked from_dict: malformed
non-dict 'extra' sections broke typing_status_text resolution (uses the
already-coerced extra dict).
Design notes: status text is a side-effect display channel only — never
enters the transcript, no prompt-cache impact. Lifecycle guarantees from
the stuck-status fix family are preserved (per-thread tracking,
clear-on-finish via existing stop_typing paths). Related: #45109
(closed; same direction via lifecycle states), #59010/#51363 (native
task cards — complementary, larger scope).
Adds PlatformConfig.typing_status_text for the two platforms that render
text for the working-state line: Slack's assistant.threads.setStatus
status (hardcoded 'is thinking...') and Google Chat's visible marker
message (hardcoded 'Hermes is thinking…'). None keeps each platform's
built-in default; to_dict omits the field when unset so existing configs
serialize unchanged. Plumbing mirrors typing_indicator exactly (typed
field, from_dict extra fallback, shared-key bridge).
Also documents that Slack's status line requires the assistant:write
scope — without it setStatus fails silently and Slack shows its own
generic placeholder, which previously made the behaviour undiagnosable
from config alone.
/simplify-code findings on the #66222 salvage:
- consume_detached_task_result moves to agent/async_utils.py (shared home);
gateway/run.py and the Discord adapter both had near-identical copies of
the same pattern (a third lives in the telegram adapter). One canonical
implementation, both new callsites import it.
- Reconnect backoff formula min(30 * 2^(n-1), 300) was copied verbatim at
3 sites in run.py (primary watcher x2, secondary-profile reconnect), the
third hardcoding the cap. Hoisted to module-level _reconnect_backoff()
with a single _RECONNECT_BACKOFF_CAP so a future tune can't silently
miss one path.
Behavior-preserving: 70 gateway teardown/liveness/reconnect tests green.
Replace REST-based Discord liveness probe with local WebSocket/heartbeat
state detection. REST success doesn't prove Gateway event delivery — a
half-closed WebSocket can leave Bot.start() alive while REST returns 200.
Now samples ready/open/ACK state and heartbeat latency; consecutive
unhealthy samples emit one retryable fatal code so GatewayRunner rebuilds
the adapter through the existing reconnect path.
Also fixes three lifecycle gaps in the recovery path:
1. asyncio.wait_for() can remain blocked if adapter cleanup swallows
cancellation — now uses bounded asyncio.wait() with task detachment.
2. Multiplexed secondary-profile adapters had no profile-scoped reconnect
owner — now uses one runner-owned reconnect slot per profile.
3. An in-flight turn could send its final text through the disconnected
adapter after a replacement was registered — now resolves the live
same-profile replacement for unsent final responses only (message IDs
never migrate, edits/deletes stay on the old transport).
Adds an opt-in Linux/systemd event-loop watchdog (gateway.systemd_watchdog_seconds,
default 0) for the failure mode where the whole asyncio loop stops making
progress and no in-process liveness task can run. stdlib-only sd_notify,
Type=notify/WatchdogSec generation, READY/STOPPING lifecycle.
Co-authored-by: 王鑫 <wx.xw@bytedance.com>
C1: The watchdog's os._exit(1) bypassed drain_log_queue() and
remove_pid_file()/release_gateway_runtime_lock() — the three things
_exit_after_graceful_shutdown does before exiting. The watchdog's own
logger.critical('shutdown watchdog fired') was silently dropped because
it was still in the async QueueListener queue when os._exit ran. PID
file and runtime lock were stranded on next boot.
W1: Dropped 2MB infographic PNG — dead asset with zero references in
the repo. Binary blobs in git history are permanent; every clone pays
the cost forever.
Authorship: @HexLab98's commits preserved via rebase-merge.
A frozen asyncio loop mid-SIGTERM drain cannot run the drain timeout or
status rewrites, so KeepAlive never sees a dead process. Arm an OS-thread
watchdog at stop() (drain+60s → faulthandler dump + os._exit) and rewrite
state/gateway.heartbeat from a loop task for external liveness checks.
- _spawn_supervised restart cap was lifetime-cumulative (never reset), so a
watcher crashing _MAX_SUPERVISED_RESTARTS times over a days-long process was
permanently abandoned despite the 'consecutive' wording. Make it time-based:
reset the attempt counter when the task ran healthily (>= _SUPERVISED_HEALTHY_SECS
= 300s) before crashing, so only rapid repeated crashes accumulate toward the
ceiling. Reword docstring/log. New regression: healthy-run-then-crash is not
abandoned.
- _derive_stream_stale_timeout read only 'model', so the reasoning stale-floor
never applied to Bedrock (payloads key the model as 'modelId', dotted
us.anthropic.claude-opus-4-6-v1:0 form). Resolve model||modelId and normalize
the Bedrock dotted/region-prefixed form to match the floor. (Sonnet's dotted
vs dashed floor-table key is a pre-existing table mismatch, left out of scope.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-applied against v0.18.2. Still unconditional KeepAlive <true/> with no
ThrottleInterval, and upstream's new restart_loop_guard only skips session
auto-resume (never throttles the boot), so the launchd respawn storm is
unguarded.
- launchd plist: add ThrottleInterval=30 + ExitTimeOut=25.
- status: record_start_and_check_storm — portable breaker (atomic gateway-starts.log,
backs off when too many starts land in a window); run_gateway sleeps it before
asyncio.run. Env HERMES_GATEWAY_MAX_STARTS (<=0 disables) / _START_WINDOW_S.
Separate file from upstream's restart_loop.json — no collision.
- tests: breaker (threshold/prune/atomic) + plist throttle keys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-applied against v0.18.2. Upstream now wraps each watcher's INNER loop in
try/except (kept — not re-added), but the 9 long-lived watchers in start() are
still bare asyncio.create_task: no handle, no exception logging, no restart. A
raise in _platform_reconnect_watcher's OUTER while-loop / pre-try region still
dies silently, permanently losing platform reconnection. Add _spawn_supervised
(track + log + bounded-backoff restart; no respawn on clean return to avoid
busy-spin) and route all 9 watchers through it.
- tests: clean-return spawned exactly once; exception-restart bounded at ceiling+1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(tui): rename /billing slash command to /topup
Behavior-preserving rename of the /billing command surface to /topup.
Changes: billing.ts → topup.ts (export topupCommands, name 'topup', new
help string), registry.ts import+spread updated, billingOverlay.tsx
overview header 'Usage credits' → 'Top up credits', billingCommand.test.ts
→ topupCommand.test.ts with import/lookup/call updated. RPC method names
(billing.state, billing.charge, etc.) and component/symbol names unchanged.
* refactor(tui): extract overlay primitives to shared module
Lift MenuRow, ActionRow, footer, and barCells() out of billingOverlay.tsx
into overlayPrimitives.tsx so the upcoming subscriptionOverlay.tsx can
import them instead of duplicating. spendBar now calls barCells() —
output is byte-identical. Pure behavior-preserving refactor.
* feat(tui): add /subscription + /topup CTAs to /usage output
Every /usage render now ends with 'Run /subscription to change plan
· /topup to add credits' — both the healthy (with-calls) and depleted
(no-calls) paths. Strings-only change, no WS1 dependency.
* feat(tui): add subscription wire types
Add SubscriptionTierOption, SubscriptionStateResponse, and
SubscriptionManageLinkResponse to gatewayTypes.ts. Type-only — no
usages yet. Mirrors the BillingStateResponse conventions (snake_case,
Decimals as strings) and reuses BillingErrorPayload for error mapping.
* feat(gateway): add subscription.state + subscription.manage_link RPCs
- agent/subscription_view.py: SubscriptionState dataclass + fail-open
build_subscription_state() (mirrors billing_view pattern) +
get_subscription_manage_link() for the Stripe deep-link.
- hermes_cli/nous_billing.py: get_subscription_state() +
post_subscription_manage_link() HTTP helpers for the two NAS endpoints
(WS1 Phase A/C). The manage-link endpoint raises BillingScopeRequired
when Remote-Spending is missing (Phase 4 step-up trigger).
- tui_gateway/server.py: _serialize_subscription_state() +
subscription.state RPC (fail-open) + subscription.manage_link RPC
(returns {ok,kind,url} or typed error envelope via
_serialize_billing_error). NOT added to _LONG_HANDLERS — synchronous
HTTP round-trip, not a device flow.
* feat(tui): add subscription overlay state types + store slot
Add SubscriptionScreen, SubscriptionOverlayCtx, SubscriptionOverlayState
to interfaces.ts and a 'subscription' slot to OverlayState. Wire it into
overlayStore.ts (buildOverlayState + $isBlocked). NOT added to
resetFlowOverlays preserve list — flow-scoped like billing, drops on
turn end.
* feat(tui): build SubscriptionOverlay — overview + confirm + handoff
Pure-render Ink component mirroring billingOverlay.tsx's structure.
Overview screen covers all 5 states (free-upgradeable, mid-tier,
top-tier, not-admin, downgrade-pending) + dunning. Confirm screen is
y/n deep-link to Stripe (NO in-terminal charge). Handoff is the
transient 'Opening Stripe' screen. Imports shared primitives from
overlayPrimitives.tsx. 8 render tests via renderSync covering every
state.
* feat(tui): add /subscription command + overlay wiring
- subscription.ts: SubscriptionOverlayCtx closure (openManageLink,
refreshState, requestRemoteSpending) + run handler that fetches
subscription.state and opens the overlay. Alias /upgrade.
- registry.ts: spread subscriptionCommands into SLASH_COMMANDS.
- appOverlays.tsx: render SubscriptionOverlay when overlay.subscription set.
- useInputHandlers.ts: Esc closes subscription overlay; promptOverlay OR
includes subscription so input is intercepted while open.
- subscriptionCommand.test.ts: 4 tests (fetch+open, logged-out sys line,
/upgrade alias, /subscription resolves).
* fix(tui/subscription): stop saying Stripe in deep-link copy + fix manage link kind type
Replace all user-facing 'Stripe' mentions in the /subscription overlay and
sys messages with 'your subscription page' — the deep-link target is NAS's
own /manage-subscription page, not the Stripe hosted portal. Stripe only
legitimately appears later at actual Checkout. Also add 'manage' to the
SubscriptionManageLinkResponse.kind union (NAS emits kind:'manage'; was
previously missing from the TypeScript type causing silent narrowing errors).
* feat(tui/subscription): render cancellation-scheduled note with headline precedence
Parse cancelAtPeriodEnd + cancellationEffectiveAt from the NAS contract
(camelCase) in the agent parser (_parse_current), emit cancel_at_period_end
+ cancellation_effective_at from the gateway serializer, extend the
SubscriptionStateResponse type, and render a warn note in OverviewScreen:
'Cancels on {date} — your plan stays active until then.'
Headline precedence when multiple flags co-occur:
past-due > cancel-scheduled > downgrade-pending > active
The downgradeNote guard is tightened to suppress when cancel is scheduled,
so at most one status line renders at a time.
* feat(tui/subscription): team-context screen — redirect to /topup for team orgs
Parse the NAS context:'personal'|'team' field (defaults to 'personal' for
unknown/missing values), emit it on the gateway wire, add it to
SubscriptionStateResponse. When context is 'team', SubscriptionOverlay
renders a dedicated read-only screen instead of the tier picker:
'This terminal is connected to {org_name}. Teams run on shared
credits — use /topup to add funds. Personal subscriptions live
on your personal account.'
The screen closes on Enter or Esc. The personal/tier-picker path is
unchanged.
* fix(subscription): drop manage-link gateway RPC, build URL locally
The NAS POST /api/billing/subscription/manage-link endpoint was dropped
(it added no server work — the target is the static /manage-subscription
page, not a Stripe-minted secret). Build the URL client-side instead:
{portal_base}/manage-subscription?org_id=<org.id>.
- Remove subscription.manage_link gateway RPC (server.py)
- Remove get_subscription_manage_link helper (subscription_view.py)
- Remove post_subscription_manage_link (nous_billing.py)
- Remove SubscriptionManageLinkResponse type (gatewayTypes.ts)
- Add org_id to SubscriptionState + wire through serializer + TS type
- openManageLink() builds the URL locally via buildManageUrl(), opens
it with the existing openExternalUrl(), no gateway round-trip
- Drop targetTierId param from openManageLink (v1 sends everyone to
/manage-subscription; no tier deep-link needed)
- Fix stale test expectations (Stripe copy → subscription page copy)
* chore(subscription): drop unused format_money import
* feat(cli): /subscription + /upgrade, /billing→/topup rename, /usage CTAs
Add the classic-CLI half of the terminal billing surface to match the TUI:
- /subscription (alias /upgrade) command + /topup (renamed /billing, keeps
'billing' as a back-compat alias) in the command registry.
- Drop the stale 'billing' entry from _SLACK_VIA_HERMES_ONLY (now cli_only).
* feat(subscription): CLI /subscription handler, drop dunning, current:null no-plan
- CLI _show_subscription mirrors the TUI overlay (plan read + tier list + usage
bar + browser deep-link via subscription_manage_url); credits render as counts.
- Adapt to the updated NAS read contract: remove is_past_due/dunning everywhere
(a card-failing subscriber returns as a normal plan now), and treat no-plan as
current:null (parser returns None) rather than an all-null object.
- HERMES_DEV_SUBSCRIPTION_FIXTURE env-driven fixtures + ui-tui fixture harness
drive every state (CLI + live TUI) with no portal.
Verified against handoff 2026-06-24_subscription-tui-handoff.md.
* feat(billing): CF-4 Remote-Spending revoked-terminal UX (NAS PR #481)
Wire the Remote-Spending gate denial contract end to end:
- nous_billing: BillingRemoteSpendingRevoked (403 remote_spending_revoked →
reconnect) + BillingSessionRevoked (401 session_revoked → re-login), distinct
from insufficient_scope; capture actor/code/recovery; 503 stays transient.
- gateway _serialize_billing_error threads the new typed kinds + actor/code/
recovery to the TUI.
- TUI renderBillingError: actor-aware revoke copy, kills the spend overlay
immediately (no 15-min zombie button), handles session_revoked, the dual-
emitted cli_billing_disabled/remote_spending_disabled, role_required,
idempotency_conflict; poll treats a mid-poll revoke as ambiguous (check
balance before retry), not a failure.
- CLI _billing_render_charge_error: same denial matrix, actor-aware copy.
Tests: gate-contract mapping + envelope (py) and revoke/session/disabled (TUI).
Per handoff 2026-06-24_remote-spending-TUI-contract-handoff.md.
* refactor(subscription): remove dead step-up scaffolding from /subscription
/subscription only opens a browser deep-link to manage-subscription — that needs
no billing scope, so it can never hit insufficient_scope. Drop the never-fired
'stepup' screen type, requestRemoteSpending ctx fn, and resumeScreen bookkeeping
(leftovers from a superseded plan). The resumable step-up lives on /topup, where
the charge actually gets gated.
* feat(tui/topup): resumable 'Allow Remote Spending' step-up on the charge path
Phase 4: when a charge returns insufficient_scope, the /topup modal no longer
tears down with a 'run /billing again' ConfirmReq. Instead it stays MOUNTED and
switches to a step-up screen:
- charge() is now awaitable, returning a discriminated outcome (submitted |
needs_remote_spending | error) so the overlay can route without closing.
- StepUpScreen: 'Allow Remote Spending' → await the device-flow grant (browser
opens via the existing out-of-band billing.step_up.verification event) →
replay the held charge (pendingCharge.amount) and settle, with no command
re-run. Never surfaces the raw billing:manage scope.
- armStepUp's fire-and-forget ConfirmReq replaced by requestRemoteSpending();
the leaky 'billing:manage' / 'Re-authorize' / 'run /billing again' copy is gone.
Tests: charge-outcome routing, step-up grant/deny, and a render test asserting
the step-up copy holds the amount and never leaks billing:manage.
Per handoff 2026-06-24_remote-spending-TUI-contract-handoff.md §2 (Grady #6).
* feat(billing): shared dollar usage model + two-bar view (drop "credits")
Single source of truth for the /usage and /subscription usage bars across
TUI + CLI. Reads the NAS account-info dollar fields (subscription/top-up/total
remaining, monthly allowance, renewal) and produces a surface-agnostic model:
two full-resolution bars (plan allowance + purchased top-up), a status
classification (free | healthy | low | depleted), and a human renewal date.
- agent/billing_usage.py: UsageModel/UsageBar, usage_model_from_account
(fail-open), build_usage_model (HERMES_DEV_CREDITS_FIXTURE-aware),
format_renews (ISO -> "Jul 24, 2026", Windows-safe), $5 low-balance threshold.
- tui_gateway/server.py: _serialize_usage_model/_serialize_usage_bar, a
usage.bars RPC, and the model embedded into subscription.state so the overlay
renders the same bars from its single fetch.
- Dollars only, never "credits"; two separate bars (not a crammed
three-segment one) for legibility at terminal widths.
- tests/agent/test_billing_usage.py: status classification, bar math
(clamp/over-cap), NaN/Inf rejection, fail-open invariants.
* feat(tui): dollar usage bars on /usage + /subscription, drop tier picker
Render the shared two-bar dollar model in both overlays; strip "credits" and
the in-terminal tier selection per UX feedback.
- overlayPrimitives.tsx: UsageBars (themed plan/top-up bars — gold allowance,
green top-up) + usageBarsText for the /usage panel. Plan name labels the
bar; "$X left of $Y · N% used" (disambiguated so the % matches); top-up
"never expires".
- subscriptionOverlay.tsx: status line dedupes ($X left once; bar carries the
breakdown), human renewal date, state-matched nudges (free upsell / <$5
low alert) with box-safe ASCII markers (! / >) instead of the width-unstable
emoji that broke the border. Tier picker removed — overview shows usage +
plan, then "Manage on portal" / "Close" (free users get "Start a
subscription"). No "credits" anywhere.
- session.ts: /usage renders the dollar bars + balance summary, falling back
to the legacy credits lines only when the model is unavailable; CTA reworded.
- gatewayTypes.ts: UsageModelData/UsageBarData wire types + usage on
SessionUsageResponse/SubscriptionStateResponse.
- Tests updated to the new contract (no "credits", "left of", dedup, markers).
* feat(cli): mirror dollar usage bars on /usage + /subscription
CLI parity with the TUI billing rework, from the same shared usage model.
- _print_nous_credits_block (/usage) and _subscription_overview render the
two-bar dollar view (plan name on the bar, "$X left of $Y · N% used",
top-up "never expires", total spendable) instead of the credits-worded block.
- Dollars only — dropped the tier catalog (no more "$N/mo (… credits)") and
every user-facing "credits"; team copy says "shared balance".
- Human renewal date via the shared format_renews; status line dedupes the
"$X left"; free upsell + <$5 low alert with ASCII markers.
- /subscription manage modal no longer dumps the raw manage-subscription URL
in its detail — the [1] Open / [2] Copy link / [3] Cancel options carry it.
Title is "Manage your subscription" (no in-terminal plan change). The raw URL
stays only in the non-interactive / not-admin fallbacks, which have no menu.
- /usage token-usage panel (model, tokens, cost, context) left untouched.
* feat(billing): embed dollar usage model into billing.state for /topup
The /topup overview renders the same two-bar dollar usage (plan + top-up) as
/usage and /subscription. Embed the shared usage model into the billing.state
RPC payload (mirrors subscription.state) so the overlay gets the bars from its
single fetch, and add the `usage` field to BillingStateResponse.
* feat(tui/topup): reorder overview + in-flight reauth with press-Enter resume
Reworks the /topup overlay per the Jun 19 review and the no-preflight decision.
Overview:
- Balance leads in the title ("Top up · balance $X"); the shared two-bar dollar
usage (plan + top-up) renders below. Dropped the old monthly-cap spend bar.
- "Add funds" is the first action (was "Buy credits"); auto-reload / monthly
limit / manage-on-portal follow. Dollars only — no "credits" anywhere.
- No "Enable terminal billing" menu item and NO scope preflight: whether the
terminal can charge is discovered reactively at pay time. (We deliberately do
not read/refresh the OAuth token to gate UI.)
Step-up (reached only on a charge's insufficient_scope 403):
- New 4-phase flow that keeps the modal mounted: prompt (one-time-setup
heads-up) → waiting (browser authorize) → granted (explicit "Press Enter to
resume") → replay the held charge → settle. The press-Enter beat is the
reassuring "you're back, finish your purchase" moment.
- Renamed user copy "Allow Remote Spending" → "Enable terminal billing"; never
leaks the raw billing:manage scope (guarded by the render test).
- topup.ts error copy de-crufted to terminal-billing wording, emoji removed.
Tests: step-up prompt copy, the no-raw-scope invariant, and new overview tests
(balance-in-title, Add-funds-first, two-bar usage, no "credits").
* feat(cli/topup): mirror overview reorder + in-flight reauth resume
CLI parity with the TUI /topup rehaul, from the same shared usage model.
- _billing_overview: balance in the title, the two-bar dollar usage (plan name
on the plan bar, top-up "never expires") in place of the old cap spend bar,
"Add funds" first, dollars throughout — no "credits", no scope preflight.
- _billing_handle_scope_required: now takes the held amount + idempotency key
and runs the in-flight flow — "Enable terminal billing" → browser device-flow
→ re-check the org kill-switch → press-Enter to resume → replay the held
charge (reusing the key so a double-submit collapses to one). Stops leaking
the raw billing:manage scope.
- Charge-error + buy/auto-reload copy de-crufted to terminal-billing/dollars.
- Tests updated to the new overview + buy copy.
* fix(billing): guard non-JSON 2xx responses in the billing HTTP client
A 2xx response with a non-JSON body — e.g. a reverse-proxy / SPA fallback HTML
page served when a billing route isn't actually mounted on a deployment — hit
json.loads() on the success path of _request() and raised a raw
json.JSONDecodeError. That escaped the typed-BillingError contract, so callers'
`except BillingError` missed it and fell through to a generic fail-open that
rendered as a misleading "not logged in" (observed when /api/billing/subscription
was briefly unshipped on staging: 200 text/html, x-matched-path /[...notFound]).
Now a non-JSON 2xx body raises a typed BillingError(error="endpoint_unavailable")
so surfaces degrade gracefully ("could not load …") instead of crashing or
mislabeling a valid session as logged-out. The 4xx/5xx path already guarded its
.json(); this closes the same hole on the success path.
Test: tests/hermes_cli/test_nous_billing_request.py — non-JSON 2xx → typed
error (not JSONDecodeError, not BillingAuthError), empty body → {}, valid JSON
parses.
* feat(billing/dev): add HERMES_DEV_BILLING_FIXTURE for offline card/scope testing
build_billing_state short-circuits to a fixture when HERMES_DEV_BILLING_FIXTURE
is set (mirrors HERMES_DEV_CREDITS_FIXTURE for the usage model). States:
nocard | card | card-autoreload | notadmin | billing-off | logged-out — so the
card-on-file gate, admin role, and kill-switch paths are exercisable offline
without a live portal. Env-var gated; returns None when unset (no prod leak).
Adds 8 behavior tests asserting the card/admin/billing-on contract per state.
* refactor(billing): fold /credits into /topup
/credits is redundant now that /topup shows the dollar balance + portal handoff.
Make 'credits' (and 'billing') aliases of /topup so typing /credits still works,
resolving to topup everywhere (CLI, gateway, Slack, TUI, autocomplete, help).
Remove the standalone /credits surface across 6 places:
- CLI _show_credits handler + dispatch
- gateway _handle_credits_command -> renamed _handle_topup_command, copy softened
to 'Manage billing on the portal' (the messaging billing surface; /topup is now
gateway-available so messaging keeps billing — credits was the only one before)
- TUI commands/credits.ts + creditsCommand.test.ts (deleted), registry entry
- tui_gateway credits.view RPC + the CreditsViewResponse type
- Slack _SLACK_VIA_HERMES_ONLY: credits -> topup
Sweep user-facing /credits -> /topup (usage-block hint, depletion notice) and
stale doc-comments. OpenRouter's /credits endpoint URL left untouched. Tests
updated (test_credits_folds_into_topup) or pruned for the removed symbols.
* fix(billing): card-on-file heads-up, no-card portal gate, /usage bar ordering, modal glyph
In-terminal charge (POST /charge against the org's server-held card, no card ref
leaves the client):
- card present: confirm screen shows 'Your card saved on the portal will be
charged' + a 'Manage on portal' escape option (CLI); heads-up line (TUI)
- no card on file: /topup overview + buy flow detect it and route to the portal
to add a card, instead of offering a charge that 403s no_payment_method
/usage bar ordering: route the dollar block through _cprint consistently. The
Plan: line (_cprint) and the bar (raw print) flushed to different buffers under
patch_stdout and interleaved nondeterministically; now Plan: -> bar -> status/CTA
is stable across all states.
Modal glyph: strip the leading emoji from bordered _prompt_text_input_modal
titles — it measures 1 char but renders 2 columns, shifting the box's right
border (the stray '|'). Includes the f-string 'Pay $X?' title.
Small /credits -> /topup string bits in cli.py ride along with the surrounding
charge edits (the fold lives in the sibling refactor commit).
* refactor(billing): apply safe simplify-pass fixes
Three low-risk cleanups from a parallel simplify review (reuse/quality/efficiency):
- dev fixture portal URL: reuse the prod host (was drifted to staging-* — a real
mismatch vs subscription_view's _DEV_FIXTURE_PORTAL)
- TUI billingOverlay choose(): collapse two byte-identical branches (needsCard +
the not-full else both = portal-or-close at index 0) into one tail; the only
divergent path (full && !needsCard → buy/auto/limit) stays explicit
- /topup overview comment: correct the stale 'buy_flow detects no_payment_method'
note (the overview's no-card gate fires first, so reaching Add funds implies a
card on file)
Skipped (judgment): the orphaned CreditsView.depleted field (harmless, on a live
dataclass), the defensive card gates in _billing_buy_flow/_confirm_and_charge
(cheap correct defense on the money path), and folding the no-card handoff into a
shared helper (touches 4 money-path sites for tidiness — not worth the risk here).
* fix(billing): reactive charge gating — drop card preflight, react to 403 (scope→reauth, no-card→portal)
* refactor(billing): drop the /credits alias entirely
The /credits fold made it an alias of /topup; now remove that too. Typing
/credits is an unknown command, not a silent redirect — billing lives only on
/topup (with /billing kept as the old command's back-compat name). Dropped the
alias from the registry CommandDef and the TUI topup.ts; updated the test to
assert /credits resolves to nothing (no command, no alias).
* docs(billing): fix stale comment in _billing_overview — describe reactive no-card path
The comment still described the removed overview-level card gate ('no-card case
handled above'). Corrected to: the buy flow reacts to the server's
no_payment_method 403 and hands off to the portal at charge time (no preflight).
* refactor(billing): simplify-pass — share usage-payload helper, drop dead bar wire fields + redundant admin gate
* refactor(billing): drop the /billing alias too — /topup is the only billing command
Following /credits removal, retire the old /billing name as well. /topup now has
NO aliases — both /credits and /billing are unknown commands. Dropped the alias
from the registry CommandDef and TUI topup.ts; fixed the one live user-facing
straggler (the not-logged-in message said 'then /billing' → /topup) and the
_show_billing docstring/default-arg references. Test asserts /topup carries no
aliases and neither old name resolves.
* fix(billing): code-review fixes — money-path + parity bugs
Money path (TUI):
- auto-reload "Turn off" now echoes current threshold/top_up_amount so the
PATCH succeeds (was sending {enabled:false} → invalid_request → stayed ON)
- charge poll honors the 5-min cap on the 429/503 throttle branch too (was
rescheduling forever); cap folded into one timedOut() helper
- step-up resume reacts to the replay outcome instead of unconditionally
closing on a reassuring line with no charge made
- synchronous submit guard on Confirm so two key events can't double-charge
Gateway:
- billing.step_up routes typed errors through _serialize_billing_error (was a
raw {error:'error'} dict → generic copy for session_revoked)
- billing.state / subscription.state / usage.bars / session.usage moved to
_LONG_HANDLERS (blocking portal HTTP no longer stalls the main stdin loop)
CLI:
- _billing_render_charge_error handles insufficient_scope without leaking the
raw billing:manage scope name on a post-grant replay re-raise
Python model:
- subscription_view tier parse None-coalesces tierOrder/dollarsPerMonth so a
free tier's 0 survives ($0, not "—"; correct sort order)
TUI parity/robustness:
- /usage shows formatted renews_display, not raw ISO renews_at
- subscription overview guards a null pending_downgrade_at (was "on null.")
- subscription overview surfaces a message instead of silently closing when
portal_url is missing
- buildManageUrl wraps new URL() so a malformed portal_url can't throw out of
the Ink key handler
* fix(billing): cross-surface bar direction, formatted cancel/downgrade dates, Slack alias gating
- CLI plan bar now fills by REMAINING (fuel-gauge), matching the shared model's
fill_fraction, the top-up bar, and the TUI — same account renders identically
on both surfaces (#8)
- subscription serializer emits cancellation_effective_display /
pending_downgrade_display (format_renews); TUI shows 'Jul 1, 2026' not raw ISO (#14b)
- _SLACK_VIA_HERMES_ONLY now includes the 'billing' alias so it follows its
canonical /topup via /hermes instead of leaking a native Slack slot (#9)
* fix(billing): thread idempotency key through the TUI step-up replay (#2)
Mint a stable idempotency key when the purchase amount is chosen; it rides
pendingCharge into both the Confirm charge and the post-grant step-up replay,
so a retried charge dedups server-side (the gateway already echoes the key).
A fresh amount selection gets a fresh key. Combined with the sync submit guard,
a double-submit now collapses to one charge.
* refactor(billing): remove dead /subscription tier-picker scaffolding (#18)
The in-terminal plan picker was cut (deep-link only), leaving a whole unreached
state machine. Removed end-to-end:
- TUI: ConfirmScreen, HandoffScreen, the 'confirm'/'handoff' screen types,
pendingTargetTierId, and the now-dead onPatch threading (collapsed the dispatch
to a single overview screen + folded the duplicate Box wrapper)
- gateway: the tiers serialization + SubscriptionTierOption wire type
- model: SubscriptionTier, _parse_tier, _coalesce, _dev_tiers and the tiers field
(never displayed on either surface, so this supersedes the tier-parse fix)
- tests: dropped the confirm/handoff/tier-passthrough tests; slimmed the overview
render tests
Net: a large dead-code cull (no behavior change — the picker never ran).
* test(billing): parametrize usage-model tests; drop dead is_low/is_free props
Collapse the fail-open + status-classification cases into parametrized tables
(same coverage, ~80 fewer lines) and remove the now-unused UsageModel.is_low /
is_free properties (only a test pinned them).
* fix(billing): revert dead 'billing' Slack-via-hermes entry — the alias was dropped
#9 was based on a stale review diff: /billing is no longer an alias of /topup
(dropped earlier), so routing it via /hermes filtered a name that doesn't exist.
* test(billing): cull redundant TUI billing tests (parametrize, merge dupes)
usageCommand: collapse 3 CTA tests into one + a panel helper.
billingStepUp: merge the two step-up render asserts.
topupCommand: parametrize requestRemoteSpending + the revoked-actor pair, drop
the redundant happy-path-submitted test. Money-path + error-mapping coverage
preserved.
* refactor(billing): extract _usage_bar_lines — one source of truth for the CLI bars
The plan + top-up bar format was copy-pasted across _print_nous_credits_block,
_subscription_overview, and _billing_overview. Extract a helper returning the
ready-to-print lines; each caller keeps its own print fn (the _cprint-ordering
constraint stays) and resolves its plan-name label. Centralizes the format so
the three surfaces can't drift.
* feat(billing): NAS V3 subscription-change HTTP client wrappers
Add the four write-side wrappers for the V3 subscription contract to nous_billing,
each a thin _request() call (reusing auth, JSON, 401-retry, typed errors):
- post_subscription_preview → POST /subscription/preview (chargeless quote)
- put_subscription_pending_change→ PUT /subscription/pending-change (downgrade/cancel)
- delete_subscription_pending_change → DELETE .../pending-change (resume/undo)
- post_subscription_upgrade → POST /subscription/upgrade (the money route)
pending-change takes a discriminated body (tier_change | cancellation); upgrade
requires an Idempotency-Key (mandatory, validated client-side before any I/O).
Tests assert the exact method/path/body/header each wrapper puts on the wire.
* feat(billing): subscription tier catalog + change-preview models
Reinstate the catalog the in-terminal picker needs (was culled when /subscription
was deep-link-only): SubscriptionTier + SubscriptionState.tiers + _parse_tier, with
_coalesce so the free tier's 0 tierOrder/price survives a falsy-or. Parse the
catalog from GET /subscription's tiers and seed _dev_tiers into every fixture.
Add SubscriptionChangePreview + subscription_change_preview_from_payload for the
POST /preview quote (effect/amountDueNowCents/effectiveAt/reason + tier delta); a
malformed/missing effect fails safe to 'blocked' so a bad quote never reads as a
charge. Module docstring updated: the overlay is no longer deep-link-only.
* feat(billing): gateway RPCs for the V3 subscription change flow
Add subscription.preview / .change / .resume / .upgrade RPCs, each wrapping its
nous_billing call and reusing _serialize_billing_error for the typed envelope
(so a 403 still drives the device step-up). upgrade mints + echoes the
idempotency key and surfaces status + recovery_url so the TUI can route an
SCA/decline to the portal. Re-add the tier catalog to _serialize_subscription_state
(price pre-formatted) for the picker. All four are pool-routed (_LONG_HANDLERS) —
preview + upgrade hit Stripe and must not stall the main stdin loop.
* feat(billing): in-terminal subscription change flow (TUI)
/subscription is no longer deep-link-only: it drives the change in-terminal
against the V3 contract via the new gateway RPCs. The overlay is a state machine
overview → picker → confirm → result:
- picker lists the tier catalog with upgrade/downgrade hints (current + free
excluded; free=cancel, on the overview);
- confirm shows the previewed effect — pay $X now (upgrade) / scheduled at date
(downgrade) / cancel at period end / blocked-with-reason — then applies it;
- an upgrade's SCA/decline routes to the portal via the result screen's recovery
link; resume/cancel/downgrade are chargeless.
Starting a NEW subscription still deep-links (needs a fresh card). insufficient_scope
points to /topup (the step-up stays there, not duplicated here). Adds the wire
types (tiers + preview/upgrade responses), widens the overlay ctx + screen state,
and threads onPatch. Render tests cover every screen.
* feat(billing): in-terminal step-up + clearer scheduled-change UX (TUI)
Two improvements to the /subscription overlay:
Step-up re-auth in place. When a mutation (preview/change/upgrade/resume) returns
insufficient_scope, route to a new 'stepup' screen that grants terminal billing
via billing.step_up and AUTO-REPLAYS the held action on grant — no bounce to
/topup. Scope routing is centralized in previewAndRoute/applyPendingAndRoute/
resumeAndRoute (shared by the picker, confirm, overview + the step-up replay). The
browser opens via the shared global verification handler; copy never leaks the raw
billing:manage scope.
Make a scheduled change unmissable. A downgrade/cancel was one buried warn line
that read as 'nothing happened'. Now the overview leads with a banner
(⏳ Scheduled change · Ultra ──▶ Plus · <date> · you keep Ultra until then), the
status line echoes the transition (Plan: Ultra → Plus), 'Keep <tier> (undo)' is
promoted to the first olive action, the result screen says 'your plan doesn't
change today', and confirm gets a charged-now / scheduled chip.
* feat(billing): full in-terminal subscription change flow in the classic CLI
Bring the CLI to parity with the TUI overlay — /subscription is no longer
deep-link-only. A paid admin/owner gets picker → preview → confirm → apply,
mirroring the /topup buy flow's modal idioms:
- _subscription_change_menu (change / undo-or-cancel / manage-on-portal),
- _subscription_pick_tier (catalog with upgrade/downgrade hints),
- _subscription_preview_and_confirm (POST /preview → effect-aware confirm),
- _subscription_apply (schedule / cancel / resume chargeless; upgrade charges
the sub's card, SCA/decline → portal),
- _subscription_handle_scope_required (insufficient_scope → step_up_nous_billing_scope
inline, then replays the held preview/mutation — reusing the upgrade idempotency key).
Also the scheduled-change UX fix: the overview leads with a prominent banner
(⏳ Scheduled change · Super ──▶ Plus · <date> · you keep Super until then) and the
status line echoes the transition, matching the TUI. Members / non-interactive /
free still deep-link. Tests drive every branch via a mocked modal + nous_billing.
* fix(billing): close TUI subscription money-path holes (ultracode review)
- Un-consented charge (P1): the step-up now HOLDS at a 'granted' phase requiring
an explicit Continue, and an abortedRef gates the grant's late .then — a cancel
during the browser flow can no longer replay the held upgrade + charge.
- Missing idempotency key (P2): mint it when building an upgrade 'pending' so it
rides into confirm AND the step-up replay (was always undefined → gateway minted
a fresh key per call, defeating dedup).
- Navigate-away re-charge (P2): confirm 'back' is guarded by submittingRef while an
apply is in flight.
- Ambiguous charge (P2): a transport-null upgrade is reported as 'may or may not
have charged — re-check', never a flat failure that invites a blind retry.
- Typed step-up denial (P2): requestRemoteSpending returns {granted,error,message};
the screen maps session_revoked / remote_spending_revoked / rate_limited to the
right recovery instead of always 'an admin must allow it'.
* fix(billing): close CLI subscription money-path holes (ultracode review)
- Bounded step-up (P2): bust the 30s token cache after a grant (it held the
pre-grant unscoped token; _request only busts on 401, not 403) and replay ONCE
with allow_stepup=False so a still-denied scope can't re-prompt/re-open in a loop.
- Stray-keystroke charge (P3→near-P2): the upgrade confirm defaults to 'Go back',
not 'Pay ' — a bare Enter can't move money.
- Fail-open on unknown effect (P3→near-P2): an unrecognized preview effect now
fails SAFE (portal hand-off) instead of scheduling a real PUT.
- 'cancel' word collision (P3): the Close row uses value 'close' so typing 'cancel'
can't hit it and falsely report 'Cancelled'.
- blocked effect re-offers the portal; undo is promoted to the first row when a
change is pending (TUI parity).
* fix(billing): guard the step-up resume against double-fire (2nd ultracode pass, BUG A)
The P1 fix split the auto-replay into a user-triggered resume() on the granted
screen, where the default row is the charging action — but resume() had no
re-entrancy guard, so a double-Enter fired two replays (the upgrade dedups on the
shared key, but schedule/cancel/resume replays carry none → duplicate PUT/DELETEs).
Mirror billingOverlay.resume(): flip to a 'resuming' phase + a resumingRef so it
fires at most once, and block 'back' once resuming (no re-mount → no second submit).
* fix(billing): CLI charge-route ambiguous-charge caveat (2nd ultracode pass, BUG B)
The TUI hardened upgradeResult(null) but the CLI charging route did not: a
transport/timeout/500 (or unknown 2xx status) on post_subscription_upgrade — after
NAS may have already prorated + charged — printed a flat failure, and a manual
re-run mints a FRESH idempotency key the server can't dedup → a real second charge.
Now the charge route reports 'your card may or may not have been charged — re-run
/subscription to check before trying again' and steers away from a blind retry
(the CLI can't persist the key across a command re-run). Also thread allow_stepup
through the preview→apply replay (BUG C.1) and route the requires_action/
payment_failed portal lines through _cprint for deterministic ordering.
* fix(billing): cap the TUI step-up replay to avoid a resume-deadlock (final pass, R1)
The round-2 resume guard ('resuming' phase + resumingRef) could deadlock: on a
REPEAT insufficient_scope during the post-grant replay, the route helpers did
onPatch({screen:'stepup'}) — a no-op since we're already mounted on stepup (no key
→ no remount) — leaving phase='resuming'/resumingRef=true frozen on 'Applying your
change…'. Thread allowStepUp through previewAndRoute/applyPendingAndRoute/
resumeAndRoute; the resume() replay passes false, so a repeat scope denial surfaces
a 'still isn't enabled' result instead (mirrors the CLI's allow_stepup=False cap).
Also: applyPendingAndRoute(pending=null) now routes to overview, not a stranded
Promise.resolve().
* fix(billing): narrow the CLI ambiguous-charge catch to indeterminate outcomes (final pass, R2)
The round-2 fix caught EVERY non-scope BillingError as 'may or may not have been
charged' — but typed pre-charge rejections (BillingRateLimited 429, BillingSessionRevoked
401, BillingRemoteSpendingRevoked 403, role_required/no_payment_method 4xx) never
reached Stripe, so the ambiguity copy was wrong and dropped their real recovery hints.
Now route those to _subscription_render_error, and reserve the ambiguous copy for
genuinely indeterminate outcomes (network_error / endpoint_unavailable / status None /
5xx). Tests: rate-limit stays deterministic; a real transport failure stays ambiguous.
* feat(billing): card visibility + guided add-card path in /topup and /subscription
Consume the NAS card-resolver contract (card.resolvedVia + chargeability) across
both surfaces, degrading cleanly on today's NAS (fields absent → prior behavior):
- WHICH card: the payment lines render provenance — 'Visa ····4242 — the card on
your subscription' (resolvedVia → label; unknown rung/older NAS → masked card +
the old generic line). Link payment methods render the brand alone (last4 is
empty — never 'Link ····').
- Presence at a glance: the /topup overview now shows 'Card: …' or 'No saved
card on file' for the full-menu case, plus a warning when the resolver marks
the card needs_repair (failing auto-reloads) on overview/buy/confirm.
- Add-card path: with no card on file, 'Add funds' becomes a guided screen —
open the portal billing page, then 'I've added it — check again' re-fetches
billing state and continues straight into the purchase (also recovers a
transient display miss). Cards are never entered in-terminal.
- /subscription upgrade confirm names the exact card ('Visa ····4242 — the card
on your subscription — will be charged'), best-effort via billing.state and
only when the resolution rung matches what a subscription charge actually
uses (subPin/customerDefault, mirroring Stripe's precedence); otherwise the
generic line stands. Fail-soft: any lookup error keeps the generic line.
- Gateway serializes display/resolved_via/needs_repair; TUI ctx gains
refreshState (topup) + fetchCard (subscription); new offline fixtures
card-sub / card-repair.
Tests: TUI ctx mocks extended; CLI suites cover provenance + repair-warning
render, the Link guard, the add-card path (continue-after-recheck + abandon),
the sub-confirm card line, and keep the confirm-time lookup offline in tests.
* fix(billing): consume server canChangePlan, preserve distinct refusal codes, drop dead chargeability
- Parse canChangePlan verbatim from NAS payloads into BillingState and
SubscriptionState; fall back to the legacy OWNER/ADMIN check only when the
server omits the field (FINANCE_ADMIN stops being locked out where NAS
authorizes it). Role model updated to the 5-role enum.
- Add the autoReload.card union (canonical | distinct | none) end-to-end:
parse + gateway serialization, distinct carries payment_method_id/brand/last4
with nullable display fields.
- stripe_unavailable (503, transient) and upgrade_cap_exceeded (429, daily cap)
now survive to the wire as their own codes instead of collapsing into
rate_limited; new exception types subclass BillingRateLimited so existing
backoff call sites keep working.
- Remove card.chargeability / needs_repair parsing, serialization, fixtures and
the cli warning blocks: NAS #670 removed the field, so the repair path was
permanently dead. The future card-health signal belongs to the NAS W1/W3 work.
- Tests: five-role fixtures, canChangePlan override/fallback, all three
auto-reload card variants, 429-vs-503 code preservation end-to-end.
* feat(tui): render the full NAS billing refusal surface
- billingOverlay: divergence notice when auto-refill charges a distinct card
(portal deep-link to reconcile); needs_repair warnings removed with the field.
- topup: explicit copy for consent_required, org_access_denied,
upgrade_cap_exceeded, auto_top_up_disabled_failures and stripe_unavailable
(honors retry_after); processing_error is an explicit charge-failure case;
transport loss during charge polling now reads as an unconfirmed outcome
(check balance before retrying), matching the revocation path.
- subscriptionOverlay: branch on upgrade reason, not status, so an SCA-needing
upgrade routes to portal verification even while NAS pre-#711 labels it
payment_failed; after an upgrade, poll subscription state until the tier
flips (bounded), rendering applying/still-applying rather than assuming
immediacy.
- Capability-neutral refusal copy (owner, admin, or finance admin) replaces
the stale org admin/owner wording.
- gatewayTypes: BillingAutoReload.card union added, needs_repair removed.
* docs(billing): client-side billing state and refusal lifecycle table
Enumerates, from the code, every billing.state shape and typed refusal the
gateway serves and the exact TUI copy + recovery each renders. Acceptance from
the billing-integration handoff: no NAS billing state or typed refusal falls
through to a generic toast; unknown codes still degrade to the default branch
that surfaces the server message.
`_prepare_inbound_message_text` (async) called `_decide_image_input_mode`
inline for every inbound image. That decision is synchronous and does
blocking network I/O on the way to a capability answer:
- `agent.models_dev.fetch_models_dev` — an HTTP GET to models.dev (15s
timeout) whenever the 1-hour in-memory cache is cold or models.dev is slow.
- `agent.model_metadata.query_ollama_supports_vision` — HTTP probes
(`detect_local_server_type` + `/api/show`) against a local Ollama server
when the active provider fronts one.
Running that inline blocks the gateway event loop for up to the request
timeout — so a single user attaching an image freezes EVERY session on that
gateway (no other messages processed, no heartbeats) until the fetch/probe
returns or times out. This is the same off-the-loop class as the cron-fire
verifier and the async_is_safe_url work.
Wrap the call in `asyncio.to_thread` so the blocking capability lookup runs
on a worker thread and the loop stays responsive. The decision result and
routing are unchanged.
Test: a gateway image-routing runtime test asserts the capability lookup runs
off the main (event-loop) thread; it runs on the main thread before the fix.
run_agent._dispatch_delegate_task forces background=True for every top-level
delegation, and async_delivery_supported() returns True for any session that
never binds the capability. On runners that cannot receive a completion after
their turn ends, that combination silently discards every subagent result: the
model gets a dispatch handle, ends its turn, and reports 'waiting for results'.
Two such runners never bind the capability:
* hermes -z (one-shot) prints one final response and exits. It bypasses cli.py,
so nothing drains process_registry.completion_queue (only the interactive
process_loop and the gateway watchers do).
* cron run_job clears the HERMES_SESSION_* routing keys, so a completion event
carries session_key="" — _enrich_async_delegation_routing cannot resolve it
and _inject_watch_notification drops it ("no routing metadata"). By then
run_job has already shipped the job's final response via _deliver_result;
there is no turn left to re-enter. Worse, get_current_session_key() can fall
back to the ambient os.environ HERMES_SESSION_KEY, so a cron subagent's output
can be routed into an unrelated user chat rather than merely dropped.
Add declare_stateless_channel() and bind it in both runners, routing
delegate_task to its existing inline/synchronous path — the same fallback the
stateless HTTP adapter already relies on, and the fix suggested in #63142. The
helper binds only the capability: set_session_vars() would also latch
_session_context_engaged, which a pure single-process one-shot must not trigger.
Also correct two agent-facing strings that hardcoded 'stateless HTTP API' as the
only channel without async delivery (delegate_tool, terminal_tool); they now name
the actual condition.
Repro (before): hermes -z 'Use delegate_task to spawn a subagent that replies
BANANA. Report its reply.' -> "Waiting for the subagent's response...", exit 0,
no BANANA. After: BANANA is returned in-turn.
Fixes#53027Fixes#63142
Same bug family as #47828, at the config-persistence layer instead of
the in-memory agent layer:
- cli.py (#25106): the --global /model handlers (both the typed-name
path in _handle_model_switch and the picker path in
_apply_model_switch_result) wrote model.default/model.provider to
config.yaml but never touched base_url/api_mode at all. A provider
switch left the OLD endpoint on disk; the next launch reconnected to
the previous provider's host under the new model name.
- gateway/slash_commands.py (#25107): both persist-global blocks (the
picker-tap callback and the typed /model --global path) guarded the
write with two INDEPENDENT ifs — `if result.base_url: ...` and
`if target_provider != "custom": clear_model_endpoint_credentials(...)`.
For named providers the second if always cleared stale values, masking
the bug. For a custom provider with an empty resolved base_url, neither
branch fired, so the previous custom endpoint's base_url/api_key/
api_mode survived untouched in config.yaml.
Fix: explicit set-if-truthy/clear-if-falsy for base_url and api_mode at
all four call sites, matching the already-correct pattern in
tui_gateway/server.py:_persist_model_switch (fixed for #48305).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Follow-up fixes for salvaged PR #65637:
1. Clear _dirty_transcripts in rewrite_transcript + rewind_session —
stale pending messages were re-inserted after /retry, /undo, /compress.
2. Narrow _is_fts_corruption_error to specific SQLite error strings —
bare 'fts' substring matched 'shifts', 'gifts', etc.
3. Move DB write outside _transcript_retry_lock — holding the lock
during writes serialized all sessions' transcript appends and blocked
during FTS rebuild. Now the lock guards only the pending queue.
4. Push rebuild_fts() into SessionDB — SessionStore was reaching into
_conn/_lock private attrs. SessionDB.rebuild_fts() follows the same
pattern as optimize_fts().
5. Cap pending per session at 200 — prevents unbounded memory growth
when DB is persistently broken. Oldest messages dropped with warning.
Added 4 new tests: dirty-clear on rewrite/rewind, FTS matcher false
positives, pending cap enforcement.
The previous commit fixes the canonical agent/skill_utils.parse_frontmatter.
Six more modules reimplement the '---' fence check locally and had the
same bug:
- tools/skill_manager_tool.py _validate_frontmatter — rejected BOM'd
skill_manage create/edit content outright
- tools/skills_hub.py GitHubSource._parse_frontmatter_quick and
OptionalSkillSource._parse_frontmatter — hub browse/install metadata
- hermes_cli/skills_hub.py — local skill install validation
- gateway/run.py — skill slug discovery for disabled-skill hints
- agent/prompt_builder.py _strip_yaml_frontmatter — BOM'd context files
(AGENTS.md) leaked raw frontmatter into the system prompt
- tools/blueprints.py _split_frontmatter — str.lstrip() does not strip
U+FEFF (not whitespace), so the existing lstrip never covered it
Sibling-surface regression tests added.
Bug class also fixed upstream in cline/cline#12218 (found by the weekly
Cline PR scout).
Unifies the two gateway subsystems that were fighting each other: the
'never lose a session' recovery machinery (#54878 stale-route self-heal,
find_latest_gateway_session_for_peer reopening agent_close/ws_orphan_reap
rows) and the session reset/expiry machinery (expiry watcher, /new,
/resume, resume_pending freshness gate).
The unified contract:
- INTENTIONAL boundaries (expiry finalization, auto-reset, /new,
/resume switch) are recorded durably via promote_to_session_reset(),
which upgrades accidental recoverable end_reasons (agent_close,
ws_orphan_reap) to the explicit boundary while preserving other
explicit reasons (compression, etc.). Recovery then correctly refuses
to resurrect them.
- ACCIDENTAL ends (crash, cleanup bug, mistaken reaper) stay
recoverable — genuine crash recovery is untouched.
On top of the cherry-picked contributor commits:
- promote_to_session_reset widened to ws_orphan_reap + parameterized
reason so auto-reset paths stay auditable (idle/daily/suspended/
resume_pending_expired) (#61220, #61993, #63539)
- get_or_create_session auto-reset, reset_session (/new), and
switch_session (/resume) all write through the promote path — the
first-reason-wins end_session no-op could previously leave a reset
session resurrectable behind a stale agent_close row (#61993)
- resume_pending freshness gate now honors session_reset.mode=none:
explicit opt-out of automatic resets also opts out of the zombie
gate (#61052)
- resume recovery note extracted to build_resume_recovery_note() and
made adapter-aware via a new interactive_resume capability flag:
webhook/api_server auto-resume turns now CONTINUE the interrupted
task instead of emitting an unanswerable 'session restored'
acknowledgement that abandoned the work (#57056)
- tests updated to call the real note builder instead of mirroring it
E2E-validated against a real SessionDB + SessionStore in a temp
HERMES_HOME: expiry->agent_close->no-resurrection, /new promote,
crash recovery preserved, mode=none opt-out, routing-table flag sync.
When a gateway session with resume_pending=True is not recovered within the
auto-continue freshness window (e.g. because repeated API calls timed out on a
large context), get_or_create_session correctly creates a new session. However
two gaps existed:
1. The user received no notification — resume_pending_expired fell through the
generic "inactive for Xh" else-branch in run.py, which produces wrong wording
and (for session_reset.mode: none users) is gated on policy.notify that
evaluates to False.
2. The old session was ended in state.db with the hardcoded generic reason
"session_reset", making it impossible to distinguish from a normal idle/daily
reset in post-mortem analysis.
Fix:
- gateway/run.py: add an explicit resume_pending_expired case for the agent
context note ("gateway restart recovery timed out") and the user-facing
notification. Always notify for this reason — like suspended — because the
user had an active session that was silently replaced.
- gateway/session.py: pass auto_reset_reason as the DB end_reason instead of
the hardcoded "session_reset", so all auto-reset paths are auditable.
- tests: extend TestResumePendingExpiredAutoReset in test_session_reset_notify.py
with five new cases that cover the reason, activity flag, DB end_reason,
non-regression of the idle path, and freshness-disabled bypass.
Closes#58933
Co-authored-by: Cursor <cursoragent@cursor.com>
When the session expiry watcher finalizes a session (daily/idle reset)
and the next message triggers the #54878 self-healing path
(get_or_create_session detects sessions.json / state.db mismatch),
the recovered session was silently reopened without checking whether
it should have been reset. This caused sessions to persist indefinitely
across reset boundaries.
Fix: after dropping the stale sessions.json entry in the self-healing
path, call _should_reset() against the old entry's updated_at. If a
reset is due, set db_end_session_id to skip DB recovery and create a
fresh session — matching the normal reset flow.
Address review feedback on #63068:
1. Replace unconditional reopen_session() + end_session() with a
conditional promote_to_session_reset() method in SessionDB.
The new method only promotes live rows or rows ended with
agent_close — explicit boundaries (compression, session_reset,
new_command) are preserved via first-writer-wins semantics.
2. Rewrite tests to use real SessionDB instead of MagicMock:
- 7 unit tests for promote_to_session_reset edge cases
- 3 integration tests verifying the actual recovery contract
in find_latest_gateway_session_for_peer after promotion
Add a runtime_validator callback to generate_title() / auto_title_session()
/ maybe_auto_title(). Callers snapshot the session's model+provider when
spawning the background titler; the validator runs right before the LLM
request and skips it silently when the live runtime no longer matches —
so a stale title request can't reload a model that strict_single_load
already evicted after a user model switch. Fail-open: a raising validator
never disables titling.
Wired at all four call sites (cli, gateway, tui_gateway, acp_adapter).
Surgical reapply of PR #19137 (base was 8k+ commits stale; the original
patch predates the pinned-language prompts, the atomic-write helper, and
the moved TUI/ACP call sites). Original work by @Thatgfsj. Closes#19027.
Bare /reasoning and /fast now render a native one-tap picker on
picker-capable platforms, with automatic fallback to the text status
card everywhere else — parity with the /model picker UX.
- gateway/slash_commands.py: generic send_choice_picker capability gate
(detected on the adapter type, like send_model_picker); selection and
typed arguments flow through one shared application path so they can
never diverge; choices built from VALID_REASONING_EFFORTS so future
levels appear automatically
- telegram: flat inline-keyboard picker (cp:<idx> callbacks), authorized
users only (same gate as approval buttons)
- discord: ChoicePickerView select menu, auth mirrors ExecApprovalView,
2-minute timeout with expiry edit
- matrix: reaction-based picker; reaction set extended to 12 slots to
fit the full effort ladder + subcommands
- locales: picker_title + choice labels in all 16 languages
- docs: ADDING_A_PLATFORM.md capability table
Closes#61110.
When no agent is active, '/queue <prompt>' previously fell through
dispatch with its raw text intact instead of being treated as a
normal prompt. Rewrite event.text to the bare payload (mirroring the
/steer no-active-agent path just below) and return a usage hint when
the payload is empty.
Salvaged from PR #29290 (queue half only — the /footer mid-run
dispatch half already landed on main via #65521).
When multiplex_profiles is true, background tasks spawned by /background
command failed with UnscopedSecretError because _resolve_session_agent_runtime()
was called without a profile secret scope. This fix wraps the task in
_profile_runtime_scope, mirroring the pattern used by _run_agent.
Fixes#60726
The platform callback verifier can do blocking network I/O (e.g. the
google-chat adapter fetches Google signing certs on a cache miss), which
would stall the event loop if called inline. Run sync verifiers via
asyncio.to_thread (await coroutine verifiers directly), and treat a
crashing verifier as a 401 rather than a 500 through the dispatch path —
a broken verifier must never admit an event.
