Reads header bytes 28-31 after every COMMIT and compares against actual file size. Raises sqlite3.DatabaseError on torn-extend (actual_pages < page_count). Also sets PRAGMA wal_autocheckpoint=100 in connect().
Refs: #31208 (Bug E - same file, coordinate), #30973 (wal_autocheckpoint)
Refs: #30445, #30896, #30908 (corruption reports)
`detect_crashed_workers` calls `_pid_alive` on every `running` task whose
claim is held by this host. The check can transiently return False for a
freshly-spawned worker (fork → /proc-visibility lag, or reap-race
between SIGCHLD and parent reaping). When a second dispatcher ticks
inside that window it reclaims the task and spawns a duplicate worker.
Add `DEFAULT_CRASH_GRACE_SECONDS = 30` and an
`HERMES_KANBAN_CRASH_GRACE_SECONDS` env-var override.
`detect_crashed_workers` skips the liveness check when
`time.time() - started_at < grace`. The existing 15-minute claim TTL
still reclaims genuinely-crashed workers; grace only suppresses the
launch-window false positive.
`HERMES_KANBAN_CRASH_GRACE_SECONDS=0` is set on the `kanban_home`
fixture in `test_kanban_core_functionality.py` so existing tests that
assert immediate reclaim retain pre-fix semantics.
Companion to merged PR #23442 (`release_stale_claims`, closes#23025),
which addressed the same multi-dispatcher race in the stale-claim path.
Related: #20015 (`_pid_alive` false-negative behaviour),
When code inside a write_txn block raises an OperationalError that SQLite
has already auto-rolled-back (typical for disk I/O error,
database is locked, and database disk image is malformed), the
explicit ROLLBACK in write_txn.__exit__ itself raises
cannot rollback - no transaction is active and the secondary exception
replaces the original in the traceback. Operators see a misleading error
and lose the diagnostic information they need.
Swallow the rollback-time OperationalError so the caller always sees the
original cause.
Confirmed reproducer: tests/hermes_cli/test_kanban_db.py::
test_write_txn_preserves_original_exception_when_rollback_fails
apply_wal_with_fallback() treated "disk i/o error" as a permanent
WAL-incompatibility marker, identical to "locking protocol" (NFS) and
"not authorized" (FUSE). But EIO during PRAGMA journal_mode=WAL is
typically TRANSIENT — page-cache pressure, brief lock contention,
recoverable storage hiccups — not a permanent filesystem property.
Treating transient EIO as a permanent downgrade signal produces the
mixed-journal-mode-across-processes corruption pattern:
1. Process A opens kanban.db, hits transient EIO on the WAL pragma,
silently downgrades to journal_mode=DELETE.
2. Process B (no EIO) opens the same file moments later and
successfully sets journal_mode=WAL.
3. A writes rollback-journal frames while B writes WAL frames. SQLite
documents this as unsupported and corrupts the file:
https://www.sqlite.org/wal.html ("all connections to the same
database must use the same locking protocol").
This was the root cause of repeated kanban.db corruption on hosts with
multiple gateway processes plus CLI invocations against the same DB
(observed pattern: corruption shortly after gateway startup, after the
process logged "WAL journal_mode unsupported on this filesystem (disk
I/O error) — falling back to journal_mode=DELETE"). The fallback
warning told the truth — fallback DID happen — but the premise
("unsupported on this filesystem") was wrong; the EIO was a one-shot
event and sibling processes successfully used WAL.
Fix has two layers:
1. Remove "disk i/o error" from _WAL_INCOMPAT_MARKERS. EIO now re-raises
so callers can retry instead of silently corrupting the DB. The two
remaining markers ("locking protocol", "not authorized") are
deterministic per filesystem so they remain safe permanent-downgrade
signals.
2. Belt-and-suspenders: before downgrading on ANY marker match, peek the
on-disk journal mode. If the header says WAL, refuse to downgrade and
re-raise the original error. This guards against any future addition
to _WAL_INCOMPAT_MARKERS turning out to be transient in some
environment we haven't yet seen.
Tests:
- tests/test_hermes_state_wal_fallback.py:
* Flipped test_falls_back_on_disk_io_error → test_reraises_on_disk_io_error
asserting EIO is re-raised, not silently swallowed.
* Added test_does_not_downgrade_when_disk_says_wal covering the
on-disk-header safety guard for the existing legitimate markers.
- tests/hermes_cli/test_kanban_db.py:
* test_connect_falls_back_to_delete_on_locking_protocol now uses a
truly-fresh DB (instead of the kanban_home fixture which pre-inits
in WAL). On NFS the very first process touching the file legitimately
downgrades; on a file already in WAL the new guard correctly refuses.
A standalone reproducer lives at /tmp/kanban-stress/repro_bugD_eio_wal_downgrade.py
(not committed): without fix the DB silently flips from WAL to DELETE
mid-process; with fix the EIO surfaces and the file stays WAL.
Refs: Bug D in the kanban-corruption investigation series (Bugs A and C
shipped in ebe7374f3 and e02147d5e respectively). Bug D explains every
corruption incident this week including those that survived A's
single-dispatcher mitigation, because every CLI invocation is a
separate process whose WAL pragma can transiently fail.
Production corruption #6 left b-tree pages with zeroed headers but intact old cell content — the Bug E pattern. This fix applies three pragma calls on every connect():
- synchronous=FULL (was NORMAL): closes the WAL-checkpoint reordering window where a crash between WAL commit and main-DB write leaves a partially-written b-tree page header. Cost is <1ms per commit on local SSD; negligible at kanban write volume.
- secure_delete=ON: forces SQLite to zero freed page bytes on disk. If a torn write or hardware fault later corrupts a page, the underlying cell content is zero, so corruption is detectable and no stale rows can resurface as live data.
- cell_size_check=ON: adds a read-side guard so corrupt cells surface as errors at read time rather than as silent wrong-data returns.
All three are connection-scoped and re-applied on every connect(). secure_delete also writes a persistent flag into the DB header on the first call against a fresh DB, making the protection durable across processes for new DBs.
Tests added for all four required cases: each pragma active on a fresh connection, and all three re-applied after close+reopen. Also adds the required negative test (migration path does not reset pragmas).
Asserts that when hermes update runs on a fork whose local HEAD matches
origin/main but commit_count == 0, the early-return path still consults
_sync_with_upstream_if_needed() before printing "Already up to date!".
Locks in the fix from the parent commit so the upstream-sync call cannot
silently regress out of the commit_count == 0 branch.
DEFAULT_CODEX_MODELS shipped three slugs that the chatgpt.com Codex
backend rejects with HTTP 400 'The <slug> model is not supported when
using Codex with a ChatGPT account.' on every account tested live:
gpt-5.2-codex
gpt-5.1-codex-max
gpt-5.1-codex-mini
Live verified against https://chatgpt.com/backend-api/codex/models
which returns gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex,
gpt-5.3-codex-spark, gpt-5.2 for ChatGPT Pro accounts.
When _fetch_models_from_api fell back to DEFAULT_CODEX_MODELS (offline
first-run, transient API failure) the picker surfaced these dead slugs
and crashed on selection. The forward-compat synthesis table chained
them downstream too.
If OpenAI re-enables them on the OAuth-backed Codex backend, live
discovery will pick them up automatically — the defaults list is only
consulted when live discovery is unavailable.
Test fixture pivoted to use gpt-5.3-codex (templated by 4 entries) as
the synthesis driver so the forward-compat test still exercises the
synthesis path.
In profile mode, _load_provider_state previously returned None when a
provider was absent from the profile's auth.json — even if the user had
authenticated at the global root. This broke runtime credential resolvers
that read state directly (resolve_nous_access_token,
resolve_nous_runtime_credentials), causing profiles without their own
nous login to fail with 'Hermes is not logged into Nous Portal' despite
a valid global session.
Push the existing read-only global fallback (already used by
get_provider_auth_state and read_credential_pool) into _load_provider_state
so every caller benefits, and simplify get_provider_auth_state into a thin
wrapper. Writes still target the profile only — profile state continues to
shadow global state on the next read after a per-profile login. Behavior in
classic (non-profile) mode is unchanged because _load_global_auth_store
returns an empty dict.
Adds 5 tests covering the new contract on _load_provider_state directly.
Existing 770 auth/credential/nous tests still pass.
Closes#32992.
The chat path resolves Codex credentials via `resolve_codex_runtime_credentials`
which only reads `providers.openai-codex.tokens` (the singleton). The auxiliary
path uses `_read_codex_access_token` which checks the credential_pool first.
For users whose tokens live only in the pool — manual seed, partial re-auth,
restore from backup, or any state where the singleton is empty but the pool
is healthy — the chat path raised AuthError or (worse, since OpenAI(api_key='')
silently attaches no header) the wire saw HTTP 401 "Missing Authentication header"
while the auxiliary path worked fine.
This adds a pool fallback to `resolve_codex_runtime_credentials`: when the
singleton has no usable access_token, scan `credential_pool.openai-codex` for
the first entry that has a non-empty access_token and isn't in an exhaustion
cooldown window (`last_error_reset_at` in the future). If found, return that
token with `source="credential_pool"`. If no usable entry exists, the original
AuthError propagates as before.
Regression tests cover:
- Empty singleton + healthy pool entry → pool token returned
- Pool fallback skips entries currently in cooldown
- Empty singleton + empty/wedged pool → AuthError propagates (existing contract preserved)
When the Codex OAuth token endpoint returns 429 (usage-limit / quota
exhaustion), refresh_codex_oauth_pure raised a generic auth error that the
gateway surfaced as 'Primary provider auth failed: No Codex credentials
stored. Run hermes auth', prompting re-auth that cannot lift a quota cap.
Classify 429 distinctly (codex_rate_limited, relogin_required=False) with a
non-alarming quota message that honors Retry-After, log it as
'Primary provider rate-limited (429)', and stop format_auth_error from
appending the re-authenticate remediation. Also log the fallback provider's
literal config key instead of the resolved runtime category.
Refs #32790
Codex re-auth via `hermes setup` / `hermes model` wrote fresh OAuth
tokens to providers.openai-codex.tokens but left the credential_pool
device_code entry holding the consumed refresh token and stale error
markers. Since the runtime selects from the pool, the next request
spent a dead token and got a 401 token_invalidated. Update the
singleton-seeded pool entries in lockstep and clear their error state.
Fixes#33000
Operators behind reverse proxies that don't reliably forward
X-Forwarded-Host / X-Forwarded-Proto / X-Forwarded-Prefix (manual
nginx setups, on-prem ingresses, custom-domain Fly deploys with
incomplete proxy chains) had no way to force the absolute base URL
the OAuth callback redirects from. The dashboard would reconstruct
the redirect_uri from request headers, the IDP would echo it back,
and the user would land on the wrong host or wrong path — 404.
Add `dashboard.public_url` to config.yaml with env override
HERMES_DASHBOARD_PUBLIC_URL. When set, it is the complete authority —
scheme + host + optional path prefix (e.g. https://example.com/hermes) —
and becomes the base for the OAuth `redirect_uri`. X-Forwarded-Prefix
is IGNORED on this code path because the operator has explicitly
declared the public URL; we no longer need to guess from proxy
headers, and stacking the prefix on top would double-prefix the
common case where the prefix is already baked into public_url.
When unset, the existing proxy_headers + X-Forwarded-Prefix
reconstruction runs untouched. Existing Fly.io deploys continue to
work without configuration — this is purely additive.
Precedence mirrors dashboard.oauth.client_id:
env (non-empty) > config.yaml > reconstructed from request
Implementation:
- hermes_cli/config.py: add dashboard.public_url to DEFAULT_CONFIG
with a multi-paragraph doc comment explaining the use case,
the X-Forwarded-Prefix interaction, and the validation rules.
- hermes_cli/dashboard_auth/prefix.py: factored out the existing
_REJECT_CHARS frozenset, added _normalise_public_url() validator
(requires http/https scheme + non-empty host + no header-injection
chars), _load_dashboard_section() loader (robust to load_config
raising, non-dict shapes), and resolve_public_url() entry point
with the env-overrides-config precedence. A malformed value
silently falls through to ""; the caller treats "" as "reconstruct
from request" so a typo never breaks the login flow.
- hermes_cli/dashboard_auth/routes.py: rewrite _redirect_uri()
docstring to spell out the three resolution tiers; add the
public_url short-circuit before the existing X-Forwarded-Prefix
splicing. Source-level comment notes that X-Forwarded-Prefix is
intentionally ignored when public_url is set so a future reader
doesn't try to "fix" the missing prefix layering.
- cli-config.yaml.example: extend the existing dashboard section
with a public_url block.
- website/docs/user-guide/features/web-dashboard.md: new "Public
URL override" section between the provider configuration and
the OAuth flow walkthrough. Documents the env-vs-config table,
the validation rules, and the `http://` `public_url` ↔ Secure
cookie footgun.
Test coverage — new TestPublicUrlOverride class (8 tests):
- env var overrides request reconstruction (the primary motivating
case)
- config.yaml used when env unset
- env wins over config (precedence pin)
- public_url with a path prefix already baked in (the Q1-a case the
user explicitly chose)
- public_url suppresses X-Forwarded-Prefix layering (defends
against the double-prefix bug)
- trailing slash stripped from public_url (no //auth/callback)
- malformed public_url falls through to reconstruction (six
hostile inputs: javascript:, ftp:, missing scheme, missing host,
quote chars, CRLF injection)
- empty env string doesn't shadow config.yaml entry (CI / Fly
provisioned-but-empty secret case)
Mutation-tested: flipping the precedence in resolve_public_url() trips
exactly test_env_overrides_config_public_url; weakening the validator
(accept any scheme) trips exactly test_malformed_public_url_falls_through_to_reconstruction.
Both other tests in each pair stay green, confirming the suite
discriminates the specific regression each test pins.
Mission-control style deploys reverse-proxy the dashboard at a path
prefix (e.g. mission-control.tilos.com/hermes/* -> :9119) and inject
X-Forwarded-Prefix: /hermes on every request. The SPA mount already
honoured this for asset URLs and the bootstrap __HERMES_BASE_PATH__,
but the OAuth gate didn't:
1. The gate's Location: header to /login and the 401 envelope's
login_url were built bare ("/login?next=..."). Under a /hermes
prefix the browser follows that to mission-control.tilos.com/login
which the proxy doesn't route to the dashboard.
2. _redirect_uri (the OAuth callback URL handed to the IDP) used
request.url_for() which doesn't honour X-Forwarded-Prefix
(Starlette/uvicorn only proxy_headers Host + Proto + For). The
IDP redirects back to /auth/callback instead of /hermes/auth/
callback → 404 in the user's browser.
3. Cookies were set with Path=/ which leaks them to other apps on
the same origin and won't be sent back on requests under the
prefix in the first place.
Fix threads the normalised prefix through every boundary:
* New hermes_cli/dashboard_auth/prefix.py — single source of truth
for X-Forwarded-Prefix parsing. web_server._normalise_prefix
becomes a re-export so the SPA mount, the gate, and the cookies
helper all agree.
* middleware._unauth_response builds login_url = f"{prefix}/login".
* routes._redirect_uri splices the prefix into the path component
of the IDP-bound URL (with full validation of the header).
* cookies.{set,clear}_{session,pkce}_cookie now take prefix="".
Path attribute switches to /hermes when set; cookie name switches
name variant (see below). Every caller passes the request's
normalised prefix.
Cookie hardening (Teknium's lesser-note #1 in the PR review): adopt
the __Host- / __Secure- cookie name prefixes per draft-west-cookie-
prefixes. The variant is selected from (use_https, prefix):
* Loopback HTTP → bare "hermes_session_at" (both prefixes require
Secure, incompatible with HTTP).
* HTTPS, direct deploy (Path=/) → "__Host-hermes_session_at".
Strongest spec: bound to exact origin, no Domain attribute, Secure
required.
* HTTPS, behind a proxy prefix (Path=/hermes) →
"__Secure-hermes_session_at". __Host- forbids Path != "/"; the
explicit Path=/hermes covers same-origin app isolation.
Setter and reader BOTH consult the prefix because the cookie *name*
changes — a reader that looked up the bare name when the setter wrote
__Secure- would never find the value. The reader falls back across
all three variants so a request whose shape changed mid-session (e.g.
post-deploy from no-prefix to /hermes) still picks up the existing
cookie until it expires.
Test coverage:
- tests/hermes_cli/test_dashboard_auth_prefix.py — new file. 11 tests
pinning:
• Location: /hermes/login on the gate's HTML redirect
• 401 envelope login_url carries the prefix
• Malformed X-Forwarded-Prefix is ignored (header-injection
defence; the script-tag value is normalised to empty string)
• _redirect_uri splices /hermes into the path (the property
that prevents the IDP-returns-to-404 failure)
• PKCE cookie uses Path=/hermes + __Secure- when proxied
• Session cookies use __Host- when direct, __Secure- when
proxied, bare on loopback HTTP
• End-to-end round trip with hand-managed PKCE cookie carriage
(TestClient can't simulate a Path=/hermes cookie automatically)
- tests/hermes_cli/test_dashboard_auth_cookies.py — rewritten to pin
each (use_https, prefix) shape produces its expected cookie name,
plus reader-side coverage that __Host- and __Secure- variants are
both recognised.
- Existing tests across middleware / 401-reauth / etc. updated to
match the new cookie names (substring contains instead of
startswith).
Mutation-tested: reverting _unauth_response to build the bare
"/login" URL trips exactly the two tests that pin the prefix
carriage, confirming the suite discriminates the regression.
The gate's _unauth_response set next=<path> on the /login redirect URL,
but nothing downstream read it: render_login_html ignored next=,
auth_login dropped it, and auth_callback read next= from its own query
string — which an IDP never sets on the callback URL (real IDPs only
echo back code+state). The _validate_post_login_target plumbing in the
callback was unreachable on the happy path, so users always landed on
"/" regardless of what they originally requested.
Worse: reading next= from the callback URL was a latent open-redirect
sink, since an attacker could craft /auth/callback?...&next=/admin and
have the server honour it post-auth.
Fix carries next= through the round trip on a server-controlled channel:
1. login_page reads request.query_params['next'] and passes it (post-
validation) to render_login_html.
2. render_login_html threads next= URL-encoded into each provider
button's href, with HTML-attribute escaping as defence in depth.
3. auth_login accepts ?next= as a query param, re-validates, and
appends it as a fourth segment (next=<urlquoted>) in the PKCE
cookie payload alongside provider/state/verifier.
4. auth_callback no longer accepts a next: str = "" query param. It
parses next= out of the PKCE cookie and validates that with the
same same-origin rules. Any attacker-supplied ?next= on the
callback URL is silently ignored — server-only carrier.
Test coverage adds three classes:
- TestAuthCallbackNext drives /login → /auth/login → IDP-bounce →
/auth/callback end-to-end without smuggling next= onto the callback
URL (which is what the previous tests did and why they didn't
catch the bug). Includes test_attacker_callback_next_param_is_ignored
to pin the security property that the URL value is never read.
- TestRenderLoginHtmlNext covers the rendering function at the
unit boundary so a regression that drops next_path is caught
without spinning up the full app.
- TestAuthLoginPkceCookieNext inspects the Set-Cookie header on
/auth/login responses so a regression in cookie encoding is caught
without driving the full round trip.
Mutation-tested: reverting auth_callback to read next= from the URL
trips 3 of 6 TestAuthCallbackNext tests (the safe-path and attacker-
hardening ones), confirming the suite discriminates between the cookie
read and the URL read.
When the OAuth gate is active, start_server runs uvicorn with
proxy_headers=True so the dashboard can honour X-Forwarded-Proto from
Fly's TLS terminator (cookies, redirect URI reconstruction). A side
effect: ws.client.host is rewritten to the X-Forwarded-For value, which
on Fly is the real internet client IP — never loopback. The loopback
peer guard in _ws_client_is_allowed then rejected every WS upgrade in
gated mode (4403 close) even after a successful OAuth round trip and
ticket consumption, silently breaking /api/pty, /api/ws, /api/pub, and
/api/events.
Fix: in gated mode, bypass the peer-IP check. The OAuth gate +
single-use ticket is the auth. The Host/Origin guard in
_ws_host_origin_is_allowed still runs and is what protects against
DNS-rebinding here, not the peer IP.
Loopback mode behaviour is unchanged: the legacy ?token= path is the
only auth there and we don't want LAN hosts guessing tokens.
Regression coverage: TestWsRequestIsAllowedGated pins all four
behaviours — non-loopback peer allowed in gated mode, non-loopback peer
rejected in loopback mode, loopback peer allowed in loopback mode, and
the Host/Origin guard still firing on a rebinding attempt with gated
mode + matching peer.
The stub auth provider's _sign/_unsign helpers joined payload and HMAC
with a 'b"."' separator and recovered the parts via bytes.rsplit. HMAC-SHA256
digests are random bytes, so ~12% of the time the digest contains 0x2E
('.') and rsplit picks the wrong split point -- HMAC verification then
spuriously rejects valid tokens.
test_stub_refresh_round_trips was failing ~25% of the time in isolation
because of this.
Switch to a fixed-length suffix (32 bytes, sliced off in _unsign): no
separator means no collision class. After the fix, 10/10 runs pass.
The Nous OAuth provider plugin (plugins/dashboard_auth/nous) is bundled
and auto-loaded — same as before — but previously refused to register
unless BOTH HERMES_DASHBOARD_OAUTH_CLIENT_ID and HERMES_DASHBOARD_PORTAL_URL
were set, then the gate's fail-closed branch told the operator 'install
the default Nous provider'. That message is misleading: the provider IS
installed; it's just unconfigured. And the contract only really needs
the per-instance client_id — the portal URL is the same for everyone
in production.
Three changes:
1. plugins/dashboard_auth/nous/__init__.py:
- HERMES_DASHBOARD_PORTAL_URL is now optional and defaults to
'https://portal.nousresearch.com'. Override only for staging
(portal.rewbs.uk) or a custom deployment. Empty string also
falls back to the default so an empty Fly secret can't point
the dashboard at nowhere.
- Plugin exposes a module-level LAST_SKIP_REASON: str that the gate
reads when no providers register. Cleared on each register() call.
Skip reasons are human-readable and actionable
('HERMES_DASHBOARD_OAUTH_CLIENT_ID is not set. The Nous Portal
provisions this env var…').
2. plugins/dashboard_auth/nous/plugin.yaml:
- requires_env drops HERMES_DASHBOARD_PORTAL_URL; only the client_id
is mandatory. Description updated to reflect this.
3. hermes_cli/web_server.py:
- When the gate fail-closes for 'no providers', it now reads each
bundled plugin's LAST_SKIP_REASON and embeds them in the SystemExit
message. Operator sees the specific config fix needed:
Bundled providers reported these issues:
• nous: HERMES_DASHBOARD_OAUTH_CLIENT_ID is not set. …
instead of the prior generic 'Install the default Nous provider'.
Tests:
- TestPluginRegister rewritten to assert the new defaults +
LAST_SKIP_REASON contents (6 tests, +1 new for empty-string env).
- New gate test test_start_server_surfaces_nous_skip_reason_when_unconfigured.
- test_get_method_is_not_allowed widened to handle the SPA-shell 200
path explicitly — assertion now verifies no JSON ticket leaks
rather than asserting a specific status code (covers all four of
401/404/405/200).
Docs updated: web-dashboard.md's 'Default provider' section now shows
the env-var table with required/optional columns and embeds the
fail-closed error message verbatim so operators can match what they
see at the prompt.
Phase 7 surfaces the OAuth gate state to users.
web/src/components/AuthWidget.tsx (new):
Sidebar widget that fetches /api/auth/me on mount and renders a
compact 'Logged in as <user_id…> via <provider>' row with a logout
icon. Contract V1 (Nous Portal) emits no email/display_name claims,
so user_id is the display value (truncated to 14 chars + ellipsis);
display_name and email fallthroughs are forward-compat for OQ-C1.
Renders nothing on 401 from /api/auth/me — that's the signal the
gate isn't engaged (loopback mode), in which case the widget would
be confusing.
Logout POSTs /auth/logout (which clears cookies + redirects to
/login) then full-page-navigates to /login itself; the SPA's fetch
wrapper doesn't follow that redirect, so the navigation is explicit.
web/src/App.tsx: mounts <AuthWidget /> above <SidebarFooter />.
Component is self-hiding in loopback mode so there's no need for a
conditional mount.
web/src/lib/api.ts:
- getAuthMe() + logout() helpers
- AuthMeResponse type
- StatusResponse gets optional auth_required + auth_providers fields
so the existing StatusPage can render a gated/loopback badge.
hermes_cli/web_server.py: /api/status payload now includes
- auth_required: bool — whether app.state.auth_required is True
- auth_providers: list[str] — registered DashboardAuthProvider names
Lazy-imports list_providers so early-startup status calls don't
crash if the dashboard_auth module is still being set up.
tests/hermes_cli/test_dashboard_auth_status_endpoint.py: 3 new tests
covering the new status fields in both gated and loopback modes plus
a regression that no existing field got dropped from the payload.
The hermes status CLI is unchanged in this commit — that command
tracks model providers + OAuth credentials, not running-dashboard
state. The /api/status endpoint is the canonical place to query
dashboard auth-gate state, consumed by the React StatusPage already.
Contract V1 of nous-account-service PR #180 ships no refresh tokens, so
the original Phase 6 silent-refresh design is replaced with a thinner
'401 → redirect to /login' UX. The dashboard's gated middleware now
emits a structured envelope on any auth failure; the SPA's fetch
wrapper sees it and full-page-navigates the user through re-auth.
hermes_cli/dashboard_auth/cookies.py:
set_session_cookies(refresh_token='') SKIPS writing the
hermes_session_rt cookie. Forward-compat: a non-empty refresh_token
still emits the cookie unchanged, so a future Portal contract that
starts issuing RTs flips the persistence on with no other change.
clear_session_cookies still emits a Max-Age=0 deletion for the RT
cookie so stale cookies from earlier deployments get flushed on
logout / session expiry. Deprecation marker + rationale in
module docstring per the user's docstring-only deprecation pattern.
hermes_cli/dashboard_auth/middleware.py:
_unauth_response now builds a structured JSON envelope for API 401s:
{ error: 'session_expired' | 'unauthenticated',
detail: 'Unauthorized',
reason: <internal>,
login_url: '/login?next=<safe-path>' }
HTML redirects also carry next= so a user landing on /sessions
without a cookie bounces back to /sessions after re-auth.
_safe_next_target validates same-origin: drops protocol-relative
paths (//evil.com), absolute URLs, and any /login or /auth/* loop.
Dead cookies are cleared on the 401 path so the browser stops
replaying invalid tokens.
hermes_cli/dashboard_auth/routes.py:
/auth/callback accepts next= query param and validates via
_validate_post_login_target (same rules as the gate's
_safe_next_target — defence-in-depth because next= survived a full
IDP round trip and attacker-controlled state can re-enter via the
callback URL). Open-redirect attempts land at '/' instead.
web/src/lib/api.ts:
fetchJSON parses the 401 envelope and full-page-navigates to
body.login_url ONLY on the known session-expiry error codes.
Domain-level 401s (e.g. permission errors) bubble up as regular
errors. credentials: 'include' added so cookie auth works for all
fetches routed through this wrapper. sessionStorage.lastLocation is
preserved for future use by AuthWidget / hermes_status.
Test files marked with pytest.mark.xdist_group so the four files that
mutate web_server.app.state.auth_required serialize onto the same xdist
worker — eliminates 'works locally, fails in CI' app-state bleed.
20 new tests in test_dashboard_auth_401_reauth.py:
- set_session_cookies(refresh_token='') skips RT cookie
- clear_session_cookies still emits RT deletion
- 401 envelope shape (unauthenticated vs session_expired)
- dead cookie cleared on invalid-token 401
- login_url carries next= for deep paths
- login loop avoided when path is /login/auth/api-auth
- protocol-relative URL rejected
- _safe_next_target unit tests (accept same-origin, reject loops/abs)
- /auth/callback respects safe next= but rejects open redirects
2 pre-existing tests updated to accept the new /login?next=%2F shape.
Full dashboard-auth suite: 168 passed, 1 skipped (Phase 0 pre-existing).
Phase 5 task 5.2. Four WebSocket endpoints — /api/pty, /api/ws, /api/pub,
/api/events — previously authed with the same constant-time check against
`_SESSION_TOKEN`. Replaced with a single helper that branches on
`app.state.auth_required`:
Loopback / --insecure: legacy ?token=<_SESSION_TOKEN> path (unchanged).
Gated: ?ticket=<single-use> consumed against the
dashboard-auth ticket store.
Critical security property: gated mode UNCONDITIONALLY rejects the
?token= path. A leaked _SESSION_TOKEN value from a log line is not
replayable for WS access in gated deployments.
`_build_sidecar_url` now branches too: loopback uses the legacy token;
gated mode mints a server-internal ticket via mint_ticket() with
pseudo-user 'pty-sidecar' / provider 'server-internal' so audit logs can
distinguish PTY-internal sidecar tickets from browser tickets. PTY
children open /api/pub exactly once at startup so single-use suffices.
Ticket rejections audit-log as WS_TICKET_REJECTED with truncated reason
+ client IP + WS path. Operators debugging 'WS keeps closing' issues see
which endpoint and why.
17 new tests:
- POST /api/auth/ws-ticket: 200 with cookie, 401/302 without, distinct
per call, GET-not-allowed.
- _ws_auth_ok loopback: token accept/reject, missing-token reject,
ticket-param-ignored.
- _ws_auth_ok gated: ticket accept, single-use rejection, unknown reject,
legacy-token-rejected-in-gated assertion, audit-log emission.
- _build_sidecar_url: loopback uses token=, gated uses ticket=, no-bound
returns None.
Phase 5 task 5.1. Browsers cannot set Authorization on a WebSocket
upgrade, so in gated mode the SPA needs an alternative way to bind the
upgrade to its authenticated session.
hermes_cli/dashboard_auth/ws_tickets.py — in-memory single-use ticket
store with 30s TTL. Thread-safe (threading.Lock), token_urlsafe(32)
values, ticket value truncated to 8 chars in error messages for log
hygiene. Module-level state with _reset_for_tests() helper.
hermes_cli/dashboard_auth/routes.py — adds POST /api/auth/ws-ticket.
Auth-required (the gate middleware already attaches Session to
request.state.session). Returns {ticket, ttl_seconds}; emits
WS_TICKET_MINTED audit event with user_id + provider + ip.
hermes_cli/dashboard_auth/audit.py — adds WS_TICKET_REJECTED enum
value for the consume-side rejection event (wired into the WS
endpoints in task 5.2).
11 new tests covering round-trip, single-use, TTL boundary, unknown
ticket rejection, secret-hygiene truncation in error messages, and
concurrent mint+consume from 20 threads.
Phase 3, Task 3.5. Three changes to web_server.py:
1. start_server replaces the legacy SystemExit-refusing-to-bind guard
with: if app.state.auth_required and no providers registered, exit
with a clear message; otherwise log the gate-on banner. --insecure
keeps its existing behaviour.
2. uvicorn proxy_headers flag is computed from app.state.auth_required.
Loopback / --insecure keep it False (so _ws_client_is_allowed sees
the real peer for the loopback gate); gated mode flips it True so
X-Forwarded-Proto from Fly's TLS terminator is honoured for cookie
Secure-flag decisions in detect_https().
3. _serve_index no longer injects window.__HERMES_SESSION_TOKEN__ when
the gate is on — the SPA reads identity from /api/auth/me using
cookie auth instead. window.__HERMES_AUTH_REQUIRED__ flag lets the
SPA pick between ticket-auth (gated) and token-auth (loopback) for
/api/pty + /api/ws (Phase 5 will wire this in the React layer).
4 new behavioural tests; loopback regression harness still green.
Phase 3, Tasks 3.2 + 3.3 + 3.4. These three pieces are mutually
dependent so they land together.
middleware.py - gated_auth_middleware engages when app.state.auth_required
is True. Allowlists /login, /auth/*, /api/auth/providers, and static
asset paths; everything else demands a valid session_at cookie. Verifies
by trying every registered provider's verify_session in turn (multi-
provider stack); attaches verified Session to request.state.session.
Returns 401 JSON for /api/* and 302 -> /login for HTML. ProviderError
during verify -> 503.
routes.py - APIRouter with:
GET /login server-rendered HTML
GET /auth/login?provider=N 302 to IDP + PKCE cookie
GET /auth/callback?code,state completes login, sets session cookies
POST /auth/logout clears cookies + best-effort revoke
GET /api/auth/providers public bootstrap endpoint (503 if zero)
GET /api/auth/me verified session as JSON (auth-required)
login_page.py - Inline-CSS HTML template, no React, no JavaScript.
web_server.py - Mounted gated_auth_middleware between host_header and
auth_middleware (FastAPI runs middlewares in registration order: host
check -> cookie auth -> token auth). auth_middleware short-circuits
when auth_required so cookie auth is authoritative in gated mode.
Router is included before mount_spa so the catch-all doesn't swallow
/login or /auth/*.
17 new behavioural tests; loopback regression harness still green.
Phase 3, Task 3.1. Three cookies:
- hermes_session_at: OAuth access token (HttpOnly, TTL = token TTL)
- hermes_session_rt: OAuth refresh token (HttpOnly, 30d max-age)
- hermes_session_pkce: PKCE state + verifier + provider hint (10min)
All SameSite=Lax + Path=/. Secure flag is set ONLY when the request
scheme is https — uvicorn proxy_headers=True (enabled in gated mode at
Phase 3.5) rewrites scheme from X-Forwarded-Proto so Fly's TLS
terminator works.
Phase 2, Task 2.1. Self-contained fake IDP — start_login redirects
straight back to {redirect_uri}?code=stub_code&state=<s> so tests can
walk the OAuth round trip in-process. Tokens are HMAC-signed JSON blobs
(not real JWTs) — enough structure for verify_session to detect tamper
and expiry without pulling in pyjwt.
Lives in tests/ only — never registered as a real plugin. Phase 3's
end-to-end tests import StubAuthProvider directly.
Convention: exp <= now counts as expired (TTL=0 means born-expired)
— matches what Phase 6's silent-refresh test will need.
Phase 1, Task 1.4. Records every auth event (login start/success/failure,
logout, refresh success/failure, revoke, session verify failure, WS
ticket mint) as one JSON object per line. Token-like kwargs (access_token,
refresh_token, code, code_verifier, state, ticket, cookie, Authorization)
are dropped before serialisation so the log never contains live secrets.
Write failures log at WARNING but never raise — auth flows must not fail
because the audit logger broke.
Phase 1, Task 1.3. Mirrors the existing register_image_gen_provider
pattern (plugins.py:531) — wrong-type or duplicate-name registrations
log at WARNING and silently return rather than raising, so a misbehaving
auth plugin cannot crash the host.
Deviation from plan: the plan's draft raised TypeError on non-provider
input; switched to silent-warn to match the established image_gen
convention. Test updated to match.
Phase 1, Task 1.2. Verifies registration order is preserved, duplicate
names are rejected with ValueError, and non-compliant providers fail at
register time (not later when the middleware tries to dispatch).
Phase 1, Task 1.1. New package hermes_cli/dashboard_auth/ contains:
base.py - DashboardAuthProvider ABC with 5 abstract methods
(start_login, complete_login, verify_session,
refresh_session, revoke_session), Session + LoginStart
frozen dataclasses, three exception types
(ProviderError / InvalidCodeError / RefreshExpiredError),
and assert_protocol_compliance() for plugins to call
in their own tests.
registry.py - Module-level register/get/list/clear with a lock.
Nothing reads the registry yet — Phase 2 adds the StubAuthProvider and
Phase 3 wires the gate middleware. The plugin hook lands in Task 1.3.
Phase 0, Task 0.3. start_server now computes should_require_auth(host,
allow_public) and records it on app.state.auth_required BEFORE the
existing legacy SystemExit guard fires. This gives middleware, the SPA
token-injection path, and WS endpoints a consistent read source for
'is the gate active'. The flag is set but no one reads it yet — Phase 3
registers the gate middleware.
Note: 4 pre-existing test failures in tests/hermes_cli/test_web_server.py
(PtyWebSocket) + test_update_hangup_protection.py reproduce on pristine
HEAD and are unrelated to this change (starlette TestClient WS regression).
Phase 0, Task 0.2. Single source of truth for 'is the auth gate active?'.
Reuses the existing _LOOPBACK_HOST_VALUES frozenset so this stays in sync
with the DNS-rebinding host-header check. RFC1918/CGNAT/link-local are
treated as public — exact threat model the gate exists for.
Phase 0, Task 0.1 of the dashboard-oauth plan. Establishes a baseline for
the loopback dashboard's auth surface so future phases can prove they
didn't regress the existing _SESSION_TOKEN flow when adding the OAuth gate.
* remove Vercel AI Gateway provider and Vercel Sandbox terminal backend
Both Vercel-hosted integrations are removed end-to-end. Users on the AI
Gateway should switch to OpenRouter or one of the other aggregators
(Nous Portal, Kilo Code). Users on the Vercel Sandbox backend should
switch to Docker, Modal, Daytona, or SSH.
What's removed:
- `plugins/model-providers/ai-gateway/` provider plugin
- `hermes_cli/vercel_auth.py` Vercel-Sandbox auth helper
- `tools/environments/vercel_sandbox.py` terminal backend
- `ai-gateway` provider wiring across auth, doctor, setup, models,
config, status, providers, main, web_server, model_normalize, dump
- `vercel_sandbox` backend wiring across terminal_tool, file_tools,
code_execution_tool, file_operations, approval, skills_tool,
environments/local, credential_files, lazy_deps, prompt_builder,
cli, gateway/run
- `AI_GATEWAY_BASE_URL` constant, `_AI_GATEWAY_HEADERS` auxiliary-client
header set, run_agent base-URL header/reasoning special-cases
- `[vercel]` pyproject extra and `vercel`/`vercel-workers` from uv.lock
- env vars: `AI_GATEWAY_API_KEY`, `AI_GATEWAY_BASE_URL`, `VERCEL_TOKEN`,
`VERCEL_PROJECT_ID`, `VERCEL_TEAM_ID`, `VERCEL_OIDC_TOKEN`,
`TERMINAL_VERCEL_RUNTIME`
- Tests: deletes test_ai_gateway_models.py and
test_vercel_sandbox_environment.py; scrubs references across 23
surviving test files (no entire tests deleted unless they were
dedicated to AI Gateway / Sandbox)
- Docs: provider tables, env-var reference, setup guides, security
notes, tool config, terminal-backend tables — English plus zh-Hans
i18n parity
- `hermes-agent` skill: provider table entry and remote-backend list
What stays (intentional):
- `popular-web-designs/templates/vercel.md` — CSS design reference,
unrelated to Vercel-the-AI-product
- `x-vercel-id` in `stream_diag.py` headers — generic Vercel CDN
response header, useful diag signal on any Vercel-hosted endpoint
- `vercel-labs/agent-browser` URL in browser config — lightpanda
browser project, different OSS effort
- `userStories.json` historical contributor entry mentioning Vercel
Sandbox — archive, not active docs
Validation:
- 1153 tests in the 22 targeted files pass (`scripts/run_tests.sh`)
- Full repo `py_compile` clean
- Live import of every touched module + invariant check (no
`ai-gateway` in `PROVIDER_REGISTRY`, no `_AI_GATEWAY_HEADERS`, no
`vercel_sandbox` in `_REMOTE_TERMINAL_BACKENDS`)
* test: convert profile-count check from change-detector to invariant
The hardcoded "== 34" assertion broke when ai-gateway was removed.
Per AGENTS.md change-detector-test guidance, assert the relationship
(registry count >= number of plugin dirs) instead of a literal count.
Counts shift when providers are added/removed; that's expected.
qwen3.7-max on OpenCode Go rejects the OpenAI-compatible (oa-compat)
format with HTTP 401 but works correctly via the Anthropic Messages
endpoint (/v1/messages with x-api-key auth). Route it the same way
MiniMax models are routed: anthropic_messages api_mode.
Changes:
- hermes_cli/models.py: add qwen3.7-max routing + curated list
- hermes_cli/setup.py: add to setup wizard model list
- hermes_cli/auth.py: update provider comment
- tests: add assertions for qwen3.7-max api_mode routing
'hermes login' was removed (the command now just prints a deprecation
message and exits). The bundled hermes-agent SKILL.md, in-code error
messages, the tip rotation, the proxy adapters, and the docs site
still pointed agents and users at the dead command — so models loading
the skill kept running 'hermes login --provider openai-codex' and
getting a dead-end print.
Replacements use the canonical 'hermes auth add <provider>' surface
(or bare 'hermes auth' for the interactive manager).
Files:
- skills/autonomous-ai-agents/hermes-agent/SKILL.md (+ regenerated docs page)
- hermes_cli/tips.py (tip rotation)
- agent/google_oauth.py (gemini-cli error message)
- agent/conversation_loop.py (nous re-auth troubleshooting line)
- agent/credential_sources.py (docstring)
- hermes_cli/proxy/cli.py + hermes_cli/proxy/adapters/nous_portal.py (proxy auth hints)
- tests/hermes_cli/test_proxy.py (updated assertions)
- website/docs/reference/faq.md, website/docs/user-guide/features/subscription-proxy.md
- zh-Hans i18n mirrors for the above
'hermes logout' is still a live command and is left untouched.
The 'hermes login' stub in hermes_cli/auth.py:login_command() and
the cli-commands.md 'Deprecated' rows are intentionally kept as
the discoverable deprecation surface.
Updates curated picker lists for both the OpenRouter fallback snapshot
(`OPENROUTER_MODELS`) and the Nous Portal list (`_PROVIDER_MODELS['nous']`).
Regenerates website/static/api/model-catalog.json via
`scripts/build_model_catalog.py` to keep the docs-hosted manifest in
sync (drift guard in `test_in_repo_lists_match_manifest`).
tests/hermes_cli/test_models.py fixtures updated — they pinned the
old model id as their live-fetch sample.
* feat(mcp): Nous-approved MCP catalog with interactive picker
Adds an optional-mcps/ directory mirroring optional-skills/: curated,
Nous-approved MCP servers shipped with the repo but disabled by default.
Presence in optional-mcps/ = approval. No community tier, no trust signals.
Entries are added by merging a PR.
New surface:
hermes mcp Interactive catalog picker (default)
hermes mcp catalog Plain-text list, scriptable
hermes mcp install <name> Install a catalog entry
Picker behavior:
not installed -> install (clone/bootstrap if needed, prompt for creds)
installed/off -> enable
installed/on -> menu (disable / uninstall / reinstall)
Manifest schema (manifest_version: 1) supports:
- transport: stdio (command/args, ${INSTALL_DIR} substitution) or http (url)
- install: optional git clone + bootstrap commands (for repos that need
local venv setup, like the n8n bridge); omit for npx/uvx servers
- auth: api_key (prompts -> ~/.hermes/.env), oauth (provider-mediated
or native MCP), or none
Catalog entries are never auto-updated. Users re-run `hermes mcp install`
to refresh. Credentials always go to ~/.hermes/.env (the .env-is-for-secrets
rule), never to per-server env blocks.
Ships n8n as the reference manifest (https://github.com/CyberSamuraiX/hermes-n8n-mcp).
Tests: 19 catalog tests + E2E install/uninstall round-trip via the shipped
manifest.
* feat(mcp): tool-selection checklist + Linear catalog entry
Adds install-time tool selection so users only enable the MCP tools they
actually want, and ships Linear as a second reference catalog entry to
demonstrate the http+oauth path alongside n8n's stdio+api_key+git-bootstrap.
Tool selection flow:
install (clone/auth/credentials) ->
probe server for available tools ->
curses checklist with pre-checked rows ->
write mcp_servers.<name>.tools.include
Pre-check priority:
1. user's prior tools.include (reinstall preserves selection)
2. manifest's tools.default_enabled (curated subset)
3. all probed tools (default)
Probe-failure fallback (server unreachable, OAuth not yet complete,
backing service offline):
- manifest declared default_enabled -> applied directly
- no default declared -> no filter written (all-on when reachable)
- both cases point user at hermes mcp configure <name>
Manifest schema additions:
tools:
default_enabled: [list, of, tool, names] # optional
Updates:
- optional-mcps/linear/manifest.yaml -- new reference entry (http+oauth)
- optional-mcps/n8n/manifest.yaml -- tools.default_enabled set to the
8 read-mostly tools; mutating tools (activate/deactivate, container_logs)
pruned by default
- docs: new 'Tool selection at install time' section in features/mcp.md
Tests: 7 new tests in TestToolSelection covering probe-success / probe-fail
matrix, manifest-default filtering, reinstall-preserves-selection, and
invalid-default-enabled rejection. 26 catalog tests + 32 existing
mcp_config tests passing.
* feat(mcp): polish — picker unification, include-mode convergence, hardening
Addresses review findings on PR #30870. Lands all improvements that
belong in this PR before merge; defers separate cleanup (consolidating
two probe implementations, change-detector tests) to follow-ups.
Picker UX (mcp_picker.py)
- Unifies catalog + custom (user-added) MCPs in one view with distinct
status badges (available / enabled / installed (disabled) /
custom — enabled / custom — disabled)
- Adds 'Configure tools (probe server + re-pick)' action to both the
catalog-installed and custom-row submenus — the existing
hermes mcp configure flow was previously unreachable from the picker
- Loops until ESC/q so the user can manage several entries in one
session instead of having to re-launch
- Uninstall message now mentions .env credentials are preserved with a
pointer to clean them up manually if no longer needed
- Surfaces a 'requires a newer Hermes' warning per future-manifest
entry instead of silently hiding it
Catalog (mcp_catalog.py)
- catalog_diagnostics() exposes which manifests were skipped and why
(future_manifest vs invalid) so UIs can give actionable feedback
- _do_git_install detects SHA-shaped refs (regex /[0-9a-f]{7,40}/)
and skips the doomed 'git clone --branch <sha>' attempt — clone --branch
only accepts branches/tags, so SHAs always failed noisily before
falling back to the full-clone path
- Probe-success all-tools-enabled message now mentions that new tools
the server adds later will be auto-enabled (no-filter mode)
Convergence (tools_config.py)
- _configure_mcp_tools_interactive now writes tools.include (whitelist)
instead of tools.exclude (blacklist), matching the catalog flow and
hermes mcp configure. The on-disk config shape no longer depends on
which UI the user touched last
- Two existing tests updated to assert the new include-mode contract
Discoverability
- Setup wizard final step now prints 'Browse curated MCPs: hermes mcp'
- Three tip-corpus entries pointing at the new catalog
- Docs updated with: trust model (manifests run code locally, gated by
PR review, but read before installing), runtime ${ENV_VAR} substitution
semantics, and the manifest_version forward-compat behavior
Tests
- 7 new tests covering future-manifest diagnostics, custom MCP picker
rows, SHA-ref git-install path, branch-ref git-install path, and the
tools_config include-mode write contract
- 80 MCP-related tests passing across test_mcp_catalog.py,
test_mcp_config.py, test_mcp_tools_config.py
* fix(mcp): drop setup-wizard catalog hint to satisfy supply-chain scanner
The wizard line 'Browse curated MCPs: hermes mcp' triggered the
CI supply-chain scanner because it pattern-matches on edits to any
file named hermes_cli/setup.py — that filename matches the Python
'install-hook file' heuristic even though this setup.py is the
user-facing 'hermes setup' wizard, not a packaging install hook.
The catalog is already surfaced via three tip-corpus entries in
hermes_cli/tips.py (which the scanner doesn't flag), so dropping the
wizard mention loses no discoverability. Worth revisiting after a
scanner allowlist for this specific file lands.
Two posture fixes surfaced by the web-pentest skill self-test against
the dashboard (issue #32267).
1. /dashboard-plugins/<name>/<path> previously returned 200 for any
file inside the plugin's dashboard directory — including
plugin_api.py and __pycache__/*.pyc. The path is unauthenticated by
architecture (SPA loads JS via <script src> and CSS via <link href>,
neither of which can attach a custom auth header), so the fix is
not "require token" — it's "restrict to browser-fetchable suffixes."
Allowlist now: .js .mjs .css .json .html .svg .png .jpg .jpeg .gif
.webp .ico .woff .woff2 .ttf .otf .map. Everything else → 404.
This stops a private user-installed plugin's Python source from
being readable by anyone reachable on the dashboard's loopback port
(other local users on a shared box, sidecar containers sharing the
host netns).
2. save_env_value() now refuses to persist env-var names that
influence how the next subprocess executes: LD_PRELOAD,
LD_LIBRARY_PATH, LD_AUDIT, DYLD_*, PYTHONPATH, PYTHONHOME,
PYTHONSTARTUP, NODE_OPTIONS, NODE_PATH, PATH, SHELL, EDITOR,
VISUAL, PAGER, BROWSER, GIT_SSH_COMMAND, GIT_EXEC_PATH; plus
HERMES_HOME / HERMES_PROFILE / HERMES_CONFIG / HERMES_ENV.
PUT /api/env is authed but the session token lives in the SPA HTML
where any future plugin XSS or local process can read it. Without
this gate, a token-holder could plant LD_PRELOAD in .env and the
next hermes process start would load attacker code via the dotenv
to os.environ chain. This is enforced on write only — pre-existing
.env values are left alone (the gate is in save_env_value, not in
load_env). PUT /api/env now returns 400 with the explanatory
message instead of an opaque 500.
IMPORTANT: HERMES_* overall is NOT blocked — only the four runtime
location names. Integration credentials following the HERMES_*
convention (HERMES_GEMINI_*, HERMES_LANGFUSE_*, HERMES_SPOTIFY_*,
HERMES_QWEN_BASE_URL, ...) keep working.
Regression tests cover both fixes (30 new test cases). No existing
tests changed; 257 passing in tests/hermes_cli/.
Closes#32267.
All four failures were broken by the security cluster (#10082 / #10133 /
#4609 / symlink-reject batch) merging on May 25. They were red on
origin/main HEAD when #32042 and #32061 ran, gating PRs that touched
unrelated code.
1) tests/hermes_cli/test_update_zip_symlink_reject.py
test_update_via_zip_accepts_normal_member called the real
_update_via_zip without sandboxing PROJECT_ROOT — so the function's
shutil.copytree() actually copied the fake README from the test ZIP
over the real repo's README.md, which then made
test_readme_mentions_powershell_installer fail in any test run that
happened to pick this test up earlier. Mock PROJECT_ROOT to an
isolated tmp_path / install_dir, stub subprocess so pip/uv reinstall
doesn't actually run, and assert the fake README lands in the
sandbox (not the real tree).
2) tests/tools/test_windows_native_support.py
test_readme_mentions_powershell_installer was the victim of (1) —
nothing wrong with the test itself, the fix in (1) clears it.
3) tests/tools/test_file_read_guards.py
test_proc_fd_other_not_blocked called _is_blocked_device('/proc/self/fd/3')
expecting False. But _is_blocked_device runs realpath() and on
pytest xdist workers fd 3 happens to be dup'd to /dev/urandom
(because the worker subprocess inherits open fds from pytest's
collection pipe machinery). Switch to the lower-level
_is_blocked_device_path which is the path-pattern check the test
actually means to exercise; realpath-resolution coverage already
lives in test_symlink_to_blocked_device_is_blocked.
4) tests/tools/test_transcription_tools.py
Module installed a faster_whisper stub via sys.modules without
setting __spec__, then later @pytest.mark.skipif called
importlib.util.find_spec('faster_whisper') which raises
'ValueError: __spec__ is None' for modules with a None spec attr.
Set __spec__ on the stub to a real ModuleSpec.
Validation: 195/195 green across the 4 affected files.
_update_via_zip downloads a source ZIP from GitHub and calls
zipfile.ZipFile.extractall. The existing zip-slip path guard validates
each member's path stays under tmp_dir, but does not check member type
— so a ZIP containing a symlink member would still be materialized by
extractall, and a symlink target could point outside the extracted
tree (or to a sensitive system path).
This isn't a high-likelihood threat for hermes-agent's actual GitHub
source ZIPs (we don't ship symlinks), but the extractall path runs as
the user's account and a compromised mirror could plant arbitrary files
via the symlink → target → write chain.
Reject any member whose Unix mode bits (upper 16 bits of external_attr)
are S_IFLNK before extractall. Hermes source ZIPs contain only regular
files and directories; a symlink member is unambiguously suspicious.
Regression tests cover: symlink member rejection (raises ValueError,
caught by the outer try/except as a clean SystemExit, no extraction),
and the happy-path verification that a normal ZIP doesn't trigger the
symlink reject message.
Salvaged from PR #15881 by @codeblackhole1024. The remaining pieces of
that PR were already on main or contradicted explicit design decisions:
- config.yaml write-deny: already in agent/file_safety.py's
control_file_names denylist (the modern guard); the proposed addition
to build_write_denied_paths was the legacy path.
- Quick commands danger detection: contradicts the explicit
cli.py:8491-8492 comment 'shell=True is intentional: quick_commands
are user-defined shell snippets from config.yaml — not agent/LLM
controlled.'
- Memory plugin shlex.split for dep checks: already on main
(hermes_cli/memory_setup.py:133).
Co-authored-by: teknium1 <127238744+teknium1@users.noreply.github.com>
Codex / Responses-API requests had three latent timeout bugs that combined
into the long silent hangs reported on #21444:
1. The non-stream stale-call detector estimated context tokens from
``api_kwargs["messages"]`` only. Codex / Responses-API payloads carry
their conversational load in ``input`` (with ``instructions`` and
``tools``), so every Codex turn logged ``context=~0 tokens`` and the
detector never applied its >50k / >100k tier bumps.
2. ``providers.<id>.request_timeout_seconds`` was silently dropped on the
main Codex path. The chat_completions path and the auxiliary Codex
adapter both forwarded it; the main path skipped it through three
places (``build_api_kwargs``, ``ResponsesApiTransport.build_kwargs``,
``_preflight_codex_api_kwargs``).
3. The streaming stale detector had the same payload-shape bug for
``codex_responses`` requests, which route through the non-streaming
detector (it's the path that emits the user-facing
"No response from provider for 300s (non-streaming, ...)" warning that
reporters keep pasting).
This commit:
- Adds ``estimate_request_context_tokens`` in ``chat_completion_helpers``,
used by both the non-stream and stream detectors. Handles ``messages``
(Chat Completions), ``input + instructions + tools`` (Responses API),
bare lists, and an unknown-dict fallback.
- Forwards ``timeout`` through ``ResponsesApiTransport.build_kwargs``
and ``_preflight_codex_api_kwargs`` (with guards against
zero/negative/inf/bool values), and wires
``_resolved_api_call_timeout()`` into the Codex branch of
``build_api_kwargs``.
- Lowers the implicit non-stream stale defaults so fallback providers
kick in faster when upstream stalls:
* base 300s -> 90s
* >50k 450s -> 150s
* >100k 600s -> 240s
These only apply when the user has *not* set
``providers.<id>.stale_timeout_seconds`` or
``HERMES_API_CALL_STALE_TIMEOUT``. Explicit config still wins.
- Adds regression tests for the estimator shapes, the new defaults, the
context-tier scaling, transport timeout pass-through, and preflight
timeout pass-through / rejection of invalid values.
Closes#21444
Supersedes #21652#24126#31855
Co-authored-by: Hoang V. Pham <26063003+hehehe0803@users.noreply.github.com>
Add an opt-in Python plugin surface for speech-to-text backends,
mirroring the TTS hook pattern. New backends (OpenRouter, SenseAudio,
Gemini-STT, custom proprietary engines) can be implemented as plugins
without modifying tools/transcription_tools.py.
Built-ins always win
--------------------
The 6 built-in STT providers (local/faster-whisper, local_command,
groq, openai, mistral, xai) keep their native handlers. Plugins
attempting to register under a built-in name are rejected at
registration time with a warning and re-checked defensively at
dispatch.
Resolution order
----------------
1. stt.provider matches a built-in → built-in dispatch (unchanged)
2. stt.provider matches a registered plugin →
a. if plugin.is_available() returns False → unavailability envelope
identifying the plugin (not the generic "No STT provider"
message — the user explicitly opted into this plugin)
b. otherwise plugin.transcribe() with model + language forwarded
from stt.<provider>.{model,language} config
3. No match → legacy "No STT provider available" error (unchanged)
Per-provider config namespace
-----------------------------
Plugins read their config from stt.<provider> in config.yaml, mirroring
how built-ins read stt.openai.model / stt.mistral.model. The dispatcher
forwards `model` and `language` from this section. Caller's explicit
`model=` argument overrides the config-set model.
Files
-----
- agent/transcription_provider.py: TranscriptionProvider ABC
- agent/transcription_registry.py: register/get/list providers,
built-in shadow guard, _reset_for_tests
- hermes_cli/plugins.py: register_transcription_provider() on
PluginContext
- tools/transcription_tools.py: BUILTIN_STT_PROVIDERS frozenset,
_dispatch_to_plugin_provider() with availability gate, wire-in
after xai branch and before "No STT provider" error
- tests/agent/test_transcription_registry.py: 27 tests
- tests/hermes_cli/test_plugins_transcription_registration.py: 3 tests
- tests/tools/test_transcription_plugin_dispatch.py: 28 tests
(covering built-in short-circuit, plugin dispatch, exception
envelope, non-dict guard, availability gate, language forwarding)
- tests/plugins/transcription/check_parity_vs_main.py: 10-scenario
subprocess-pinned parity harness vs origin/main
- website/docs/user-guide/features/{tts,plugins}.md: docs
Behavior parity
---------------
10 scenarios, 8 OK + 2 expected DIFFs:
no_provider_error → plugin (plugin-installed scenario)
no_provider_error → plugin_unavailable (plugin-installed-unavailable
scenario; PR returns cleaner envelope)
Zero behavior change for users not opting into a plugin.
Issue follow-up to #30398.
PR #31416 (avoid persisting borrowed credential secrets) added
sanitize_borrowed_credential_payload, which strips access_token from
any auth.json pool entry whose (provider, source) isn't in the
_PERSISTABLE_PROVIDER_SOURCES allowlist.
(copilot, gh_cli) is borrowed (not in the allowlist), so the test
fixture's pre-seeded access_token now gets stripped at load_pool()
time, leaving the pool empty. resolve_target('1') then fails with
'No credential #1. Provider: copilot.'
Fix: align the test with the new contract. At runtime, copilot tokens
are hydrated by resolve_copilot_token() — mock that path so the pool
gets an entry the test can remove. The behavior under test
(suppression of gh_cli + env variants on remove) is unchanged.
CI repro on origin/main HEAD; reproduced locally with stock checkout.
