The dict-form guard from PR #67878 only covered the mapping shape
({model: {context_length: ...}}). The list-of-dicts shape
([{id: model, context_length: ...}]) is also a supported config form
(per _declared_model_ids) and was still being replaced with a flat
list of strings, destroying per-model metadata.
Sibling site for #67841.
When custom_providers[].models uses the mapping form to store
per-model metadata (e.g. context_length), _save_discovered_models_to_config
must not replace it with a flat list of strings. Add a guard that skips
entries whose models value is a dict, preserving the user's curated
metadata.
The regression was introduced by PR #65652, which added the auto-save
helper without considering the dict form.
Consolidates the three Qwen provider slugs (alibaba / Qwen Cloud,
alibaba-coding-plan / Alibaba Cloud Coding Plan, qwen-oauth / Qwen CLI
OAuth) under a single 'Qwen' group row in the interactive provider
pickers, matching the existing OpenAI / Kimi / MiniMax / xAI groups.
Display-only via PROVIDER_GROUPS — slug identity, --provider, and
/model <provider:model> paths are unchanged. Because group_providers()
is the shared fold, the CLI 'hermes model' picker, the setup wizard,
and the Telegram /model keyboard all pick up the grouping with no
per-surface changes.
Salvage of PR #67447 — the original PR fixed 3 of 7 missing keys.
gateway/config.py reads 4 more top-level keys (stt_echo_transcripts,
reset_triggers, always_log_local, filter_silence_narration) that
produced the same false 'Unknown top-level config key' warning.
Add all 4 and extend the regression test to cover them.
Hermes writes known_plugin_toolsets via tools_config and bridges
group_sessions_per_user / thread_sessions_per_user in gateway/config,
but doctor treated them as unknown top-level keys. Add them to
_EXTRA_KNOWN_ROOT_KEYS so validation matches keys Hermes itself uses.
* feat(desktop): add custom endpoint settings (supersedes #42745)
Salvages PR #42745 (elashera:custom-endpoints-desktop), which could no
longer merge cleanly against main. Re-integrated the work onto current
main and reconciled the conflicts:
- Settings nav: wired the new 'Custom Endpoints' provider sub-view into
main's data-driven navGroups/OverlayNav layout (PR predated that
refactor) and added it to PROVIDER_VIEWS.
- providers-settings: kept BOTH main's LocalEndpointRow affordance and
the PR's fuller CRUD panel; unified ProvidersSettingsProps to carry
onClose + onConfigSaved + onMainModelChanged.
- web_server: kept main's _normalize_main_model_assignment + api_key
propagation AND the PR's provider base_url lookup in
_apply_model_assignment_sync.
- model_switch: dropped the PR's bare direct-custom-config picker block;
main already implements it (source='model-config', with live model
discovery). Updated the salvaged test to assert main's behavior.
- Merged additive import/type blocks in hermes.ts and types/hermes.ts.
Backend endpoints, i18n labels (en/ja/zh/zh-hant), and the
custom-endpoints-settings.tsx panel carried over. 28 custom-endpoint
tests pass.
Co-authored-by: elashera <emilio.jesus.lasheras.romero@nttdata.com>
* chore(contributors): map elashera's commit email
Salvage of #42745 (superseded by #67759) preserves @elashera's
authorship, whose corporate commit email had no contributor mapping.
Adds contributors/emails/ mapping so check-attribution passes.
Verified: GitHub user 'elashera' id=135239963 matches their own
noreply commit email (135239963+elashera@users.noreply.github.com).
---------
Co-authored-by: elashera <emilio.jesus.lasheras.romero@nttdata.com>
* fix: speed up CLI /model picker by skipping non-current custom provider probing
The CLI /model picker calls build_models_payload() with default
probe_custom_providers=True, which live-fetches /v1/models from every
saved custom endpoint on every open. The GUI/desktop picker already
passes probe_custom_providers=False for snappiness.
Match the GUI behavior: skip probing non-current custom providers, but
still probe the current one so its model list stays accurate. Users can
force a full re-fetch with /model --refresh.
Fixes#65650
Related: #63583
* fix(cli): forward force_refresh to model picker probe flags
When /model --refresh is used, the CLI model picker must probe all
custom providers to refresh their model lists — not skip them.
Normal bare /model still skips non-current probes for speed.
Mirrors the existing desktop/TUI behavior. Add regression test for
both normal and refresh flag forwarding.
Fixes#65650
* fix: auto-save discovered models to config for discover-once caching
After a successful /v1/models probe, persist the discovered model list
back to config.yaml under the matching custom_providers entry. This
makes discover_models: false meaningful out of the box — users get a
populated cache after the first probe instead of a stale 1-model list.
- Add _save_discovered_models_to_config() helper
- Call after successful fetch_api_models in section 4 probe path
- Skip config write when model list hasn't changed
- Idempotent — no-op on empty api_url or model_ids
Tests: 4 new tests covering auto-save, empty-probe skip, unchanged
skip, and no-op-on-empty-args. All 4 pass.
Refs: #65652, #65650
---------
Co-authored-by: ajzrva-sys <302567740+ajzrva-sys@users.noreply.github.com>
grok-4.5 is xAI's newest release (their versioning is non-monotonic:
4.5 > 4.20) and is the model xAI's own docs use for the server-side
x_search tool. Users who explicitly pinned x_search.model keep their
choice; everyone else picks up the new default via the config
deep-merge — no _config_version bump needed.
- tools/x_search_tool.py: DEFAULT_X_SEARCH_MODEL
- hermes_cli/config.py: DEFAULT_CONFIG x_search.model + comment
- agent/reasoning_timeouts.py: 300s stale-timeout floor entry for
grok-4.5 (grok-4.20-reasoning entry kept for pinned users)
- docs: x-search.md en + zh-Hans (config sample + troubleshooting)
- tests: default-model assertion + timeout-floor positive case
Two follow-ups to the per-job model pin surface (#67472 / #49948 review):
- cron/scheduler.py: pass target_model=<effective job model> to
resolve_runtime_provider() on the primary path, so providers with
model-specific api_mode routing derive the mode from the model the job
actually runs (per-job pin > env > config default) instead of the stale
persisted default. The auth-fallback path already did this for its
fb_model.
- hermes_cli/web_server.py: POST /api/cron/jobs (and its sync worker) no
longer hardcodes profile="default" when the request carries no profile
param. A pool backend scoped to a named profile now resolves its own
profile via get_active_profile_name(), so pre-profileScoped desktop
clients can't write a named profile's job into ~/.hermes. Unscoped /
custom HERMES_HOME keeps the legacy default fallback.
Tests: target_model capture test on run_job; two profile-default tests on
the create endpoint.
* fix(desktop): stop contradicting the Ready pill with the one-time-install hint
When a provider's server-computed status is 'ready' (post_setup install
verifiably satisfied, e.g. cua-driver on PATH), the PostSetupRunner row
still said 'This backend needs a one-time install (…)'. Swap the copy for
a muted installed-confirmation one-liner and keep the Run setup button for
repair re-runs. Gated purely on the provider status prop so it composes
with the server-driven resting state work in the sibling lane.
* feat(tools): surface the web search/extract capability split in the Capabilities UI
The runtime has dispatched web_search and web_extract to independently
configurable backends for a long time (web.search_backend /
web.extract_backend overrides with web.backend as the shared fallback),
but the Capabilities tab still presented one monolithic 'Web Search &
Extract' choice that only wrote web.backend.
Backend:
- GET /api/tools/toolsets/web/config now returns active_search_backend /
active_extract_backend resolved via the REAL runtime getters
(tools.web_tools._get_search_backend/_get_extract_backend), plus each
provider row's web_backend key and supported capabilities (from the
registry's supports_search/supports_extract flags).
- PUT /api/tools/toolsets/web/provider accepts an optional capability
('search'|'extract') that writes web.<capability>_backend without
touching web.backend; validates the provider actually supports the
requested capability (ddgs/brave-free are search-only). Omitted →
unchanged legacy apply_provider_selection path.
- New tools_config.web_provider_capabilities() helper reads the plugin
registry's capability flags.
Frontend: 'Search: <backend>' / 'Extract: <backend>' pills above the web
provider matrix, per-row 'Search backend'/'Extract backend' assignment
pills, and 'Use for Search'/'Use for Extract' actions gated on each
backend's declared capabilities.
Tests: endpoint tests assert the runtime getters resolve to the written
backend (searxng for search, firecrawl for extract) after the endpoint
write; vitest covers badges, capability-gated buttons, and non-web
toolsets staying untouched.
* feat(desktop): deep-link Capabilities key rows to Settings → API Keys
Set env-var rows in the toolset config panel now offer 'Manage in API
Keys' in the row actions menu — an internal route change to
/settings?tab=keys&key=<ENV_KEY>. KeysSettings consumes the ?key= param
via the shared useDeepLinkHighlight hook (same mechanism as the command
palette's ?field= config deep links and ?session= archived-session
links): scrolls the credential card into view, flashes it, and expands
it. Applies generically to every env-var row, and only when the key is
set (unset keys are managed inline via Set). i18n in en/zh/zh-hant/ja.
* feat(desktop): point the vision Capabilities detail at Settings → Models
The vision toolset has no TOOL_CATEGORIES provider matrix — its
provider/model resolution runs through the auxiliary model config
(agent/auxiliary_client.py), so the Capabilities detail pane looked
empty with no hint of where the model choice lives.
Add a short explainer + an internal deep link
(/settings?tab=config:model&aux=vision) rendered only for
toolset.name === 'vision'. ModelSettings consumes the ?aux= param via
the shared useDeepLinkHighlight hook and scrolls/flashes the matching
auxiliary task row (rows now carry aux-task-<key> anchor ids). No
external URLs. i18n in en/zh/zh-hant/ja.
* test(desktop): use type-alias imports for the react-router mock (lint)
* chore: drop accidentally committed node_modules symlinks
* chore: drop remaining committed node_modules symlinks (apps/desktop, apps/shared)
Refactor the cherry-picked #40338 backend half:
- Move option merging from import-time _SCHEMA_OVERRIDES mutation to a
per-request overlay in GET /api/config/schema — options now reflect the
current config.yaml (no restart needed) and the module-level
CONFIG_SCHEMA is never mutated. The endpoint gains an optional
?profile= param scoped via _config_profile_scope.
- Keep builtin display order first, customs appended (drop the
sorted(set(...)) re-sort) — matches desktop enumOptionsFor.
- Only command-type provider blocks count (type absent or 'command' plus
non-empty command string), enumerated from the canonical
<kind>.providers.* location AND the legacy top-level <kind>.<name>
fallback — the same dual resolution as _get_named_provider_config /
_get_named_stt_provider_config. Builtin-name collisions are excluded
case-insensitively against the RUNTIME builtin sets (not the display
shortlist), mirroring apps/desktop/src/app/settings/helpers.ts
commandProviderNames (#67209).
- Drop the plugin.yaml 'provides: [tts]' manifest scan — that convention
does not exist (manifests carry provides_tools/provides_hooks only);
plugin TTS/STT providers register at runtime via
ctx.register_tts_provider(). Instead, opportunistically include names
from agent.tts_registry / agent.transcription_registry when plugins
happen to be loaded in this process.
- Current tts.provider/stt.provider value preserved in options.
- Tests: custom command provider merge (tts+stt), builtin-order
preservation, EDGE collision exclusion, non-command block exclusion,
current-value preservation, per-request freshness, legacy top-level
block support.
* fix(windows): suppress console-window flash in tools post-setup subprocess spawns
The desktop GUI runs post-setup hooks via a detached, console-less
'hermes tools post-setup <key>' child (spawned with windows_detach_flags).
But the hook implementations in tools_config.py ran their inner installers
(npm install, agent-browser install, uv/pip installs, ensurepip, cua-driver
version probes and installer) without Windows creationflags — and on
Windows a console-less parent spawning a console/.cmd child materializes a
brand-new console window, the 'terminal flash' reported on the
Capabilities > Browser Automation setup journey.
Add _post_setup_no_window_flags(), a local wrapper around
windows_hide_flags() (CREATE_NO_WINDOW only — DETACHED_PROCESS would sever
stdio and break capture_output), and pass it at every post-setup subprocess
call site. Spawns that stream live output to the user's console
(verbose cua-driver install) only hide when stdout is not a tty, so
interactive CLI installs keep their output. POSIX behavior is unchanged
(the helper returns 0 off-Windows).
* fix(desktop): make Capabilities post-setup idempotent — Installed state instead of unconditional Run setup
The GUI panel rendered the primary 'Run setup' CTA whenever a provider
declared post_setup, ignoring the server-computed readiness status the
config endpoint already serves. Users on Windows clicked 'Run setup' on
an already-installed Local Browser and watched it 'install' again.
Frontend: PostSetupRunner now takes installed (provider.status === 'ready')
and renders an 'Installed' pill + small 'Re-run setup' text button in that
state; onComplete still refetches the toolset config, so a fresh install
flips the row to Installed once the endpoint reports ready.
Backend:
- _POST_SETUP_READY extended: agent_browser now tracks the FULL local
install (_local_browser_runnable: CLI + Chromium-or-Lightpanda) instead
of the bare CLI check; new entries for the cloud 'browserbase' hook
(CLI only — cloud rows host their own Chromium) and camofox (npm
package present).
- _run_post_setup prints distinct 'already installed, nothing to do'
messages for the agent-browser/Chromium/Camofox early-exits so the GUI
action log tells the truth on re-runs vs fresh installs.
i18n: new postSetupInstalled/postSetupRerun/postSetupInstalledHint strings
in en, ja, zh, zh-hant + types.
* fix(desktop): let managed Nous Subscription rows activate from the GUI via the Portal sign-in flow
PUT /api/tools/toolsets/{name}/provider intentionally skips the Nous
Portal auth gate the CLI runs inline (ensure_nous_portal_access) — but no
desktop surface handled it. Selecting 'Nous Subscription (Browser Use
cloud)' from Capabilities wrote browser.cloud_provider=browser-use +
use_gateway=true and then silently never activated: _is_provider_active
requires feature.managed_by_nous, which stays false without the
entitlement, and the credential was never used.
Backend: after apply_provider_selection, the endpoint now checks the
managed row's entitlement (get_nous_subscription_features force_fresh +
the same per-category coverage gate the CLI applies) and reports the gap
with additive response fields {needs_nous_auth: true, feature}. The
selection is still persisted — activation is what's gated.
Frontend: handleSelect surfaces a 'Sign in to Nous Portal' warning toast
with a Sign-in action instead of the misleading success toast. The action
drives the EXISTING Nous Portal OAuth device-code flow (provider id
'nous' in _OAUTH_PROVIDER_CATALOG): POST /api/providers/oauth/nous/start,
open verification_url, poll /poll/{session}; on approval the panel
refetches the toolset config so is_active/status flip.
i18n: nousAuthNeeded*/nousAuthSignIn/nousAuthDone*/nousAuthFailed strings
in en, ja, zh, zh-hant + types.
Follow-up to the salvaged #56724: the runtime's _generate_xai_tts reads
voice_id, language, speed, auto_speech_tags, optimize_streaming_latency,
sample_rate, and bit_rate — but never text_normalization, and the xAI
/v1/tts payload builder has no such field. Surfacing it in the desktop
GUI would be a dead knob, so remove it from DEFAULT_CONFIG, constants.ts
(labels/descriptions/SECTIONS), and the ja/zh/zh-hant locale catalogs.
The other six xAI keys are all verified against tools/tts_tool.py.
Sibling-site fix for the flaw addressed in the global 'Configure all
platforms' flow: the per-platform checklist also returned to the menu
without opening provider setup when a selected toolset was already
enabled but lacked provider configuration. Adds a matching regression
test.
* fix(env): recognize export-prefixed .env lines in save/remove (#40041)
load_env() parses bash-compatible 'export KEY=value' lines (#6659), so a
hand-added 'export GITHUB_TOKEN=ghp_...' shows as set (green light) in the
desktop Tools & Keys page. But save_env_value/remove_env_value only matched
plain 'KEY=' lines:
- DELETE /api/env 404'd ('not found in .env') — the token could not be
removed through the UI
- PUT /api/env appended a SECOND line; a later delete removed the new line
while the export line silently resurrected the old value
Both writers now match assignments through a shared _env_line_defines_key()
helper that understands the export prefix. Commented-out lines are still
ignored.
Regression tests drive the real dashboard endpoint handlers against a temp
HERMES_HOME with runtime-constructed classic-PAT-shaped fixtures, covering
save-does-not-500, export-line remove, export-line replace-without-duplicate,
and the plain-line path staying intact.
Fixes#40041
* fix(credentials): unify provider key delete/update across .env, auth.json, config.yaml (#51071#59761#62269)
A provider API key can live in three stores at once: ~/.hermes/.env,
auth.json credential_pool (env-seeded 'env:<VAR>' entries persisted by the
pool loader), and config.yaml mirrors (model.api_key, auxiliary.*.api_key,
custom_providers[*].api_key). The desktop/dashboard endpoints and the TUI
gateway RPCs only ever mutated .env, so the stores diverged:
- #51071/#59761: DELETE /api/env removed the key from .env but left the
credential_pool entry (the loader is additive-only and never prunes),
so the provider kept appearing in the model picker — surviving restart
via the stale pool entry + provider_models_cache.json row.
- #62269: PUT /api/env rewrote .env but left the OLD key in config.yaml
(model.api_key wins over env at client construction), producing 401s
with a key the UI no longer showed.
New hermes_cli/credential_lifecycle.py is the single choke point:
- remove_provider_env_credential(): clears the .env entry, prunes
env:<VAR> pool entries across ALL providers (a shared var like
GITHUB_TOKEN can seed several), suppresses the env source so a lingering
shell export can't re-seed it (matching 'hermes auth remove' semantics),
drops the affected providers' model-cache rows, and scrubs value-matched
config.yaml api_key mirrors. Returns 'found' spanning every store so a
stale pool-only entry is cleanable through the same delete button.
- save_provider_env_credential(): writes .env, rotates any config.yaml
mirror that held the PREVIOUS value (value-matched — an unrelated inline
key is untouched), and lifts a prior env-source suppression so re-adding
behaves like 'hermes auth add'.
OAuth preservation: only entries with source == 'env:<VAR>' are pruned.
OAuth/device-code/manual/borrowed pool entries and providers.<id> OAuth
token blocks are never touched by a key-only delete. (model.disconnect in
the TUI gateway still clears OAuth via clear_provider_auth — that surface
is a full provider disconnect, which is the documented intent there.)
Rerouted call sites: PUT/DELETE /api/env (dashboard + desktop),
tui_gateway model.save_key / model.disconnect, save_env_value_secure
(TUI/gateway secret capture), and hermes config set/unset for env-shaped
keys.
E2E tests drive the real endpoint handlers against temp-HERMES_HOME
fixtures (.env + auth.json + config.yaml with runtime-constructed fake
keys) and assert cross-store consistency after delete/update, pool-reload
survival ('restart'), OAuth preservation, models-cache invalidation, and
the suppress/unsuppress round-trip.
Fixes#51071Fixes#59761Fixes#62269
* fix(desktop): compute truthful per-provider readiness for Capabilities tool config
Backend: GET /api/tools/toolsets/{name}/config now sends a per-provider
'status' field ('ready' | 'needs_keys' | 'needs_auth' | 'needs_setup')
computed by the new provider_readiness_status() in tools_config:
- env vars declared: all set -> ready, else needs_keys
- Nous-managed rows: Portal login + per-category tool-gateway entitlement
(MANAGED_FEATURE_COVERAGE_CATEGORY) -> ready, else needs_auth
- post_setup 'xai_grok' rows: Grok OAuth or XAI_API_KEY -> ready, else
needs_auth
- other keyless post_setup rows: installed-state predicate
(_POST_SETUP_READY: kittentts/piper/ddgs/langfuse via find_spec,
agent_browser via _has_agent_browser, cua_driver via PATH probe);
unknown hooks fall back to is_active as the setup-completed signal
- genuinely-free keyless rows (Edge TTS) stay ready
Existing fields are untouched; 'status' is additive so older desktops
keep working.
* fix(desktop): render server readiness status in Capabilities provider pills
The panel's providerConfigured() heuristic pilled every zero-env-var
provider 'Ready' — including logged-out Nous Subscription rows, xAI TTS
without Grok OAuth, and never-installed KittenTTS/Piper. Render the
backend's per-provider 'status' instead:
- ready -> existing 'Ready' pill
- needs_auth -> warn pill 'Needs sign-in'
- needs_setup -> warn pill 'Needs setup'
- needs_keys -> no pill (env-var fields are the signal)
Keyed rows keep deriving ready/needs_keys from local envState so saving
or clearing a key updates the pill without a refetch. Older backends
without 'status' fall back to the legacy env-var heuristic (narrow
compat path, desktop/runtime update on separate clocks).
Adds the warn tone to the settings Pill primitive and needsSignIn /
needsSetup strings to all locale catalogs (en, zh, zh-hant, ja).
Follow-ups on the salvaged unknown-root-key warning from PR #67345:
- Add image_gen, video_gen, plugins, smart_model_routing, platform_toolsets,
session_reset, multiplex_profiles, profile_routes, platforms,
require_mention, unauthorized_dm_behavior, and signal to
_EXTRA_KNOWN_ROOT_KEYS — all are read from the raw user YAML (gateway,
registries, plugin CLI) or written by our own setup wizard, but absent
from DEFAULT_CONFIG. Without this, doctor would warn on configs Hermes
itself wrote.
- Convert tests/hermes_cli/test_config_validation.py back to LF line
endings (the PR's rewrite introduced CRLF).
Extracted from the config-validation portion of PR #67345 (the token-cost
half was not salvaged). Unknown top-level config keys now warn (naming the
key) instead of being silently ignored; known roots derive from
DEFAULT_CONFIG.keys() plus a small extras set for valid-on-disk roots
absent from defaults.
Two config-hygiene improvements (warning-only, non-blocking):
1) Unknown top-level config keys now surface a warning naming the key (known roots derived from DEFAULT_CONFIG.keys() as single source of truth) so typos like 'skillz:'/'secrity:' are no longer silently ignored. Provider.* unknown-key behavior preserved.
2) hermes doctor reports deprecated/legacy config keys (display.tool_progress_overrides, delegation.max_async_children, compression.summary_*) and legacy env vars (HERMES_TOOL_PROGRESS*, TERMINAL_CWD, QQ_HOME_CHANNEL*) with their modern replacements, as non-failing warnings. No auto-delete/migrate.
Tests: config_validation + doctor suites green (100 passed).
A final response generated but not confirmed-delivered was the one
artifact the gateway could lose without a trace: crash or planned
restart between finalize and platform ACK dropped it silently, and the
resume path re-ran the whole turn at full cost (#58818 P1, #41696,
#63695's gateway half).
gateway/delivery_ledger.py records each outbound final response in
state.db (same conventions as the async-delegation ledger: WAL, owner
pid + process-start-time liveness, bounded retention):
pending -> attempting -> delivered | failed
startup sweep on dead-owner rows -> redeliver | abandoned
Contract (the lessons from the closed delivery-outbox attempt #61790):
- obligation recorded BEFORE the first send attempt; cleared only on
SendResult.success (destination acceptance, #51184)
- ambiguity is labeled, never silently retried: rows that were mid-send
when the process died redeliver with a visible '♻️ Recovered reply —
may be a duplicate' prefix (honest at-least-once)
- stable ids from session_key + inbound message id + content, so
distinct threads/topics can never collide
- poison rows bounded: 3 attempts / 24h freshness -> abandoned; claim
atomically re-stamps ownership so racing sweeps can't double-claim
- redelivery clears resume_pending for the session so the resume path
never re-runs a turn whose answer the ledger already holds
- best-effort everywhere: ledger failure can never block or delay a send
- slash-command/ephemeral/empty responses are not recorded; cron and
proactive delivery stay on DeliveryRouter (separate subsystem)
Config: gateway.delivery_ledger (default on; no version bump needed).
Validation: 30 ledger+producer tests; 352 blast-radius gateway tests
green; cross-process E2E (record in process A, kill it mid-send, claim
+ marker + redeliver in a fresh process B against the same state.db).
Dusk1e's fix wired _restore_session_cwd into _handle_resume_command, but that
handler was extracted from cli.py into hermes_cli/cli_commands_mixin.py
(094aa85c3) after the PR's base, so the original cli.py hunk no longer applied
(a naive cherry-pick fuzzily misplaced it inside new_session(), where
session_meta is undefined). Transplanted the call to the end of the handler in
its current home; /sessions <id> delegates here so both command forms are
covered. Dusk1e's regression tests carry over unchanged.
Co-authored-by: Dusk1e <yusufalweshdemir@gmail.com>
Deduplicates the sidecar handling across 9 sites:
- substitute_api_content(): 2 API-bound pop+substitute sites
(chat_completion_helpers, transport — conversation_loop keeps its
inline pop because the current-turn compose fallback needs the value)
- drop_stale_api_content(): 4 rewrite-drop sites
(context_compressor x2, replay_cleanup, agent_runtime_helpers)
- extract_api_content_sidecar(): 3 gateway forwarding sites
(gateway/session, gateway/slash_commands, cli_commands_mixin)
Also: restore the eager _pending_cli_user_message clear on the early
row-creation try (was lost when the crash-persist moved after prefetch —
a crash in compression between the two tries would leave a stale staged
input), and fix a comment indentation nit in run_agent.py.
Co-authored-by: Soju06 <qlskssk@gmail.com>
The first LLM call of every gateway turn gets ~0% provider prompt-cache
hit rate (in-turn calls: 97-99%) because the bytes sent for a turn's
user message are not the bytes replayed next turn: memory-prefetch and
pre_llm_call context are injected into the API copy only, the #48677
persist override writes cleaned content to the DB row, and
get_messages_as_conversation sanitize/strips user and assistant content
on load. Any of these diverges the request prefix at that message and
re-prefills everything after it — measured 27.9s for the first call vs
2.4-5.8s cached at a median ~156k-token context.
Persist what you send: a nullable messages.api_content column stores the
exact content string sent to the API when it differs from the clean
stored content, and replay substitutes it verbatim (no sanitize, no
strip). The injection composition lives in one helper
(turn_context.compose_user_api_content); the turn prologue stamps its
output onto the live user message, the api_messages build sends the
stamped bytes, and every outgoing copy pops the field so it never
reaches a provider. The crash-resilience user-turn persist moves after
prefetch/pre_llm_call so the user row is written once with its final
sidecar; _ensure_db_session stays before preflight compression (session
rotation needs the parent row under PRAGMA foreign_keys=ON). The
current-turn index trackers are re-anchored after compaction rebuilds
the message list, in-place preflight compaction backfills the stamp onto
the already-inserted row, and gateway replay forwards the sidecar only
when the replay pipeline did not rewrite the content. Rewrite paths that
would leave stale bytes (historical image strip, merge-summary-into-
tail, consecutive-user repair merge, stale-confirmation redaction) drop
the sidecar; the chat-completions transport and the max-iterations
summary path strip it defensively. codex_app_server and MoA turns are
excluded from stamping because their wire bytes differ from the
composition. A missing or dropped sidecar degrades to today's behavior
(one cache-boundary miss), never to wrong content.
The sidebar refresh fired three /api/profiles/sessions calls (recents, cron,
messaging), and each one reopened every selected profile's state.db and re-ran
list_sessions_rich + session_count — ~3N DB opens/counts per refresh, on every
turn/broadcast/reconnect.
Add GET /api/profiles/sessions/sidebar: one pass that opens each profile DB
once and runs the three source-scoped queries together (recents scoped to the
active profile; cron + messaging cross-profile), returning the three windows in
one payload. Same read-only projection, 300s active heuristic, and caller-
supplied source taxonomy (recents_exclude / messaging_exclude / source=cron) as
the per-slice endpoint.
Renderer refreshSessions now makes one listSidebarSessions call and distributes
recents/cron/messaging to their stores (cron *jobs* stay a separate getCronJobs
API). Electron splices remote profiles per slice via fetchProfilesSessionSlice
(reusing the proven per-slice merge) so remote correctness is preserved; the
no-remote common case gets the single-open fast path.
From the Desktop performance audit (P1: "Batch sidebar session slices").
Hot-reload and multi-entry load_hermes_dotenv can hit the same UTF-32
file repeatedly; gate the refuse-to-mangle warning on a module-level
seen-set (house style: _WARNED_KEYS sibling) so logs are not spammed.
Notepad "Unicode" saves write UTF-16 with a BOM. The sanitizer decoded
those bytes as utf-8-sig with errors=replace, glued U+FFFD onto the first
key name, stripped NULs, and rewrote the mangled content permanently.
Sniff leading BOMs before any text decode (UTF-32 before UTF-16, because
UTF-32-LE's BOM starts with UTF-16-LE's FF FE). Decode UTF-16 correctly
and rewrite as clean UTF-8. Refuse UTF-32 (leave untouched + warning).
After errors=replace, do not persist a first line that starts with U+FFFD.
Does not touch _load_dotenv_with_fallback (#65124's surface).
_quote_env_value previously left internal spaces unquoted (only #/"/'
and leading/trailing whitespace triggered). Spaced macOS paths written
via hermes setup SSH / Google Chat SA path / hermes config set produced
lines that python-dotenv still parsed but shell `set -a; . file` word-split.
Extend needs_quoting with any(c.isspace()); escaping dialect unchanged.
Built-in memory (MEMORY.md/USER.md) is controlled by memory_enabled, not
memory.provider — but the desktop dropdown offered 'builtin' as a normal
provider-plugin value and gave it plugin-shaped affordances (config panel,
OAuth connect row), and the empty sentinel rendered as '(none)' even though
built-in memory was active.
- Label the empty memory.provider option 'Built-in only' (all locales).
- Drop the literal 'builtin' option from the desktop ENUM_OPTIONS and the
backend config-schema select; _normalize_memory_provider_name already maps
legacy builtin/built-in/none values to ''. A stored legacy literal stays
visible via enumOptionsFor's current-value passthrough.
- Gate MemoryConnect and ProviderConfigPanel behind a new
isExternalMemoryProvider() helper so built-in aliases never get
provider-plugin affordances.
_make_tui_argv() called _ensure_tui_workspace(tui_dir) unconditionally
before checking for a prebuilt bundle. That function sys.exit(1)s when
ui-tui/ doesn't exist, which it never does on a pip/pipx install — the
wheel ships hermes_cli/tui_dist/entry.js but never ships ui-tui/ at all
(that directory only exists in a git checkout).
Every dashboard Chat tab connection on a pip/pipx install therefore
hard-exited before ever reaching _find_bundled_tui(), surfacing as the
unhelpful "Chat unavailable: 1" banner despite having a fully valid
bundled entry.js on disk.
Move the bundled-wheel/HERMES_TUI_DIR shortcut ahead of the workspace
check. --dev is unaffected (it never uses the bundled path and still
requires the workspace), and the checkout-without-bundle path is
unaffected (bundled lookup returns None, falls through to the existing
git-restore/npm-install/build flow).
Adds a contributors/emails mapping for the original author.
Fixes#56665
Co-authored-by: lucaskvasirr <lucaskvasir@duck.com>
Adds --once to /model across CLI, TUI, and gateway: switch model for the
next turn only, restoring the previous model in a finally block so
success, exception, and interrupt all revert. Parsing extends
parse_model_flags_detailed(); resolve_persist_behavior() treats --once
as a persistence opt-out; --global + --once is rejected.
Salvaged from PR #29923 (image-generation lane split to #59815 per
review; conflict resolution against current main by the maintainers).
Route recovered messages through the live Discord ingress policy, preserve dedup and completion invariants, bound and retain the recovery ledger, and expose the opt-in config with docs and backup coverage.
Salvaged from PR #25653 by @Black0Fox0 — the config-key and env-wiring
halves of that PR landed via #67018; this carries the surviving dashboard
schema override so browser.headed renders as a labeled boolean toggle.
Description updated to reflect the merged cleanup-skip behavior.
Replace REST-based Discord liveness probe with local WebSocket/heartbeat
state detection. REST success doesn't prove Gateway event delivery — a
half-closed WebSocket can leave Bot.start() alive while REST returns 200.
Now samples ready/open/ACK state and heartbeat latency; consecutive
unhealthy samples emit one retryable fatal code so GatewayRunner rebuilds
the adapter through the existing reconnect path.
Also fixes three lifecycle gaps in the recovery path:
1. asyncio.wait_for() can remain blocked if adapter cleanup swallows
cancellation — now uses bounded asyncio.wait() with task detachment.
2. Multiplexed secondary-profile adapters had no profile-scoped reconnect
owner — now uses one runner-owned reconnect slot per profile.
3. An in-flight turn could send its final text through the disconnected
adapter after a replacement was registered — now resolves the live
same-profile replacement for unsent final responses only (message IDs
never migrate, edits/deletes stay on the old transport).
Adds an opt-in Linux/systemd event-loop watchdog (gateway.systemd_watchdog_seconds,
default 0) for the failure mode where the whole asyncio loop stops making
progress and no in-process liveness task can run. stdlib-only sd_notify,
Type=notify/WatchdogSec generation, READY/STOPPING lifecycle.
Co-authored-by: 王鑫 <wx.xw@bytedance.com>
Follow-up to PR #66479 salvage. The three new env vars
(HERMES_LOCAL_STREAM_STALE_TIMEOUT, HERMES_GATEWAY_MAX_STARTS,
HERMES_GATEWAY_START_WINDOW_S) were introduced as bare env-var reads with no
config.yaml surface or documentation — violating the .env-is-for-secrets-only
policy (behavioral settings must live in config.yaml, bridged to env internally).
- config.yaml: add gateway.respawn_storm {max_starts, window_seconds} to
DEFAULT_CONFIG, mirroring the existing restart_loop_guard pattern.
- gateway.py: read config.yaml first, env vars override as escape-hatch.
- environment-variables.md: document all three new env vars, noting the
config.yaml alternative for the gateway ones.
- chat_completion_helpers.py: cross-reference the env-var docs from the
local stale timeout comment.
Re-applied against v0.18.2. Still unconditional KeepAlive <true/> with no
ThrottleInterval, and upstream's new restart_loop_guard only skips session
auto-resume (never throttles the boot), so the launchd respawn storm is
unguarded.
- launchd plist: add ThrottleInterval=30 + ExitTimeOut=25.
- status: record_start_and_check_storm — portable breaker (atomic gateway-starts.log,
backs off when too many starts land in a window); run_gateway sleeps it before
asyncio.run. Env HERMES_GATEWAY_MAX_STARTS (<=0 disables) / _START_WINDOW_S.
Separate file from upstream's restart_loop.json — no collision.
- tests: breaker (threshold/prune/atomic) + plist throttle keys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(shared): move terminal-billing wire types to @hermes/shared
The billing/subscription wire shapes (plus UsageBarData/UsageModelData,
which they reference) move verbatim from ui-tui/src/gatewayTypes.ts into
apps/shared/src/billing-types.ts so the desktop app can share the same
gateway contract. gatewayTypes.ts re-exports every moved name from the
new @hermes/shared/billing subpath, so no ui-tui consumer changes.
The subpath export keeps DOM-less ui-tui from pulling the barrel (whose
WebSocket helpers need the DOM lib). ui-tui also now declares its
@hermes/shared dependency explicitly instead of relying on workspace
hoisting.
* test(cli): pin nous_billing wire-layer status-to-exception mapping
The HTTP layer's error handling had zero coverage through _request:
only 2xx parsing and request shaping were tested, and the mapping cases
in test_remote_spending_gate_contract.py hit _raise_for_error directly.
Adds 19 tests driving _request via a monkeypatched urlopen: the
401-refresh-retry path (success, terminal plain/session_revoked,
idempotency-key preservation, base re-resolution), 403 variants through
the wire, 429/503 retry-after, non-JSON error bodies, 404/502
fallbacks, and URLError normalization.
Two behaviors are pinned as findings rather than fixed: a JSON-body
retryAfter hint is ignored unless the Retry-After header is present,
and a bare socket.timeout propagates uncaught (real urllib wraps
timeouts in URLError before this layer).
* fix(cli): normalize read-phase timeouts to the typed billing error
urlopen wraps connect-phase timeouts in URLError (already mapped to
network_error), but a timeout during resp.read() raises a bare
TimeoutError that escaped the typed-BillingError contract and reached
callers as an unhandled exception. Catch it narrowly and normalize.
The boundary test now asserts normalization instead of documenting the
leak.
* fix(shared): stop typing mutation success payloads as error payloads
BillingMutationResponse.payload was declared BillingErrorPayload, but on
ok:true the gateway passes through the raw NAS success body (rail,
changeType, cancelAtPeriodEnd, ...). The TUI never reads it so nothing
broke, but the shared contract now feeds the desktop app too — widen the
field deliberately and document both shapes.
* feat(shared): typed billing refusal and charge-failure unions
- BillingRefusalCode covers every code the gateway serializes today, with a
(string & {}) arm so unknown future codes (the NAS W3 card-health family)
stay assignable — consumers keep their unknown-code fallback.
- ChargeFailureReason models the four NAS terminal reasons plus the raw
subscription_payment_intent_requires_action code NAS leaks pre-#711.
- billing.state now carries the server-derived can_change_plan the gateway
emits; capability comments updated (canChangePlan is capability-based, not
an OWNER/ADMIN role gate).
* feat(shared): closed Known* halves for the refusal and charge-failure unions
- KnownBillingRefusalCode / KnownChargeFailureReason are closed literal sets,
so classification tables, copy maps and tests can be Record-exhaustive and
break at compile time when a code is added but not mapped. The wire types
keep the (string & {}) open arm for unknown future codes.
- Add network_error (client-originated transport code the gateway already
serializes) to the known set.
- Export the union types from the root barrel alongside the other billing
names.
* feat(shared): canonical billing refusal policy and charge-settlement driver
- billing-policy.ts: one exhaustive Record<KnownBillingRefusalCode,
BillingRefusalPolicy> classifying every known code (recovery kind,
mid-poll ambiguity, idempotency-key reuse) with a documented unknown-code
fallback. Surfaces keep their own copy; the behavior classification now
has a single home that breaks the build when a new code goes unmapped.
- charge-settlement.ts: the settlement poll state machine (2s cadence,
5-minute cap, bounded retry-after backoff, ambiguous-on-revocation) as a
pure dependency-injected driver returning a discriminated outcome.
- The TUI's pollCharge becomes a thin renderer over the shared driver —
byte-identical output, and the desktop poller can now share the same
machine instead of a drifting copy.
On Windows (and some Linux setups), an application like VS Code's
js-debug can hold 127.0.0.1:9222 while a Chromium browser launched
with --remote-debugging-port=9222 silently binds [::1]:9222 only.
The IPv4-only probe then (a) missed the live browser entirely and
(b) hung against the squatter — which accepts TCP but never answers
the /json/version HTTP probe — repeatedly, driving the whole connect
past the desktop GUI's RPC deadline:
'error: request timed out: browser.manage'.
Fix, applied to both the gateway browser.manage RPC and the CLI
/browser connect path via shared helpers in browser_connect.py:
- discover_local_cdp_url(): probe BOTH loopbacks (127.0.0.1 first,
then [::1]) and adopt whichever actually speaks CDP.
- local_port_in_use() + find_free_debug_port(): when neither loopback
speaks CDP but the port is held by another application, report the
squatter explicitly and launch the debug browser on a nearby free
port instead of fighting a bind conflict on 9222.
- Bound the gateway's post-launch wait to a 10s deadline (was up to
20 unbounded probe cycles) so connect always answers inside the
client RPC timeout.
- _wait_for_browser_debug_ready_or_exit() also probes dual-stack so a
successful launch pushed onto [::1] is classified 'ready'.
Verified on a live Windows repro (VS Code holding 127.0.0.1:9222,
Chrome 148 on [::1]:9222): connect now resolves http://[::1]:9222
in ~4.5s instead of timing out.
Follow-up to the salvaged #66609 (4 primary readers) and #41604 (context
files): two more jobs.json readers rejected a BOM'd file —
- hermes_cli/backup.py _count_cron_jobs: a BOM made the count None,
silently disabling the post-update cron-loss auto-restore safety net
- agent/curator_backup.py _backup_cron_jobs_into: BOM broke the job
count (spurious parse_warning) and propagated the BOM into snapshots
Both now read utf-8-sig; curator snapshots are written BOM-free so
rollback restores a file load_jobs can read. AUTHOR_MAP entry added
for deacon-botdoctor.
Tests: BOM'd-live-file auto-restore + BOM'd snapshot count/BOM-free copy.
