Commit graph

19255 commits

Author SHA1 Message Date
teknium1
ed33ebca1d refactor: canonical config loaders for behavioral reads + guarded raw-read primitive (kills the managed-scope/env-expansion drift class)
The disease: ~15 scattered raw yaml.safe_load(config.yaml) reads that
silently miss managed-scope overlay, ${ENV_VAR} expansion, profile-aware
pathing, and root-model normalization. Every new config feature needed an
N-site sweep (incident chain 9cbcc0c9c8732293cf87b0e47a98f91928aa0443). This commit assigns every raw read to an owner and adds a
lint-guard test so the class cannot regrow.

New primitive (additive-only change to hermes_cli/config.py):
  read_user_config_raw(path=None) — reads the user file EXACTLY as
  written; docstring states it is ONLY legal for write-back round-trips
  and raw-file diagnostics. Behavioral reads must use
  load_config()/load_config_readonly().

BEHAVIOR FIXES (class-a sites migrated to a canonical loader — these
previously read values that could DIFFER from the effective config):

  gateway/run.py _try_resolve_fallback_provider → _load_gateway_runtime_config
    keys: fallback_providers/fallback_model (provider, model, base_url,
    api_key). Drift fixed: a managed-pinned fallback chain was ignored;
    an api_key of "${OPENROUTER_API_KEY}" reached the resolver unexpanded.
  gateway/run.py GatewayRunner._load_provider_routing → same loader
    key: provider_routing. Drift fixed: managed-pinned routing prefs and
    ${VAR} templates were ignored.
  gateway/run.py GatewayRunner._load_fallback_model → same loader
    keys: fallback chain. Same drift as above.
  gateway/run.py GatewayRunner._refresh_fallback_model
    keeps the raw primitive (its last-known-good-on-parse-failure contract
    forbids the fail-open loader, which returns {} on a torn write) but now
    applies managed overlay + env expansion inline. Drift fixed: chain
    edits under managed scope / env templates were previously frozen out.
  tui_gateway/server.py _load_cfg (72 behavioral call sites)
    now = raw read + managed overlay (pre-existing) + NEW ${VAR} expansion,
    split from a new _load_cfg_raw() write-back primitive. Drift fixed:
    e.g. custom_prompt: "hello ${VAR}", agent.system_prompt, model,
    api_key/base_url templates reached sessions unexpanded. DEFAULT_CONFIG
    is deliberately NOT merged (callers treat missing keys as unset;
    `_load_cfg() == {}` sentinels and _save_cfg round-trips depend on it).
  tui_gateway/server.py _profile_configured_cwd
    keys: terminal.cwd of a NON-launch profile. Drift fixed: managed
    overlay + ${VAR} expansion now apply (load_config() would resolve the
    wrong profile's home, so the raw primitive + inline pipeline is used).
  plugins/platforms/telegram/adapter.py _reload_dm_topics_from_config
    → load_config_readonly(). keys: platforms.telegram.extra.dm_topics.
    Drift fixed: managed overlay + profile-aware pathing + expansion.
  plugins/memory/holographic _load_plugin_config → load_config_readonly().
    keys: plugins.hermes-memory-store.*. Same drift class.

WRITE-BACK ROUND-TRIPS (class-b: stay raw BY DESIGN via read_user_config_raw;
merging defaults/overlay would pollute the saved user file):
  gateway/slash_commands.py: model persist x2, _save_gateway_config_key,
    memory/skills write_approval toggles
  gateway/platforms/yuanbao.py auto-sethome
  tui_gateway/server.py _write_config_key + all cfg→_save_cfg blocks
    (reasoning show/hide/full/clamp, details_mode[.section], prompt)
    → new _load_cfg_raw()
  plugins/memory/holographic save_config

RAW-FILE DIAGNOSTICS + presence-sensitive bridges (class-c: stay raw,
now via the shared primitive with an explanatory comment):
  hermes_cli/doctor.py x5 (model validation, stale-root-keys, .env drift,
    deprecation sweep, memory-provider probe — the latter two keep their
    inline managed overlay where they had one)
  gateway/run.py _bridge_max_turns_from_config and the module-level
    TERMINAL_*/HERMES_* env bridge (bridging merged defaults would export
    all of DEFAULT_CONFIG into the environment; both keep their inline
    overlay + expansion)
  hermes_cli/send_cmd.py env bridge (same presence-sensitivity)
  hermes_cli/gateway.py multiplex-conflict probe (reads the DEFAULT root's
    config, not the active profile's — load_config is the wrong owner)
  hermes_cli/profiles.py / hermes_cli/web_server.py / tools/wake_word.py
    multi-profile reads (load_config targets only the ACTIVE profile home)
  cron/jobs.py _resolve_default_model_snapshot and cron/scheduler.py
    run_job config read keep their existing inline overlay+expansion but
    now share the primitive (their fail-open + last-value semantics and
    the deliberate no-defaults merge are preserved exactly).

Failure-semantics audit: every migrated site preserves its exact previous
behavior on missing file ({} / early return) and parse failure (raise into
the caller's existing except, warn, last-known-good, or fail-open) —
read_user_config_raw intentionally mirrors bare open()+safe_load semantics
(raises on parse errors, {} only on FileNotFoundError/non-dict root).

Guard: tests/hermes_cli/test_config_read_guard.py scans the tree for
yaml.safe_load within 6 lines of a 'config.yaml' reference outside an
explicit ALLOWLIST (hermes_cli/config.py, gateway/config.py, gateway/run.py
fallback path, hermes_cli/managed_scope.py which reads the MANAGED file,
gateway/readiness.py parse-health probe) and fails on new offenders.

E2E: tests/hermes_cli/test_config_loader_e2e.py runs a subprocess with a
temp HERMES_HOME (config.yaml containing ${E2E_PROMPT_SUFFIX}) plus a
HERMES_MANAGED_DIR overlay pinning agent.reasoning_effort, asserting
tui _load_cfg resolves "hello world"/"high" while _load_cfg_raw +
_save_cfg round-trip the template and user value verbatim with no
managed/default leakage.
2026-07-29 10:53:29 -07:00
teknium1
c92e2c0fbf test: age the disk L2 entry too in the server-swap redetection test
The TTL-expiry test aged only the in-process probe cache; with the new
disk L2 the fresh disk verdict (correctly) served 'ollama' and the swap
to lm-studio wasn't re-detected. In real time-flow the 300s disk TTL
always lapses before the 1h in-proc TTL — the test now compresses both
expiries, matching the scenario it describes. 25/25 pass.
2026-07-29 10:52:13 -07:00
teknium1
d7a4065568 perf(local-endpoints): disk L2 for server-type + ollama ctx probes, faster timeouts
Local-model users paid a fresh probe waterfall on EVERY CLI cold start
inside AIAgent.__init__: detect_local_server_type (up to 4 HTTP GETs,
2s timeout each on a hung server) + /api/show (3s timeout). The
existing caches were in-process only, so back-to-back invocations
(chat -q, cron ticks, subagents) re-paid the network every time.

- New 300s-TTL disk L2 at HERMES_HOME/cache/local_endpoint_probes.json
  for detect_local_server_type verdicts and query_ollama_num_ctx
  results. Only SUCCESSFUL probes persist (a down server never pins a
  negative verdict); stale entries pruned on write; corrupted cache
  degrades to a miss; atomic writes. 300s is strictly fresher than the
  1h in-process TTL that already accepts server-swap staleness.
- models.dev fetch timeout 15 -> (5, 10) connect/read tuple: a
  blackholed connect stalled the first-turn critical path 15s; now
  fails in 5s (matches the OpenRouter fetch convention, #46620).
- _auto_detect_local_model timeout 5 -> (2, 3): runs inside
  _get_model_config() at startup against a LOCAL endpoint; a hung local
  server cost 5s before the banner.

E2E (real HTTP server, two fresh subprocesses, isolated HERMES_HOME):
proc1 = 2 HTTP hits, proc2 = 0 HTTP hits, identical results
(ollama/131072), probe wall 74.5 -> 35.5 ms. 222 targeted tests green
incl. 9 new disk-L2 contract tests.
2026-07-29 10:52:13 -07:00
brooklyn!
cfa43f520a
Merge pull request #74245 from NousResearch/bb/pinned-messaging-index
Pinning a messaging session removes it from the sidebar
2026-07-29 12:41:16 -05:00
brooklyn!
f9f3811105
Merge pull request #74247 from NousResearch/bb/session-render-jank
fix(desktop): anchor the transcript through a session load
2026-07-29 12:39:50 -05:00
brooklyn!
c8f911112c
Merge pull request #74033 from NousResearch/jb/session-detail-profile-stamp
fix(sessions): always stamp owning profile so cross-profile open works toward default
2026-07-29 12:39:09 -05:00
teknium1
0c1a872af7 perf(install): run connectivity probes in parallel — blocked-network worst case 16s -> 8s
install.sh probed pypi.org and duckduckgo.com serially with
--max-time 8 each, so a fully blocked network cost 16s before the user
saw any useful guidance. The two probes are independent; running them
as background jobs and gathering verdicts caps the worst case at one
--max-time (8s) while the good path stays instant.

Verified live: reachable URLs 0.24s (both probes concurrent);
blackholed 10.255.255.x URLs 8.02s total (was 2x8s), warning text
unchanged. bash -n clean.
2026-07-29 10:33:43 -07:00
teknium1
fedd689d37 perf(config): one raw config.yaml parse per process instead of 3-4
Counted with an open()/read_text audit hook on a real 'hermes --version'
run: config.yaml was parsed 3x before load_config() even ran — once by
env_loader._load_secrets_config, once by main.py's early redact/ipv4
bridge (bespoke yaml.load), once by hermes_logging._read_logging_config.
Each raw parse is 1-3 ms with libyaml plus an open/stat — pure
duplication since all three want the same raw dict.

All three now route through read_raw_config()'s existing (mtime_ns,
size)-keyed shared cache:

- env_loader._load_secrets_config: uses the shared reader when reading
  the process HERMES_HOME (the cache key's home); other homes (profile
  seeding) keep the isolated direct parse. Parse-error isolation is
  preserved — the shared reader also swallows errors and returns {}.
- main.py early bridge: drops the bespoke yaml.load for read_raw_config
  (managed-scope overlay unchanged).
- hermes_logging._read_logging_config: prefers the shared reader,
  falls back to the direct fast_safe_load parse when hermes_cli.config
  isn't importable.

Measured: config.yaml opens per 'hermes --version' 3 -> 1.
348 targeted tests green (env_loader + secret sources + applied-homes +
bitwarden + hermes_logging + config).
2026-07-29 10:32:25 -07:00
brooklyn!
abd9edbea3
Merge pull request #74234 from NousResearch/bb/session-pins-server-owned
Pins are server-owned, so they survive paging and follow you between apps
2026-07-29 12:31:38 -05:00
teknium1
0034da2975 perf(stream): replace per-chunk repr() with delta-length byte estimate
The streaming hot loop computed len(repr(chunk)) on EVERY chunk to feed
the retry-diagnostic byte counter — a full recursive pydantic repr at
5.5-8.8 us per chunk (measured), ~20-30 ms of pure CPU per 3,000-chunk
response, paid on every streaming response on every platform.

New _estimate_chunk_bytes() sizes the chunk from its delta payload
strings (content / reasoning / tool-call arguments) plus a 40-byte
framing floor: 2.1-2.4 us per chunk (~3x cheaper), independent of
pydantic field count, never raises on unknown shapes (Anthropic events,
stub providers fall back to the floor). Both call sites switched
(chat-completions loop + anthropic event loop).

The counter feeds only the stream-retry diagnostic log line
(agent/stream_diag.py) — an estimate proportional to traffic preserves
its purpose (distinguishing 'died at 0 bytes' from 'died mid-stream').

6 new contract tests; 64 targeted stream tests green.
2026-07-29 10:21:59 -07:00
teknium1
3a69e34702 perf(update): cut redundant network + subprocess work from hermes update
Four fixes on the updater path:

1. uv self update freshness gate + timeout (managed_uv.py): the network
   self-update ran on EVERY hermes update — including the 'Already up to
   date!' fast path — with NO timeout (unbounded hang risk offline). Now
   skipped when it succeeded within 7 days (stamp file under
   HERMES_HOME/cache), capped at 60s, force= override available. The
   CVE-driven vulnerable-runtime repair probe is NEVER gated — it still
   runs on every invocation.

2. Drop the second network fetch from the pull step (main.py): the update
   flow fetched origin/<branch>, counted commits, then ran
   'git pull --ff-only origin <branch>' — a SECOND fetch of the same ref
   (~0.5-1.5s). Now merges the already-fetched tracking ref via
   'git merge --ff-only origin/<branch>'; the diverged-history reset
   fallback is unchanged.

3. Probe the upstream remote locally before fetching it (_cmd_update_check):
   non-fork installs have no 'upstream' remote, and --check burned a
   failed network attempt (~0.3-1s) on every run before falling back to
   origin. 'git remote get-url upstream' (~1ms local) now gates the fetch.

4. Desktop rebuild check reads the content-hash stamp in-process before
   spawning 'hermes desktop --build-only' (a full CLI re-import, ~1-3s)
   just to learn nothing changed. Stamp errors fall through to the
   subprocess path unchanged.

Savings on a no-op 'hermes update': ~2-6s (uv self-update 0.5-3s +
second fetch 0.5-1.5s + desktop spawn 1-3s when applicable).
187 targeted updater tests green incl. 5 new stamp-gate tests.
2026-07-29 10:16:44 -07:00
teknium1
9179fb72ea refactor: extract web_server Pydantic models to web_models.py (pure schema move) 2026-07-29 10:15:07 -07:00
teknium1
bf15259e33 refactor(gateway): shared media-cache mime dispatch for adapter downloads (per-adapter overrides preserve historical mappings) 2026-07-29 10:14:59 -07:00
teknium1
7b6a67f82e refactor(gateway): extract duplicated StreamConsumerConfig setup into _build_stream_consumer_config (preserves both sites' divergent fallback semantics via parameter) 2026-07-29 10:14:45 -07:00
teknium1
326764e255 refactor: table-driven config migration registry (17 if-blocks → (version, fn) table, byte-identical semantics) 2026-07-29 10:14:32 -07:00
teknium1
21c7ae8563 refactor: split SessionDB into Search/Schema/Portability mixins (mechanical move, ~2.9K LOC out of hermes_state.py) 2026-07-29 10:14:19 -07:00
teknium1
3d48f893da refactor: single build_subprocess_env() factory for all child-process spawns (profile + secret-scrub single owner) 2026-07-29 10:14:11 -07:00
teknium1
1a7f73b8ea refactor: migrate hand-rolled error envelopes to shared tool_error()
Replace json.dumps({"error": ...}) boilerplate with the documented
tools/registry.py tool_error() helper across 13 files.

Migrated: 59 sites (58 code sites + 1 docstring example in
path_security.py), incl. multi-key envelopes passed via kwargs
(available_actions, path/already_read, pattern/already_searched,
parameters/hint, needs_reauth/server, error_type/tool/result_type).
Also removed 2 now-redundant local tool_error imports in mcp_tool.py
in favor of a module-level import.

Skipped (not byte/shape-compatible with tool_error):
- {"success": false, "error": ...} envelopes (browser_tool,
  browser_camofox, browser_dialog_tool, web_tools, tts_tool,
  skills_tool, image_generation_tool, project_tools, memory_tool,
  cronjob_tools, x_search_tool, xai_video_tools) — leading keys
  differ; key order would change.
- terminal_tool/code_execution_tool envelopes carrying output/
  exit_code/status leading keys.
- tool_search.py:912-area multi-key success paths (non-error).
- mcp_tool.py MCPSampling._error — returns MCP-spec ErrorData
  object, not a JSON string; incompatible.
- send_message_tool._error — returns a dict (not str) and applies
  secret redaction; return type must be preserved.

Behavior note: sites that previously omitted ensure_ascii=False now
emit raw UTF-8 (tool_error's canonical behavior) — JSON-equivalent.

Tests: 23 targeted files (tool_search, discord, file_tools/read
guards/operations, registry, clarify, homeassistant, code_execution,
send_message, delegate, terminal, mcp, model_tools, sanitize_tool_error,
retaindb plugin) — all pass. ruff clean.
2026-07-29 10:14:00 -07:00
teknium1
f57cb2e482 refactor: use utils atomic writes in cron/skill_manager 2026-07-29 10:13:50 -07:00
teknium1
6f7f7cd064 refactor: use shared strip_ansi for inline ANSI regexes 2026-07-29 10:13:50 -07:00
teknium1
7c198c5e44 refactor: single shared Retry-After parser 2026-07-29 10:13:50 -07:00
Brooklyn Nicholson
a7cc924204 perf(desktop): add a session-load scenario to the harness
`submit` measures the scroll jump when a turn is appended; nothing measured
the jump when a session is opened, which is the prepend/settle path. Clicks
sidebar rows and tracks how far the bottom turn moves after first paint.
2026-07-29 12:13:36 -05:00
Brooklyn Nicholson
8c92983fde fix(desktop): anchor the transcript through a session load
Opening a session painted a small first budget, then prepended the rest of
the turns above the viewport with nothing holding the scroll position, so the
view was stranded near the top of the transcript until use-stick-to-bottom's
ResizeObserver caught up a frame or two later. The settle loop couldn't cover
for it either: keyed on sessionKey alone it measured an EMPTY viewport on a
cold load, saw a stable height, and declared the load settled hundreds of ms
before the messages arrived.

Anchor before the backfill prepends (the mechanism "Show earlier" already
uses) and re-arm the settle loop when the transcript actually lands.

Measured over 12 real sidebar-click loads: 12,179 -> 544 px of post-paint
movement per load, worst single jump 13,369 -> 2,296 px. Render churn is
unchanged (fewer commits and total renders, same wasted count).
2026-07-29 12:13:36 -05:00
Brooklyn Nicholson
46b55c2c91 refactor(desktop): extract the pin index and cover the messaging slice
Pebrd's fix is right but the `useMemo` it edits didn't list
`messagingSessions` as a dependency, so the index would go stale as the
messaging slice refreshed. Rather than regex around logic buried in a
1500-line component, lift it into `buildSessionByAnyId` where the
contract can be tested directly — the extraction LeonSGP43 proposed in
sibling PR #49896.

The test asserts the invariant that actually matters: a pin resolves
from every slice the sidebar fetches. Reverting the function to its
two-slice form fails it on the messaging and lineage-root cases.

Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
2026-07-29 12:12:55 -05:00
Pebrd
ab694586b2 fix: resolve pinned Telegram sessions into sidebar Pinned section
The sessionByAnyId map only indexed cronSessions and visibleSessions
(local CLI chats), so a pinned gateway session (Telegram, etc.) stored
its pinId in localStorage but could never be resolved back to a
SessionInfo object — the Pinned section rendered empty for those.

Include messagingSessions in the lookup so pinned gateway chats appear
correctly at the top of the sidebar.
2026-07-29 12:09:19 -05:00
teknium1
58b78f5a0a perf(banner): scope startup update-check fetch to origin/main
The banner update-check ran an unscoped 'git fetch origin', transferring
all ~1,400 remote heads (measured 3.0s dry-run vs 0.55s scoped, and up to
70s on a cold ref store) and frequently burning its full 10s timeout on
slow links. cmd_update already scopes its fetch for exactly this reason.

A scoped 'git fetch origin main' updates both the origin/main tracking
ref (full-clone count path) and FETCH_HEAD (shallow compare path), so
behind-count semantics are unchanged — verified empirically on a full
clone (rewound tracking ref restored to tip, count correct) and a
--depth 1 shallow clone (FETCH_HEAD updated, boundary preserved).
2026-07-29 10:09:14 -07:00
Teknium
5081551f09 fix(voice): full-duplex agent-turn listener — interrupt by voice during generation AND playback
Replaces the half-duplex per-playback barge monitors with ONE listener
that runs for the entire agent turn in continuous voice mode: armed at
utterance-submit, disarmed when the turn is fully done (response + TTS
finished). Fixes Teknium's live report that voice interruption never
works: (a) not while the LLM is generating, (b) not while TTS plays.

Root causes:
- HALF-DUPLEX GAP: the barge monitor only spawned when TTS playback
  STARTED (cli.py streaming/whole-file paths, gateway _tts_stream_begin).
  During LLM generation there was NO microphone listener at all.
- PLAYBACK DEAFNESS: the monitor calibrated its VAD noise floor WHILE the
  speaker was blasting TTS (speaker bleed baked into the floor), then
  multiplied it by 8x with a 1s strictly-consecutive block requirement —
  normal speech could rarely reach the trigger, and the 2s grace
  swallowed early interjections.

New model — tools/voice_mode.full_duplex_listen():
- Pre-playback calibration: quiet-room noise floor established at turn
  start and HELD through playback (never recalibrated against bleed).
- Phase-aware trigger: generation = floor x voice.barge_in_threshold_multiplier
  (new config, default 3.0, justified by synthetic-frame tests);
  playback = additionally clamped to a 1500-RMS minimum so bleed alone
  can't trip; 4000-RMS ceiling keeps speech always reachable.
- Windowed-majority detection (>=80% of a 300ms window) instead of the
  strictly-consecutive counter that reset on intra-word energy dips.
- Grace on playback ONSET only (voice.barge_in_grace_seconds, default
  down 2.0 -> 0.5) — suppresses the onset transient, not the mic.
- Debug diagnostics at every decision point (calibrated floor, per-window
  RMS above 50% of trigger, trip/no-trip, grace suppressions) — always
  logger.debug, mirrored to stderr under HERMES_VOICE_DEBUG=1.

Phase behavior (CLI cli.py + tui_gateway/server.py, same model):
- generation: speech interrupts the in-flight turn via the SAME seam the
  typed/Ctrl+C interrupt uses (agent.interrupt()), cuts any pending TTS
  pipeline so the stale reply never plays, and submits the captured
  interjection (pre-roll capture, first syllable kept) as the next turn.
- playback: cuts TTS (streaming pipeline stop + fallback speak stop
  events + file player) and submits the capture.
- stop phrase honored in BOTH phases: mid-generation 'stop' interrupts
  the turn AND ends the voice chat (stop everything).
- one listener instance spans generation -> playback (no re-arm race);
  double-arm refused (CLI _voice_fd_active / gateway _fd_listener_active).

Gateway specifics: _arm_full_duplex_listener() at _run_prompt_submit turn
start and inside _tts_stream_begin; _speak_text_with_barge registers its
(stop, done) pair in _fd_speak_pipelines so fallback speaks are cut and
tracked; _tts_stream_barge_in_monitor kept as a shim that arms the new
listener. Desktop renderer owns its own mic path (voice-barge-in.ts) and
is unaffected; if desktop backend-mic mode is used it inherits via the
gateway.

Tests: full_duplex_listen synthetic-RMS suite (speech-over-bleed trips,
bleed alone doesn't, quiet floor held through playback, grace window,
multiplier math 3x vs 8x, windowed-majority dips), CLI listener phase
tests (generation interrupt seam, playback cut, lifecycle spans phases,
double-arm, config forwarding, stop-phrase-mid-generation), gateway
generation-phase interrupt + stop-phrase tests. Generation-interrupt
test sabotage-verified.
2026-07-29 10:08:53 -07:00
ai-ag2026
8c196ed85c test: isolate computer-use approval globals between tests
tools/computer_use/tool.py keeps the CLI approval flow in module-globals:
_approval_callback plus the per-session unlock stores _always_allow /
_session_auto_approve. Any test that installs a callback (or drives CLI
init far enough that the real one is registered) and does not reset it
poisons every later computer-use test in the same process:

* a leaked callback that raises — dead UI infra or a stale two-argument
  signature (the contract is (action, args, summary)) — becomes
  verdict='deny' in _request_approval, so dispatch tests fail with an
  empty backend call list;
* a leaked callback that blocks (the real CLI one waits on an answer
  queue) hangs a single-process run forever.

Both are order-dependent: tests/tools/test_computer_use.py passes 220/220
in isolation but shows dispatch failures in single-process full-suite runs
(and, with a blocking leak, a permanent hang observed via py-spy inside
_request_approval -> callback -> queue.get with no timeout).

Fix: an autouse teardown-only fixture resets callback + unlock stores
after every test; tests that install their own callback keep it for their
own duration. Regression pair included: a 'forgetful' test leaves a stale
two-arg callback behind, the next test asserts dispatch still routes to
the backend — red without the fixture (1 failed), green with it (225
passed together with the whole computer-use file).
2026-07-29 10:06:27 -07:00
Jasmine Naderi
33fe1cc9e5 fix(test): patch threading.Event class for CUA timeout tests
_start_lifecycle_locked reassigns a fresh threading.Event() at line
786, so patching the pre-made instance's wait() is lost and the real
30s wait races pytest-timeout. Patch threading.Event itself with a
FakeEvent whose wait() returns False immediately (#69372).
2026-07-29 10:06:27 -07:00
teknium1
490056d6d3 perf: lazy mcp SDK import + tool-discovery mtime cache + browser_tool import diet
Three cold-start cuts, measured with .venv python, PYTHONPATH=worktree,
median of 3-5 fresh subprocesses:

1. tools/mcp_oauth.py — availability now via importlib.util.find_spec("mcp");
   SDK classes (OAuthClientProvider et al.) import lazily on first use via
   _ensure_sdk_loaded(). Module-level names kept as None placeholders so the
   test-patch surface (patch.object(mcp_oauth, "OAuthClientProvider", ...))
   still works. import tools.mcp_oauth: 242ms -> 56ms (mcp SDK no longer
   loaded at import time).

2. tools/registry.py — discover_builtin_tools AST scan memoized in an
   mtime_ns+size-keyed disk cache at ~/.hermes/cache/tool_discovery_cache.json
   (atomic write via utils.atomic_json_write, best-effort/never raises;
   corrupt or missing cache -> full rescan + rewrite; per-file stat mismatch
   -> rescan just that file). Scan of 100 files: 158ms cold -> 3ms warm.

3. tools/browser_tool.py — top-level `import requests` and
   agent.auxiliary_client.call_llm moved to lazy first-use (PEP 562
   __getattr__ preserves patch("tools.browser_tool.requests.get") and
   patch("tools.browser_tool.call_llm") surfaces; internal call sites go
   through _lazy_call_llm which reads module globals so patches are honored).

Entry-point imports (median ms, before -> after):
  import model_tools   392 -> 245   (warm discovery cache)
  import cli           152 -> 151   (unchanged; cli doesn't hit these paths)
  import gateway.run   234 -> 230

Functional verification:
- get_tool_definitions(quiet_mode=True) under temp HERMES_HOME: identical
  sorted 30-tool name set before vs after (empty diff).
- Discovery cache: delete cache -> 158ms, second run -> 3ms; corrupt cache
  -> clean full rescan; touching one file -> single-file rescan (12ms).
- Tests green: tests/tools/test_registry.py, all test_mcp_oauth*,
  test_mcp_dashboard_oauth, test_mcp_tool_401_handling, all
  tests/tools/test_browser*.py, tests/hermes_cli/test_mcp_{config,startup,
  dashboard_oauth}.py, test_skills_tool_discovery_cache.py.
2026-07-29 10:02:03 -07:00
Brooklyn Nicholson
8ce8b70dca feat(desktop): pins sync between apps sharing a gateway
The pin bridge only ever pushed: localStorage to the backend, never
back. Two apps on the same gateway each kept their own localStorage, so
a pin made on the Mac never appeared on the Windows app.

Now that a pinned row is guaranteed to be in the page, its absence says
nothing about its pin state — which makes the server row authoritative.
`pullRemotePins` adopts pins this app hasn't seen and drops local pins
the server says are gone, keying on the durable lineage root so a pin
survives compression tip rotation. Adopted pins are recorded as already
mirrored rather than echoed back as a redundant write.

A list request in flight when we PATCH still carries the old value, so
honouring it would silently undo the pin the user just made. Writes are
guarded for the lifetime of their own request — no timers, no wall-clock
windows. A runtime predating the flag sends no `pinned` at all; that's
treated as no opinion and leaves the local set alone.

Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Co-authored-by: Kyzcreig <9063726+Kyzcreig@users.noreply.github.com>
Co-authored-by: aman-merchant <274313970+aman-merchant@users.noreply.github.com>
Co-authored-by: rerdi92 <76791321+rerdi92@users.noreply.github.com>
2026-07-29 12:00:41 -05:00
Brooklyn Nicholson
3290807e60 feat(api): session lists carry the pin flag and never drop a pinned row
The three list endpoints (`/api/sessions`, `/api/profiles/sessions`, and
the batched sidebar route) now request the pinned back-fill and expose
`pinned` as a real JSON boolean alongside `archived`.

Both merge paths re-window rows after sorting, which would have thrown
away exactly what the back-fill fetched, so pinned rows survive the cap.
The sidebar's "load more" signal discounts them — they arrive past the
LIMIT by design and would otherwise fake a full page on a short list.

Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Co-authored-by: Kyzcreig <9063726+Kyzcreig@users.noreply.github.com>
Co-authored-by: konsisumer <11262660+konsisumer@users.noreply.github.com>
Co-authored-by: eason2026 <209090628+eason2026@users.noreply.github.com>
2026-07-29 12:00:31 -05:00
Brooklyn Nicholson
1b317d23fb fix(sessions): a pinned conversation can't be paged out of the list
`list_sessions_rich` returns one recency-ordered window, so a pinned
conversation that hadn't been touched in a while simply wasn't in the
payload. The desktop's Pinned section resolves pins against the loaded
rows, so the pin rendered as nothing until something dragged the row
back onto the page.

A pin is a "this must always be reachable" statement, which makes
falling off the page a bug rather than a paging outcome. `include_pinned`
adds one bounded query for the rows carrying `pinned = 1` that the
window missed, reusing the page's own WHERE clause — an archived or
filtered-out conversation stays out, and a pin is never a filter bypass.
It runs before compression projection, so a back-filled root surfaces
under its live tip exactly like a row that made the page on its own.

Co-authored-by: hrnbld <260600092+hrnbld@users.noreply.github.com>
Co-authored-by: liuhao1024 <11816344+liuhao1024@users.noreply.github.com>
Co-authored-by: Tamaz-sujashvili <56168197+Tamaz-sujashvili@users.noreply.github.com>
Co-authored-by: ferminquant <14808645+ferminquant@users.noreply.github.com>
2026-07-29 12:00:21 -05:00
Teknium
3334db67a4 docs: fold in remaining live fixes from overnight-sweep PR cluster
Salvaged from the same PR family (#50691, #59335, #67832 by @virtuadex):

- sessions.md: gateway routing index is the gateway_routing table in
  state.db; sessions.json is a legacy mirror behind
  gateway.write_sessions_json (from #59335)
- mcp.md: MCP server reads state.db first, sessions.json fallback
  (from #59335)
- slash-commands.md: /model flags --once/--session/--refresh/--provider
  + persist_switch_by_default semantics (from #67832); /reasoning
  messaging row gains level list + --global; /history timestamps note
  (from #50691)
- environment-variables.md: TERMINAL_DOCKER_ENV,
  TERMINAL_DOCKER_EXTRA_ARGS (from #50691); MEM0_MODE/HOST/USER_ID/
  AGENT_ID rewritten for the current tri-mode plugin
- cron-script-only.md: interpreter table row matches bash-from-PATH
2026-07-29 09:45:11 -07:00
virtuadex
782d0219a0 docs: sync overnight sweep with registry, gateway, curator, cron
Salvaged from #72422 by @virtuadex (conflicts resolved against current
main; superseded slash-command hunks dropped):

- SECURITY.md + SECURITY.es.md: gateway adapters live under
  plugins/platforms/<name>/, registry in gateway/platform_registry.py
- gateway-internals.md (EN + zh-Hans): key-files table rows for
  platform_registry.py and plugins/platforms/, deferred-loading section
- slash-commands.md: /reasoning full level list (max/ultra) + --global;
  CLI-only notes list gains /prompt, /pet, /hatch, /timestamps
- cron.md + cron-script-only.md: script runner accuracy — bash resolved
  from PATH with /bin/bash fallback, script paths confined to
  ~/.hermes/scripts/, provider credentials stripped via
  _sanitize_subprocess_env
- curator.md: cron-referenced skills protected from auto-archive,
  never-used grace floor
2026-07-29 09:45:11 -07:00
Teknium
c6f98b0cba docs: teach providers: dict as the canonical custom-provider format
Fixes #67278. Since config v12 (see #8776), custom_providers: list is
legacy — the migration converts it to the providers: dict and the
resolver reads the dict first (runtime_provider.py). Docs still taught
the legacy list as primary.

- integrations/providers.md: all 8 YAML examples converted to
  providers: dict (field mapping code-verified: api, default_model,
  transport), one consolidated legacy-format note
- configuring-models.md, configuration.md, credential-pools.md,
  migrate-from-openclaw.md, provider-runtime.md, faq.md: examples and
  prose flipped to dict-first; legacy list noted as still-read
- adding-providers.md untouched (sole mention is a literal test
  filename)
2026-07-29 09:44:59 -07:00
teknium1
1fe06115d1 refactor: extract DEFAULT_CONFIG + OPTIONAL_ENV_VARS to config_defaults.py (pure data move) 2026-07-29 09:34:18 -07:00
teknium1
19055492aa fix: route stray HERMES_HOME hardcodes through get_hermes_home() (profile + native-Windows safety) 2026-07-29 09:33:48 -07:00
Austin Pickett
b8ceba97ed fix(web): make PTY resume sanitizer work against real PTY output
Review follow-up on the salvaged #47772 work. Three defects made the
filter a no-op or actively harmful in production, plus both Copilot
review items.

1. The blank-line burst filter never fired. pty_bridge.py spawns via
   ptyprocess.PtyProcess.spawn() and never calls setraw(), so the PTY
   line discipline runs with ONLCR: every LF the child writes reaches
   xterm as CRLF. The /\n{50,}/ pattern requires consecutive LF, so a
   real 1000-row burst matched nothing (verified against a live PTY:
   b"A"+b"\n"*5+b"B" is read back as b"A\r\n\r\n\r\n\r\n\r\nB").
   Now matches /(?:\r?\n){50,}/.

2. Bursts split across WebSocket frames were not collapsed. bridge.read()
   does os.read(fd, 65536) per drain tick and each read is forwarded as
   its own frame, so a burst spans frames and each fragment fell under
   the 50 threshold (3000 rows survived in a 40-byte-read simulation).
   The sanitizer now holds back a trailing newline run — including a lone
   trailing CR, since a frame can split a CRLF pair — and resolves it on
   the next frame or on flush.

3. Erase-code stripping was permanent, not resume-scoped. resumeParam is
   the durable session identity and is never cleared after connect, so
   every spinner/progress/status redraw in a resumed session lost its
   ESC[K and left stale glyphs. Suppression is now bounded to
   PTY_RESUME_SANITIZE_WINDOW_MS (30s) after connect; burst collapsing
   still applies for the life of the socket.

Copilot review items:
- flush() no longer writes a buffered partial CSI into xterm. #pending
  only ever holds an incomplete sequence, and emitting one leaves the
  parser in an in-escape state that swallows output after reconnect. A
  buffered newline run is still emitted (collapsed).
- Test expectations updated accordingly.

Tests: 29 cases (was 18), now using CRLF fixtures that match real PTY
output, plus cross-frame burst reassembly, CRLF-pair frame splits, and
post-window erase preservation. Full web suite 135 passing.
2026-07-29 12:21:48 -04:00
灵越羽毛
88e67508bc fix(web): replace stateless PTY regex with tested PtyResumeSanitizer stream helper
Addresses sweeper review feedback:

- Stateful buffer guards against CSI sequences split across WebSocket frames
- Newline threshold raised to \n{50,} — only targets Ink's pathological bursts
- Streaming TextDecoder with {stream: true} handles split UTF-8 bytes
- onclose flush drains any buffered partial escape
- Extract into web/src/lib/pty-resume-sanitizer.ts for independent testing

Added 17 vitest cases covering:
- applyPtyFilters: pass-through, burst collapse, short newlines, erase-line,
  erase-char, SGR preservation, empty string, mixed content
- PtyResumeSanitizer: complete chunks, 2/3-frame CSI splits, bare \x1b,
  \x1b[digit prefix, empty-chunk buffer integrity, instance isolation,
  onclose flush
2026-07-29 12:21:48 -04:00
灵越羽毛
5e37c94906 fix(web): gate ANSI filtering behind resumeParam and reuse TextDecoder
Addresses Copilot review:
- Reuse a single TextDecoder instance instead of allocating per message
- Only filter erase codes during session resume (resumeParam != null)
- Extract filter chain into named helper sanitizeResumeOutput()

Refs #47313
2026-07-29 12:21:48 -04:00
灵越羽毛
4b6e1e440e fix(web): strip ANSI erase codes and blank-line bursts from PTY stream
Ink two-pass virtual scrolling during session resume floods the
PTY output with \x1b[K (erase-line), \x1b[NX (erase-char), and
thousand-line \n bursts. In the Dashboard INLINE mode, xterm
main scrollback buffer absorbs these as blank rows.

Filter all three in ws.onmessage before they reach xterm.

Refs #47313.
2026-07-29 12:21:48 -04:00
Teknium
aff380ddbe
Merge pull request #74176 from NousResearch/fix/cua-driver-refresh-pin-confirmed-version
fix: pin cua-driver refresh to the release check-update confirmed
2026-07-29 09:19:52 -07:00
Teknium
5fdc9d2a9d fix: pin cua-driver refresh to the release check-update confirmed
The upstream cua-driver installer scripts on trycua/cua@main carry a
baked default version that Release Please bumps in the release PR
*before* the release assets are published. During that window an
unpinned installer run 404s on the asset download and the
`hermes update` cua-driver refresh fails with:

    error: download failed: The remote server returned an error: (404) Not Found.
    ⚠ cua-driver refreshing did not complete. Re-run manually: ...

Observed live 2026-07-29: baked version 0.14.0 vs latest published
release 0.13.1 — every `hermes update` run with an out-of-date driver
hit the warning until upstream publishes the assets.

We already know the correct version: `cua-driver check-update --json`
returns `latest_version` straight from the GitHub Releases API, whose
entries by definition have published assets. When the check positively
confirms an update, export that version as CUA_DRIVER_RS_VERSION into
the installer child env (both install.sh and install.ps1 honour it over
their baked default), so the refresh downloads the release that
actually exists instead of racing the upstream release pipeline.

Malformed / missing latest_version values fall back to the previous
unpinned behaviour. The explicit `hermes computer-use install
--upgrade` force path and fresh installs are unchanged.
2026-07-29 09:02:19 -07:00
Teknium
7de33cc57e
Merge pull request #64536 from victor-kyriazakos/feat/gateway-health-diagnostics
feat(monitoring): gateway health & diagnostics OTLP export
2026-07-29 08:58:33 -07:00
Teknium
43874d1a96 docs: accuracy sweep + coverage for 2 months of shipped features
Accuracy pass (all 373 pages audited against code, 13 parallel audits):
- configuration.md: 12 stale defaults/keys (file-sync rewrite, clarify
  timeout, streaming knobs, iteration budget, TTS/STT enums)
- reference/: commands/env-vars/toolsets/tools synced with
  COMMAND_REGISTRY, argparse tree, OPTIONAL_ENV_VARS, TOOLSETS
  (28 env vars added, 3 phantom removed, mcp__ naming, webhook
  platform restricted toolset)
- features/, messaging/, developer-guide/, guides/: ~60 factual fixes
  (web_extract truncation, dashboard auth fail-closed, delegation
  blocked tools, adapter signatures, session schema v23, phantom
  Matrix env vars, hermes setup tts, auth spotify, webhook --skills)
- zh-Hans: explicit heading IDs fix 2 broken WSL2 anchors

New coverage for features shipped in the last 2 months (verified
against code before writing):
- compression.in_place, verify-on-stop (+v31/v32 migration reality),
  ${env:VAR} SecretRef, display.timestamp_format, session:compress
  hook + thread_id/chat_type fields
- /journey learning timeline, per-channel model/system-prompt
  overrides, /sessions search, clarify multi-select, -z --usage-file,
  uninstall --dry-run, config get/unset
- MCP elicitation, extra_headers, discover_models, api-server run cap,
  Bedrock cachePoint, Discord reasoning_style, Google Chat clarify
  cards, vibe reactions, resume cwd restore, Yuanbao forwarded
  messages, api_content sidecar, roaming pet, tool_progress log mode,
  WhatsApp polls/locations, kanban per-task model + lifecycle hooks
2026-07-29 08:48:05 -07:00
Teknium
068d7812a4 docs: document Hermes Relay and the desktop feature wave
- New page user-guide/messaging/relay.md: what Relay is, enrollment
  (hermes gateway enroll), capability handshake (media, native
  approval/clarify, threads, typing), config keys, troubleshooting —
  derived from gateway/relay/ + gateway_enroll.py (PRs #48147 #48242
  #71300 #71363 #71404 #71624 #69721).
- desktop.md: artifacts viewer, timeline rail, find-in-page,
  multi-window, git review/worktrees, multi-terminal + persistence,
  theme import, rebindable shortcuts, quick entry, context-usage
  popover, keep-awake, command palette, Hermes Cloud mode, memory
  graph — all verified against apps/desktop/ source.
2026-07-29 08:48:05 -07:00
Teknium
355b376225 docs(skills): regenerate skills catalog; fix generator sentence truncation
- Fix generate-skill-docs.py short-description truncation: split on
  sentence boundary (dot + space/end) instead of the first dot, which
  mangled descriptions containing dotted paths ('.hermes/plans/') or
  abbreviations.
- Regenerate all 181 skill pages + both catalog indexes + sidebar:
  adds missing pages for inspecting-hermes-desktop-dom, tldraw-offline
  (fixes broken catalog link), and pinecone-research.
- Delete orphaned kanban-codex-lane page (skill removed in #39028).
2026-07-29 08:48:05 -07:00
Teknium
c19fd5c505 fix(cli): wire hermes import-agent parser into main argparse tree
The import-agent subcommand (24c3c27ba8) shipped with its parser builder
and handler logic but build_import_agent_parser() was never called from
main.py, making the documented and unit-tested command uninvocable.
Register it alongside the other subcommand builders.
2026-07-29 08:48:05 -07:00
Teknium
707f316687 chore: map contributor email shag@agentmail.to -> sg-shag 2026-07-29 08:42:16 -07:00