288 commits

Author	SHA1	Message	Date
freqyfreqy	8de26e280e	docs(lsp): replace "git worktree" with "git repository" in LSP docs ... The word "worktree" (a git subcommand feature for parallel checkouts) was used interchangeably with "repository" in the LSP docs, causing confusion. LSP only requires a git-initialized directory, not an actual worktree. Fixes two instances: section "When LSP runs" and the troubleshooting "Editing a file outside any git repo" heading.	2026-05-13 23:05:20 -07:00
Teknium	d5775fe988	feat(codex-runtime): skip unavailable plugins during migration (#25437) ... Followup to PR #24182 — caught when scanning OpenClaw for recent codex fixes we hadn't considered. OpenClaw learned the hard way (#80815) that migrating plugins which codex itself reports as unavailable produces config that fails at activation time. Our /codex-runtime codex_app_server enable path queries codex's plugin/list and migrates everything where installed=true. We were trusting codex's installation state and ignoring its availability field. So a plugin that's installed=true but availability=UNAVAILABLE (broken local install) or REQUIRES_AUTH (OAuth expired or never completed) would get an [plugins."<n>@openai-curated"] entry in ~/.codex/config.toml — and the user's first codex turn after enabling the runtime would fail because codex refuses to activate it. Fix: filter on availability in _query_codex_plugins(). Only emit plugins where availability is empty (older codex versions without the field — preserve backward compat) or explicitly AVAILABLE. Tests: test_plugin_discovery_skips_unavailable_plugins — verifies 4 cases: - good-plugin (installed=True, availability=AVAILABLE) → migrated - broken-plugin (installed=True, availability=UNAVAILABLE) → skipped - auth-pending (installed=True, availability=REQUIRES_AUTH) → skipped - legacy-plugin (installed=True, no availability field) → migrated (older codex versions; preserve backward compat) Docs: Added bullet to 'What's NOT migrated' list in the docs page calling out the availability filter and why. Other OpenClaw codex PRs I reviewed but did NOT apply (with reasoning): - #81591 (load Codex for selectable models): we resolve runtime per-call already, no startup-time gating to fix - #81510 (cron compatibility): we documented cron as untested; their fix is for OpenClaw-specific cron orchestration shape - #81223 (rotate incompatible context-engine threads): we don't have a Lossless context engine equivalent - #80688 (constrain sandbox): we don't have an outer-sandbox concept - #80616 (release on turn_aborted): we already handle status= interrupted in turn/completed correctly - #80278 (expose activeModel in plugin SDK): not our surface - #80792 (default destructive_actions on): we don't expose that knob 56 codex-runtime migration tests still green (+1 new).	2026-05-13 22:20:27 -07:00
Teknium	091d8e1030	feat(codex-runtime): optional codex app-server runtime for OpenAI/Codex models (#24182) ... * feat(codex-runtime): scaffold optional codex app-server runtime Foundational commit for an opt-in alternate runtime that hands OpenAI/Codex turns to a 'codex app-server' subprocess instead of Hermes' tool dispatch. Default behavior is unchanged. Lands in three pieces: 1. agent/transports/codex_app_server.py — JSON-RPC 2.0 over stdio speaker for codex's app-server protocol (codex-rs/app-server). Spawn, init handshake, request/response, notification queue, server-initiated request queue (for approval round-trips), interrupt-friendly blocking reads. Tested against real codex 0.130.0 binary end-to-end during development. 2. hermes_cli/runtime_provider.py: - Adds 'codex_app_server' to _VALID_API_MODES. - Adds _maybe_apply_codex_app_server_runtime() helper, called at the end of _resolve_runtime_from_pool_entry(). Inert unless 'model.openai_runtime: codex_app_server' is set in config.yaml AND provider in {openai, openai-codex}. Other providers cannot be rerouted (anthropic, openrouter, etc. preserved). 3. tests/agent/transports/test_codex_app_server_runtime.py — 24 tests covering api_mode registration, the rewriter helper (default-off, case-insensitive, opt-in, non-eligible providers preserved), version parser, missing-binary handling, error class. Does NOT require codex CLI installed. This commit is wire-only: the api_mode is recognized but AIAgent does not yet branch on it. Followup commits add the session adapter, event projector, approval bridge, transcript projection (so memory/skill review still works), plugin migration, and slash command. Existing tests remain green: - tests/cli/test_cli_provider_resolution.py (29 passed) - tests/agent/test_credential_pool_routing.py (included above) * feat(codex-runtime): add codex item projector for memory/skill review The translator that lets Hermes' self-improvement loop keep working under the Codex runtime: converts codex 'item/*' notifications into Hermes' standard {role, content, tool_calls, tool_call_id} message shape that agent/curator.py already knows how to read. Item taxonomy (matches codex-rs/app-server-protocol/src/protocol/v2/item.rs): - userMessage → {role: user, content} - agentMessage → {role: assistant, content: text} - reasoning → stashed in next assistant's 'reasoning' field - commandExecution → assistant tool_call(name='exec_command') + tool result - fileChange → assistant tool_call(name='apply_patch') + tool result - mcpToolCall → assistant tool_call(name='mcp.<server>.<tool>') + tool result - dynamicToolCall → assistant tool_call(name=<tool>) + tool result - plan/hookPrompt/etc → opaque assistant note, no fabricated tool_calls Invariants preserved: - Message role alternation never violated: each tool item produces at most one assistant + one tool message in that order, correlated by call_id. - Streaming deltas (item/<type>/outputDelta, item/agentMessage/delta) don't materialize messages — only item/completed does. Mirrors how Hermes already only writes the assistant message after streaming ends. - Tool call ids are deterministic (codex item id-based) so replays produce identical messages and prefix caches stay valid (AGENTS.md pitfall #16). - JSON args use sorted_keys for the same reason. Real wire formats verified against codex 0.130.0 by capturing live notifications from thread/shellCommand and including one as a fixture (COMMAND_EXEC_COMPLETED). 23 new tests, all green: - Streaming deltas don't materialize (3 paths) - Turn/thread frame events are silent - commandExecution: 5 tests including non-zero exit annotation + deterministic id stability across replays - agentMessage + reasoning attachment + reasoning consumption - fileChange: summary without inlined content - mcpToolCall: namespaced naming + error surfacing - userMessage: text fragments only (drops images/etc) - opaque items: no fabricated tool_calls - Helpers: deterministic id stability + sorted JSON args - Role alternation invariant across all four tool-shaped item types This commit is a pure addition. AIAgent integration (the wire that uses the projector) is the next commit. * feat(codex-runtime): add session adapter + approval bridge The third self-contained module: CodexAppServerSession owns one Codex thread per Hermes session, drives turn/start, consumes streaming notifications via CodexEventProjector, handles server-initiated approval requests, and translates cancellation into turn/interrupt. The adapter has a single public per-turn method: result = session.run_turn(user_input='...', turn_timeout=600) # result.final_text → assistant text for the caller # result.projected_messages → list ready to splice into AIAgent.messages # result.tool_iterations → tick count for _iters_since_skill nudge # result.interrupted → True on Ctrl+C / deadline / interrupt # result.error → error string when the turn cannot complete # result.turn_id, thread_id → for sessions DB / resume Behavior: - ensure_started() spawns codex, does the initialize handshake, and issues thread/start with cwd + permissions profile. Idempotent. - run_turn() blocks until turn/completed, drains server-initiated requests (approvals) before reading notifications so codex never deadlocks waiting for us, projects every item/completed via the projector, and increments tool_iterations for the skill nudge gate. - request_interrupt() is thread-safe (threading.Event); the next loop iteration issues turn/interrupt and unwinds. - turn_timeout deadlock guard issues turn/interrupt and records an error if the turn never completes. - close() escalates terminate → kill via the underlying client. Approval bridge: Codex emits server-initiated requests for execCommandApproval and applyPatchApproval. The adapter translates Hermes' approval choice vocabulary onto codex's decision vocabulary: Hermes 'once' → codex 'approved' Hermes 'session' or 'always' → codex 'approvedForSession' Hermes 'deny' / anything else → codex 'denied' Routing precedence: 1. _ServerRequestRouting.auto_approve_* flags (cron / non-interactive) 2. approval_callback wired by the CLI (defers to tools.approval.prompt_dangerous_approval()) 3. Fail-closed denial when neither is wired Unknown server-request methods are answered with JSON-RPC error -32601 so codex doesn't hang waiting for us. Permission profile mapping mirrors AGENTS.md: Hermes 'auto' → codex 'workspace-write' Hermes 'approval-required' → codex 'read-only-with-approval' Hermes 'unrestricted/yolo' → codex 'full-access' 20 new tests, all green. Combined with prior commits this PR now has 67 tests across three modules: - test_codex_app_server_runtime.py: 24 (api_mode + transport surface) - test_codex_event_projector.py: 23 (item taxonomy projections) - test_codex_app_server_session.py: 20 (turn loop + approvals + interrupts) Full tests/agent/transports/ directory: 249/249 pass — no regressions to existing transport tests. Still no wire into AIAgent.run_conversation(); that integration commit is small and goes next. * feat(codex-runtime): wire codex_app_server runtime into AIAgent The integration commit. AIAgent.run_conversation() now early-returns to a new helper _run_codex_app_server_turn() when self.api_mode == 'codex_app_server', bypassing the chat_completions tool loop entirely. Three small surgical edits to run_agent.py (~105 LOC total): 1. Line ~1204 (constructor api_mode validation set): Add 'codex_app_server' so an explicit api_mode='codex_app_server' passed to AIAgent() isn't silently rewritten to 'chat_completions'. 2. Line ~12048 (run_conversation, just before the while loop): Early-return to _run_codex_app_server_turn() when self.api_mode is 'codex_app_server'. Placed AFTER all standard pre-loop setup — logging context, session DB, surrogate sanitization, _user_turn_count and _turns_since_memory increments, _ext_prefetch_cache, memory manager on_turn_start — so behavior outside the model-call loop is identical between paths. Default Hermes flow is unchanged when the flag is off. 3. End-of-class (line ~15497): New method _run_codex_app_server_turn(). Lazy-instantiates one CodexAppServerSession per AIAgent (reused across turns), runs the turn, splices projected_messages into messages, increments _iters_since_skill by tool_iterations (since the chat_completions loop normally does that per iteration), fires _spawn_background_review on the same cadence as the default path. Counter accounting: _turns_since_memory ← already incremented at run_conversation:11817 (gated on memory store configured) — codex helper does NOT touch it (would double-count). _user_turn_count ← already incremented at run_conversation:11793 — codex helper does NOT touch it. _iters_since_skill ← incremented in the chat_completions loop per tool iteration. Codex helper increments by turn.tool_iterations since the loop is bypassed. User message: ALREADY appended to messages by run_conversation pre-loop (line 11823) before the early-return reaches us. Helper does NOT append again. Regression test test_user_message_not_duplicated guards this. Approval callback wiring: Lazy-fetches tools.terminal_tool._get_approval_callback at session spawn time, passes to CodexAppServerSession. CLI threads with prompt_toolkit get interactive approvals; gateway/cron contexts get the codex-side fail-closed deny. Error path: Codex session exceptions become a 'partial' result with completed=False and a final_response that explicitly tells the user how to switch back: 'Codex app-server turn failed: ... Fall back to default runtime with /codex-runtime auto.' Same return-dict shape as the chat_completions path so all callers (gateway, CLI, batch_runner, ACP) work unchanged. 9 new integration tests in tests/run_agent/test_codex_app_server_integration.py: - api_mode='codex_app_server' is accepted on AIAgent construction - run_conversation returns the expected codex shape (final_response, codex_thread_id, codex_turn_id, completed, partial) - Projected messages are spliced into messages list - _iters_since_skill ticks per tool iteration - _user_turn_count delegated to standard flow (not double-counted) - User message appears exactly once (regression guard) - _spawn_background_review IS invoked (memory/skill review keeps working) - chat.completions.create is NEVER called (loop fully bypassed) - Session exception → partial result with /codex-runtime auto hint - Interrupted turn → partial result with error preserved Adjacent test runs confirm no regressions: - tests/run_agent/test_memory_nudge_counter_hydration.py: green - tests/run_agent/test_background_review.py: green - tests/run_agent/test_fallback_model.py: green - tests/agent/transports/: 249/249 green Still missing for full feature: /codex-runtime slash command, plugin migration helper, docs page, live e2e test gated on codex binary. Those are the remaining followup commits. * feat(codex-runtime): add /codex-runtime slash command (CLI + gateway) User-facing toggle for the optional codex app-server runtime. Follows the 'Adding a Slash Command (All Platforms)' pattern from AGENTS.md exactly: single CommandDef in the central registry → CLI handler → gateway handler → running-agent guard → all surfaces (autocomplete, /help, Telegram menu, Slack subcommands) update automatically. Surface: /codex-runtime — show current state + codex CLI status /codex-runtime auto — Hermes default runtime /codex-runtime codex_app_server — codex subprocess runtime /codex-runtime on / off — synonyms Files changed: hermes_cli/codex_runtime_switch.py (new): Pure-Python state machine shared by CLI and gateway. Parse args, read/write model.openai_runtime in the config dict, gate enabling behind a codex --version check (don't let users opt in to a runtime they have no binary for; print npm install hint instead). Returns a CodexRuntimeStatus dataclass that callers render however suits their surface. hermes_cli/commands.py: Single CommandDef entry, no aliases (codex-runtime is its own thing). cli.py: Dispatch in process_command() + _handle_codex_runtime() handler that delegates to the shared module and renders results via _cprint. gateway/run.py: Dispatch in _handle_message() + _handle_codex_runtime_command() that returns a string (gateway sends as message). On a successful change that requires a new session, _evict_cached_agent() forces the next inbound message to construct a fresh AIAgent with the new api_mode — avoids prompt-cache invalidation mid-session. gateway/run.py running-agent guard: /codex-runtime joins /model in the early-intercept block so a runtime flip mid-turn can't split a turn across two transports. Tests: tests/hermes_cli/test_codex_runtime_switch.py — 25 tests covering the state machine: arg parsing (10 cases incl. case-insensitive and synonyms), reading current runtime (5 cases incl. malformed configs), writing runtime (3 cases), apply() entry point covering read-only, no-op, codex-missing-blocked, codex-present-success, disable-no-binary-check, and persist-failure paths (8 cases). All green. Adjacent test suites confirm no regressions: - tests/hermes_cli/test_commands.py + test_codex_runtime_switch.py: 167/167 green - tests/agent/transports/: 283/283 green when combined with prior commits Still missing: plugin migration helper, docs page, live e2e test gated on codex binary. Followup commits. * feat(codex-runtime): auto-migrate Hermes MCP servers to ~/.codex/config.toml Translates the user's mcp_servers config from ~/.hermes/config.yaml into the TOML format codex's MCP client expects. Wired into the /codex-runtime codex_app_server enable path so users get their MCP tool surface in the spawned subprocess automatically. The migration runs on every enable. Failures are non-fatal — the runtime change still proceeds and the user gets a warning so they can fix the codex config manually. What translates (mapping verified against codex-rs/core/src/config/edit.rs): Hermes mcp_servers.<n>.command/args/env → codex stdio transport Hermes mcp_servers.<n>.url/headers → codex streamable_http transport Hermes mcp_servers.<n>.timeout → codex tool_timeout_sec Hermes mcp_servers.<n>.connect_timeout → codex startup_timeout_sec Hermes mcp_servers.<n>.cwd → codex stdio cwd Hermes mcp_servers.<n>.enabled: false → codex enabled = false What does NOT translate (warned + skipped per server): Hermes-specific keys (sampling, etc.) — codex's MCP client has no equivalent. Listed in the per-server skipped[] field of the report. What's NOT migrated (intentional): AGENTS.md — codex respects this file natively in its cwd. Hermes' own AGENTS.md (project-level) is already in the worktree, so codex picks it up without translation. No code needed. Idempotency design: All managed content lives between a 'managed by hermes-agent' marker and the next non-mcp_servers section header. _strip_existing_managed_block removes the prior managed region cleanly, preserving any user-added codex config (model, providers.openai, sandbox profiles, etc.) above or below. Files added: hermes_cli/codex_runtime_plugin_migration.py — pure-Python migration helper. Public API: migrate(hermes_config, codex_home=None, dry_run=False) returns MigrationReport with .migrated/.errors/ .skipped_keys_per_server. No external TOML dependency — minimal formatter handles strings/numbers/booleans/lists/inline-tables. tests/hermes_cli/test_codex_runtime_plugin_migration.py — 39 tests covering: - per-server translation (12): stdio/http/sse, cwd, timeouts, enabled flag, command+url precedence, sampling drop, unknown keys - TOML formatter (8): types, escaping, inline tables, error case - existing-block stripping (4): no marker, alone, with user content above, with user content below - end-to-end migrate() (8): empty, dry-run, round-trip, idempotent re-run, preserves user config, error reporting, invalid input, summary formatting Files changed: hermes_cli/codex_runtime_switch.py — apply() now calls migrate() in the codex_app_server enable branch. Migration failure logs a warning in the result message but does NOT fail the runtime change. Disable path (auto) explicitly skips migration. tests/hermes_cli/test_codex_runtime_switch.py — 3 new tests: test_enable_triggers_mcp_migration, test_disable_does_not_trigger_migration, test_migration_failure_does_not_block_enable. All 325 feature tests green: - tests/agent/transports/: 249 (incl. 67 new) - tests/run_agent/test_codex_app_server_integration.py: 9 - tests/hermes_cli/test_codex_runtime_switch.py: 28 (3 new) - tests/hermes_cli/test_codex_runtime_plugin_migration.py: 39 (new) * perf(codex-runtime): cache codex --version check within apply() Single /codex-runtime invocation could spawn 'codex --version' up to 3 times (state report, enable gate, success message). Each spawn is ~50ms, so the cumulative cost wasn't a crisis, but it was wasteful and turned a trivial slash command into something noticeably laggy on slower systems. Refactored to lazy-once via a closure over a nonlocal cache. First call spawns; subsequent calls in the same apply() reuse the result. Behavior unchanged — same return shape, same error handling, same install hint when codex is missing. Just one subprocess per call instead of three. Two regression-guard tests added: - test_binary_check_cached_within_apply: enable path → call_count == 1 - test_binary_check_cached_on_read_only_call: state-report path → call_count == 1 Total tests for /codex-runtime now 30 (was 28); all 143 codex-runtime tests still green. * fix(codex-runtime): correct protocol field names found via live e2e test Three real bugs caught only by running a turn end-to-end against codex 0.130.0 with a real ChatGPT subscription. Unit tests passed because they asserted on our own (incorrect) wire shapes; the wire format from codex-rs/app-server-protocol/src/protocol/v2/* is the source of truth and my initial reading of the README was incomplete. Bug 1: thread/start.permissions wire format Was sending {"profileId": "workspace-write"}. Real format per PermissionProfileSelectionParams enum (tagged union): {"type": "profile", "id": "workspace-write"} AND requires the experimentalApi capability declared during initialize. AND requires a matching [permissions] table in ~/.codex/config.toml or codex fails the request with 'default_permissions requires a [permissions] table'. Fix: stop overriding permissions on thread/start. Codex picks its default profile (read-only unless user configures otherwise), which matches what codex CLI users expect — they configure their default permission profile in ~/.codex/config.toml the standard way. Trying to be clever about profile selection broke every turn we tested. Live error before fix: 'Invalid request: missing field type' on every turn/start, even though our turn/start payload was correct — the field codex was complaining about was inside the permissions sub-object we shouldn't have been sending. Bug 2: server-request method names Was matching 'execCommandApproval' and 'applyPatchApproval'. Real names per common.rs ServerRequest enum: item/commandExecution/requestApproval item/fileChange/requestApproval item/permissions/requestApproval (new third method) Fix: match the documented names. Added handler for item/permissions/requestApproval that always declines — codex sometimes asks to escalate permissions mid-turn and silent acceptance would surprise users. Live symptom before fix: agent.log showed 'Unknown codex server request: item/commandExecution/requestApproval' and codex stalled because we replied with -32601 (unsupported method) instead of an approval decision. The agent reported back 'The write command was rejected' even though Hermes never showed the user an approval prompt. Bug 3: approval decision values Was sending decision strings 'approved'/'approvedForSession'/'denied'. Real values per CommandExecutionApprovalDecision enum (camelCase): accept, acceptForSession, decline, cancel (also AcceptWithExecpolicyAmendment and ApplyNetworkPolicyAmendment variants we don't currently use). Fix: rename _approval_choice_to_codex_decision return values; update auto_approve_* fallbacks; update fail-closed default from 'denied' to 'decline'. Test mapping table updated to match. Live test verified after fixes: $ hermes (with model.openai_runtime: codex_app_server) > Run the shell command: echo hermes-codex-livetest > .../proof.txt then read it back Approval prompt fired with 'Codex requests exec in <cwd>'. User chose 'Allow once'. Codex executed the command, wrote the file, read it back. Final response: 'Read back from proof.txt: hermes-codex-livetest'. File contents on disk match. agent.log confirms: codex app-server thread started: id=019e200e profile=workspace-write cwd=/tmp/hermes-codex-livetest/workspace All 20 session tests still green after wire-format updates. * fix(codex-runtime): correct apply_patch approval params + ship docs Live e2e revealed FileChangeRequestApprovalParams doesn't carry the changeset (just itemId, threadId, turnId, reason, grantRoot) — Codex's 'reason' field describes what the patch wants to do. Test config and display logic updated to use it. The first 'apply_patch (0 change(s))' display from the live test is now 'apply_patch: <reason>'. Adds website/docs/user-guide/features/codex-app-server-runtime.md covering enable/disable, prerequisites, approval UX, MCP migration behavior, permission profile delegation to ~/.codex/config.toml, known limitations, and the architecture diagram. Wired into the Automation category in sidebars.ts. Live e2e validation across the path matrix: ✓ thread/start handshake ✓ turn/start with text input ✓ commandExecution items + projection ✓ item/commandExecution/requestApproval → Hermes UI → response ✓ Approve once → command runs ✓ Deny → command rejected, codex falls back to read-only message ✓ Multi-turn (codex remembers prior turn's results) ✓ apply_patch via Codex's fileChange path ✓ item/fileChange/requestApproval → Hermes UI ✓ MCP server migration loads inside spawned codex (verified via 'use the filesystem MCP tool' prompt) ✓ /codex-runtime auto → codex_app_server toggle cycle ✓ Disable doesn't trigger migration ✓ Enable with codex CLI present succeeds + migrates ✓ Hermes-side interrupt path (turn/interrupt request issued cleanly even if codex finishes before the interrupt lands) Known live-validated limitations now documented in the docs page: - delegate_task subagents unavailable on this runtime - permission profile selection delegated to ~/.codex/config.toml - apply_patch approval prompt has no inline changeset (codex protocol doesn't expose it) 145/145 codex-runtime tests still green. * feat(codex-runtime): native plugin migration + UX polish (quirks 2/4/5/10/11) Major: migrate native Codex plugins (#7 in OpenClaw's PR list) Discovers installed curated plugins via codex's plugin/list RPC and writes [plugins."<name>@<marketplace>"] entries to ~/.codex/config.toml so they're enabled in the spawned Codex sessions. This is the 'YouTube-video-worthy' bit Pash highlighted: when a user has google-calendar, github, etc. installed in their Codex CLI, those plugins activate automatically when they enable Hermes' codex runtime. Implementation: - hermes_cli/codex_runtime_plugin_migration.py: new _query_codex_plugins() helper spawns 'codex app-server' briefly and walks plugin/list. Returns (plugins, error) — failures are non-fatal so MCP migration still works. - render_codex_toml_section() now takes plugins + permissions args. - migrate() defaults: discover_plugins=True, default_permission_profile= 'workspace-write'. Explicit None on either disables that side. - _strip_existing_managed_block() now also strips [plugins.*] and [permissions]/[permissions.*] sections inside the managed block, so re-runs replace plugins cleanly without touching codex's own config. Quirk fixes: #2 Default permissions profile written on enable. Without this, Codex's read-only default kicks in and EVERY write triggers an approval prompt. Now writes [permissions] default = 'workspace-write' so the runtime feels normal out of the box. Set default_permission_profile=None to opt out. #4 apply_patch approval prompt now shows what's changing. Codex's FileChangeRequestApprovalParams doesn't carry the changeset. Session adapter now caches the fileChange item from item/started notifications and looks it up by itemId when codex requests approval. Prompt shows '1 add, 1 update: /tmp/new.py, /tmp/old.py' instead of 'apply_patch (0 change(s))'. Side benefit: also drains pending notifications BEFORE handling a server request, so the projector and per-turn caches are up to date when the approval decision fires. Bounded to 8 notifications per loop iter to avoid starving codex's response. #5/#10 Exec approval prompt never shows empty cwd. When codex omits cwd in CommandExecutionRequestApprovalParams, fall back to the session's cwd. If somehow neither is available, show '<unknown>' explicitly instead of an empty string. Also surfaces 'reason' from the approval params when codex provides it — gives users more context on why codex wants to run something. #11 Banner indicates the codex_app_server runtime when active. New 'Runtime: codex app-server (terminal/file ops/MCP run inside codex)' line appears in the welcome banner only when the runtime is on. Default banner is unchanged. Tests: - 7 new tests in test_codex_runtime_plugin_migration.py covering plugin discovery (mocked), failure handling, dry-run skip, opt-out flag, idempotent re-runs, and permissions writing. - 3 new tests in test_codex_app_server_session.py covering the enriched approval prompts: cwd fallback, change summary on apply_patch, fallback when no item/started cache exists. - All 26 session tests + 46 migration tests green; 153 total in PR. * feat(codex-runtime): hermes-tools MCP callback + native plugin migration The big architectural addition: when codex_app_server runtime is on, Hermes registers its own tool surface as an MCP server in ~/.codex/config.toml so the codex subprocess can call back into Hermes for tools codex doesn't ship with — web_search, browser_*, vision, image_generate, skills, TTS. Also: 'migrate native codex plugins' (Pash's YouTube-video-worthy bit) — when the user has plugins like Linear, GitHub, Gmail, Calendar, Canva installed via 'codex plugin', Hermes discovers them via plugin/list and writes [plugins.<name>@openai-curated] entries so they activate automatically. New module: agent/transports/hermes_tools_mcp_server.py FastMCP stdio server exposing 17 Hermes tools. Each call dispatches through model_tools.handle_function_call() — same code path as the Hermes default runtime. Run with: python -m agent.transports.hermes_tools_mcp_server [--verbose] Exposed: web_search, web_extract, browser_navigate / _click / _type / _press / _snapshot / _scroll / _back / _get_images / _console / _vision, vision_analyze, image_generate, skill_view, skills_list, text_to_speech. NOT exposed (deliberately): - terminal/shell/read_file/write_file/patch — codex has built-ins - delegate_task/memory/session_search/todo — _AGENT_LOOP_TOOLS in model_tools.py:493, require running AIAgent context. Documented as a limitation and surfaced in the slash command output. Migration changes (hermes_cli/codex_runtime_plugin_migration.py): - _query_codex_plugins() spawns 'codex app-server' briefly to walk plugin/list and pull installed openai-curated plugins. Failures are non-fatal — MCP migration still completes. - render_codex_toml_section() now takes plugins + permissions args AND wraps the managed block with a MIGRATION_END_MARKER comment so the stripper can reliably find both ends, even when the block contains top-level keys (default_permissions = ...). - migrate() defaults: discover_plugins=True, expose_hermes_tools=True, default_permission_profile=':workspace' (built-in codex profile name — must be prefixed with ':'). All three opt-out via explicit args. - _build_hermes_tools_mcp_entry() builds the codex stdio entry with HERMES_HOME and PYTHONPATH passthrough so a worktree-launched Hermes points the MCP subprocess at the same module layout. Live-caught wire bugs fixed during this turn: 1. Permission profile config key is top-level , NOT a [permissions] table. The [permissions] table is for *user-defined* profiles with structured fields. Built-in profile names start with ':' (':workspace', ':read-only', ':danger-no-sandbox'). Was emitting which codex rejected with 'invalid type: string "X", expected struct PermissionProfileToml'. 2. Built-in profile is , NOT . Codex rejected with 'unknown built-in profile'. 3. Codex's MCP layer sends for tool-call confirmation. We weren't handling it, so codex stalled and returned 'MCP tool call was rejected'. Now: auto-accept for our own hermes-tools server (user already opted in by enabling the runtime), decline for third-party servers. Quirk fixes shipped (from the limitations list): #2 default permissions: workspace profile written on enable. No more approval prompt on every write. #4 apply_patch approval shows what's changing: cache fileChange items from item/started, look up by itemId when codex sends item/fileChange/requestApproval. Prompt: '1 add, 1 update: /tmp/new.py, /tmp/old.py' instead of '0 change(s)'. #5/#10 exec approval cwd never empty: fall back to session cwd, then '<unknown>'. Also surfaces 'reason' from codex when present. #11 banner shows 'Runtime: codex app-server' line when active so users understand why tool counts may not match what's reachable. Tests: - 5 new tests in test_codex_runtime_plugin_migration.py covering plugin discovery, expose_hermes_tools entry generation, idempotent re-runs, opt-out flag, permissions profile. - 3 new tests in test_codex_app_server_session.py covering enriched approval prompts (cwd fallback, fileChange summary). - 2 new tests for mcpServer/elicitation/request handling (accept hermes-tools, decline others). - New test file test_hermes_tools_mcp_server.py covering module surface, EXPOSED_TOOLS safety invariants (no shell/file_ops, no agent-loop tools), and main() error paths. - 166 codex-runtime tests total, all green. Live e2e validated against codex 0.130.0 + ChatGPT subscription: ✓ /codex-runtime codex_app_server enables, migrates filesystem MCP, registers hermes-tools, writes default_permissions = ':workspace' ✓ Banner shows 'Runtime: codex app-server' line in subsequent sessions ✓ Shell command runs without approval prompt (workspace profile works) ✓ Multi-turn — codex remembers prior turn's results ✓ apply_patch path via fileChange request approval ✓ web_search via hermes-tools MCP callback returns real Firecrawl results: 'OpenAI Codex CLI – Getting Started' end-to-end in 13s ✓ Disable cycle clean Docs updated: website/docs/user-guide/features/codex-app-server-runtime.md Full re-write covering native plugin migration, the hermes-tools callback architecture, the prerequisites change ('codex login is separate from hermes auth login codex'), the trade-off table now reflecting which Hermes tools work via callback, and the limitations list updated with what's actually unavailable on this runtime. * feat(codex-runtime): pin user-config preservation invariant for quirk #6 Quirk #6 from the limitations list — user MCP servers / overrides / codex-only sections in ~/.codex/config.toml that live OUTSIDE the hermes-managed block must survive re-migration verbatim. This already worked thanks to the MIGRATION_MARKER + MIGRATION_END_MARKER pair I added when fixing the default_permissions wire format (so the strip can find both ends of the managed region even with top-level keys like default_permissions). But it was an emergent property without a test pinning it. Now explicitly tested: - User MCP server above the managed block survives migration - User MCP server below the managed block survives migration - Both above + below survive a second re-migration - User content (model, providers, sandbox, otel, etc.) outside our region is left untouched Docs added a section "Editing ~/.codex/config.toml safely" explaining the marker contract — so users know they can add their own MCP servers, override permissions, configure codex-only options, etc. without fear of Hermes overwriting their work. 167 codex-runtime tests, all green. * docs(codex-runtime): clarify the actual tool surface — shell covers terminal/read/write/find Previous docs and PR description undersold what codex's built-in toolset actually provides. apply_patch alone made it sound like the runtime could only edit files in patch format — implying you'd lose terminal use, read_file, write_file, search/find. That was wrong. Codex's 'shell' tool runs arbitrary shell commands inside the sandbox, which covers everything you'd do in bash: cat/head/tail (read), echo> or heredocs (write), find/rg/grep (search), ls/cd (navigate), build/ test/git/etc. apply_patch is for structured multi-file edits on top of that. update_plan is its in-runtime todo. view_image loads images. And codex has its own web_search built in (in addition to the Firecrawl-backed one Hermes exposes via MCP callback). Docs now have a 'What tools the model actually has' section right after Why, breaking the surface into three clearly-labeled buckets: 1. Codex's built-in toolset (always on) — shell, apply_patch, update_plan, view_image, web_search; covers everything terminal- adjacent. 2. Native Codex plugins (auto-migrated from your codex plugin install) — Linear, GitHub, Gmail, Calendar, Outlook, Canva, etc. 3. Hermes tool callback (MCP server in ~/.codex/config.toml) — web_search/web_extract via Firecrawl, browser_*, vision_analyze, image_generate, skill_view/skills_list, text_to_speech. Plus a 'What's NOT available' callout listing the four agent-loop tools (delegate_task, memory, session_search, todo) that need running AIAgent context and can't reach the codex runtime. Trade-offs table broken out: shell, apply_patch, update_plan, view_image, sandbox each get their own row with a one-line description so users can see at a glance what's available natively. Architecture diagram updated to list the codex built-ins by name instead of 'apply_patch + shell + sandbox'. No code changes — purely docs clarification. 167 codex-runtime tests still green. * fix(codex-runtime): _spawn_background_review signature + review fork api_mode downgrade Two real bugs in the self-improvement loop integration that the previous test mocked away. Bug 1: wrong call signature The codex helper was calling self._spawn_background_review() with no args after every turn. That function actually requires: messages_snapshot=list (positional or keyword) review_memory=bool (at least one trigger must be True) review_skills=bool So the call would have raised TypeError at runtime — except the only test that exercised this path mocked _spawn_background_review entirely and just asserted spawn.called, so the wrong-arg shape never surfaced. Bug 2: review fork inherits codex_app_server api_mode The review fork is constructed with: api_mode = _parent_runtime.get('api_mode') So when the parent is codex_app_server, the review fork ALSO runs as codex_app_server. But the review fork's whole job is to call agent-loop tools (memory, skill_manage) which require Hermes' own dispatch — they short-circuit with 'must be handled by the agent loop' on the codex runtime. So the review fork would have run, decided to save something, called memory or skill_manage, and silently no-op'd. Fixed in run_agent.py:_spawn_background_review() — when the parent api_mode is 'codex_app_server', the review fork is downgraded to 'codex_responses' (same OAuth credentials, same openai-codex provider, but talks to OpenAI's Responses API directly so Hermes owns the loop). Also rewrote the codex helper's review wiring to match the chat_completions path: - Computes _should_review_memory in the pre-loop block (was already being computed; now passed through to the helper as an arg). - Computes _should_review_skills AFTER the codex turn returns + counters tick (line ~15432 pattern in chat_completions). - Calls _spawn_background_review(messages_snapshot=, review_memory=, review_skills=) only when at least one trigger fires. - Adds the external memory provider sync (_sync_external_memory_for_turn) that the chat_completions path runs after every turn. Tests: Replaced the broken test_background_review_invoked (which only asserted spawn.called) with three sharper tests: - test_background_review_NOT_invoked_below_threshold: single turn at default thresholds → no review fires (would have caught the original 'every turn calls spawn with no args' bug) - test_background_review_skill_trigger_fires_above_threshold: 10 tool_iterations at threshold=10 → review fires with messages_snapshot=list, review_skills=True, counter resets - test_background_review_signature_never_breaks: regression guard asserting positional args are always empty and kwargs include messages_snapshot New TestReviewForkApiModeDowngrade class: - test_codex_app_server_parent_downgrades_review_fork: drives the real _spawn_background_review function (no mock at that level), asserts the review_agent gets api_mode='codex_responses' when the parent was codex_app_server. Live-validated against real run_conversation: - Counter ticked from 0 to 5 after a 5-tool-iteration turn - _spawn_background_review fired exactly once with kwargs-only signature - review_skills=True, review_memory=False - messages_snapshot was 12 entries (5 assistant tool_calls + 5 tool results + 1 final assistant + initial system/user) - Counter reset to 0 after fire 170 codex-runtime tests, all green. Docs: added a Self-improvement loop section to the codex runtime page explaining both how the trigger logic stays equivalent and that the review fork is auto-downgraded to codex_responses for the agent-loop tools. Also clarified that apply_patch and update_plan ARE codex's built-in tools (the previous version made it sound like they were separate from 'codex's stuff' — they're not, all five tools listed in 'What tools the model actually has' section 1 are codex built-ins). * feat(codex-runtime): expose kanban tools through Hermes MCP callback Kanban workers spawn as separate hermes chat -q subprocesses that read the user's config.yaml. If model.openai_runtime: codex_app_server is set globally (which is the whole point of opt-in), every dispatched worker ALSO comes up on the codex runtime. That mostly works — codex's built-in shell + apply_patch + update_plan do the actual task work fine — but it had one critical break: the worker handoff tools (kanban_complete, kanban_block, kanban_comment, kanban_heartbeat) are Hermes-registered tools, not codex built-ins. On the codex runtime, codex builds its own tool list and these never reach the model, so the worker would do the work but not be able to report back, hanging until the dispatcher's timeout escalates it as zombie. Fix: add all 9 kanban tools to the EXPOSED_TOOLS list in the Hermes MCP callback. They dispatch statelessly through handle_function_call() just like web_search and the others — they read HERMES_KANBAN_TASK from env (set by the dispatcher), gate correctly (worker tools require the env var, orchestrator tools require it unset), and write to ~/.hermes/kanban.db. Why kanban tools work via stateless dispatch when delegate_task/memory/ session_search/todo don't: those four are listed in _AGENT_LOOP_TOOLS (model_tools.py:493) and short-circuit in handle_function_call() with 'must be handled by the agent loop' — they need to mutate AIAgent's mid-loop state. Kanban tools have no such requirement; they're pure side-effect functions against the kanban.db plus state_meta. Tools exposed: Worker handoff (require HERMES_KANBAN_TASK): kanban_complete, kanban_block, kanban_comment, kanban_heartbeat Read-only board queries: kanban_show, kanban_list Orchestrator (require HERMES_KANBAN_TASK unset): kanban_create, kanban_unblock, kanban_link Tests: - test_kanban_worker_tools_exposed: complete/block/comment/heartbeat in EXPOSED_TOOLS (regression guard for the would-hang-worker bug) - test_kanban_orchestrator_tools_exposed: create/show/list/unblock/link Docs: - New 'Workflow features' section in the docs page covering /goal, kanban, and cron behavior on this runtime - /goal: works fully via run_conversation feedback; only caveat is approval-prompt noise on long writes-heavy goals (mitigated by the default :workspace permission profile) - Kanban: enumerated which tools are reachable via the callback and why the env var propagates correctly through the codex subprocess to the MCP server subprocess - Cron: documented as 'not specifically tested' — same rules as the CLI apply since cron runs through AIAgent.run_conversation - Trade-offs table gained rows for /goal, kanban worker, kanban orchestrator 172/172 codex-runtime tests green (+2 from kanban tests). * docs(codex-runtime): wire /codex-runtime into slash-commands ref + flag aux token cost Three docs gaps caught during a final audit: 1. /codex-runtime was only in the feature docs page, not in the slash-commands reference. Added rows to both the CLI section and the Messaging section so users discover it where they'd look for slash command syntax. 2. CODEX_HOME and HERMES_KANBAN_TASK weren't in environment-variables.md. CODEX_HOME lets users redirect Codex CLI's config dir (the migration honors it). HERMES_KANBAN_TASK is set by the kanban dispatcher and propagates to the codex subprocess + the hermes-tools MCP subprocess so kanban worker tools gate correctly — documented as 'don't set manually' since it's an internal handoff. 3. Aux client behavior on this runtime. When openai_runtime= codex_app_server is on with the openai-codex provider, every aux task (title generation, context compression, vision auto-detect, session search summarization, the background self-improvement review fork) flows through the user's ChatGPT subscription by default. This is true for the existing codex_responses path too, but it's more visible / important here because users explicitly opted in for subscription billing. Added a 'Auxiliary tasks and ChatGPT subscription token cost' section to the docs page with a YAML example showing how to override specific aux tasks to a cheaper model (typically google/gemini-3-flash-preview via OpenRouter). Also documents how the self-improvement review fork gets auto-downgraded from codex_app_server to codex_responses by the fix earlier in this PR. No code changes — pure docs. 172 codex-runtime tests still green. * docs+test(codex-runtime): pin HOME passthrough, document multi-profile + CODEX_HOME OpenClaw hit a real footgun in openclaw/openclaw#81562: when spawning codex app-server they were synthesizing a per-agent HOME alongside CODEX_HOME. That made every subprocess codex's shell tool launches (gh, git, aws, npm, gcloud, ...) see a fake $HOME and miss the user's real config files. They had to back it out in PR #81562 — keep CODEX_HOME isolation, leave HOME alone. Audit confirms Hermes' codex spawn doesn't have this problem. We do os.environ.copy() and only overlay CODEX_HOME (when provided) and RUST_LOG. HOME passes through unchanged. But it was an emergent property without a test pinning it, so adding a regression guard: test_spawn_env_preserves_HOME — confirms parent HOME survives intact in the subprocess env test_spawn_env_sets_CODEX_HOME_when_provided — confirms codex_home arg still isolates codex state correctly Docs additions: 'HOME environment variable passthrough' section — calls out the contract explicitly: CODEX_HOME isolates codex's own state, HOME stays user-real so gh/git/aws/npm/etc. find their normal config. Cites openclaw#81562 as the cautionary tale. 'Multi-profile / multi-tenant setups' section — addresses the related concern: profiles share ~/.codex/ by default. For users who want per-profile codex isolation (separate auth, separate plugins), documents the manual CODEX_HOME=<profile-scoped-dir> approach. Explains why we DON'T auto-scope CODEX_HOME per profile: doing so would silently invalidate existing codex login state for anyone upgrading to this PR with tokens already at ~/.codex/auth.json. Opt-in is safer than surprising users. 174 codex-runtime tests (+2 from HOME guards), all green. * fix(codex-runtime): TOML control-char escapes + atomic config.toml write Two footguns caught in a final audit pass before merge. Bug 1: TOML control characters not escaped The _format_toml_value() helper escaped backslashes and double quotes but passed literal control characters (\n, \t, \r, \f, \b) through unchanged. TOML basic strings don't allow literal control characters — a path or env var containing a newline would produce invalid TOML that codex refuses to load. Realistic exposure: pathological cases like a HERMES_HOME with a trailing newline (env var concatenation accident), or a PYTHONPATH with a tab from a multi-line shell heredoc. Fix: escape all five TOML basic-string control sequences (\b \t \n \f \r) in addition to \\ and \" that we already did. Order matters — backslash must come first or the other escapes get re-escaped. Bug 2: config.toml write wasn't atomic If the python process crashed between target.mkdir() and the write_text() finishing, a half-written config.toml could be left behind. On NFS / Windows / some FUSE mounts this is a real concern; on ext4/APFS small writes are usually atomic in practice but not guaranteed. Fix: write to a tempfile.mkstemp() temp file in the same directory, then Path.replace() (atomic same-dir rename on POSIX, ReplaceFile on Windows). On rename failure, clean up the temp file so repeated failed migrations don't pile up .config.toml.* files. Tests: - test_string_with_newline_escaped — \n in value → \n in output - test_string_with_tab_escaped — \t in value → \t in output - test_string_with_other_controls_escaped — \r, \f, \b - test_windows_path_escaped_correctly — backslash doubling - test_atomic_write_no_temp_leak_on_success — no .config.toml.* left over after a successful write - test_atomic_write_cleanup_on_rename_failure — temp file removed when Path.replace raises (simulated disk full) 180 codex-runtime tests, all green (+6 from this commit). Footguns audited but NOT fixed (with rationale): - Concurrent migrations race. Two Hermes processes hitting /codex-runtime codex_app_server within seconds of each other could cause one writer to lose entries. Low probability (you'd have to enable from two surfaces simultaneously) and low impact (just re-run migration). Adding fcntl/msvcrt locking is more code than it's worth here. The atomic rename above means each individual write is consistent — only the merge step is racy. - Codex protocol version drift. We pin MIN_CODEX_VERSION=0.125 and check at runtime but don't reject too-new versions. Right call — the protocol has been stable through 0.125 → 0.130. If OpenAI breaks it later we'd see the error in test_codex_app_server_runtime on CI before users hit it.	2026-05-13 17:18:15 -07:00
Teknium	9d42c2c286	feat(video_gen): unified video_generate tool with pluggable provider backends (#25126) ... * feat(video_gen): unified video_generate tool with pluggable provider backends One core video_generate tool, every backend a plugin. Mirrors the image_gen + memory_provider + context_engine architecture: ABC, registry, plugin-context registration hook, and per-plugin model catalogs surfaced through hermes tools. Surface (one schema, every backend): - operation: generate / edit / extend - modalities: text-to-video (prompt only), image-to-video (prompt + image_url), video edit (prompt + video_url), video extend (video_url) - reference_image_urls, duration, aspect_ratio, resolution, negative_prompt, audio, seed, model override - Providers ignore unknown kwargs and declare what they support via VideoGenProvider.capabilities() — backend-specific quirks stay in the backend, the agent learns one tool Backends shipped: - plugins/video_gen/xai/ — Grok-Imagine, full generate/edit/extend + image-to-video + reference images (salvaged from PR #10600 by @Jaaneek, reshaped into the plugin interface) - plugins/video_gen/fal/ — Veo 3.1 (t2v + i2v), Kling O3 i2v, Pixverse v6 i2v with model-aware payload building that drops keys a model doesn't declare Wiring: - agent/video_gen_provider.py — VideoGenProvider ABC, normalize_operation, success_response / error_response, save_b64_video / save_bytes_video, $HERMES_HOME/cache/videos/ - agent/video_gen_registry.py — thread-safe register/get/list + get_active_provider() reading video_gen.provider from config.yaml - hermes_cli/plugins.py — PluginContext.register_video_gen_provider() - hermes_cli/tools_config.py — Video Generation category in hermes tools, plugin-only providers list, model picker per plugin, config write to video_gen.{provider,model} - toolsets.py — new video_gen toolset - tests: 31 new tests covering ABC, registry, tool dispatch, both plugins - docs: developer-guide/video-gen-provider-plugin.md (parallel to the image-gen guide), sidebar + toolsets-reference + plugin guides updated Supersedes: #25035 (FAL), #17972 (FAL), #14543 (xAI), #13847 (HappyHorse), #10458 (provider categories), #10786 (xAI media+search bundle), #2984 (FAL duplicate), #19086 (Google Veo standalone — easy port to plugin interface). Co-authored-by: Jaaneek <Jaaneek@users.noreply.github.com> * feat(video_gen): dynamic schema reflects active backend's capabilities Address the 'capability variance' question — instead of one tool with a static schema that lies about what every backend supports, the video_generate tool now rebuilds its description at get_definitions() time based on the configured video_gen.provider and video_gen.model. The agent sees backend-specific guidance up-front: - 'fal-ai/veo3.1/image-to-video': 'image-to-video only — image_url is REQUIRED; text-only prompts will be rejected' - 'fal-ai/veo3.1' (t2v): no image_url restriction shown - xAI grok-imagine-video: 'operations: generate, edit, extend; up to 7 reference_image_urls' - Backends without edit/extend: 'not supported on this backend — surface that they need to switch backends via hermes tools' This is the same pattern PR #22694 used for delegate_task self-capping — documented in the dynamic-tool-schemas skill. Cache invalidation is free: get_tool_definitions() already memoizes on config.yaml mtime, so a mid-session backend swap rebuilds the schema automatically. Tested: - Empirical FAL OpenAPI schema check confirms image-to-video models require image_url (FAL returns HTTP 422 otherwise) — client-side rejection in FALVideoGenProvider.generate() now prevents the wasted round-trip - Live E2E: fal-ai/veo3.1/image-to-video + prompt-only → clean missing_image_url error; fal-ai/veo3.1 + prompt-only → dispatches - 6 new tests cover the builder (no config / image-only / full-surface / text-only / unknown provider / registry wiring), all passing - 37/37 in the slice, 134/134 in the broader regression set * test(video_gen/xai): full surface integration tests + cleaner schema Verified end-to-end that the xAI plugin handles every documented mode from PR #10600's surface: text-to-video, image-to-video, reference-images-to-video, video edit, video extend (with and without prompt). All five modes route to the correct xAI endpoint (/videos/generations, /videos/edits, /videos/extensions) with the right payload shape (image / reference_images / video keys), and all five client-side rejections fire before the network: edit-without-prompt, extend-without-video_url, image+refs conflict, >7 references, and duration/aspect_ratio clamping. 15 new integration tests grouped into four classes (endpoint routing, modalities, validation, clamping). httpx is stubbed via a small fake AsyncClient that records POSTs so the tests assert the actual payload the plugin would send to xAI — not just the success/error envelope. Also cleaned up a description redundancy: when a model's operations match the backend's overall set, we no longer print the duplicate 'operations supported by this model' line. xAI's description now reads: Active backend: xAI . model: grok-imagine-video - operations supported by this backend: edit, extend, generate - modalities supported by this backend: image, reference_images, text - aspect_ratio choices: 16:9, 1:1, 2:3, 3:2, 3:4, 4:3, 9:16 - resolution choices: 480p, 720p - duration range: 1-15s - reference_image_urls: up to 7 images Co-authored-by: Jaaneek <Jaaneek@users.noreply.github.com> * feat(video_gen): collapse surface to t2v + i2v, family-based auto-routing Two design changes per Teknium: 1) Drop edit/extend from the tool surface entirely. Only text-to-video and image-to-video remain. The agent sees a clean tool with two modalities; backend-specific quirks like xAI's edit/extend endpoints stay out of the unified schema. 2) FAL: pick a model FAMILY once, the plugin routes between the family's text-to-video and image-to-video endpoints based on whether image_url was passed. Users no longer pick 'fal-ai/veo3.1' AND 'fal-ai/veo3.1/image-to-video' as separate options — they pick 'veo3.1', and the plugin handles the rest. Catalog rewritten as families: veo3.1 fal-ai/veo3.1 / fal-ai/veo3.1/image-to-video pixverse-v6 fal-ai/pixverse/v6/text-to-video / fal-ai/pixverse/v6/image-to-video kling-o3-standard fal-ai/kling-video/o3/standard/text-to-video / fal-ai/kling-video/o3/standard/image-to-video xAI uses a single endpoint (/videos/generations) for both modes, routed by the presence of the 'image' field in the payload — no edit/extend exposure. Schema changes: - VIDEO_GENERATE_SCHEMA: drop operation, drop video_url. Final params: prompt (required), image_url, reference_image_urls, duration, aspect_ratio, resolution, negative_prompt, audio, seed, model. - VideoGenProvider ABC: drop normalize_operation, VALID_OPERATIONS, DEFAULT_OPERATION. capabilities() drops 'operations' key. - success_response: add 'modality' field ('text' | 'image') so the agent and logs can see which endpoint was actually hit. Dynamic schema builder simplified — no operations bullet, no 'switch backends if you need edit/extend' guidance. When the active backend supports both modalities (the common case), description reads: Active backend: FAL . model: pixverse-v6 - supports both text-to-video (omit image_url) and image-to-video (pass image_url) - routes automatically - aspect_ratio choices: 16:9, 9:16, 1:1 - resolution choices: 360p, 540p, 720p, 1080p - duration range: 1-15s - audio: pass audio=true to enable native audio (pricing tier) - negative_prompt: supported Tests: 51 in the video_gen slice, 216 across the broader image+video sweep, all passing. New FAL routing tests prove pixverse-v6 + no image hits text-to-video endpoint, pixverse-v6 + image_url hits image-to-video endpoint, same for veo3.1 and kling-o3-standard. Docs updated: developer-guide page rewrites the 'model families' pattern as a first-class section so external plugin authors know the convention. toolsets-reference and toolsets.py descriptions match the new surface. Co-authored-by: Jaaneek <Jaaneek@users.noreply.github.com> * feat(video_gen/fal): expand catalog to 6 families, cheap + premium tiers Catalog now covers everything Teknium specced from FAL: Cheap tier: ltx-2.3 fal-ai/ltx-2.3-22b/text-to-video / image-to-video pixverse-v6 fal-ai/pixverse/v6/text-to-video / image-to-video Premium tier: veo3.1 fal-ai/veo3.1 / fal-ai/veo3.1/image-to-video seedance-2.0 bytedance/seedance-2.0/text-to-video / image-to-video kling-v3-4k fal-ai/kling-video/v3/4k/text-to-video / image-to-video happy-horse fal-ai/happy-horse/text-to-video / image-to-video DEFAULT_MODEL moved from veo3.1 (premium) to pixverse-v6 (cheap, sane defaults, both modalities) — better first-run UX for users who haven't explicitly picked a model. New family-entry knob: image_param_key. Kling v3 4K's image-to-video endpoint expects start_image_url instead of image_url; declaring image_param_key='start_image_url' on the family lets _build_payload remap correctly. Other families default to plain image_url. Per-family capability flags reflect each model's docs: - LTX 2.3 + Happy Horse: minimal payloads (no duration/aspect/resolution enum exposed by FAL — let endpoint apply defaults) - Seedance: 6 aspect ratios incl 21:9, durations 4-15, audio supported, negative prompts NOT supported per docs - Kling v3 4K: 16:9/9:16/1:1, 3-15s, audio + negative - Veo 3.1: unchanged, 16:9/9:16, 4/6/8s Tests: +5 covering the new families (full catalog, Kling 4K start_image_url remap, Seedance routing, LTX payload minimality, Happy Horse minimality). 56/56 in the slice green. Note: I did NOT add the FAL-hosted xAI Grok-Imagine variant. Hermes already has a direct xAI plugin that talks to xAI's own API; routing the same model through FAL's wrapper would duplicate the surface without adding capabilities. Users on FAL who want Grok-Imagine should use the xAI plugin directly; flag if you want both routes available. * test(video_gen): tool-surface routing matrix — every model x modality End-to-end matrix test driven through _handle_video_generate() — the actual function the agent's video_generate tool call lands in. Writes config.yaml, invokes the registered handler with a raw args dict, then asserts the outbound HTTP/SDK call hit the right endpoint with the right payload shape. Parametrized over FAL_FAMILIES.keys() so the matrix auto-discovers new families as they're added (add a family to FAL_FAMILIES and you get both modalities tested for free). Coverage: - All 6 FAL families x {text-only, text+image} = 12 cases - xAI x {text-only, text+image} = 2 cases - tool-level model= arg overrides config = 2 cases For each case, verifies: - result['success'] is True - result['modality'] matches input shape ('text' if no image_url, 'image' otherwise) - outbound endpoint URL matches the family's text_endpoint or image_endpoint - text-only payloads carry no image-shaped keys - text+image payloads carry the family's image key (image_url for most, start_image_url for kling-v3-4k, wrapped 'image' object for xAI) All 16 cases passing. Confirms the tool surface routes every (provider, model, modality) combination correctly with zero leakage. * feat(video_gen): keep video_gen out of first-run setup, surface in status Two changes: 1. video_gen joins _DEFAULT_OFF_TOOLSETS, so it is NOT pre-selected in the first-run toolset checklist. Video gen is niche, paid, and slow — most users don't want it nagging them during initial setup. Anyone who wants it opts in via 'hermes tools' -> Video Generation, which already routes to the provider+model picker. 2. The 'hermes setup' status panel learns about video_gen — but only shows the row when a plugin reports available. Users without FAL_KEY/XAI_API_KEY see nothing about video gen; users with one of those keys see 'Video Generation (FAL) ✓' as confirmation it's wired. Verified live: - Fresh install (no creds): zero video_gen mentions in wizard. - With FAL_KEY: status row appears with active backend name. - 160/160 in the setup + tools_config + video_gen test slice. Rationale: image_gen is on by default because it's a featured creative tool used in casual chat (telegrams, etc). Video gen is heavier — long wait, paid per-second pricing. Default-off matches user intent better. --------- Co-authored-by: Jaaneek <Jaaneek@users.noreply.github.com>	2026-05-13 16:39:41 -07:00
Teknium	80c4b27437	docs(lsp): document follow-up fixes from #24630 (#24709) ... - Note that typescript-language-server pulls in the typescript SDK automatically (peer-dep relationship was previously implicit and caused initialize failures when the SDK was absent). - Add a Troubleshooting entry for the new Backend warnings section in hermes lsp status, with the shellcheck install commands across apt / brew / scoop. Reflects what shipped in PR #24630.	2026-05-12 18:44:33 -07:00
Teknium	83b93898c2	feat(lsp): semantic diagnostics from real language servers in write_file/patch (#24168) ... * feat(lsp): semantic diagnostics from real language servers in write_file/patch Wire ~26 language servers (pyright, gopls, rust-analyzer, typescript-language-server, clangd, bash-language-server, ...) into the post-write lint check used by write_file and patch. The model now sees type errors, undefined names, missing imports, and project-wide semantic issues introduced by its edits, not just syntax errors. LSP is gated on git workspace detection: when the agent's cwd or the file being edited is inside a git worktree, LSP runs against that workspace; otherwise the existing in-process syntax checks are the only tier. This keeps users on user-home cwds (Telegram/Discord gateway chats) from spawning daemons. The post-write check is layered: in-process syntax check first (microseconds), then LSP semantic diagnostics second when syntax is clean. Diagnostics are delta-filtered against a baseline captured at write start, so the agent only sees errors its edit introduced. A flaky/missing language server can never break a write -- every LSP failure path falls back silently to the syntax-only result. New module agent/lsp/ split into: - protocol.py: Content-Length JSON-RPC framer + envelope helpers - client.py: async LSPClient (spawn, initialize, didOpen/didChange, ContentModified retry, push/pull diagnostic stores) - workspace.py: git worktree walk-up + per-server NearestRoot resolver - servers.py: registry of 26 language servers (extension match, root resolver, spawn builder per language) - install.py: auto-install dispatch (npm install --prefix, go install with GOBIN, pip install --target) into HERMES_HOME/lsp/bin/ - manager.py: LSPService (per-(server_id, root) client registry, lazy spawn, broken-set, in-flight dedupe, sync facade for tools layer) - reporter.py: <diagnostics> block formatter (severity-1-only, 20-per-file) - cli.py: hermes lsp {status,list,install,install-all,restart,which} Wired into tools/file_operations.py: - write_file/patch_replace now call _snapshot_lsp_baseline before write - _check_lint_delta gains a third tier: LSP semantic diagnostics when syntax is clean - All LSP code paths swallow exceptions; write_file's contract unchanged Config: 'lsp' section in DEFAULT_CONFIG with enabled (default true), wait_mode, wait_timeout, install_strategy (default 'auto'), and per-server overrides (disabled, command, env, initialization_options). Tests: tests/agent/lsp/ -- 49 tests covering protocol framing (encode and read_message round-trip, EOF/truncation/missing Content-Length), workspace gate (git walk-up, exclude markers, fallback to file location), reporter (severity filter, max-per-file cap, truncation), service-level delta filter, and an in-process mock LSP server that exercises the full client lifecycle including didChange version bumps, dedup, crash recovery, and idempotent teardown. Live E2E verified end-to-end through ShellFileOperations: pyright auto-installed via npm into HERMES_HOME, baseline captured, type error introduced, single delta diagnostic surfaced with correct line/column/code/ source, then patch fix removes the diagnostic from the output. Docs: new website/docs/user-guide/features/lsp.md page covering supported languages, configuration knobs, performance characteristics, and troubleshooting; cli-commands.md updated with the 'hermes lsp' reference; sidebar updated. * feat(lsp): structured logging, backend gate, defensive walk caps Cherry-picks the substantive ideas from #24155 (different scope, same problem space) onto our PR. agent/lsp/eventlog.py (new): dedicated structured logger ``hermes.lint.lsp`` with steady-state silence. Module-level dedup sets keep a 1000-write session at exactly ONE INFO line ("active for <root>") at the default INFO threshold; clean writes log at DEBUG so they never reach agent.log under normal config. State transitions (server starts, no project root for a file, server unavailable) fire at INFO/WARNING once per (server_id, key); novel events (timeouts, unexpected errors) fire WARNING per call. Grep recipe: ``rg 'lsp\\['``. agent/lsp/manager.py: wire the eventlog into _get_or_spawn and get_diagnostics_sync so users can answer "did LSP fire on this edit?" with a single grep, plus surface "binary not on PATH" warnings once instead of silently retrying every write. tools/file_operations.py: backend-type gate. ``_lsp_local_only()`` returns False for non-local backends (Docker / Modal / SSH / Daytona); ``_snapshot_lsp_baseline`` and ``_maybe_lsp_diagnostics`` now skip entirely on remote envs. The host-side language server can't see files inside a sandbox, so this prevents pretending to lint a file the host process can't open. agent/lsp/protocol.py: 8 KiB cap on the header block in ``read_message``. A pathological server that streams headers without ever emitting CRLF-CRLF would have looped forever consuming bytes; now raises ``LSPProtocolError`` instead. agent/lsp/workspace.py: 64-step cap on ``find_git_worktree`` and ``nearest_root`` upward walks, plus try/except containment around ``Path(...).resolve()`` and child ``.exists()`` calls. Defensive against pathological inputs (symlink loops, encoding errors, permission failures mid-walk) — the lint hook is hot-path code and must never raise. Tests: - tests/agent/lsp/test_eventlog.py: 18 tests covering steady-state silence (clean writes stay DEBUG), state-transition INFO-once semantics (active for, no project root), action-required WARNING-once (server unavailable), per-call WARNING (timeouts, spawn failures), and the "1000 clean writes => 1 INFO" contract. - tests/agent/lsp/test_backend_gate.py: 5 tests verifying _lsp_local_only / snapshot_baseline / maybe_lsp_diagnostics skip the LSP layer for non-local backends and route correctly for LocalEnvironment. - tests/agent/lsp/test_protocol.py: new test_read_message_rejects_runaway_header exercising the 8 KiB cap. Validation: - 73/73 LSP tests pass (49 original + 18 eventlog + 5 backend-gate + 1 framer cap) - 198/198 pass when run alongside existing file_operations tests - Live E2E re-run with pyright still surfaces "ERROR [2:12] Type ... reportReturnType (Pyright)" through the full path, then patch fix removes it on the next call. * feat(lsp): atexit cleanup + separate lsp_diagnostics JSON field Two improvements salvaged from #24414's plugin-form alternative, keeping our core-integrated design: 1. atexit cleanup of spawned language servers ---------------------------------------------------------------- ``agent/lsp/__init__.get_service`` now registers an ``atexit`` handler on first creation that tears down the LSPService on Python exit. Without this, every ``hermes chat`` exit was leaking pyright/gopls/etc. processes for a few seconds while their stdout buffers drained -- they got reaped by the kernel eventually but a watchful ``ps aux`` would catch them. The handler runs once per process (gated by ``_atexit_registered``); idempotent ``shutdown_service`` ensures double-fire is a no-op. Errors during shutdown are swallowed at debug level since by the time atexit fires the user has already seen the agent's final response. 2. Separate ``lsp_diagnostics`` field on WriteResult / PatchResult ---------------------------------------------------------------- Previously the LSP layer folded its diagnostic block into the ``lint.output`` string, conflating the syntax-check tier with the semantic tier. The agent (and any downstream parsers) now read syntax errors and semantic errors as independent signals: { "bytes_written": 42, "lint": {"status": "ok", "output": ""}, "lsp_diagnostics": "<diagnostics file=...>\nERROR [2:12] ..." } ``_check_lint_delta`` returns to its original two-tier shape (syntax check + delta filter); ``write_file`` and ``patch_replace`` independently fetch LSP diagnostics via ``_maybe_lsp_diagnostics`` and pass them into the new field. ``patch_replace`` propagates the inner write_file's ``lsp_diagnostics`` so the outer PatchResult carries the patch's delta correctly. Tests: 19 new - tests/agent/lsp/test_lifecycle.py (8 tests): atexit registration fires once and only once across N get_service calls; the registered callable is our internal shutdown wrapper; shutdown_service is idempotent and safe when never started; exceptions during shutdown are swallowed; inactive service is cached so we don't rebuild on every check. - tests/agent/lsp/test_diagnostics_field.py (11 tests): WriteResult / PatchResult dataclass shape, to_dict include/omit semantics, channel separation (lint and lsp_diagnostics carry independent signals), write_file populates the field via _maybe_lsp_diagnostics only when the syntax tier is clean, patch_replace propagates the field forward from its internal write_file. Validation: - 92/92 LSP tests pass (73 prior + 8 lifecycle + 11 diagnostics field) - 217/217 pass with file_operations + LSP combined - Live E2E reverified: clean writes -> both fields empty/none; type error introduced -> lint clean (parses), lsp_diagnostics carries the pyright reportReturnType block; patch fix -> both fields clean again. * fix(lsp): broken-set short-circuit so a wedged server isn't paid every write Discovered while auditing failure paths: a language server binary that hangs (sleep forever, no LSP traffic on stdin/stdout) caused EVERY subsequent write to re-pay the 8s snapshot_baseline timeout. Five writes = ~64s of dead time. The bug: ``_get_or_spawn`` adds the (server_id, root) pair to ``_broken`` inside its inner exception handler, but when the OUTER ``_loop.run`` timeout fires, it cancels the inner task before that handler runs. The pair never makes it to broken-set, so the next write re-enters the spawn path and re-pays the timeout. Fix: - New ``_mark_broken_for_file`` helper at the service layer marks the (server_id, workspace_root) pair broken from the OUTSIDE when the outer timeout fires. Called from the except branches in ``snapshot_baseline``, ``get_diagnostics_sync`` (asyncio.TimeoutError + generic Exception). Also kills any orphan client process that survived the cancelled future, fire-and-forget with a 1s ceiling. - ``enabled_for`` now consults the broken-set BEFORE returning True. Files in already-broken (server_id, root) pairs short-circuit to False, so the file_operations layer skips the LSP path entirely with no spawn cost. Until the service is restarted (``hermes lsp restart``) or the process exits. - A single eventlog WARNING is emitted on first mark-broken so the user knows which server gave up. Subsequent edits in the same project stay silent. Tests: 7 new in tests/agent/lsp/test_broken_set.py — covers the key shape (server_id, per_server_root), enabled_for short-circuit, sibling-file skip in same project, project isolation (broken in A doesn't affect B), graceful no-op for missing-server / no-workspace, and an end-to-end test that snapshots after a failure and verifies the next ``enabled_for`` returns False. Validation: - Live retest of the wedged-binary scenario: 5 sequential writes, first 8.88s (the one snapshot timeout), subsequent four ~0.84s (no LSP cost). Down from 5x12.85s = 64s before this fix. - 99/99 LSP tests pass (92 prior + 7 broken-set) - 224/224 pass with file_operations + LSP combined - Happy path E2E reverified — clean write, type error introduced, patch fix all behave correctly with the new broken-set logic. Note: the FIRST write to a wedged binary still pays 8s (the snapshot_baseline timeout). We could shorten that, but pyright/ tsserver normally take 2-3s and slow CI rust-analyzer can need 5+ seconds, so 8s is the conservative ceiling. Subsequent writes are instant.	2026-05-12 16:31:54 -07:00
Teknium	38441a7d77	docs(camofox): expand externally-managed sessions section (#24584) ... Adds behavior detail to the existing 'Externally managed Camofox sessions' subsection in features/browser.md: - Three-row settings table (config key + env var + effect). - 'What changes when user_id is set' — soft-cleanup behavior, why DELETE /sessions/<user_id> is skipped. - 'How tab adoption works' — 4-step lookup against GET /tabs, listItemId matching, fallback to new-tab creation, no mid-run re-polling. - Picking session_key: how to attach to a specific existing tab vs share-profile-only behavior with the default per-task session_key. - Concurrency note that Camofox does not arbitrate per-tab focus.	2026-05-12 15:20:42 -07:00
Dan Benyamin	62fd905340	feat(browser): support externally managed Camofox sessions ... Allow integrations to share a visible Camofox identity with Hermes and recover existing tabs without carrying local patches. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-12 15:14:49 -07:00
Teknium	ced1990c1c	feat(computer-use): refresh cua-driver on hermes update + add install --upgrade (#24063) ... cua-driver was only installed once on toolset enable: `_run_post_setup` early-returns when the binary is already on PATH, so upstream fixes (e.g. v0.1.6 Safari window-focus fix) never reached existing users without manual reinstall. Two refresh points now: - `hermes update` re-runs the upstream installer at the end of the update if cua-driver is on PATH (macOS-only, no-op otherwise). Ties driver freshness to the user-controlled update cadence — no startup latency, no per-launch GitHub API call. - `hermes computer-use install --upgrade` for manual force-refresh. The upstream `install.sh` always pulls the latest release, so re-running is the canonical upgrade path. No version-comparison logic needed. `hermes computer-use status` now shows the installed version, and points at `--upgrade` for refreshing.	2026-05-11 17:10:58 -07:00
Teknium	3b122cc1ac	feat(kanban): stranded_in_ready diagnostic for unclaimed tasks (#23578) ... Surface ready tasks that nobody claims within a threshold (default 30 min) regardless of why. One identity-agnostic signal that catches: - Operator typo'd the assignee - Profile was deleted, leaving its tasks stranded - External worker pool (Codex CLI lane, custom daemon) is down - Dispatcher misconfigured (wrong board / wrong HERMES_HOME) Today the dispatcher correctly skips these (no respawn loop, good) but nothing surfaces the fact that operator-actionable work is accumulating. The new `stranded_in_ready` rule does that without requiring a manual lane registry — it reads the most recent ready- transition event (`created` / `promoted` / `reclaimed` / `unblocked`) and fires when (now - last_ready_ts) > threshold. Severity escalates with age: warning at threshold, error at 2x, critical at 6x. The cli_hint and reassign actions point operators at the right next step. Out of scope deliberately: - Lane registry (#20157 closed) — this signal supersedes it. - Pushing the diagnostic into messaging gateways — diagnostics are pull-only via 'hermes kanban diagnostics' for now; gateway push is a separate UX decision. Tests: 10 new + 461 existing kanban tests pass. E2E verified end- to-end via 'hermes kanban diagnostics --json' against a 2h-old stranded task — surfaces as error severity with correct actions.	2026-05-10 21:58:44 -07:00
Teknium1	ae83a54be4	docs(kanban): worker lane contract page + review-required convention ... Closes the architectural-pin part of #19931. Most of what that issue asked for is already implemented (logs under kanban root, env-pinned workspace, dispatcher routing of unknown assignees, lifecycle ownership, structured handoff conventions). What was missing: 1. A written contract integrators can point at when adding a new worker lane shape, and 2. The "code-changing workers should not auto-promote success to done" convention. This commit ships both as docs+convention layered on existing primitives. No kernel changes — the kanban_complete / kanban_block / kanban_comment surfaces already support the review-required pattern; we just hadn't written it down or made it visible to workers. Changes: - `agent/prompt_builder.py::KANBAN_GUIDANCE`: append the review-required exception to step 5 of the lifecycle. Workers get the cue auto-injected into their system prompt — drop structured metadata into a kanban_comment first, then end with kanban_block(reason="review-required: <summary>") instead of kanban_complete when the work needs review. Total prompt size went from ~3000 to ~3275 chars; well under the 4096 budget enforced by test_kanban_guidance_size. - `skills/devops/kanban-worker/SKILL.md`: add a worked example to the existing "Good summary + metadata shapes" section between the Coding-task and Research-task examples. Same shape as the others (kanban_comment with structured handoff JSON, then kanban_block with the human-readable reason). Plus a one-line guide on when to use kanban_complete vs the review-required pattern. - `website/docs/user-guide/features/kanban-worker-lanes.md` (new): the integrator-facing contract. Covers the hierarchy, the three things every lane must provide (assignee, spawn mechanism, lifecycle terminator), the env vars the dispatcher injects, the review-required convention, the failure modes the kernel handles for free, and an explicit "external CLI worker lane" deferred- pending-concrete-asker section that links to #19931 and #19924. - `website/sidebars.ts`: link the new page under user-guide/features. The "specialist worker lanes for external CLI tools (Codex / Claude Code / OpenCode)" runner is NOT shipped here. The dispatcher's spawn_fn parameter already supports plugin-shaped extension; the per-CLI integration work (auth, sandbox policy, exit-code mapping) needs a concrete asker. The new docs page tells would-be integrators the contract any such lane must satisfy. Refs #19931	2026-05-10 18:15:52 -07:00
Teknium	6e5c49bdc4	refactor(kanban-orchestrator): drop hardcoded specialist roster, add Step-0 profile discovery ... The skill enumerated 8 specialist profile names (researcher, analyst, writer, reviewer, backend-eng, frontend-eng, ops, pm) as "the standard roster" and told orchestrators to "assume these exist." Almost no real Hermes setup matches that fleet — single-profile setups, Docker-worker setups, and curated-team setups all violate it — so following the skill literally produced cards assigned to non-existent profiles, which the dispatcher silently failed to spawn (no autocorrect, no fallback, just sits in `ready` forever). Changes: - Drop the standard-specialist-roster table. - Add a "Profiles are user-configured — not a fixed roster" section at the top with a Step 0 that prescribes `hermes profile list` (or asking the user) before fanning out. Cache the result in working memory. - Rewrite the worked task-graph example with placeholder names (<profile-A>, <profile-B>, <profile-C>) so the structure is still teachable but doesn't invite copy-paste of role names that may not exist. - Reframe the "If no specialist fits" anti-temptation rule: don't invent profile names; ask the user. - Add a "Inventing profile names that doesn't exist" entry to Pitfalls. - Bump skill version 2.0.0 → 3.0.0 (semantic break: previous behavior promised a roster the skill no longer enumerates). - Update website/docs/user-guide/features/kanban.md to drop the matching "(researcher, writer, analyst, backend-eng, reviewer, ops)" line and explain the discovery prompt instead. - Re-run website/scripts/generate-skill-docs.py to refresh the auto-generated skill page + catalog. Closes #21131 in spirit — addresses the same hardcoded-names footgun @yehuosi flagged, with a different shape than their PR (delete the roster rather than replace each name with placeholder, since the roster table was the load-bearing footgun and the worked example is salvageable with placeholder profile names). Co-authored-by: yehuosi <yehuosi@users.noreply.github.com>	2026-05-10 12:59:11 -07:00
Teknium	d4b26df897	perf(browser): route browser_console eval through supervisor's persistent CDP WS (180x faster) (#23226) ... Adds CDPSupervisor.evaluate_runtime() and wires it into _browser_eval as a fast path when a supervisor is alive for the current task_id. Replaces the ~180ms agent-browser subprocess fork+exec+Node-startup hop with a ~1ms Runtime.evaluate over the supervisor's already-connected WebSocket. Falls through to the existing agent-browser CLI path when no supervisor is running (e.g. backends without CDP, or before the first browser_navigate attaches one), so behaviour is unchanged where it can't apply. JS-side exceptions surface directly without falling through to the subprocess (the subprocess would just re-raise the same error, slower); supervisor-side failures (loop down, no session) fall through cleanly. Benchmark — 30 iterations of `1 + 1` against headless Chrome: supervisor WS mean= 0.96ms median= 0.91ms agent-browser subprocess mean=179.35ms median=167.73ms → 187x speedup mean Tests: 14 unit tests (mocked supervisor + response-shape coverage), 5 real-Chrome e2e tests in test_browser_supervisor.py (gated on Chrome being installed). Browser test suite: 355 passed, 1 skipped.	2026-05-10 07:37:55 -07:00
Teknium	5aa755e4e6	feat(plugins): run any LLM call from inside a plugin via ctx.llm (#23194) ... * feat(plugins): host-owned LLM access via ctx.llm Plugins can now ask the host to run a one-shot chat or structured completion against the user's active model and auth, without ever seeing an OAuth token or API key. Closes the gap where plugins that needed bounded structured inference (receipts, CRM extraction, support classification) had to either bring their own provider keys or register a tool the agent had to call. New surface on PluginContext: - ctx.llm.complete(messages, ...) - ctx.llm.complete_structured(instructions, input, json_schema, ...) - async siblings ctx.llm.acomplete / acomplete_structured Backed by the existing auxiliary_client.call_llm pipeline — every provider, fallback chain, vision routing, and timeout policy Hermes already supports applies automatically. Trust gate (fail-closed by default): - plugins.entries.<id>.llm.allow_model_override - plugins.entries.<id>.llm.allowed_models (allowlist; '*' = any) - plugins.entries.<id>.llm.allow_agent_id_override - plugins.entries.<id>.llm.allow_profile_override Embedded model@profile shorthand goes through the same gate as explicit profile=, so it can't bypass the auth-profile policy. Conflicting explicit and embedded profiles fail closed. Also lands: - plugins/plugin-llm-example/ — reference plugin that registers /receipt-extract, demonstrating image+text structured input, jsonschema validation, and the trust-gate config. - website/docs/developer-guide/plugin-llm-access.md — full API docs. - 45 unit tests covering trust gates, JSON parsing, schema validation, image encoding, async surface, and config loading. Validation: - 2628 tests pass in tests/agent/ - E2E: bundled plugin loaded with isolated HERMES_HOME, slash command produced parsed JSON via stubbed call_llm - response_format extra_body wired correctly for both json_object and json_schema modes * docs(plugin-llm): rewrite quickstart and framing The quickstart now uses a meeting-notes-to-tasks example instead of a receipt extractor, and the page leads with hook-time / gateway pre-filter / scheduled-job framing rather than the OpenClaw KB/support/CRM/finance/migration enumeration that the original upstream PR used. Receipt example moved to a separate worked example link so the docs page itself doesn't echo any of the upstream framing. Also clarifies where ctx.llm fits in the broader plugin surface (table comparing register_tool / register_platform / register_hook / etc.) and what makes this lane different from auxiliary_client internals. No code change. * docs(plugin-llm): reframe as any LLM call, not just structured output The original draft leaned heavily on complete_structured() and made the chat lane (complete() / acomplete()) feel like a footnote. Restructure so: - The page title and description say 'any LLM call.' - The lead shows BOTH a plain chat call (error rewriter) AND a structured call (triage scorer) up top. - Quick start has two complete plugin examples — /tldr (chat) and /paste-to-tasks (structured). - New 'When to use which' table for choosing complete() vs complete_structured() vs the async siblings. - Trust-gate sections explicitly note 'all four methods,' and the request-shaping list calls out chat-only fields (messages) and structured-only fields (instructions, input, json_schema) alongside each other. - The 'Where this fits' section now says 'for any reason, structured or not.' The receipt-extractor reference plugin still exists under plugins/plugin-llm-example/ — but the docs page no longer treats it as the canonical surface example. It's now described as 'a third worked example, this time with image input.' No code change. * feat(plugin-llm): split provider/model into independent explicit kwargs The first cut accepted a single 'provider/model' slug on every method and split it internally. That looked clean but broke under live test: the model-override path tried to use the slug's vendor prefix as a literal Hermes provider id, which silently switched the user off their aggregator (e.g. plugin asks for 'openai/gpt-4o-mini' on a user who routes through OpenRouter — host attempted to call the 'openai' provider directly, failed because OPENAI_API_KEY wasn't set). New shape mirrors the host's main config: ctx.llm.complete( messages=[...], provider='openrouter', # gated, optional model='openai/gpt-4o-mini', # gated, optional profile='work', # gated, optional ... ) Each is independently gated by its own allow_*_override flag. Granting model-override does NOT auto-grant provider-override. Allowlists are now per-axis (allowed_providers, allowed_models) matched literally against whatever string the plugin sends. Dropped 'model@profile' embedded-suffix shorthand entirely. Hermes doesn't use that pattern anywhere else; profile= is its own kwarg. Live E2E (against real OpenRouter via Teknium's config) confirms: - zero-config call works - default-deny blocks each override with a helpful error - model-only override stays on user's active provider (the bug) - provider+model override switches cleanly - allowlist refuses non-listed entries - structured output round-trip parses + schema-validates Tests: 49 cases (up from 45); all green. Docs updated to match the new shape, including a 'most plugins never need this section' callout on the trust-gate config block. * fix+cleanup(plugin-llm): real attribution, hook-mode coverage, move example out of core Three integration fixes for the ctx.llm surface: 1. Attribution bug — result.provider and result.model now reflect what call_llm actually used, not placeholder fallbacks ('auto', 'default'). New _resolve_attribution() helper: - explicit overrides win (what the call targeted) - response.model wins for the recorded model (provider canonicalisation: 'gpt-4o' → 'gpt-4o-2024-08-06' etc.) - falls back to _read_main_provider() / _read_main_model() when no override is set, so audit logs reflect the user's active main provider/model - 'auto' / 'default' only when EVERYTHING is empty Live verified: zero-config call now records provider='openrouter', model='anthropic/claude-4.7-opus-20260416' instead of provider='auto', model='default'. 2. Hook-mode coverage — TestHookMode confirms ctx.llm.complete works from inside a registered post_tool_call callback. The docs page promised hook integration; now there's a test that exercises the lazy-import path through the real invoke_hook machinery. Two cases: traceback-rewrite hook with conditional ctx.llm.complete, and minimal hook regression for the sync-hook + sync-llm path. 3. Reference plugin moved out of core. plugins/plugin-llm-example/ is gone from hermes-agent — it now lives in the new NousResearch/hermes-example-plugins companion repo. The docs page links there. Hermes' bundled plugins should be plugins users actually run; reference / docs-companion plugins live externally. Test count: 56 (up from 49). Wider sweep on tests/hermes_cli/ + tests/gateway/ + tests/tools/ + tests/agent/ shows 16770 passing; the 12 failures are all pre-existing on origin/main (verified by stashing this branch's changes and re-running) — kanban-boards, delegate-task, gateway-restart, tts-routing — none touch the plugin_llm surface. * chore(plugins): move all example plugins to companion repo Reference / docs-companion plugins now live exclusively in NousResearch/hermes-example-plugins, not bundled with the core repo: - example-dashboard - strike-freedom-cockpit A new fourth example, plugin-llm-async-example, was added to that repo demonstrating ctx.llm's async surface (acomplete()) with asyncio.gather() — registers /translate <lang>: <text> which fires forward translation + sentiment classifier in parallel, then a back-translation for QA. Live-tested at 2.5s for three real provider round-trips (would be ~5-6s sequential). Docs updated: - developer-guide/plugin-llm-access.md links both sync and async examples in the Reference section - user-guide/features/extending-the-dashboard.md repoints both demo sections to the companion repo with corrected install paths - user-guide/features/built-in-plugins.md drops the two demo rows - AGENTS.md notes that example plugins live in the companion repo Net: hermes-agent's plugins/ directory now contains only plugins users actually run (memory providers, dashboard tabs that ship real features, the disk-cleanup hook, platform adapters). All four demo / reference plugins live externally where they can be cloned on demand instead of inflating the core install.	2026-05-10 07:09:28 -07:00
Teknium	9cdcf31cae	docs(web-search): explain auxiliary-model summarization for web_extract (#23211) ... web_extract runs returned page content through the web_extract auxiliary model when pages exceed 5 000 chars (single-pass up to 500k, chunked up to 2M, refused above that). The user-guide page didn't mention this — users were surprised that long-page extracts produced summaries instead of raw markdown, and that those summaries cost main-model tokens by default. Adds: - size-driven behavior table (under 5k / 5k–500k / 500k–2M / over 2M) - which auxiliary task does the work (auxiliary.web_extract) - how to route summaries to a cheap model regardless of main - escape hatch: browser_navigate when you need raw content - troubleshooting entry for summarization timeouts	2026-05-10 06:40:23 -07:00
Eric Litovsky	236cbe16b6	feat(kanban): add orchestrator board tools	2026-05-10 05:58:44 -07:00
Teknium	252d68fd45	docs: deep audit — fix stale config keys, missing commands, and registry drift (#22784) ... * docs: deep audit — fix stale config keys, missing commands, and registry drift Cross-checked ~80 high-impact docs pages (getting-started, reference, top-level user-guide, user-guide/features) against the live registries: hermes_cli/commands.py COMMAND_REGISTRY (slash commands) hermes_cli/auth.py PROVIDER_REGISTRY (providers) hermes_cli/config.py DEFAULT_CONFIG (config keys) toolsets.py TOOLSETS (toolsets) tools/registry.py get_all_tool_names() (tools) python -m hermes_cli.main <subcmd> --help (CLI args) reference/ - cli-commands.md: drop duplicate hermes fallback row + duplicate section, add stepfun/lmstudio to --provider enum, expand auth/mcp/curator subcommand lists to match --help output (status/logout/spotify, login, archive/prune/ list-archived). - slash-commands.md: add missing /sessions and /reload-skills entries + correct the cross-platform Notes line. - tools-reference.md: drop bogus '68 tools' headline, drop fictional 'browser-cdp toolset' (these tools live in 'browser' and are runtime-gated), add missing 'kanban' and 'video' toolset sections, fix MCP example to use the real mcp_<server>_<tool> prefix. - toolsets-reference.md: list browser_cdp/browser_dialog inside the 'browser' row, add missing 'kanban' and 'video' toolset rows, drop the stale '38 tools' count for hermes-cli. - profile-commands.md: add missing install/update/info subcommands, document fish completion. - environment-variables.md: dedupe GMI_API_KEY/GMI_BASE_URL rows (kept the one with the correct gmi-serving.com default). - faq.md: Anthropic/Google/OpenAI examples — direct providers exist (not just via OpenRouter), refresh the OpenAI model list. getting-started/ - installation.md: PortableGit (not MinGit) is what the Windows installer fetches; document the 32-bit MinGit fallback. - installation.md / termux.md: installer prefers .[termux-all] then falls back to .[termux]. - nix-setup.md: Python 3.12 (not 3.11), Node.js 22 (not 20); fix invalid 'nix flake update --flake' invocation. - updating.md: 'hermes backup restore --state pre-update' doesn't exist — point at the snapshot/quick-snapshot flow; correct config key 'updates.pre_update_backup' (was 'update.backup'). user-guide/ - configuration.md: api_max_retries default 3 (not 2); display.runtime_footer is the real key (not display.runtime_metadata_footer); checkpoints defaults enabled=false / max_snapshots=20 (not true / 50). - configuring-models.md: 'hermes model list' / 'hermes model set ...' don't exist — hermes model is interactive only. - tui.md: busy_indicator -> tui_status_indicator with values kaomoji|emoji|unicode|ascii (not kawaii|minimal|dots|wings|none). - security.md: SSH backend keys (TERMINAL_SSH_HOST/USER/KEY) live in .env, not config.yaml. - windows-wsl-quickstart.md: there is no 'hermes api' subcommand — the OpenAI-compatible API server runs inside hermes gateway. user-guide/features/ - computer-use.md: approvals.mode (not security.approval_level); fix broken ./browser-use.md link to ./browser.md. - fallback-providers.md: top-level fallback_providers (not model.fallback_providers); the picker is subcommand-based, not modal. - api-server.md: API_SERVER_* are env vars — write to per-profile .env, not 'hermes config set' which targets YAML. - web-search.md: drop web_crawl as a registered tool (it isn't); deep-crawl modes are exposed through web_extract. - kanban.md: failure_limit default is 2, not '~5'. - plugins.md: drop hard-coded '33 providers' count. - honcho.md: fix unclosed quote in echo HONCHO_API_KEY snippet; document that 'hermes honcho' subcommand is gated on memory.provider=honcho; reconcile subcommand list with actual --help output. - memory-providers.md: legacy 'hermes honcho setup' redirect documented. Verified via 'npm run build' — site builds cleanly; broken-link count went from 149 to 146 (no regressions, fixed a few in passing). * docs: round 2 audit fixes + regenerate skill catalogs Follow-up to the previous commit on this branch: Round 2 manual fixes: - quickstart.md: KIMI_CODING_API_KEY mentioned alongside KIMI_API_KEY; voice-mode and ACP install commands rewritten — bare 'pip install ...' doesn't work for curl-installed setups (no pip on PATH, not in repo dir); replaced with 'cd ~/.hermes/hermes-agent && uv pip install -e ".[voice]"'. ACP already ships in [all] so the curl install includes it. - cli.md / configuration.md: 'auxiliary.compression.model' shown as 'google/gemini-3-flash-preview' (the doc's own claimed default); actual default is empty (= use main model). Reworded as 'leave empty (default) or pin a cheap model'. - built-in-plugins.md: added the bundled 'kanban/dashboard' plugin row that was missing from the table. Regenerated skill catalogs: - ran website/scripts/generate-skill-docs.py to refresh all 163 per-skill pages and both reference catalogs (skills-catalog.md, optional-skills-catalog.md). This adds the entries that were genuinely missing — productivity/teams-meeting-pipeline (bundled), optional/finance/* (entire category — 7 skills: 3-statement-model, comps-analysis, dcf-model, excel-author, lbo-model, merger-model, pptx-author), creative/hyperframes, creative/kanban-video-orchestrator, devops/watchers, productivity/shop-app, research/searxng-search, apple/macos-computer-use — and rewrites every other per-skill page from the current SKILL.md. Most diffs are tiny (one line of refreshed metadata). Validation: - 'npm run build' succeeded. - Broken-link count moved 146 -> 155 — the +9 are zh-Hans translation shells that lag every newly-added skill page (pre-existing pattern). No regressions on any en/ page.	2026-05-09 13:19:51 -07:00
Teknium	8f711f79a4	fix(tools): install cua-driver when Computer Use is enabled via 'hermes tools' (#22765) ... Returning users who enabled '🖱️ Computer Use (macOS)' via 'hermes tools' saw '✓ Saved configuration' but no install — cua-driver was never on PATH and the toolset failed at first use. Two compounding causes: 1. _toolset_needs_configuration_prompt fell through to _toolset_has_keys, which returned True for any provider with empty env_vars. cua-driver has no env vars, so the gate skipped _configure_toolset entirely and _run_post_setup('cua_driver') never ran. 2. No stable CLI entry-point existed for re-running the install when the picker no-op'd it (e.g. when toggling the toolset off+on inside one picker session, where 'added' is empty). Changes: - hermes_cli/tools_config.py: add _POST_SETUP_INSTALLED registry mapping post_setup keys to installed-state predicates. The gate now returns True when any visible provider has a registered post_setup whose predicate fails. cua_driver is the only opt-in for now; other post_setup hooks keep their existing behaviour. - hermes_cli/main.py: add 'hermes computer-use install' and 'hermes computer-use status' as a stable docs target. install reuses the same _run_post_setup('cua_driver') path that the picker invokes; status reports whether cua-driver is on PATH. - tools/computer_use/cua_backend.py: install hint now points users at 'hermes computer-use install' first. - website/docs/user-guide/features/computer-use.md: document the new command as the primary install path. - website/docs/reference/cli-commands.md: catalog 'hermes computer-use' alongside 'hermes tools'. - tests/hermes_cli/test_post_setup_gating.py: regression coverage for the gate predicate (missing -> setup forced, installed -> setup skipped, broken predicate -> non-blocking, unregistered keys -> behaviour unchanged). Fixes #22737. Reported by @f-trycua.	2026-05-09 13:02:25 -07:00
Teknium	9de893e3b0	feat(windows): close native-Windows install gaps — crash-free startup, UTF-8 stdio, tzdata dep, docs ... Native Windows (with Git for Windows installed) can now run the Hermes CLI and gateway end-to-end without crashing. install.ps1 already existed and the Git Bash terminal backend was already wired up — this PR fills the remaining gaps discovered by auditing every Windows-unsafe primitive (`signal.SIGKILL`, `os.kill(pid, 0)` probes, bare `fcntl`/`termios` imports) and by comparing hermes against how Claude Code, OpenCode, Codex, and Cline handle native Windows. ## What changed ### UTF-8 stdio (new module) - `hermes_cli/stdio.py` — single `configure_windows_stdio()` entry point. Flips the console code page to CP_UTF8 (65001), reconfigures `sys.stdout`/`stderr`/`stdin` to UTF-8, sets `PYTHONIOENCODING` + `PYTHONUTF8` for subprocesses. No-op on non-Windows. Opt out via `HERMES_DISABLE_WINDOWS_UTF8=1`. - Called early in `cli.py::main`, `hermes_cli/main.py::main`, and `gateway/run.py::main` so Unicode banners (box-drawing, geometric symbols, non-Latin chat text) don't `UnicodeEncodeError` on cp1252 consoles. ### Crash sites fixed - `hermes_cli/main.py:7970` (hermes update → stuck gateway sweep): raw `os.kill(pid, _signal.SIGKILL)` → `gateway.status.terminate_pid(pid, force=True)` which routes through `taskkill /T /F` on Windows. - `hermes_cli/profiles.py::_stop_gateway_process`: same fix — also converted SIGTERM path to `terminate_pid()` and widened OSError catch on the intermediate `os.kill(pid, 0)` probe. - `hermes_cli/kanban_db.py:2914, 3041`: raw `signal.SIGKILL` → `getattr(signal, "SIGKILL", signal.SIGTERM)` fallback (matches the pattern already used in `gateway/status.py`). ### OSError widening on `os.kill(pid, 0)` probes Windows raises `OSError` (WinError 87) for a gone PID instead of `ProcessLookupError`. Widened the catch at: - `gateway/run.py:15101` (`--replace` wait-for-exit loop — without this, the loop busy-spins the full 10s every Windows gateway start) - `hermes_cli/gateway.py:228, 460, 940` - `hermes_cli/profiles.py:777` - `tools/process_registry.py::_is_host_pid_alive` - `tools/browser_tool.py:1170, 1206` ### Dashboard PTY graceful degradation `hermes_cli/pty_bridge.py` depends on `fcntl`/`termios`/`ptyprocess`, none of which exist on native Windows. Previously a Windows dashboard would crash on `import hermes_cli.web_server` because of a top-level import. Now: - `hermes_cli/web_server.py` wraps the pty_bridge import in `try/except ImportError` and sets `_PTY_BRIDGE_AVAILABLE=False`. - The `/api/pty` WebSocket handler returns a friendly "use WSL2 for this tab" message instead of exploding. - Every other dashboard feature (sessions, jobs, metrics, config editor) runs natively on Windows. ### Dependency - `pyproject.toml`: add `tzdata>=2023.3; sys_platform == 'win32'` so Python's `zoneinfo` works on Windows (which has no IANA tzdata shipped with the OS). Credits @sprmn24 (PR #13182). ### Docs - README.md: removed "Native Windows is not supported"; added PowerShell one-liner and Git-for-Windows prerequisite note. - `website/docs/getting-started/installation.md`: new Windows section with capability matrix (everything native except the dashboard `/chat` PTY tab, which is WSL2-only). - `website/docs/user-guide/windows-wsl-quickstart.md`: reframed as "WSL2 as an alternative to native" rather than "the only way". - `website/docs/developer-guide/contributing.md`: updated cross-platform guidance with the `signal.SIGKILL` / `OSError` rules we enforce now. - `website/docs/user-guide/features/web-dashboard.md`: acknowledged native Windows works for everything except the embedded PTY pane. ## Why this shape Pulled from a survey of how other agent codebases handle native Windows (Claude Code, OpenCode, Codex, Cline): - All four treat Git Bash as the canonical shell on Windows, same as hermes already does in `tools/environments/local.py::_find_bash()`. - None of them force `SetConsoleOutputCP` — but they don't have to, Node/Rust write UTF-16 to the Win32 console API. Python does not get that for free, so we flip CP_UTF8 via ctypes. - None of them ship PowerShell-as-primary-shell (Claude Code exposes PS as a secondary tool; scope creep for this PR). - All of them use `taskkill /T /F` for force-kill on Windows, which is exactly what `gateway.status.terminate_pid(force=True)` does. ## Non-goals (deliberate scope limits) - No PowerShell-as-a-second-shell tool — worth designing separately. - No terminal routing rewrite (#12317, #15461, #19800 cluster) — that's the hardest design call and needs a separate doc. - No wholesale `open()` → `open(..., encoding="utf-8")` sweep (Tianworld cluster) — will do as follow-up if users hit actual breakage; most modern code already specifies it. ## Validation - 28 new tests in `tests/tools/test_windows_native_support.py` — all platform-mocked, pass on Linux CI. Cover: - `configure_windows_stdio` idempotency, opt-out, env-preservation - `terminate_pid` taskkill routing, failure → OSError, FileNotFoundError fallback - `getattr(signal, "SIGKILL", …)` fallback shape - `_is_host_pid_alive` OSError widening (Windows-gone-PID behavior) - Source-level checks that all entry points call `configure_windows_stdio` - pty_bridge import-guard present in `web_server.py` - README no longer says "not supported" - 12 pre-existing tests in `tests/tools/test_windows_compat.py` still pass. - `tests/hermes_cli/` ran fully (3909 passed, 9 failures — all confirmed pre-existing on main by stash-test). - `tests/gateway/` ran fully (5021 passed, 1 pre-existing failure). - `tests/tools/test_process_registry.py` + `test_browser_*` pass. - Manual smoke: `import hermes_cli.stdio; import gateway.run; import hermes_cli.web_server` — all clean, `_PTY_BRIDGE_AVAILABLE=True` on Linux (as expected). ## Files - New: `hermes_cli/stdio.py`, `tests/tools/test_windows_native_support.py` - Modified: `cli.py`, `gateway/run.py`, `hermes_cli/main.py`, `hermes_cli/profiles.py`, `hermes_cli/gateway.py`, `hermes_cli/kanban_db.py`, `hermes_cli/pty_bridge.py`, `hermes_cli/web_server.py`, `tools/browser_tool.py`, `tools/process_registry.py`, `pyproject.toml`, `README.md`, and 4 docs pages. Credits to everyone whose prior PR work informed these fixes — see the co-author trailers. All of the PRs listed in `~/.hermes/plans/windows-support-prs.md` fixing `os.kill` / `signal.SIGKILL` / UTF-8 stdio / tzdata / README patterns found the same issues; this PR consolidates them. Co-authored-by: Philip D'Souza <9472774+PhilipAD@users.noreply.github.com> Co-authored-by: Arecanon <42595053+ArecaNon@users.noreply.github.com> Co-authored-by: XiaoXiao0221 <263113677+XiaoXiao0221@users.noreply.github.com> Co-authored-by: Lars Hagen <1360677+lars-hagen@users.noreply.github.com> Co-authored-by: Luan Dias <65574834+luandiasrj@users.noreply.github.com> Co-authored-by: Ruzzgar <ruzzgarcn@gmail.com> Co-authored-by: sprmn24 <oncuevtv@gmail.com> Co-authored-by: adybag14-cyber <252811164+adybag14-cyber@users.noreply.github.com> Co-authored-by: Prasanna28Devadiga <54196612+Prasanna28Devadiga@users.noreply.github.com>	2026-05-08 14:27:40 -07:00
Teknium	850413f120	feat(computer-use): cua-driver backend, universal any-model schema ... Background macOS desktop control via cua-driver MCP — does NOT steal the user's cursor or keyboard focus, works with any tool-capable model. Replaces the Anthropic-native `computer_20251124` approach from the abandoned #4562 with a generic OpenAI function-calling schema plus SOM (set-of-mark) captures so Claude, GPT, Gemini, and open models can all drive the desktop via numbered element indices. - `tools/computer_use/` package — swappable ComputerUseBackend ABC + CuaDriverBackend (stdio MCP client to trycua/cua's cua-driver binary). - Universal `computer_use` tool with one schema for all providers. Actions: capture (som/vision/ax), click, double_click, right_click, middle_click, drag, scroll, type, key, wait, list_apps, focus_app. - Multimodal tool-result envelope (`_multimodal=True`, OpenAI-style `content: [text, image_url]` parts) that flows through handle_function_call into the tool message. Anthropic adapter converts into native `tool_result` image blocks; OpenAI-compatible providers get the parts list directly. - Image eviction in convert_messages_to_anthropic: only the 3 most recent screenshots carry real image data; older ones become text placeholders to cap per-turn token cost. - Context compressor image pruning: old multimodal tool results have their image parts stripped instead of being skipped. - Image-aware token estimation: each image counts as a flat 1500 tokens instead of its base64 char length (~1MB would have registered as ~250K tokens before). - COMPUTER_USE_GUIDANCE system-prompt block — injected when the toolset is active. - Session DB persistence strips base64 from multimodal tool messages. - Trajectory saver normalises multimodal messages to text-only. - `hermes tools` post-setup installs cua-driver via the upstream script and prints permission-grant instructions. - CLI approval callback wired so destructive computer_use actions go through the same prompt_toolkit approval dialog as terminal commands. - Hard safety guards at the tool level: blocked type patterns (curl|bash, sudo rm -rf, fork bomb), blocked key combos (empty trash, force delete, lock screen, log out). - Skill `apple/macos-computer-use/SKILL.md` — universal (model-agnostic) workflow guide. - Docs: `user-guide/features/computer-use.md` plus reference catalog entries. 44 new tests in tests/tools/test_computer_use.py covering schema shape (universal, not Anthropic-native), dispatch routing, safety guards, multimodal envelope, Anthropic adapter conversion, screenshot eviction, context compressor pruning, image-aware token estimation, run_agent helpers, and universality guarantees. 469/469 pass across tests/tools/test_computer_use.py + the affected agent/ test suites. - `model_tools.py` provider-gating: the tool is available to every provider. Providers without multi-part tool message support will see text-only tool results (graceful degradation via `text_summary`). - Anthropic server-side `clear_tool_uses_20250919` — deferred; client-side eviction + compressor pruning cover the same cost ceiling without a beta header. - macOS only. cua-driver uses private SkyLight SPIs (SLEventPostToPid, SLPSPostEventRecordTo, _AXObserverAddNotificationAndCheckRemote) that can break on any macOS update. Pin with HERMES_CUA_DRIVER_VERSION. - Requires Accessibility + Screen Recording permissions — the post-setup prints the Settings path. Supersedes PR #4562 (pyautogui/Quartz foreground backend, Anthropic- native schema). Credit @0xbyt4 for the original #3816 groundwork whose context/eviction/token design is preserved here in generic form.	2026-05-08 11:07:38 -07:00
Teknium	486b14b423	feat(cron): routing intent — deliver=all fans out to every connected channel (#21495) ... Adds one reserved token to the cron `deliver` field: - `all` — expand to every platform with a configured home channel Resolves at fire time, not create time, so a job created before Telegram was wired up picks it up once `TELEGRAM_HOME_CHANNEL` is set. Composes with existing targets: `origin,all`, `all,telegram:-100:17`. Inspired by Vellum Assistant's reminder routing-intent system. ## Changes - cron/scheduler.py: _expand_routing_tokens + integrate into _resolve_delivery_targets - tools/cronjob_tools.py: schema description updated - tests/cron/test_scheduler.py: TestRoutingIntents (5 cases) - website/docs/user-guide/features/cron.md: docs + table rows ## Validation - tests/cron/test_scheduler.py -k 'Routing or Deliver' → 57 passed	2026-05-08 04:17:21 -07:00
helix4u	faa13e49f8	docs(web): fix SearXNG env configuration	2026-05-07 17:54:47 -07:00
Teknium	cff821e2dc	docs: register triage_specifier in the aux-models enumerations (#21494) ... The kanban specifier landed in #21435 with feature-page docs (the kanban page itself + the CLI reference table), but three other docs pages enumerate every auxiliary task slot and were missed: user-guide/configuration.md Auxiliary Models section — interactive picker example + full auxiliary config reference YAML block. user-guide/features/fallback-providers.md Both 'Auxiliary Tasks' and 'Fallback Reference' tables. user-guide/features/kanban-tutorial.md Triage-column bullet now mentions the ✨ Specify button + CLI + slash command. No other docs enumerate the aux task slots (verified with grep -r 'title_generation\|auxiliary.session_search' website/docs/).	2026-05-07 13:07:18 -07:00
Teknium	24d48ffb82	feat(kanban): add specify — auxiliary LLM fleshes out triage tasks (#21435) ... * feat(kanban): add `specify` — auxiliary LLM fleshes out triage tasks The Triage column shipped with a placeholder 'a specifier will flesh out the spec', but the specifier itself was never built. This wires it up as a dedicated CLI verb. `hermes kanban specify <id>` calls the auxiliary LLM (configured under `auxiliary.triage_specifier`) to expand a rough one-liner into a concrete spec — tightened title plus a body with Goal / Approach / Acceptance criteria / Out-of-scope sections — then atomically flips `status: triage -> todo` and recomputes ready so parent-free tasks go straight to the dispatcher on the same tick. Surface: hermes kanban specify <task_id> # single task hermes kanban specify --all [--tenant T] # sweep triage column hermes kanban specify ... --author NAME # audit-comment author hermes kanban specify ... --json # one JSON line per task Design choices: - Parent gating is preserved. specify_triage_task flips to 'todo', then recompute_ready promotes to 'ready' only when parents are done — same rule as a normal parent-gated todo. - No daemon, no background watcher. Every invocation is explicit — keeps cost predictable and doesn't fight the dispatcher loop. - Response parse is lenient: strict JSON preferred, markdown-fence tolerated, raw-body fallback on malformed JSON so the LLM can't strand a task in triage. - All failure modes (no aux client, API error, task moved out of triage mid-call) return SpecifyOutcome(ok=False, reason=...) so --all continues past individual failures. Changes: hermes_cli/kanban_db.py + specify_triage_task() hermes_cli/kanban_specify.py NEW (~220 LOC — prompt, parse, call) hermes_cli/kanban.py + specify subcommand + _cmd_specify hermes_cli/config.py + auxiliary.triage_specifier task slot website/docs/user-guide/features/kanban.md specify + config notes website/docs/reference/cli-commands.md CLI reference entry tests/hermes_cli/test_kanban_specify_db.py NEW (10 tests) tests/hermes_cli/test_kanban_specify.py NEW (20 tests) Validation: 30/30 targeted tests pass. E2E: triage task -> specify -> ends in 'ready' with events [created, specified, promoted] and the audit comment recorded under the configured author. * feat(kanban): wire specifier into dashboard and gateway slash Follow-ups to the initial PR #21435 — closes the two gaps I'd left as post-merge: dashboard button and first-class gateway surface. Dashboard (plugins/kanban/dashboard/) - POST /tasks/:id/specify NEW endpoint. Thin wrapper around kanban_specify.specify_task(). Returns the CLI outcome shape ({ok, task_id, reason, new_title}); ok=false with a human reason is a 200, not a 4xx, so the UI can render it inline without treating 'no aux client configured' as a crash. - Runs sync in FastAPI's threadpool because the LLM call can take tens of seconds on reasoning models. - Pins HERMES_KANBAN_BOARD around the specify call so the module's argless kb.connect() lands on the right board. - dist/index.js: doSpecify callback threaded through the drawer → TaskDetail → StatusActions prop chain. ✨ Specify button appears ONLY when task.status === 'triage' (elsewhere the backend would reject anyway — hide the button to keep the action row clean). Busy state (Specifying…) + inline success/error banner under the button using the response.reason text. - dist/style.css: tiny hermes-kanban-msg-ok / -err classes using existing --color vars so themes reskin cleanly. Gateway slash (/kanban specify) - Already works via the existing run_slash → build_parser → kanban_command pipeline. No code change needed — slash commands inherit the argparse tree automatically. Added coverage: test_run_slash_specify_end_to_end (create --triage, specify, verify promotion + retitle) and test_run_slash_specify_help_is_reachable. Tests - tests/plugins/test_kanban_dashboard_plugin.py: 3 new tests for the REST endpoint — happy path, non-triage rejection as ok=false 200, missing aux client as ok=false 200. - tests/hermes_cli/test_kanban_cli.py: 2 new slash-surface tests. Docs - website/docs/user-guide/features/kanban.md: dashboard action row description mentions ✨ Specify + all three surfaces. REST table gains /tasks/:id/specify. Slash examples include /kanban specify. Validation: 340/340 targeted tests pass. E2E via TestClient: create a triage task over REST → POST /specify with mocked aux client → task moves to 'ready' column on /board with new title and body applied.	2026-05-07 13:04:41 -07:00
Teknium	47bf5d7ecb	test+docs: cover transform_llm_output hook + release author map ... - tests/test_transform_llm_output_hook.py: dispatch semantics (kwargs contract, first-non-empty-string-wins, empty-string pass-through, raising-plugin fail-open, no-plugins = no-op) - tests/hermes_cli/test_plugins.py: assert the new hook name is in VALID_HOOKS alongside the other transform_* hooks - website/docs/user-guide/features/hooks.md: summary-table entry + full section mirroring transform_tool_result / transform_terminal_output - scripts/release.py: map barnacleboy.jezzahehn@agentmail.to -> JezzaHehn (existing entry only covers the gmail address)	2026-05-07 05:46:05 -07:00
Teknium	6b3a9b4bfa	docs(curator): update CLI docs for synchronous-by-default manual run ... Follow-up to the previous commit which flipped 'hermes curator run' default from async to sync. Updates the curator.md feature page and cli-commands.md reference to show --background as the opt-in async flag and note that the default now blocks until the LLM pass finishes.	2026-05-07 05:27:47 -07:00
Teknium	49c3c2e0d3	docs(kanban): fix worker skill setup instructions too (#20960) ... Follow-up to #20958. The worker skill section had the same stale 'hermes skills install devops/kanban-worker' command — kanban-worker is also bundled, so that command fails with 'Could not fetch from any source.' Replace with bundled-skill verification + restore pattern, matching the orchestrator section. Uses <your-worker-profile> placeholder since assignees vary (researcher, writer, ops, linguist, reviewer, etc.) rather than a single fixed 'worker' profile.	2026-05-06 18:40:30 -07:00
Gille	45cbf93899	docs(kanban): fix orchestrator skill setup instructions (#20958)	2026-05-06 18:14:30 -07:00
brooklyn!	f1a8e99942	fix(tui): honor skin highlight colors (#20895)	2026-05-06 14:01:56 -07:00
Teknium	d514dd4055	docs(tool-gateway): rewrite as pitch-first marketing page (#20827) ... Previous version read like internal API docs \u2014 leading with env var tables, config YAML, and 'precedence' rules before ever explaining the product. Complete rewrite inverts the structure so readers see value first, mechanics last. Structure now: - Lede: 'One subscription. Every tool built in.' + pitch paragraph - CTA: subscribe/manage button styled as a real call-to-action - What's included: emoji-led table with expanded descriptions per tool. Image gen lists all 9 models by name (FLUX 2 Klein/Pro, Z-Image Turbo, Nano Banana Pro, GPT Image 1.5/2, Ideogram V3, Recraft V4 Pro, Qwen) - Why it's here: value bullets \u2014 one bill, one signup, one key, same quality, bring-your-own anytime - Get started: two-command flow (hermes model \u2192 hermes status) - Eligibility: paid-tier note with upgrade link - Mix and match: three realistic usage patterns - Using individual image models: ID reference table for power users - --- separator --- - Configuration reference (demoted): use_gateway flag, disabling, self-hosted gateway env vars moved below the fold where they belong - FAQ: streamlined, removed redundant content Fact-checked against code: - 9 FAL models confirmed from tools/image_generation_tool.py FAL_MODELS - Status section output verified against hermes_cli/status.py - Portal subscription URL preserved - Self-hosted env vars (TOOL_GATEWAY_DOMAIN etc.) kept accurate Verified: docusaurus build SUCCESS, page renders, no new broken links.	2026-05-06 13:20:09 -07:00
kshitij	48c241840a	docs: add Web Search + Extract feature page with SearXNG setup guide	2026-05-06 10:20:05 -07:00
Teknium	6388aafbd6	feat(dashboard): add 'default-large' built-in theme with 18px base size (#20820) ... Same Hermes Teal palette as the default theme, but with baseSize 18px, lineHeight 1.65, and spacious density so the whole dashboard scales up. Gives users a one-click bigger-text preset and a copyable reference for authoring custom YAML themes with their own typography settings.	2026-05-06 09:10:44 -07:00
Teknium	773cf48c50	docs(plugins): close the gaps \u2014 image-gen-provider-plugin guide + publishing a skill tap (#20800) ... Two pluggable surfaces were mentioned in the interfaces map without a real authoring guide behind them: 1. **Image-gen backends** — only had 'See bundled examples' pointers. Now a full developer-guide/image-gen-provider-plugin.md (270 lines) mirroring the memory/context/model provider docs: - How discovery works, directory structure, plugin.yaml - ImageGenProvider ABC with every overridable method (name, display_name, is_available, list_models, default_model, get_setup_schema, generate) - Full authoring walkthrough with a working MyBackendImageGenProvider - Response-format reference (success_response / error_response) - Handling b64 vs URL output (save_b64_image helper) - User overrides at ~/.hermes/plugins/image_gen/<name>/ - Testing recipe + pip distribution - Reference examples (openai, openai-codex, xai) 2. **Skill taps** — features/skills.md mentioned the CLI commands but never explained the repo contract for publishing a tap. Added 'Publishing a custom skill tap' section under Skills Hub covering: - Repo layout (skills/<name>/SKILL.md by default) - Minimal working example - Non-default path configuration (taps.json) - Installing individual skills without subscribing - Trust-level handling - Full tap management CLI + in-session /skills tap commands Wired into: - website/sidebars.ts: image-gen-provider-plugin added to Extending group - website/docs/user-guide/features/plugins.md: pluggable interfaces table + 'What plugins can do' table now link to the real guides instead of 'See bundled examples' - website/docs/guides/build-a-hermes-plugin.md: top info map and inline sub-sections updated, 'Full guide:' line added to image-gen block, tap section mentions publishing Verified: docusaurus build SUCCESS, new page renders at /docs/developer-guide/image-gen-provider-plugin, anchor #publishing-a-custom-skill-tap resolves from plugins.md + build-a-hermes-plugin.md. Pre-existing zh-Hans broken links unchanged.	2026-05-06 08:40:05 -07:00
Teknium	b62a82e0c3	docs: pluggable surfaces coverage — model-provider guide, full plugin map, opt-in fix (#20749) ... * docs(providers): add model-provider-plugin authoring guide + fix stale refs New docs: - website/docs/developer-guide/model-provider-plugin.md — full authoring guide (directory layout, minimal example, ProviderProfile fields, overridable hooks, user overrides, api_mode selection, auth types, testing, pip distribution) - Wired into website/sidebars.ts under 'Extending' - Cross-references added in: - guides/build-a-hermes-plugin.md (tip block) - developer-guide/adding-providers.md - developer-guide/provider-runtime.md User guide: - user-guide/features/plugins.md: Plugin types table grows from 3 to 4 with 'Model providers' row Stale comment cleanup (providers/*.py → plugins/model-providers/<name>/): - hermes_cli/main.py:_is_profile_api_key_provider docstring - hermes_cli/doctor.py:_build_apikey_providers_list docstring - hermes_cli/auth.py: PROVIDER_REGISTRY + alias auto-extension comments - hermes_cli/models.py: CANONICAL_PROVIDERS auto-extension comment AGENTS.md: - Project-structure tree: added plugins/model-providers/ row - New section: 'Model-provider plugins' explaining discovery, override semantics, PluginManager integration, kind auto-coerce heuristic Verified: docusaurus build succeeds, new page renders, all 3 cross-links resolve. 347/347 targeted tests pass (tests/providers/, tests/hermes_cli/test_plugins.py, tests/hermes_cli/test_runtime_provider_resolution.py, tests/run_agent/test_provider_parity.py). * docs(plugins): add 'pluggable interfaces at a glance' maps to plugins.md + build-a-hermes-plugin Devs landing on either the user-guide plugin page or the build-a-plugin guide now get an upfront table of every distinct pluggable surface with a link to the right authoring doc. Previously they'd have to read the full general-plugin guide to discover that model providers / platforms / memory / context engines are separate systems. user-guide/features/plugins.md: - New 'Pluggable interfaces — where to go for each' section below the existing 4-kinds table - 10 rows covering every register_* surface (tool, hook, slash command, CLI subcommand, skill, model provider, platform, memory, context engine, image-gen) - Explicit note: TTS/STT are NOT plugin-extensible yet — documented with a pointer to the current config.yaml 'command providers' pattern and a note that register_tts_provider()/register_stt_provider() may come later guides/build-a-hermes-plugin.md: - New :::info 'Not sure which guide you need?' map at the top so devs see all pluggable interfaces before investing in this 737-line general-plugin walkthrough - Existing bottom :::tip expanded to include platform adapters alongside model/memory/context plugins Verified: - All 8 cross-doc links in the new plugins.md table resolve in a docusaurus build (SUCCESS, no new broken links) - TTS link corrected (features/voice → features/tts; latter exists) - Pre-existing broken links/anchors (cron-script-only, llms.txt, adding-platform-adapters#step-by-step-checklist) are unchanged * docs(plugins): correct TTS/STT pluggability \u2014 they ARE plugins (command-providers) Previous commit incorrectly said TTS/STT 'aren't plugin-extensible'. They are, via the config-driven command-provider pattern \u2014 any CLI that reads text and writes audio (or vice versa for STT) is automatically a plugin with zero Python. The tts.md docs cover this extensively and I missed it. plugins.md: - TTS row: 'Config-driven (not a Python plugin)', points at tts.md#custom-command-providers - STT row: points at tts.md#voice-message-transcription-stt (STT docs live in tts.md despite the filename) - Expanded note: TTS/STT use config-driven shell-command templates as their plugin surface (full tts.providers.<name> registry for TTS; HERMES_LOCAL_STT_COMMAND escape hatch for STT) - Any CLI that reads/writes files is automatically a plugin \u2014 no Python register_* API needed - Future register_tts_provider()/register_stt_provider() hooks mentioned as nice-to-have for SDK/streaming cases, not as the primary story build-a-hermes-plugin.md: - Same map update: TTS/STT rows explicit, footer note corrected Verified: - tts.md anchors (custom-command-providers, voice-message-transcription-stt) exist and resolve in docusaurus build (SUCCESS, no new broken links) * docs(plugins): expand pluggable interfaces table with MCP / event hooks / shell hooks / skill taps Broadened the scope beyond Python register_* hooks. Hermes has MULTIPLE plugin-style extension surfaces; they're now all in one table instead of being scattered across feature docs. Added rows for: - **MCP servers** — config.yaml mcp_servers.<name> auto-registers external tools from any MCP server. Huge extensibility surface, previously not linked from the plugin map. - **Gateway event hooks** — drop HOOK.yaml + handler.py into ~/.hermes/hooks/<name>/ to fire on gateway:startup, session:*, agent:*, command:* events. Separate from Python plugin hooks. - **Shell hooks** — hooks: block in config.yaml runs shell commands on events (notifications, auditing, etc.). - **Skill sources (taps)** — hermes skills tap add <repo> to pull in new skill registries beyond the built-in sources. Both docs updated: - user-guide/features/plugins.md: table column renamed to 'How' (mixes Python API + config-driven + drop-in-dir surfaces accurately) - guides/build-a-hermes-plugin.md: :::info map at top mirrors the new surfaces with a forward-link to the consolidated table Note block rewritten: instead of singling out TTS/STT as the 'different style' exception, now honestly describes that Hermes deliberately supports three plugin styles — Python APIs, config-driven commands, and drop-in manifest directories — and devs should pick the one that fits their integration. Not included (considered and rejected): - Transport layer (register_transport) — internal, not user-facing - Tool-call parsers — internal, VLLM phase-2 thing - Cloud browser providers — hardcoded registry, not drop-in yet - Terminal backends — hardcoded if/elif, not drop-in yet - Skill sources (the ABC) — hardcoded list, only taps are user-extensible Verified: - All 5 new anchors resolve (gateway-event-hooks, shell-hooks, skills-hub, custom-command-providers, voice-message-transcription-stt) - Docusaurus build SUCCESS, zero new broken links - Same 3 pre-existing broken links on main (cron-script-only, llms.txt, adding-platform-adapters#step-by-step-checklist) * docs(plugins): cover every pluggable surface in both the overview and how-to Both plugins.md and build-a-hermes-plugin.md now cover every extension surface end-to-end \u2014 general plugin APIs, specialized plugin types, config-driven surfaces \u2014 with concrete authoring patterns for each. plugins.md: - 'What plugins can do' table grows from 9 rows (general ctx.register_* only) to 14 rows covering register_platform, register_image_gen_provider, register_context_engine, MemoryProvider subclass, register_provider (model). Each row links to its full authoring guide. - New 'Plugin sub-categories' section under Plugin Discovery explains how plugins/platforms/, plugins/image_gen/, plugins/memory/, plugins/context_engine/, plugins/model-providers/ are routed to different loaders \u2014 PluginManager vs the per-category own-loader systems. - Explicit mention of user-override semantics at ~/.hermes/plugins/model-providers/ and ~/.hermes/plugins/memory/. build-a-hermes-plugin.md: - New '## Specialized plugin types' section (5 sub-sections): - Model provider plugins \u2014 ProviderProfile + plugin.yaml example, auto-wiring summary, link to full guide - Platform plugins \u2014 BasePlatformAdapter + register_platform() skeleton - Memory provider plugins \u2014 MemoryProvider subclass example - Context engine plugins \u2014 ContextEngine subclass example - Image-generation backends \u2014 ImageGenProvider + kind: backend example - New '## Non-Python extension surfaces' section (5 sub-sections): - MCP servers \u2014 config.yaml mcp_servers.<name> example - Gateway event hooks \u2014 HOOK.yaml + handler.py example - Shell hooks \u2014 hooks: block in config.yaml example - Skill sources (taps) \u2014 hermes skills tap add example - TTS / STT command templates \u2014 tts.providers.<name> with type: command - Distribute via pip / NixOS promoted from ### to ## (they were orphaned after the reorganization) Each specialized / non-Python section has a concrete, copy-pasteable example plus a 'Full guide:' link to the authoritative doc. Devs arriving at the build-a-hermes-plugin guide now see every extension surface at their disposal, not just the general tool/hook/slash-command surface. Verified: - Docusaurus build SUCCESS, zero new broken links - All new cross-links (developer-guide/model-provider-plugin, adding-platform-adapters, memory-provider-plugin, context-engine-plugin, user-guide/features/mcp, skills#skills-hub, hooks#gateway-event-hooks, hooks#shell-hooks, tts#custom-command-providers, tts#voice-message-transcription-stt) resolve - Same 3 pre-existing broken links on main (cron-script-only, llms.txt, adding-platform-adapters#step-by-step-checklist) * docs(plugins): fix opt-in inconsistency — not every plugin is gated The 'Every plugin is disabled by default' statement was wrong. Several plugin categories intentionally bypass plugins.enabled: - Bundled platform plugins (IRC, Teams) auto-load so shipped gateway channels are available out of the box. Activation per channel is via gateway.platforms.<name>.enabled. - Bundled backends (plugins/image_gen/*) auto-load so the default backend 'just works'. Selection via <category>.provider config. - Memory providers are all discovered; one is active via memory.provider. - Context engines are all discovered; one is active via context.engine. - Model providers: all 33 discovered at first get_provider_profile(); user picks via --provider / config. The plugins.enabled allow-list specifically gates: - Standalone plugins (general tools/hooks/slash commands) - User-installed backends - User-installed platforms (third-party gateway adapters) - Pip entry-point backends Which matches the actual code in hermes_cli/plugins.py:737 where the bundled+backend/platform check bypasses the allow-list. Rewrote '## Plugins are opt-in' to: - Retitle to 'Plugins are opt-in (with a few exceptions)' - Narrow opening claim to 'General plugins and user-installed backends are disabled by default' - Added 'What the allow-list does NOT gate' subsection with a full table of which bypass the gate and how they're activated instead - Fixed migration section wording (bundled platform/backend plugins never needed grandfathering) Verified: docusaurus build SUCCESS, zero new broken links.	2026-05-06 07:24:42 -07:00
Jun Han	0d945d1541	docs: update VS Code setup instructions for ACP Client integration	2026-05-05 14:16:30 -07:00
liuyuqi	a11234dd68	docs(browser): document WSL-to-Windows Chrome MCP bridge	2026-05-05 14:12:49 -07:00
Yuan Tao-Wen	39560c948d	docs(voice): add Doubao speech integration examples (TTS + STT)	2026-05-05 13:54:33 -07:00
0xVox	5bd75c73ed	docs(kanban): document handoff evidence metadata	2026-05-05 13:52:46 -07:00
Michel Belleau	5f8e59b0f1	docs(discord): fix Server Members Intent + SSRC-mapping drift; add /voice join slash Choice ... Salvage of #11350. Kept: - Code: add an explicit /voice join Choice in the slash UI (runner accepts both 'join' and 'channel' but only 'channel' was in autocomplete). - Docs: Server Members Intent is conditional (only needed if DISCORD_ALLOWED_USERS contains usernames); SSRC → user_id mapping uses the voice websocket SPEAKING opcode, not the Members intent. Dropped from the original PR: - HERMES_DISCORD_VOICE_PACKET_DUMP — this env var doesn't exist on main (it was in a different PR that isn't merged). - DISCORD_PROXY docs — already documented on current main. - DISCORD_ALLOW_MENTION_* docs — already on main. - "barge-in mode" rewrite — current main actually does pause the listener during TTS (VoiceReceiver.pause() at discord.py:192); there is no barge_in_guard/barge_in_rms on main. Co-authored-by: Michel Belleau <michel.belleau@malaiwah.com>	2026-05-05 13:50:43 -07:00
Wysie	af312ccc97	docs: fix Camofox Docker setup instructions	2026-05-05 13:41:46 -07:00
kshitijk4poor	20a4f79ed1	feat: provider modules — ProviderProfile ABC, 33 providers, fetch_models, transport single-path ... Introduces providers/ package — single source of truth for every inference provider. Adding a simple api-key provider now requires one providers/<name>.py file with zero edits anywhere else. What this PR ships: - providers/ package (ProviderProfile ABC + 33 profiles across 4 api_modes) - ProviderProfile declarative fields: name, api_mode, aliases, display_name, env_vars, base_url, models_url, auth_type, fallback_models, hostname, default_headers, fixed_temperature, default_max_tokens, default_aux_model - 4 overridable hooks: prepare_messages, build_extra_body, build_api_kwargs_extras, fetch_models - chat_completions.build_kwargs: profile path via _build_kwargs_from_profile, legacy flag path retained for lmstudio/tencent-tokenhub (which have session-aware reasoning probing that doesn't map cleanly to hooks yet) - run_agent.py: profile path for all registered providers; legacy path variable scoping fixed (all flags defined before branching) - Auto-wires: auth.PROVIDER_REGISTRY, models.CANONICAL_PROVIDERS, doctor health checks, config.OPTIONAL_ENV_VARS, model_metadata._URL_TO_PROVIDER - GeminiProfile: thinking_config translation (native + openai-compat nested) - New tests/providers/ (79 tests covering profile declarations, transport parity, hook overrides, e2e kwargs assembly) Deltas vs original PR (salvaged onto current main): - Added profiles: alibaba-coding-plan, azure-foundry, minimax-oauth (were added to main since original PR) - Skipped profiles: lmstudio, tencent-tokenhub stay on legacy path (their reasoning_effort probing has no clean hook equivalent yet) - Removed lmstudio alias from custom profile (it's a separate provider now) - Skipped openrouter/custom from PROVIDER_REGISTRY auto-extension (resolve_provider special-cases them; adding breaks runtime resolution) - runtime_provider: profile.api_mode only as fallback when URL detection finds nothing (was breaking minimax /v1 override) - Preserved main's legacy-path improvements: deepseek reasoning_content preserve, gemini Gemma skip, OpenRouter response caching, Anthropic 1M beta recovery, etc. - Kept agent/copilot_acp_client.py in place (rejected PR's relocation — main has 7 fixes landed since; relocation would revert them) - _API_KEY_PROVIDER_AUX_MODELS alias kept for backward compat with existing test imports Co-authored-by: kshitijk4poor <82637225+kshitijk4poor@users.noreply.github.com> Closes #14418	2026-05-05 13:40:01 -07:00
Tony Simons	e4723f671a	docs(cron): add context_from chaining section ... Resolved merge against current main (new No-agent mode section added in parallel). Co-authored-by: Tony Simons <tony@tonysimons.dev>	2026-05-05 13:34:03 -07:00
r266-tech	91f339b981	docs(plugins): document ctx.dispatch_tool() in plugin capabilities table	2026-05-05 13:33:56 -07:00
r266-tech	c28c2a2380	docs(tts): document per-provider max_text_length caps ... PR #13743 replaced the global MAX_TEXT_LENGTH=4000 with a per-provider table and a user-override 'max_text_length:' key, but the user-guide TTS page documented no length behaviour at all. Users hitting truncation had no way to discover the new caps or the override. Add an 'Input length limits' subsection after the existing Configuration YAML block: provider default caps (Edge 5000 / OpenAI 4096 / xAI 15000 / MiniMax 10000 / Mistral 4000 / Gemini 5000 / ElevenLabs model-aware / NeuTTS,KittenTTS 2000), ElevenLabs model_id -> cap table (5k-40k), an override example, and the validation rules (non-positive / non-integer / boolean values fall through to the provider default).	2026-05-05 13:28:53 -07:00
Teknium	b10e38e392	fix(skills): pin protects against deletion only, not edits (#20220) ... Previously, pinning a skill blocked every skill_manage write action (edit, patch, delete, write_file, remove_file). The 'hard fence' design conflated two concerns: 1. Pin as deletion protection — don't let the curator archive or the agent delete a stable skill. 2. Pin as content freeze — don't let the agent rewrite it mid-conversation. In practice (1) is what users pin for: they want a skill to survive curator passes. (2) created friction — agents finding a new pitfall in a pinned skill had to ask the user to unpin, then the agent patches, then the user re-pins. The dance discouraged skill maintenance and pinned skills went stale. This narrows the _pinned_guard to skill_manage(action='delete') only. Patches, edits, and supporting-file writes go through on pinned skills so the agent can keep improving them. The curator's own pinned-skip behavior (agent/curator.py:271 for auto-archive, line 349 for the LLM review prompt) is unchanged — curator still never touches pinned skills. Changes: - tools/skill_manager_tool.py: remove _pinned_guard calls from _edit_skill, _patch_skill, _write_file, _remove_file; keep on _delete_skill. Updated _pinned_guard docstring and error message. - tools/skill_manager_tool.py: updated skill_manage model-facing tool description to reflect the new semantic. - website/docs/user-guide/features/curator.md: updated pinning section. - tests/tools/test_skill_manager_tool.py: flipped refuses-pinned tests for edit/patch/write_file/remove_file into allowed-when-pinned; kept test_delete_refuses_pinned (strengthened assertion to check the 'cannot be deleted' wording). Closes #18354	2026-05-05 05:43:10 -07:00
Aamir Jawaid	93869b48ab	docs: add Microsoft Teams to platform lists across docs ... Update all platform enumeration lists to include Teams: index.md, quickstart.md, integrations/index.md, sessions.md, slash-commands.md, updating.md, hooks.md, hermes-agent skill. Skipped PII redaction docs — Teams uses AAD object IDs, not phone numbers, so redaction doesn't apply there. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-04 20:59:18 -07:00
Teknium	a1bed18194	docs: clarify that the Docker terminal backend is a single persistent container (#20003) ... The docs were ambiguous about whether the Docker terminal backend spins up a fresh container per command or reuses a long-lived one. It's the latter — Hermes starts one container on first use and routes every terminal, file, and execute_code call through docker exec into that same container for the life of the process (across /new, /reset, and delegate_task subagents). Working-directory changes, installed packages, and files in /workspace persist from one tool call to the next, like a local shell. - configuration.md: lead the Docker Backend section with the persistence model before the YAML example; sharpen the Backend Overview table row. - features/tools.md: expand the Docker Backend block (previously just a 2-line YAML stub) with a clear statement of the persistent-container semantics and a pointer to the full lifecycle section. - docker.md: tighten the 'Docker as a terminal backend' bullet and the 'Skills and credential files' paragraph to call out the single-container model explicitly.	2026-05-04 20:09:31 -07:00
Teknium	9cda237bb1	docs(cron): lead with agent-driven setup for no-agent mode (#19871) ... The shipped no-agent docs introduced the feature via CLI first and mentioned the chat path as a two-line afterthought. That buries the actual value prop: the cronjob tool exposes no_agent directly to the agent, so a user can describe a watchdog in plain language and Hermes wires up the script + schedule + delivery without anyone opening an editor. Changes: * cron-script-only.md: promote 'Create One from Chat' above 'Create One from the CLI', flesh it out with a worked transcript (the actual tool calls the agent makes), add subsections covering 'what the agent decides for you' (when to pick no_agent=True vs LLM mode) and 'managing watchdogs from chat' (pause/resume/edit/ remove all agent-accessible). * user-guide/features/cron.md: - Add 'no-agent mode' to the top-level feature list with a cross- link, plus a sentence up top making it clear everything is agent-accessible through the cronjob tool. - Add 'The agent sets these up for you' subsection to the no-agent section showing the exact tool call shape. * automate-with-cron.md: tighten the existing tip box to mention the agent-driven path, not just CLI scheduling. No behavior change — docs only.	2026-05-04 12:39:19 -07:00
Teknium	3db6b9cc87	feat(cron): add no_agent mode for script-only cron jobs (watchdog pattern) (#19709) ... * feat(cron): add no_agent mode for script-only cron jobs (watchdog pattern) Adds a no_agent=True option to the cronjob system. When enabled, the scheduler runs the attached script on schedule and delivers its stdout directly to the job's target — no LLM, no agent loop, no token spend. This is the classic bash-watchdog pattern (memory alert every 5 min, disk alert every 15 min, CI ping) reimplemented as a first-class Hermes primitive instead of a systemd timer + curl + bot token triplet living outside the system. ## What hermes cron create "every 5m" \ --no-agent \ --script memory-watchdog.sh \ --deliver telegram \ --name memory-watchdog Agent tool: cronjob(action='create', schedule='every 5m', script='memory-watchdog.sh', no_agent=True, deliver='telegram') Semantics: - Script stdout (trimmed) → delivered verbatim as the message - Empty stdout → silent tick (no delivery; watchdog pattern) - wakeAgent=false gate → silent tick (same gate LLM jobs use) - Non-zero exit/timeout → delivered as an error alert (broken watchdogs shouldn't fail silently) - No LLM ever invoked; no tokens spent; no provider fallback applied ## Implementation cron/jobs.py * create_job gains no_agent: bool = False * prompt becomes Optional (no_agent jobs don't need one) * Validation: no_agent=True requires a script at create time * Field roundtrips via load_jobs / save_jobs / update_job cron/scheduler.py * run_job: new short-circuit branch at the top that runs the script, wraps its output into the (success, doc, final_response, error) tuple downstream delivery already expects, and returns before any AIAgent import or construction * _run_job_script: picks interpreter by extension — .sh/.bash run under /bin/bash, anything else under sys.executable (Python). Shell support unlocks the bash-watchdog pattern without wrapping scripts in Python. Extension is explicit; we deliberately do NOT trust the file's own shebang. Path-containment guard (scripts dir) unchanged. tools/cronjob_tools.py * Schema: new no_agent boolean property with clear trigger guidance * cronjob() accepts no_agent and validates mode-specific shape: - no_agent=True requires script; prompt/skills optional - no_agent=False keeps the existing 'prompt or skill required' rule * update path rejects flipping no_agent=True on a job without a script * _format_job surfaces no_agent in list output * Handler lambda forwards no_agent from tool args hermes_cli/main.py, hermes_cli/cron.py * 'hermes cron create --no-agent' and edit's --no-agent / --agent pair for toggling at CLI parity with the agent tool * Existing --script help text updated to describe both modes * List / create / edit output now shows 'Mode: no-agent (...)' when set ## Tests tests/cron/test_cron_no_agent.py — 18 tests covering: * create_job: no_agent shape, validation, field persistence * update_job: flag roundtrip across reload * cronjob tool: schema validation, update toggling, mode-specific requirements, prompt-relaxation rule * run_job short-circuit: - success path delivers stdout verbatim - empty stdout → SILENT_MARKER (no delivery downstream) - wakeAgent=false gate → silent - script failure → error alert - run_job does NOT import AIAgent (verified via mock) * _run_job_script: - .sh executes via bash (no shebang required) - .bash executes via bash - .py still runs via sys.executable (regression) - path-traversal still blocked (security regression) All 18 new tests pass. 341/342 pre-existing cron tests still pass; the one failure (test_script_empty_output_noted) was already broken on main and is unrelated to this change. ## Docs website/docs/guides/cron-script-only.md — new dedicated guide covering the watchdog pattern, interpreter rules, delivery mapping, worked examples (memory / disk alerts), and the comparison table vs hermes send, regular LLM cron jobs, and OS-level cron. website/docs/user-guide/features/cron.md — new 'No-agent mode' section in the cron feature reference, cross-linked to the guide. website/docs/guides/automate-with-cron.md — new tip box pointing users to no-agent mode when they don't need LLM reasoning. ## Compatibility - Existing jobs: unchanged. no_agent defaults to False, existing code paths untouched until the flag is set. - Schema additive only; older jobs.json without the field load fine via .get() with False default. - New CLI flags are opt-in and don't alter existing flag behavior. * fix(cron): lazy-import AIAgent + SessionDB so no_agent ticks pay zero The unconditional `from run_agent import AIAgent` + SessionDB() init at the top of run_job() meant every no_agent tick still paid the full agent module load cost (~300ms + transitive imports + DB open) even though it never touched any of that machinery. Move both to live under the default (LLM) path, after the no_agent short-circuit has returned. Now a no_agent tick's sys.modules stays clean — verified end-to-end: assert 'run_agent' not in sys.modules # before run_job(no_agent_job) assert 'run_agent' not in sys.modules # after The existing mock-based unit test (test_run_job_no_agent_never_invokes_aiagent) kept passing because patch() replaces the class AFTER import; the leak was only visible via real subprocess-style verification. End-to-end demo confirmed: agent calls cronjob(no_agent=True) → script runs → stdout delivered → no LLM machinery loaded. * docs(cron): tighten no_agent tool schema — defaults, silent semantics, pick rule Previous description buried the important bits in one long sentence. Agents could plausibly miss three things an LLM-facing schema should make unmissable: 1. What the default is — now first sentence + JSON Schema `default: false` 2. What 'silent run' actually means for the user — now spelled out: 'nothing is sent to the user and they won't see anything happened' 3. When to pick True vs False — now a concrete decision rule with examples on both sides (watchdogs/metrics/pollers → True; summarize/draft/pick/rephrase → False) Also adds explicit 'prompt and skills are ignored when True' since the agent could otherwise still pass them out of habit. No behavior change — schema text only.	2026-05-04 12:31:01 -07:00
helix4u	eeb05cf556	docs: default custom tool creation to plugins ... Steers custom tool creation toward the plugin route by default. The adding-tools.md guide is now explicitly for built-in core Hermes tools only. Key fixes: - Plugin quickstart: ctx.register_tool() now uses correct keyword-arg API (name=, toolset=, schema=, handler=) instead of broken 3-arg call - Handler signature: (params, **kwargs) instead of (params) - Handler return: json.dumps({...}) instead of plain string - AGENTS.md: mentions plugin route before built-in tool instructions - learning-path.md: plugins listed before core tool development - contributing.md: separates plugin vs core tool paths Based on PR #13138 by @helix4u.	2026-05-04 05:53:16 -07:00