fix(tools): install cua-driver when Computer Use is enabled via 'hermes tools' (#22765)

Returning users who enabled '🖱️ Computer Use (macOS)' via 'hermes tools'
saw '✓ Saved configuration' but no install — cua-driver was never on
PATH and the toolset failed at first use. Two compounding causes:

1. _toolset_needs_configuration_prompt fell through to _toolset_has_keys,
   which returned True for any provider with empty env_vars. cua-driver
   has no env vars, so the gate skipped _configure_toolset entirely and
   _run_post_setup('cua_driver') never ran.

2. No stable CLI entry-point existed for re-running the install when
   the picker no-op'd it (e.g. when toggling the toolset off+on inside
   one picker session, where 'added' is empty).

Changes:

- hermes_cli/tools_config.py: add _POST_SETUP_INSTALLED registry
  mapping post_setup keys to installed-state predicates. The gate
  now returns True when any visible provider has a registered
  post_setup whose predicate fails. cua_driver is the only opt-in
  for now; other post_setup hooks keep their existing behaviour.
- hermes_cli/main.py: add 'hermes computer-use install' and
  'hermes computer-use status' as a stable docs target. install
  reuses the same _run_post_setup('cua_driver') path that the
  picker invokes; status reports whether cua-driver is on PATH.
- tools/computer_use/cua_backend.py: install hint now points users
  at 'hermes computer-use install' first.
- website/docs/user-guide/features/computer-use.md: document the
  new command as the primary install path.
- website/docs/reference/cli-commands.md: catalog 'hermes
  computer-use' alongside 'hermes tools'.
- tests/hermes_cli/test_post_setup_gating.py: regression coverage
  for the gate predicate (missing -> setup forced, installed ->
  setup skipped, broken predicate -> non-blocking, unregistered
  keys -> behaviour unchanged).

Fixes #22737. Reported by @f-trycua.
This commit is contained in:
Teknium 2026-05-09 13:02:25 -07:00 committed by GitHub
parent 6e5489c9f3
commit 8f711f79a4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 205 additions and 4 deletions

View file

@ -8886,6 +8886,7 @@ def _build_provider_choices() -> list[str]:
_BUILTIN_SUBCOMMANDS = frozenset( _BUILTIN_SUBCOMMANDS = frozenset(
{ {
"acp", "auth", "backup", "checkpoints", "claw", "completion", "acp", "auth", "backup", "checkpoints", "claw", "completion",
"computer-use",
"config", "cron", "curator", "dashboard", "debug", "doctor", "config", "cron", "curator", "dashboard", "debug", "doctor",
"dump", "fallback", "gateway", "hooks", "import", "insights", "dump", "fallback", "gateway", "hooks", "import", "insights",
"kanban", "login", "logout", "logs", "mcp", "memory", "model", "kanban", "login", "logout", "logs", "mcp", "memory", "model",
@ -10506,6 +10507,54 @@ Examples:
tools_command(args) tools_command(args)
tools_parser.set_defaults(func=cmd_tools) tools_parser.set_defaults(func=cmd_tools)
# =========================================================================
# computer-use command — manage Computer Use (cua-driver) on macOS
# =========================================================================
computer_use_parser = subparsers.add_parser(
"computer-use",
help="Manage the Computer Use (cua-driver) backend (macOS)",
description=(
"Install or check the cua-driver binary used by the\n"
"`computer_use` toolset. macOS-only.\n\n"
"Use `hermes computer-use install` to fetch and run the\n"
"upstream cua-driver installer. This is equivalent to the\n"
"post-setup hook that `hermes tools` runs when you first\n"
"enable the Computer Use toolset, and is a stable target\n"
"for re-running the install if it didn't fire (e.g. when\n"
"toggling the toolset on a returning-user setup)."
),
)
computer_use_sub = computer_use_parser.add_subparsers(dest="computer_use_action")
computer_use_sub.add_parser(
"install",
help="Install or repair the cua-driver binary (macOS)",
)
computer_use_sub.add_parser(
"status",
help="Print whether cua-driver is installed and on PATH",
)
def cmd_computer_use(args):
action = getattr(args, "computer_use_action", None)
if action == "install":
from hermes_cli.tools_config import _run_post_setup
_run_post_setup("cua_driver")
return
if action == "status":
import shutil
path = shutil.which("cua-driver")
if path:
print(f"cua-driver: installed at {path}")
return
print("cua-driver: not installed")
print(" Run: hermes computer-use install")
return
# No subcommand → show help
computer_use_parser.print_help()
computer_use_parser.set_defaults(func=cmd_computer_use)
# ========================================================================= # =========================================================================
# mcp command — manage MCP server connections # mcp command — manage MCP server connections
# ========================================================================= # =========================================================================

View file

@ -12,6 +12,7 @@ the `platform_toolsets` key.
import json as _json import json as _json
import logging import logging
import os import os
import shutil
import sys import sys
from pathlib import Path from pathlib import Path
from typing import Dict, List, Optional, Set from typing import Dict, List, Optional, Set
@ -1424,12 +1425,52 @@ def _visible_providers(cat: dict, config: dict) -> list[dict]:
return visible return visible
_POST_SETUP_INSTALLED: dict = {
# post_setup_key -> predicate(): True when the install side-effect
# is already satisfied. Used by `_toolset_needs_configuration_prompt`
# to force the provider-setup flow when a no-key provider still needs
# a binary/dependency install (otherwise an already-configured user
# who toggles the toolset on via `hermes tools` gets a silent no-op
# because the gate sees "no env vars to ask about" and skips the
# provider-setup flow that would have run the post_setup hook).
#
# Only entries here are gated; other post_setup hooks (kittentts,
# piper, agent_browser, etc.) keep their existing behaviour. Add an
# entry when (a) the post_setup is the ONLY install side-effect for
# a no-key provider, and (b) an installed-state check is cheap and
# doesn't trigger a heavy import.
"cua_driver": lambda: bool(shutil.which("cua-driver")),
}
def _post_setup_already_installed(post_setup_key: str) -> bool:
"""Return True when the post_setup install side-effect is satisfied."""
predicate = _POST_SETUP_INSTALLED.get(post_setup_key)
if predicate is None:
# No install-state check registered → assume satisfied (don't
# change behaviour for hooks we haven't explicitly opted in).
return True
try:
return bool(predicate())
except Exception:
return True
def _toolset_needs_configuration_prompt(ts_key: str, config: dict) -> bool: def _toolset_needs_configuration_prompt(ts_key: str, config: dict) -> bool:
"""Return True when enabling this toolset should open provider setup.""" """Return True when enabling this toolset should open provider setup."""
cat = TOOL_CATEGORIES.get(ts_key) cat = TOOL_CATEGORIES.get(ts_key)
if not cat: if not cat:
return not _toolset_has_keys(ts_key, config) return not _toolset_has_keys(ts_key, config)
# If any visible provider has a registered post_setup install-state
# check that hasn't been satisfied (e.g. cua-driver binary not on
# PATH yet), force the configuration flow so `_configure_provider`
# invokes `_run_post_setup` and the install actually runs.
for provider in _visible_providers(cat, config):
post_setup = provider.get("post_setup")
if post_setup and not _post_setup_already_installed(post_setup):
return True
if ts_key == "tts": if ts_key == "tts":
tts_cfg = config.get("tts", {}) tts_cfg = config.get("tts", {})
return not isinstance(tts_cfg, dict) or "provider" not in tts_cfg return not isinstance(tts_cfg, dict) or "provider" not in tts_cfg

View file

@ -0,0 +1,71 @@
"""Tests for the post_setup install-state gate in `_toolset_needs_configuration_prompt`.
Regression coverage for the cua-driver silent-no-op bug (issue #22737).
When a no-key provider's only install side-effect is a `post_setup` hook
(cua-driver, etc.), the gate function used to fall through to the
`_toolset_has_keys` catch-all, which returned True for any provider with
empty `env_vars` causing `hermes tools` to write the toolset to config
and exit ` Saved` without ever invoking the post_setup install. These
tests pin the new predicate-aware behaviour so the regression doesn't
sneak back in.
"""
from __future__ import annotations
class TestPostSetupGate:
def test_cua_driver_missing_forces_setup(self, monkeypatch, tmp_path):
"""When cua-driver isn't on PATH, the gate must return True so the
provider-setup flow runs and triggers `_run_post_setup`."""
from hermes_cli import tools_config
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
monkeypatch.setattr(tools_config.shutil, "which", lambda name: None)
assert tools_config._toolset_needs_configuration_prompt(
"computer_use", {}
) is True
def test_cua_driver_installed_skips_setup(self, monkeypatch, tmp_path):
"""When cua-driver is already on PATH, the gate must return False
so a re-save through `hermes tools` doesn't re-prompt the user."""
from hermes_cli import tools_config
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
monkeypatch.setattr(
tools_config.shutil,
"which",
lambda name: "/usr/local/bin/cua-driver" if name == "cua-driver" else None,
)
assert tools_config._toolset_needs_configuration_prompt(
"computer_use", {}
) is False
def test_post_setup_predicate_exception_does_not_block(self, monkeypatch):
"""A predicate that raises must be treated as 'satisfied' so a
broken check can't strand the user in an infinite setup loop."""
from hermes_cli import tools_config
def _boom():
raise RuntimeError("predicate broken")
monkeypatch.setitem(tools_config._POST_SETUP_INSTALLED, "cua_driver", _boom)
assert tools_config._post_setup_already_installed("cua_driver") is True
def test_unregistered_post_setup_treated_as_satisfied(self):
"""post_setup keys without a registered predicate must default to
'satisfied' so we don't change behaviour for hooks we haven't
explicitly opted in (kittentts, piper, agent_browser, etc.)."""
from hermes_cli import tools_config
assert tools_config._post_setup_already_installed("does_not_exist") is True
def test_cua_driver_predicate_registered(self):
"""Keep an explicit pin on the cua_driver entry so accidental
deletion of the registry row would fail this test rather than
silently restore the original silent-no-op bug."""
from hermes_cli import tools_config
assert "cua_driver" in tools_config._POST_SETUP_INSTALLED

View file

@ -84,7 +84,9 @@ def cua_driver_binary_available() -> bool:
def cua_driver_install_hint() -> str: def cua_driver_install_hint() -> str:
return ( return (
"cua-driver is not installed. Install with:\n" "cua-driver is not installed. Install with one of:\n"
" hermes computer-use install\n"
"Or run the upstream installer directly:\n"
' /bin/bash -c "$(curl -fsSL ' ' /bin/bash -c "$(curl -fsSL '
'https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"\n' 'https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"\n'
"Or run `hermes tools` and enable the Computer Use toolset to install it automatically." "Or run `hermes tools` and enable the Computer Use toolset to install it automatically."

View file

@ -66,6 +66,7 @@ hermes [global-options] <command> [subcommand/options]
| `hermes mcp` | Manage MCP server configurations and run Hermes as an MCP server. | | `hermes mcp` | Manage MCP server configurations and run Hermes as an MCP server. |
| `hermes plugins` | Manage Hermes Agent plugins (install, enable, disable, remove). | | `hermes plugins` | Manage Hermes Agent plugins (install, enable, disable, remove). |
| `hermes tools` | Configure enabled tools per platform. | | `hermes tools` | Configure enabled tools per platform. |
| `hermes computer-use` | Install or check the cua-driver backend (macOS Computer Use). |
| `hermes sessions` | Browse, export, prune, rename, and delete sessions. | | `hermes sessions` | Browse, export, prune, rename, and delete sessions. |
| `hermes insights` | Show token/cost/activity analytics. | | `hermes insights` | Show token/cost/activity analytics. |
| `hermes fallback` | Interactive manager for the fallback provider chain. | | `hermes fallback` | Interactive manager for the fallback provider chain. |
@ -958,6 +959,26 @@ hermes tools [--summary]
Without `--summary`, this launches the interactive per-platform tool configuration UI. Without `--summary`, this launches the interactive per-platform tool configuration UI.
## `hermes computer-use`
```bash
hermes computer-use <subcommand>
```
Subcommands:
| Subcommand | Description |
|------------|-------------|
| `install` | Run the upstream cua-driver installer (macOS only). |
| `status` | Print whether `cua-driver` is on `$PATH`. |
`hermes computer-use install` is the stable entry point for installing the
[cua-driver](https://github.com/trycua/cua) binary used by the
`computer_use` toolset. It runs the same upstream installer that
`hermes tools` invokes when you first enable Computer Use, so it's safe
to use for re-running the install if the toolset toggle didn't trigger
it (for example, on returning-user setups).
## `hermes sessions` ## `hermes sessions`
```bash ```bash

View file

@ -27,9 +27,25 @@ cua-driver is the open-source equivalent.
## Enabling ## Enabling
Pick whichever path is most convenient — both run the same upstream installer:
**Option 1: dedicated CLI command (most direct).**
```
hermes computer-use install
```
This fetches and runs the upstream cua-driver installer:
`curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh`.
Use `hermes computer-use status` to verify the install.
**Option 2: enable the toolset interactively.**
1. Run `hermes tools`, pick `🖱️ Computer Use (macOS)``cua-driver (background)`. 1. Run `hermes tools`, pick `🖱️ Computer Use (macOS)``cua-driver (background)`.
2. The setup runs the upstream installer: 2. The setup runs the upstream installer (same as Option 1).
`curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh`.
After installing, regardless of which path you took:
3. Grant macOS permissions when prompted: 3. Grant macOS permissions when prompted:
- **System Settings → Privacy & Security → Accessibility** → allow the - **System Settings → Privacy & Security → Accessibility** → allow the
terminal (or Hermes app). terminal (or Hermes app).
@ -143,7 +159,8 @@ HERMES_COMPUTER_USE_BACKEND=noop # records calls, no side effects
## Troubleshooting ## Troubleshooting
**`computer_use backend unavailable: cua-driver is not installed`** — Run **`computer_use backend unavailable: cua-driver is not installed`** — Run
`hermes tools` and enable Computer Use. `hermes computer-use install` to fetch the cua-driver binary, or run
`hermes tools` and enable the Computer Use toolset.
**Clicks seem to have no effect** — Capture and verify. A modal you **Clicks seem to have no effect** — Capture and verify. A modal you
didn't see may be blocking input. Dismiss it with `escape` or the close didn't see may be blocking input. Dismiss it with `escape` or the close