cua_driver_update_check now short-circuits to None when no driver
resolves; CI has none installed, so pin resolve_cua_driver_cmd in the
update-check and env-sanitization tests.
Cut steady-state Computer Use latency without changing default behavior
or waiting on cua-driver:
- Cap screenshots via set_config(max_image_dimension) on session start
(config: computer_use.max_image_dimension, default 1456)
- Cache aux-vision routing per (provider, model) so captures skip
repeated load_config()
- Add computer_use.capture_after_mode (default som) so users can opt
follow-ups down to ax (elements only) for speed
Auto-detect now disables the cursor overlay on darwin as well as
headless/WSL2 Linux. After start_session, also call
set_agent_cursor_enabled(false) when the policy is on so older drivers
without --no-overlay still tear the overlay down.
Co-authored-by: David Metcalfe <80915+DavidMetcalfe@users.noreply.github.com>
The hermes-sweeper review #4701565902 (2026-07-15) flagged two
consistency issues in `_cua_driver_supports_no_overlay` and one
additive-config concern:
1. `cua-backend.py:260` — the `cua-driver --help` support probe
inherited the full parent environment. cua-driver is a third-party
binary; every other spawn site in this file (manifest probe at
`:214`, MCP spawn at `:697`, install probe at `:997`) uses
`_sanitize_subprocess_env(cua_driver_child_env())`. The `--help`
probe should match. This was a low-impact leak (only help output
exits), but inconsistency is the wrong default for a third-party
subprocess.
2. `cua_backend.py:238` — when the manifest returned a `command`
different from the input `driver_cmd` parameter (e.g. a relocated
executable at `/opt/relocated/cua-driver` while the system binary
is at `/usr/bin/cua-driver`), the support probe ran against
`_CUA_DRIVER_CMD` (the default) instead of the manifest-discovered
`command`. Two failure modes:
- The wrapper binary supports `--no-overlay` but the system binary
doesn't → probe returns False → overlay kept despite capability.
- The system binary supports `--no-overlay` but the wrapper doesn't
→ probe returns True → MCP spawn crashes on the unknown flag.
3. The original commit bumped `_config_version` 31→32 for an additive
default (`computer_use.no_overlay: None`). AGENTS.md specifies that
additive defaults in existing sections are handled by deep merge
and should NOT trigger a version bump. After cherry-picking onto
current `origin/main` (which is already at 33), the bump is
effectively dropped — resolved to main's 33.
Changes:
- Add `env=_sanitize_subprocess_env(cua_driver_child_env())` to the
`--help` subprocess (with the same import + rationale comment as
the manifest probe).
- Pass `driver_cmd=command` (or `driver_cmd=driver_cmd` for the
fallback path) into `_mcp_args_with_overlay_flag`, so the support
probe runs against the binary that will actually be launched.
Tests (3 new):
- `test_help_probe_passes_sanitized_env` — verifies `subprocess.run`
is called with an `env=` kwarg.
- `test_manifest_command_drives_support_probe` — verifies the probe
runs against the manifest command when it differs from the input
driver_cmd.
- `test_fallback_uses_input_driver_cmd_for_support_probe` — verifies
the fallback path (no command in manifest) uses the input
driver_cmd.
- `test_probe_distinguishes_support_between_binaries` — sanity check
that the lru_cache key on `driver_cmd` prevents cross-binary
cache leakage.
File-revert negative test confirmed all three of the new
"manifest/probe" tests are load-bearing: with the pre-fix code, they
fail (probe runs against the default binary instead of the resolved
one); with the fix, they pass. 20/20 tests in
`tests/computer_use/test_cua_no_overlay.py` green.
`TestMcpInvocationResolution` (8/8) still green.
Refs: sweeper review #4701565902
Address review feedback from cross-vendor review (Flash + GPT-OSS):
1. Auto-detect now checks for headless Linux (no DISPLAY), WSL2
(/proc/version contains 'microsoft'), instead of all Linux.
Desktop Linux with a compositor keeps the overlay.
2. Add _cua_driver_supports_no_overlay() that probes cua-driver --help
to check if the flag is supported. Older drivers (< 0.6.x) reject
unknown flags, so passing --no-overlay would crash the MCP spawn.
3. Update tests to cover headless vs desktop Linux, WSL2 detection,
version probe, and the unsupported-driver fallback path.
- Add autouse fixture to TestMcpInvocationResolution to disable
--no-overlay flag so existing tests assert baseline args
- Make test_config_load_failure_fails_safe and test_missing_section_enables
platform-aware (Linux auto-detect returns True, macOS/Windows False)
cua-driver's cursor overlay rendering loop can consume CPU indefinitely
when idle (#28152, #47032). On Linux/WSL2, the overlay serves no visual
purpose and the rendering path is the primary source of idle CPU usage.
Add computer_use.no_overlay config option (default: auto-detect) that
passes --no-overlay to cua-driver when enabled. Auto-detection disables
the overlay on Linux (covers WSL2, headless, containers) where it has no
benefit, and keeps it enabled on macOS/Windows where it is visually
useful.
Refs: #28152, #47032
PR #58889 fixed the CLI-fallback transport; review of that fix found the
same leak class at four sibling spawn sites of the third-party cua-driver
binary:
- _resolve_mcp_invocation (cua-driver manifest): no env= at all — full
parent environment inherited
- cua_driver_update_check (check-update --json): telemetry env but no
secret sanitization
- doctor._drive_health_report (<binary> mcp Popen): telemetry env only
- permissions._run (every macOS/Linux permission probe): telemetry env only
All now route through _sanitize_subprocess_env(cua_driver_child_env()),
matching the sanctioned MCP spawn and the #53503/#55709/#58889 strip-by-
default policy for non-terminal spawns. Sanitization degrades gracefully
(falls back to the telemetry env) so doctor/permission probes never break
on an import error.
4 regression tests covering each site.
_CuaDriverSession._call_tool_via_cli() (the EAGAIN/silent-empty MCP
fallback transport) invokes `cua-driver call <tool> <json>` via
subprocess.run() with no env= argument, so the third-party cua-driver
binary inherits the full, unsanitized parent environment. The primary
MCP spawn site (_lifecycle_coro) already applies
_sanitize_subprocess_env(cua_driver_child_env()) before opening the
stdio client, per the same policy #53503/#55709 established for other
subprocess spawn points — this fallback path, added alongside the
EAGAIN/silent-empty-capture hardening, missed it.
Fix: apply the same env=_sanitize_subprocess_env(cua_driver_child_env())
to the subprocess.run() call in _call_tool_via_cli(), mirroring the
sanctioned spawn site exactly (telemetry policy applied first, then
Hermes-managed secrets filtered).
* feat(computer_use): disable cua-driver telemetry by default, add opt-in
cua-driver ships anonymous PostHog usage telemetry ENABLED by default
upstream (fires cua_driver_install / cua_driver_doctor events to
eu.i.posthog.com). Hermes now disables it for our users unless they
explicitly opt in.
- New config key `computer_use.cua_telemetry` (default false) in
DEFAULT_CONFIG.
- `cua_backend.cua_driver_child_env()` injects
`CUA_DRIVER_RS_TELEMETRY_ENABLED=0` into the child env when telemetry is
disabled (the default); leaves the var untouched on opt-in so the driver
uses its own default. Reads config fail-safe — any error defaults to
telemetry off.
- Routed every cua-driver spawn site through the policy: MCP backend
(StdioServerParameters env), `cua_driver_update_check`, doctor's
health_report Popen, the install.sh/install.ps1 runner, and the
`--version` / status probes.
- Docs: new Telemetry subsection in computer-use.md (EN).
- Tests: tests/computer_use/test_cua_telemetry.py — default disables,
explicit-false disables, opt-in leaves var untouched, config-failure
fails safe, inherited-enabled is overridden off.
Verified live on Linux against the real cua-driver-rs 0.6.0 binary: with
the var=0 the driver reports "telemetry: disabled via
CUA_DRIVER_RS_TELEMETRY_ENABLED" and sends no event; with it unset it logs
"sending event: cua_driver_doctor". 213 computer_use + install tests green.
* fix(dashboard): fold computer_use config category into agent tab
The new computer_use.cua_telemetry key created a single-field dashboard
config category, tripping test_no_single_field_categories (web_server's
invariant that categories with <2 fields must be merged to avoid tab
sprawl). Add computer_use -> agent to _CATEGORY_MERGE, matching the
existing onboarding/telegram single-field folds.
Make the computer_use toolset platform-agnostic by driving cua-driver on
macOS, Windows, and Linux. Consumes the 8 cua-driver decoupling surfaces
(capability discovery, structuredContent AX tree, opaque element_token,
click button enum, explicit mimeType, machine-readable manifest,
structured list_windows, structured health_report), each degrading
gracefully on older drivers.
Adds `hermes computer-use doctor` (drives cua-driver health_report with a
per-OS check matrix and an exit 0/1/2 ok/degraded/blocked contract), full
typed wrappers for the previously-uncovered cua-driver tools plus a generic
call_tool escape hatch, per-session agent-cursor lifecycle, platform-aware
system-prompt guidance (host-deterministic, cache-safe), and honors
HERMES_CUA_DRIVER_CMD end-to-end.
Replaces the macOS-only skills/apple/macos-computer-use skill with a
cross-platform skills/computer-use skill, and refreshes the EN + zh-Hans
docs.
Supersedes #44221 (Windows-enablement salvage of #30660).
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
