feat(computer_use): cross-platform cua-driver (macOS/Windows/Linux)

Make the computer_use toolset platform-agnostic by driving cua-driver on
macOS, Windows, and Linux. Consumes the 8 cua-driver decoupling surfaces
(capability discovery, structuredContent AX tree, opaque element_token,
click button enum, explicit mimeType, machine-readable manifest,
structured list_windows, structured health_report), each degrading
gracefully on older drivers.

Adds `hermes computer-use doctor` (drives cua-driver health_report with a
per-OS check matrix and an exit 0/1/2 ok/degraded/blocked contract), full
typed wrappers for the previously-uncovered cua-driver tools plus a generic
call_tool escape hatch, per-session agent-cursor lifecycle, platform-aware
system-prompt guidance (host-deterministic, cache-safe), and honors
HERMES_CUA_DRIVER_CMD end-to-end.

Replaces the macOS-only skills/apple/macos-computer-use skill with a
cross-platform skills/computer-use skill, and refreshes the EN + zh-Hans
docs.

Supersedes #44221 (Windows-enablement salvage of #30660).

Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
This commit is contained in:
Francesco Bonacci 2026-06-21 20:04:05 -07:00 committed by Teknium
parent 17dfc6bec4
commit f2e37549c6
22 changed files with 4130 additions and 657 deletions

View file

@ -47,6 +47,7 @@ ACP_REGISTRY_MANIFEST = REPO_ROOT / "acp_registry" / "agent.json"
AUTHOR_MAP = {
"21178861+ScotterMonk@users.noreply.github.com": "ScotterMonk", # PR #50145 salvage (cron output truncation: adapter-aware chunking, #50126)
"rrandqua@gmail.com": "TutkuEroglu", # PR #50481 salvage (AGENTS.md stale token-lock adapter path)
"f@trycua.com": "f-trycua", # PR #50507 salvage (cross-platform computer_use; supersedes #44221/#30660)
"pedro.m.simoes@gmail.com": "pmos69", # PR #29474 salvage (native Antigravity OAuth provider; Gemini CLI sunset #29294/#49701)
"mediratta01.pally@gmail.com": "orbisai0security", # PR #9560 salvage (session.py path-traversal guard, V-009)
"panghuer023@users.noreply.github.com": "panghuer023", # PR #37994 salvage (interrupt unblocks pending gateway approval; #8697)