docs(computer-use): document Windows UIPI elevated-window limitation (#51121)

A Medium-integrity Hermes agent cannot drive High-integrity (admin)
windows on Windows — UIPI blocks UIA enumeration and mouse injection
(SOM returns 0 elements, clicks silently no-op, screenshots still work,
keyboard partially bypasses). OS constraint affecting every Windows
automation stack, not a cua-driver bug. Document the symptom + the
run-elevated workaround. Closes #49067.
This commit is contained in:
Teknium 2026-06-23 08:41:33 -07:00 committed by GitHub
parent 5ecf3bf0e0
commit 351afd353d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -255,6 +255,19 @@ of screenshot context, not ~600K.
drawing (Logic, Final Cut, some games) have sparse or empty AX trees.
Fall back to pixel coordinates if the tree is empty — or skip the
task entirely.
- **Windows: elevated (admin) windows can't be driven from a normal
agent.** Windows UIPI (User Interface Privilege Isolation) enforces
integrity-level boundaries: a Medium-integrity process (the default
Hermes agent) cannot enumerate the UIA tree of, or inject mouse input
into, a window owned by a High-integrity (Administrator) process.
Symptom: `capture(mode='som')` returns 0 elements and `click(...)`
reports success while doing nothing, even though the screenshot
renders fine (GDI capture sits below the integrity check). Keyboard
events partially bypass UIPI, so Tab / Enter can still navigate an
elevated dialog. This is an OS constraint, not a cua-driver bug — it
affects every Windows automation stack. To drive elevated windows,
run the Hermes agent itself at High integrity (launch from an
elevated terminal); otherwise target non-elevated windows.
- **Platform-specific deployment gotchas:**
- **macOS** uses private SkyLight SPIs. Apple can change them in any
OS update. Hermes warns when the installed cua-driver is older than