mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-06-27 11:22:03 +00:00
docs(computer-use): document Windows UIPI elevated-window limitation (#51121)
A Medium-integrity Hermes agent cannot drive High-integrity (admin) windows on Windows — UIPI blocks UIA enumeration and mouse injection (SOM returns 0 elements, clicks silently no-op, screenshots still work, keyboard partially bypasses). OS constraint affecting every Windows automation stack, not a cua-driver bug. Document the symptom + the run-elevated workaround. Closes #49067.
This commit is contained in:
parent
5ecf3bf0e0
commit
351afd353d
1 changed files with 13 additions and 0 deletions
|
|
@ -255,6 +255,19 @@ of screenshot context, not ~600K.
|
|||
drawing (Logic, Final Cut, some games) have sparse or empty AX trees.
|
||||
Fall back to pixel coordinates if the tree is empty — or skip the
|
||||
task entirely.
|
||||
- **Windows: elevated (admin) windows can't be driven from a normal
|
||||
agent.** Windows UIPI (User Interface Privilege Isolation) enforces
|
||||
integrity-level boundaries: a Medium-integrity process (the default
|
||||
Hermes agent) cannot enumerate the UIA tree of, or inject mouse input
|
||||
into, a window owned by a High-integrity (Administrator) process.
|
||||
Symptom: `capture(mode='som')` returns 0 elements and `click(...)`
|
||||
reports success while doing nothing, even though the screenshot
|
||||
renders fine (GDI capture sits below the integrity check). Keyboard
|
||||
events partially bypass UIPI, so Tab / Enter can still navigate an
|
||||
elevated dialog. This is an OS constraint, not a cua-driver bug — it
|
||||
affects every Windows automation stack. To drive elevated windows,
|
||||
run the Hermes agent itself at High integrity (launch from an
|
||||
elevated terminal); otherwise target non-elevated windows.
|
||||
- **Platform-specific deployment gotchas:**
|
||||
- **macOS** uses private SkyLight SPIs. Apple can change them in any
|
||||
OS update. Hermes warns when the installed cua-driver is older than
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue