mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-07-24 16:54:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

fix(skills/tldraw-offline): correct the computer-use delivery note

Browse source 
The skill claimed Chromium/Electron 'reject synthetic clicks' so computer-use
can't drive the canvas. The Cua team disproved this on the exact v1.11.0
AppImage (Linux/X11): background delivery returns background_unavailable, but
that's the first rung, not a wall — cua-driver returns escalation:'foreground'
and its X11 XTest path (x11_xtest_fg) with delivery_mode:'foreground' clicks
through, dismissing the consent dialog and landing canvas clicks.

Corrected the note to say: climb to foreground on background_unavailable, don't
conclude Electron is unclickable. Ref: NousResearch/hermes-agent#67052.
This commit is contained in:
teknium1 2026-07-18 12:29:35 -07:00 • committed by Teknium
parent e321b83392
commit 849c17752d
1 changed files with 10 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

15
optional-skills/creative/tldraw-offline/SKILL.md
View file

@ -137,11 +137,16 @@ that standard the counter is verified: dispatched clicks stepped it
logs `inotify_add_watch ... No space left on device`, `script-status` shows
`state: "not-watching"` / `hasEntry: false`, and the script never executes.
This is a host limit, not a script bug — a normal desktop is unaffected.
- **Real background GUI clicks (computer-use style) do NOT reach the canvas.**
Chromium/Electron reject synthetic pointer input to an unfocused/occluded
renderer, so `computer_use` no-focus automation can't click buttons. This is
irrelevant to the actual product path (agents use `/exec`, not clicks) — but
don't try to "verify" a script by having a background agent click the GUI.
- **Driving the canvas with computer-use needs FOREGROUND delivery, not
background.** cua-driver's default `background` delivery returns
`background_unavailable` against Electron's occluded renderer — but that is the
first rung, not a wall. cua-driver returns an `escalation: "foreground"` hint;
set `delivery_mode: "foreground"` (pair with `bring_to_front`) and its X11 XTest
path (`x11_xtest_fg`) clicks land — this dismisses the "Run Script" consent
dialog and clicks the canvas. Verified by the Cua team on this exact v1.11.0
AppImage (Linux/X11). Do NOT conclude "Electron rejects synthetic clicks" and
give up in background mode — climb to foreground. (The real product path is
still `/exec`, not clicks; this note is for computer-use-driven testing.)
The pattern: