hermes-agent/tools/computer_use
Teknium 30e5d0092d
feat(computer-use): add whole-screen/desktop capture target
capture(app='screen'|'desktop') now resolves to the OS shell/desktop
window (Windows Progman/WorkerW desktop or Shell_TrayWnd taskbar, macOS
Finder/Dock) so 'show me my screen' and 'click the taskbar' work.
Previously capture() only matched application windows, and the schema
advertised 'or the whole screen' without any code path delivering it.

cua-driver is window-oriented (no virtual-desktop or per-monitor MCP
tool), so a single image still cannot span multiple monitors — the
schema now states this and the no-desktop-window path returns a clear
message instead of silently grabbing the frontmost app.
2026-06-22 12:21:58 -07:00
..
__init__.py feat(computer-use): cua-driver backend, universal any-model schema 2026-05-08 11:07:38 -07:00
backend.py feat(computer_use): cross-platform cua-driver (macOS/Windows/Linux) 2026-06-22 06:42:30 -07:00
cua_backend.py feat(computer-use): add whole-screen/desktop capture target 2026-06-22 12:21:58 -07:00
doctor.py feat(computer_use): disable cua-driver telemetry by default, add opt-in (#50842) 2026-06-22 09:57:16 -07:00
schema.py feat(computer-use): add whole-screen/desktop capture target 2026-06-22 12:21:58 -07:00
tool.py fix(computer_use): reconcile Linux gate with stale "gated off" comments 2026-06-22 06:42:30 -07:00
vision_routing.py fix(computer_use): honor custom vision routing 2026-06-07 02:09:20 -07:00