hermes-agent/tools/computer_use
Jeff 52cf24e441
fix(tools): downscale computer_use captures before aux-vision routing
Full-resolution desktop captures (1920x1032+) tokenize to thousands
of vision tokens and overflow small local vision models' context
windows - the aux call came back "the vision API rejected the image"
and the model got no description at all. Cap the long side at 1456px
before writing the temp image for vision_analyze: SOM badges stay
legible, the request fits comfortably, and per-capture vision latency
drops roughly in half. Also drop the hardcoded "macOS" from the
describe prompt now that captures come from Windows hosts too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 06:19:20 -07:00
..
__init__.py feat(computer-use): cua-driver backend, universal any-model schema 2026-05-08 11:07:38 -07:00
backend.py fix(computer-use): add set_value to ComputerUseBackend ABC and _NoopBackend stub 2026-05-22 01:14:15 -07:00
cua_backend.py fix(computer_use): honor custom vision routing 2026-06-07 02:09:20 -07:00
overlay.py feat(tools): on-screen overlay for Windows computer_use 2026-06-15 06:19:20 -07:00
schema.py feat(tools): add Windows UIA backend for computer_use 2026-06-15 06:19:20 -07:00
tool.py fix(tools): downscale computer_use captures before aux-vision routing 2026-06-15 06:19:20 -07:00
vision_routing.py fix(computer_use): honor custom vision routing 2026-06-07 02:09:20 -07:00
windows_backend.py feat(tools): on-screen overlay for Windows computer_use 2026-06-15 06:19:20 -07:00