hermes-agent/tools/computer_use
briandevans bb694bad42 fix(computer-use): cap AX elements array to prevent context blowup (#22865)
`computer_use(action='capture', mode='ax')` returned the full AX element
list verbatim in the JSON response. Dense Electron / Obsidian / JetBrains
UIs publish 500+ AX nodes (one reproduction in #22865 returned 597
elements against Obsidian), so a single capture could consume enough
context to trigger compression failures or render the session unusable.
The human-readable `_format_elements` summary is already capped at 40
lines, so the truncation gap was invisible to anyone reading the summary
output.

Add a `max_elements` argument to the tool schema, default 100, that
trims the AX `elements` array. When the cap fires, the response surfaces
`total_elements` and `truncated_elements` and appends a "raise
max_elements or pass app= to narrow" hint to the summary so the model
knows the JSON view is partial and can re-issue with a tighter scope.

Validation is centralized in `_coerce_max_elements`: missing /
non-integer / sub-1 inputs fall back to the default cap, so the
protection can never be silently disabled by a malformed tool-call
argument. The cap only affects AX-mode JSON; `mode='som'` and
`mode='vision'` keep returning a screenshot + image-aware summary
unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 19:07:32 -07:00
..
__init__.py feat(computer-use): cua-driver backend, universal any-model schema 2026-05-08 11:07:38 -07:00
backend.py feat(computer-use): cua-driver backend, universal any-model schema 2026-05-08 11:07:38 -07:00
cua_backend.py fix(computer-use): surface app=… filter no-match instead of silently using frontmost (#24170 bug 1) 2026-05-21 17:15:35 -07:00
schema.py fix(computer-use): cap AX elements array to prevent context blowup (#22865) 2026-05-21 19:07:32 -07:00
tool.py fix(computer-use): cap AX elements array to prevent context blowup (#22865) 2026-05-21 19:07:32 -07:00
vision_routing.py fix(computer_use): add helper to decide capture vision routing 2026-05-21 17:38:19 -07:00