Siddharth Balyan c9a3f36f56 feat: add video_analyze tool for native video understanding (#19301) ... * feat: add video_analyze tool for native video understanding Adds a video_analyze tool that sends video files to multimodal LLMs (e.g. Gemini) for analysis via the OpenRouter-compatible video_url content type. Mirrors vision_analyze in structure, error handling, and registration pattern. Key design: - Base64 encodes entire video (no frame extraction, no ffmpeg dep) - Uses 'video_url' content block type (OpenRouter standard) - Supports mp4, webm, mov, avi, mkv, mpeg formats - 50 MB hard cap, 20 MB warning threshold - 180s minimum timeout (videos take longer than images) - AUXILIARY_VIDEO_MODEL env override, falls back to AUXILIARY_VISION_MODEL - Same SSRF protection, retry logic, and cleanup as vision_analyze Default disabled: registered in 'video' toolset (not in _HERMES_CORE_TOOLS). Users opt in via: hermes tools enable video, or enabled_toolsets=['video']. * feat(video): add models.dev capability pre-check + CONFIGURABLE_TOOLSETS entry - Pre-checks model video capability via models.dev modalities.input before expensive base64 encoding. Fails early with helpful message suggesting video-capable alternatives (gemini, mimo-v2.5-pro). - Passes optimistically if model unknown or lookup fails. - Adds ModelInfo.supports_video_input() helper. - Adds 'video' to CONFIGURABLE_TOOLSETS and _DEFAULT_OFF_TOOLSETS so 'hermes tools enable video' works from CLI. - 8 new tests for the capability check (37 total). * refactor(video): remove models.dev capability pre-check Removes _check_video_model_capability and ModelInfo.supports_video_input. The vision_analyze tool doesn't pre-check image capability either — both tools rely on the same pattern: send request, handle API errors gracefully with categorized user-facing messages. The pre-check was inconsistent (only worked for some providers/models) so drop it for parity. * cleanup: compress comments, fix fragile timeout coupling - Replace _VISION_DOWNLOAD_TIMEOUT * 2 with hardcoded 60s (no silent breakage if vision timeout changes independently) - Strip verbose comments and redundant log lines throughout - No behavioral changes		2026-05-04 00:04:36 +05:30
..
browser_providers	feat: ungate Tool Gateway — subscription-based access with per-tool opt-in	2026-04-16 12:36:49 -07:00
environments	fix(ci): stabilize current main test regressions	2026-04-30 06:36:50 -07:00
neutts_samples	refactor(tts): replace NeuTTS optional skill with built-in provider + setup flow	2026-03-17 02:33:12 -07:00
__init__.py	Merge branch 'main' into rewbs/tool-use-charge-to-subscription	2026-03-31 08:48:54 +09:00
ansi_strip.py	fix: strip ANSI at the source — clean terminal output before it reaches the model	2026-03-23 07:43:12 -07:00
approval.py	fix(approval): extend sensitive write target to cover shell RC and credential files	2026-05-03 08:49:13 -07:00
binary_extensions.py	fix(tools): address PR review — remove _extract_raw_output, BudgetConfig everywhere, read_file hardening	2026-04-08 02:24:32 -07:00
browser_camofox.py	refactor(config): add cfg_get() helper; migrate 20 nested-get call sites (#17304)	2026-04-28 23:17:39 -07:00
browser_camofox_state.py	feat(browser): add persistent Camofox sessions and VNC URL discovery (salvage #4400) (#4419)	2026-04-01 04:18:50 -07:00
browser_cdp_tool.py	chore: remove unused imports and dead locals (ruff F401, F841) (#17010)	2026-04-28 06:46:45 -07:00
browser_dialog_tool.py	feat(browser): CDP supervisor — dialog detection + response + cross-origin iframe eval (#14540)	2026-04-23 22:23:37 -07:00
browser_supervisor.py	fix(browser_supervisor): verify thread and loop health before returning cached supervisor	2026-04-30 20:33:33 -07:00
browser_tool.py	refactor(config): add cfg_get() helper; migrate 20 nested-get call sites (#17304)	2026-04-28 23:17:39 -07:00
budget_config.py	fix: preserve existing thresholds, remove pre-read byte guard	2026-04-08 02:24:32 -07:00
checkpoint_manager.py	feat(checkpoints): auto-prune orphan and stale shadow repos at startup (#16303)	2026-04-26 19:05:52 -07:00
clarify_tool.py	refactor: add tool_error/tool_result helpers + read_raw_config, migrate 129 callsites	2026-04-07 13:36:38 -07:00
code_execution_tool.py	fix(tools): serialize concurrent hermes_tools RPC calls from execute_code	2026-04-30 03:31:16 -07:00
credential_files.py	refactor(config): add cfg_get() helper; migrate 20 nested-get call sites (#17304)	2026-04-28 23:17:39 -07:00
cronjob_tools.py	fix(cron): accept list-form deliver values so deliver=['telegram'] works (#17456)	2026-04-29 06:35:34 -07:00
debug_helpers.py	refactor: codebase-wide lint cleanup — unused imports, dead code, and inefficient patterns (#5821)	2026-04-07 10:25:31 -07:00
delegate_tool.py	fix(delegate): honor runtime default model during provider resolution	2026-04-30 19:58:55 -07:00
discord_tool.py	fix(discord_tool): key capability cache by token instead of single global	2026-04-30 20:37:12 -07:00
env_passthrough.py	refactor(config): add cfg_get() helper; migrate 20 nested-get call sites (#17304)	2026-04-28 23:17:39 -07:00
feishu_doc_tool.py	fix(feishu-comment): use get_hermes_home(); drop dead asyncio wrapper; AUTHOR_MAP	2026-04-17 19:04:11 -07:00
feishu_drive_tool.py	fix(feishu-comment): use get_hermes_home(); drop dead asyncio wrapper; AUTHOR_MAP	2026-04-17 19:04:11 -07:00
file_operations.py	chore: remove unused imports and dead locals (ruff F401, F841) (#17010)	2026-04-28 06:46:45 -07:00
file_state.py	feat(delegate): cross-agent file state coordination for concurrent subagents (#13718)	2026-04-21 16:41:26 -07:00
file_tools.py	fix(tools): write_file handler now rejects missing 'content'/'path' args instead of silently writing zero-byte files (#19096)	2026-05-03 08:52:41 -07:00
fuzzy_match.py	fix(patch): gate 'did you mean?' to no-match + extend to v4a/skill_manage	2026-04-21 02:03:46 -07:00
homeassistant_tool.py	fix: clean up description escaping, add string-data tests	2026-04-13 04:45:07 -07:00
image_generation_tool.py	fix(image-gen): force-refresh plugin providers in long-lived sessions	2026-04-23 03:01:18 -07:00
interrupt.py	fix(interrupt): propagate to concurrent-tool workers + opt-in debug trace (#11907)	2026-04-17 20:39:25 -07:00
kanban_tools.py	feat(kanban): durable multi-profile collaboration board (#17805)	2026-04-30 13:36:47 -07:00
managed_tool_gateway.py	fix(tools): add debug logging for token refresh and tighten domain check	2026-04-02 12:40:03 +11:00
mcp_oauth.py	fix(mcp-oauth): preserve server_url path for protected-resource validation (#16031)	2026-04-26 05:43:54 -07:00
mcp_oauth_manager.py	fix(mcp-oauth): preserve server_url path for protected-resource validation (#16031)	2026-04-26 05:43:54 -07:00
mcp_tool.py	fix: clean up defensive shims and finish CI stabilization from #17660 (#17801)	2026-04-29 23:53:17 -07:00
memory_tool.py	refactor: consolidate symlink-safe atomic replace into shared helper	2026-04-28 04:58:22 -07:00
mixture_of_agents_tool.py	Fix (mixture_of_agents): replace deprecated Gemini model and forward max_tokens to OpenRouter (#6621)	2026-04-23 15:14:11 -07:00
neutts_synth.py	fix(tts): document NeuTTS provider and align install guidance (#1903)	2026-03-18 02:55:30 -07:00
openrouter_client.py	refactor: route ad-hoc LLM consumers through centralized provider router	2026-03-11 20:02:36 -07:00
osv_check.py	feat: OSV malware check for MCP extension packages (#5305)	2026-04-05 12:46:07 -07:00
patch_parser.py	fix(patch): gate 'did you mean?' to no-match + extend to v4a/skill_manage	2026-04-21 02:03:46 -07:00
path_security.py	refactor: extract shared helpers to deduplicate repeated code patterns (#7917)	2026-04-11 13:59:52 -07:00
process_registry.py	fix(process): reconcile session.exited against real child exit in poll/wait (#17430)	2026-04-29 04:59:21 -07:00
registry.py	perf(tools): memoize get_tool_definitions + TTL-cache check_fn results (#17098)	2026-04-28 18:20:17 -07:00
rl_training_tool.py	refactor: codebase-wide lint cleanup — unused imports, dead code, and inefficient patterns (#5821)	2026-04-07 10:25:31 -07:00
schema_sanitizer.py	refactor(schema): consolidate nullable-union stripping in schema_sanitizer	2026-04-28 04:58:03 -07:00
send_message_tool.py	feat(gateway/signal): add support for multiple images sending	2026-04-30 04:28:08 -07:00
session_search_tool.py	fix(session_search): order recent mode by last activity instead of start time	2026-04-30 20:17:15 -07:00
skill_manager_tool.py	fix(curator): authoritative absorbed_into on delete + restore cron skill links on rollback (#18671) (#18731)	2026-05-02 01:29:57 -07:00
skill_usage.py	fix: use skill activity in curator status	2026-04-30 10:31:47 -07:00
skills_guard.py	feat(skills-guard): gate agent-created scanner on config.skills.guard_agent_created (default off)	2026-04-23 06:20:47 -07:00
skills_hub.py	feat(skills): install skills from a direct HTTP(S) URL (#16323)	2026-04-26 20:57:10 -07:00
skills_sync.py	refactor: consolidate symlink-safe atomic replace into shared helper	2026-04-28 04:58:22 -07:00
skills_tool.py	fix(skills): also bump_use on skill_view tool invocation	2026-04-30 05:07:34 -07:00
slash_confirm.py	feat(gateway,cli): confirm /reload-mcp to warn about prompt cache invalidation	2026-04-29 21:56:47 -07:00
terminal_tool.py	fix(terminal): skip sudo prompt when local NOPASSWD sudo works	2026-04-30 20:38:09 -07:00
tirith_security.py	fix: guard against None tirith path in security scanner	2026-04-23 03:08:53 -07:00
todo_tool.py	fix(tools): enforce ID uniqueness in TODO store during replace operations	2026-04-11 16:22:50 -07:00
tool_backend_helpers.py	fix(cli): coerce use_gateway config flags in tool routing	2026-04-26 19:02:55 -07:00
tool_output_limits.py	feat(skills): add design-md skill for Google's DESIGN.md spec (#14876)	2026-04-23 21:51:19 -07:00
tool_result_storage.py	fix(tools): neutralize shell injection in _write_to_sandbox via path quoting (#7940)	2026-04-11 14:26:11 -07:00
transcription_tools.py	fix(ci): stabilize main test suite regressions (#17660)	2026-04-29 23:18:55 -07:00
tts_tool.py	feat(tts): add Piper as a native local TTS provider (closes #8508) (#17885)	2026-04-30 02:53:20 -07:00
url_safety.py	fix(security): treat quoted false as false in browser SSRF guards	2026-04-26 18:27:13 -07:00
vision_tools.py	feat: add video_analyze tool for native video understanding (#19301)	2026-05-04 00:04:36 +05:30
voice_mode.py	fix: point optional-dep install hints at the venv's python (#11938)	2026-04-17 21:16:33 -07:00
web_tools.py	perf(tools): memoize get_tool_definitions + TTL-cache check_fn results (#17098)	2026-04-28 18:20:17 -07:00
website_policy.py	refactor: codebase-wide lint cleanup — unused imports, dead code, and inefficient patterns (#5821)	2026-04-07 10:25:31 -07:00
xai_http.py	feat(xai): upgrade to Responses API, add TTS provider	2026-04-16 02:24:08 -07:00
yuanbao_tools.py	chore: remove unused imports and dead locals (ruff F401, F841) (#17010)	2026-04-28 06:46:45 -07:00