hermes-agent/tests/tools
Dylan Socolobsky c6dba918b3
fix(tests): fix several failing/flaky tests on main (#6777)
* fix(tests): mock is_safe_url in tests that use example.com

Tests using example.com URLs were failing because is_safe_url does a real DNS lookup which fails in environments where example.com doesn't resolve, causing the request to be blocked before reaching the already-mocked HTTP client. This should fix around 17 failing tests.

These tests test logic, caching, etc. so mocking this method should not modify them in any way. TestMattermostSendUrlAsFile was already doing this so we follow the same pattern.

* fix(test): use case-insensitive lookup for model context length check

DEFAULT_CONTEXT_LENGTHS uses inconsistent casing (MiniMax keys are lowercase, Qwen keys are mixed-case) so the test was broken in some cases since it couldn't find the model.

* fix(test): patch is_linux in systemd gateway restart test

The test only patched is_macos to False but didn't patch is_linux to True. On macOS hosts, is_linux() returns False and the systemd restart code path is skipped entirely, making the assertion fail.

* fix(test): use non-blocklisted env var in docker forward_env tests

GITHUB_TOKEN is in api_key_env_vars and thus in _HERMES_PROVIDER_ENV_BLOCKLIST so the env var is silently dropped, we replace it with a non-blocked one like DATABASE_URL so the tests actually work.

* fix(test): fully isolate _has_any_provider_configured from host env

_has_any_provider_configured() checks all env vars from PROVIDER_REGISTRY (not just the 5 the tests were clearing) and also calls get_auth_status() which detects gh auth token for Copilot. On machines with any of these set, the function returns True before reaching the code path under test.

Clear all registry vars and mock get_auth_status so host credentials don't interfere.

* fix(test): correct path to hermes_base_env.py in tool parser tests

Path(__file__).parent.parent resolved to tests/, not the project root.
The file lives at environments/hermes_base_env.py so we need one more parent level.

* fix(test): accept optional HTML fields in Matrix send payload

_send_matrix sometimes adds format and formatted_body when the markdown library is installed. The test was doing an exact dict equality check which broke. Check required fields instead.

* fix(test): add config.yaml to codex vision requirements test

The test only wrote auth.json but not config.yaml, so _read_main_provider() returned empty and vision auto-detect never tried the codex provider. Add a config.yaml pointing at openai-codex so the fallback path actually resolves the client.

* fix(test): clear OPENROUTER_API_KEY in _isolate_hermes_home

run_agent.py calls load_hermes_dotenv() at import time, which injects API keys from ~/.hermes/.env into os.environ before any test fixture runs. This caused test_agent_loop_tool_calling to make real API calls instead of skipping, which ends up making some tests fail.

* fix(test): add get_rate_limit_state to agent mock in usage report tests

_show_usage now calls agent.get_rate_limit_state() for rate limit
  display. The SimpleNamespace mock was missing this method.

* fix(test): update expected Camofox config version from 12 to 13

* fix(test): mock _get_enabled_platforms in nous managed defaults test

Importing gateway.run leaks DISCORD_BOT_TOKEN into os.environ, which makes _get_enabled_platforms() return ["cli", "discord"] instead of just ["cli"]. tools_command loops per platform, so apply_nous_managed_defaults
  runs twice: the first call sets config values, the second sees them as
  already configured and returns an empty set, causing the assertion to
  fail.
2026-04-09 13:17:06 -07:00
..
__init__.py test: reorganize test structure and add missing unit tests 2026-02-26 03:20:08 +03:00
test_ansi_strip.py fix: strip ANSI at the source — clean terminal output before it reaches the model 2026-03-23 07:43:12 -07:00
test_approval.py fix(gateway): isolate approval session key per turn 2026-04-03 17:50:01 -07:00
test_base_environment.py feat(environments): unified spawn-per-call execution layer 2026-04-08 17:23:15 -07:00
test_browser_camofox.py fix: URL-based auth for third-party Anthropic endpoints + CI test fixes (#4148) 2026-03-30 20:36:56 -07:00
test_browser_camofox_persistence.py fix(doctor): only check the active memory provider, not all providers unconditionally (#6285) 2026-04-08 13:44:58 -07:00
test_browser_camofox_state.py fix(tests): fix several failing/flaky tests on main (#6777) 2026-04-09 13:17:06 -07:00
test_browser_cdp_override.py feat: switch managed browser provider from Browserbase to Browser Use (#5750) 2026-04-07 08:40:22 -04:00
test_browser_cleanup.py fix(doctor): only check the active memory provider, not all providers unconditionally (#6285) 2026-04-08 13:44:58 -07:00
test_browser_console.py fix: add browser_console to browser toolset and core tools list (#1084) 2026-03-17 02:02:57 -07:00
test_browser_content_none_guard.py fix(browser): guard LLM response content against None in snapshot and vision (#3642) 2026-03-28 17:25:04 -07:00
test_browser_homebrew_paths.py fix(browser): preserve agent-browser paths with spaces 2026-04-08 02:35:48 -07:00
test_browser_secret_exfil.py fix: rewrite test mock secrets and add redaction fixture 2026-04-01 12:03:56 -07:00
test_browser_ssrf_local.py fix(browser): skip SSRF check for local backends (Camofox, headless Chromium) (#4292) 2026-03-31 10:40:13 -07:00
test_checkpoint_manager.py fix: reduce file tool log noise 2026-03-13 22:14:00 -07:00
test_clarify_tool.py test(tools): add unit tests for clarify_tool.py 2026-02-27 03:29:26 -05:00
test_clipboard.py feat(clipboard): add native Windows image paste support 2026-04-07 12:49:39 -07:00
test_code_execution.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_command_guards.py fix: make tirith block verdicts approvable instead of hard-blocking (#3428) 2026-03-27 13:22:01 -07:00
test_config_null_guard.py fix: guard config.get() against YAML null values to prevent AttributeError (#3377) 2026-03-27 04:03:00 -07:00
test_credential_files.py feat: wire skills.external_dirs into all remaining discovery paths 2026-04-03 21:14:42 -07:00
test_cron_prompt_injection.py fix: cron prompt injection scanner bypass for multi-word variants 2026-02-26 13:55:54 +03:00
test_cronjob_tools.py fix(tools): remove unnecessary crontab requirement from cronjob tool (#1638) 2026-03-17 01:40:02 -07:00
test_daytona_environment.py fix: update tests for unified spawn-per-call execution model 2026-04-08 17:23:15 -07:00
test_debug_helpers.py fix(tests): isolate HERMES_HOME in tests and adjust log directory for debug session 2026-03-02 04:34:21 -08:00
test_delegate.py fix(delegate): share credential pools with subagents + per-task leasing 2026-04-06 23:01:11 -07:00
test_delegate_toolset_scope.py fix(security): restrict subagent toolsets to parent's enabled set (#3269) 2026-03-26 14:50:26 -07:00
test_docker_environment.py fix(tests): fix several failing/flaky tests on main (#6777) 2026-04-09 13:17:06 -07:00
test_docker_find.py fix: Docker backend fails when docker is not in PATH (macOS gateway) 2026-03-10 20:45:13 -07:00
test_env_passthrough.py feat: env var passthrough for skills and user config (#2807) 2026-03-24 08:19:34 -07:00
test_file_operations.py fix: search_files now reports error for non-existent paths instead of silent empty results 2026-03-08 16:47:20 -07:00
test_file_read_guards.py feat(file_tools): harden read_file with size guard, dedup, and device blocking (#4315) 2026-03-31 12:53:19 -07:00
test_file_staleness.py fix(file_tools): refresh staleness timestamp after writes (#4390) 2026-04-01 00:50:08 -07:00
test_file_tools.py fix: strip ANSI at the source — clean terminal output before it reaches the model 2026-03-23 07:43:12 -07:00
test_file_tools_live.py feat(environments): unified spawn-per-call execution layer 2026-04-08 17:23:15 -07:00
test_file_write_safety.py fix(security): harden terminal safety and sandbox file writes (#1653) 2026-03-17 02:22:12 -07:00
test_force_dangerous_override.py fix(skills): honor policy table for dangerous verdicts 2026-03-14 11:27:02 -07:00
test_fuzzy_match.py test: reorganize test structure and add missing unit tests 2026-02-26 03:20:08 +03:00
test_hidden_dir_filter.py fix: use Path.parts for hidden directory filter in skill listing 2026-03-04 18:34:16 +03:00
test_homeassistant_tool.py fix: add service domain blocklist and entity_id validation to HA tools 2026-03-01 11:53:50 +03:00
test_interrupt.py feat: concurrent tool execution with ThreadPoolExecutor 2026-03-13 02:51:51 -07:00
test_llm_content_none_guard.py fix: guard aux LLM calls against None content + reasoning fallback + retry (salvage #3389) (#3449) 2026-03-27 15:28:19 -07:00
test_local_env_blocklist.py fix: add macOS Homebrew paths to browser and terminal PATH resolution 2026-03-23 22:45:55 -07:00
test_managed_browserbase_and_modal.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_managed_media_gateways.py Gate tool-gateway behind an env var, so it's not in users' faces until we're ready. Even if users enable it, it'll be blocked server-side for now, until we unlock for non-admin users on tool-gateway. 2026-03-30 13:28:10 +09:00
test_managed_modal_environment.py feat(environments): unified spawn-per-call execution layer 2026-04-08 17:23:15 -07:00
test_managed_server_tool_support.py fix(tests): fix several failing/flaky tests on main (#6777) 2026-04-09 13:17:06 -07:00
test_managed_tool_gateway.py feat: switch managed browser provider from Browserbase to Browser Use (#5750) 2026-04-07 08:40:22 -04:00
test_mcp_dynamic_discovery.py feat(mcp): dynamic tool discovery via notifications/tools/list_changed (#3812) 2026-03-29 15:52:54 -07:00
test_mcp_oauth.py feat: implement MCP OAuth 2.1 PKCE client support (#5420) 2026-04-05 22:08:00 -07:00
test_mcp_probe.py fix: remove stale test skips, fix regex backtracking, file search bug, and test flakiness 2026-04-04 10:18:57 -07:00
test_mcp_stability.py fix(mcp): stability fix pack — reload timeout, shutdown cleanup, event loop handler, OAuth non-blocking (#4757) 2026-04-03 02:29:20 -07:00
test_mcp_structured_content.py fix: use camelCase structuredContent attr, prefer structured over text 2026-04-07 18:00:01 -07:00
test_mcp_tool.py fix(gateway): normalize step_callback prev_tools for backward compat 2026-04-02 20:54:27 -07:00
test_mcp_tool_issue_948.py fix: remove stale test skips, fix regex backtracking, file search bug, and test flakiness 2026-04-04 10:18:57 -07:00
test_memory_tool.py fix(memory): profile-scoped memory isolation and clone support (#4845) 2026-04-03 13:10:11 -07:00
test_mixture_of_agents_tool.py refactor: tighten MoA traceback logging scope (#1307) 2026-03-14 07:53:56 -07:00
test_modal_sandbox_fixes.py fix: update tests for unified spawn-per-call execution model 2026-04-08 17:23:15 -07:00
test_modal_snapshot_isolation.py fix: update tests for unified spawn-per-call execution model 2026-04-08 17:23:15 -07:00
test_notify_on_complete.py fix(process): correct detached crash recovery state 2026-04-08 03:35:43 -07:00
test_osv_check.py feat: OSV malware check for MCP extension packages (#5305) 2026-04-05 12:46:07 -07:00
test_parse_env_var.py fix(docker): add explicit env allowlist for container credentials (#1436) 2026-03-17 02:34:35 -07:00
test_patch_parser.py fix: handle addition-only hunks in V4A patch parser (#3325) 2026-03-26 19:38:04 -07:00
test_process_registry.py fix(process): correct detached crash recovery state 2026-04-08 03:35:43 -07:00
test_read_loop_detection.py fix: remove post-compression file-read history injection (#2226) 2026-03-20 14:54:25 -07:00
test_registry.py perf(ttft): salvage easy-win startup optimizations from #3346 (#3395) 2026-03-27 07:49:44 -07:00
test_rl_training_tool.py fix: call _stop_training_run on early-return failure paths 2026-03-10 17:09:51 -07:00
test_search_hidden_dirs.py fix: exclude hidden directories from find/grep search backends (#1558) 2026-03-17 02:02:57 -07:00
test_send_message_missing_platforms.py fix(tests): fix several failing/flaky tests on main (#6777) 2026-04-09 13:17:06 -07:00
test_send_message_tool.py fix(tools): redact query secrets in send_message errors 2026-04-06 16:49:52 -07:00
test_session_search.py feat(sessions): add --source flag for third-party session isolation (#3255) 2026-03-26 14:35:31 -07:00
test_singularity_preflight.py fix(tests): use case-insensitive regex in singularity preflight tests 2026-03-16 19:01:39 +03:00
test_skill_env_passthrough.py fix(skills): stop marking persisted env vars missing on remote backends (#3650) 2026-03-28 17:52:32 -07:00
test_skill_improvements.py feat(skills): size limits for agent writes + fuzzy matching for patch (#4414) 2026-04-01 04:19:19 -07:00
test_skill_manager_tool.py fix: repair 57 failing CI tests across 14 files (#5823) 2026-04-07 09:58:45 -07:00
test_skill_size_limits.py feat(skills): size limits for agent writes + fuzzy matching for patch (#4414) 2026-04-01 04:19:19 -07:00
test_skill_view_path_check.py refactor: use Path.is_relative_to() for skill_view boundary check 2026-03-04 05:30:43 -08:00
test_skill_view_traversal.py fix(security): block path traversal in skill_view file_path (fixes #220) 2026-03-02 02:00:09 -08:00
test_skills_guard.py fix(skills): preserve trust for skills-sh identifiers + reduce resolution churn (#3251) 2026-03-26 13:40:21 -07:00
test_skills_hub.py fix(skills): validate hub bundle paths before install (#3986) 2026-03-30 08:37:19 -07:00
test_skills_hub_clawhub.py fix: improve clawhub skill search matching 2026-03-14 23:15:04 -07:00
test_skills_sync.py feat: nix flake — uv2nix build, NixOS module, persistent container mode (#20) 2026-03-26 01:08:02 +05:30
test_skills_tool.py fix(skills): stop marking persisted env vars missing on remote backends (#3650) 2026-03-28 17:52:32 -07:00
test_ssh_environment.py fix: update tests for unified spawn-per-call execution model 2026-04-08 17:23:15 -07:00
test_symlink_prefix_confusion.py fix: use is_relative_to() for symlink boundary check in skills_guard 2026-03-04 17:23:23 +03:00
test_terminal_disk_usage.py fix(terminal): log disk warning check failures at debug level (salvage #2372) (#2394) 2026-03-21 17:10:17 -07:00
test_terminal_exit_semantics.py feat: add exit code context for common CLI tools in terminal results (#5144) 2026-04-04 16:57:24 -07:00
test_terminal_none_command_guard.py fix(terminal): guard invalid command values 2026-04-08 21:37:51 -07:00
test_terminal_requirements.py Merge branch 'main' into rewbs/tool-use-charge-to-subscription 2026-03-31 08:48:54 +09:00
test_terminal_timeout_output.py fix(terminal): preserve partial output when command times out (#3868) 2026-03-29 21:51:44 -07:00
test_terminal_tool.py fix: handle empty sudo password and false prompts 2026-04-09 02:50:07 -07:00
test_terminal_tool_requirements.py Gate tool-gateway behind an env var, so it's not in users' faces until we're ready. Even if users enable it, it'll be blocked server-side for now, until we unlock for non-admin users on tool-gateway. 2026-03-30 13:28:10 +09:00
test_threaded_process_handle.py feat(environments): unified spawn-per-call execution layer 2026-04-08 17:23:15 -07:00
test_tirith_security.py fix: send_animation metadata, MarkdownV2 inline code splitting, tirith cosign-free install (#1626) 2026-03-16 23:39:41 -07:00
test_todo_tool.py fix: update test_non_empty_has_markers to match todo filtering behavior 2026-03-08 23:07:38 +03:00
test_tool_call_parsers.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_tool_result_storage.py fix: preserve existing thresholds, remove pre-read byte guard 2026-04-08 02:24:32 -07:00
test_transcription.py feat(auth): same-provider credential pools with rotation, custom endpoint support, and interactive CLI (#2647) 2026-03-31 03:10:01 -07:00
test_transcription_tools.py fix(tools): use correct import path for mistralai SDK 2026-04-08 13:47:08 -07:00
test_url_safety.py fix(security): add SSRF protection to vision_tools and web_tools (hardened) 2026-03-23 15:40:42 -07:00
test_vision_tools.py fix(tests): fix several failing/flaky tests on main (#6777) 2026-04-09 13:17:06 -07:00
test_voice_cli_integration.py fix: voice pipeline hardening — 7 bug fixes with tests 2026-03-14 14:27:21 +03:00
test_voice_mode.py fix: allow voice mode in WSL when PulseAudio bridge is configured 2026-03-31 12:13:33 -07:00
test_web_tools_config.py Merge branch 'main' into rewbs/tool-use-charge-to-subscription 2026-03-31 08:48:54 +09:00
test_web_tools_tavily.py fix(tests): fix several failing/flaky tests on main (#6777) 2026-04-09 13:17:06 -07:00
test_website_policy.py fix: resolve 7 failing CI tests (#3936) 2026-03-30 08:10:14 -07:00
test_windows_compat.py fix: guard POSIX-only process functions for Windows compatibility 2026-03-01 01:54:27 +03:00
test_write_deny.py fix: resolve symlink bypass in write deny list on macOS 2026-02-26 13:30:55 +03:00
test_yolo_mode.py fix(security): harden terminal safety and sandbox file writes (#1653) 2026-03-17 02:22:12 -07:00