mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
48be2e0e4d
* ci(tests): install ripgrep from prebuilt tarball instead of apt
apt-get update + install of ripgrep takes ~4 min on the GHA Ubuntu
runners (the apt-get update against archive.ubuntu.com is the slow
part; ripgrep itself is small). Switching to the upstream musl
binary tarball cuts the step to a few seconds.
- Pinned to ripgrep 15.1.0 with sha256 verification (same hash as
published in the releases sha256 sidecar file).
- Drops the `rg` binary into /usr/local/bin so it is on PATH for
every subsequent step without GITHUB_PATH manipulation.
- Applied to both the test and e2e jobs in tests.yml.
* fix(cli): compile syntax check to tempdir, not source __pycache__
`_validate_critical_files_syntax` runs `py_compile.compile()` on each
critical bootstrap file after a successful `git pull`. The default
`py_compile` writes the resulting `.pyc` next to the source under
`__pycache__/`, which causes two real problems:
1. Parallel test workers walking the same source tree (e.g. running
the suite under per-file process isolation) can race against each
other on the `__pycache__` write — manifests as flaky 'directory
not empty' errors during teardown.
2. In production, the post-pull syntax check leaves a `.pyc` behind
that the next interpreter run might pick up — fine when the
interpreter version matches, sketchy if it doesn't.
Fix: write the compiled output to a `tempfile.TemporaryDirectory()`
that's discarded on function exit. We only care about the compile-or-not
signal, not the artifact.
* test(runner): per-file process isolation, drop manual state reset + xdist
Replace fragile manual _reset_module_state test fixtures with robust
per-file subprocess isolation. Each test file runs in a fresh
`python -m pytest <file>` subprocess via ThreadPoolExecutor. No xdist,
no custom pytest plugin, no shared worker state.
Key changes:
* scripts/run_tests_parallel.py — new runner: discovers test files,
runs N in parallel via ThreadPoolExecutor, captures stdout per file,
treats exit code 5 (no tests collected) as pass, kills all children
on exit. Change from cpu_count to cpu_count*2. The runner is
I/O-bound (waiting on subprocess.communicate() from pytest children)
The parent process does almost no CPU work, so 2x oversubscription
keeps more pipes full. When a file fails, immediately show the last
30 lines of pytest output (stack traces + FAILED summary) plus a
ready-to-copy repro command:
python -m pytest tests/agent/test_auxiliary_client.py
* scripts/run_tests.sh — delegates to run_tests_parallel.py
* .github/workflows/tests.yml — test step: python
scripts/run_tests_parallel.py
* pyproject.toml — drop pytest-xdist, pytest-split; simplify addopts
* tests/conftest.py — remove ~200 lines of manual state-reset fixtures
* AGENTS.md — update Testing section for per-file design
* test(runner): speed gateway test antipattern scan up
* fix(test): web search provider plugin test missing xai
* fix(tests): make 14 test files pass under per-file subprocess isolation
Tests that relied on cross-file state pollution from xdist workers
fail when run in isolation (per-file subprocess model). Root causes
and fixes:
Tool registry not populated:
- test_video_generation_tool_surface_matrix: add discover_builtin_tools()
- test_web_providers_brave_free/ddgs/searxng/general: autouse fixtures
registering all 8 bundled web providers, reset after each test
- test_website_policy: same provider registration pattern
- test_web_tools_tavily: same pattern across 3 dispatch test classes
- Also add is_safe_url/check_website_access mocks where SSRF check
blocks example.com (DNS resolution fails in isolated envs)
Stale check_fn cache:
- test_kanban_tools: invalidate_check_fn_cache() + _clear_tool_defs_cache()
in both kanban guidance tests (prior test cached False for kanban_show)
- test_discord_tool: cache invalidation in setup/teardown
- test_homeassistant_tool: invalidate_check_fn_cache() before registry queries
Module-level state pollution:
- test_auxiliary_client: autouse fixture clearing _aux_unhealthy_until cache
- test_skill_commands: set_session_vars() instead of patch.dict(os.environ)
(ContextVar takes precedence over os.environ)
- test_dm_topics: overwrite sys.modules + separate telegram.constants mock
+ force-reimport of gateway.platforms.telegram
- test_terminal_tool_requirements: removed duplicate class declaration,
autouse _clear_caches fixture
* change(tests): run_tests.sh explicitly includes env vars
instead of manually dropping some vars, now we just only include some
* fix(tests): 5 more isolation/NixOS fixes
- test_approval_plugin_hooks: isolate HERMES_HOME so real user's
command_allowlist doesn't short-circuit the approval path
- test_google_chat: skipif when Platform.GOOGLE_CHAT not in enum
(feature not merged on this branch)
- test_write_deny: test systemd prefix against tmp_path instead of
/etc/systemd which resolves to /nix/store on NixOS
- test_pty_bridge: use shutil.which('cat') instead of /bin/cat
(doesn't exist on NixOS)
- profiles.py: rmtree onexc handler chmod's parent dirs too, fixing
profile deletion when copytree preserved read-only modes from
nix store
* fix(tests): clear unhealthy cache in autouse fixture for auxiliary_client
* fix(tests): skip send_message when telegram not installed; handle missing worker_id in browser_supervisor
* fix: py3.11 rmtree onexc compat + belt-and-suspenders unhealthy cache clear for expired codex test
* fix: address PR #29016 review feedback
- Remove tracked .pytest-cache/ artifact and add to .gitignore
- Fix stale 'xdist worker' comment in conftest.py
- Deduplicate web provider registration into tests/tools/conftest.py
shared helper (register_all_web_providers), replacing 8 copy-pasted
blocks across 6 test files
- Update PR description: remove stale recovered-test-files claim,
fix worker count to match code (cpu_count*2)
* fix: eliminate race in stale-cache achievements test
The background scan thread could complete and overwrite _SNAPSHOT_CACHE
before evaluate_all() returned the stale data — only 10 fake sessions
made the scan finish instantly. Added scan_delay param to _FakeSessionDB
and set it to 2s in the stale-cache test so the background thread can't
win the race.
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| conftest.py | ||
| test_accretion_caps.py | ||
| test_ansi_strip.py | ||
| test_approval.py | ||
| test_approval_heartbeat.py | ||
| test_approval_plugin_hooks.py | ||
| test_base_environment.py | ||
| test_browser_camofox.py | ||
| test_browser_camofox_persistence.py | ||
| test_browser_camofox_state.py | ||
| test_browser_cdp_override.py | ||
| test_browser_cdp_tool.py | ||
| test_browser_chromium_check.py | ||
| test_browser_cleanup.py | ||
| test_browser_cloud_fallback.py | ||
| test_browser_cloud_provider_cache.py | ||
| test_browser_console.py | ||
| test_browser_content_none_guard.py | ||
| test_browser_eval_supervisor_path.py | ||
| test_browser_hardening.py | ||
| test_browser_homebrew_paths.py | ||
| test_browser_hybrid_routing.py | ||
| test_browser_lightpanda.py | ||
| test_browser_orphan_reaper.py | ||
| test_browser_secret_exfil.py | ||
| test_browser_ssrf_local.py | ||
| test_browser_supervisor.py | ||
| test_browser_supervisor_healthcheck.py | ||
| test_budget_config.py | ||
| test_checkpoint_manager.py | ||
| test_clarify_gateway.py | ||
| test_clarify_tool.py | ||
| test_clipboard.py | ||
| test_code_execution.py | ||
| test_code_execution_modes.py | ||
| test_code_execution_windows_env.py | ||
| test_command_guards.py | ||
| test_computer_use.py | ||
| test_config_null_guard.py | ||
| test_credential_files.py | ||
| test_credential_pool_env_fallback.py | ||
| test_cron_approval_mode.py | ||
| test_cron_prompt_injection.py | ||
| test_cronjob_tools.py | ||
| test_daytona_environment.py | ||
| test_debug_helpers.py | ||
| test_delegate.py | ||
| test_delegate_composite_toolsets.py | ||
| test_delegate_subagent_timeout_diagnostic.py | ||
| test_delegate_toolset_scope.py | ||
| test_discord_tool.py | ||
| test_docker_environment.py | ||
| test_docker_find.py | ||
| test_dockerfile_node_modules_perms.py | ||
| test_dockerfile_pid1_reaping.py | ||
| test_env_passthrough.py | ||
| test_feishu_tools.py | ||
| test_file_operations.py | ||
| test_file_operations_edge_cases.py | ||
| test_file_ops_cwd_tracking.py | ||
| test_file_read_guards.py | ||
| test_file_staleness.py | ||
| test_file_state_registry.py | ||
| test_file_sync.py | ||
| test_file_sync_back.py | ||
| test_file_sync_perf.py | ||
| test_file_tools.py | ||
| test_file_tools_container_config.py | ||
| test_file_tools_live.py | ||
| test_file_write_safety.py | ||
| test_force_dangerous_override.py | ||
| test_fuzzy_match.py | ||
| test_hardline_blocklist.py | ||
| test_heartbeat_stale_thresholds.py | ||
| test_hidden_dir_filter.py | ||
| test_homeassistant_tool.py | ||
| test_image_generation.py | ||
| test_image_generation_env.py | ||
| test_image_generation_plugin_dispatch.py | ||
| test_init_session_cwd_respect.py | ||
| test_interrupt.py | ||
| test_kanban_codex_lane_skill.py | ||
| test_kanban_tools.py | ||
| test_lazy_deps.py | ||
| test_llm_content_none_guard.py | ||
| test_local_background_child_hang.py | ||
| test_local_env_blocklist.py | ||
| test_local_env_cwd_recovery.py | ||
| test_local_env_windows_msys.py | ||
| test_local_interrupt_cleanup.py | ||
| test_local_shell_init.py | ||
| test_local_tempdir.py | ||
| test_managed_browserbase_and_modal.py | ||
| test_managed_media_gateways.py | ||
| test_managed_modal_environment.py | ||
| test_managed_tool_gateway.py | ||
| test_mcp_cancelled_error_propagation.py | ||
| test_mcp_circuit_breaker.py | ||
| test_mcp_dynamic_discovery.py | ||
| test_mcp_empty_error_message.py | ||
| test_mcp_image_content.py | ||
| test_mcp_invalid_url.py | ||
| test_mcp_oauth.py | ||
| test_mcp_oauth_bidirectional.py | ||
| test_mcp_oauth_cold_load_expiry.py | ||
| test_mcp_oauth_integration.py | ||
| test_mcp_oauth_manager.py | ||
| test_mcp_oauth_metadata.py | ||
| test_mcp_probe.py | ||
| test_mcp_reconnect_signal.py | ||
| test_mcp_sse_transport.py | ||
| test_mcp_stability.py | ||
| test_mcp_structured_content.py | ||
| test_mcp_tool.py | ||
| test_mcp_tool_401_handling.py | ||
| test_mcp_tool_issue_948.py | ||
| test_mcp_tool_session_expired.py | ||
| test_mcp_utility_capability_gating.py | ||
| test_memory_tool.py | ||
| test_memory_tool_import_fallback.py | ||
| test_memory_tool_schema.py | ||
| test_microsoft_graph_auth.py | ||
| test_microsoft_graph_client.py | ||
| test_mixture_of_agents_tool.py | ||
| test_modal_bulk_upload.py | ||
| test_modal_sandbox_fixes.py | ||
| test_modal_snapshot_isolation.py | ||
| test_notify_on_complete.py | ||
| test_osv_check.py | ||
| test_parse_env_var.py | ||
| test_patch_parser.py | ||
| test_process_registry.py | ||
| test_read_loop_detection.py | ||
| test_registry.py | ||
| test_resolve_path.py | ||
| test_schema_sanitizer.py | ||
| test_search_hidden_dirs.py | ||
| test_send_message_missing_platforms.py | ||
| test_send_message_telegram_proxy.py | ||
| test_send_message_tool.py | ||
| test_session_search.py | ||
| test_shared_container_task_id.py | ||
| test_signal_media.py | ||
| test_singularity_preflight.py | ||
| test_skill_env_passthrough.py | ||
| test_skill_improvements.py | ||
| test_skill_manager_tool.py | ||
| test_skill_provenance.py | ||
| test_skill_size_limits.py | ||
| test_skill_usage.py | ||
| test_skill_view_path_check.py | ||
| test_skill_view_traversal.py | ||
| test_skills_guard.py | ||
| test_skills_hub.py | ||
| test_skills_hub_browse_sh.py | ||
| test_skills_hub_clawhub.py | ||
| test_skills_sync.py | ||
| test_skills_tool.py | ||
| test_slash_confirm.py | ||
| test_spotify_client.py | ||
| test_ssh_bulk_upload.py | ||
| test_ssh_environment.py | ||
| test_symlink_prefix_confusion.py | ||
| test_sync_back_backends.py | ||
| test_terminal_compound_background.py | ||
| test_terminal_config_env_sync.py | ||
| test_terminal_exit_semantics.py | ||
| test_terminal_foreground_timeout_cap.py | ||
| test_terminal_none_command_guard.py | ||
| test_terminal_output_transform_hook.py | ||
| test_terminal_requirements.py | ||
| test_terminal_task_cwd.py | ||
| test_terminal_timeout_output.py | ||
| test_terminal_tool.py | ||
| test_terminal_tool_pty_fallback.py | ||
| test_terminal_tool_requirements.py | ||
| test_threaded_process_handle.py | ||
| test_tirith_security.py | ||
| test_todo_tool.py | ||
| test_tool_backend_helpers.py | ||
| test_tool_output_limits.py | ||
| test_tool_result_storage.py | ||
| test_transcription.py | ||
| test_transcription_dotenv_fallback.py | ||
| test_transcription_tools.py | ||
| test_tts_command_providers.py | ||
| test_tts_dotenv_fallback.py | ||
| test_tts_gemini.py | ||
| test_tts_kittentts.py | ||
| test_tts_max_text_length.py | ||
| test_tts_mistral.py | ||
| test_tts_opus_routing.py | ||
| test_tts_piper.py | ||
| test_tts_speed.py | ||
| test_tts_xai_speech_tags.py | ||
| test_url_safety.py | ||
| test_vercel_sandbox_environment.py | ||
| test_video_analyze.py | ||
| test_video_generation_dispatch.py | ||
| test_video_generation_dynamic_schema.py | ||
| test_video_generation_tool_surface_matrix.py | ||
| test_vision_native_fast_path.py | ||
| test_vision_tools.py | ||
| test_voice_cli_integration.py | ||
| test_voice_mode.py | ||
| test_watch_patterns.py | ||
| test_web_providers.py | ||
| test_web_providers_brave_free.py | ||
| test_web_providers_ddgs.py | ||
| test_web_providers_searxng.py | ||
| test_web_providers_xai.py | ||
| test_web_tools_config.py | ||
| test_web_tools_tavily.py | ||
| test_website_policy.py | ||
| test_windows_compat.py | ||
| test_windows_native_support.py | ||
| test_write_deny.py | ||
| test_x_search_tool.py | ||
| test_yolo_mode.py | ||
| test_zombie_process_cleanup.py | ||