mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-27 17:58:07 +00:00
e869accc1a
Deferred MCP/plugin tools become invisible once the tool_search bridge activates — live benchmarking (48 runs, Claude Haiku 4.5) showed models substituting visible core tools (terminal/web_search/browser) for deferred capabilities or declaring them nonexistent instead of searching: 16/24 task success vs 24/24 with eager loading. Skills never had this failure mode because every skill keeps a ~21-token name+description listing line in the system prompt. This ports that exact pattern to the tool bridge: when tool_search activates, a grouped manifest of every deferred tool (name + first sentence of description, clipped to 60 chars, grouped per MCP server / toolset) is embedded in the tool_search bridge description. - tools/tool_search.py: build_catalog_listing() with deterministic ordering (byte-stable across assemblies -> prompt prefix stays cacheable); token-budget fallbacks full -> names-only -> legacy bare count; bridge_tool_schemas(listing=...) embeds it and instructs the model to skip tool_search when the exact name is visible (one fewer round-trip per use) - config: tools.tool_search.listing auto|on|off (default auto), listing_max_tokens (default 4000, clamped 200..20000); legacy bool shapes keep working - tests: 8 new tests (config parsing/clamps, short-desc clipping, deterministic rendering, budget fallbacks, bridge embedding, assembly on/off paths); full file green (47 passed) - docs: tool-search.md config table + rationale - scripts/tool_search_livetest2.py: benchmark harness v2 with real per-call token accounting (normalize_usage spy) and a third 'listing' mode for A/B/C comparison |
||
|---|---|---|
| .. | ||
| ci | ||
| lib | ||
| tests | ||
| whatsapp-bridge | ||
| add_contributor.py | ||
| analyze_livetest.py | ||
| benchmark_browser_eval.py | ||
| build_model_catalog.py | ||
| build_skills_index.py | ||
| capture-cage-terminal.sh | ||
| check-windows-footguns.py | ||
| check_subprocess_stdin.py | ||
| contributor_audit.py | ||
| dev-sandbox.sh | ||
| discord-voice-doctor.py | ||
| docker_config_migrate.py | ||
| docker_rebootstrap_nous_session.py | ||
| generate_conformance_vectors.py | ||
| hermes-gateway | ||
| install.cmd | ||
| install.ps1 | ||
| install.sh | ||
| install_psutil_android.py | ||
| iso-certify.py | ||
| keystroke_diagnostic.py | ||
| kill_modal.sh | ||
| lint_diff.py | ||
| LIVETEST_README.md | ||
| profile-tui.py | ||
| release.py | ||
| run_tests.sh | ||
| run_tests_parallel.py | ||
| sample_and_compress.py | ||
| tool_search_livetest.py | ||
| tool_search_livetest2.py | ||