Commit graph

2 commits

Author SHA1 Message Date
teknium1
4ddb443c12
fix(lint): explicit encoding on write_text in livetest harness (PLW1514) 2026-07-22 05:16:30 -07:00
teknium1
3ee3394907
feat(tools): skills-style catalog listing for tool_search progressive disclosure
Deferred MCP/plugin tools become invisible once the tool_search bridge
activates — live benchmarking (48 runs, Claude Haiku 4.5) showed models
substituting visible core tools (terminal/web_search/browser) for deferred
capabilities or declaring them nonexistent instead of searching: 16/24
task success vs 24/24 with eager loading.

Skills never had this failure mode because every skill keeps a ~21-token
name+description listing line in the system prompt. This ports that exact
pattern to the tool bridge: when tool_search activates, a grouped manifest
of every deferred tool (name + first sentence of description, clipped to
60 chars, grouped per MCP server / toolset) is embedded in the tool_search
bridge description.

- tools/tool_search.py: build_catalog_listing() with deterministic
  ordering (byte-stable across assemblies -> prompt prefix stays
  cacheable); token-budget fallbacks full -> names-only -> legacy bare
  count; bridge_tool_schemas(listing=...) embeds it and instructs the
  model to skip tool_search when the exact name is visible (one fewer
  round-trip per use)
- config: tools.tool_search.listing auto|on|off (default auto),
  listing_max_tokens (default 4000, clamped 200..20000); legacy bool
  shapes keep working
- tests: 8 new tests (config parsing/clamps, short-desc clipping,
  deterministic rendering, budget fallbacks, bridge embedding, assembly
  on/off paths); full file green (47 passed)
- docs: tool-search.md config table + rationale
- scripts/tool_search_livetest2.py: benchmark harness v2 with real
  per-call token accounting (normalize_usage spy) and a third 'listing'
  mode for A/B/C comparison
2026-07-22 05:16:30 -07:00