feat(tool_search): probe-validate blind tool_call args against the deferred schema

Port from nearai/ironclaw#5149 (the describe-first live-hardening fix in
their progressive tool disclosure work): when a model invokes a deferred
tool through the tool_call bridge without the schema-required arguments,
return the tool's parameter schema instead of dispatching blind.

Pre-fix, a blind call produced an opaque downstream failure
("[TOOL_ERROR] Tool execution failed: KeyError: 'document_id'") that
teaches the model nothing about what the tool expects — IronClaw observed
cheap models looping ~30 identical invalid calls until the iteration
budget died. Post-fix, the model repairs the call in one round-trip.

- tools/tool_search.py: new validate_deferred_call_args() — key-absence
  check of schema 'required' fields only; no type checking (coerce_tool_args
  already repairs types downstream); fails open on any validator error so
  it can never block a legitimate dispatch.
- model_tools.py: probe after the scope gate in the bridge dispatch.
- agent/tool_executor.py: probe in both unwrap sites (concurrent +
  sequential) before the underlying tool replaces the bridge; sequential
  path flattens the payload to match its {"error": str} wrapping.
- tests: TestDeferredCallSchemaProbe — blind call returns schema (not
  KeyError), valid/optional calls dispatch, unvalidatable tools fail open,
  out-of-scope rejection unchanged.
This commit is contained in:
teknium1 2026-07-05 17:30:17 -07:00 committed by Teknium
parent 9b97dea1e6
commit 8fbe2e388f
4 changed files with 199 additions and 4 deletions

View file

@ -934,6 +934,61 @@ def scoped_deferrable_names(tool_defs: List[Dict[str, Any]]) -> frozenset[str]:
return frozenset(names)
def validate_deferred_call_args(name: str, args: Dict[str, Any]) -> Optional[str]:
"""Probe-validate ``tool_call`` arguments against the deferred tool's schema.
A deferred tool's parameter schema is invisible to the model until it
calls ``tool_describe`` so models routinely invoke deferred tools
"blind" by name alone, omitting required arguments. Dispatching such a
call produces an opaque downstream failure (``KeyError: 'document_id'``)
that tells the model nothing about what the tool expects, and cheap
models loop on it until the iteration budget dies.
Port of the describe-first probe-validation fix from nearai/ironclaw#5149:
when required arguments are missing, return the tool's parameter schema
instead of dispatching blind the model repairs the call in one
round-trip. Valid calls (and any call we can't confidently validate)
dispatch untouched, so this can never block a legitimate invocation.
Only *key absence* of schema-``required`` fields counts as invalid.
No type checking, no null rejection nullable/typed edge cases are the
tool's own business, and ``coerce_tool_args`` already handles type repair
downstream. Returns a JSON error string when invalid, ``None`` when the
call should dispatch.
"""
try:
from tools.registry import registry as _registry
schema = _registry.get_schema(name)
if not isinstance(schema, dict):
return None
fn = schema.get("function") if schema.get("type") == "function" else schema
if not isinstance(fn, dict):
return None
params = fn.get("parameters")
if not isinstance(params, dict):
return None
required = params.get("required")
if not isinstance(required, list) or not required:
return None
missing = [r for r in required if isinstance(r, str) and r not in args]
if not missing:
return None
return json.dumps({
"error": (
f"tool_call to '{name}' is missing required argument(s): "
f"{', '.join(missing)}. The tool was NOT invoked."
),
"parameters": params,
"hint": (
"Retry tool_call with 'arguments' matching the parameters "
"schema above."
),
}, ensure_ascii=False)
except Exception: # pragma: no cover — never block dispatch on validator bugs
logger.debug("validate_deferred_call_args failed for %s", name, exc_info=True)
return None
def resolve_underlying_call(args: Dict[str, Any]) -> Tuple[Optional[str], Dict[str, Any], Optional[str]]:
"""Parse a ``tool_call`` invocation into (underlying_name, args, error_msg).
@ -992,4 +1047,5 @@ __all__ = [
"dispatch_tool_describe",
"resolve_underlying_call",
"scoped_deferrable_names",
"validate_deferred_call_args",
]