Commit graph

15 commits

Author SHA1 Message Date
Teknium
3913ac2ba0
test: pair load_config_readonly stubs at pruned-file sites (post-merge with main's readonly sweep)
Main's 243c9182b1/a16fd675df/7142dc4580 added load_config_readonly
sibling stubs across 38 files; our pruned versions of 11 of those files
kept only the load_config stubs. Re-applied the pairing at every
surviving site (26 patch()/setattr sites) — same return_value/
side_effect as the adjacent load_config stub. 494 tests green across
the 11 files.
2026-07-29 15:37:24 -07:00
teknium1
7142dc4580 test: complete the readonly-stub pairing sweep (6 more files)
Final pass: per_model_threshold_init_ordering, memory_provider_init,
plugin_context_engine_init, api_max_retries_config,
invalid_context_length_warning, tool_call_guardrail_runtime all stub
agent_init config reads that now resolve through load_config_readonly().
Verified by running the complete 59-file suspect list (every test file
that stubs load_config in string or attribute form and intersects the
swapped modules): 3,063 tests, 0 failed.
2026-07-29 15:28:15 -07:00
Teknium
7c5a98d888
Merge remote-tracking branch 'origin/main' into tests/prune-low-value
# Conflicts:
#	tests/run_agent/test_conversation_fallback_state.py
2026-07-29 15:24:14 -07:00
Soju06
ce9f6712ff refactor(agent): remove the inter-tool delay
The 1.0s sleep between sequential tool calls has been present since the
initial commit with no documented rationale. It sleeps between local
tool executions — the next LLM request goes out only after the whole
batch — so it rate-limits nothing, and the parallel read-only path
already runs with no delay. Every multi-tool turn pays (N-1) seconds
of dead time. Remove the sleep, the internal tool_delay plumbing, and
dead test assignments. AIAgent.__init__ keeps tool_delay as a
deprecated no-op keyword for one release so existing programmatic
callers construct cleanly; passing it emits a DeprecationWarning.
2026-07-29 15:21:26 -07:00
Teknium
6b81590c55
test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions
Systematic prune per AGENTS.md test policy, one pass over every major
test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli,
cron, tui_gateway, honcho/openviking, root-level):

- DELETE: source-reading tests (read_text/getsource on prod files),
  change-detector tests (exact catalog counts, model-name snapshots,
  config version literals), mock-echo tests (assert a mock returns what
  it was told), assertion-free/trivial tests, near-duplicate
  parametrizations (boundaries + one representative kept), async/sync
  twin duplicates, cosmetic within-file variations.
- KEEP (mandatory): security/redaction/approval guards, message-role
  alternation invariants, prompt-caching/deterministic-call-id
  invariants, issue-number regression tests (deduped), E2E tests.
- 6 test files deleted outright (script-style/no-assert or fully
  redundant); conftest.py, fakes/, fixtures/ untouched.
- tests/acp/conftest.py added: autouse fixture stubs the live
  models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server
  tests performed on every session create — test_server.py 147s → 3.4s,
  and the tests are now genuinely hermetic.
- Sleep-based slowness shrunk where safe (codex_ttfb_watchdog,
  compression_concurrent_fork, etc.); no wall-clock assertion tightened.

Verification: full hermetic suite via scripts/run_tests.sh —
2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall
(baseline: 583s wall, 13,564s subprocess CPU).
2026-07-29 13:10:23 -07:00
Alex Fournier
14bed44c8c Reapply "feat(observability): integrate NeMo Relay runtime and shared metrics"
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-27 21:10:51 -07:00
Jeffrey Quesnelle
841a5a744a
Revert "feat(observability): integrate NeMo Relay runtime and shared metrics" 2026-07-27 22:28:08 -04:00
Alex Fournier
9bc521b138 refactor(runtime): manage Relay LLM tool and subagent execution
Signed-off-by: Alex Fournier <afournier@nvidia.com>
2026-07-19 08:57:38 -04:00
doncazper
36308f0667 feat(plugins): pass approve rule keys to approval gate 2026-07-07 15:14:30 -07:00
kshitijk4poor
74cc9ee3f0 Revert "Merge pull request #58698 from kshitijk4poor/feat/pre-tool-call-approve-escalation"
This reverts commit 368e5f197e, reversing
changes made to abf9638f4e.
2026-07-06 02:36:52 +05:30
kshitijk4poor
f512d6f020 feat(plugins): pre_tool_call approve action escalates to human gate
Extend the pre_tool_call plugin hook return contract with a new directive:

    {"action": "approve", "message": "why this needs human confirmation"}

Previously a pre_tool_call hook could only veto a tool call (action: block)
or allow it silently. It could not escalate to the existing human-approval
flow. This unlocks user-defined runtime approval rules on ANY tool (HTTP
writes, file writes to sensitive paths, email sends), enforced at runtime —
resolving #51221 as a pure plugin, with no core approval.py rule schema.

Mechanism:
- get_pre_tool_call_directive() returns (action, message) for block|approve;
  get_pre_tool_call_block_message() kept as a block-only back-compat shim.
- resolve_pre_tool_block() is the single dispatch-site chokepoint: fetches
  the directive and, for approve, invokes the human gate; fail-closed to a
  block on denial, timeout, or gate exception. ALL FOUR tool-dispatch sites
  now call it: tool_executor (concurrent + sequential), agent_runtime_helpers,
  and model_tools.handle_function_call.
- request_tool_approval() escalates via the SAME machinery as Tier-2
  dangerous commands: session/permanent allowlist, prompt_dangerous_approval
  (CLI) / submit_pending (gateway), [o]nce/[s]ession/[a]lways/[d]eny,
  timeout fail-closed, approvals.cron_mode for cron contexts.

Architecture: extracted the shared decision core into _run_approval_gate(),
called by BOTH check_dangerous_command() and request_tool_approval() so the
fail-closed / cron / gateway / yolo / persist policy lives in ONE place and
cannot drift. Fixed a latent divergence — the plugin path now honors --yolo.

Approval grain: [a]lways is keyed on tool_name + a hash of the reason (an
explicit plugin rule_key overrides), so distinct reasons on the same tool
persist independently instead of one 'always' blanketing the whole tool.

Non-interactive: cron honors approvals.cron_mode (parity with commands); any
other non-interactive non-gateway context fails CLOSED for the plugin path
(the command path keeps its historical fail-open default, unchanged).

No new config schema, no new env vars, no new hook events.
2026-07-05 12:48:11 +05:30
Teknium
186bf25cb1 test(guardrail): assert halt message reaches stream_delta_callback
Regression guard for #30770 — verifies the guardrail-halt branch in
agent/conversation_loop.py pushes the synthesized halt message through
stream_delta_callback before breaking out of the loop.  Without the
emit, chat-completions SSE writers drain an empty queue and clients
(Open WebUI, etc.) see a finish chunk with zero content delta —
indistinguishable from a crash.

Verified: the test fails when the production fix is reverted.
2026-05-24 07:38:24 -07:00
ooovenenoso
d759a67c0f fix: add recovery hints to loop guard warnings 2026-05-19 00:12:12 -07:00
Mind-Dragon
0704589ceb fix(agent): make tool loop guardrails warning-first 2026-04-30 20:43:15 -07:00
Mind-Dragon
58b89965c8 fix(agent): add tool-call loop guardrails 2026-04-30 20:43:15 -07:00