mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-25 17:04:52 +00:00
32a4faa2d5
PR #56756 added reference_max_tokens to cap MoA advisor output and cut turn latency. The value is correctly threaded through five layers of MoA code (moa_config → conversation_loop → aggregate_moa_context → _run_references_parallel → _run_reference → call_llm(task='moa_reference', max_tokens=800, ...)). However, _build_call_kwargs() in auxiliary_client.py silently drops max_tokens for all OpenAI-compatible providers (PR #34845, which fixed endpoints and NVIDIA NIM keep it. This means reference_max_tokens never reached the API for the vast majority of providers. The bug affects every OpenAI-compatible MoA reference/aggregator slot: Z.AI (coding plan), OpenRouter, OpenAI, GitHub Copilot, and local providers. Only Anthropic-compat endpoints (MiniMax, /anthropic URLs) worked — by coincidence, not MoA-aware design. Fix: thread the 'task' parameter through all six _build_call_kwargs() call sites. When task starts with 'moa_', max_tokens is always included in the request kwargs regardless of provider. Non-MoA auxiliary tasks (compression, titles, vision, etc.) keep PR #34845 behavior unchanged. Verified end-to-end: - Z.AI GLM-5.2 with max_tokens=50 → returned exactly 50 tokens - Z.AI GLM-5.2 with max_tokens=20 → returned exactly 20 tokens - Z.AI GLM-5.2 uncapped → returned 315 tokens - 7 new regression tests covering 4 providers, Anthropic wire, non-MoA tasks, and prefix-matching boundary - 288 auxiliary_client tests pass (was 281, +7 new), 84 MoA tests pass - Zero regressions
2 lines
28 B
Text
2 lines
28 B
Text
Janig88
|
|
# PR #58402 salvage
|