Commit graph

1 commit

Author SHA1 Message Date
srojk34
2d3eac5fbd fix(moa): apply prompt-caching decoration to the aggregator's one-shot synthesis call
22c5048d9 restored Anthropic-style cache_control for two of MoA's three
call paths: the acting aggregator (MoAChatCompletions.create, the
persistent `provider: moa` model) and the advisor fan-out (_run_reference).
aggregate_moa_context() -- the /moa <prompt> one-shot command's synthesis
call -- is the third, independent call path and was never covered: its
call_llm(task="moa_aggregator", ...) sent a single undecorated user message
containing the full joined reference output, re-billing the entire input on
every invocation even when the resolved aggregator slot is a cache-honoring
route (Claude on OpenRouter/native Anthropic, MiniMax, Qwen/DashScope).

- Generalize _maybe_apply_advisor_cache_control to
  _maybe_apply_moa_cache_control (it never had advisor-specific logic --
  same policy function, same breakpoint layout as the main loop, judged
  purely on the passed-in runtime) and reuse it in aggregate_moa_context
  the same way _run_reference already does.
- Compute _slot_runtime(aggregator) once and reuse it for both the
  decoration call and the call_llm kwargs, instead of calling it twice.

Mutation-verified: reverting the moa_loop.py change makes the new
regression test fail by asserting a plain string aggregator-message
content where the cache-honoring case expects native cache_control
content blocks.
2026-07-04 20:59:02 +05:30