hermes-agent/tests/hermes_cli/test_moa_config.py
Teknium c6575df927
feat(moa): expose MoA presets as selectable virtual models (#46081)
* feat(moa): expose MoA presets as selectable virtual models

Reconstructed onto current main (PR #46081's base had diverged with no common
ancestor, marking the PR dirty so CI never dispatched). MoA is now a virtual
provider: each named preset is a selectable model under provider 'moa', and the
preset's aggregator is the acting model that answers and calls tools.

Reference models fan out in parallel via a bounded ThreadPoolExecutor (the same
batch pattern delegate_task uses) — all references dispatched at once, collected
when every one finishes, then handed to the aggregator. Output order is
preserved, failures and the MoA-recursion guard stay isolated per reference.

- Removed the old mixture_of_agents model tool and moa toolset.
- Added moa as a virtual provider in the provider/model inventory.
- /moa is shortcut behavior over model selection (default preset / named preset
  / one-shot prompt).
- Dashboard + Desktop manage named presets; presets appear in model pickers.
- Parallel reference fan-out in agent/moa_loop.py with regression test.

* fix(moa): thread moa_config through _run_agent to _run_agent_inner

The reconstructed gateway MoA wiring declared moa_config on _run_agent (the
profile-scoping wrapper) and used it inside _run_agent_inner, but the wrapper
never forwarded it — _run_agent_inner had no such parameter, so the runtime hit
NameError: name 'moa_config' is not defined on the compression-failure session
sync path. Add moa_config to _run_agent_inner's signature and forward it from
both wrapper call sites (multiplex and non-multiplex). Caught by
tests/gateway/test_compression_failure_session_sync.py on CI shard test(4).

* fix(moa): classify moa as a virtual provider in the catalog

The moa virtual provider has no PROVIDER_REGISTRY/ProviderProfile entry, so
provider_catalog() fell through to the default auth_type="api_key" with no
env vars — tripping two catalog invariants:
  - test_provider_catalog: api_key providers must expose a credential env var
  - test_provider_parity: every hermes-model provider must be desktop-configurable

moa already declares auth_type="virtual" in HERMES_OVERLAYS; consult that
overlay as an auth_type fallback so the catalog reports moa as virtual (no real
credential, no network endpoint). Exempt virtual providers from the desktop
parity union check the same way 'custom' is exempt — derived from the catalog,
not a hardcoded slug, so future virtual providers are covered too.
2026-06-25 13:52:06 -07:00

97 lines
3.3 KiB
Python

from hermes_cli.moa_config import (
DEFAULT_MOA_AGGREGATOR,
DEFAULT_MOA_PRESET_NAME,
DEFAULT_MOA_REFERENCE_MODELS,
build_moa_turn_prompt,
decode_moa_turn,
exact_moa_preset_name,
normalize_moa_config,
resolve_moa_preset,
set_active_moa_preset,
)
def test_normalize_moa_config_uses_default_named_preset():
cfg = normalize_moa_config({})
assert cfg["default_preset"] == DEFAULT_MOA_PRESET_NAME
assert list(cfg["presets"]) == [DEFAULT_MOA_PRESET_NAME]
assert cfg["reference_models"] == DEFAULT_MOA_REFERENCE_MODELS
assert cfg["aggregator"] == DEFAULT_MOA_AGGREGATOR
def test_normalize_moa_config_preserves_named_presets():
cfg = normalize_moa_config(
{
"default_preset": "coding",
"presets": {
"coding": {
"reference_models": [{"provider": "openai-codex", "model": "gpt-5.5"}],
"aggregator": {"provider": "openrouter", "model": "anthropic/claude-opus-4.8"},
},
"review": {
"reference_models": [{"provider": "openrouter", "model": "deepseek/deepseek-v4-pro"}],
"aggregator": {"provider": "openrouter", "model": "anthropic/claude-opus-4.8"},
},
},
}
)
assert cfg["default_preset"] == "coding"
assert set(cfg["presets"]) == {"coding", "review"}
assert cfg["reference_models"] == [{"provider": "openai-codex", "model": "gpt-5.5"}]
def test_legacy_flat_config_becomes_default_preset():
cfg = normalize_moa_config(
{
"reference_models": [{"provider": "openai-codex", "model": "gpt-5.5"}],
"aggregator": {"provider": "openrouter", "model": "anthropic/claude-opus-4.8"},
}
)
assert cfg["presets"][DEFAULT_MOA_PRESET_NAME]["reference_models"] == [
{"provider": "openai-codex", "model": "gpt-5.5"}
]
def test_exact_preset_matching_is_not_fuzzy():
config = {"presets": {"coding": {}, "review": {}}}
assert exact_moa_preset_name(config, "coding") == "coding"
assert exact_moa_preset_name(config, "cod") is None
assert exact_moa_preset_name(config, "coding please fix this") is None
def test_active_preset_toggle_validation():
config = {"default_preset": "coding", "presets": {"coding": {}, "review": {}}}
active = set_active_moa_preset(config, "review")
assert active["active_preset"] == "review"
inactive = set_active_moa_preset(active, "")
assert inactive["active_preset"] == ""
def test_resolve_moa_preset_returns_requested_model_set():
cfg = normalize_moa_config(
{
"presets": {
"coding": {"reference_models": [{"provider": "openai-codex", "model": "gpt-5.5"}]},
"review": {"reference_models": [{"provider": "openrouter", "model": "deepseek/deepseek-v4-pro"}]},
}
}
)
assert resolve_moa_preset(cfg, "review")["reference_models"] == [
{"provider": "openrouter", "model": "deepseek/deepseek-v4-pro"}
]
def test_build_moa_turn_prompt_encodes_one_shot_default_preset():
prompt = build_moa_turn_prompt("write a file then inspect it")
decoded_prompt, cfg = decode_moa_turn(prompt)
assert decoded_prompt == "write a file then inspect it"
assert cfg is not None
assert cfg["reference_models"] == DEFAULT_MOA_REFERENCE_MODELS