hermes-agent/tests/hermes_cli/test_image_gen_picker.py
Teknium ff9752410a
feat(plugins): pluggable image_gen backends + OpenAI provider (#13799)
* feat(plugins): pluggable image_gen backends + OpenAI provider

Adds a ImageGenProvider ABC so image generation backends register as
bundled plugins under `plugins/image_gen/<name>/`. The plugin scanner
gains three primitives to make this work generically:

- `kind:` manifest field (`standalone` | `backend` | `exclusive`).
  Bundled `kind: backend` plugins auto-load — no `plugins.enabled`
  incantation. User-installed backends stay opt-in.
- Path-derived keys: `plugins/image_gen/openai/` gets key
  `image_gen/openai`, so a future `tts/openai` cannot collide.
- Depth-2 recursion into category namespaces (parent dirs without a
  `plugin.yaml` of their own).

Includes `OpenAIImageGenProvider` as the first consumer (gpt-image-1.5
default, plus gpt-image-1, gpt-image-1-mini, DALL-E 3/2). Base64
responses save to `$HERMES_HOME/cache/images/`; URL responses pass
through.

FAL stays in-tree for this PR — a follow-up ports it into
`plugins/image_gen/fal/` so the in-tree `image_generation_tool.py`
slims down. The dispatch shim in `_handle_image_generate` only fires
when `image_gen.provider` is explicitly set to a non-FAL value, so
existing FAL setups are untouched.

- 41 unit tests (scanner recursion, kind parsing, gate logic,
  registry, OpenAI payload shapes)
- E2E smoke verified: bundled plugin autoloads, registers, and
  `_handle_image_generate` routes to OpenAI when configured

* fix(image_gen/openai): don't send response_format to gpt-image-*

The live API rejects it: 'Unknown parameter: response_format'
(verified 2026-04-21 with gpt-image-1.5). gpt-image-* models return
b64_json unconditionally, so the parameter was both unnecessary and
actively broken.

* feat(image_gen/openai): gpt-image-2 only, drop legacy catalog

gpt-image-2 is the latest/best OpenAI image model (released 2026-04-21)
and there's no reason to expose the older gpt-image-1.5 / gpt-image-1 /
dall-e-3 / dall-e-2 alongside it — slower, lower quality, or awkward
(dall-e-2 squares only). Trim the catalog down to a single model.

Live-verified end-to-end: landscape 1536x1024 render of a Moog-style
synth matches prompt exactly, 2.4MB PNG saved to cache.

* feat(image_gen/openai): expose gpt-image-2 as three quality tiers

Users pick speed/fidelity via the normal model picker instead of a
hidden quality knob. All three tier IDs resolve to the single underlying
gpt-image-2 API model with a different quality parameter:

  gpt-image-2-low     ~15s   fast iteration
  gpt-image-2-medium  ~40s   default
  gpt-image-2-high    ~2min  highest fidelity

Live-measured on OpenAI's API today: 15.4s / 40.8s / 116.9s for the
same 1024x1024 prompt.

Config:
  image_gen.openai.model: gpt-image-2-high
  # or
  image_gen.model: gpt-image-2-low
  # or env var for scripts/tests
  OPENAI_IMAGE_MODEL=gpt-image-2-medium

Live-verified end-to-end with the low tier: 18.8s landscape render of a
golden retriever in wildflowers, vision-confirmed exact match.

* feat(tools_config): plugin image_gen providers inject themselves into picker

'hermes tools' → Image Generation now shows plugin-registered backends
alongside Nous Subscription and FAL.ai without tools_config.py needing
to know about them. OpenAI appears as a third option today; future
backends appear automatically as they're added.

Mechanism:
- ImageGenProvider gains an optional get_setup_schema() hook
  (name, badge, tag, env_vars). Default derived from display_name.
- tools_config._plugin_image_gen_providers() pulls the schemas from
  every registered non-FAL plugin provider.
- _visible_providers() appends those rows when rendering the Image
  Generation category.
- _configure_provider() handles the new image_gen_plugin_name marker:
  writes image_gen.provider and routes to the plugin's list_models()
  catalog for the model picker.
- _toolset_needs_configuration_prompt('image_gen') stops demanding a
  FAL key when any plugin provider reports is_available().

FAL is skipped in the plugin path because it already has hardcoded
TOOL_CATEGORIES rows — when it gets ported to a plugin in a follow-up
PR the hardcoded rows go away and it surfaces through the same path
as OpenAI.

Verified live: picker shows Nous Subscription / FAL.ai / OpenAI.
Picking OpenAI prompts for OPENAI_API_KEY, then shows the
gpt-image-2-low/medium/high model picker sourced from the plugin.

397 tests pass across plugins/, tools_config, registry, and picker.

* fix(image_gen): close final gaps for plugin-backend parity with FAL

Two small places that still hardcoded FAL:

- hermes_cli/setup.py status line: an OpenAI-only setup showed
  'Image Generation: missing FAL_KEY'. Now probes plugin providers
  and reports '(OpenAI)' when one is_available() — or falls back to
  'missing FAL_KEY or OPENAI_API_KEY' if nothing is configured.

- image_generate tool schema description: said 'using FAL.ai, default
  FLUX 2 Klein 9B'. Rewrote provider-neutral — 'backend and model are
  user-configured' — and notes the 'image' field can be a URL or an
  absolute path, which the gateway delivers either way via
  extract_local_files().
2026-04-21 21:30:10 -07:00

174 lines
6.2 KiB
Python

"""Tests for plugin image_gen providers injecting themselves into the picker.
Covers `_plugin_image_gen_providers`, `_visible_providers`, and
`_toolset_needs_configuration_prompt` handling of plugin providers.
"""
from __future__ import annotations
import pytest
from agent import image_gen_registry
from agent.image_gen_provider import ImageGenProvider
class _FakeProvider(ImageGenProvider):
def __init__(self, name: str, available: bool = True, schema=None, models=None):
self._name = name
self._available = available
self._schema = schema or {
"name": name.title(),
"badge": "test",
"tag": f"{name} test tag",
"env_vars": [{"key": f"{name.upper()}_API_KEY", "prompt": f"{name} key"}],
}
self._models = models or [
{"id": f"{name}-model-v1", "display": f"{name} v1",
"speed": "~5s", "strengths": "test", "price": "$"},
]
@property
def name(self) -> str:
return self._name
def is_available(self) -> bool:
return self._available
def list_models(self):
return list(self._models)
def default_model(self):
return self._models[0]["id"] if self._models else None
def get_setup_schema(self):
return dict(self._schema)
def generate(self, prompt, aspect_ratio="landscape", **kw):
return {"success": True, "image": f"{self._name}://{prompt}"}
@pytest.fixture(autouse=True)
def _reset_registry():
image_gen_registry._reset_for_tests()
yield
image_gen_registry._reset_for_tests()
class TestPluginPickerInjection:
def test_plugin_providers_returns_registered(self, monkeypatch):
from hermes_cli import tools_config
image_gen_registry.register_provider(_FakeProvider("myimg"))
rows = tools_config._plugin_image_gen_providers()
names = [r["name"] for r in rows]
plugin_names = [r.get("image_gen_plugin_name") for r in rows]
assert "Myimg" in names
assert "myimg" in plugin_names
def test_fal_skipped_to_avoid_duplicate(self, monkeypatch):
from hermes_cli import tools_config
# Simulate a FAL plugin being registered — the picker already has
# hardcoded FAL rows in TOOL_CATEGORIES, so plugin-FAL must be
# skipped to avoid showing FAL twice.
image_gen_registry.register_provider(_FakeProvider("fal"))
image_gen_registry.register_provider(_FakeProvider("openai"))
rows = tools_config._plugin_image_gen_providers()
names = [r.get("image_gen_plugin_name") for r in rows]
assert "fal" not in names
assert "openai" in names
def test_visible_providers_includes_plugins_for_image_gen(self, monkeypatch):
from hermes_cli import tools_config
image_gen_registry.register_provider(_FakeProvider("someimg"))
cat = tools_config.TOOL_CATEGORIES["image_gen"]
visible = tools_config._visible_providers(cat, {})
plugin_names = [p.get("image_gen_plugin_name") for p in visible if p.get("image_gen_plugin_name")]
assert "someimg" in plugin_names
def test_visible_providers_does_not_inject_into_other_categories(self, monkeypatch):
from hermes_cli import tools_config
image_gen_registry.register_provider(_FakeProvider("someimg"))
# Browser category must NOT see image_gen plugins.
browser = tools_config.TOOL_CATEGORIES["browser"]
visible = tools_config._visible_providers(browser, {})
assert all(p.get("image_gen_plugin_name") is None for p in visible)
class TestPluginCatalog:
def test_plugin_catalog_returns_models(self):
from hermes_cli import tools_config
image_gen_registry.register_provider(_FakeProvider("catimg"))
catalog, default = tools_config._plugin_image_gen_catalog("catimg")
assert "catimg-model-v1" in catalog
assert default == "catimg-model-v1"
def test_plugin_catalog_empty_for_unknown(self):
from hermes_cli import tools_config
catalog, default = tools_config._plugin_image_gen_catalog("does-not-exist")
assert catalog == {}
assert default is None
class TestConfigPrompt:
def test_image_gen_satisfied_by_plugin_provider(self, monkeypatch, tmp_path):
"""When a plugin provider reports is_available(), the picker should
not force a setup prompt on the user."""
from hermes_cli import tools_config
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
monkeypatch.delenv("FAL_KEY", raising=False)
image_gen_registry.register_provider(_FakeProvider("avail-img", available=True))
assert tools_config._toolset_needs_configuration_prompt("image_gen", {}) is False
def test_image_gen_still_prompts_when_nothing_available(self, monkeypatch, tmp_path):
from hermes_cli import tools_config
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
monkeypatch.delenv("FAL_KEY", raising=False)
image_gen_registry.register_provider(_FakeProvider("unavail-img", available=False))
assert tools_config._toolset_needs_configuration_prompt("image_gen", {}) is True
class TestConfigWriting:
def test_picking_plugin_provider_writes_provider_and_model(self, monkeypatch, tmp_path):
"""When a user picks a plugin-backed image_gen provider with no
env vars needed, ``_configure_provider`` should write both
``image_gen.provider`` and ``image_gen.model``."""
from hermes_cli import tools_config
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
image_gen_registry.register_provider(_FakeProvider("noenv", schema={
"name": "NoEnv",
"badge": "free",
"tag": "",
"env_vars": [],
}))
# Stub out the interactive model picker — no TTY in tests.
monkeypatch.setattr(tools_config, "_prompt_choice", lambda *a, **kw: 0)
config: dict = {}
provider_row = {
"name": "NoEnv",
"env_vars": [],
"image_gen_plugin_name": "noenv",
}
tools_config._configure_provider(provider_row, config)
assert config["image_gen"]["provider"] == "noenv"
assert config["image_gen"]["model"] == "noenv-model-v1"