mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-25 00:51:20 +00:00
feat(plugins): pluggable image_gen backends + OpenAI provider (#13799)
* feat(plugins): pluggable image_gen backends + OpenAI provider
Adds a ImageGenProvider ABC so image generation backends register as
bundled plugins under `plugins/image_gen/<name>/`. The plugin scanner
gains three primitives to make this work generically:
- `kind:` manifest field (`standalone` | `backend` | `exclusive`).
Bundled `kind: backend` plugins auto-load — no `plugins.enabled`
incantation. User-installed backends stay opt-in.
- Path-derived keys: `plugins/image_gen/openai/` gets key
`image_gen/openai`, so a future `tts/openai` cannot collide.
- Depth-2 recursion into category namespaces (parent dirs without a
`plugin.yaml` of their own).
Includes `OpenAIImageGenProvider` as the first consumer (gpt-image-1.5
default, plus gpt-image-1, gpt-image-1-mini, DALL-E 3/2). Base64
responses save to `$HERMES_HOME/cache/images/`; URL responses pass
through.
FAL stays in-tree for this PR — a follow-up ports it into
`plugins/image_gen/fal/` so the in-tree `image_generation_tool.py`
slims down. The dispatch shim in `_handle_image_generate` only fires
when `image_gen.provider` is explicitly set to a non-FAL value, so
existing FAL setups are untouched.
- 41 unit tests (scanner recursion, kind parsing, gate logic,
registry, OpenAI payload shapes)
- E2E smoke verified: bundled plugin autoloads, registers, and
`_handle_image_generate` routes to OpenAI when configured
* fix(image_gen/openai): don't send response_format to gpt-image-*
The live API rejects it: 'Unknown parameter: response_format'
(verified 2026-04-21 with gpt-image-1.5). gpt-image-* models return
b64_json unconditionally, so the parameter was both unnecessary and
actively broken.
* feat(image_gen/openai): gpt-image-2 only, drop legacy catalog
gpt-image-2 is the latest/best OpenAI image model (released 2026-04-21)
and there's no reason to expose the older gpt-image-1.5 / gpt-image-1 /
dall-e-3 / dall-e-2 alongside it — slower, lower quality, or awkward
(dall-e-2 squares only). Trim the catalog down to a single model.
Live-verified end-to-end: landscape 1536x1024 render of a Moog-style
synth matches prompt exactly, 2.4MB PNG saved to cache.
* feat(image_gen/openai): expose gpt-image-2 as three quality tiers
Users pick speed/fidelity via the normal model picker instead of a
hidden quality knob. All three tier IDs resolve to the single underlying
gpt-image-2 API model with a different quality parameter:
gpt-image-2-low ~15s fast iteration
gpt-image-2-medium ~40s default
gpt-image-2-high ~2min highest fidelity
Live-measured on OpenAI's API today: 15.4s / 40.8s / 116.9s for the
same 1024x1024 prompt.
Config:
image_gen.openai.model: gpt-image-2-high
# or
image_gen.model: gpt-image-2-low
# or env var for scripts/tests
OPENAI_IMAGE_MODEL=gpt-image-2-medium
Live-verified end-to-end with the low tier: 18.8s landscape render of a
golden retriever in wildflowers, vision-confirmed exact match.
* feat(tools_config): plugin image_gen providers inject themselves into picker
'hermes tools' → Image Generation now shows plugin-registered backends
alongside Nous Subscription and FAL.ai without tools_config.py needing
to know about them. OpenAI appears as a third option today; future
backends appear automatically as they're added.
Mechanism:
- ImageGenProvider gains an optional get_setup_schema() hook
(name, badge, tag, env_vars). Default derived from display_name.
- tools_config._plugin_image_gen_providers() pulls the schemas from
every registered non-FAL plugin provider.
- _visible_providers() appends those rows when rendering the Image
Generation category.
- _configure_provider() handles the new image_gen_plugin_name marker:
writes image_gen.provider and routes to the plugin's list_models()
catalog for the model picker.
- _toolset_needs_configuration_prompt('image_gen') stops demanding a
FAL key when any plugin provider reports is_available().
FAL is skipped in the plugin path because it already has hardcoded
TOOL_CATEGORIES rows — when it gets ported to a plugin in a follow-up
PR the hardcoded rows go away and it surfaces through the same path
as OpenAI.
Verified live: picker shows Nous Subscription / FAL.ai / OpenAI.
Picking OpenAI prompts for OPENAI_API_KEY, then shows the
gpt-image-2-low/medium/high model picker sourced from the plugin.
397 tests pass across plugins/, tools_config, registry, and picker.
* fix(image_gen): close final gaps for plugin-backend parity with FAL
Two small places that still hardcoded FAL:
- hermes_cli/setup.py status line: an OpenAI-only setup showed
'Image Generation: missing FAL_KEY'. Now probes plugin providers
and reports '(OpenAI)' when one is_available() — or falls back to
'missing FAL_KEY or OPENAI_API_KEY' if nothing is configured.
- image_generate tool schema description: said 'using FAL.ai, default
FLUX 2 Klein 9B'. Rewrote provider-neutral — 'backend and model are
user-configured' — and notes the 'image' field can be a URL or an
absolute path, which the gateway delivers either way via
extract_local_files().
This commit is contained in:
parent
d1acf17773
commit
ff9752410a
13 changed files with 2122 additions and 67 deletions
242
agent/image_gen_provider.py
Normal file
242
agent/image_gen_provider.py
Normal file
|
|
@ -0,0 +1,242 @@
|
||||||
|
"""
|
||||||
|
Image Generation Provider ABC
|
||||||
|
=============================
|
||||||
|
|
||||||
|
Defines the pluggable-backend interface for image generation. Providers register
|
||||||
|
instances via ``PluginContext.register_image_gen_provider()``; the active one
|
||||||
|
(selected via ``image_gen.provider`` in ``config.yaml``) services every
|
||||||
|
``image_generate`` tool call.
|
||||||
|
|
||||||
|
Providers live in ``<repo>/plugins/image_gen/<name>/`` (built-in, auto-loaded
|
||||||
|
as ``kind: backend``) or ``~/.hermes/plugins/image_gen/<name>/`` (user, opt-in
|
||||||
|
via ``plugins.enabled``).
|
||||||
|
|
||||||
|
Response shape
|
||||||
|
--------------
|
||||||
|
All providers return a dict that :func:`success_response` / :func:`error_response`
|
||||||
|
produce. The tool wrapper JSON-serializes it. Keys:
|
||||||
|
|
||||||
|
success bool
|
||||||
|
image str | None URL or absolute file path
|
||||||
|
model str provider-specific model identifier
|
||||||
|
prompt str echoed prompt
|
||||||
|
aspect_ratio str "landscape" | "square" | "portrait"
|
||||||
|
provider str provider name (for diagnostics)
|
||||||
|
error str only when success=False
|
||||||
|
error_type str only when success=False
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import abc
|
||||||
|
import base64
|
||||||
|
import datetime
|
||||||
|
import logging
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict, List, Optional, Tuple
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
VALID_ASPECT_RATIOS: Tuple[str, ...] = ("landscape", "square", "portrait")
|
||||||
|
DEFAULT_ASPECT_RATIO = "landscape"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# ABC
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class ImageGenProvider(abc.ABC):
|
||||||
|
"""Abstract base class for an image generation backend.
|
||||||
|
|
||||||
|
Subclasses must implement :meth:`generate`. Everything else has sane
|
||||||
|
defaults — override only what your provider needs.
|
||||||
|
"""
|
||||||
|
|
||||||
|
@property
|
||||||
|
@abc.abstractmethod
|
||||||
|
def name(self) -> str:
|
||||||
|
"""Stable short identifier used in ``image_gen.provider`` config.
|
||||||
|
|
||||||
|
Lowercase, no spaces. Examples: ``fal``, ``openai``, ``replicate``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def display_name(self) -> str:
|
||||||
|
"""Human-readable label shown in ``hermes tools``. Defaults to ``name.title()``."""
|
||||||
|
return self.name.title()
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
"""Return True when this provider can service calls.
|
||||||
|
|
||||||
|
Typically checks for a required API key. Default: True
|
||||||
|
(providers with no external dependencies are always available).
|
||||||
|
"""
|
||||||
|
return True
|
||||||
|
|
||||||
|
def list_models(self) -> List[Dict[str, Any]]:
|
||||||
|
"""Return catalog entries for ``hermes tools`` model picker.
|
||||||
|
|
||||||
|
Each entry::
|
||||||
|
|
||||||
|
{
|
||||||
|
"id": "gpt-image-1.5", # required
|
||||||
|
"display": "GPT Image 1.5", # optional; defaults to id
|
||||||
|
"speed": "~10s", # optional
|
||||||
|
"strengths": "...", # optional
|
||||||
|
"price": "$...", # optional
|
||||||
|
}
|
||||||
|
|
||||||
|
Default: empty list (provider has no user-selectable models).
|
||||||
|
"""
|
||||||
|
return []
|
||||||
|
|
||||||
|
def get_setup_schema(self) -> Dict[str, Any]:
|
||||||
|
"""Return provider metadata for the ``hermes tools`` picker.
|
||||||
|
|
||||||
|
Used by ``tools_config.py`` to inject this provider as a row in
|
||||||
|
the Image Generation provider list. Shape::
|
||||||
|
|
||||||
|
{
|
||||||
|
"name": "OpenAI", # picker label
|
||||||
|
"badge": "paid", # optional short tag
|
||||||
|
"tag": "One-line description...", # optional subtitle
|
||||||
|
"env_vars": [ # keys to prompt for
|
||||||
|
{"key": "OPENAI_API_KEY",
|
||||||
|
"prompt": "OpenAI API key",
|
||||||
|
"url": "https://platform.openai.com/api-keys"},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
Default: minimal entry derived from ``display_name``. Override to
|
||||||
|
expose API key prompts and custom badges.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"name": self.display_name,
|
||||||
|
"badge": "",
|
||||||
|
"tag": "",
|
||||||
|
"env_vars": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
def default_model(self) -> Optional[str]:
|
||||||
|
"""Return the default model id, or None if not applicable."""
|
||||||
|
models = self.list_models()
|
||||||
|
if models:
|
||||||
|
return models[0].get("id")
|
||||||
|
return None
|
||||||
|
|
||||||
|
@abc.abstractmethod
|
||||||
|
def generate(
|
||||||
|
self,
|
||||||
|
prompt: str,
|
||||||
|
aspect_ratio: str = DEFAULT_ASPECT_RATIO,
|
||||||
|
**kwargs: Any,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Generate an image.
|
||||||
|
|
||||||
|
Implementations should return the dict from :func:`success_response`
|
||||||
|
or :func:`error_response`. ``kwargs`` may contain forward-compat
|
||||||
|
parameters future versions of the schema will expose — implementations
|
||||||
|
should ignore unknown keys.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_aspect_ratio(value: Optional[str]) -> str:
|
||||||
|
"""Clamp an aspect_ratio value to the valid set, defaulting to landscape.
|
||||||
|
|
||||||
|
Invalid values are coerced rather than rejected so the tool surface is
|
||||||
|
forgiving of agent mistakes.
|
||||||
|
"""
|
||||||
|
if not isinstance(value, str):
|
||||||
|
return DEFAULT_ASPECT_RATIO
|
||||||
|
v = value.strip().lower()
|
||||||
|
if v in VALID_ASPECT_RATIOS:
|
||||||
|
return v
|
||||||
|
return DEFAULT_ASPECT_RATIO
|
||||||
|
|
||||||
|
|
||||||
|
def _images_cache_dir() -> Path:
|
||||||
|
"""Return ``$HERMES_HOME/cache/images/``, creating parents as needed."""
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
|
||||||
|
path = get_hermes_home() / "cache" / "images"
|
||||||
|
path.mkdir(parents=True, exist_ok=True)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def save_b64_image(
|
||||||
|
b64_data: str,
|
||||||
|
*,
|
||||||
|
prefix: str = "image",
|
||||||
|
extension: str = "png",
|
||||||
|
) -> Path:
|
||||||
|
"""Decode base64 image data and write it under ``$HERMES_HOME/cache/images/``.
|
||||||
|
|
||||||
|
Returns the absolute :class:`Path` to the saved file.
|
||||||
|
|
||||||
|
Filename format: ``<prefix>_<YYYYMMDD_HHMMSS>_<short-uuid>.<ext>``.
|
||||||
|
"""
|
||||||
|
raw = base64.b64decode(b64_data)
|
||||||
|
ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||||
|
short = uuid.uuid4().hex[:8]
|
||||||
|
path = _images_cache_dir() / f"{prefix}_{ts}_{short}.{extension}"
|
||||||
|
path.write_bytes(raw)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def success_response(
|
||||||
|
*,
|
||||||
|
image: str,
|
||||||
|
model: str,
|
||||||
|
prompt: str,
|
||||||
|
aspect_ratio: str,
|
||||||
|
provider: str,
|
||||||
|
extra: Optional[Dict[str, Any]] = None,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Build a uniform success response dict.
|
||||||
|
|
||||||
|
``image`` may be an HTTP URL or an absolute filesystem path (for b64
|
||||||
|
providers like OpenAI). Callers that need to pass through additional
|
||||||
|
backend-specific fields can supply ``extra``.
|
||||||
|
"""
|
||||||
|
payload: Dict[str, Any] = {
|
||||||
|
"success": True,
|
||||||
|
"image": image,
|
||||||
|
"model": model,
|
||||||
|
"prompt": prompt,
|
||||||
|
"aspect_ratio": aspect_ratio,
|
||||||
|
"provider": provider,
|
||||||
|
}
|
||||||
|
if extra:
|
||||||
|
for k, v in extra.items():
|
||||||
|
payload.setdefault(k, v)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def error_response(
|
||||||
|
*,
|
||||||
|
error: str,
|
||||||
|
error_type: str = "provider_error",
|
||||||
|
provider: str = "",
|
||||||
|
model: str = "",
|
||||||
|
prompt: str = "",
|
||||||
|
aspect_ratio: str = DEFAULT_ASPECT_RATIO,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Build a uniform error response dict."""
|
||||||
|
return {
|
||||||
|
"success": False,
|
||||||
|
"image": None,
|
||||||
|
"error": error,
|
||||||
|
"error_type": error_type,
|
||||||
|
"model": model,
|
||||||
|
"prompt": prompt,
|
||||||
|
"aspect_ratio": aspect_ratio,
|
||||||
|
"provider": provider,
|
||||||
|
}
|
||||||
120
agent/image_gen_registry.py
Normal file
120
agent/image_gen_registry.py
Normal file
|
|
@ -0,0 +1,120 @@
|
||||||
|
"""
|
||||||
|
Image Generation Provider Registry
|
||||||
|
==================================
|
||||||
|
|
||||||
|
Central map of registered providers. Populated by plugins at import-time via
|
||||||
|
``PluginContext.register_image_gen_provider()``; consumed by the
|
||||||
|
``image_generate`` tool to dispatch each call to the active backend.
|
||||||
|
|
||||||
|
Active selection
|
||||||
|
----------------
|
||||||
|
The active provider is chosen by ``image_gen.provider`` in ``config.yaml``.
|
||||||
|
If unset, :func:`get_active_provider` applies fallback logic:
|
||||||
|
|
||||||
|
1. If exactly one provider is registered, use it.
|
||||||
|
2. Otherwise if a provider named ``fal`` is registered, use it (legacy
|
||||||
|
default — matches pre-plugin behavior).
|
||||||
|
3. Otherwise return ``None`` (the tool surfaces a helpful error pointing
|
||||||
|
the user at ``hermes tools``).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import threading
|
||||||
|
from typing import Dict, List, Optional
|
||||||
|
|
||||||
|
from agent.image_gen_provider import ImageGenProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
_providers: Dict[str, ImageGenProvider] = {}
|
||||||
|
_lock = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
def register_provider(provider: ImageGenProvider) -> None:
|
||||||
|
"""Register an image generation provider.
|
||||||
|
|
||||||
|
Re-registration (same ``name``) overwrites the previous entry and logs
|
||||||
|
a debug message — this makes hot-reload scenarios (tests, dev loops)
|
||||||
|
behave predictably.
|
||||||
|
"""
|
||||||
|
if not isinstance(provider, ImageGenProvider):
|
||||||
|
raise TypeError(
|
||||||
|
f"register_provider() expects an ImageGenProvider instance, "
|
||||||
|
f"got {type(provider).__name__}"
|
||||||
|
)
|
||||||
|
name = provider.name
|
||||||
|
if not isinstance(name, str) or not name.strip():
|
||||||
|
raise ValueError("Image gen provider .name must be a non-empty string")
|
||||||
|
with _lock:
|
||||||
|
existing = _providers.get(name)
|
||||||
|
_providers[name] = provider
|
||||||
|
if existing is not None:
|
||||||
|
logger.debug("Image gen provider '%s' re-registered (was %r)", name, type(existing).__name__)
|
||||||
|
else:
|
||||||
|
logger.debug("Registered image gen provider '%s' (%s)", name, type(provider).__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def list_providers() -> List[ImageGenProvider]:
|
||||||
|
"""Return all registered providers, sorted by name."""
|
||||||
|
with _lock:
|
||||||
|
items = list(_providers.values())
|
||||||
|
return sorted(items, key=lambda p: p.name)
|
||||||
|
|
||||||
|
|
||||||
|
def get_provider(name: str) -> Optional[ImageGenProvider]:
|
||||||
|
"""Return the provider registered under *name*, or None."""
|
||||||
|
if not isinstance(name, str):
|
||||||
|
return None
|
||||||
|
with _lock:
|
||||||
|
return _providers.get(name.strip())
|
||||||
|
|
||||||
|
|
||||||
|
def get_active_provider() -> Optional[ImageGenProvider]:
|
||||||
|
"""Resolve the currently-active provider.
|
||||||
|
|
||||||
|
Reads ``image_gen.provider`` from config.yaml; falls back per the
|
||||||
|
module docstring.
|
||||||
|
"""
|
||||||
|
configured: Optional[str] = None
|
||||||
|
try:
|
||||||
|
from hermes_cli.config import load_config
|
||||||
|
|
||||||
|
cfg = load_config()
|
||||||
|
section = cfg.get("image_gen") if isinstance(cfg, dict) else None
|
||||||
|
if isinstance(section, dict):
|
||||||
|
raw = section.get("provider")
|
||||||
|
if isinstance(raw, str) and raw.strip():
|
||||||
|
configured = raw.strip()
|
||||||
|
except Exception as exc:
|
||||||
|
logger.debug("Could not read image_gen.provider from config: %s", exc)
|
||||||
|
|
||||||
|
with _lock:
|
||||||
|
snapshot = dict(_providers)
|
||||||
|
|
||||||
|
if configured:
|
||||||
|
provider = snapshot.get(configured)
|
||||||
|
if provider is not None:
|
||||||
|
return provider
|
||||||
|
logger.debug(
|
||||||
|
"image_gen.provider='%s' configured but not registered; falling back",
|
||||||
|
configured,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Fallback: single-provider case
|
||||||
|
if len(snapshot) == 1:
|
||||||
|
return next(iter(snapshot.values()))
|
||||||
|
|
||||||
|
# Fallback: prefer legacy FAL for backward compat
|
||||||
|
if "fal" in snapshot:
|
||||||
|
return snapshot["fal"]
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _reset_for_tests() -> None:
|
||||||
|
"""Clear the registry. **Test-only.**"""
|
||||||
|
with _lock:
|
||||||
|
_providers.clear()
|
||||||
|
|
@ -133,6 +133,9 @@ def _get_enabled_plugins() -> Optional[set]:
|
||||||
# Data classes
|
# Data classes
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_VALID_PLUGIN_KINDS: Set[str] = {"standalone", "backend", "exclusive"}
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class PluginManifest:
|
class PluginManifest:
|
||||||
"""Parsed representation of a plugin.yaml manifest."""
|
"""Parsed representation of a plugin.yaml manifest."""
|
||||||
|
|
@ -146,6 +149,23 @@ class PluginManifest:
|
||||||
provides_hooks: List[str] = field(default_factory=list)
|
provides_hooks: List[str] = field(default_factory=list)
|
||||||
source: str = "" # "user", "project", or "entrypoint"
|
source: str = "" # "user", "project", or "entrypoint"
|
||||||
path: Optional[str] = None
|
path: Optional[str] = None
|
||||||
|
# Plugin kind — see plugins.py module docstring for semantics.
|
||||||
|
# ``standalone`` (default): hooks/tools of its own; opt-in via
|
||||||
|
# ``plugins.enabled``.
|
||||||
|
# ``backend``: pluggable backend for an existing core tool (e.g.
|
||||||
|
# image_gen). Built-in (bundled) backends auto-load;
|
||||||
|
# user-installed still gated by ``plugins.enabled``.
|
||||||
|
# ``exclusive``: category with exactly one active provider (memory).
|
||||||
|
# Selection via ``<category>.provider`` config key; the
|
||||||
|
# category's own discovery system handles loading and the
|
||||||
|
# general scanner skips these.
|
||||||
|
kind: str = "standalone"
|
||||||
|
# Registry key — path-derived, used by ``plugins.enabled``/``disabled``
|
||||||
|
# lookups and by ``hermes plugins list``. For a flat plugin at
|
||||||
|
# ``plugins/disk-cleanup/`` the key is ``disk-cleanup``; for a nested
|
||||||
|
# category plugin at ``plugins/image_gen/openai/`` the key is
|
||||||
|
# ``image_gen/openai``. When empty, falls back to ``name``.
|
||||||
|
key: str = ""
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|
@ -366,6 +386,33 @@ class PluginContext:
|
||||||
self.manifest.name, engine.name,
|
self.manifest.name, engine.name,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# -- image gen provider registration ------------------------------------
|
||||||
|
|
||||||
|
def register_image_gen_provider(self, provider) -> None:
|
||||||
|
"""Register an image generation backend.
|
||||||
|
|
||||||
|
``provider`` must be an instance of
|
||||||
|
:class:`agent.image_gen_provider.ImageGenProvider`. The
|
||||||
|
``provider.name`` attribute is what ``image_gen.provider`` in
|
||||||
|
``config.yaml`` matches against when routing ``image_generate``
|
||||||
|
tool calls.
|
||||||
|
"""
|
||||||
|
from agent.image_gen_provider import ImageGenProvider
|
||||||
|
from agent.image_gen_registry import register_provider
|
||||||
|
|
||||||
|
if not isinstance(provider, ImageGenProvider):
|
||||||
|
logger.warning(
|
||||||
|
"Plugin '%s' tried to register an image_gen provider that does "
|
||||||
|
"not inherit from ImageGenProvider. Ignoring.",
|
||||||
|
self.manifest.name,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
register_provider(provider)
|
||||||
|
logger.info(
|
||||||
|
"Plugin '%s' registered image_gen provider: %s",
|
||||||
|
self.manifest.name, provider.name,
|
||||||
|
)
|
||||||
|
|
||||||
# -- hook registration --------------------------------------------------
|
# -- hook registration --------------------------------------------------
|
||||||
|
|
||||||
def register_hook(self, hook_name: str, callback: Callable) -> None:
|
def register_hook(self, hook_name: str, callback: Callable) -> None:
|
||||||
|
|
@ -465,11 +512,16 @@ class PluginManager:
|
||||||
manifests: List[PluginManifest] = []
|
manifests: List[PluginManifest] = []
|
||||||
|
|
||||||
# 1. Bundled plugins (<repo>/plugins/<name>/)
|
# 1. Bundled plugins (<repo>/plugins/<name>/)
|
||||||
# Repo-shipped generic plugins live next to hermes_cli/. Memory and
|
#
|
||||||
# context_engine subdirs are handled by their own discovery paths, so
|
# Repo-shipped plugins live next to hermes_cli/. Two layouts are
|
||||||
# skip those names here. Bundled plugins are discovered (so they
|
# supported (see ``_scan_directory`` for details):
|
||||||
# show up in `hermes plugins`) but only loaded when added to
|
#
|
||||||
# `plugins.enabled` in config.yaml — opt-in like any other plugin.
|
# - flat: ``plugins/disk-cleanup/plugin.yaml`` (standalone)
|
||||||
|
# - category: ``plugins/image_gen/openai/plugin.yaml`` (backend)
|
||||||
|
#
|
||||||
|
# ``memory/`` and ``context_engine/`` are skipped at the top level —
|
||||||
|
# they have their own discovery systems. Porting those to the
|
||||||
|
# category-namespace ``kind: exclusive`` model is a future PR.
|
||||||
repo_plugins = Path(__file__).resolve().parent.parent / "plugins"
|
repo_plugins = Path(__file__).resolve().parent.parent / "plugins"
|
||||||
manifests.extend(
|
manifests.extend(
|
||||||
self._scan_directory(
|
self._scan_directory(
|
||||||
|
|
@ -492,36 +544,69 @@ class PluginManager:
|
||||||
manifests.extend(self._scan_entry_points())
|
manifests.extend(self._scan_entry_points())
|
||||||
|
|
||||||
# Load each manifest (skip user-disabled plugins).
|
# Load each manifest (skip user-disabled plugins).
|
||||||
# Later sources override earlier ones on name collision — user plugins
|
# Later sources override earlier ones on key collision — user
|
||||||
# take precedence over bundled, project plugins take precedence over
|
# plugins take precedence over bundled, project plugins take
|
||||||
# user. Dedup here so we only load the final winner.
|
# precedence over user. Dedup here so we only load the final
|
||||||
|
# winner. Keys are path-derived (``image_gen/openai``,
|
||||||
|
# ``disk-cleanup``) so ``tts/openai`` and ``image_gen/openai``
|
||||||
|
# don't collide even when both manifests say ``name: openai``.
|
||||||
disabled = _get_disabled_plugins()
|
disabled = _get_disabled_plugins()
|
||||||
enabled = _get_enabled_plugins() # None = opt-in default (nothing enabled)
|
enabled = _get_enabled_plugins() # None = opt-in default (nothing enabled)
|
||||||
winners: Dict[str, PluginManifest] = {}
|
winners: Dict[str, PluginManifest] = {}
|
||||||
for manifest in manifests:
|
for manifest in manifests:
|
||||||
winners[manifest.name] = manifest
|
winners[manifest.key or manifest.name] = manifest
|
||||||
for manifest in winners.values():
|
for manifest in winners.values():
|
||||||
# Explicit disable always wins.
|
lookup_key = manifest.key or manifest.name
|
||||||
if manifest.name in disabled:
|
|
||||||
|
# Explicit disable always wins (matches on key or on legacy
|
||||||
|
# bare name for back-compat with existing user configs).
|
||||||
|
if lookup_key in disabled or manifest.name in disabled:
|
||||||
loaded = LoadedPlugin(manifest=manifest, enabled=False)
|
loaded = LoadedPlugin(manifest=manifest, enabled=False)
|
||||||
loaded.error = "disabled via config"
|
loaded.error = "disabled via config"
|
||||||
self._plugins[manifest.name] = loaded
|
self._plugins[lookup_key] = loaded
|
||||||
logger.debug("Skipping disabled plugin '%s'", manifest.name)
|
logger.debug("Skipping disabled plugin '%s'", lookup_key)
|
||||||
continue
|
continue
|
||||||
# Opt-in gate: plugins must be in the enabled allow-list.
|
|
||||||
# If the allow-list is missing (None), treat as "nothing enabled"
|
# Exclusive plugins (memory providers) have their own
|
||||||
# — users have to explicitly enable plugins to load them.
|
# discovery/activation path. The general loader records the
|
||||||
# Memory and context_engine providers are excluded from this gate
|
# manifest for introspection but does not load the module.
|
||||||
# since they have their own single-select config (memory.provider
|
if manifest.kind == "exclusive":
|
||||||
# / context.engine), not the enabled list.
|
|
||||||
if enabled is None or manifest.name not in enabled:
|
|
||||||
loaded = LoadedPlugin(manifest=manifest, enabled=False)
|
loaded = LoadedPlugin(manifest=manifest, enabled=False)
|
||||||
loaded.error = "not enabled in config (run `hermes plugins enable {}` to activate)".format(
|
loaded.error = (
|
||||||
manifest.name
|
"exclusive plugin — activate via <category>.provider config"
|
||||||
)
|
)
|
||||||
self._plugins[manifest.name] = loaded
|
self._plugins[lookup_key] = loaded
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"Skipping '%s' (not in plugins.enabled)", manifest.name
|
"Skipping '%s' (exclusive, handled by category discovery)",
|
||||||
|
lookup_key,
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Built-in backends auto-load — they ship with hermes and must
|
||||||
|
# just work. Selection among them (e.g. which image_gen backend
|
||||||
|
# services calls) is driven by ``<category>.provider`` config,
|
||||||
|
# enforced by the tool wrapper.
|
||||||
|
if manifest.kind == "backend" and manifest.source == "bundled":
|
||||||
|
self._load_plugin(manifest)
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Everything else (standalone, user-installed backends,
|
||||||
|
# entry-point plugins) is opt-in via plugins.enabled.
|
||||||
|
# Accept both the path-derived key and the legacy bare name
|
||||||
|
# so existing configs keep working.
|
||||||
|
is_enabled = (
|
||||||
|
enabled is not None
|
||||||
|
and (lookup_key in enabled or manifest.name in enabled)
|
||||||
|
)
|
||||||
|
if not is_enabled:
|
||||||
|
loaded = LoadedPlugin(manifest=manifest, enabled=False)
|
||||||
|
loaded.error = (
|
||||||
|
"not enabled in config (run `hermes plugins enable {}` to activate)"
|
||||||
|
.format(lookup_key)
|
||||||
|
)
|
||||||
|
self._plugins[lookup_key] = loaded
|
||||||
|
logger.debug(
|
||||||
|
"Skipping '%s' (not in plugins.enabled)", lookup_key
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
self._load_plugin(manifest)
|
self._load_plugin(manifest)
|
||||||
|
|
@ -545,9 +630,37 @@ class PluginManager:
|
||||||
) -> List[PluginManifest]:
|
) -> List[PluginManifest]:
|
||||||
"""Read ``plugin.yaml`` manifests from subdirectories of *path*.
|
"""Read ``plugin.yaml`` manifests from subdirectories of *path*.
|
||||||
|
|
||||||
*skip_names* is an optional allow-list of names to ignore (used
|
Supports two layouts, mixed freely:
|
||||||
for the bundled scan to exclude ``memory`` / ``context_engine``
|
|
||||||
subdirs that have their own discovery path).
|
* **Flat** — ``<root>/<plugin-name>/plugin.yaml``. Key is
|
||||||
|
``<plugin-name>`` (e.g. ``disk-cleanup``).
|
||||||
|
* **Category** — ``<root>/<category>/<plugin-name>/plugin.yaml``,
|
||||||
|
where the ``<category>`` directory itself has no ``plugin.yaml``.
|
||||||
|
Key is ``<category>/<plugin-name>`` (e.g. ``image_gen/openai``).
|
||||||
|
Depth is capped at two segments.
|
||||||
|
|
||||||
|
*skip_names* is an optional allow-list of names to ignore at the
|
||||||
|
top level (kept for back-compat; the current call sites no longer
|
||||||
|
pass it now that categories are first-class).
|
||||||
|
"""
|
||||||
|
return self._scan_directory_level(
|
||||||
|
path, source, skip_names=skip_names, prefix="", depth=0
|
||||||
|
)
|
||||||
|
|
||||||
|
def _scan_directory_level(
|
||||||
|
self,
|
||||||
|
path: Path,
|
||||||
|
source: str,
|
||||||
|
*,
|
||||||
|
skip_names: Optional[Set[str]],
|
||||||
|
prefix: str,
|
||||||
|
depth: int,
|
||||||
|
) -> List[PluginManifest]:
|
||||||
|
"""Recursive implementation of :meth:`_scan_directory`.
|
||||||
|
|
||||||
|
``prefix`` is the category path already accumulated ("" at root,
|
||||||
|
"image_gen" one level in). ``depth`` is the recursion depth; we
|
||||||
|
cap at 2 so ``<root>/a/b/c/`` is ignored.
|
||||||
"""
|
"""
|
||||||
manifests: List[PluginManifest] = []
|
manifests: List[PluginManifest] = []
|
||||||
if not path.is_dir():
|
if not path.is_dir():
|
||||||
|
|
@ -556,22 +669,73 @@ class PluginManager:
|
||||||
for child in sorted(path.iterdir()):
|
for child in sorted(path.iterdir()):
|
||||||
if not child.is_dir():
|
if not child.is_dir():
|
||||||
continue
|
continue
|
||||||
if skip_names and child.name in skip_names:
|
if depth == 0 and skip_names and child.name in skip_names:
|
||||||
continue
|
continue
|
||||||
manifest_file = child / "plugin.yaml"
|
manifest_file = child / "plugin.yaml"
|
||||||
if not manifest_file.exists():
|
if not manifest_file.exists():
|
||||||
manifest_file = child / "plugin.yml"
|
manifest_file = child / "plugin.yml"
|
||||||
if not manifest_file.exists():
|
|
||||||
logger.debug("Skipping %s (no plugin.yaml)", child)
|
if manifest_file.exists():
|
||||||
|
manifest = self._parse_manifest(
|
||||||
|
manifest_file, child, source, prefix
|
||||||
|
)
|
||||||
|
if manifest is not None:
|
||||||
|
manifests.append(manifest)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
# No manifest at this level. If we're still within the depth
|
||||||
|
# cap, treat this directory as a category namespace and recurse
|
||||||
|
# one level in looking for children with manifests.
|
||||||
|
if depth >= 1:
|
||||||
|
logger.debug("Skipping %s (no plugin.yaml, depth cap reached)", child)
|
||||||
|
continue
|
||||||
|
|
||||||
|
sub_prefix = f"{prefix}/{child.name}" if prefix else child.name
|
||||||
|
manifests.extend(
|
||||||
|
self._scan_directory_level(
|
||||||
|
child,
|
||||||
|
source,
|
||||||
|
skip_names=None,
|
||||||
|
prefix=sub_prefix,
|
||||||
|
depth=depth + 1,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
return manifests
|
||||||
|
|
||||||
|
def _parse_manifest(
|
||||||
|
self,
|
||||||
|
manifest_file: Path,
|
||||||
|
plugin_dir: Path,
|
||||||
|
source: str,
|
||||||
|
prefix: str,
|
||||||
|
) -> Optional[PluginManifest]:
|
||||||
|
"""Parse a single ``plugin.yaml`` into a :class:`PluginManifest`.
|
||||||
|
|
||||||
|
Returns ``None`` on parse failure (logs a warning).
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
if yaml is None:
|
if yaml is None:
|
||||||
logger.warning("PyYAML not installed – cannot load %s", manifest_file)
|
logger.warning("PyYAML not installed – cannot load %s", manifest_file)
|
||||||
continue
|
return None
|
||||||
data = yaml.safe_load(manifest_file.read_text()) or {}
|
data = yaml.safe_load(manifest_file.read_text()) or {}
|
||||||
manifest = PluginManifest(
|
|
||||||
name=data.get("name", child.name),
|
name = data.get("name", plugin_dir.name)
|
||||||
|
key = f"{prefix}/{plugin_dir.name}" if prefix else name
|
||||||
|
|
||||||
|
raw_kind = data.get("kind", "standalone")
|
||||||
|
if not isinstance(raw_kind, str):
|
||||||
|
raw_kind = "standalone"
|
||||||
|
kind = raw_kind.strip().lower()
|
||||||
|
if kind not in _VALID_PLUGIN_KINDS:
|
||||||
|
logger.warning(
|
||||||
|
"Plugin %s: unknown kind '%s' (valid: %s); treating as 'standalone'",
|
||||||
|
key, raw_kind, ", ".join(sorted(_VALID_PLUGIN_KINDS)),
|
||||||
|
)
|
||||||
|
kind = "standalone"
|
||||||
|
|
||||||
|
return PluginManifest(
|
||||||
|
name=name,
|
||||||
version=str(data.get("version", "")),
|
version=str(data.get("version", "")),
|
||||||
description=data.get("description", ""),
|
description=data.get("description", ""),
|
||||||
author=data.get("author", ""),
|
author=data.get("author", ""),
|
||||||
|
|
@ -579,13 +743,13 @@ class PluginManager:
|
||||||
provides_tools=data.get("provides_tools", []),
|
provides_tools=data.get("provides_tools", []),
|
||||||
provides_hooks=data.get("provides_hooks", []),
|
provides_hooks=data.get("provides_hooks", []),
|
||||||
source=source,
|
source=source,
|
||||||
path=str(child),
|
path=str(plugin_dir),
|
||||||
|
kind=kind,
|
||||||
|
key=key,
|
||||||
)
|
)
|
||||||
manifests.append(manifest)
|
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.warning("Failed to parse %s: %s", manifest_file, exc)
|
logger.warning("Failed to parse %s: %s", manifest_file, exc)
|
||||||
|
return None
|
||||||
return manifests
|
|
||||||
|
|
||||||
# -----------------------------------------------------------------------
|
# -----------------------------------------------------------------------
|
||||||
# Entry-point scanning
|
# Entry-point scanning
|
||||||
|
|
@ -609,6 +773,7 @@ class PluginManager:
|
||||||
name=ep.name,
|
name=ep.name,
|
||||||
source="entrypoint",
|
source="entrypoint",
|
||||||
path=ep.value,
|
path=ep.value,
|
||||||
|
key=ep.name,
|
||||||
)
|
)
|
||||||
manifests.append(manifest)
|
manifests.append(manifest)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
|
|
@ -670,10 +835,16 @@ class PluginManager:
|
||||||
loaded.error = str(exc)
|
loaded.error = str(exc)
|
||||||
logger.warning("Failed to load plugin '%s': %s", manifest.name, exc)
|
logger.warning("Failed to load plugin '%s': %s", manifest.name, exc)
|
||||||
|
|
||||||
self._plugins[manifest.name] = loaded
|
self._plugins[manifest.key or manifest.name] = loaded
|
||||||
|
|
||||||
def _load_directory_module(self, manifest: PluginManifest) -> types.ModuleType:
|
def _load_directory_module(self, manifest: PluginManifest) -> types.ModuleType:
|
||||||
"""Import a directory-based plugin as ``hermes_plugins.<name>``."""
|
"""Import a directory-based plugin as ``hermes_plugins.<slug>``.
|
||||||
|
|
||||||
|
The module slug is derived from ``manifest.key`` so category-namespaced
|
||||||
|
plugins (``image_gen/openai``) import as
|
||||||
|
``hermes_plugins.image_gen__openai`` without colliding with any
|
||||||
|
future ``tts/openai``.
|
||||||
|
"""
|
||||||
plugin_dir = Path(manifest.path) # type: ignore[arg-type]
|
plugin_dir = Path(manifest.path) # type: ignore[arg-type]
|
||||||
init_file = plugin_dir / "__init__.py"
|
init_file = plugin_dir / "__init__.py"
|
||||||
if not init_file.exists():
|
if not init_file.exists():
|
||||||
|
|
@ -686,7 +857,9 @@ class PluginManager:
|
||||||
ns_pkg.__package__ = _NS_PARENT
|
ns_pkg.__package__ = _NS_PARENT
|
||||||
sys.modules[_NS_PARENT] = ns_pkg
|
sys.modules[_NS_PARENT] = ns_pkg
|
||||||
|
|
||||||
module_name = f"{_NS_PARENT}.{manifest.name.replace('-', '_')}"
|
key = manifest.key or manifest.name
|
||||||
|
slug = key.replace("/", "__").replace("-", "_")
|
||||||
|
module_name = f"{_NS_PARENT}.{slug}"
|
||||||
spec = importlib.util.spec_from_file_location(
|
spec = importlib.util.spec_from_file_location(
|
||||||
module_name,
|
module_name,
|
||||||
init_file,
|
init_file,
|
||||||
|
|
@ -767,10 +940,12 @@ class PluginManager:
|
||||||
def list_plugins(self) -> List[Dict[str, Any]]:
|
def list_plugins(self) -> List[Dict[str, Any]]:
|
||||||
"""Return a list of info dicts for all discovered plugins."""
|
"""Return a list of info dicts for all discovered plugins."""
|
||||||
result: List[Dict[str, Any]] = []
|
result: List[Dict[str, Any]] = []
|
||||||
for name, loaded in sorted(self._plugins.items()):
|
for key, loaded in sorted(self._plugins.items()):
|
||||||
result.append(
|
result.append(
|
||||||
{
|
{
|
||||||
"name": name,
|
"name": loaded.manifest.name,
|
||||||
|
"key": loaded.manifest.key or loaded.manifest.name,
|
||||||
|
"kind": loaded.manifest.kind,
|
||||||
"version": loaded.manifest.version,
|
"version": loaded.manifest.version,
|
||||||
"description": loaded.manifest.description,
|
"description": loaded.manifest.description,
|
||||||
"source": loaded.manifest.source,
|
"source": loaded.manifest.source,
|
||||||
|
|
|
||||||
|
|
@ -408,13 +408,36 @@ def _print_setup_summary(config: dict, hermes_home):
|
||||||
("Browser Automation", False, missing_browser_hint)
|
("Browser Automation", False, missing_browser_hint)
|
||||||
)
|
)
|
||||||
|
|
||||||
# FAL (image generation)
|
# Image generation — FAL (direct or via Nous), or any plugin-registered
|
||||||
|
# provider (OpenAI, etc.)
|
||||||
if subscription_features.image_gen.managed_by_nous:
|
if subscription_features.image_gen.managed_by_nous:
|
||||||
tool_status.append(("Image Generation (Nous subscription)", True, None))
|
tool_status.append(("Image Generation (Nous subscription)", True, None))
|
||||||
elif subscription_features.image_gen.available:
|
elif subscription_features.image_gen.available:
|
||||||
tool_status.append(("Image Generation", True, None))
|
tool_status.append(("Image Generation", True, None))
|
||||||
else:
|
else:
|
||||||
tool_status.append(("Image Generation", False, "FAL_KEY"))
|
# Fall back to probing plugin-registered providers so OpenAI-only
|
||||||
|
# setups don't show as "missing FAL_KEY".
|
||||||
|
_img_backend = None
|
||||||
|
try:
|
||||||
|
from agent.image_gen_registry import list_providers
|
||||||
|
from hermes_cli.plugins import _ensure_plugins_discovered
|
||||||
|
|
||||||
|
_ensure_plugins_discovered()
|
||||||
|
for _p in list_providers():
|
||||||
|
if _p.name == "fal":
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
if _p.is_available():
|
||||||
|
_img_backend = _p.display_name
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if _img_backend:
|
||||||
|
tool_status.append((f"Image Generation ({_img_backend})", True, None))
|
||||||
|
else:
|
||||||
|
tool_status.append(("Image Generation", False, "FAL_KEY or OPENAI_API_KEY"))
|
||||||
|
|
||||||
# TTS — show configured provider
|
# TTS — show configured provider
|
||||||
tts_provider = config.get("tts", {}).get("provider", "edge")
|
tts_provider = config.get("tts", {}).get("provider", "edge")
|
||||||
|
|
|
||||||
|
|
@ -847,6 +847,51 @@ def _configure_toolset(ts_key: str, config: dict):
|
||||||
_configure_simple_requirements(ts_key)
|
_configure_simple_requirements(ts_key)
|
||||||
|
|
||||||
|
|
||||||
|
def _plugin_image_gen_providers() -> list[dict]:
|
||||||
|
"""Build picker-row dicts from plugin-registered image gen providers.
|
||||||
|
|
||||||
|
Each returned dict looks like a regular ``TOOL_CATEGORIES`` provider
|
||||||
|
row but carries an ``image_gen_plugin_name`` marker so downstream
|
||||||
|
code (config writing, model picker) knows to route through the
|
||||||
|
plugin registry instead of the in-tree FAL backend.
|
||||||
|
|
||||||
|
FAL is skipped — it's already exposed by the hardcoded
|
||||||
|
``TOOL_CATEGORIES["image_gen"]`` entries. When FAL gets ported to
|
||||||
|
a plugin in a follow-up PR, the hardcoded entries go away and this
|
||||||
|
function surfaces it alongside OpenAI automatically.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from agent.image_gen_registry import list_providers
|
||||||
|
from hermes_cli.plugins import _ensure_plugins_discovered
|
||||||
|
|
||||||
|
_ensure_plugins_discovered()
|
||||||
|
providers = list_providers()
|
||||||
|
except Exception:
|
||||||
|
return []
|
||||||
|
|
||||||
|
rows: list[dict] = []
|
||||||
|
for provider in providers:
|
||||||
|
if getattr(provider, "name", None) == "fal":
|
||||||
|
# FAL has its own hardcoded rows today.
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
schema = provider.get_setup_schema()
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if not isinstance(schema, dict):
|
||||||
|
continue
|
||||||
|
rows.append(
|
||||||
|
{
|
||||||
|
"name": schema.get("name", provider.display_name),
|
||||||
|
"badge": schema.get("badge", ""),
|
||||||
|
"tag": schema.get("tag", ""),
|
||||||
|
"env_vars": schema.get("env_vars", []),
|
||||||
|
"image_gen_plugin_name": provider.name,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
def _visible_providers(cat: dict, config: dict) -> list[dict]:
|
def _visible_providers(cat: dict, config: dict) -> list[dict]:
|
||||||
"""Return provider entries visible for the current auth/config state."""
|
"""Return provider entries visible for the current auth/config state."""
|
||||||
features = get_nous_subscription_features(config)
|
features = get_nous_subscription_features(config)
|
||||||
|
|
@ -857,6 +902,12 @@ def _visible_providers(cat: dict, config: dict) -> list[dict]:
|
||||||
if provider.get("requires_nous_auth") and not features.nous_auth_present:
|
if provider.get("requires_nous_auth") and not features.nous_auth_present:
|
||||||
continue
|
continue
|
||||||
visible.append(provider)
|
visible.append(provider)
|
||||||
|
|
||||||
|
# Inject plugin-registered image_gen backends (OpenAI today, more
|
||||||
|
# later) so the picker lists them alongside FAL / Nous Subscription.
|
||||||
|
if cat.get("name") == "Image Generation":
|
||||||
|
visible.extend(_plugin_image_gen_providers())
|
||||||
|
|
||||||
return visible
|
return visible
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -876,7 +927,24 @@ def _toolset_needs_configuration_prompt(ts_key: str, config: dict) -> bool:
|
||||||
browser_cfg = config.get("browser", {})
|
browser_cfg = config.get("browser", {})
|
||||||
return not isinstance(browser_cfg, dict) or "cloud_provider" not in browser_cfg
|
return not isinstance(browser_cfg, dict) or "cloud_provider" not in browser_cfg
|
||||||
if ts_key == "image_gen":
|
if ts_key == "image_gen":
|
||||||
return not fal_key_is_configured()
|
# Satisfied when the in-tree FAL backend is configured OR any
|
||||||
|
# plugin-registered image gen provider is available.
|
||||||
|
if fal_key_is_configured():
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
from agent.image_gen_registry import list_providers
|
||||||
|
from hermes_cli.plugins import _ensure_plugins_discovered
|
||||||
|
|
||||||
|
_ensure_plugins_discovered()
|
||||||
|
for provider in list_providers():
|
||||||
|
try:
|
||||||
|
if provider.is_available():
|
||||||
|
return False
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return True
|
||||||
|
|
||||||
return not _toolset_has_keys(ts_key, config)
|
return not _toolset_has_keys(ts_key, config)
|
||||||
|
|
||||||
|
|
@ -1095,6 +1163,88 @@ def _configure_imagegen_model(backend_name: str, config: dict) -> None:
|
||||||
_print_success(f" Model set to: {chosen}")
|
_print_success(f" Model set to: {chosen}")
|
||||||
|
|
||||||
|
|
||||||
|
def _plugin_image_gen_catalog(plugin_name: str):
|
||||||
|
"""Return ``(catalog_dict, default_model_id)`` for a plugin provider.
|
||||||
|
|
||||||
|
``catalog_dict`` is shaped like the legacy ``FAL_MODELS`` table —
|
||||||
|
``{model_id: {"display", "speed", "strengths", "price", ...}}`` —
|
||||||
|
so the existing picker code paths work without change. Returns
|
||||||
|
``({}, None)`` if the provider isn't registered or has no models.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from agent.image_gen_registry import get_provider
|
||||||
|
from hermes_cli.plugins import _ensure_plugins_discovered
|
||||||
|
|
||||||
|
_ensure_plugins_discovered()
|
||||||
|
provider = get_provider(plugin_name)
|
||||||
|
except Exception:
|
||||||
|
return {}, None
|
||||||
|
if provider is None:
|
||||||
|
return {}, None
|
||||||
|
try:
|
||||||
|
models = provider.list_models() or []
|
||||||
|
default = provider.default_model()
|
||||||
|
except Exception:
|
||||||
|
return {}, None
|
||||||
|
catalog = {m["id"]: m for m in models if isinstance(m, dict) and "id" in m}
|
||||||
|
return catalog, default
|
||||||
|
|
||||||
|
|
||||||
|
def _configure_imagegen_model_for_plugin(plugin_name: str, config: dict) -> None:
|
||||||
|
"""Prompt the user to pick a model for a plugin-registered backend.
|
||||||
|
|
||||||
|
Writes selection to ``image_gen.model``. Mirrors
|
||||||
|
:func:`_configure_imagegen_model` but sources its catalog from the
|
||||||
|
plugin registry instead of :data:`IMAGEGEN_BACKENDS`.
|
||||||
|
"""
|
||||||
|
catalog, default_model = _plugin_image_gen_catalog(plugin_name)
|
||||||
|
if not catalog:
|
||||||
|
return
|
||||||
|
|
||||||
|
cur_cfg = config.setdefault("image_gen", {})
|
||||||
|
if not isinstance(cur_cfg, dict):
|
||||||
|
cur_cfg = {}
|
||||||
|
config["image_gen"] = cur_cfg
|
||||||
|
current_model = cur_cfg.get("model") or default_model
|
||||||
|
if current_model not in catalog:
|
||||||
|
current_model = default_model
|
||||||
|
|
||||||
|
model_ids = list(catalog.keys())
|
||||||
|
ordered = [current_model] + [m for m in model_ids if m != current_model]
|
||||||
|
|
||||||
|
widths = {
|
||||||
|
"model": max(len(m) for m in model_ids),
|
||||||
|
"speed": max((len(catalog[m].get("speed", "")) for m in model_ids), default=6),
|
||||||
|
"strengths": max((len(catalog[m].get("strengths", "")) for m in model_ids), default=0),
|
||||||
|
}
|
||||||
|
|
||||||
|
print()
|
||||||
|
header = (
|
||||||
|
f" {'Model':<{widths['model']}} "
|
||||||
|
f"{'Speed':<{widths['speed']}} "
|
||||||
|
f"{'Strengths':<{widths['strengths']}} "
|
||||||
|
f"Price"
|
||||||
|
)
|
||||||
|
print(color(header, Colors.CYAN))
|
||||||
|
|
||||||
|
rows = []
|
||||||
|
for mid in ordered:
|
||||||
|
row = _format_imagegen_model_row(mid, catalog[mid], widths)
|
||||||
|
if mid == current_model:
|
||||||
|
row += " ← currently in use"
|
||||||
|
rows.append(row)
|
||||||
|
|
||||||
|
idx = _prompt_choice(
|
||||||
|
f" Choose {plugin_name} model:",
|
||||||
|
rows,
|
||||||
|
default=0,
|
||||||
|
)
|
||||||
|
|
||||||
|
chosen = ordered[idx]
|
||||||
|
cur_cfg["model"] = chosen
|
||||||
|
_print_success(f" Model set to: {chosen}")
|
||||||
|
|
||||||
|
|
||||||
def _configure_provider(provider: dict, config: dict):
|
def _configure_provider(provider: dict, config: dict):
|
||||||
"""Configure a single provider - prompt for API keys and set config."""
|
"""Configure a single provider - prompt for API keys and set config."""
|
||||||
env_vars = provider.get("env_vars", [])
|
env_vars = provider.get("env_vars", [])
|
||||||
|
|
@ -1151,10 +1301,28 @@ def _configure_provider(provider: dict, config: dict):
|
||||||
_print_success(f" {provider['name']} - no configuration needed!")
|
_print_success(f" {provider['name']} - no configuration needed!")
|
||||||
if managed_feature:
|
if managed_feature:
|
||||||
_print_info(" Requests for this tool will be billed to your Nous subscription.")
|
_print_info(" Requests for this tool will be billed to your Nous subscription.")
|
||||||
|
# Plugin-registered image_gen provider: write image_gen.provider
|
||||||
|
# and route model selection to the plugin's own catalog.
|
||||||
|
plugin_name = provider.get("image_gen_plugin_name")
|
||||||
|
if plugin_name:
|
||||||
|
img_cfg = config.setdefault("image_gen", {})
|
||||||
|
if not isinstance(img_cfg, dict):
|
||||||
|
img_cfg = {}
|
||||||
|
config["image_gen"] = img_cfg
|
||||||
|
img_cfg["provider"] = plugin_name
|
||||||
|
_print_success(f" image_gen.provider set to: {plugin_name}")
|
||||||
|
_configure_imagegen_model_for_plugin(plugin_name, config)
|
||||||
|
return
|
||||||
# Imagegen backends prompt for model selection after backend pick.
|
# Imagegen backends prompt for model selection after backend pick.
|
||||||
backend = provider.get("imagegen_backend")
|
backend = provider.get("imagegen_backend")
|
||||||
if backend:
|
if backend:
|
||||||
_configure_imagegen_model(backend, config)
|
_configure_imagegen_model(backend, config)
|
||||||
|
# In-tree FAL is the only non-plugin backend today. Keep
|
||||||
|
# image_gen.provider clear so the dispatch shim falls through
|
||||||
|
# to the legacy FAL path.
|
||||||
|
img_cfg = config.setdefault("image_gen", {})
|
||||||
|
if isinstance(img_cfg, dict) and img_cfg.get("provider") not in (None, "", "fal"):
|
||||||
|
img_cfg["provider"] = "fal"
|
||||||
return
|
return
|
||||||
|
|
||||||
# Prompt for each required env var
|
# Prompt for each required env var
|
||||||
|
|
@ -1189,10 +1357,23 @@ def _configure_provider(provider: dict, config: dict):
|
||||||
|
|
||||||
if all_configured:
|
if all_configured:
|
||||||
_print_success(f" {provider['name']} configured!")
|
_print_success(f" {provider['name']} configured!")
|
||||||
|
plugin_name = provider.get("image_gen_plugin_name")
|
||||||
|
if plugin_name:
|
||||||
|
img_cfg = config.setdefault("image_gen", {})
|
||||||
|
if not isinstance(img_cfg, dict):
|
||||||
|
img_cfg = {}
|
||||||
|
config["image_gen"] = img_cfg
|
||||||
|
img_cfg["provider"] = plugin_name
|
||||||
|
_print_success(f" image_gen.provider set to: {plugin_name}")
|
||||||
|
_configure_imagegen_model_for_plugin(plugin_name, config)
|
||||||
|
return
|
||||||
# Imagegen backends prompt for model selection after env vars are in.
|
# Imagegen backends prompt for model selection after env vars are in.
|
||||||
backend = provider.get("imagegen_backend")
|
backend = provider.get("imagegen_backend")
|
||||||
if backend:
|
if backend:
|
||||||
_configure_imagegen_model(backend, config)
|
_configure_imagegen_model(backend, config)
|
||||||
|
img_cfg = config.setdefault("image_gen", {})
|
||||||
|
if isinstance(img_cfg, dict) and img_cfg.get("provider") not in (None, "", "fal"):
|
||||||
|
img_cfg["provider"] = "fal"
|
||||||
|
|
||||||
|
|
||||||
def _configure_simple_requirements(ts_key: str):
|
def _configure_simple_requirements(ts_key: str):
|
||||||
|
|
|
||||||
303
plugins/image_gen/openai/__init__.py
Normal file
303
plugins/image_gen/openai/__init__.py
Normal file
|
|
@ -0,0 +1,303 @@
|
||||||
|
"""OpenAI image generation backend.
|
||||||
|
|
||||||
|
Exposes OpenAI's ``gpt-image-2`` model at three quality tiers as an
|
||||||
|
:class:`ImageGenProvider` implementation. The tiers are implemented as
|
||||||
|
three virtual model IDs so the ``hermes tools`` model picker and the
|
||||||
|
``image_gen.model`` config key behave like any other multi-model backend:
|
||||||
|
|
||||||
|
gpt-image-2-low ~15s fastest, good for iteration
|
||||||
|
gpt-image-2-medium ~40s default — balanced
|
||||||
|
gpt-image-2-high ~2min slowest, highest fidelity
|
||||||
|
|
||||||
|
All three hit the same underlying API model (``gpt-image-2``) with a
|
||||||
|
different ``quality`` parameter. Output is base64 JSON → saved under
|
||||||
|
``$HERMES_HOME/cache/images/``.
|
||||||
|
|
||||||
|
Selection precedence (first hit wins):
|
||||||
|
|
||||||
|
1. ``OPENAI_IMAGE_MODEL`` env var (escape hatch for scripts / tests)
|
||||||
|
2. ``image_gen.openai.model`` in ``config.yaml``
|
||||||
|
3. ``image_gen.model`` in ``config.yaml`` (when it's one of our tier IDs)
|
||||||
|
4. :data:`DEFAULT_MODEL` — ``gpt-image-2-medium``
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from typing import Any, Dict, List, Optional, Tuple
|
||||||
|
|
||||||
|
from agent.image_gen_provider import (
|
||||||
|
DEFAULT_ASPECT_RATIO,
|
||||||
|
ImageGenProvider,
|
||||||
|
error_response,
|
||||||
|
resolve_aspect_ratio,
|
||||||
|
save_b64_image,
|
||||||
|
success_response,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Model catalog
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# All three IDs resolve to the same underlying API model with a different
|
||||||
|
# ``quality`` setting. ``api_model`` is what gets sent to OpenAI;
|
||||||
|
# ``quality`` is the knob that changes generation time and output fidelity.
|
||||||
|
|
||||||
|
API_MODEL = "gpt-image-2"
|
||||||
|
|
||||||
|
_MODELS: Dict[str, Dict[str, Any]] = {
|
||||||
|
"gpt-image-2-low": {
|
||||||
|
"display": "GPT Image 2 (Low)",
|
||||||
|
"speed": "~15s",
|
||||||
|
"strengths": "Fast iteration, lowest cost",
|
||||||
|
"quality": "low",
|
||||||
|
},
|
||||||
|
"gpt-image-2-medium": {
|
||||||
|
"display": "GPT Image 2 (Medium)",
|
||||||
|
"speed": "~40s",
|
||||||
|
"strengths": "Balanced — default",
|
||||||
|
"quality": "medium",
|
||||||
|
},
|
||||||
|
"gpt-image-2-high": {
|
||||||
|
"display": "GPT Image 2 (High)",
|
||||||
|
"speed": "~2min",
|
||||||
|
"strengths": "Highest fidelity, strongest prompt adherence",
|
||||||
|
"quality": "high",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
DEFAULT_MODEL = "gpt-image-2-medium"
|
||||||
|
|
||||||
|
_SIZES = {
|
||||||
|
"landscape": "1536x1024",
|
||||||
|
"square": "1024x1024",
|
||||||
|
"portrait": "1024x1536",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_openai_config() -> Dict[str, Any]:
|
||||||
|
"""Read ``image_gen`` from config.yaml (returns {} on any failure)."""
|
||||||
|
try:
|
||||||
|
from hermes_cli.config import load_config
|
||||||
|
|
||||||
|
cfg = load_config()
|
||||||
|
section = cfg.get("image_gen") if isinstance(cfg, dict) else None
|
||||||
|
return section if isinstance(section, dict) else {}
|
||||||
|
except Exception as exc:
|
||||||
|
logger.debug("Could not load image_gen config: %s", exc)
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_model() -> Tuple[str, Dict[str, Any]]:
|
||||||
|
"""Decide which tier to use and return ``(model_id, meta)``."""
|
||||||
|
env_override = os.environ.get("OPENAI_IMAGE_MODEL")
|
||||||
|
if env_override and env_override in _MODELS:
|
||||||
|
return env_override, _MODELS[env_override]
|
||||||
|
|
||||||
|
cfg = _load_openai_config()
|
||||||
|
openai_cfg = cfg.get("openai") if isinstance(cfg.get("openai"), dict) else {}
|
||||||
|
candidate: Optional[str] = None
|
||||||
|
if isinstance(openai_cfg, dict):
|
||||||
|
value = openai_cfg.get("model")
|
||||||
|
if isinstance(value, str) and value in _MODELS:
|
||||||
|
candidate = value
|
||||||
|
if candidate is None:
|
||||||
|
top = cfg.get("model")
|
||||||
|
if isinstance(top, str) and top in _MODELS:
|
||||||
|
candidate = top
|
||||||
|
|
||||||
|
if candidate is not None:
|
||||||
|
return candidate, _MODELS[candidate]
|
||||||
|
|
||||||
|
return DEFAULT_MODEL, _MODELS[DEFAULT_MODEL]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Provider
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class OpenAIImageGenProvider(ImageGenProvider):
|
||||||
|
"""OpenAI ``images.generate`` backend — gpt-image-2 at low/medium/high."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "openai"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def display_name(self) -> str:
|
||||||
|
return "OpenAI"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
if not os.environ.get("OPENAI_API_KEY"):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
import openai # noqa: F401
|
||||||
|
except ImportError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
def list_models(self) -> List[Dict[str, Any]]:
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"id": model_id,
|
||||||
|
"display": meta["display"],
|
||||||
|
"speed": meta["speed"],
|
||||||
|
"strengths": meta["strengths"],
|
||||||
|
"price": "varies",
|
||||||
|
}
|
||||||
|
for model_id, meta in _MODELS.items()
|
||||||
|
]
|
||||||
|
|
||||||
|
def default_model(self) -> Optional[str]:
|
||||||
|
return DEFAULT_MODEL
|
||||||
|
|
||||||
|
def get_setup_schema(self) -> Dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"name": "OpenAI",
|
||||||
|
"badge": "paid",
|
||||||
|
"tag": "gpt-image-2 at low/medium/high quality tiers",
|
||||||
|
"env_vars": [
|
||||||
|
{
|
||||||
|
"key": "OPENAI_API_KEY",
|
||||||
|
"prompt": "OpenAI API key",
|
||||||
|
"url": "https://platform.openai.com/api-keys",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
def generate(
|
||||||
|
self,
|
||||||
|
prompt: str,
|
||||||
|
aspect_ratio: str = DEFAULT_ASPECT_RATIO,
|
||||||
|
**kwargs: Any,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
prompt = (prompt or "").strip()
|
||||||
|
aspect = resolve_aspect_ratio(aspect_ratio)
|
||||||
|
|
||||||
|
if not prompt:
|
||||||
|
return error_response(
|
||||||
|
error="Prompt is required and must be a non-empty string",
|
||||||
|
error_type="invalid_argument",
|
||||||
|
provider="openai",
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not os.environ.get("OPENAI_API_KEY"):
|
||||||
|
return error_response(
|
||||||
|
error=(
|
||||||
|
"OPENAI_API_KEY not set. Run `hermes tools` → Image "
|
||||||
|
"Generation → OpenAI to configure, or `hermes setup` "
|
||||||
|
"to add the key."
|
||||||
|
),
|
||||||
|
error_type="auth_required",
|
||||||
|
provider="openai",
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
import openai
|
||||||
|
except ImportError:
|
||||||
|
return error_response(
|
||||||
|
error="openai Python package not installed (pip install openai)",
|
||||||
|
error_type="missing_dependency",
|
||||||
|
provider="openai",
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
)
|
||||||
|
|
||||||
|
tier_id, meta = _resolve_model()
|
||||||
|
size = _SIZES.get(aspect, _SIZES["square"])
|
||||||
|
|
||||||
|
# gpt-image-2 returns b64_json unconditionally and REJECTS
|
||||||
|
# ``response_format`` as an unknown parameter. Don't send it.
|
||||||
|
payload: Dict[str, Any] = {
|
||||||
|
"model": API_MODEL,
|
||||||
|
"prompt": prompt,
|
||||||
|
"size": size,
|
||||||
|
"n": 1,
|
||||||
|
"quality": meta["quality"],
|
||||||
|
}
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = openai.OpenAI()
|
||||||
|
response = client.images.generate(**payload)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.debug("OpenAI image generation failed", exc_info=True)
|
||||||
|
return error_response(
|
||||||
|
error=f"OpenAI image generation failed: {exc}",
|
||||||
|
error_type="api_error",
|
||||||
|
provider="openai",
|
||||||
|
model=tier_id,
|
||||||
|
prompt=prompt,
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
)
|
||||||
|
|
||||||
|
data = getattr(response, "data", None) or []
|
||||||
|
if not data:
|
||||||
|
return error_response(
|
||||||
|
error="OpenAI returned no image data",
|
||||||
|
error_type="empty_response",
|
||||||
|
provider="openai",
|
||||||
|
model=tier_id,
|
||||||
|
prompt=prompt,
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
)
|
||||||
|
|
||||||
|
first = data[0]
|
||||||
|
b64 = getattr(first, "b64_json", None)
|
||||||
|
url = getattr(first, "url", None)
|
||||||
|
revised_prompt = getattr(first, "revised_prompt", None)
|
||||||
|
|
||||||
|
if b64:
|
||||||
|
try:
|
||||||
|
saved_path = save_b64_image(b64, prefix=f"openai_{tier_id}")
|
||||||
|
except Exception as exc:
|
||||||
|
return error_response(
|
||||||
|
error=f"Could not save image to cache: {exc}",
|
||||||
|
error_type="io_error",
|
||||||
|
provider="openai",
|
||||||
|
model=tier_id,
|
||||||
|
prompt=prompt,
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
)
|
||||||
|
image_ref = str(saved_path)
|
||||||
|
elif url:
|
||||||
|
# Defensive — gpt-image-2 returns b64 today, but fall back
|
||||||
|
# gracefully if the API ever changes.
|
||||||
|
image_ref = url
|
||||||
|
else:
|
||||||
|
return error_response(
|
||||||
|
error="OpenAI response contained neither b64_json nor URL",
|
||||||
|
error_type="empty_response",
|
||||||
|
provider="openai",
|
||||||
|
model=tier_id,
|
||||||
|
prompt=prompt,
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
)
|
||||||
|
|
||||||
|
extra: Dict[str, Any] = {"size": size, "quality": meta["quality"]}
|
||||||
|
if revised_prompt:
|
||||||
|
extra["revised_prompt"] = revised_prompt
|
||||||
|
|
||||||
|
return success_response(
|
||||||
|
image=image_ref,
|
||||||
|
model=tier_id,
|
||||||
|
prompt=prompt,
|
||||||
|
aspect_ratio=aspect,
|
||||||
|
provider="openai",
|
||||||
|
extra=extra,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Plugin entry point
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Plugin entry point — wire ``OpenAIImageGenProvider`` into the registry."""
|
||||||
|
ctx.register_image_gen_provider(OpenAIImageGenProvider())
|
||||||
7
plugins/image_gen/openai/plugin.yaml
Normal file
7
plugins/image_gen/openai/plugin.yaml
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
name: openai
|
||||||
|
version: 1.0.0
|
||||||
|
description: "OpenAI image generation backend (gpt-image-2). Saves generated images to $HERMES_HOME/cache/images/."
|
||||||
|
author: NousResearch
|
||||||
|
kind: backend
|
||||||
|
requires_env:
|
||||||
|
- OPENAI_API_KEY
|
||||||
111
tests/agent/test_image_gen_registry.py
Normal file
111
tests/agent/test_image_gen_registry.py
Normal file
|
|
@ -0,0 +1,111 @@
|
||||||
|
"""Tests for agent/image_gen_registry.py — provider registration & active lookup."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from agent import image_gen_registry
|
||||||
|
from agent.image_gen_provider import ImageGenProvider
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeProvider(ImageGenProvider):
|
||||||
|
def __init__(self, name: str, available: bool = True):
|
||||||
|
self._name = name
|
||||||
|
self._available = available
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return self._name
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return self._available
|
||||||
|
|
||||||
|
def generate(self, prompt, aspect_ratio="landscape", **kw):
|
||||||
|
return {"success": True, "image": f"{self._name}://{prompt}"}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _reset_registry():
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
|
yield
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
class TestRegisterProvider:
|
||||||
|
def test_register_and_lookup(self):
|
||||||
|
provider = _FakeProvider("fake")
|
||||||
|
image_gen_registry.register_provider(provider)
|
||||||
|
assert image_gen_registry.get_provider("fake") is provider
|
||||||
|
|
||||||
|
def test_rejects_non_provider(self):
|
||||||
|
with pytest.raises(TypeError):
|
||||||
|
image_gen_registry.register_provider("not a provider") # type: ignore[arg-type]
|
||||||
|
|
||||||
|
def test_rejects_empty_name(self):
|
||||||
|
class Empty(ImageGenProvider):
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def generate(self, prompt, aspect_ratio="landscape", **kw):
|
||||||
|
return {}
|
||||||
|
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
image_gen_registry.register_provider(Empty())
|
||||||
|
|
||||||
|
def test_reregister_overwrites(self):
|
||||||
|
a = _FakeProvider("same")
|
||||||
|
b = _FakeProvider("same")
|
||||||
|
image_gen_registry.register_provider(a)
|
||||||
|
image_gen_registry.register_provider(b)
|
||||||
|
assert image_gen_registry.get_provider("same") is b
|
||||||
|
|
||||||
|
def test_list_is_sorted(self):
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("zeta"))
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("alpha"))
|
||||||
|
names = [p.name for p in image_gen_registry.list_providers()]
|
||||||
|
assert names == ["alpha", "zeta"]
|
||||||
|
|
||||||
|
|
||||||
|
class TestGetActiveProvider:
|
||||||
|
def test_single_provider_autoresolves(self, tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("solo"))
|
||||||
|
active = image_gen_registry.get_active_provider()
|
||||||
|
assert active is not None and active.name == "solo"
|
||||||
|
|
||||||
|
def test_fal_preferred_on_multi_without_config(self, tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("fal"))
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("openai"))
|
||||||
|
active = image_gen_registry.get_active_provider()
|
||||||
|
assert active is not None and active.name == "fal"
|
||||||
|
|
||||||
|
def test_explicit_config_wins(self, tmp_path, monkeypatch):
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
(tmp_path / "config.yaml").write_text(
|
||||||
|
yaml.safe_dump({"image_gen": {"provider": "openai"}})
|
||||||
|
)
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("fal"))
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("openai"))
|
||||||
|
active = image_gen_registry.get_active_provider()
|
||||||
|
assert active is not None and active.name == "openai"
|
||||||
|
|
||||||
|
def test_missing_configured_provider_falls_back(self, tmp_path, monkeypatch):
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
(tmp_path / "config.yaml").write_text(
|
||||||
|
yaml.safe_dump({"image_gen": {"provider": "replicate"}})
|
||||||
|
)
|
||||||
|
# Only FAL is registered — configured provider doesn't exist
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("fal"))
|
||||||
|
active = image_gen_registry.get_active_provider()
|
||||||
|
# Falls back to FAL preference (legacy default) rather than None
|
||||||
|
assert active is not None and active.name == "fal"
|
||||||
|
|
||||||
|
def test_none_when_empty(self, tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
assert image_gen_registry.get_active_provider() is None
|
||||||
174
tests/hermes_cli/test_image_gen_picker.py
Normal file
174
tests/hermes_cli/test_image_gen_picker.py
Normal file
|
|
@ -0,0 +1,174 @@
|
||||||
|
"""Tests for plugin image_gen providers injecting themselves into the picker.
|
||||||
|
|
||||||
|
Covers `_plugin_image_gen_providers`, `_visible_providers`, and
|
||||||
|
`_toolset_needs_configuration_prompt` handling of plugin providers.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from agent import image_gen_registry
|
||||||
|
from agent.image_gen_provider import ImageGenProvider
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeProvider(ImageGenProvider):
|
||||||
|
def __init__(self, name: str, available: bool = True, schema=None, models=None):
|
||||||
|
self._name = name
|
||||||
|
self._available = available
|
||||||
|
self._schema = schema or {
|
||||||
|
"name": name.title(),
|
||||||
|
"badge": "test",
|
||||||
|
"tag": f"{name} test tag",
|
||||||
|
"env_vars": [{"key": f"{name.upper()}_API_KEY", "prompt": f"{name} key"}],
|
||||||
|
}
|
||||||
|
self._models = models or [
|
||||||
|
{"id": f"{name}-model-v1", "display": f"{name} v1",
|
||||||
|
"speed": "~5s", "strengths": "test", "price": "$"},
|
||||||
|
]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return self._name
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return self._available
|
||||||
|
|
||||||
|
def list_models(self):
|
||||||
|
return list(self._models)
|
||||||
|
|
||||||
|
def default_model(self):
|
||||||
|
return self._models[0]["id"] if self._models else None
|
||||||
|
|
||||||
|
def get_setup_schema(self):
|
||||||
|
return dict(self._schema)
|
||||||
|
|
||||||
|
def generate(self, prompt, aspect_ratio="landscape", **kw):
|
||||||
|
return {"success": True, "image": f"{self._name}://{prompt}"}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _reset_registry():
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
|
yield
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
|
|
||||||
|
|
||||||
|
class TestPluginPickerInjection:
|
||||||
|
def test_plugin_providers_returns_registered(self, monkeypatch):
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("myimg"))
|
||||||
|
|
||||||
|
rows = tools_config._plugin_image_gen_providers()
|
||||||
|
names = [r["name"] for r in rows]
|
||||||
|
plugin_names = [r.get("image_gen_plugin_name") for r in rows]
|
||||||
|
|
||||||
|
assert "Myimg" in names
|
||||||
|
assert "myimg" in plugin_names
|
||||||
|
|
||||||
|
def test_fal_skipped_to_avoid_duplicate(self, monkeypatch):
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
# Simulate a FAL plugin being registered — the picker already has
|
||||||
|
# hardcoded FAL rows in TOOL_CATEGORIES, so plugin-FAL must be
|
||||||
|
# skipped to avoid showing FAL twice.
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("fal"))
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("openai"))
|
||||||
|
|
||||||
|
rows = tools_config._plugin_image_gen_providers()
|
||||||
|
names = [r.get("image_gen_plugin_name") for r in rows]
|
||||||
|
assert "fal" not in names
|
||||||
|
assert "openai" in names
|
||||||
|
|
||||||
|
def test_visible_providers_includes_plugins_for_image_gen(self, monkeypatch):
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("someimg"))
|
||||||
|
|
||||||
|
cat = tools_config.TOOL_CATEGORIES["image_gen"]
|
||||||
|
visible = tools_config._visible_providers(cat, {})
|
||||||
|
plugin_names = [p.get("image_gen_plugin_name") for p in visible if p.get("image_gen_plugin_name")]
|
||||||
|
assert "someimg" in plugin_names
|
||||||
|
|
||||||
|
def test_visible_providers_does_not_inject_into_other_categories(self, monkeypatch):
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("someimg"))
|
||||||
|
|
||||||
|
# Browser category must NOT see image_gen plugins.
|
||||||
|
browser = tools_config.TOOL_CATEGORIES["browser"]
|
||||||
|
visible = tools_config._visible_providers(browser, {})
|
||||||
|
assert all(p.get("image_gen_plugin_name") is None for p in visible)
|
||||||
|
|
||||||
|
|
||||||
|
class TestPluginCatalog:
|
||||||
|
def test_plugin_catalog_returns_models(self):
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("catimg"))
|
||||||
|
|
||||||
|
catalog, default = tools_config._plugin_image_gen_catalog("catimg")
|
||||||
|
assert "catimg-model-v1" in catalog
|
||||||
|
assert default == "catimg-model-v1"
|
||||||
|
|
||||||
|
def test_plugin_catalog_empty_for_unknown(self):
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
catalog, default = tools_config._plugin_image_gen_catalog("does-not-exist")
|
||||||
|
assert catalog == {}
|
||||||
|
assert default is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestConfigPrompt:
|
||||||
|
def test_image_gen_satisfied_by_plugin_provider(self, monkeypatch, tmp_path):
|
||||||
|
"""When a plugin provider reports is_available(), the picker should
|
||||||
|
not force a setup prompt on the user."""
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
monkeypatch.delenv("FAL_KEY", raising=False)
|
||||||
|
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("avail-img", available=True))
|
||||||
|
|
||||||
|
assert tools_config._toolset_needs_configuration_prompt("image_gen", {}) is False
|
||||||
|
|
||||||
|
def test_image_gen_still_prompts_when_nothing_available(self, monkeypatch, tmp_path):
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
monkeypatch.delenv("FAL_KEY", raising=False)
|
||||||
|
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("unavail-img", available=False))
|
||||||
|
|
||||||
|
assert tools_config._toolset_needs_configuration_prompt("image_gen", {}) is True
|
||||||
|
|
||||||
|
|
||||||
|
class TestConfigWriting:
|
||||||
|
def test_picking_plugin_provider_writes_provider_and_model(self, monkeypatch, tmp_path):
|
||||||
|
"""When a user picks a plugin-backed image_gen provider with no
|
||||||
|
env vars needed, ``_configure_provider`` should write both
|
||||||
|
``image_gen.provider`` and ``image_gen.model``."""
|
||||||
|
from hermes_cli import tools_config
|
||||||
|
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
image_gen_registry.register_provider(_FakeProvider("noenv", schema={
|
||||||
|
"name": "NoEnv",
|
||||||
|
"badge": "free",
|
||||||
|
"tag": "",
|
||||||
|
"env_vars": [],
|
||||||
|
}))
|
||||||
|
|
||||||
|
# Stub out the interactive model picker — no TTY in tests.
|
||||||
|
monkeypatch.setattr(tools_config, "_prompt_choice", lambda *a, **kw: 0)
|
||||||
|
|
||||||
|
config: dict = {}
|
||||||
|
provider_row = {
|
||||||
|
"name": "NoEnv",
|
||||||
|
"env_vars": [],
|
||||||
|
"image_gen_plugin_name": "noenv",
|
||||||
|
}
|
||||||
|
tools_config._configure_provider(provider_row, config)
|
||||||
|
|
||||||
|
assert config["image_gen"]["provider"] == "noenv"
|
||||||
|
assert config["image_gen"]["model"] == "noenv-model-v1"
|
||||||
357
tests/hermes_cli/test_plugin_scanner_recursion.py
Normal file
357
tests/hermes_cli/test_plugin_scanner_recursion.py
Normal file
|
|
@ -0,0 +1,357 @@
|
||||||
|
"""Tests for PR1 pluggable image gen: scanner recursion, kinds, path keys.
|
||||||
|
|
||||||
|
Covers ``_scan_directory`` recursion into category namespaces
|
||||||
|
(``plugins/image_gen/openai/``), ``kind`` parsing, path-derived registry
|
||||||
|
keys, and the new gate logic (bundled backends auto-load; user backends
|
||||||
|
still opt-in; exclusive kind skipped; unknown kinds → standalone warning).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from hermes_cli.plugins import PluginManager, PluginManifest
|
||||||
|
|
||||||
|
|
||||||
|
# ── Helpers ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def _write_plugin(
|
||||||
|
root: Path,
|
||||||
|
segments: list[str],
|
||||||
|
*,
|
||||||
|
manifest_extra: Dict[str, Any] | None = None,
|
||||||
|
register_body: str = "pass",
|
||||||
|
) -> Path:
|
||||||
|
"""Create a plugin dir at ``root/<segments...>/`` with plugin.yaml + __init__.py.
|
||||||
|
|
||||||
|
``segments`` lets tests build both flat (``["my-plugin"]``) and
|
||||||
|
category-namespaced (``["image_gen", "openai"]``) layouts.
|
||||||
|
"""
|
||||||
|
plugin_dir = root
|
||||||
|
for seg in segments:
|
||||||
|
plugin_dir = plugin_dir / seg
|
||||||
|
plugin_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
manifest = {
|
||||||
|
"name": segments[-1],
|
||||||
|
"version": "0.1.0",
|
||||||
|
"description": f"Test plugin {'/'.join(segments)}",
|
||||||
|
}
|
||||||
|
if manifest_extra:
|
||||||
|
manifest.update(manifest_extra)
|
||||||
|
(plugin_dir / "plugin.yaml").write_text(yaml.dump(manifest))
|
||||||
|
(plugin_dir / "__init__.py").write_text(
|
||||||
|
f"def register(ctx):\n {register_body}\n"
|
||||||
|
)
|
||||||
|
return plugin_dir
|
||||||
|
|
||||||
|
|
||||||
|
def _enable(hermes_home: Path, name: str) -> None:
|
||||||
|
"""Append ``name`` to ``plugins.enabled`` in ``<hermes_home>/config.yaml``."""
|
||||||
|
cfg_path = hermes_home / "config.yaml"
|
||||||
|
cfg: dict = {}
|
||||||
|
if cfg_path.exists():
|
||||||
|
try:
|
||||||
|
cfg = yaml.safe_load(cfg_path.read_text()) or {}
|
||||||
|
except Exception:
|
||||||
|
cfg = {}
|
||||||
|
plugins_cfg = cfg.setdefault("plugins", {})
|
||||||
|
enabled = plugins_cfg.setdefault("enabled", [])
|
||||||
|
if isinstance(enabled, list) and name not in enabled:
|
||||||
|
enabled.append(name)
|
||||||
|
cfg_path.write_text(yaml.safe_dump(cfg))
|
||||||
|
|
||||||
|
|
||||||
|
# ── Scanner recursion ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestCategoryNamespaceRecursion:
|
||||||
|
def test_category_namespace_discovered(self, tmp_path, monkeypatch):
|
||||||
|
"""``<root>/image_gen/openai/plugin.yaml`` is discovered with key
|
||||||
|
``image_gen/openai`` when the ``image_gen`` parent has no manifest."""
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
user_plugins = hermes_home / "plugins"
|
||||||
|
|
||||||
|
_write_plugin(user_plugins, ["image_gen", "openai"])
|
||||||
|
_enable(hermes_home, "image_gen/openai")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert "image_gen/openai" in mgr._plugins
|
||||||
|
loaded = mgr._plugins["image_gen/openai"]
|
||||||
|
assert loaded.manifest.key == "image_gen/openai"
|
||||||
|
assert loaded.manifest.name == "openai"
|
||||||
|
assert loaded.enabled is True
|
||||||
|
|
||||||
|
def test_flat_plugin_key_matches_name(self, tmp_path, monkeypatch):
|
||||||
|
"""Flat plugins keep their bare name as the key (back-compat)."""
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
user_plugins = hermes_home / "plugins"
|
||||||
|
|
||||||
|
_write_plugin(user_plugins, ["my-plugin"])
|
||||||
|
_enable(hermes_home, "my-plugin")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert "my-plugin" in mgr._plugins
|
||||||
|
assert mgr._plugins["my-plugin"].manifest.key == "my-plugin"
|
||||||
|
|
||||||
|
def test_depth_cap_two(self, tmp_path, monkeypatch):
|
||||||
|
"""Plugins nested three levels deep are not discovered.
|
||||||
|
|
||||||
|
``<root>/a/b/c/plugin.yaml`` should NOT be picked up — cap is
|
||||||
|
two segments.
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
user_plugins = hermes_home / "plugins"
|
||||||
|
|
||||||
|
_write_plugin(user_plugins, ["a", "b", "c"])
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
non_bundled = [
|
||||||
|
k for k, p in mgr._plugins.items()
|
||||||
|
if p.manifest.source != "bundled"
|
||||||
|
]
|
||||||
|
assert non_bundled == []
|
||||||
|
|
||||||
|
def test_category_dir_with_manifest_is_leaf(self, tmp_path, monkeypatch):
|
||||||
|
"""If ``image_gen/plugin.yaml`` exists, ``image_gen`` itself IS the
|
||||||
|
plugin and its children are ignored."""
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
user_plugins = hermes_home / "plugins"
|
||||||
|
|
||||||
|
# parent has a manifest → stop recursing
|
||||||
|
_write_plugin(user_plugins, ["image_gen"])
|
||||||
|
# child also has a manifest — should NOT be found because we stop
|
||||||
|
# at the parent.
|
||||||
|
_write_plugin(user_plugins, ["image_gen", "openai"])
|
||||||
|
_enable(hermes_home, "image_gen")
|
||||||
|
_enable(hermes_home, "image_gen/openai")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
# The bundled plugins/image_gen/openai/ exists in the repo — filter
|
||||||
|
# it out so we're only asserting on the user-dir layout.
|
||||||
|
user_plugins_in_registry = {
|
||||||
|
k for k, p in mgr._plugins.items() if p.manifest.source != "bundled"
|
||||||
|
}
|
||||||
|
assert "image_gen" in user_plugins_in_registry
|
||||||
|
assert "image_gen/openai" not in user_plugins_in_registry
|
||||||
|
|
||||||
|
|
||||||
|
# ── Kind parsing ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestKindField:
|
||||||
|
def test_default_kind_is_standalone(self, tmp_path, monkeypatch):
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
_write_plugin(hermes_home / "plugins", ["p1"])
|
||||||
|
_enable(hermes_home, "p1")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert mgr._plugins["p1"].manifest.kind == "standalone"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("kind", ["backend", "exclusive", "standalone"])
|
||||||
|
def test_valid_kinds_parsed(self, kind, tmp_path, monkeypatch):
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
_write_plugin(
|
||||||
|
hermes_home / "plugins",
|
||||||
|
["p1"],
|
||||||
|
manifest_extra={"kind": kind},
|
||||||
|
)
|
||||||
|
# Not all kinds auto-load, but manifest should parse.
|
||||||
|
_enable(hermes_home, "p1")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert "p1" in mgr._plugins
|
||||||
|
assert mgr._plugins["p1"].manifest.kind == kind
|
||||||
|
|
||||||
|
def test_unknown_kind_falls_back_to_standalone(self, tmp_path, monkeypatch, caplog):
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
_write_plugin(
|
||||||
|
hermes_home / "plugins",
|
||||||
|
["p1"],
|
||||||
|
manifest_extra={"kind": "bogus"},
|
||||||
|
)
|
||||||
|
_enable(hermes_home, "p1")
|
||||||
|
|
||||||
|
with caplog.at_level("WARNING"):
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert mgr._plugins["p1"].manifest.kind == "standalone"
|
||||||
|
assert any(
|
||||||
|
"unknown kind" in rec.getMessage() for rec in caplog.records
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ── Gate logic ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestBackendGate:
|
||||||
|
def test_user_backend_still_gated_by_enabled(self, tmp_path, monkeypatch):
|
||||||
|
"""User-installed ``kind: backend`` plugins still require opt-in —
|
||||||
|
they're not trusted by default."""
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
user_plugins = hermes_home / "plugins"
|
||||||
|
|
||||||
|
_write_plugin(
|
||||||
|
user_plugins,
|
||||||
|
["image_gen", "fancy"],
|
||||||
|
manifest_extra={"kind": "backend"},
|
||||||
|
)
|
||||||
|
# Do NOT opt in.
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
loaded = mgr._plugins["image_gen/fancy"]
|
||||||
|
assert loaded.enabled is False
|
||||||
|
assert "not enabled" in (loaded.error or "")
|
||||||
|
|
||||||
|
def test_user_backend_loads_when_enabled(self, tmp_path, monkeypatch):
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
user_plugins = hermes_home / "plugins"
|
||||||
|
|
||||||
|
_write_plugin(
|
||||||
|
user_plugins,
|
||||||
|
["image_gen", "fancy"],
|
||||||
|
manifest_extra={"kind": "backend"},
|
||||||
|
)
|
||||||
|
_enable(hermes_home, "image_gen/fancy")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert mgr._plugins["image_gen/fancy"].enabled is True
|
||||||
|
|
||||||
|
def test_exclusive_kind_skipped(self, tmp_path, monkeypatch):
|
||||||
|
"""``kind: exclusive`` plugins are recorded but not loaded — the
|
||||||
|
category's own discovery system handles them (memory today)."""
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
_write_plugin(
|
||||||
|
hermes_home / "plugins",
|
||||||
|
["some-backend"],
|
||||||
|
manifest_extra={"kind": "exclusive"},
|
||||||
|
)
|
||||||
|
_enable(hermes_home, "some-backend")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
loaded = mgr._plugins["some-backend"]
|
||||||
|
assert loaded.enabled is False
|
||||||
|
assert "exclusive" in (loaded.error or "")
|
||||||
|
|
||||||
|
|
||||||
|
# ── Bundled backend auto-load (integration with real bundled plugin) ────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestBundledBackendAutoLoad:
|
||||||
|
def test_bundled_image_gen_openai_autoloads(self, tmp_path, monkeypatch):
|
||||||
|
"""The bundled ``plugins/image_gen/openai/`` plugin loads without
|
||||||
|
any opt-in — it's ``kind: backend`` and shipped in-repo."""
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert "image_gen/openai" in mgr._plugins
|
||||||
|
loaded = mgr._plugins["image_gen/openai"]
|
||||||
|
assert loaded.manifest.source == "bundled"
|
||||||
|
assert loaded.manifest.kind == "backend"
|
||||||
|
assert loaded.enabled is True, f"error: {loaded.error}"
|
||||||
|
|
||||||
|
|
||||||
|
# ── PluginContext.register_image_gen_provider ───────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestRegisterImageGenProvider:
|
||||||
|
def test_accepts_valid_provider(self, tmp_path, monkeypatch):
|
||||||
|
from agent import image_gen_registry
|
||||||
|
from agent.image_gen_provider import ImageGenProvider
|
||||||
|
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
|
|
||||||
|
class FakeProvider(ImageGenProvider):
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "fake-test"
|
||||||
|
|
||||||
|
def generate(self, prompt, aspect_ratio="landscape", **kw):
|
||||||
|
return {"success": True, "image": "test://fake"}
|
||||||
|
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
plugin_dir = _write_plugin(
|
||||||
|
hermes_home / "plugins",
|
||||||
|
["my-img-plugin"],
|
||||||
|
register_body=(
|
||||||
|
"from agent.image_gen_provider import ImageGenProvider\n"
|
||||||
|
" class P(ImageGenProvider):\n"
|
||||||
|
" @property\n"
|
||||||
|
" def name(self): return 'fake-ctx'\n"
|
||||||
|
" def generate(self, prompt, aspect_ratio='landscape', **kw):\n"
|
||||||
|
" return {'success': True, 'image': 'x://y'}\n"
|
||||||
|
" ctx.register_image_gen_provider(P())"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
_enable(hermes_home, "my-img-plugin")
|
||||||
|
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
assert mgr._plugins["my-img-plugin"].enabled is True
|
||||||
|
assert image_gen_registry.get_provider("fake-ctx") is not None
|
||||||
|
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
|
|
||||||
|
def test_rejects_non_provider(self, tmp_path, monkeypatch, caplog):
|
||||||
|
from agent import image_gen_registry
|
||||||
|
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
|
|
||||||
|
import os
|
||||||
|
hermes_home = Path(os.environ["HERMES_HOME"]) # set by hermetic conftest fixture
|
||||||
|
_write_plugin(
|
||||||
|
hermes_home / "plugins",
|
||||||
|
["bad-img-plugin"],
|
||||||
|
register_body="ctx.register_image_gen_provider('not a provider')",
|
||||||
|
)
|
||||||
|
_enable(hermes_home, "bad-img-plugin")
|
||||||
|
|
||||||
|
with caplog.at_level("WARNING"):
|
||||||
|
mgr = PluginManager()
|
||||||
|
mgr.discover_and_load()
|
||||||
|
|
||||||
|
# Plugin loaded (register returned normally) but nothing was
|
||||||
|
# registered in the provider registry.
|
||||||
|
assert mgr._plugins["bad-img-plugin"].enabled is True
|
||||||
|
assert image_gen_registry.get_provider("not a provider") is None
|
||||||
|
|
||||||
|
image_gen_registry._reset_for_tests()
|
||||||
0
tests/plugins/image_gen/__init__.py
Normal file
0
tests/plugins/image_gen/__init__.py
Normal file
243
tests/plugins/image_gen/test_openai_provider.py
Normal file
243
tests/plugins/image_gen/test_openai_provider.py
Normal file
|
|
@ -0,0 +1,243 @@
|
||||||
|
"""Tests for the bundled OpenAI image_gen plugin (gpt-image-2, three tiers)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
import plugins.image_gen.openai as openai_plugin
|
||||||
|
|
||||||
|
|
||||||
|
# 1×1 transparent PNG — valid bytes for save_b64_image()
|
||||||
|
_PNG_HEX = (
|
||||||
|
"89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4"
|
||||||
|
"890000000d49444154789c6300010000000500010d0a2db40000000049454e44"
|
||||||
|
"ae426082"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _b64_png() -> str:
|
||||||
|
import base64
|
||||||
|
return base64.b64encode(bytes.fromhex(_PNG_HEX)).decode()
|
||||||
|
|
||||||
|
|
||||||
|
def _fake_response(*, b64=None, url=None, revised_prompt=None):
|
||||||
|
item = SimpleNamespace(b64_json=b64, url=url, revised_prompt=revised_prompt)
|
||||||
|
return SimpleNamespace(data=[item])
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _tmp_hermes_home(tmp_path, monkeypatch):
|
||||||
|
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||||
|
yield tmp_path
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def provider(monkeypatch):
|
||||||
|
monkeypatch.setenv("OPENAI_API_KEY", "test-key")
|
||||||
|
return openai_plugin.OpenAIImageGenProvider()
|
||||||
|
|
||||||
|
|
||||||
|
def _patched_openai(fake_client: MagicMock):
|
||||||
|
fake_openai = MagicMock()
|
||||||
|
fake_openai.OpenAI.return_value = fake_client
|
||||||
|
return patch.dict("sys.modules", {"openai": fake_openai})
|
||||||
|
|
||||||
|
|
||||||
|
# ── Metadata ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestMetadata:
|
||||||
|
def test_name(self, provider):
|
||||||
|
assert provider.name == "openai"
|
||||||
|
|
||||||
|
def test_default_model(self, provider):
|
||||||
|
assert provider.default_model() == "gpt-image-2-medium"
|
||||||
|
|
||||||
|
def test_list_models_three_tiers(self, provider):
|
||||||
|
ids = [m["id"] for m in provider.list_models()]
|
||||||
|
assert ids == ["gpt-image-2-low", "gpt-image-2-medium", "gpt-image-2-high"]
|
||||||
|
|
||||||
|
def test_catalog_entries_have_display_speed_strengths(self, provider):
|
||||||
|
for entry in provider.list_models():
|
||||||
|
assert entry["display"].startswith("GPT Image 2")
|
||||||
|
assert entry["speed"]
|
||||||
|
assert entry["strengths"]
|
||||||
|
|
||||||
|
|
||||||
|
# ── Availability ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestAvailability:
|
||||||
|
def test_no_api_key_unavailable(self, monkeypatch):
|
||||||
|
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
|
||||||
|
assert openai_plugin.OpenAIImageGenProvider().is_available() is False
|
||||||
|
|
||||||
|
def test_api_key_set_available(self, monkeypatch):
|
||||||
|
monkeypatch.setenv("OPENAI_API_KEY", "test")
|
||||||
|
assert openai_plugin.OpenAIImageGenProvider().is_available() is True
|
||||||
|
|
||||||
|
|
||||||
|
# ── Model resolution ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestModelResolution:
|
||||||
|
def test_default_is_medium(self):
|
||||||
|
model_id, meta = openai_plugin._resolve_model()
|
||||||
|
assert model_id == "gpt-image-2-medium"
|
||||||
|
assert meta["quality"] == "medium"
|
||||||
|
|
||||||
|
def test_env_var_override(self, monkeypatch):
|
||||||
|
monkeypatch.setenv("OPENAI_IMAGE_MODEL", "gpt-image-2-high")
|
||||||
|
model_id, meta = openai_plugin._resolve_model()
|
||||||
|
assert model_id == "gpt-image-2-high"
|
||||||
|
assert meta["quality"] == "high"
|
||||||
|
|
||||||
|
def test_env_var_unknown_falls_back(self, monkeypatch):
|
||||||
|
monkeypatch.setenv("OPENAI_IMAGE_MODEL", "bogus-tier")
|
||||||
|
model_id, _ = openai_plugin._resolve_model()
|
||||||
|
assert model_id == openai_plugin.DEFAULT_MODEL
|
||||||
|
|
||||||
|
def test_config_openai_model(self, tmp_path):
|
||||||
|
import yaml
|
||||||
|
(tmp_path / "config.yaml").write_text(
|
||||||
|
yaml.safe_dump({"image_gen": {"openai": {"model": "gpt-image-2-low"}}})
|
||||||
|
)
|
||||||
|
model_id, meta = openai_plugin._resolve_model()
|
||||||
|
assert model_id == "gpt-image-2-low"
|
||||||
|
assert meta["quality"] == "low"
|
||||||
|
|
||||||
|
def test_config_top_level_model(self, tmp_path):
|
||||||
|
"""``image_gen.model: gpt-image-2-high`` also works (top-level)."""
|
||||||
|
import yaml
|
||||||
|
(tmp_path / "config.yaml").write_text(
|
||||||
|
yaml.safe_dump({"image_gen": {"model": "gpt-image-2-high"}})
|
||||||
|
)
|
||||||
|
model_id, meta = openai_plugin._resolve_model()
|
||||||
|
assert model_id == "gpt-image-2-high"
|
||||||
|
assert meta["quality"] == "high"
|
||||||
|
|
||||||
|
|
||||||
|
# ── Generate ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class TestGenerate:
|
||||||
|
def test_empty_prompt_rejected(self, provider):
|
||||||
|
result = provider.generate("", aspect_ratio="square")
|
||||||
|
assert result["success"] is False
|
||||||
|
assert result["error_type"] == "invalid_argument"
|
||||||
|
|
||||||
|
def test_missing_api_key(self, monkeypatch):
|
||||||
|
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
|
||||||
|
result = openai_plugin.OpenAIImageGenProvider().generate("a cat")
|
||||||
|
assert result["success"] is False
|
||||||
|
assert result["error_type"] == "auth_required"
|
||||||
|
|
||||||
|
def test_b64_saves_to_cache(self, provider, tmp_path):
|
||||||
|
import base64
|
||||||
|
png_bytes = bytes.fromhex(_PNG_HEX)
|
||||||
|
fake_client = MagicMock()
|
||||||
|
fake_client.images.generate.return_value = _fake_response(b64=_b64_png())
|
||||||
|
|
||||||
|
with _patched_openai(fake_client):
|
||||||
|
result = provider.generate("a cat", aspect_ratio="landscape")
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert result["model"] == "gpt-image-2-medium"
|
||||||
|
assert result["aspect_ratio"] == "landscape"
|
||||||
|
assert result["provider"] == "openai"
|
||||||
|
assert result["quality"] == "medium"
|
||||||
|
|
||||||
|
saved = Path(result["image"])
|
||||||
|
assert saved.exists()
|
||||||
|
assert saved.parent == tmp_path / "cache" / "images"
|
||||||
|
assert saved.read_bytes() == png_bytes
|
||||||
|
|
||||||
|
call_kwargs = fake_client.images.generate.call_args.kwargs
|
||||||
|
# All tiers hit the single underlying API model.
|
||||||
|
assert call_kwargs["model"] == "gpt-image-2"
|
||||||
|
assert call_kwargs["quality"] == "medium"
|
||||||
|
assert call_kwargs["size"] == "1536x1024"
|
||||||
|
# gpt-image-2 rejects response_format — we must NOT send it.
|
||||||
|
assert "response_format" not in call_kwargs
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("tier,expected_quality", [
|
||||||
|
("gpt-image-2-low", "low"),
|
||||||
|
("gpt-image-2-medium", "medium"),
|
||||||
|
("gpt-image-2-high", "high"),
|
||||||
|
])
|
||||||
|
def test_tier_maps_to_quality(self, provider, monkeypatch, tier, expected_quality):
|
||||||
|
monkeypatch.setenv("OPENAI_IMAGE_MODEL", tier)
|
||||||
|
fake_client = MagicMock()
|
||||||
|
fake_client.images.generate.return_value = _fake_response(b64=_b64_png())
|
||||||
|
|
||||||
|
with _patched_openai(fake_client):
|
||||||
|
result = provider.generate("a cat")
|
||||||
|
|
||||||
|
assert result["model"] == tier
|
||||||
|
assert result["quality"] == expected_quality
|
||||||
|
assert fake_client.images.generate.call_args.kwargs["quality"] == expected_quality
|
||||||
|
# Always the same underlying API model regardless of tier.
|
||||||
|
assert fake_client.images.generate.call_args.kwargs["model"] == "gpt-image-2"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("aspect,expected_size", [
|
||||||
|
("landscape", "1536x1024"),
|
||||||
|
("square", "1024x1024"),
|
||||||
|
("portrait", "1024x1536"),
|
||||||
|
])
|
||||||
|
def test_aspect_ratio_mapping(self, provider, aspect, expected_size):
|
||||||
|
fake_client = MagicMock()
|
||||||
|
fake_client.images.generate.return_value = _fake_response(b64=_b64_png())
|
||||||
|
|
||||||
|
with _patched_openai(fake_client):
|
||||||
|
provider.generate("a cat", aspect_ratio=aspect)
|
||||||
|
|
||||||
|
assert fake_client.images.generate.call_args.kwargs["size"] == expected_size
|
||||||
|
|
||||||
|
def test_revised_prompt_passed_through(self, provider):
|
||||||
|
fake_client = MagicMock()
|
||||||
|
fake_client.images.generate.return_value = _fake_response(
|
||||||
|
b64=_b64_png(), revised_prompt="A photo of a cat",
|
||||||
|
)
|
||||||
|
|
||||||
|
with _patched_openai(fake_client):
|
||||||
|
result = provider.generate("a cat")
|
||||||
|
|
||||||
|
assert result["revised_prompt"] == "A photo of a cat"
|
||||||
|
|
||||||
|
def test_api_error_returns_error_response(self, provider):
|
||||||
|
fake_client = MagicMock()
|
||||||
|
fake_client.images.generate.side_effect = RuntimeError("boom")
|
||||||
|
|
||||||
|
with _patched_openai(fake_client):
|
||||||
|
result = provider.generate("a cat")
|
||||||
|
|
||||||
|
assert result["success"] is False
|
||||||
|
assert result["error_type"] == "api_error"
|
||||||
|
assert "boom" in result["error"]
|
||||||
|
|
||||||
|
def test_empty_response_data(self, provider):
|
||||||
|
fake_client = MagicMock()
|
||||||
|
fake_client.images.generate.return_value = SimpleNamespace(data=[])
|
||||||
|
|
||||||
|
with _patched_openai(fake_client):
|
||||||
|
result = provider.generate("a cat")
|
||||||
|
|
||||||
|
assert result["success"] is False
|
||||||
|
assert result["error_type"] == "empty_response"
|
||||||
|
|
||||||
|
def test_url_fallback_if_api_changes(self, provider):
|
||||||
|
"""Defensive: if OpenAI ever returns URL instead of b64, pass through."""
|
||||||
|
fake_client = MagicMock()
|
||||||
|
fake_client.images.generate.return_value = _fake_response(
|
||||||
|
b64=None, url="https://example.com/img.png",
|
||||||
|
)
|
||||||
|
|
||||||
|
with _patched_openai(fake_client):
|
||||||
|
result = provider.generate("a cat")
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert result["image"] == "https://example.com/img.png"
|
||||||
|
|
@ -774,13 +774,40 @@ def check_fal_api_key() -> bool:
|
||||||
|
|
||||||
|
|
||||||
def check_image_generation_requirements() -> bool:
|
def check_image_generation_requirements() -> bool:
|
||||||
"""True if FAL credentials and fal_client SDK are both available."""
|
"""True if any image gen backend is available.
|
||||||
|
|
||||||
|
Providers are considered in this order:
|
||||||
|
|
||||||
|
1. The in-tree FAL backend (FAL_KEY or managed gateway).
|
||||||
|
2. Any plugin-registered provider whose ``is_available()`` returns True.
|
||||||
|
|
||||||
|
Plugins win only when the in-tree FAL path is NOT ready, which matches
|
||||||
|
the historical behavior: shipping hermes with a FAL key configured
|
||||||
|
should still expose the tool. The active selection among ready
|
||||||
|
providers is resolved per-call by ``image_gen.provider``.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
if not check_fal_api_key():
|
if check_fal_api_key():
|
||||||
return False
|
|
||||||
fal_client # noqa: F401 — SDK presence check
|
fal_client # noqa: F401 — SDK presence check
|
||||||
return True
|
return True
|
||||||
except ImportError:
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Probe plugin providers. Discovery is idempotent and cheap.
|
||||||
|
try:
|
||||||
|
from agent.image_gen_registry import list_providers
|
||||||
|
from hermes_cli.plugins import _ensure_plugins_discovered
|
||||||
|
|
||||||
|
_ensure_plugins_discovered()
|
||||||
|
for provider in list_providers():
|
||||||
|
try:
|
||||||
|
if provider.is_available():
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -827,10 +854,11 @@ from tools.registry import registry, tool_error
|
||||||
IMAGE_GENERATE_SCHEMA = {
|
IMAGE_GENERATE_SCHEMA = {
|
||||||
"name": "image_generate",
|
"name": "image_generate",
|
||||||
"description": (
|
"description": (
|
||||||
"Generate high-quality images from text prompts using FAL.ai. "
|
"Generate high-quality images from text prompts. The underlying "
|
||||||
"The underlying model is user-configured (default: FLUX 2 Klein 9B, "
|
"backend (FAL, OpenAI, etc.) and model are user-configured and not "
|
||||||
"sub-1s generation) and is not selectable by the agent. Returns a "
|
"selectable by the agent. Returns either a URL or an absolute file "
|
||||||
"single image URL. Display it using markdown: "
|
"path in the `image` field; display it with markdown "
|
||||||
|
" and the gateway will deliver it."
|
||||||
),
|
),
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
|
|
@ -851,13 +879,104 @@ IMAGE_GENERATE_SCHEMA = {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _read_configured_image_provider():
|
||||||
|
"""Return the value of ``image_gen.provider`` from config.yaml, or None.
|
||||||
|
|
||||||
|
We only consult the plugin registry when this is explicitly set — an
|
||||||
|
unset value keeps users on the legacy in-tree FAL path even when other
|
||||||
|
providers happen to be registered (e.g. a user has OPENAI_API_KEY set
|
||||||
|
for other features but never asked for OpenAI image gen).
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from hermes_cli.config import load_config
|
||||||
|
cfg = load_config()
|
||||||
|
section = cfg.get("image_gen") if isinstance(cfg, dict) else None
|
||||||
|
if isinstance(section, dict):
|
||||||
|
value = section.get("provider")
|
||||||
|
if isinstance(value, str) and value.strip():
|
||||||
|
return value.strip()
|
||||||
|
except Exception as exc:
|
||||||
|
logger.debug("Could not read image_gen.provider: %s", exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _dispatch_to_plugin_provider(prompt: str, aspect_ratio: str):
|
||||||
|
"""Route the call to a plugin-registered provider when one is selected.
|
||||||
|
|
||||||
|
Returns a JSON string on dispatch, or ``None`` to fall through to the
|
||||||
|
built-in FAL path.
|
||||||
|
|
||||||
|
Dispatch only fires when ``image_gen.provider`` is explicitly set AND
|
||||||
|
it does not point to ``fal`` (FAL still lives in-tree in this PR;
|
||||||
|
a later PR ports it into ``plugins/image_gen/fal/``). Any other value
|
||||||
|
that matches a registered plugin provider wins.
|
||||||
|
"""
|
||||||
|
configured = _read_configured_image_provider()
|
||||||
|
if not configured or configured == "fal":
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Import locally so plugin discovery isn't triggered just by
|
||||||
|
# importing this module (tests rely on that).
|
||||||
|
from agent.image_gen_registry import get_provider
|
||||||
|
from hermes_cli.plugins import _ensure_plugins_discovered
|
||||||
|
|
||||||
|
_ensure_plugins_discovered()
|
||||||
|
provider = get_provider(configured)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.debug("image_gen plugin dispatch skipped: %s", exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
if provider is None:
|
||||||
|
return json.dumps({
|
||||||
|
"success": False,
|
||||||
|
"image": None,
|
||||||
|
"error": (
|
||||||
|
f"image_gen.provider='{configured}' is set but no plugin "
|
||||||
|
f"registered that name. Run `hermes plugins list` to see "
|
||||||
|
f"available image gen backends."
|
||||||
|
),
|
||||||
|
"error_type": "provider_not_registered",
|
||||||
|
})
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = provider.generate(prompt=prompt, aspect_ratio=aspect_ratio)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Image gen provider '%s' raised: %s",
|
||||||
|
getattr(provider, "name", "?"), exc,
|
||||||
|
)
|
||||||
|
return json.dumps({
|
||||||
|
"success": False,
|
||||||
|
"image": None,
|
||||||
|
"error": f"Provider '{getattr(provider, 'name', '?')}' error: {exc}",
|
||||||
|
"error_type": "provider_exception",
|
||||||
|
})
|
||||||
|
if not isinstance(result, dict):
|
||||||
|
return json.dumps({
|
||||||
|
"success": False,
|
||||||
|
"image": None,
|
||||||
|
"error": "Provider returned a non-dict result",
|
||||||
|
"error_type": "provider_contract",
|
||||||
|
})
|
||||||
|
return json.dumps(result)
|
||||||
|
|
||||||
|
|
||||||
def _handle_image_generate(args, **kw):
|
def _handle_image_generate(args, **kw):
|
||||||
prompt = args.get("prompt", "")
|
prompt = args.get("prompt", "")
|
||||||
if not prompt:
|
if not prompt:
|
||||||
return tool_error("prompt is required for image generation")
|
return tool_error("prompt is required for image generation")
|
||||||
|
aspect_ratio = args.get("aspect_ratio", DEFAULT_ASPECT_RATIO)
|
||||||
|
|
||||||
|
# Route to a plugin-registered provider if one is active (and it's
|
||||||
|
# not the in-tree FAL path).
|
||||||
|
dispatched = _dispatch_to_plugin_provider(prompt, aspect_ratio)
|
||||||
|
if dispatched is not None:
|
||||||
|
return dispatched
|
||||||
|
|
||||||
return image_generate_tool(
|
return image_generate_tool(
|
||||||
prompt=prompt,
|
prompt=prompt,
|
||||||
aspect_ratio=args.get("aspect_ratio", DEFAULT_ASPECT_RATIO),
|
aspect_ratio=aspect_ratio,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue