hermes-agent/plugins/model-providers
teknium1 1c0884516c fix(openrouter): give auxiliary calls the sticky routing key
Auxiliary LLM calls routed through OpenRouter now carry the `session_id`
sticky routing key, so they pin to the same provider endpoint as the
conversation they belong to instead of routing independently.

`OpenRouterProfile.build_extra_body` sourced the key only from its explicit
argument. Auxiliary call sites — compression, title generation, vision,
web_extract, session_search, MoA slots — funnel through
`agent.auxiliary_client`, which has no session handle and never passes one.
Those calls sent NO sticky key at all, so OpenRouter fell back to hashing
the opening messages and each aux call could land on a different provider
than the conversation it served. Per OpenRouter's prompt-caching docs a
present `session_id` is used directly as the routing key and activates
stickiness on the first successful request rather than only after a cache
hit, so the missing key also delayed pinning for the main loop.

Resolve it from the ambient conversation contextvar first, explicit argument
as fallback — the same resolution the Nous Portal profile uses (f2f4df064d)
and the same one `nous_portal_tags` already used for the `conversation=` tag.
The ambient value is the session-lineage ROOT, so the key also stays stable
for installs that opt out of the default `compression.in_place: true` and
across delegate-subagent trees.

The xAI `x-grok-conv-id` cache-affinity header in `build_api_kwargs_extras`
had the identical gap and is fixed the same way; it remains model-gated to
`x-ai/grok-*` / `xai/grok-*`.

Fixes #70820. Credit to @webtecnica, who reported it and submitted #70883
first; that PR threaded `session_id` through `call_llm` to reach the same
goal. This takes the ambient-contextvar route instead because none of the
34 `call_llm` call sites currently pass a `session_id`, so the parameter
alone would not have changed any live request.

Tests: 2 cases in tests/providers/test_provider_profiles.py, both verified
to fail against the pre-fix behavior via a sabotage run. E2E-validated on a
real AIAgent driving the out-of-turn compaction path.
2026-07-25 22:46:35 -07:00
..
alibaba
alibaba-coding-plan chore(model-picker): refresh provider picker descriptions 2026-05-31 15:02:26 -07:00
anthropic fix(security): cover remaining catalog credential paths 2026-07-11 12:28:55 +05:30
arcee
azure-foundry feat(azure-foundry): add Microsoft Entra ID auth 2026-05-18 10:14:38 -07:00
bedrock fix(models): pass model.base_url to fetch_models in /model picker 2026-06-16 13:09:40 -07:00
copilot fix(copilot): clamp reasoning effort to the nearest supported level, not xhigh->high 2026-07-16 08:47:10 -07:00
copilot-acp fix(models): pass model.base_url to fetch_models in /model picker 2026-06-16 13:09:40 -07:00
custom fix(ollama): emit top-level reasoning_effort=none on /v1/chat/completions (#25758) 2026-07-15 06:38:28 -07:00
deepinfra fix(deepinfra): harden multimodal provider routing 2026-07-14 02:59:39 +05:30
deepseek feat(reasoning): add max and ultra effort levels (#62650) 2026-07-12 00:26:49 -07:00
fireworks fix(providers): align Fireworks integration with project policy 2026-07-11 05:43:35 -07:00
gemini feat(providers): remove google-gemini-cli + google-antigravity OAuth providers (#50492) 2026-06-21 19:53:27 -07:00
gmi fix: align gmi fallback_models ordering with curated list 2026-07-20 02:25:44 -07:00
huggingface
kilocode
kimi-coding feat(kimi): discover K3 on coding endpoint 2026-07-16 13:33:02 -07:00
minimax fix: route minimax m3 reasoning controls through profile 2026-06-15 07:08:43 -07:00
nous fix(nous-portal): give auxiliary calls the Portal sticky routing key 2026-07-25 22:15:00 -07:00
novita docs: update NovitaAI provider positioning (#25532) 2026-05-14 01:31:12 -07:00
nvidia
ollama-cloud fix(ollama-cloud): capability-gate reasoning_effort + correct disable semantics 2026-07-16 07:58:04 -07:00
openai-codex
opencode-zen feat(reasoning): add max and ultra effort levels (#62650) 2026-07-12 00:26:49 -07:00
openrouter fix(openrouter): give auxiliary calls the sticky routing key 2026-07-25 22:46:35 -07:00
qwen-oauth perf: avoid broad message prep deepcopies 2026-07-09 14:35:14 +05:30
stepfun
upstage fix(upstage): collapse unknown future efforts to high; behavior-contract tests 2026-07-15 00:09:24 +05:30
vertex feat(vertex): add Google Vertex AI provider for Gemini (OAuth2) 2026-07-01 05:25:33 -07:00
xai
xiaomi fix(vision): proactive downgrade for providers rejecting list-type tool content (#41072) 2026-06-07 21:50:57 -07:00
zai feat(reasoning): add max and ultra effort levels (#62650) 2026-07-12 00:26:49 -07:00
README.md

Model Provider Plugins

Each subdirectory is a self-contained provider profile plugin. The directory layout mirrors plugins/platforms/:

plugins/model-providers/
├── openrouter/
│   ├── __init__.py      # registers the ProviderProfile
│   └── plugin.yaml      # manifest: name, kind, version, description
├── anthropic/
│   ├── __init__.py
│   └── plugin.yaml
└── ...

How discovery works

providers/__init__.py._discover_providers() scans this directory (and $HERMES_HOME/plugins/model-providers/) the first time anything calls get_provider_profile() or list_providers(). Each __init__.py is imported and expected to call providers.register_provider(profile).

User plugins at $HERMES_HOME/plugins/model-providers/<name>/ override bundled plugins of the same name — last-writer-wins in register_provider(). Drop a file there to replace a built-in.

Adding a new provider

  1. Create plugins/model-providers/<your_provider>/__init__.py:

    from providers import register_provider
    from providers.base import ProviderProfile
    
    my_provider = ProviderProfile(
        name="your-provider",
        aliases=("alias1", "alias2"),
        display_name="Your Provider",
        description="One-line description shown in the setup picker",
        signup_url="https://your-provider.example.com/keys",
        env_vars=("YOUR_PROVIDER_API_KEY", "YOUR_PROVIDER_BASE_URL"),
        base_url="https://api.your-provider.example.com/v1",
        default_aux_model="your-cheap-model",
    )
    
    register_provider(my_provider)
    
  2. Create plugins/model-providers/<your_provider>/plugin.yaml:

    name: your-provider-profile
    kind: model-provider
    version: 1.0.0
    description: Short sentence about the provider
    author: Your Name
    

Nothing else needs to change. auth.py, config.py, models.py, doctor.py, model_metadata.py, runtime_provider.py, and the chat_completions transport all auto-wire from the registry.

Non-trivial profiles

Override the ProviderProfile hooks in a subclass for per-provider quirks — see plugins/model-providers/openrouter/__init__.py for build_extra_body and build_api_kwargs_extras examples, and plugins/model-providers/gemini/__init__.py for thinking_config translation.