mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-29 06:31:32 +00:00
3d66787a04
* fix(vision): route auxiliary.vision.provider=openai to api.openai.com, skip text-only main for vision Fixes #31179. Three coupled fixes so a configured aux vision backend actually serves vision tasks instead of silently routing images to the user's main provider: 1. agent/auxiliary_client.py: `auxiliary.<task>.provider: openai` resolves to `custom` + `https://api.openai.com/v1`. "openai" was not in PROVIDER_REGISTRY (we have `openai-codex` for OAuth and `custom` for manual base_url), so the obvious config name silently failed to build a client. User-supplied base_url is still preserved; only the provider name normalises to `custom` so resolution doesn't hit the PROVIDER_REGISTRY-only path. 2. agent/auxiliary_client.py: the vision auto-detect chain now skips the user's main provider when models.dev reports `supports_vision=False`. Without this guard, a misconfigured aux provider would fall back to `auto`, which happily returned the main-provider client. The caller would then send image content to e.g. api.deepseek.com with model `gpt-4o-mini` and get a cryptic `unknown variant 'image_url', expected 'text'` from the provider's parser. 3. tools/vision_tools.py + tools/browser_tool.py: `check_vision_requirements` now mirrors the runtime fallback chain (explicit provider, then auto), so `vision_analyze` shows up whenever vision is actually serviceable. `browser_vision` gets a new `check_browser_vision_requirements` check_fn that AND-gates browser + vision availability, so it doesn't get advertised to the model when the call would fail at runtime. Reproduction (config from the bug report): model.provider: deepseek model.default: deepseek-v4-pro auxiliary.vision.provider: openai auxiliary.vision.model: gpt-4o-mini Before: resolve_vision_provider_client() returns None for the explicit provider, fallback auto returns the deepseek client with model='gpt-4o-mini', image hits api.deepseek.com → 'unknown variant image_url'. vision_analyze hidden from tool list; browser_vision exposed but fails at call time. After: resolves to custom + api.openai.com/v1 with model gpt-4o-mini. vision_analyze and browser_vision both gate correctly on capability. Tests: tests/agent/test_vision_routing_31179.py covers all three fixes (12 cases including the user's exact scenario, base_url preservation, text-only-main skip, capability-unknown permissive fallback, and tool gating parity). Existing 382 tests across auxiliary/vision/image_routing suites still pass. * test(vision): use exact hostname check to silence CodeQL substring-sanitization alert * fix(auxiliary): drop model name from vision-skip debug log to silence CodeQL The new `logger.debug(...)` added in the previous commit interpolated both `main_provider` and `vision_model` (a public model slug \u2014 not sensitive). CodeQL's `py/clear-text-logging-sensitive-data` heuristic re-flagged it twice because the rule mis-detects multi-value interpolations near tainted-via-config provider strings. Drop the model from the log args (provider alone is enough to diagnose the skip; the same sibling branch a few lines up already logs provider only). Behavior unchanged; CodeQL false positive cleared. |
||
|---|---|---|
| .. | ||
| lsp | ||
| secret_sources | ||
| transports | ||
| __init__.py | ||
| account_usage.py | ||
| agent_init.py | ||
| agent_runtime_helpers.py | ||
| anthropic_adapter.py | ||
| async_utils.py | ||
| auxiliary_client.py | ||
| azure_identity_adapter.py | ||
| background_review.py | ||
| bedrock_adapter.py | ||
| browser_provider.py | ||
| browser_registry.py | ||
| chat_completion_helpers.py | ||
| codex_responses_adapter.py | ||
| codex_runtime.py | ||
| context_compressor.py | ||
| context_engine.py | ||
| context_references.py | ||
| conversation_compression.py | ||
| conversation_loop.py | ||
| copilot_acp_client.py | ||
| credential_pool.py | ||
| credential_sources.py | ||
| curator.py | ||
| curator_backup.py | ||
| display.py | ||
| error_classifier.py | ||
| file_safety.py | ||
| gemini_cloudcode_adapter.py | ||
| gemini_native_adapter.py | ||
| gemini_schema.py | ||
| google_code_assist.py | ||
| google_oauth.py | ||
| i18n.py | ||
| image_gen_provider.py | ||
| image_gen_registry.py | ||
| image_routing.py | ||
| insights.py | ||
| iteration_budget.py | ||
| lmstudio_reasoning.py | ||
| manual_compression_feedback.py | ||
| markdown_tables.py | ||
| memory_manager.py | ||
| memory_provider.py | ||
| message_sanitization.py | ||
| model_metadata.py | ||
| models_dev.py | ||
| moonshot_schema.py | ||
| nous_rate_guard.py | ||
| onboarding.py | ||
| plugin_llm.py | ||
| portal_tags.py | ||
| process_bootstrap.py | ||
| prompt_builder.py | ||
| prompt_caching.py | ||
| rate_limit_tracker.py | ||
| redact.py | ||
| retry_utils.py | ||
| shell_hooks.py | ||
| skill_bundles.py | ||
| skill_commands.py | ||
| skill_preprocessing.py | ||
| skill_utils.py | ||
| stream_diag.py | ||
| subdirectory_hints.py | ||
| system_prompt.py | ||
| think_scrubber.py | ||
| title_generator.py | ||
| tool_dispatch_helpers.py | ||
| tool_executor.py | ||
| tool_guardrails.py | ||
| tool_result_classification.py | ||
| trajectory.py | ||
| usage_pricing.py | ||
| video_gen_provider.py | ||
| video_gen_registry.py | ||
| web_search_provider.py | ||
| web_search_registry.py | ||