hermes-agent/tests/agent
Julien Talbot b577697189 fix(model_metadata): add xAI Grok context length fallbacks
xAI /v1/models does not return context_length metadata, so Hermes
probes down to the 128k default whenever a user configures a custom
provider pointing at https://api.x.ai/v1. This forces every xAI user
to manually override model.context_length in config.yaml (2M for
Grok 4.20 / 4.1-fast / 4-fast) or lose most of the usable context
window.

Add DEFAULT_CONTEXT_LENGTHS entries for the Grok family so the
fallback lookup returns the correct value via substring matching.
Values sourced from models.dev (2026-04) and cross-checked against
the xAI /v1/models listing:

  - grok-4.20-*          2,000,000  (reasoning, non-reasoning, multi-agent)
  - grok-4-1-fast-*      2,000,000
  - grok-4-fast-*        2,000,000
  - grok-4 / grok-4-0709   256,000
  - grok-code-fast-1       256,000
  - grok-3*                131,072
  - grok-2 / latest        131,072
  - grok-2-vision*           8,192
  - grok (catch-all)       131,072

Keys are ordered longest-first so that specific variants match before
the catch-all, consistent with the existing Claude/Gemma/MiniMax entries.

Add TestDefaultContextLengths.test_grok_models_context_lengths and
test_grok_substring_matching to pin the values and verify the full
lookup path. All 77 tests in test_model_metadata.py pass.
2026-04-10 03:04:19 -07:00
..
__init__.py test: add unit tests for 8 modules (batch 2) 2026-02-26 13:54:20 +03:00
test_anthropic_adapter.py fix(anthropic): omit tool-streaming beta on MiniMax endpoints 2026-04-09 17:53:52 -07:00
test_auxiliary_client.py fix: restore codex fallback auth-store lookup 2026-04-09 01:56:10 -07:00
test_auxiliary_config_bridge.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_auxiliary_named_custom_providers.py fix(auxiliary): resolve named custom providers and 'main' alias in auxiliary routing (#5978) 2026-04-07 17:59:47 -07:00
test_context_compressor.py test: add coverage for token-budget tail protection 2026-04-08 23:54:23 -07:00
test_context_references.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_credential_pool.py fix: reduce credential exhaustion TTL from 24 hours to 1 hour (#6504) 2026-04-09 02:37:23 -07:00
test_credential_pool_routing.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_crossloop_client_cache.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_display.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_display_emoji.py feat(tools): centralize tool emoji metadata in registry + skin integration 2026-03-15 20:21:21 -07:00
test_error_classifier.py fix(error_classifier): disambiguate usage-limit patterns in _classify_by_message 2026-04-09 16:24:13 -07:00
test_external_skills.py feat(skills): support external skill directories via config (#3678) 2026-03-29 00:33:30 -07:00
test_insights.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_local_stream_timeout.py fix: increase stream read timeout default to 120s, auto-raise for local LLMs (#6967) 2026-04-09 22:35:30 -07:00
test_memory_plugin_e2e.py feat(memory): pluggable memory provider interface with profile isolation, review fixes, and honcho CLI restoration (#4623) 2026-04-02 15:33:51 -07:00
test_memory_provider.py fix: mem0 API v2 compat, prefetch context fencing, secret redaction (#5423) 2026-04-05 22:43:33 -07:00
test_memory_user_id.py fix: thread gateway user_id to memory plugins for per-user scoping (#5895) 2026-04-07 11:14:12 -07:00
test_minimax_auxiliary_url.py fix: provider/model resolution — salvage 4 PRs + MiniMax aux URL fix (#5983) 2026-04-07 22:23:28 -07:00
test_minimax_provider.py fix(anthropic): omit tool-streaming beta on MiniMax endpoints 2026-04-09 17:53:52 -07:00
test_model_metadata.py fix(model_metadata): add xAI Grok context length fallbacks 2026-04-10 03:04:19 -07:00
test_model_metadata_local_ctx.py refactor(tests): re-architect tests + fix CI failures (#5946) 2026-04-07 17:19:07 -07:00
test_models_dev.py feat: overhaul context length detection with models.dev and provider-aware resolution (#2158) 2026-03-20 06:04:33 -07:00
test_prompt_builder.py feat: switch managed browser provider from Browserbase to Browser Use (#5750) 2026-04-07 08:40:22 -04:00
test_prompt_caching.py fix(prompt-caching): skip top-level cache_control on role:tool for OpenRouter 2026-03-21 16:54:43 -07:00
test_rate_limit_tracker.py feat: capture provider rate limit headers and show in /usage (#6541) 2026-04-09 03:43:14 -07:00
test_redact.py test(redact): add regression tests for lowercase variable redaction (#4367) (#5185) 2026-04-05 00:10:16 -07:00
test_skill_commands.py fix: sanitize Telegram command names to strip invalid characters 2026-04-06 11:27:28 -07:00
test_smart_model_routing.py fix: hermes update causes dual gateways on macOS (launchd) (#1567) 2026-03-16 12:36:29 -07:00
test_subagent_progress.py feat(api): structured run events via /v1/runs SSE endpoint 2026-04-05 12:05:13 -07:00
test_subdirectory_hints.py fix(agent): catch PermissionError in subdirectory hint discovery 2026-04-09 03:10:30 -07:00
test_title_generator.py feat: auto-generate session titles after first exchange 2026-03-17 04:14:40 -07:00
test_usage_pricing.py feat: use endpoint metadata for custom model context and pricing (#1906) 2026-03-18 03:04:07 -07:00