mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
1908dd09fc
Ollama sizes a model's real context window by free VRAM at load time, often far below the GGUF trained max that /api/show reports, and its OpenAI-compatible endpoint has no options passthrough — per-request num_ctx and keep_alive are silently dropped, so the window cannot be controlled from the client. The compressor was being sized to the trained max (e.g. 262K for a model actually running at 32K). - Add query_ollama_loaded_context() reading the effective window from /api/ps (60s cache, never persisted — transient load state). - Reconcile after each successful response via sync_ollama_loaded_context(): resize the compressor to the loaded window and warn when it is below the tool-use minimum. Selection surfaces keep showing the trained max; explicit model.context_length still wins. No-op for non-Ollama providers. - Refresh model.ollama_keep_alive through the native API (rate-limited /api/generate ping) since /v1 drops it. - Remove the inert num_ctx/keep_alive request-body plumbing; warn that model.ollama_num_ctx has no effect and point at OLLAMA_CONTEXT_LENGTH / Modelfile num_ctx. Keep detection for the pre-flight window check. - Disable thinking on Ollama thinking models via reasoning_effort 'none' — the only switch its /v1 handler parses (think is dropped). |
||
|---|---|---|
| .. | ||
| test_custom_profile.py | ||
| test_deepseek_profile.py | ||
| test_fireworks_profile.py | ||
| test_kimi_profile.py | ||
| test_minimax_profile.py | ||
| test_ollama_cloud_profile.py | ||
| test_opencode_go_profile.py | ||
| test_zai_profile.py | ||