hermes-agent/tests
teknium1 585f8528b2 fix: deep review — prefix matching, tool_calls extraction, query perf, serialization
Issues found and fixed during deep code path review:

1. CRITICAL: Prefix matching returned wrong prices for dated model names
   - 'gpt-4o-mini-2024-07-18' matched gpt-4o ($2.50) instead of gpt-4o-mini ($0.15)
   - Same for o3-mini→o3 (9x), gpt-4.1-mini→gpt-4.1 (5x), gpt-4.1-nano→gpt-4.1 (20x)
   - Fix: use longest-match-wins strategy instead of first-match
   - Removed dangerous key.startswith(bare) reverse matching

2. CRITICAL: Top Tools section was empty for CLI sessions
   - run_agent.py doesn't set tool_name on tool response messages (pre-existing)
   - Insights now also extracts tool names from tool_calls JSON on assistant
     messages, which IS populated for all sessions
   - Uses max() merge strategy to avoid double-counting between sources

3. SELECT * replaced with explicit column list
   - Skips system_prompt and model_config blobs (can be thousands of chars)
   - Reduces memory and I/O for large session counts

4. Sets in overview dict converted to sorted lists
   - models_with_pricing / models_without_pricing were Python sets
   - Sets aren't JSON-serializable — would crash json.dumps()

5. Negative duration guard
   - end > start check prevents negative durations from clock drift

6. Model breakdown sort fallback
   - When all tokens are 0, now sorts by session count instead of arbitrary order

7. Removed unused timedelta import

Added 6 new tests: dated model pricing (4), tool_calls JSON extraction,
JSON serialization safety. Total: 69 tests.
2026-03-06 14:50:57 -08:00
..
agent test: comprehensive tests for model metadata + firecrawl config 2026-03-05 18:22:39 -08:00
cron test: strengthen assertions across 7 test files (batch 1) 2026-03-05 18:39:37 -08:00
fakes test: add HA integration tests with fake in-process server 2026-02-28 14:28:04 +03:00
gateway Merge PR #433: fix(whatsapp): replace Linux-only fuser with cross-platform port cleanup 2026-03-06 04:52:25 -08:00
hermes_cli test: add 26 tests for set_config_value secret routing 2026-03-06 04:26:18 -08:00
honcho_integration Merge PR #193: add unit tests for 5 security/logic-critical modules (batch 4) 2026-03-04 19:35:01 -08:00
integration test(daytona): add unit and integration tests for Daytona backend 2026-03-05 10:26:22 -08:00
tools fix: correct mock tool name 'search' → 'search_files' in test_code_execution 2026-03-06 03:53:43 -08:00
__init__.py A bit of restructuring for simplicity and organization 2025-10-01 23:29:25 +00:00
conftest.py fix(tests): isolate HERMES_HOME in tests and adjust log directory for debug session 2026-03-02 04:34:21 -08:00
test_413_compression.py fix: preflight context compression + error handler ordering for model switches 2026-03-04 14:42:41 -08:00
test_atomic_json_write.py refactor: extract atomic_json_write helper, add 24 checkpoint tests 2026-03-06 05:50:12 -08:00
test_auth_codex_provider.py refactor(auth): transition Codex OAuth tokens to Hermes auth store 2026-03-01 19:59:24 -08:00
test_auth_nous_provider.py Fix nous refresh token rotation failure in case where api key mint/retrieval fails 2026-03-02 17:18:15 +11:00
test_batch_runner_checkpoint.py refactor: extract atomic_json_write helper, add 24 checkpoint tests 2026-03-06 05:50:12 -08:00
test_cli_init.py test: strengthen assertions across 7 test files (batch 1) 2026-03-05 18:39:37 -08:00
test_cli_provider_resolution.py Add OpenAI Codex provider runtime and responses integration (without .agent/PLANS.md) 2026-02-25 18:20:38 -08:00
test_codex_execution_paths.py fix: update mock agent signature to accept task_id after PR #419 2026-03-05 01:41:50 -08:00
test_codex_models.py Harden Codex auth refresh and responses compatibility 2026-02-25 19:27:54 -08:00
test_external_credential_detection.py refactor(auth): transition Codex OAuth tokens to Hermes auth store 2026-03-01 19:59:24 -08:00
test_flush_memories_codex.py refactor(cli): Finalize OpenAI Codex Integration with OAuth 2026-02-28 21:47:51 -08:00
test_hermes_state.py test: strengthen assertions across 7 test files (batch 1) 2026-03-05 18:39:37 -08:00
test_honcho_client_config.py fix(honcho): auto-enable when API key is present 2026-03-01 03:12:37 -05:00
test_insights.py fix: deep review — prefix matching, tool_calls extraction, query perf, serialization 2026-03-06 14:50:57 -08:00
test_model_tools.py test: strengthen assertions across 3 more test files (batch 2) 2026-03-05 18:46:30 -08:00
test_provider_parity.py feat(agent): add summary handling for reasoning items 2026-03-01 20:03:03 -08:00
test_run_agent.py Merge PR #436: fix: use _max_tokens_param in max-iterations retry path 2026-03-06 04:46:24 -08:00
test_run_agent_codex_responses.py refactor(cli): Finalize OpenAI Codex Integration with OAuth 2026-02-28 21:47:51 -08:00
test_runtime_provider_resolution.py fix: prioritize OPENROUTER_API_KEY over OPENAI_API_KEY 2026-03-03 00:28:26 +03:00
test_toolset_distributions.py test: add unit tests for 8 modules (batch 2) 2026-02-26 13:54:20 +03:00
test_toolsets.py test: add unit tests for 8 untested modules 2026-02-26 13:27:58 +03:00
test_trajectory_compressor.py test: add 25 unit tests for trajectory_compressor 2026-02-28 21:28:28 +03:00