Commit graph

599 commits

Author SHA1 Message Date
0xbyt4
6053236158 fix: prioritize OPENROUTER_API_KEY over OPENAI_API_KEY
When both OPENROUTER_API_KEY and OPENAI_API_KEY are set (e.g. OPENAI_API_KEY
in .bashrc), the wrong key was sent to OpenRouter causing auth failures.

Fixed key resolution order in cli.py and runtime_provider.py.

Fixes #289
2026-03-03 00:28:26 +03:00
0xbyt4
11615014a4 fix: eliminate shell noise from terminal output with fence markers
- Wrap commands with unique fence markers (printf FENCE; cmd; printf FENCE)
  to isolate real output from shell init/exit noise (oh-my-zsh, macOS
  session restore/save, docker plugin errors, etc.)
- Expand _clean_shell_noise to cover zsh/macOS patterns and strip from
  both beginning and end (fallback when fences are missing)
- Fix BSD find compatibility: fallback to simple find when -printf
  produces empty output (macOS)
- Fix test_terminal_disk_usage: use sys.modules to get the real module
  instead of the shadowed function from tools/__init__.py
- Add 13 new unit tests for fence extraction and zsh noise patterns
2026-03-02 22:53:21 +03:00
0xbyt4
11a2ecb936 fix: resolve thread safety issues and shutdown deadlock in MCP client
- Add threading.Lock protecting all shared state (_servers, _mcp_loop, _mcp_thread)
- Fix deadlock in shutdown_mcp_servers: _stop_mcp_loop was called inside
  a _lock block but also acquires _lock (non-reentrant)
- Fix race condition in _ensure_mcp_loop with concurrent callers
- Change idempotency to per-server (retry failed servers, skip connected)
- Dynamic toolset injection via startswith("hermes-") instead of hardcoded list
- Parallel shutdown via asyncio.gather instead of sequential loop
- Add tests for partial failure retry, parallel shutdown, dynamic injection
2026-03-02 22:08:32 +03:00
0xbyt4
151e8d896c fix(tests): isolate discover_mcp_tools tests from global _servers state
Patch _servers to empty dict in tests that call discover_mcp_tools()
with mocked config, preventing interference from real MCP connections
that may exist when running within the full test suite.
2026-03-02 21:38:01 +03:00
0xbyt4
aa2ecaef29 fix: resolve orphan subprocess leak on MCP server shutdown
Refactor MCP connections from AsyncExitStack to task-per-server
architecture. Each server now runs as a long-lived asyncio Task
with `async with stdio_client(...)`, ensuring anyio cancel-scope
cleanup happens in the same Task that opened the connection.
2026-03-02 21:22:00 +03:00
0xbyt4
3c252ae44b feat: add MCP (Model Context Protocol) client support
Connect to external MCP servers via stdio transport, discover their tools
at startup, and register them into the hermes-agent tool registry.

- New tools/mcp_tool.py: config loading, server connection via background
  event loop, tool handler factories, discovery, and graceful shutdown
- model_tools.py: trigger MCP discovery after built-in tool imports
- cli.py: call shutdown_mcp_servers in _run_cleanup
- pyproject.toml: add mcp>=1.2.0 as optional dependency
- 27 unit tests covering config, schema conversion, handlers, registration,
  SDK interaction, toolset injection, graceful fallback, and shutdown

Config format (in ~/.hermes/config.yaml):
  mcp_servers:
    filesystem:
      command: "npx"
      args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
2026-03-02 21:03:14 +03:00
BP602
6789084ec0
Fix ClawHub Skills Hub adapter for updated API 2026-03-02 16:11:49 +01:00
teknium1
7652afb8de Merge PR #243: fix(honcho): auto-enable when API key is present
Authored by Bartok9. Fixes #241.
2026-03-02 05:13:33 -08:00
teknium1
7862e7010c test: add additional multiline bypass tests for find patterns
Extra test coverage for newline bypass detection (DOTALL fix).
Inspired by Bartok9's PR #245.
2026-03-02 04:46:27 -08:00
teknium1
4faf2a6cf4 Merge PR #233: fix(security): add re.DOTALL to prevent multiline bypass of dangerous command detection
Authored by Farukest. Fixes #232.
2026-03-02 04:44:06 -08:00
teknium1
6d2481ee5c Merge PR #231: fix: use task-specific glob pattern in disk usage calculation
Authored by Farukest. Fixes #230.
2026-03-02 04:38:58 -08:00
teknium1
ca5525bcd7 fix(tests): isolate HERMES_HOME in tests and adjust log directory for debug session
Added a fixture to redirect HERMES_HOME to a temporary directory during tests, preventing writes to the user's home directory. Updated the test for DebugSession to create a dedicated log directory for saving logs, ensuring test isolation and accuracy in assertions.
2026-03-02 04:34:21 -08:00
teknium1
56b53bff6e Merge PR #229: fix(agent): copy conversation_history to avoid mutating caller's list
Authored by Farukest. Fixes #228.

# Conflicts:
#	tests/test_run_agent.py
2026-03-02 04:21:39 -08:00
teknium1
c4ea996612 fix: repair flush sentinel test — mock auxiliary client and add guard
The TestFlushSentinelNotLeaked test from PR #227 had two issues:
1. flush_memories() uses get_text_auxiliary_client() which could bypass
   agent.client entirely — mock it to return (None, None)
2. No assertion that the API was actually called — added guard assert

Without these fixes the test passed vacuously (API never called).
2026-03-02 03:21:08 -08:00
teknium1
39bfd226b8 Merge PR #225: fix: preserve empty content in ReadResult.to_dict()
Authored by Farukest. Fixes #224.
2026-03-02 03:13:31 -08:00
teknium1
234b67f5fd fix: mock time in retry exhaustion tests to prevent backoff sleep
The TestRetryExhaustion tests from PR #223 didn't mock time.sleep/time.time,
causing the retry backoff loops (275s+ total) to run in real time. Tests would
time out instead of running quickly.

Added _make_fast_time_mock() helper that creates a mock time module where
time.time() advances 500s per call (so sleep_end is always in the past) and
time.sleep() is a no-op. Both tests now complete in <1s.
2026-03-02 02:59:41 -08:00
teknium1
e27e3a4f8a Merge PR #223: fix: correct off-by-one in retry exhaustion checks
Authored by Farukest. Fixes #222.
2026-03-02 02:54:10 -08:00
teknium1
1cb2311bad fix(security): block path traversal in skill_view file_path (fixes #220)
skill_view accepted arbitrary file_path values like '../../.env' and
would read files outside the skill directory, exposing API keys and
other sensitive data.

Added two layers of defense:
1. Reject paths with '..' components (fast, catches obvious traversal)
2. resolve() containment check with trailing '/' to prevent prefix
   collisions (catches symlinks and edge cases)

Fix approach from PR #242 (@Bartok9). Vulnerability reported by
@Farukest (#220, PR #221). Tests rewritten to properly mock SKILLS_DIR.

Closes #220
2026-03-02 02:00:09 -08:00
teknium1
25c65bc99e fix(agent): handle None content in context compressor (fixes #211)
The OpenAI API returns content: null on assistant messages that only
contain tool calls. msg.get('content', '') returns None (not '') when
the key exists with value None, causing TypeError on len() and string
concatenation in _generate_summary and compress.

Fix: msg.get('content') or '' — handles both missing keys and None.

Tests from PR #216 (@Farukest). Fix also in PR #215 (@cutepawss).
Both PRs had stale branches and couldn't be merged directly.

Closes #211
2026-03-02 01:35:52 -08:00
teknium1
afb680b50d fix(cli): fix max_turns comment and test for correct priority order
Priority is: CLI arg > config file > env var > default
(not env var > config file as the old comment stated)

The test failed because config.yaml had max_turns at both root level
and inside agent section. The test cleared agent.max_turns but the
root-level value still took precedence over the env var. Fixed the
test to clear both, and corrected the comment to match the intended
priority order.
2026-03-02 01:18:52 -08:00
teknium1
e265006fd6 test: add coverage for chat_topic in SessionSource and session context prompt
Tests added:
- Roundtrip serialization of chat_topic via to_dict/from_dict
- chat_topic defaults to None when missing from dict
- Channel Topic line appears in session context prompt when set
- Channel Topic line is omitted when chat_topic is None

Follow-up to PR #248 (feat: Discord channel topic in session context).
2026-03-02 00:53:21 -08:00
teknium1
719f2eef32 Merge branch 'pr-217'
# Conflicts:
#	gateway/session.py
2026-03-02 00:18:41 -08:00
Robin Fernandes
5e5e0efc60 Fix nous refresh token rotation failure in case where api key mint/retrieval fails 2026-03-02 17:18:15 +11:00
teknium1
e5893075f9 feat(agent): add summary handling for reasoning items
Enhanced the AIAgent class to capture and normalize summary information for reasoning items. Implemented logic to handle summaries as lists, ensuring proper formatting for API interactions. Updated tests to validate the inclusion of summaries in reasoning items, both for existing and default cases.
2026-03-01 20:03:03 -08:00
teknium1
5e598a588f refactor(auth): transition Codex OAuth tokens to Hermes auth store
Updated the authentication mechanism to store Codex OAuth tokens in the Hermes auth store located at ~/.hermes/auth.json instead of the previous ~/.codex/auth.json. This change includes refactoring related functions for reading and saving tokens, ensuring better management of authentication states and preventing conflicts between different applications. Adjusted tests to reflect the new storage structure and improved error handling for missing or malformed tokens.
2026-03-01 19:59:24 -08:00
teknium1
8bc2de4ab6 feat(provider-routing): add OpenRouter provider routing configuration
Introduced a new `provider_routing` section in the CLI configuration to control how requests are routed across providers when using OpenRouter. This includes options for sorting providers by throughput, latency, or price, as well as allowing or ignoring specific providers, setting the order of provider attempts, and managing data collection policies. Updated relevant classes and documentation to support these features, enhancing flexibility in provider selection.
2026-03-01 18:24:27 -08:00
teknium1
11f5c1ecf0 fix(tests): use bare @pytest.mark.asyncio for hook emit tests
Remove loop_scope="function" parameter from async test decorators in
test_hooks.py. This matches the existing convention in the repo
(test_telegram_documents.py) and avoids requiring pytest-asyncio 0.23+.

All 144 new tests from PR #191 now pass.
2026-03-01 05:28:55 -08:00
0xbyt4
3b745633e4
test: add unit tests for 8 untested modules (batch 3) (#191)
* test: add unit tests for 8 untested modules (batch 3)

New test files (143 tests total):
- tools/debug_helpers.py: DebugSession enable/disable, log, save, session info
- tools/skills_guard.py: scan_file, scan_skill, trust levels, install policy, structural checks
- tools/skills_sync.py: manifest read/write, skill discovery, sync logic
- gateway/sticker_cache.py: cache CRUD, sticker injection text builders
- gateway/channel_directory.py: channel resolution, display formatting, session building
- gateway/hooks.py: hook discovery, sync/async emit, wildcard matching
- gateway/mirror.py: session lookup, JSONL append, mirror_to_session
- honcho_integration/client.py: config from env/file, session name resolution, linked workspaces

Also documents a gap in skills_guard: multi-word prompt injection
variants like "ignore all prior instructions" bypass the regex scanner.

* test: strengthen sticker injection tests with exact format assertions

Replace loose "contains" checks with exact output matching for
build_sticker_injection and build_animated_sticker_injection.
Add edge cases: set_name without emoji, empty description, empty emoji.

* test: remove skills_guard gap-documenting test to avoid conflict with fix PR
2026-03-01 05:28:12 -08:00
0xbyt4
900d48714a Merge remote-tracking branch 'origin/main' into test/expand-coverage-4
# Conflicts:
#	tests/agent/test_auxiliary_client.py
2026-03-01 12:11:54 +03:00
0xbyt4
3fdf03390e Merge remote-tracking branch 'origin/main' into feature/homeassistant-integration
# Conflicts:
#	run_agent.py
2026-03-01 11:59:12 +03:00
0xbyt4
25fb9aafcb fix: add service domain blocklist and entity_id validation to HA tools
Block dangerous HA service domains (shell_command, command_line,
python_script, pyscript, hassio, rest_command) that allow arbitrary
code execution or SSRF. Add regex validation for entity_id to prevent
path traversal attacks. 17 new tests covering both security features.
2026-03-01 11:53:50 +03:00
Bartok Moltbot
ed0e860abb fix(honcho): auto-enable when API key is present
Fixes #241

When users set HONCHO_API_KEY via `hermes config set` or environment
variable, they expect the integration to activate. Previously, the
`enabled` flag defaulted to `false` when reading from global config,
requiring users to also explicitly enable Honcho.

This change auto-enables Honcho when:
- An API key is present (from config file or env var)
- AND `enabled` is not explicitly set to `false` in the config

Users who want to disable Honcho while keeping the API key can still
set `enabled: false` in their config.

Also adds unit tests for the auto-enable behavior.
2026-03-01 03:12:37 -05:00
teknium1
41d8a80226 fix(display): fix subagent progress tree-view visual nits
Two fixes to the subagent progress display from PR #186:

1. Task index prefix: show 1-indexed prefix ([1], [2], ...) for ALL
   tasks in batch mode (task_count > 1). Single tasks get no prefix.
   Previously task 0 had no prefix while others did, making batch
   output confusing.

2. Completion indicator: use spinner.print_above() instead of raw
   print() for per-task completion lines (✓ [1/2] ...). Raw print
   collided with the active spinner, mushing the completion text
   onto the spinner line. Now prints cleanly above.

Added task_count parameter to _build_child_progress_callback and
_run_single_child. Updated tests accordingly.
2026-02-28 23:29:49 -08:00
teknium1
4ec386cc72 fix(display): use spaces instead of ANSI \033[K in print_above() for prompt_toolkit compat
print_above() used \033[K (erase-to-end-of-line) to clear the spinner
line before printing text above it. This causes garbled escape codes when
prompt_toolkit's patch_stdout is active in CLI mode.

Switched to the same spaces-based clearing approach used by stop() —
overwrite with blanks, then carriage return back to start of line.

Updated test assertion to match the new clearing method.
2026-02-28 23:19:23 -08:00
lila
dd69f16c3e
feat(gateway): expose subagent tool calls and thinking to user (fixes #169) (#186)
When subagents run via delegate_task, the user now sees real-time
progress instead of silence:

CLI: tree-view activity lines print above the delegation spinner
  🔀 Delegating: research quantum computing
     ├─ 💭 "I'll search for papers first..."
     ├─ 🔍 web_search  "quantum computing"
     ├─ 📖 read_file  "paper.pdf"
     └─ ⠹ working... (18.2s)

Gateway (Telegram/Discord): batched progress summaries sent every
5 tool calls to avoid message spam. Remaining tools flushed on
subagent completion.

Changes:
- agent/display.py: add KawaiiSpinner.print_above() to print
  status lines above an active spinner without disrupting animation.
  Uses captured stdout (self._out) so it works inside the child's
  redirect_stdout(devnull).

- tools/delegate_tool.py: add _build_child_progress_callback()
  that creates a per-child callback relaying tool calls and
  thinking events to the parent's spinner (CLI) or progress
  queue (gateway). Each child gets its own callback instance,
  so parallel subagents don't share state. Includes _flush()
  for gateway batch completion.

- run_agent.py: fire tool_progress_callback with '_thinking'
  event when the model produces text content. Guarded by
  _delegate_depth > 0 so only subagents fire this (prevents
  gateway spam from main agent). REASONING_SCRATCHPAD/think/
  reasoning XML tags are stripped before display.

Tests: 21 new tests covering print_above, callback builder,
thinking relay, SCRATCHPAD filtering, batching, flush, thread
isolation, delegate_depth guard, and prefix handling.
2026-02-28 23:18:00 -08:00
teknium1
1db5598294 feat(tests): add live integration tests for file operations and shell noise filtering
- Introduce a new test suite in `test_file_tools_live.py` to validate file operations and ensure accurate command execution in a real environment.
- Implement assertions to check for shell noise contamination in outputs, enhancing the reliability of command results.
- Create fixtures for setting up a local environment and populating directories with known file contents for comprehensive testing.
- Refactor shell noise handling in `process_registry.py` and `local.py` to support multiple noise patterns, improving output cleanliness.
2026-02-28 22:57:58 -08:00
teknium1
70dfec9638 test(redact): add sensitive text redaction
- Introduce a new test suite for the `redact_sensitive_text` function, covering various sensitive data formats including API keys, tokens, and environment variables.
- Ensure that sensitive information is properly masked in logs and outputs while non-sensitive data remains unchanged.
- Add tests for different scenarios including JSON fields, authorization headers, and environment variable assignments.
- Implement a redacting formatter for logging to enhance security during log output.
2026-02-28 21:56:27 -08:00
teknium1
500f0eab4a refactor(cli): Finalize OpenAI Codex Integration with OAuth
- Enhanced Codex model discovery by fetching available models from the API, with fallback to local cache and defaults.
- Updated the context compressor's summary target tokens to 2500 for improved performance.
- Added external credential detection for Codex CLI to streamline authentication.
- Refactored various components to ensure consistent handling of authentication and model selection across the application.
2026-02-28 21:47:51 -08:00
Teknium
5a79e423fe
Merge branch 'main' into codex/align-codex-provider-conventions-mainrepo 2026-02-28 18:13:38 -08:00
Farukest
7166647ca1
fix(security): add re.DOTALL to prevent multiline bypass of dangerous command detection 2026-03-01 03:23:29 +03:00
Farukest
f7300a858e
fix(tools): use task-specific glob pattern in disk usage calculation 2026-03-01 03:17:50 +03:00
Farukest
e87859e82c
fix(agent): copy conversation_history to avoid mutating caller's list 2026-03-01 03:06:13 +03:00
Farukest
de101a8202
fix(agent): strip _flush_sentinel from API messages 2026-03-01 02:51:31 +03:00
Farukest
7f1f4c2248
fix(tools): preserve empty content in ReadResult.to_dict() 2026-03-01 02:42:15 +03:00
Farukest
c33f8d381b
fix: correct off-by-one in retry exhaustion checks
The retry exhaustion checks used > instead of >= to compare
retry_count against max_retries. Since the while loop condition is
retry_count < max_retries, the check retry_count > max_retries can
never be true inside the loop. When retries are exhausted, the loop
exits and falls through to response.choices[0] on an invalid response,
crashing with IndexError instead of returning a proper error.
2026-03-01 02:27:26 +03:00
Farukest
3f58e47c63
fix: guard POSIX-only process functions for Windows compatibility
os.setsid, os.killpg, and os.getpgid do not exist on Windows and raise
AttributeError on import or first call. This breaks the terminal tool,
code execution sandbox, process registry, and WhatsApp bridge on Windows.

Added _IS_WINDOWS platform guard in all four affected files, following
the pattern documented in CONTRIBUTING.md. On Windows, preexec_fn is
set to None and process termination falls back to proc.terminate() /
proc.kill() instead of process group signals.

Files changed:
- tools/environments/local.py (3 call sites)
- tools/process_registry.py (2 call sites)
- tools/code_execution_tool.py (3 call sites)
- gateway/platforms/whatsapp.py (3 call sites)
2026-03-01 01:54:27 +03:00
Farukest
b7f8a17c24
fix(gateway): persist transcript changes in /retry, /undo and fix /reset
/retry and /undo set session_entry.conversation_history which does not
exist on SessionEntry. The truncated history was never written to disk,
so the next message reload picked up the full unmodified transcript.

Added SessionStore.rewrite_transcript() that persists changes to both
the JSONL file and SQLite database, and updated both commands to use it.

/reset accessed self.session_store._sessions which does not exist on
SessionStore (the correct attribute is _entries). Also replaced the
hand-coded session key with _generate_session_key() to fix WhatsApp DM
sessions using the wrong key format.

Closes #210
2026-03-01 01:40:30 +03:00
0xbyt4
b759602483 fix: prevent italic regex from spanning newlines in Telegram formatter
The italic regex \*([^*]+)\* used [^*] which matches newlines, causing
bullet lists with * markers to be incorrectly converted to italic text.
Changed to [^*\n]+ to prevent cross-line matching.

Adds 43 tests for _escape_mdv2 and format_message covering code blocks,
bold/italic, headers, links, mixed formatting, and the regression case.
2026-02-28 22:01:48 +03:00
0xbyt4
9769e07cd5 test: add 25 unit tests for trajectory_compressor
Tests cover CompressionConfig (defaults, from_yaml with full/partial/empty),
TrajectoryMetrics and AggregateMetrics (to_dict, aggregation, division-by-zero
guards), _find_protected_indices (basic, all-protected, no tail, missing roles,
disabled protection), _extract_turn_content_for_summary (basic, truncation,
empty range), and token counting (empty, basic, trajectory, fallback on error).
2026-02-28 21:28:28 +03:00
0xbyt4
08250a53a1 fix: skills hub dedup prefers higher trust levels + 43 tests
- unified_search and GitHubSource.search dedup: replace naive
  `trust_level == "trusted"` check with ranked comparison so
  "builtin" results are never overwritten by "trusted" or "community"
- Add 43 unit tests covering _parse_frontmatter_quick, trust_level_for,
  HubLockFile CRUD, TapsManager ops, LobeHub _convert_to_skill_md,
  unified_search dedup (with regression test), and append_audit_log
2026-02-28 21:25:55 +03:00