mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-25 00:51:20 +00:00
feat(memory): pluggable memory provider interface with profile isolation, review fixes, and honcho CLI restoration (#4623)
* feat(memory): add pluggable memory provider interface with profile isolation Introduces a pluggable MemoryProvider ABC so external memory backends can integrate with Hermes without modifying core files. Each backend becomes a plugin implementing a standard interface, orchestrated by MemoryManager. Key architecture: - agent/memory_provider.py — ABC with core + optional lifecycle hooks - agent/memory_manager.py — single integration point in the agent loop - agent/builtin_memory_provider.py — wraps existing MEMORY.md/USER.md Profile isolation fixes applied to all 6 shipped plugins: - Cognitive Memory: use get_hermes_home() instead of raw env var - Hindsight Memory: check $HERMES_HOME/hindsight/config.json first, fall back to legacy ~/.hindsight/ for backward compat - Hermes Memory Store: replace hardcoded ~/.hermes paths with get_hermes_home() for config loading and DB path defaults - Mem0 Memory: use get_hermes_home() instead of raw env var - RetainDB Memory: auto-derive profile-scoped project name from hermes_home path (hermes-<profile>), explicit env var overrides - OpenViking Memory: read-only, no local state, isolation via .env MemoryManager.initialize_all() now injects hermes_home into kwargs so every provider can resolve profile-scoped storage without importing get_hermes_home() themselves. Plugin system: adds register_memory_provider() to PluginContext and get_plugin_memory_providers() accessor. Based on PR #3825. 46 tests (37 unit + 5 E2E + 4 plugin registration). * refactor(memory): drop cognitive plugin, rewrite OpenViking as full provider Remove cognitive-memory plugin (#727) — core mechanics are broken: decay runs 24x too fast (hourly not daily), prefetch uses row ID as timestamp, search limited by importance not similarity. Rewrite openviking-memory plugin from a read-only search wrapper into a full bidirectional memory provider using the complete OpenViking session lifecycle API: - sync_turn: records user/assistant messages to OpenViking session (threaded, non-blocking) - on_session_end: commits session to trigger automatic memory extraction into 6 categories (profile, preferences, entities, events, cases, patterns) - prefetch: background semantic search via find() endpoint - on_memory_write: mirrors built-in memory writes to the session - is_available: checks env var only, no network calls (ABC compliance) Tools expanded from 3 to 5: - viking_search: semantic search with mode/scope/limit - viking_read: tiered content (abstract ~100tok / overview ~2k / full) - viking_browse: filesystem-style navigation (list/tree/stat) - viking_remember: explicit memory storage via session - viking_add_resource: ingest URLs/docs into knowledge base Uses direct HTTP via httpx (no openviking SDK dependency needed). Response truncation on viking_read to prevent context flooding. * fix(memory): harden Mem0 plugin — thread safety, non-blocking sync, circuit breaker - Remove redundant mem0_context tool (identical to mem0_search with rerank=true, top_k=5 — wastes a tool slot and confuses the model) - Thread sync_turn so it's non-blocking — Mem0's server-side LLM extraction can take 5-10s, was stalling the agent after every turn - Add threading.Lock around _get_client() for thread-safe lazy init (prefetch and sync threads could race on first client creation) - Add circuit breaker: after 5 consecutive API failures, pause calls for 120s instead of hammering a down server every turn. Auto-resets after cooldown. Logs a warning when tripped. - Track success/failure in prefetch, sync_turn, and all tool calls - Wait for previous sync to finish before starting a new one (prevents unbounded thread accumulation on rapid turns) - Clean up shutdown to join both prefetch and sync threads * fix(memory): enforce single external memory provider limit MemoryManager now rejects a second non-builtin provider with a warning. Built-in memory (MEMORY.md/USER.md) is always accepted. Only ONE external plugin provider is allowed at a time. This prevents tool schema bloat (some providers add 3-5 tools each) and conflicting memory backends. The warning message directs users to configure memory.provider in config.yaml to select which provider to activate. Updated all 47 tests to use builtin + one external pattern instead of multiple externals. Added test_second_external_rejected to verify the enforcement. * feat(memory): add ByteRover memory provider plugin Implements the ByteRover integration (from PR #3499 by hieuntg81) as a MemoryProvider plugin instead of direct run_agent.py modifications. ByteRover provides persistent memory via the brv CLI — a hierarchical knowledge tree with tiered retrieval (fuzzy text then LLM-driven search). Local-first with optional cloud sync. Plugin capabilities: - prefetch: background brv query for relevant context - sync_turn: curate conversation turns (threaded, non-blocking) - on_memory_write: mirror built-in memory writes to brv - on_pre_compress: extract insights before context compression Tools (3): - brv_query: search the knowledge tree - brv_curate: store facts/decisions/patterns - brv_status: check CLI version and context tree state Profile isolation: working directory at $HERMES_HOME/byterover/ (scoped per profile). Binary resolution cached with thread-safe double-checked locking. All write operations threaded to avoid blocking the agent (curate can take 120s with LLM processing). * fix(memory): thread remaining sync_turns, fix holographic, add config key Plugin fixes: - Hindsight: thread sync_turn (was blocking up to 30s via _run_in_thread) - RetainDB: thread sync_turn (was blocking on HTTP POST) - Both: shutdown now joins sync threads alongside prefetch threads Holographic retrieval fixes: - reason(): removed dead intersection_key computation (bundled but never used in scoring). Now reuses pre-computed entity_residuals directly, moved role_content encoding outside the inner loop. - contradict(): added _MAX_CONTRADICT_FACTS=500 scaling guard. Above 500 facts, only checks the most recently updated ones to avoid O(n^2) explosion (~125K comparisons at 500 is acceptable). Config: - Added memory.provider key to DEFAULT_CONFIG ("" = builtin only). No version bump needed (deep_merge handles new keys automatically). * feat(memory): extract Honcho as a MemoryProvider plugin Creates plugins/honcho-memory/ as a thin adapter over the existing honcho_integration/ package. All 4 Honcho tools (profile, search, context, conclude) move from the normal tool registry to the MemoryProvider interface. The plugin delegates all work to HonchoSessionManager — no Honcho logic is reimplemented. It uses the existing config chain: $HERMES_HOME/honcho.json -> ~/.honcho/config.json -> env vars. Lifecycle hooks: - initialize: creates HonchoSessionManager via existing client factory - prefetch: background dialectic query - sync_turn: records messages + flushes to API (threaded) - on_memory_write: mirrors user profile writes as conclusions - on_session_end: flushes all pending messages This is a prerequisite for the MemoryManager wiring in run_agent.py. Once wired, Honcho goes through the same provider interface as all other memory plugins, and the scattered Honcho code in run_agent.py can be consolidated into the single MemoryManager integration point. * feat(memory): wire MemoryManager into run_agent.py Adds 8 integration points for the external memory provider plugin, all purely additive (zero existing code modified): 1. Init (~L1130): Create MemoryManager, find matching plugin provider from memory.provider config, initialize with session context 2. Tool injection (~L1160): Append provider tool schemas to self.tools and self.valid_tool_names after memory_manager init 3. System prompt (~L2705): Add external provider's system_prompt_block alongside existing MEMORY.md/USER.md blocks 4. Tool routing (~L5362): Route provider tool calls through memory_manager.handle_tool_call() before the catchall handler 5. Memory write bridge (~L5353): Notify external provider via on_memory_write() when the built-in memory tool writes 6. Pre-compress (~L5233): Call on_pre_compress() before context compression discards messages 7. Prefetch (~L6421): Inject provider prefetch results into the current-turn user message (same pattern as Honcho turn context) 8. Turn sync + session end (~L8161, ~L8172): sync_all() after each completed turn, queue_prefetch_all() for next turn, on_session_end() + shutdown_all() at conversation end All hooks are wrapped in try/except — a failing provider never breaks the agent. The existing memory system, Honcho integration, and all other code paths are completely untouched. Full suite: 7222 passed, 4 pre-existing failures. * refactor(memory): remove legacy Honcho integration from core Extracts all Honcho-specific code from run_agent.py, model_tools.py, toolsets.py, and gateway/run.py. Honcho is now exclusively available as a memory provider plugin (plugins/honcho-memory/). Removed from run_agent.py (-457 lines): - Honcho init block (session manager creation, activation, config) - 8 Honcho methods: _honcho_should_activate, _strip_honcho_tools, _activate_honcho, _register_honcho_exit_hook, _queue_honcho_prefetch, _honcho_prefetch, _honcho_save_user_observation, _honcho_sync - _inject_honcho_turn_context module-level function - Honcho system prompt block (tool descriptions, CLI commands) - Honcho context injection in api_messages building - Honcho params from __init__ (honcho_session_key, honcho_manager, honcho_config) - HONCHO_TOOL_NAMES constant - All honcho-specific tool dispatch forwarding Removed from other files: - model_tools.py: honcho_tools import, honcho params from handle_function_call - toolsets.py: honcho toolset definition, honcho tools from core tools list - gateway/run.py: honcho params from AIAgent constructor calls Removed tests (-339 lines): - 9 Honcho-specific test methods from test_run_agent.py - TestHonchoAtexitFlush class from test_exit_cleanup_interrupt.py Restored two regex constants (_SURROGATE_RE, _BUDGET_WARNING_RE) that were accidentally removed during the honcho function extraction. The honcho_integration/ package is kept intact — the plugin delegates to it. tools/honcho_tools.py registry entries are now dead code (import commented out in model_tools.py) but the file is preserved for reference. Full suite: 7207 passed, 4 pre-existing failures. Zero regressions. * refactor(memory): restructure plugins, add CLI, clean gateway, migration notice Plugin restructure: - Move all memory plugins from plugins/<name>-memory/ to plugins/memory/<name>/ (byterover, hindsight, holographic, honcho, mem0, openviking, retaindb) - New plugins/memory/__init__.py discovery module that scans the directory directly, loading providers by name without the general plugin system - run_agent.py uses load_memory_provider() instead of get_plugin_memory_providers() CLI wiring: - hermes memory setup — interactive curses picker + config wizard - hermes memory status — show active provider, config, availability - hermes memory off — disable external provider (built-in only) - hermes honcho — now shows migration notice pointing to hermes memory setup Gateway cleanup: - Remove _get_or_create_gateway_honcho (already removed in prev commit) - Remove _shutdown_gateway_honcho and _shutdown_all_gateway_honcho methods - Remove all calls to shutdown methods (4 call sites) - Remove _honcho_managers/_honcho_configs dict references Dead code removal: - Delete tools/honcho_tools.py (279 lines, import was already commented out) - Delete tests/gateway/test_honcho_lifecycle.py (131 lines, tested removed methods) - Remove if False placeholder from run_agent.py Migration: - Honcho migration notice on startup: detects existing honcho.json or ~/.honcho/config.json, prints guidance to run hermes memory setup. Only fires when memory.provider is not set and not in quiet mode. Full suite: 7203 passed, 4 pre-existing failures. Zero regressions. * feat(memory): standardize plugin config + add per-plugin documentation Config architecture: - Add save_config(values, hermes_home) to MemoryProvider ABC - Honcho: writes to $HERMES_HOME/honcho.json (SDK native) - Mem0: writes to $HERMES_HOME/mem0.json - Hindsight: writes to $HERMES_HOME/hindsight/config.json - Holographic: writes to config.yaml under plugins.hermes-memory-store - OpenViking/RetainDB/ByteRover: env-var only (default no-op) Setup wizard (hermes memory setup): - Now calls provider.save_config() for non-secret config - Secrets still go to .env via env vars - Only memory.provider activation key goes to config.yaml Documentation: - README.md for each of the 7 providers in plugins/memory/<name>/ - Requirements, setup (wizard + manual), config reference, tools table - Consistent format across all providers The contract for new memory plugins: - get_config_schema() declares all fields (REQUIRED) - save_config() writes native config (REQUIRED if not env-var-only) - Secrets use env_var field in schema, written to .env by wizard - README.md in the plugin directory * docs: add memory providers user guide + developer guide New pages: - user-guide/features/memory-providers.md — comprehensive guide covering all 7 shipped providers (Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover). Each with setup, config, tools, cost, and unique features. Includes comparison table and profile isolation notes. - developer-guide/memory-provider-plugin.md — how to build a new memory provider plugin. Covers ABC, required methods, config schema, save_config, threading contract, profile isolation, testing. Updated pages: - user-guide/features/memory.md — replaced Honcho section with link to new Memory Providers page - user-guide/features/honcho.md — replaced with migration redirect to the new Memory Providers page - sidebars.ts — added both new pages to navigation * fix(memory): auto-migrate Honcho users to memory provider plugin When honcho.json or ~/.honcho/config.json exists but memory.provider is not set, automatically set memory.provider: honcho in config.yaml and activate the plugin. The plugin reads the same config files, so all data and credentials are preserved. Zero user action needed. Persists the migration to config.yaml so it only fires once. Prints a one-line confirmation in non-quiet mode. * fix(memory): only auto-migrate Honcho when enabled + credentialed Check HonchoClientConfig.enabled AND (api_key OR base_url) before auto-migrating — not just file existence. Prevents false activation for users who disabled Honcho, stopped using it (config lingers), or have ~/.honcho/ from a different tool. * feat(memory): auto-install pip dependencies during hermes memory setup Reads pip_dependencies from plugin.yaml, checks which are missing, installs them via pip before config walkthrough. Also shows install guidance for external_dependencies (e.g. brv CLI for ByteRover). Updated all 7 plugin.yaml files with pip_dependencies: - honcho: honcho-ai - mem0: mem0ai - openviking: httpx - hindsight: hindsight-client - holographic: (none) - retaindb: requests - byterover: (external_dependencies for brv CLI) * fix: remove remaining Honcho crash risks from cli.py and gateway cli.py: removed Honcho session re-mapping block (would crash importing deleted tools/honcho_tools.py), Honcho flush on compress, Honcho session display on startup, Honcho shutdown on exit, honcho_session_key AIAgent param. gateway/run.py: removed honcho_session_key params from helper methods, sync_honcho param, _honcho.shutdown() block. tests: fixed test_cron_session_with_honcho_key_skipped (was passing removed honcho_key param to _flush_memories_for_session). * fix: include plugins/ in pyproject.toml package list Without this, plugins/memory/ wouldn't be included in non-editable installs. Hermes always runs from the repo checkout so this is belt- and-suspenders, but prevents breakage if the install method changes. * fix(memory): correct pip-to-import name mapping for dep checks The heuristic dep.replace('-', '_') fails for packages where the pip name differs from the import name: honcho-ai→honcho, mem0ai→mem0, hindsight-client→hindsight_client. Added explicit mapping table so hermes memory setup doesn't try to reinstall already-installed packages. * chore: remove dead code from old plugin memory registration path - hermes_cli/plugins.py: removed register_memory_provider(), _memory_providers list, get_plugin_memory_providers() — memory providers now use plugins/memory/ discovery, not the general plugin system - hermes_cli/main.py: stripped 74 lines of dead honcho argparse subparsers (setup, status, sessions, map, peer, mode, tokens, identity, migrate) — kept only the migration redirect - agent/memory_provider.py: updated docstring to reflect new registration path - tests: replaced TestPluginMemoryProviderRegistration with TestPluginMemoryDiscovery that tests the actual plugins/memory/ discovery system. Added 3 new tests (discover, load, nonexistent). * chore: delete dead honcho_integration/cli.py and its tests cli.py (794 lines) was the old 'hermes honcho' command handler — nobody calls it since cmd_honcho was replaced with a migration redirect. Deleted tests that imported from removed code: - tests/honcho_integration/test_cli.py (tested _resolve_api_key) - tests/honcho_integration/test_config_isolation.py (tested CLI config paths) - tests/tools/test_honcho_tools.py (tested the deleted tools/honcho_tools.py) Remaining honcho_integration/ files (actively used by the plugin): - client.py (445 lines) — config loading, SDK client creation - session.py (991 lines) — session management, queries, flush * refactor: move honcho_integration/ into the honcho plugin Moves client.py (445 lines) and session.py (991 lines) from the top-level honcho_integration/ package into plugins/memory/honcho/. No Honcho code remains in the main codebase. - plugins/memory/honcho/client.py — config loading, SDK client creation - plugins/memory/honcho/session.py — session management, queries, flush - Updated all imports: run_agent.py (auto-migration), hermes_cli/doctor.py, plugin __init__.py, session.py cross-import, all tests - Removed honcho_integration/ package and pyproject.toml entry - Renamed tests/honcho_integration/ → tests/honcho_plugin/ * docs: update architecture + gateway-internals for memory provider system - architecture.md: replaced honcho_integration/ with plugins/memory/ - gateway-internals.md: replaced Honcho-specific session routing and flush lifecycle docs with generic memory provider interface docs * fix: update stale mock path for resolve_active_host after honcho plugin migration * fix(memory): address review feedback — P0 lifecycle, ABC contract, honcho CLI restore Review feedback from Honcho devs (erosika): P0 — Provider lifecycle: - Remove on_session_end() + shutdown_all() from run_conversation() tail (was killing providers after every turn in multi-turn sessions) - Add shutdown_memory_provider() method on AIAgent for callers - Wire shutdown into CLI atexit, reset_conversation, gateway stop/expiry Bug fixes: - Remove sync_honcho=False kwarg from /btw callsites (TypeError crash) - Fix doctor.py references to dead 'hermes honcho setup' command - Cache prefetch_all() before tool loop (was re-calling every iteration) ABC contract hardening (all backwards-compatible): - Add session_id kwarg to prefetch/sync_turn/queue_prefetch - Make on_pre_compress() return str (provider insights in compression) - Add **kwargs to on_turn_start() for runtime context - Add on_delegation() hook for parent-side subagent observation - Document agent_context/agent_identity/agent_workspace kwargs on initialize() (prevents cron corruption, enables profile scoping) - Fix docstring: single external provider, not multiple Honcho CLI restoration: - Add plugins/memory/honcho/cli.py (from main's honcho_integration/cli.py with imports adapted to plugin path) - Restore full hermes honcho command with all subcommands (status, peer, mode, tokens, identity, enable/disable, sync, peers, --target-profile) - Restore auto-clone on profile creation + sync on hermes update - hermes honcho setup now redirects to hermes memory setup * fix(memory): wire on_delegation, skip_memory for cron/flush, fix ByteRover return type - Wire on_delegation() in delegate_tool.py — parent's memory provider is notified with task+result after each subagent completes - Add skip_memory=True to cron scheduler (prevents cron system prompts from corrupting user representations — closes #4052) - Add skip_memory=True to gateway flush agent (throwaway agent shouldn't activate memory provider) - Fix ByteRover on_pre_compress() return type: None -> str * fix(honcho): port profile isolation fixes from PR #4632 Ports 5 bug fixes found during profile testing (erosika's PR #4632): 1. 3-tier config resolution — resolve_config_path() now checks $HERMES_HOME/honcho.json → ~/.hermes/honcho.json → ~/.honcho/config.json (non-default profiles couldn't find shared host blocks) 2. Thread host=_host_key() through from_global_config() in cmd_setup, cmd_status, cmd_identity (--target-profile was being ignored) 3. Use bare profile name as aiPeer (not host key with dots) — Honcho's peer ID pattern is ^[a-zA-Z0-9_-]+$, dots are invalid 4. Wrap add_peers() in try/except — was fatal on new AI peers, killed all message uploads for the session 5. Gate Honcho clone behind --clone/--clone-all on profile create (bare create should be blank-slate) Also: sanitize assistant_peer_id via _sanitize_id() * fix(tests): add module cleanup fixture to test_cli_provider_resolution test_cli_provider_resolution._import_cli() wipes tools.*, cli, and run_agent from sys.modules to force fresh imports, but had no cleanup. This poisoned all subsequent tests on the same xdist worker — mocks targeting tools.file_tools, tools.send_message_tool, etc. patched the NEW module object while already-imported functions still referenced the OLD one. Caused ~25 cascade failures: send_message KeyError, process_registry FileNotFoundError, file_read_guards timeouts, read_loop_detection file-not-found, mcp_oauth None port, and provider_parity/codex_execution stale tool lists. Fix: autouse fixture saves all affected modules before each test and restores them after, matching the pattern in test_managed_browserbase_and_modal.py.
This commit is contained in:
parent
e0b2bdb089
commit
924bc67eee
69 changed files with 7501 additions and 2317 deletions
113
agent/builtin_memory_provider.py
Normal file
113
agent/builtin_memory_provider.py
Normal file
|
|
@ -0,0 +1,113 @@
|
||||||
|
"""BuiltinMemoryProvider — wraps MEMORY.md / USER.md as a MemoryProvider.
|
||||||
|
|
||||||
|
Always registered as the first provider. Cannot be disabled or removed.
|
||||||
|
This is the existing Hermes memory system exposed through the provider
|
||||||
|
interface for compatibility with the MemoryManager.
|
||||||
|
|
||||||
|
The actual storage logic lives in tools/memory_tool.py (MemoryStore).
|
||||||
|
This provider is a thin adapter that delegates to MemoryStore and
|
||||||
|
exposes the memory tool schema.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class BuiltinMemoryProvider(MemoryProvider):
|
||||||
|
"""Built-in file-backed memory (MEMORY.md + USER.md).
|
||||||
|
|
||||||
|
Always active, never disabled by other providers. The `memory` tool
|
||||||
|
is handled by run_agent.py's agent-level tool interception (not through
|
||||||
|
the normal registry), so get_tool_schemas() returns an empty list —
|
||||||
|
the memory tool is already wired separately.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
memory_store=None,
|
||||||
|
memory_enabled: bool = False,
|
||||||
|
user_profile_enabled: bool = False,
|
||||||
|
):
|
||||||
|
self._store = memory_store
|
||||||
|
self._memory_enabled = memory_enabled
|
||||||
|
self._user_profile_enabled = user_profile_enabled
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "builtin"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
"""Built-in memory is always available."""
|
||||||
|
return True
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
"""Load memory from disk if not already loaded."""
|
||||||
|
if self._store is not None:
|
||||||
|
self._store.load_from_disk()
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
"""Return MEMORY.md and USER.md content for the system prompt.
|
||||||
|
|
||||||
|
Uses the frozen snapshot captured at load time. This ensures the
|
||||||
|
system prompt stays stable throughout a session (preserving the
|
||||||
|
prompt cache), even though the live entries may change via tool calls.
|
||||||
|
"""
|
||||||
|
if not self._store:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
parts = []
|
||||||
|
if self._memory_enabled:
|
||||||
|
mem_block = self._store.format_for_system_prompt("memory")
|
||||||
|
if mem_block:
|
||||||
|
parts.append(mem_block)
|
||||||
|
if self._user_profile_enabled:
|
||||||
|
user_block = self._store.format_for_system_prompt("user")
|
||||||
|
if user_block:
|
||||||
|
parts.append(user_block)
|
||||||
|
|
||||||
|
return "\n\n".join(parts)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
"""Built-in memory doesn't do query-based recall — it's injected via system_prompt_block."""
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Built-in memory doesn't auto-sync turns — writes happen via the memory tool."""
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
"""Return empty list.
|
||||||
|
|
||||||
|
The `memory` tool is an agent-level intercepted tool, handled
|
||||||
|
specially in run_agent.py before normal tool dispatch. It's not
|
||||||
|
part of the standard tool registry. We don't duplicate it here.
|
||||||
|
"""
|
||||||
|
return []
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: Dict[str, Any], **kwargs) -> str:
|
||||||
|
"""Not used — the memory tool is intercepted in run_agent.py."""
|
||||||
|
return json.dumps({"error": "Built-in memory tool is handled by the agent loop"})
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
"""No cleanup needed — files are saved on every write."""
|
||||||
|
|
||||||
|
# -- Property access for backward compatibility --------------------------
|
||||||
|
|
||||||
|
@property
|
||||||
|
def store(self):
|
||||||
|
"""Access the underlying MemoryStore for legacy code paths."""
|
||||||
|
return self._store
|
||||||
|
|
||||||
|
@property
|
||||||
|
def memory_enabled(self) -> bool:
|
||||||
|
return self._memory_enabled
|
||||||
|
|
||||||
|
@property
|
||||||
|
def user_profile_enabled(self) -> bool:
|
||||||
|
return self._user_profile_enabled
|
||||||
335
agent/memory_manager.py
Normal file
335
agent/memory_manager.py
Normal file
|
|
@ -0,0 +1,335 @@
|
||||||
|
"""MemoryManager — orchestrates the built-in memory provider plus at most
|
||||||
|
ONE external plugin memory provider.
|
||||||
|
|
||||||
|
Single integration point in run_agent.py. Replaces scattered per-backend
|
||||||
|
code with one manager that delegates to registered providers.
|
||||||
|
|
||||||
|
The BuiltinMemoryProvider is always registered first and cannot be removed.
|
||||||
|
Only ONE external (non-builtin) provider is allowed at a time — attempting
|
||||||
|
to register a second external provider is rejected with a warning. This
|
||||||
|
prevents tool schema bloat and conflicting memory backends.
|
||||||
|
|
||||||
|
Usage in run_agent.py:
|
||||||
|
self._memory_manager = MemoryManager()
|
||||||
|
self._memory_manager.add_provider(BuiltinMemoryProvider(...))
|
||||||
|
# Only ONE of these:
|
||||||
|
self._memory_manager.add_provider(plugin_provider)
|
||||||
|
|
||||||
|
# System prompt
|
||||||
|
prompt_parts.append(self._memory_manager.build_system_prompt())
|
||||||
|
|
||||||
|
# Pre-turn
|
||||||
|
context = self._memory_manager.prefetch_all(user_message)
|
||||||
|
|
||||||
|
# Post-turn
|
||||||
|
self._memory_manager.sync_all(user_msg, assistant_response)
|
||||||
|
self._memory_manager.queue_prefetch_all(user_msg)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class MemoryManager:
|
||||||
|
"""Orchestrates the built-in provider plus at most one external provider.
|
||||||
|
|
||||||
|
The builtin provider is always first. Only one non-builtin (external)
|
||||||
|
provider is allowed. Failures in one provider never block the other.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._providers: List[MemoryProvider] = []
|
||||||
|
self._tool_to_provider: Dict[str, MemoryProvider] = {}
|
||||||
|
self._has_external: bool = False # True once a non-builtin provider is added
|
||||||
|
|
||||||
|
# -- Registration --------------------------------------------------------
|
||||||
|
|
||||||
|
def add_provider(self, provider: MemoryProvider) -> None:
|
||||||
|
"""Register a memory provider.
|
||||||
|
|
||||||
|
Built-in provider (name ``"builtin"``) is always accepted.
|
||||||
|
Only **one** external (non-builtin) provider is allowed — a second
|
||||||
|
attempt is rejected with a warning.
|
||||||
|
"""
|
||||||
|
is_builtin = provider.name == "builtin"
|
||||||
|
|
||||||
|
if not is_builtin:
|
||||||
|
if self._has_external:
|
||||||
|
existing = next(
|
||||||
|
(p.name for p in self._providers if p.name != "builtin"), "unknown"
|
||||||
|
)
|
||||||
|
logger.warning(
|
||||||
|
"Rejected memory provider '%s' — external provider '%s' is "
|
||||||
|
"already registered. Only one external memory provider is "
|
||||||
|
"allowed at a time. Configure which one via memory.provider "
|
||||||
|
"in config.yaml.",
|
||||||
|
provider.name, existing,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
self._has_external = True
|
||||||
|
|
||||||
|
self._providers.append(provider)
|
||||||
|
|
||||||
|
# Index tool names → provider for routing
|
||||||
|
for schema in provider.get_tool_schemas():
|
||||||
|
tool_name = schema.get("name", "")
|
||||||
|
if tool_name and tool_name not in self._tool_to_provider:
|
||||||
|
self._tool_to_provider[tool_name] = provider
|
||||||
|
elif tool_name in self._tool_to_provider:
|
||||||
|
logger.warning(
|
||||||
|
"Memory tool name conflict: '%s' already registered by %s, "
|
||||||
|
"ignoring from %s",
|
||||||
|
tool_name,
|
||||||
|
self._tool_to_provider[tool_name].name,
|
||||||
|
provider.name,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Memory provider '%s' registered (%d tools)",
|
||||||
|
provider.name,
|
||||||
|
len(provider.get_tool_schemas()),
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def providers(self) -> List[MemoryProvider]:
|
||||||
|
"""All registered providers in order."""
|
||||||
|
return list(self._providers)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def provider_names(self) -> List[str]:
|
||||||
|
"""Names of all registered providers."""
|
||||||
|
return [p.name for p in self._providers]
|
||||||
|
|
||||||
|
def get_provider(self, name: str) -> Optional[MemoryProvider]:
|
||||||
|
"""Get a provider by name, or None if not registered."""
|
||||||
|
for p in self._providers:
|
||||||
|
if p.name == name:
|
||||||
|
return p
|
||||||
|
return None
|
||||||
|
|
||||||
|
# -- System prompt -------------------------------------------------------
|
||||||
|
|
||||||
|
def build_system_prompt(self) -> str:
|
||||||
|
"""Collect system prompt blocks from all providers.
|
||||||
|
|
||||||
|
Returns combined text, or empty string if no providers contribute.
|
||||||
|
Each non-empty block is labeled with the provider name.
|
||||||
|
"""
|
||||||
|
blocks = []
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
block = provider.system_prompt_block()
|
||||||
|
if block and block.strip():
|
||||||
|
blocks.append(block)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Memory provider '%s' system_prompt_block() failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
return "\n\n".join(blocks)
|
||||||
|
|
||||||
|
# -- Prefetch / recall ---------------------------------------------------
|
||||||
|
|
||||||
|
def prefetch_all(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
"""Collect prefetch context from all providers.
|
||||||
|
|
||||||
|
Returns merged context text labeled by provider. Empty providers
|
||||||
|
are skipped. Failures in one provider don't block others.
|
||||||
|
"""
|
||||||
|
parts = []
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
result = provider.prefetch(query, session_id=session_id)
|
||||||
|
if result and result.strip():
|
||||||
|
parts.append(result)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(
|
||||||
|
"Memory provider '%s' prefetch failed (non-fatal): %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
return "\n\n".join(parts)
|
||||||
|
|
||||||
|
def queue_prefetch_all(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
"""Queue background prefetch on all providers for the next turn."""
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
provider.queue_prefetch(query, session_id=session_id)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(
|
||||||
|
"Memory provider '%s' queue_prefetch failed (non-fatal): %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
|
||||||
|
# -- Sync ----------------------------------------------------------------
|
||||||
|
|
||||||
|
def sync_all(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Sync a completed turn to all providers."""
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
provider.sync_turn(user_content, assistant_content, session_id=session_id)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Memory provider '%s' sync_turn failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
|
||||||
|
# -- Tools ---------------------------------------------------------------
|
||||||
|
|
||||||
|
def get_all_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
"""Collect tool schemas from all providers."""
|
||||||
|
schemas = []
|
||||||
|
seen = set()
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
for schema in provider.get_tool_schemas():
|
||||||
|
name = schema.get("name", "")
|
||||||
|
if name and name not in seen:
|
||||||
|
schemas.append(schema)
|
||||||
|
seen.add(name)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Memory provider '%s' get_tool_schemas() failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
return schemas
|
||||||
|
|
||||||
|
def get_all_tool_names(self) -> set:
|
||||||
|
"""Return set of all tool names across all providers."""
|
||||||
|
return set(self._tool_to_provider.keys())
|
||||||
|
|
||||||
|
def has_tool(self, tool_name: str) -> bool:
|
||||||
|
"""Check if any provider handles this tool."""
|
||||||
|
return tool_name in self._tool_to_provider
|
||||||
|
|
||||||
|
def handle_tool_call(
|
||||||
|
self, tool_name: str, args: Dict[str, Any], **kwargs
|
||||||
|
) -> str:
|
||||||
|
"""Route a tool call to the correct provider.
|
||||||
|
|
||||||
|
Returns JSON string result. Raises ValueError if no provider
|
||||||
|
handles the tool.
|
||||||
|
"""
|
||||||
|
provider = self._tool_to_provider.get(tool_name)
|
||||||
|
if provider is None:
|
||||||
|
return json.dumps({"error": f"No memory provider handles tool '{tool_name}'"})
|
||||||
|
try:
|
||||||
|
return provider.handle_tool_call(tool_name, args, **kwargs)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(
|
||||||
|
"Memory provider '%s' handle_tool_call(%s) failed: %s",
|
||||||
|
provider.name, tool_name, e,
|
||||||
|
)
|
||||||
|
return json.dumps({"error": f"Memory tool '{tool_name}' failed: {e}"})
|
||||||
|
|
||||||
|
# -- Lifecycle hooks -----------------------------------------------------
|
||||||
|
|
||||||
|
def on_turn_start(self, turn_number: int, message: str, **kwargs) -> None:
|
||||||
|
"""Notify all providers of a new turn.
|
||||||
|
|
||||||
|
kwargs may include: remaining_tokens, model, platform, tool_count.
|
||||||
|
"""
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
provider.on_turn_start(turn_number, message, **kwargs)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(
|
||||||
|
"Memory provider '%s' on_turn_start failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
|
||||||
|
def on_session_end(self, messages: List[Dict[str, Any]]) -> None:
|
||||||
|
"""Notify all providers of session end."""
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
provider.on_session_end(messages)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(
|
||||||
|
"Memory provider '%s' on_session_end failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
|
||||||
|
def on_pre_compress(self, messages: List[Dict[str, Any]]) -> str:
|
||||||
|
"""Notify all providers before context compression.
|
||||||
|
|
||||||
|
Returns combined text from providers to include in the compression
|
||||||
|
summary prompt. Empty string if no provider contributes.
|
||||||
|
"""
|
||||||
|
parts = []
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
result = provider.on_pre_compress(messages)
|
||||||
|
if result and result.strip():
|
||||||
|
parts.append(result)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(
|
||||||
|
"Memory provider '%s' on_pre_compress failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
return "\n\n".join(parts)
|
||||||
|
|
||||||
|
def on_memory_write(self, action: str, target: str, content: str) -> None:
|
||||||
|
"""Notify external providers when the built-in memory tool writes.
|
||||||
|
|
||||||
|
Skips the builtin provider itself (it's the source of the write).
|
||||||
|
"""
|
||||||
|
for provider in self._providers:
|
||||||
|
if provider.name == "builtin":
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
provider.on_memory_write(action, target, content)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(
|
||||||
|
"Memory provider '%s' on_memory_write failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
|
||||||
|
def on_delegation(self, task: str, result: str, *,
|
||||||
|
child_session_id: str = "", **kwargs) -> None:
|
||||||
|
"""Notify all providers that a subagent completed."""
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
provider.on_delegation(
|
||||||
|
task, result, child_session_id=child_session_id, **kwargs
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(
|
||||||
|
"Memory provider '%s' on_delegation failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
|
||||||
|
def shutdown_all(self) -> None:
|
||||||
|
"""Shut down all providers (reverse order for clean teardown)."""
|
||||||
|
for provider in reversed(self._providers):
|
||||||
|
try:
|
||||||
|
provider.shutdown()
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Memory provider '%s' shutdown failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
|
|
||||||
|
def initialize_all(self, session_id: str, **kwargs) -> None:
|
||||||
|
"""Initialize all providers.
|
||||||
|
|
||||||
|
Automatically injects ``hermes_home`` into *kwargs* so that every
|
||||||
|
provider can resolve profile-scoped storage paths without importing
|
||||||
|
``get_hermes_home()`` themselves.
|
||||||
|
"""
|
||||||
|
if "hermes_home" not in kwargs:
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
kwargs["hermes_home"] = str(get_hermes_home())
|
||||||
|
for provider in self._providers:
|
||||||
|
try:
|
||||||
|
provider.initialize(session_id=session_id, **kwargs)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Memory provider '%s' initialize failed: %s",
|
||||||
|
provider.name, e,
|
||||||
|
)
|
||||||
231
agent/memory_provider.py
Normal file
231
agent/memory_provider.py
Normal file
|
|
@ -0,0 +1,231 @@
|
||||||
|
"""Abstract base class for pluggable memory providers.
|
||||||
|
|
||||||
|
Memory providers give the agent persistent recall across sessions. One
|
||||||
|
external provider is active at a time alongside the always-on built-in
|
||||||
|
memory (MEMORY.md / USER.md). The MemoryManager enforces this limit.
|
||||||
|
|
||||||
|
Built-in memory is always active as the first provider and cannot be removed.
|
||||||
|
External providers (Honcho, Hindsight, Mem0, etc.) are additive — they never
|
||||||
|
disable the built-in store. Only one external provider runs at a time to
|
||||||
|
prevent tool schema bloat and conflicting memory backends.
|
||||||
|
|
||||||
|
Registration:
|
||||||
|
1. Built-in: BuiltinMemoryProvider — always present, not removable.
|
||||||
|
2. Plugins: Ship in plugins/memory/<name>/, activated by memory.provider config.
|
||||||
|
|
||||||
|
Lifecycle (called by MemoryManager, wired in run_agent.py):
|
||||||
|
initialize() — connect, create resources, warm up
|
||||||
|
system_prompt_block() — static text for the system prompt
|
||||||
|
prefetch(query) — background recall before each turn
|
||||||
|
sync_turn(user, asst) — async write after each turn
|
||||||
|
get_tool_schemas() — tool schemas to expose to the model
|
||||||
|
handle_tool_call() — dispatch a tool call
|
||||||
|
shutdown() — clean exit
|
||||||
|
|
||||||
|
Optional hooks (override to opt in):
|
||||||
|
on_turn_start(turn, message, **kwargs) — per-turn tick with runtime context
|
||||||
|
on_session_end(messages) — end-of-session extraction
|
||||||
|
on_pre_compress(messages) -> str — extract before context compression
|
||||||
|
on_memory_write(action, target, content) — mirror built-in memory writes
|
||||||
|
on_delegation(task, result, **kwargs) — parent-side observation of subagent work
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class MemoryProvider(ABC):
|
||||||
|
"""Abstract base class for memory providers."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
@abstractmethod
|
||||||
|
def name(self) -> str:
|
||||||
|
"""Short identifier for this provider (e.g. 'builtin', 'honcho', 'hindsight')."""
|
||||||
|
|
||||||
|
# -- Core lifecycle (implement these) ------------------------------------
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
"""Return True if this provider is configured, has credentials, and is ready.
|
||||||
|
|
||||||
|
Called during agent init to decide whether to activate the provider.
|
||||||
|
Should not make network calls — just check config and installed deps.
|
||||||
|
"""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
"""Initialize for a session.
|
||||||
|
|
||||||
|
Called once at agent startup. May create resources (banks, tables),
|
||||||
|
establish connections, start background threads, etc.
|
||||||
|
|
||||||
|
kwargs always include:
|
||||||
|
- hermes_home (str): The active HERMES_HOME directory path. Use this
|
||||||
|
for profile-scoped storage instead of hardcoding ``~/.hermes``.
|
||||||
|
- platform (str): "cli", "telegram", "discord", "cron", etc.
|
||||||
|
|
||||||
|
kwargs may also include:
|
||||||
|
- agent_context (str): "primary", "subagent", "cron", or "flush".
|
||||||
|
Providers should skip writes for non-primary contexts (cron system
|
||||||
|
prompts would corrupt user representations).
|
||||||
|
- agent_identity (str): Profile name (e.g. "coder"). Use for
|
||||||
|
per-profile provider identity scoping.
|
||||||
|
- agent_workspace (str): Shared workspace name (e.g. "hermes").
|
||||||
|
- parent_session_id (str): For subagents, the parent's session_id.
|
||||||
|
- user_id (str): Platform user identifier (gateway sessions).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
"""Return text to include in the system prompt.
|
||||||
|
|
||||||
|
Called during system prompt assembly. Return empty string to skip.
|
||||||
|
This is for STATIC provider info (instructions, status). Prefetched
|
||||||
|
recall context is injected separately via prefetch().
|
||||||
|
"""
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
"""Recall relevant context for the upcoming turn.
|
||||||
|
|
||||||
|
Called before each API call. Return formatted text to inject as
|
||||||
|
context, or empty string if nothing relevant. Implementations
|
||||||
|
should be fast — use background threads for the actual recall
|
||||||
|
and return cached results here.
|
||||||
|
|
||||||
|
session_id is provided for providers serving concurrent sessions
|
||||||
|
(gateway group chats, cached agents). Providers that don't need
|
||||||
|
per-session scoping can ignore it.
|
||||||
|
"""
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
"""Queue a background recall for the NEXT turn.
|
||||||
|
|
||||||
|
Called after each turn completes. The result will be consumed
|
||||||
|
by prefetch() on the next turn. Default is no-op — providers
|
||||||
|
that do background prefetching should override this.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Persist a completed turn to the backend.
|
||||||
|
|
||||||
|
Called after each turn. Should be non-blocking — queue for
|
||||||
|
background processing if the backend has latency.
|
||||||
|
"""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
"""Return tool schemas this provider exposes.
|
||||||
|
|
||||||
|
Each schema follows the OpenAI function calling format:
|
||||||
|
{"name": "...", "description": "...", "parameters": {...}}
|
||||||
|
|
||||||
|
Return empty list if this provider has no tools (context-only).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: Dict[str, Any], **kwargs) -> str:
|
||||||
|
"""Handle a tool call for one of this provider's tools.
|
||||||
|
|
||||||
|
Must return a JSON string (the tool result).
|
||||||
|
Only called for tool names returned by get_tool_schemas().
|
||||||
|
"""
|
||||||
|
raise NotImplementedError(f"Provider {self.name} does not handle tool {tool_name}")
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
"""Clean shutdown — flush queues, close connections."""
|
||||||
|
|
||||||
|
# -- Optional hooks (override to opt in) ---------------------------------
|
||||||
|
|
||||||
|
def on_turn_start(self, turn_number: int, message: str, **kwargs) -> None:
|
||||||
|
"""Called at the start of each turn with the user message.
|
||||||
|
|
||||||
|
Use for turn-counting, scope management, periodic maintenance.
|
||||||
|
|
||||||
|
kwargs may include: remaining_tokens, model, platform, tool_count.
|
||||||
|
Providers use what they need; extras are ignored.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def on_session_end(self, messages: List[Dict[str, Any]]) -> None:
|
||||||
|
"""Called when a session ends (explicit exit or timeout).
|
||||||
|
|
||||||
|
Use for end-of-session fact extraction, summarization, etc.
|
||||||
|
messages is the full conversation history.
|
||||||
|
|
||||||
|
NOT called after every turn — only at actual session boundaries
|
||||||
|
(CLI exit, /reset, gateway session expiry).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def on_pre_compress(self, messages: List[Dict[str, Any]]) -> str:
|
||||||
|
"""Called before context compression discards old messages.
|
||||||
|
|
||||||
|
Use to extract insights from messages about to be compressed.
|
||||||
|
messages is the list that will be summarized/discarded.
|
||||||
|
|
||||||
|
Return text to include in the compression summary prompt so the
|
||||||
|
compressor preserves provider-extracted insights. Return empty
|
||||||
|
string for no contribution (backwards-compatible default).
|
||||||
|
"""
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def on_delegation(self, task: str, result: str, *,
|
||||||
|
child_session_id: str = "", **kwargs) -> None:
|
||||||
|
"""Called on the PARENT agent when a subagent completes.
|
||||||
|
|
||||||
|
The parent's memory provider gets the task+result pair as an
|
||||||
|
observation of what was delegated and what came back. The subagent
|
||||||
|
itself has no provider session (skip_memory=True).
|
||||||
|
|
||||||
|
task: the delegation prompt
|
||||||
|
result: the subagent's final response
|
||||||
|
child_session_id: the subagent's session_id
|
||||||
|
"""
|
||||||
|
|
||||||
|
def get_config_schema(self) -> List[Dict[str, Any]]:
|
||||||
|
"""Return config fields this provider needs for setup.
|
||||||
|
|
||||||
|
Used by 'hermes memory setup' to walk the user through configuration.
|
||||||
|
Each field is a dict with:
|
||||||
|
key: config key name (e.g. 'api_key', 'mode')
|
||||||
|
description: human-readable description
|
||||||
|
secret: True if this should go to .env (default: False)
|
||||||
|
required: True if required (default: False)
|
||||||
|
default: default value (optional)
|
||||||
|
choices: list of valid values (optional)
|
||||||
|
url: URL where user can get this credential (optional)
|
||||||
|
env_var: explicit env var name for secrets (default: auto-generated)
|
||||||
|
|
||||||
|
Return empty list if no config needed (e.g. local-only providers).
|
||||||
|
"""
|
||||||
|
return []
|
||||||
|
|
||||||
|
def save_config(self, values: Dict[str, Any], hermes_home: str) -> None:
|
||||||
|
"""Write non-secret config to the provider's native location.
|
||||||
|
|
||||||
|
Called by 'hermes memory setup' after collecting user inputs.
|
||||||
|
``values`` contains only non-secret fields (secrets go to .env).
|
||||||
|
``hermes_home`` is the active HERMES_HOME directory path.
|
||||||
|
|
||||||
|
Providers with native config files (JSON, YAML) should override
|
||||||
|
this to write to their expected location. Providers that use only
|
||||||
|
env vars can leave the default (no-op).
|
||||||
|
|
||||||
|
All new memory provider plugins MUST implement either:
|
||||||
|
- save_config() for native config file formats, OR
|
||||||
|
- use only env vars (in which case get_config_schema() fields
|
||||||
|
should all have ``env_var`` set and this method stays no-op).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def on_memory_write(self, action: str, target: str, content: str) -> None:
|
||||||
|
"""Called when the built-in memory tool writes an entry.
|
||||||
|
|
||||||
|
action: 'add', 'replace', or 'remove'
|
||||||
|
target: 'memory' or 'user'
|
||||||
|
content: the entry content
|
||||||
|
|
||||||
|
Use to mirror built-in memory writes to your backend.
|
||||||
|
"""
|
||||||
66
cli.py
66
cli.py
|
|
@ -508,6 +508,8 @@ from tools.browser_tool import _emergency_cleanup_all_sessions as _cleanup_all_b
|
||||||
|
|
||||||
# Guard to prevent cleanup from running multiple times on exit
|
# Guard to prevent cleanup from running multiple times on exit
|
||||||
_cleanup_done = False
|
_cleanup_done = False
|
||||||
|
# Weak reference to the active AIAgent for memory provider shutdown at exit
|
||||||
|
_active_agent_ref = None
|
||||||
|
|
||||||
def _run_cleanup():
|
def _run_cleanup():
|
||||||
"""Run resource cleanup exactly once."""
|
"""Run resource cleanup exactly once."""
|
||||||
|
|
@ -536,6 +538,15 @@ def _run_cleanup():
|
||||||
shutdown_cached_clients()
|
shutdown_cached_clients()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
# Shut down memory provider (on_session_end + shutdown_all) at actual
|
||||||
|
# session boundary — NOT per-turn inside run_conversation().
|
||||||
|
try:
|
||||||
|
if _active_agent_ref and hasattr(_active_agent_ref, 'shutdown_memory_provider'):
|
||||||
|
_active_agent_ref.shutdown_memory_provider(
|
||||||
|
getattr(_active_agent_ref, 'conversation_history', None) or []
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
|
|
@ -2218,7 +2229,7 @@ class HermesCLI:
|
||||||
session_db=self._session_db,
|
session_db=self._session_db,
|
||||||
clarify_callback=self._clarify_callback,
|
clarify_callback=self._clarify_callback,
|
||||||
reasoning_callback=self._current_reasoning_callback(),
|
reasoning_callback=self._current_reasoning_callback(),
|
||||||
honcho_session_key=None, # resolved by run_agent via config sessions map / title
|
|
||||||
fallback_model=self._fallback_model,
|
fallback_model=self._fallback_model,
|
||||||
thinking_callback=self._on_thinking,
|
thinking_callback=self._on_thinking,
|
||||||
checkpoints_enabled=self.checkpoints_enabled,
|
checkpoints_enabled=self.checkpoints_enabled,
|
||||||
|
|
@ -2230,6 +2241,9 @@ class HermesCLI:
|
||||||
stream_delta_callback=self._stream_delta if self.streaming_enabled else None,
|
stream_delta_callback=self._stream_delta if self.streaming_enabled else None,
|
||||||
tool_gen_callback=self._on_tool_gen_start if self.streaming_enabled else None,
|
tool_gen_callback=self._on_tool_gen_start if self.streaming_enabled else None,
|
||||||
)
|
)
|
||||||
|
# Store reference for atexit memory provider shutdown
|
||||||
|
global _active_agent_ref
|
||||||
|
_active_agent_ref = self.agent
|
||||||
# Route agent status output through prompt_toolkit so ANSI escape
|
# Route agent status output through prompt_toolkit so ANSI escape
|
||||||
# sequences aren't garbled by patch_stdout's StdoutProxy (#2262).
|
# sequences aren't garbled by patch_stdout's StdoutProxy (#2262).
|
||||||
self.agent._print_fn = _cprint
|
self.agent._print_fn = _cprint
|
||||||
|
|
@ -3237,6 +3251,9 @@ class HermesCLI:
|
||||||
|
|
||||||
def reset_conversation(self):
|
def reset_conversation(self):
|
||||||
"""Reset the conversation by starting a new session."""
|
"""Reset the conversation by starting a new session."""
|
||||||
|
# Shut down memory provider before resetting — actual session boundary
|
||||||
|
if hasattr(self, 'agent') and self.agent:
|
||||||
|
self.agent.shutdown_memory_provider(self.conversation_history)
|
||||||
self.new_session()
|
self.new_session()
|
||||||
|
|
||||||
def save_conversation(self):
|
def save_conversation(self):
|
||||||
|
|
@ -3901,28 +3918,6 @@ class HermesCLI:
|
||||||
try:
|
try:
|
||||||
if self._session_db.set_session_title(self.session_id, new_title):
|
if self._session_db.set_session_title(self.session_id, new_title):
|
||||||
_cprint(f" Session title set: {new_title}")
|
_cprint(f" Session title set: {new_title}")
|
||||||
# Re-map Honcho session key to new title
|
|
||||||
if self.agent and getattr(self.agent, '_honcho', None):
|
|
||||||
try:
|
|
||||||
hcfg = self.agent._honcho_config
|
|
||||||
new_key = (
|
|
||||||
hcfg.resolve_session_name(
|
|
||||||
session_title=new_title,
|
|
||||||
session_id=self.agent.session_id,
|
|
||||||
)
|
|
||||||
if hcfg else new_title
|
|
||||||
)
|
|
||||||
if new_key and new_key != self.agent._honcho_session_key:
|
|
||||||
old_key = self.agent._honcho_session_key
|
|
||||||
self.agent._honcho.get_or_create(new_key)
|
|
||||||
self.agent._honcho_session_key = new_key
|
|
||||||
from tools.honcho_tools import set_session_context
|
|
||||||
set_session_context(self.agent._honcho, new_key)
|
|
||||||
from agent.display import honcho_session_line, write_tty
|
|
||||||
write_tty(honcho_session_line(hcfg.workspace_id, new_key) + "\n")
|
|
||||||
_cprint(f" Honcho session: {old_key} → {new_key}")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
else:
|
else:
|
||||||
_cprint(" Session not found in database.")
|
_cprint(" Session not found in database.")
|
||||||
except ValueError as e:
|
except ValueError as e:
|
||||||
|
|
@ -4387,7 +4382,6 @@ class HermesCLI:
|
||||||
user_message=btw_prompt,
|
user_message=btw_prompt,
|
||||||
conversation_history=history_snapshot,
|
conversation_history=history_snapshot,
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
sync_honcho=False,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
response = (result.get("final_response") or "") if result else ""
|
response = (result.get("final_response") or "") if result else ""
|
||||||
|
|
@ -4817,12 +4811,7 @@ class HermesCLI:
|
||||||
f" ✅ Compressed: {original_count} → {new_count} messages "
|
f" ✅ Compressed: {original_count} → {new_count} messages "
|
||||||
f"(~{approx_tokens:,} → ~{new_tokens:,} tokens)"
|
f"(~{approx_tokens:,} → ~{new_tokens:,} tokens)"
|
||||||
)
|
)
|
||||||
# Flush Honcho async queue so queued messages land before context resets
|
|
||||||
if self.agent and getattr(self.agent, '_honcho', None):
|
|
||||||
try:
|
|
||||||
self.agent._honcho.flush_all()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f" ❌ Compression failed: {e}")
|
print(f" ❌ Compression failed: {e}")
|
||||||
|
|
||||||
|
|
@ -6483,17 +6472,6 @@ class HermesCLI:
|
||||||
# One-line Honcho session indicator (TTY-only, not captured by agent).
|
# One-line Honcho session indicator (TTY-only, not captured by agent).
|
||||||
# Only show when the user explicitly configured Honcho for Hermes
|
# Only show when the user explicitly configured Honcho for Hermes
|
||||||
# (not auto-enabled from a stray HONCHO_API_KEY env var).
|
# (not auto-enabled from a stray HONCHO_API_KEY env var).
|
||||||
try:
|
|
||||||
from honcho_integration.client import HonchoClientConfig
|
|
||||||
from agent.display import honcho_session_line, write_tty
|
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
|
||||||
if hcfg.enabled and (hcfg.api_key or hcfg.base_url) and hcfg.explicitly_configured:
|
|
||||||
sname = hcfg.resolve_session_name(session_id=self.session_id)
|
|
||||||
if sname:
|
|
||||||
write_tty(honcho_session_line(hcfg.workspace_id, sname) + "\n")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
# If resuming a session, load history and display it immediately
|
# If resuming a session, load history and display it immediately
|
||||||
# so the user has context before typing their first message.
|
# so the user has context before typing their first message.
|
||||||
if self._resumed:
|
if self._resumed:
|
||||||
|
|
@ -7812,12 +7790,6 @@ class HermesCLI:
|
||||||
set_sudo_password_callback(None)
|
set_sudo_password_callback(None)
|
||||||
set_approval_callback(None)
|
set_approval_callback(None)
|
||||||
set_secret_capture_callback(None)
|
set_secret_capture_callback(None)
|
||||||
# Flush + shut down Honcho async writer (drains queue before exit)
|
|
||||||
if self.agent and getattr(self.agent, '_honcho', None):
|
|
||||||
try:
|
|
||||||
self.agent._honcho.shutdown()
|
|
||||||
except (Exception, KeyboardInterrupt):
|
|
||||||
pass
|
|
||||||
# Close session in SQLite
|
# Close session in SQLite
|
||||||
if hasattr(self, '_session_db') and self._session_db and self.agent:
|
if hasattr(self, '_session_db') and self._session_db and self.agent:
|
||||||
try:
|
try:
|
||||||
|
|
|
||||||
|
|
@ -437,6 +437,7 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||||
provider_sort=pr.get("sort"),
|
provider_sort=pr.get("sort"),
|
||||||
disabled_toolsets=["cronjob", "messaging", "clarify"],
|
disabled_toolsets=["cronjob", "messaging", "clarify"],
|
||||||
quiet_mode=True,
|
quiet_mode=True,
|
||||||
|
skip_memory=True, # Cron system prompts would corrupt user representations
|
||||||
platform="cron",
|
platform="cron",
|
||||||
session_id=_cron_session_id,
|
session_id=_cron_session_id,
|
||||||
session_db=_session_db,
|
session_db=_session_db,
|
||||||
|
|
|
||||||
|
|
@ -474,8 +474,6 @@ class GatewayRunner:
|
||||||
# Persistent Honcho managers keyed by gateway session key.
|
# Persistent Honcho managers keyed by gateway session key.
|
||||||
# This preserves write_frequency="session" semantics across short-lived
|
# This preserves write_frequency="session" semantics across short-lived
|
||||||
# per-message AIAgent instances.
|
# per-message AIAgent instances.
|
||||||
self._honcho_managers: Dict[str, Any] = {}
|
|
||||||
self._honcho_configs: Dict[str, Any] = {}
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -508,60 +506,8 @@ class GatewayRunner:
|
||||||
# Track background tasks to prevent garbage collection mid-execution
|
# Track background tasks to prevent garbage collection mid-execution
|
||||||
self._background_tasks: set = set()
|
self._background_tasks: set = set()
|
||||||
|
|
||||||
def _get_or_create_gateway_honcho(self, session_key: str):
|
|
||||||
"""Return a persistent Honcho manager/config pair for this gateway session."""
|
|
||||||
if not hasattr(self, "_honcho_managers"):
|
|
||||||
self._honcho_managers = {}
|
|
||||||
if not hasattr(self, "_honcho_configs"):
|
|
||||||
self._honcho_configs = {}
|
|
||||||
|
|
||||||
if session_key in self._honcho_managers:
|
|
||||||
return self._honcho_managers[session_key], self._honcho_configs.get(session_key)
|
|
||||||
|
|
||||||
try:
|
|
||||||
from honcho_integration.client import HonchoClientConfig, get_honcho_client
|
|
||||||
from honcho_integration.session import HonchoSessionManager
|
|
||||||
|
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
|
||||||
if not hcfg.enabled or not (hcfg.api_key or hcfg.base_url):
|
|
||||||
return None, hcfg
|
|
||||||
|
|
||||||
client = get_honcho_client(hcfg)
|
|
||||||
manager = HonchoSessionManager(
|
|
||||||
honcho=client,
|
|
||||||
config=hcfg,
|
|
||||||
context_tokens=hcfg.context_tokens,
|
|
||||||
)
|
|
||||||
self._honcho_managers[session_key] = manager
|
|
||||||
self._honcho_configs[session_key] = hcfg
|
|
||||||
return manager, hcfg
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug("Gateway Honcho init failed for %s: %s", session_key, e)
|
|
||||||
return None, None
|
|
||||||
|
|
||||||
def _shutdown_gateway_honcho(self, session_key: str) -> None:
|
|
||||||
"""Flush and close the persistent Honcho manager for a gateway session."""
|
|
||||||
managers = getattr(self, "_honcho_managers", None)
|
|
||||||
configs = getattr(self, "_honcho_configs", None)
|
|
||||||
if managers is None or configs is None:
|
|
||||||
return
|
|
||||||
|
|
||||||
manager = managers.pop(session_key, None)
|
|
||||||
configs.pop(session_key, None)
|
|
||||||
if not manager:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
manager.shutdown()
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug("Gateway Honcho shutdown failed for %s: %s", session_key, e)
|
|
||||||
|
|
||||||
def _shutdown_all_gateway_honcho(self) -> None:
|
|
||||||
"""Flush and close all persistent Honcho managers."""
|
|
||||||
managers = getattr(self, "_honcho_managers", None)
|
|
||||||
if not managers:
|
|
||||||
return
|
|
||||||
for session_key in list(managers.keys()):
|
|
||||||
self._shutdown_gateway_honcho(session_key)
|
|
||||||
|
|
||||||
# -- Setup skill availability ----------------------------------------
|
# -- Setup skill availability ----------------------------------------
|
||||||
|
|
||||||
|
|
@ -627,7 +573,6 @@ class GatewayRunner:
|
||||||
def _flush_memories_for_session(
|
def _flush_memories_for_session(
|
||||||
self,
|
self,
|
||||||
old_session_id: str,
|
old_session_id: str,
|
||||||
honcho_session_key: Optional[str] = None,
|
|
||||||
):
|
):
|
||||||
"""Prompt the agent to save memories/skills before context is lost.
|
"""Prompt the agent to save memories/skills before context is lost.
|
||||||
|
|
||||||
|
|
@ -660,9 +605,9 @@ class GatewayRunner:
|
||||||
model=model,
|
model=model,
|
||||||
max_iterations=8,
|
max_iterations=8,
|
||||||
quiet_mode=True,
|
quiet_mode=True,
|
||||||
|
skip_memory=True, # Flush agent — no memory provider
|
||||||
enabled_toolsets=["memory", "skills"],
|
enabled_toolsets=["memory", "skills"],
|
||||||
session_id=old_session_id,
|
session_id=old_session_id,
|
||||||
honcho_session_key=honcho_session_key,
|
|
||||||
)
|
)
|
||||||
# Fully silence the flush agent — quiet_mode only suppresses init
|
# Fully silence the flush agent — quiet_mode only suppresses init
|
||||||
# messages; tool call output still leaks to the terminal through
|
# messages; tool call output still leaks to the terminal through
|
||||||
|
|
@ -725,22 +670,14 @@ class GatewayRunner:
|
||||||
tmp_agent.run_conversation(
|
tmp_agent.run_conversation(
|
||||||
user_message=flush_prompt,
|
user_message=flush_prompt,
|
||||||
conversation_history=msgs,
|
conversation_history=msgs,
|
||||||
sync_honcho=False,
|
|
||||||
)
|
)
|
||||||
logger.info("Pre-reset memory flush completed for session %s", old_session_id)
|
logger.info("Pre-reset memory flush completed for session %s", old_session_id)
|
||||||
# Flush any queued Honcho writes before the session is dropped
|
|
||||||
if getattr(tmp_agent, '_honcho', None):
|
|
||||||
try:
|
|
||||||
tmp_agent._honcho.shutdown()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Pre-reset memory flush failed for session %s: %s", old_session_id, e)
|
logger.debug("Pre-reset memory flush failed for session %s: %s", old_session_id, e)
|
||||||
|
|
||||||
async def _async_flush_memories(
|
async def _async_flush_memories(
|
||||||
self,
|
self,
|
||||||
old_session_id: str,
|
old_session_id: str,
|
||||||
honcho_session_key: Optional[str] = None,
|
|
||||||
):
|
):
|
||||||
"""Run the sync memory flush in a thread pool so it won't block the event loop."""
|
"""Run the sync memory flush in a thread pool so it won't block the event loop."""
|
||||||
loop = asyncio.get_event_loop()
|
loop = asyncio.get_event_loop()
|
||||||
|
|
@ -748,7 +685,6 @@ class GatewayRunner:
|
||||||
None,
|
None,
|
||||||
self._flush_memories_for_session,
|
self._flush_memories_for_session,
|
||||||
old_session_id,
|
old_session_id,
|
||||||
honcho_session_key,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
|
|
@ -1291,7 +1227,14 @@ class GatewayRunner:
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
await self._async_flush_memories(entry.session_id, key)
|
await self._async_flush_memories(entry.session_id, key)
|
||||||
self._shutdown_gateway_honcho(key)
|
# Shut down memory provider on the cached agent
|
||||||
|
cached_agent = self._running_agents.get(key)
|
||||||
|
if cached_agent and cached_agent is not _AGENT_PENDING_SENTINEL:
|
||||||
|
try:
|
||||||
|
if hasattr(cached_agent, 'shutdown_memory_provider'):
|
||||||
|
cached_agent.shutdown_memory_provider()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
# Mark as flushed and persist to disk so the flag
|
# Mark as flushed and persist to disk so the flag
|
||||||
# survives gateway restarts.
|
# survives gateway restarts.
|
||||||
with self.session_store._lock:
|
with self.session_store._lock:
|
||||||
|
|
@ -1425,6 +1368,12 @@ class GatewayRunner:
|
||||||
logger.debug("Interrupted running agent for session %s during shutdown", session_key[:20])
|
logger.debug("Interrupted running agent for session %s during shutdown", session_key[:20])
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Failed interrupting agent during shutdown: %s", e)
|
logger.debug("Failed interrupting agent during shutdown: %s", e)
|
||||||
|
# Shut down memory provider at actual session boundary
|
||||||
|
try:
|
||||||
|
if hasattr(agent, 'shutdown_memory_provider'):
|
||||||
|
agent.shutdown_memory_provider()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
for platform, adapter in list(self.adapters.items()):
|
for platform, adapter in list(self.adapters.items()):
|
||||||
try:
|
try:
|
||||||
|
|
@ -1446,7 +1395,6 @@ class GatewayRunner:
|
||||||
self._running_agents.clear()
|
self._running_agents.clear()
|
||||||
self._pending_messages.clear()
|
self._pending_messages.clear()
|
||||||
self._pending_approvals.clear()
|
self._pending_approvals.clear()
|
||||||
self._shutdown_all_gateway_honcho()
|
|
||||||
self._shutdown_event.set()
|
self._shutdown_event.set()
|
||||||
|
|
||||||
from gateway.status import remove_pid_file, write_runtime_status
|
from gateway.status import remove_pid_file, write_runtime_status
|
||||||
|
|
@ -2992,8 +2940,6 @@ class GatewayRunner:
|
||||||
_flush_task.add_done_callback(self._background_tasks.discard)
|
_flush_task.add_done_callback(self._background_tasks.discard)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Gateway memory flush on reset failed: %s", e)
|
logger.debug("Gateway memory flush on reset failed: %s", e)
|
||||||
|
|
||||||
self._shutdown_gateway_honcho(session_key)
|
|
||||||
self._evict_cached_agent(session_key)
|
self._evict_cached_agent(session_key)
|
||||||
|
|
||||||
# Reset the session
|
# Reset the session
|
||||||
|
|
@ -4144,7 +4090,6 @@ class GatewayRunner:
|
||||||
user_message=btw_prompt,
|
user_message=btw_prompt,
|
||||||
conversation_history=history_snapshot,
|
conversation_history=history_snapshot,
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
sync_honcho=False,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
loop = asyncio.get_event_loop()
|
loop = asyncio.get_event_loop()
|
||||||
|
|
@ -4526,8 +4471,6 @@ class GatewayRunner:
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Memory flush on resume failed: %s", e)
|
logger.debug("Memory flush on resume failed: %s", e)
|
||||||
|
|
||||||
self._shutdown_gateway_honcho(session_key)
|
|
||||||
|
|
||||||
# Clear any running agent for this session key
|
# Clear any running agent for this session key
|
||||||
if session_key in self._running_agents:
|
if session_key in self._running_agents:
|
||||||
del self._running_agents[session_key]
|
del self._running_agents[session_key]
|
||||||
|
|
@ -5599,7 +5542,6 @@ class GatewayRunner:
|
||||||
}
|
}
|
||||||
|
|
||||||
pr = self._provider_routing
|
pr = self._provider_routing
|
||||||
honcho_manager, honcho_config = self._get_or_create_gateway_honcho(session_key)
|
|
||||||
reasoning_config = self._load_reasoning_config()
|
reasoning_config = self._load_reasoning_config()
|
||||||
self._reasoning_config = reasoning_config
|
self._reasoning_config = reasoning_config
|
||||||
# Set up streaming consumer if enabled
|
# Set up streaming consumer if enabled
|
||||||
|
|
@ -5672,9 +5614,6 @@ class GatewayRunner:
|
||||||
provider_data_collection=pr.get("data_collection"),
|
provider_data_collection=pr.get("data_collection"),
|
||||||
session_id=session_id,
|
session_id=session_id,
|
||||||
platform=platform_key,
|
platform=platform_key,
|
||||||
honcho_session_key=session_key,
|
|
||||||
honcho_manager=honcho_manager,
|
|
||||||
honcho_config=honcho_config,
|
|
||||||
session_db=self._session_db,
|
session_db=self._session_db,
|
||||||
fallback_model=self._fallback_model,
|
fallback_model=self._fallback_model,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -428,6 +428,11 @@ DEFAULT_CONFIG = {
|
||||||
"user_profile_enabled": True,
|
"user_profile_enabled": True,
|
||||||
"memory_char_limit": 2200, # ~800 tokens at 2.75 chars/token
|
"memory_char_limit": 2200, # ~800 tokens at 2.75 chars/token
|
||||||
"user_char_limit": 1375, # ~500 tokens at 2.75 chars/token
|
"user_char_limit": 1375, # ~500 tokens at 2.75 chars/token
|
||||||
|
# External memory provider plugin (empty = built-in only).
|
||||||
|
# Set to a provider name to activate: "openviking", "mem0",
|
||||||
|
# "hindsight", "holographic", "retaindb", "byterover".
|
||||||
|
# Only ONE external provider is allowed at a time.
|
||||||
|
"provider": "",
|
||||||
},
|
},
|
||||||
|
|
||||||
# Subagent delegation — override the provider:model used by delegate_task
|
# Subagent delegation — override the provider:model used by delegate_task
|
||||||
|
|
|
||||||
|
|
@ -55,7 +55,7 @@ def _has_provider_env_config(content: str) -> bool:
|
||||||
def _honcho_is_configured_for_doctor() -> bool:
|
def _honcho_is_configured_for_doctor() -> bool:
|
||||||
"""Return True when Honcho is configured, even if this process has no active session."""
|
"""Return True when Honcho is configured, even if this process has no active session."""
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import HonchoClientConfig
|
from plugins.memory.honcho.client import HonchoClientConfig
|
||||||
|
|
||||||
cfg = HonchoClientConfig.from_global_config()
|
cfg = HonchoClientConfig.from_global_config()
|
||||||
return bool(cfg.enabled and (cfg.api_key or cfg.base_url))
|
return bool(cfg.enabled and (cfg.api_key or cfg.base_url))
|
||||||
|
|
@ -709,19 +709,19 @@ def run_doctor(args):
|
||||||
print(color("◆ Honcho Memory", Colors.CYAN, Colors.BOLD))
|
print(color("◆ Honcho Memory", Colors.CYAN, Colors.BOLD))
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import HonchoClientConfig, resolve_config_path
|
from plugins.memory.honcho.client import HonchoClientConfig, resolve_config_path
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
hcfg = HonchoClientConfig.from_global_config()
|
||||||
_honcho_cfg_path = resolve_config_path()
|
_honcho_cfg_path = resolve_config_path()
|
||||||
|
|
||||||
if not _honcho_cfg_path.exists():
|
if not _honcho_cfg_path.exists():
|
||||||
check_warn("Honcho config not found", "run: hermes honcho setup")
|
check_warn("Honcho config not found", "run: hermes memory setup")
|
||||||
elif not hcfg.enabled:
|
elif not hcfg.enabled:
|
||||||
check_info(f"Honcho disabled (set enabled: true in {_honcho_cfg_path} to activate)")
|
check_info(f"Honcho disabled (set enabled: true in {_honcho_cfg_path} to activate)")
|
||||||
elif not (hcfg.api_key or hcfg.base_url):
|
elif not (hcfg.api_key or hcfg.base_url):
|
||||||
check_fail("Honcho API key or base URL not set", "run: hermes honcho setup")
|
check_fail("Honcho API key or base URL not set", "run: hermes memory setup")
|
||||||
issues.append("No Honcho API key — run 'hermes honcho setup'")
|
issues.append("No Honcho API key — run 'hermes memory setup'")
|
||||||
else:
|
else:
|
||||||
from honcho_integration.client import get_honcho_client, reset_honcho_client
|
from plugins.memory.honcho.client import get_honcho_client, reset_honcho_client
|
||||||
reset_honcho_client()
|
reset_honcho_client()
|
||||||
try:
|
try:
|
||||||
get_honcho_client(hcfg)
|
get_honcho_client(hcfg)
|
||||||
|
|
|
||||||
|
|
@ -3206,12 +3206,12 @@ def cmd_update(args):
|
||||||
|
|
||||||
# Sync Honcho host blocks to all profiles
|
# Sync Honcho host blocks to all profiles
|
||||||
try:
|
try:
|
||||||
from honcho_integration.cli import sync_honcho_profiles_quiet
|
from plugins.memory.honcho.cli import sync_honcho_profiles_quiet
|
||||||
synced = sync_honcho_profiles_quiet()
|
synced = sync_honcho_profiles_quiet()
|
||||||
if synced:
|
if synced:
|
||||||
print(f"\n-> Honcho: synced {synced} profile(s)")
|
print(f"\n-> Honcho: synced {synced} profile(s)")
|
||||||
except Exception:
|
except Exception:
|
||||||
pass # honcho not installed or not configured
|
pass # honcho plugin not installed or not configured
|
||||||
|
|
||||||
# Check for config migrations
|
# Check for config migrations
|
||||||
print()
|
print()
|
||||||
|
|
@ -3555,13 +3555,14 @@ def cmd_profile(args):
|
||||||
else:
|
else:
|
||||||
print(f"Cloned config, .env, SOUL.md from {source_label}.")
|
print(f"Cloned config, .env, SOUL.md from {source_label}.")
|
||||||
|
|
||||||
# Auto-clone Honcho config for the new profile
|
# Auto-clone Honcho config for the new profile (only with --clone/--clone-all)
|
||||||
|
if clone or clone_all:
|
||||||
try:
|
try:
|
||||||
from honcho_integration.cli import clone_honcho_for_profile
|
from plugins.memory.honcho.cli import clone_honcho_for_profile
|
||||||
if clone_honcho_for_profile(name):
|
if clone_honcho_for_profile(name):
|
||||||
print(f"Honcho config cloned (host: hermes.{name})")
|
print(f"Honcho config cloned (peer: {name})")
|
||||||
except Exception:
|
except Exception:
|
||||||
pass # Honcho not installed or not configured
|
pass # Honcho plugin not installed or not configured
|
||||||
|
|
||||||
# Seed bundled skills (skip if --clone-all already copied them)
|
# Seed bundled skills (skip if --clone-all already copied them)
|
||||||
if not clone_all:
|
if not clone_all:
|
||||||
|
|
@ -4449,20 +4450,17 @@ For more help on a command:
|
||||||
plugins_parser.set_defaults(func=cmd_plugins)
|
plugins_parser.set_defaults(func=cmd_plugins)
|
||||||
|
|
||||||
# =========================================================================
|
# =========================================================================
|
||||||
# honcho command
|
# honcho command — Honcho-specific config (peer, mode, tokens, profiles)
|
||||||
|
# Provider selection happens via 'hermes memory setup'.
|
||||||
# =========================================================================
|
# =========================================================================
|
||||||
honcho_parser = subparsers.add_parser(
|
honcho_parser = subparsers.add_parser(
|
||||||
"honcho",
|
"honcho",
|
||||||
help="Manage Honcho AI memory integration",
|
help="Manage Honcho memory provider config (peer, mode, profiles)",
|
||||||
description=(
|
description=(
|
||||||
"Honcho is a memory layer that persists across sessions.\n\n"
|
"Configure Honcho-specific settings. Honcho is now a memory provider\n"
|
||||||
"Each conversation is stored as a peer interaction in a workspace. "
|
"plugin — initial setup is via 'hermes memory setup'. These commands\n"
|
||||||
"Honcho builds a representation of the user over time — conclusions, "
|
"manage Honcho's own config: peer names, memory mode, token budgets,\n"
|
||||||
"patterns, context — and surfaces the relevant slice at the start of "
|
"per-profile host blocks, and cross-profile observability."
|
||||||
"each turn so Hermes knows who you are without you having to repeat yourself.\n\n"
|
|
||||||
"Modes: hybrid (Honcho + local MEMORY.md), honcho (Honcho only), "
|
|
||||||
"local (MEMORY.md only). Write frequency is configurable so memory "
|
|
||||||
"writes never block the response."
|
|
||||||
),
|
),
|
||||||
formatter_class=__import__("argparse").RawDescriptionHelpFormatter,
|
formatter_class=__import__("argparse").RawDescriptionHelpFormatter,
|
||||||
)
|
)
|
||||||
|
|
@ -4472,7 +4470,7 @@ For more help on a command:
|
||||||
)
|
)
|
||||||
honcho_subparsers = honcho_parser.add_subparsers(dest="honcho_command")
|
honcho_subparsers = honcho_parser.add_subparsers(dest="honcho_command")
|
||||||
|
|
||||||
honcho_subparsers.add_parser("setup", help="Interactive setup wizard for Honcho integration")
|
honcho_subparsers.add_parser("setup", help="Initial Honcho setup (redirects to hermes memory setup)")
|
||||||
honcho_status = honcho_subparsers.add_parser("status", help="Show current Honcho config and connection status")
|
honcho_status = honcho_subparsers.add_parser("status", help="Show current Honcho config and connection status")
|
||||||
honcho_status.add_argument("--all", action="store_true", help="Show config overview across all profiles")
|
honcho_status.add_argument("--all", action="store_true", help="Show config overview across all profiles")
|
||||||
honcho_subparsers.add_parser("peers", help="Show peer identities across all profiles")
|
honcho_subparsers.add_parser("peers", help="Show peer identities across all profiles")
|
||||||
|
|
@ -4540,11 +4538,55 @@ For more help on a command:
|
||||||
honcho_subparsers.add_parser("sync", help="Sync Honcho config to all existing profiles")
|
honcho_subparsers.add_parser("sync", help="Sync Honcho config to all existing profiles")
|
||||||
|
|
||||||
def cmd_honcho(args):
|
def cmd_honcho(args):
|
||||||
from honcho_integration.cli import honcho_command
|
sub = getattr(args, "honcho_command", None)
|
||||||
|
if sub == "setup":
|
||||||
|
# Redirect to the generic memory setup
|
||||||
|
print("\n Honcho is now configured via the memory provider system.")
|
||||||
|
print(" Running 'hermes memory setup'...\n")
|
||||||
|
from hermes_cli.memory_setup import memory_command
|
||||||
|
memory_command(args)
|
||||||
|
return
|
||||||
|
from plugins.memory.honcho.cli import honcho_command
|
||||||
honcho_command(args)
|
honcho_command(args)
|
||||||
|
|
||||||
honcho_parser.set_defaults(func=cmd_honcho)
|
honcho_parser.set_defaults(func=cmd_honcho)
|
||||||
|
|
||||||
|
# =========================================================================
|
||||||
|
# memory command
|
||||||
|
# =========================================================================
|
||||||
|
memory_parser = subparsers.add_parser(
|
||||||
|
"memory",
|
||||||
|
help="Configure external memory provider",
|
||||||
|
description=(
|
||||||
|
"Set up and manage external memory provider plugins.\n\n"
|
||||||
|
"Available providers: honcho, openviking, mem0, hindsight,\n"
|
||||||
|
"holographic, retaindb, byterover.\n\n"
|
||||||
|
"Only one external provider can be active at a time.\n"
|
||||||
|
"Built-in memory (MEMORY.md/USER.md) is always active."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
memory_sub = memory_parser.add_subparsers(dest="memory_command")
|
||||||
|
memory_sub.add_parser("setup", help="Interactive provider selection and configuration")
|
||||||
|
memory_sub.add_parser("status", help="Show current memory provider config")
|
||||||
|
memory_off_p = memory_sub.add_parser("off", help="Disable external provider (built-in only)")
|
||||||
|
|
||||||
|
def cmd_memory(args):
|
||||||
|
sub = getattr(args, "memory_command", None)
|
||||||
|
if sub == "off":
|
||||||
|
from hermes_cli.config import load_config, save_config
|
||||||
|
config = load_config()
|
||||||
|
if not isinstance(config.get("memory"), dict):
|
||||||
|
config["memory"] = {}
|
||||||
|
config["memory"]["provider"] = ""
|
||||||
|
save_config(config)
|
||||||
|
print("\n ✓ Memory provider: built-in only")
|
||||||
|
print(" Saved to config.yaml\n")
|
||||||
|
else:
|
||||||
|
from hermes_cli.memory_setup import memory_command
|
||||||
|
memory_command(args)
|
||||||
|
|
||||||
|
memory_parser.set_defaults(func=cmd_memory)
|
||||||
|
|
||||||
# =========================================================================
|
# =========================================================================
|
||||||
# tools command
|
# tools command
|
||||||
# =========================================================================
|
# =========================================================================
|
||||||
|
|
|
||||||
451
hermes_cli/memory_setup.py
Normal file
451
hermes_cli/memory_setup.py
Normal file
|
|
@ -0,0 +1,451 @@
|
||||||
|
"""hermes memory setup|status — configure memory provider plugins.
|
||||||
|
|
||||||
|
Auto-detects installed memory providers via the plugin system.
|
||||||
|
Interactive curses-based UI for provider selection, then walks through
|
||||||
|
the provider's config schema. Writes config to config.yaml + .env.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import getpass
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Curses-based interactive picker (same pattern as hermes tools)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _curses_select(title: str, items: list[tuple[str, str]], default: int = 0) -> int:
|
||||||
|
"""Interactive single-select with arrow keys.
|
||||||
|
|
||||||
|
items: list of (label, description) tuples.
|
||||||
|
Returns selected index, or default on escape/quit.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import curses
|
||||||
|
result = [default]
|
||||||
|
|
||||||
|
def _menu(stdscr):
|
||||||
|
curses.curs_set(0)
|
||||||
|
if curses.has_colors():
|
||||||
|
curses.start_color()
|
||||||
|
curses.use_default_colors()
|
||||||
|
curses.init_pair(1, curses.COLOR_GREEN, -1)
|
||||||
|
curses.init_pair(2, curses.COLOR_YELLOW, -1)
|
||||||
|
curses.init_pair(3, curses.COLOR_CYAN, -1)
|
||||||
|
cursor = default
|
||||||
|
|
||||||
|
while True:
|
||||||
|
stdscr.clear()
|
||||||
|
max_y, max_x = stdscr.getmaxyx()
|
||||||
|
|
||||||
|
# Title
|
||||||
|
try:
|
||||||
|
stdscr.addnstr(0, 0, title, max_x - 1,
|
||||||
|
curses.A_BOLD | (curses.color_pair(2) if curses.has_colors() else 0))
|
||||||
|
stdscr.addnstr(1, 0, " ↑↓ navigate ⏎ select q quit", max_x - 1,
|
||||||
|
curses.color_pair(3) if curses.has_colors() else curses.A_DIM)
|
||||||
|
except curses.error:
|
||||||
|
pass
|
||||||
|
|
||||||
|
for i, (label, desc) in enumerate(items):
|
||||||
|
y = i + 3
|
||||||
|
if y >= max_y - 1:
|
||||||
|
break
|
||||||
|
arrow = "→" if i == cursor else " "
|
||||||
|
line = f" {arrow} {label}"
|
||||||
|
if desc:
|
||||||
|
line += f" {desc}"
|
||||||
|
|
||||||
|
attr = curses.A_NORMAL
|
||||||
|
if i == cursor:
|
||||||
|
attr = curses.A_BOLD
|
||||||
|
if curses.has_colors():
|
||||||
|
attr |= curses.color_pair(1)
|
||||||
|
try:
|
||||||
|
stdscr.addnstr(y, 0, line[:max_x - 1], max_x - 1, attr)
|
||||||
|
except curses.error:
|
||||||
|
pass
|
||||||
|
|
||||||
|
stdscr.refresh()
|
||||||
|
key = stdscr.getch()
|
||||||
|
|
||||||
|
if key in (curses.KEY_UP, ord('k')):
|
||||||
|
cursor = (cursor - 1) % len(items)
|
||||||
|
elif key in (curses.KEY_DOWN, ord('j')):
|
||||||
|
cursor = (cursor + 1) % len(items)
|
||||||
|
elif key in (curses.KEY_ENTER, 10, 13):
|
||||||
|
result[0] = cursor
|
||||||
|
return
|
||||||
|
elif key in (27, ord('q')):
|
||||||
|
return
|
||||||
|
|
||||||
|
curses.wrapper(_menu)
|
||||||
|
return result[0]
|
||||||
|
|
||||||
|
except Exception:
|
||||||
|
# Fallback: numbered input
|
||||||
|
print(f"\n {title}\n")
|
||||||
|
for i, (label, desc) in enumerate(items):
|
||||||
|
marker = "→" if i == default else " "
|
||||||
|
d = f" {desc}" if desc else ""
|
||||||
|
print(f" {marker} {i + 1}. {label}{d}")
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
val = input(f"\n Select [1-{len(items)}] ({default + 1}): ")
|
||||||
|
if not val:
|
||||||
|
return default
|
||||||
|
idx = int(val) - 1
|
||||||
|
if 0 <= idx < len(items):
|
||||||
|
return idx
|
||||||
|
except (ValueError, EOFError):
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def _prompt(label: str, default: str | None = None, secret: bool = False) -> str:
|
||||||
|
"""Prompt for a value with optional default and secret masking."""
|
||||||
|
suffix = f" [{default}]" if default else ""
|
||||||
|
if secret:
|
||||||
|
sys.stdout.write(f" {label}{suffix}: ")
|
||||||
|
sys.stdout.flush()
|
||||||
|
if sys.stdin.isatty():
|
||||||
|
val = getpass.getpass(prompt="")
|
||||||
|
else:
|
||||||
|
val = sys.stdin.readline().strip()
|
||||||
|
else:
|
||||||
|
sys.stdout.write(f" {label}{suffix}: ")
|
||||||
|
sys.stdout.flush()
|
||||||
|
val = sys.stdin.readline().strip()
|
||||||
|
return val or (default or "")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Provider discovery
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _install_dependencies(provider_name: str) -> None:
|
||||||
|
"""Install pip dependencies declared in plugin.yaml."""
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path as _Path
|
||||||
|
|
||||||
|
plugin_dir = _Path(__file__).parent.parent / "plugins" / "memory" / provider_name
|
||||||
|
yaml_path = plugin_dir / "plugin.yaml"
|
||||||
|
if not yaml_path.exists():
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
with open(yaml_path) as f:
|
||||||
|
meta = yaml.safe_load(f) or {}
|
||||||
|
except Exception:
|
||||||
|
return
|
||||||
|
|
||||||
|
pip_deps = meta.get("pip_dependencies", [])
|
||||||
|
if not pip_deps:
|
||||||
|
return
|
||||||
|
|
||||||
|
# pip name → import name mapping for packages where they differ
|
||||||
|
_IMPORT_NAMES = {
|
||||||
|
"honcho-ai": "honcho",
|
||||||
|
"mem0ai": "mem0",
|
||||||
|
"hindsight-client": "hindsight_client",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check which packages are missing
|
||||||
|
missing = []
|
||||||
|
for dep in pip_deps:
|
||||||
|
import_name = _IMPORT_NAMES.get(dep, dep.replace("-", "_").split("[")[0])
|
||||||
|
try:
|
||||||
|
__import__(import_name)
|
||||||
|
except ImportError:
|
||||||
|
missing.append(dep)
|
||||||
|
|
||||||
|
if not missing:
|
||||||
|
return
|
||||||
|
|
||||||
|
print(f"\n Installing dependencies: {', '.join(missing)}")
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
[sys.executable, "-m", "pip", "install", "--quiet"] + missing,
|
||||||
|
check=True, timeout=120,
|
||||||
|
capture_output=True,
|
||||||
|
)
|
||||||
|
print(f" ✓ Installed {', '.join(missing)}")
|
||||||
|
except subprocess.CalledProcessError as e:
|
||||||
|
print(f" ⚠ Failed to install {', '.join(missing)}")
|
||||||
|
stderr = (e.stderr or b"").decode()[:200]
|
||||||
|
if stderr:
|
||||||
|
print(f" {stderr}")
|
||||||
|
print(f" Run manually: pip install {' '.join(missing)}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ⚠ Install failed: {e}")
|
||||||
|
print(f" Run manually: pip install {' '.join(missing)}")
|
||||||
|
|
||||||
|
# Also show external dependencies (non-pip) if any
|
||||||
|
ext_deps = meta.get("external_dependencies", [])
|
||||||
|
for dep in ext_deps:
|
||||||
|
dep_name = dep.get("name", "")
|
||||||
|
check_cmd = dep.get("check", "")
|
||||||
|
install_cmd = dep.get("install", "")
|
||||||
|
if check_cmd:
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
check_cmd, shell=True, capture_output=True, timeout=5
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
if install_cmd:
|
||||||
|
print(f"\n ⚠ '{dep_name}' not found. Install with:")
|
||||||
|
print(f" {install_cmd}")
|
||||||
|
|
||||||
|
|
||||||
|
def _get_available_providers() -> list:
|
||||||
|
"""Discover memory providers from plugins/memory/.
|
||||||
|
|
||||||
|
Returns list of (name, description, provider_instance) tuples.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from plugins.memory import discover_memory_providers, load_memory_provider
|
||||||
|
raw = discover_memory_providers()
|
||||||
|
except Exception:
|
||||||
|
raw = []
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for name, desc, available in raw:
|
||||||
|
try:
|
||||||
|
provider = load_memory_provider(name)
|
||||||
|
if not provider:
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
# Override description with setup hint
|
||||||
|
schema = provider.get_config_schema() if hasattr(provider, "get_config_schema") else []
|
||||||
|
has_secrets = any(f.get("secret") for f in schema)
|
||||||
|
if has_secrets:
|
||||||
|
setup_hint = "requires API key"
|
||||||
|
elif not schema:
|
||||||
|
setup_hint = "no setup needed"
|
||||||
|
else:
|
||||||
|
setup_hint = "local"
|
||||||
|
results.append((name, setup_hint, provider))
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Setup wizard
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def cmd_setup(args) -> None:
|
||||||
|
"""Interactive memory provider setup wizard."""
|
||||||
|
from hermes_cli.config import load_config, save_config
|
||||||
|
|
||||||
|
providers = _get_available_providers()
|
||||||
|
|
||||||
|
if not providers:
|
||||||
|
print("\n No memory provider plugins detected.")
|
||||||
|
print(" Install a plugin to ~/.hermes/plugins/ and try again.\n")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Build picker items
|
||||||
|
items = []
|
||||||
|
for name, desc, _ in providers:
|
||||||
|
items.append((name, f"— {desc}"))
|
||||||
|
items.append(("Built-in only", "— MEMORY.md / USER.md (default)"))
|
||||||
|
|
||||||
|
builtin_idx = len(items) - 1
|
||||||
|
selected = _curses_select("Memory provider setup", items, default=builtin_idx)
|
||||||
|
|
||||||
|
config = load_config()
|
||||||
|
if not isinstance(config.get("memory"), dict):
|
||||||
|
config["memory"] = {}
|
||||||
|
|
||||||
|
# Built-in only
|
||||||
|
if selected >= len(providers) or selected < 0:
|
||||||
|
config["memory"]["provider"] = ""
|
||||||
|
save_config(config)
|
||||||
|
print("\n ✓ Memory provider: built-in only")
|
||||||
|
print(" Saved to config.yaml\n")
|
||||||
|
return
|
||||||
|
|
||||||
|
name, _, provider = providers[selected]
|
||||||
|
|
||||||
|
# Install pip dependencies if declared in plugin.yaml
|
||||||
|
_install_dependencies(name)
|
||||||
|
|
||||||
|
schema = provider.get_config_schema() if hasattr(provider, "get_config_schema") else []
|
||||||
|
|
||||||
|
# Provider config section
|
||||||
|
provider_config = config["memory"].get(name, {})
|
||||||
|
if not isinstance(provider_config, dict):
|
||||||
|
provider_config = {}
|
||||||
|
|
||||||
|
env_path = Path(os.environ.get("HERMES_HOME", os.path.expanduser("~/.hermes"))) / ".env"
|
||||||
|
env_writes = {}
|
||||||
|
|
||||||
|
if schema:
|
||||||
|
print(f"\n Configuring {name}:\n")
|
||||||
|
|
||||||
|
for field in schema:
|
||||||
|
key = field["key"]
|
||||||
|
desc = field.get("description", key)
|
||||||
|
default = field.get("default")
|
||||||
|
is_secret = field.get("secret", False)
|
||||||
|
choices = field.get("choices")
|
||||||
|
env_var = field.get("env_var")
|
||||||
|
url = field.get("url")
|
||||||
|
|
||||||
|
if choices and not is_secret:
|
||||||
|
# Use curses picker for choice fields
|
||||||
|
choice_items = [(c, "") for c in choices]
|
||||||
|
current = provider_config.get(key, default)
|
||||||
|
current_idx = 0
|
||||||
|
if current and current in choices:
|
||||||
|
current_idx = choices.index(current)
|
||||||
|
sel = _curses_select(f" {desc}", choice_items, default=current_idx)
|
||||||
|
provider_config[key] = choices[sel]
|
||||||
|
elif is_secret:
|
||||||
|
# Prompt for secret
|
||||||
|
existing = os.environ.get(env_var, "") if env_var else ""
|
||||||
|
if existing:
|
||||||
|
masked = f"...{existing[-4:]}" if len(existing) > 4 else "set"
|
||||||
|
val = _prompt(f"{desc} (current: {masked}, blank to keep)", secret=True)
|
||||||
|
else:
|
||||||
|
hint = f" Get yours at {url}" if url else ""
|
||||||
|
if hint:
|
||||||
|
print(hint)
|
||||||
|
val = _prompt(desc, secret=True)
|
||||||
|
if val and env_var:
|
||||||
|
env_writes[env_var] = val
|
||||||
|
else:
|
||||||
|
# Regular text prompt
|
||||||
|
current = provider_config.get(key)
|
||||||
|
effective_default = current or default
|
||||||
|
val = _prompt(desc, default=str(effective_default) if effective_default else None)
|
||||||
|
if val:
|
||||||
|
provider_config[key] = val
|
||||||
|
|
||||||
|
# Write activation key to config.yaml
|
||||||
|
config["memory"]["provider"] = name
|
||||||
|
save_config(config)
|
||||||
|
|
||||||
|
# Write non-secret config to provider's native location
|
||||||
|
hermes_home = str(Path(os.environ.get("HERMES_HOME", os.path.expanduser("~/.hermes"))))
|
||||||
|
if provider_config and hasattr(provider, "save_config"):
|
||||||
|
try:
|
||||||
|
provider.save_config(provider_config, hermes_home)
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ⚠ Failed to write provider config: {e}")
|
||||||
|
|
||||||
|
# Write secrets to .env
|
||||||
|
if env_writes:
|
||||||
|
_write_env_vars(env_path, env_writes)
|
||||||
|
|
||||||
|
print(f"\n ✓ Memory provider: {name}")
|
||||||
|
print(f" ✓ Activation saved to config.yaml")
|
||||||
|
if provider_config:
|
||||||
|
print(f" ✓ Provider config saved")
|
||||||
|
if env_writes:
|
||||||
|
print(f" ✓ API keys saved to .env")
|
||||||
|
print(f"\n Start a new session to activate.\n")
|
||||||
|
|
||||||
|
|
||||||
|
def _write_env_vars(env_path: Path, env_writes: dict) -> None:
|
||||||
|
"""Append or update env vars in .env file."""
|
||||||
|
env_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
existing_lines = []
|
||||||
|
if env_path.exists():
|
||||||
|
existing_lines = env_path.read_text().splitlines()
|
||||||
|
|
||||||
|
updated_keys = set()
|
||||||
|
new_lines = []
|
||||||
|
for line in existing_lines:
|
||||||
|
key_match = line.split("=", 1)[0].strip() if "=" in line else ""
|
||||||
|
if key_match in env_writes:
|
||||||
|
new_lines.append(f"{key_match}={env_writes[key_match]}")
|
||||||
|
updated_keys.add(key_match)
|
||||||
|
else:
|
||||||
|
new_lines.append(line)
|
||||||
|
|
||||||
|
for key, val in env_writes.items():
|
||||||
|
if key not in updated_keys:
|
||||||
|
new_lines.append(f"{key}={val}")
|
||||||
|
|
||||||
|
env_path.write_text("\n".join(new_lines) + "\n")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Status
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def cmd_status(args) -> None:
|
||||||
|
"""Show current memory provider config."""
|
||||||
|
from hermes_cli.config import load_config
|
||||||
|
|
||||||
|
config = load_config()
|
||||||
|
mem_config = config.get("memory", {})
|
||||||
|
provider_name = mem_config.get("provider", "")
|
||||||
|
|
||||||
|
print(f"\nMemory status\n" + "─" * 40)
|
||||||
|
print(f" Built-in: always active")
|
||||||
|
print(f" Provider: {provider_name or '(none — built-in only)'}")
|
||||||
|
|
||||||
|
if provider_name:
|
||||||
|
provider_config = mem_config.get(provider_name, {})
|
||||||
|
if provider_config:
|
||||||
|
print(f"\n {provider_name} config:")
|
||||||
|
for key, val in provider_config.items():
|
||||||
|
print(f" {key}: {val}")
|
||||||
|
|
||||||
|
providers = _get_available_providers()
|
||||||
|
found = any(name == provider_name for name, _, _ in providers)
|
||||||
|
if found:
|
||||||
|
print(f"\n Plugin: installed ✓")
|
||||||
|
for pname, _, p in providers:
|
||||||
|
if pname == provider_name:
|
||||||
|
if p.is_available():
|
||||||
|
print(f" Status: available ✓")
|
||||||
|
else:
|
||||||
|
print(f" Status: not available ✗")
|
||||||
|
schema = p.get_config_schema() if hasattr(p, "get_config_schema") else []
|
||||||
|
secrets = [f for f in schema if f.get("secret")]
|
||||||
|
if secrets:
|
||||||
|
print(f" Missing:")
|
||||||
|
for s in secrets:
|
||||||
|
env_var = s.get("env_var", "")
|
||||||
|
url = s.get("url", "")
|
||||||
|
is_set = bool(os.environ.get(env_var))
|
||||||
|
mark = "✓" if is_set else "✗"
|
||||||
|
line = f" {mark} {env_var}"
|
||||||
|
if url and not is_set:
|
||||||
|
line += f" → {url}"
|
||||||
|
print(line)
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
print(f"\n Plugin: NOT installed ✗")
|
||||||
|
print(f" Install the '{provider_name}' memory plugin to ~/.hermes/plugins/")
|
||||||
|
|
||||||
|
providers = _get_available_providers()
|
||||||
|
if providers:
|
||||||
|
print(f"\n Installed plugins:")
|
||||||
|
for pname, desc, _ in providers:
|
||||||
|
active = " ← active" if pname == provider_name else ""
|
||||||
|
print(f" • {pname} ({desc}){active}")
|
||||||
|
|
||||||
|
print()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Router
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def memory_command(args) -> None:
|
||||||
|
"""Route memory subcommands."""
|
||||||
|
sub = getattr(args, "memory_command", None)
|
||||||
|
if sub == "setup":
|
||||||
|
cmd_setup(args)
|
||||||
|
elif sub == "status":
|
||||||
|
cmd_status(args)
|
||||||
|
else:
|
||||||
|
cmd_status(args)
|
||||||
|
|
@ -1,9 +0,0 @@
|
||||||
"""Honcho integration for AI-native memory.
|
|
||||||
|
|
||||||
This package is only active when honcho.enabled=true in config and
|
|
||||||
HONCHO_API_KEY is set. All honcho-ai imports are deferred to avoid
|
|
||||||
ImportError when the package is not installed.
|
|
||||||
|
|
||||||
Named ``honcho_integration`` (not ``honcho``) to avoid shadowing the
|
|
||||||
``honcho`` package installed by the ``honcho-ai`` SDK.
|
|
||||||
"""
|
|
||||||
|
|
@ -156,7 +156,7 @@ def _discover_tools():
|
||||||
"tools.delegate_tool",
|
"tools.delegate_tool",
|
||||||
"tools.process_registry",
|
"tools.process_registry",
|
||||||
"tools.send_message_tool",
|
"tools.send_message_tool",
|
||||||
"tools.honcho_tools",
|
# "tools.honcho_tools", # Removed — Honcho is now a memory provider plugin
|
||||||
"tools.homeassistant_tool",
|
"tools.homeassistant_tool",
|
||||||
]
|
]
|
||||||
import importlib
|
import importlib
|
||||||
|
|
@ -371,8 +371,6 @@ def handle_function_call(
|
||||||
task_id: Optional[str] = None,
|
task_id: Optional[str] = None,
|
||||||
user_task: Optional[str] = None,
|
user_task: Optional[str] = None,
|
||||||
enabled_tools: Optional[List[str]] = None,
|
enabled_tools: Optional[List[str]] = None,
|
||||||
honcho_manager: Optional[Any] = None,
|
|
||||||
honcho_session_key: Optional[str] = None,
|
|
||||||
) -> str:
|
) -> str:
|
||||||
"""
|
"""
|
||||||
Main function call dispatcher that routes calls to the tool registry.
|
Main function call dispatcher that routes calls to the tool registry.
|
||||||
|
|
@ -417,16 +415,12 @@ def handle_function_call(
|
||||||
function_name, function_args,
|
function_name, function_args,
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
enabled_tools=sandbox_enabled,
|
enabled_tools=sandbox_enabled,
|
||||||
honcho_manager=honcho_manager,
|
|
||||||
honcho_session_key=honcho_session_key,
|
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
result = registry.dispatch(
|
result = registry.dispatch(
|
||||||
function_name, function_args,
|
function_name, function_args,
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
user_task=user_task,
|
user_task=user_task,
|
||||||
honcho_manager=honcho_manager,
|
|
||||||
honcho_session_key=honcho_session_key,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
|
|
||||||
1
plugins/__init__.py
Normal file
1
plugins/__init__.py
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
# Hermes plugins package
|
||||||
213
plugins/memory/__init__.py
Normal file
213
plugins/memory/__init__.py
Normal file
|
|
@ -0,0 +1,213 @@
|
||||||
|
"""Memory provider plugin discovery.
|
||||||
|
|
||||||
|
Scans ``plugins/memory/<name>/`` directories for memory provider plugins.
|
||||||
|
Each subdirectory must contain ``__init__.py`` with a class implementing
|
||||||
|
the MemoryProvider ABC.
|
||||||
|
|
||||||
|
Memory providers are separate from the general plugin system — they live
|
||||||
|
in the repo and are always available without user installation. Only ONE
|
||||||
|
can be active at a time, selected via ``memory.provider`` in config.yaml.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
from plugins.memory import discover_memory_providers, load_memory_provider
|
||||||
|
|
||||||
|
available = discover_memory_providers() # [(name, desc, available), ...]
|
||||||
|
provider = load_memory_provider("openviking") # MemoryProvider instance
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib
|
||||||
|
import importlib.util
|
||||||
|
import logging
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import List, Optional, Tuple
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_MEMORY_PLUGINS_DIR = Path(__file__).parent
|
||||||
|
|
||||||
|
|
||||||
|
def discover_memory_providers() -> List[Tuple[str, str, bool]]:
|
||||||
|
"""Scan plugins/memory/ for available providers.
|
||||||
|
|
||||||
|
Returns list of (name, description, is_available) tuples.
|
||||||
|
Does NOT import the providers — just reads plugin.yaml for metadata
|
||||||
|
and does a lightweight availability check.
|
||||||
|
"""
|
||||||
|
results = []
|
||||||
|
if not _MEMORY_PLUGINS_DIR.is_dir():
|
||||||
|
return results
|
||||||
|
|
||||||
|
for child in sorted(_MEMORY_PLUGINS_DIR.iterdir()):
|
||||||
|
if not child.is_dir() or child.name.startswith(("_", ".")):
|
||||||
|
continue
|
||||||
|
init_file = child / "__init__.py"
|
||||||
|
if not init_file.exists():
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Read description from plugin.yaml if available
|
||||||
|
desc = ""
|
||||||
|
yaml_file = child / "plugin.yaml"
|
||||||
|
if yaml_file.exists():
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
with open(yaml_file) as f:
|
||||||
|
meta = yaml.safe_load(f) or {}
|
||||||
|
desc = meta.get("description", "")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Quick availability check — try loading and calling is_available()
|
||||||
|
available = True
|
||||||
|
try:
|
||||||
|
provider = _load_provider_from_dir(child)
|
||||||
|
if provider:
|
||||||
|
available = provider.is_available()
|
||||||
|
else:
|
||||||
|
available = False
|
||||||
|
except Exception:
|
||||||
|
available = False
|
||||||
|
|
||||||
|
results.append((child.name, desc, available))
|
||||||
|
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def load_memory_provider(name: str) -> Optional["MemoryProvider"]:
|
||||||
|
"""Load and return a MemoryProvider instance by name.
|
||||||
|
|
||||||
|
Returns None if the provider is not found or fails to load.
|
||||||
|
"""
|
||||||
|
provider_dir = _MEMORY_PLUGINS_DIR / name
|
||||||
|
if not provider_dir.is_dir():
|
||||||
|
logger.debug("Memory provider '%s' not found in %s", name, _MEMORY_PLUGINS_DIR)
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
provider = _load_provider_from_dir(provider_dir)
|
||||||
|
if provider:
|
||||||
|
return provider
|
||||||
|
logger.warning("Memory provider '%s' loaded but no provider instance found", name)
|
||||||
|
return None
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Failed to load memory provider '%s': %s", name, e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _load_provider_from_dir(provider_dir: Path) -> Optional["MemoryProvider"]:
|
||||||
|
"""Import a provider module and extract the MemoryProvider instance.
|
||||||
|
|
||||||
|
The module must have either:
|
||||||
|
- A register(ctx) function (plugin-style) — we simulate a ctx
|
||||||
|
- A top-level class that extends MemoryProvider — we instantiate it
|
||||||
|
"""
|
||||||
|
name = provider_dir.name
|
||||||
|
module_name = f"plugins.memory.{name}"
|
||||||
|
init_file = provider_dir / "__init__.py"
|
||||||
|
|
||||||
|
if not init_file.exists():
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Check if already loaded
|
||||||
|
if module_name in sys.modules:
|
||||||
|
mod = sys.modules[module_name]
|
||||||
|
else:
|
||||||
|
# Handle relative imports within the plugin
|
||||||
|
# First ensure the parent packages are registered
|
||||||
|
for parent in ("plugins", "plugins.memory"):
|
||||||
|
if parent not in sys.modules:
|
||||||
|
parent_path = Path(__file__).parent
|
||||||
|
if parent == "plugins":
|
||||||
|
parent_path = parent_path.parent
|
||||||
|
parent_init = parent_path / "__init__.py"
|
||||||
|
if parent_init.exists():
|
||||||
|
spec = importlib.util.spec_from_file_location(
|
||||||
|
parent, str(parent_init),
|
||||||
|
submodule_search_locations=[str(parent_path)]
|
||||||
|
)
|
||||||
|
if spec:
|
||||||
|
parent_mod = importlib.util.module_from_spec(spec)
|
||||||
|
sys.modules[parent] = parent_mod
|
||||||
|
try:
|
||||||
|
spec.loader.exec_module(parent_mod)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Now load the provider module
|
||||||
|
spec = importlib.util.spec_from_file_location(
|
||||||
|
module_name, str(init_file),
|
||||||
|
submodule_search_locations=[str(provider_dir)]
|
||||||
|
)
|
||||||
|
if not spec:
|
||||||
|
return None
|
||||||
|
|
||||||
|
mod = importlib.util.module_from_spec(spec)
|
||||||
|
sys.modules[module_name] = mod
|
||||||
|
|
||||||
|
# Register submodules so relative imports work
|
||||||
|
# e.g., "from .store import MemoryStore" in holographic plugin
|
||||||
|
for sub_file in provider_dir.glob("*.py"):
|
||||||
|
if sub_file.name == "__init__.py":
|
||||||
|
continue
|
||||||
|
sub_name = sub_file.stem
|
||||||
|
full_sub_name = f"{module_name}.{sub_name}"
|
||||||
|
if full_sub_name not in sys.modules:
|
||||||
|
sub_spec = importlib.util.spec_from_file_location(
|
||||||
|
full_sub_name, str(sub_file)
|
||||||
|
)
|
||||||
|
if sub_spec:
|
||||||
|
sub_mod = importlib.util.module_from_spec(sub_spec)
|
||||||
|
sys.modules[full_sub_name] = sub_mod
|
||||||
|
try:
|
||||||
|
sub_spec.loader.exec_module(sub_mod)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Failed to load submodule %s: %s", full_sub_name, e)
|
||||||
|
|
||||||
|
try:
|
||||||
|
spec.loader.exec_module(mod)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Failed to exec_module %s: %s", module_name, e)
|
||||||
|
sys.modules.pop(module_name, None)
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Try register(ctx) pattern first (how our plugins are written)
|
||||||
|
if hasattr(mod, "register"):
|
||||||
|
collector = _ProviderCollector()
|
||||||
|
try:
|
||||||
|
mod.register(collector)
|
||||||
|
if collector.provider:
|
||||||
|
return collector.provider
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("register() failed for %s: %s", name, e)
|
||||||
|
|
||||||
|
# Fallback: find a MemoryProvider subclass and instantiate it
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
for attr_name in dir(mod):
|
||||||
|
attr = getattr(mod, attr_name, None)
|
||||||
|
if (isinstance(attr, type) and issubclass(attr, MemoryProvider)
|
||||||
|
and attr is not MemoryProvider):
|
||||||
|
try:
|
||||||
|
return attr()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class _ProviderCollector:
|
||||||
|
"""Fake plugin context that captures register_memory_provider calls."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.provider = None
|
||||||
|
|
||||||
|
def register_memory_provider(self, provider):
|
||||||
|
self.provider = provider
|
||||||
|
|
||||||
|
# No-op for other registration methods
|
||||||
|
def register_tool(self, *args, **kwargs):
|
||||||
|
pass
|
||||||
|
|
||||||
|
def register_hook(self, *args, **kwargs):
|
||||||
|
pass
|
||||||
41
plugins/memory/byterover/README.md
Normal file
41
plugins/memory/byterover/README.md
Normal file
|
|
@ -0,0 +1,41 @@
|
||||||
|
# ByteRover Memory Provider
|
||||||
|
|
||||||
|
Persistent memory via the `brv` CLI — hierarchical knowledge tree with tiered retrieval (fuzzy text → LLM-driven search).
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
Install the ByteRover CLI:
|
||||||
|
```bash
|
||||||
|
curl -fsSL https://byterover.dev/install.sh | sh
|
||||||
|
# or
|
||||||
|
npm install -g byterover-cli
|
||||||
|
```
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "byterover"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
hermes config set memory.provider byterover
|
||||||
|
# Optional cloud sync:
|
||||||
|
echo "BRV_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
| Env Var | Required | Description |
|
||||||
|
|---------|----------|-------------|
|
||||||
|
| `BRV_API_KEY` | No | Cloud sync key (optional, local-first by default) |
|
||||||
|
|
||||||
|
Working directory: `$HERMES_HOME/byterover/` (profile-scoped).
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `brv_query` | Search the knowledge tree |
|
||||||
|
| `brv_curate` | Store facts, decisions, patterns |
|
||||||
|
| `brv_status` | CLI version, tree stats, sync state |
|
||||||
398
plugins/memory/byterover/__init__.py
Normal file
398
plugins/memory/byterover/__init__.py
Normal file
|
|
@ -0,0 +1,398 @@
|
||||||
|
"""ByteRover memory plugin — MemoryProvider interface.
|
||||||
|
|
||||||
|
Persistent memory via the ByteRover CLI (``brv``). Organizes knowledge into
|
||||||
|
a hierarchical context tree with tiered retrieval (fuzzy text → LLM-driven
|
||||||
|
search). Local-first with optional cloud sync.
|
||||||
|
|
||||||
|
Original PR #3499 by hieuntg81, adapted to MemoryProvider ABC.
|
||||||
|
|
||||||
|
Requires: ``brv`` CLI installed (npm install -g byterover-cli or
|
||||||
|
curl -fsSL https://byterover.dev/install.sh | sh).
|
||||||
|
|
||||||
|
Config via environment variables (profile-scoped via each profile's .env):
|
||||||
|
BRV_API_KEY — ByteRover API key (for cloud features, optional for local)
|
||||||
|
|
||||||
|
Working directory: $HERMES_HOME/byterover/ (profile-scoped context tree)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Timeouts
|
||||||
|
_QUERY_TIMEOUT = 30 # brv query — should be fast
|
||||||
|
_CURATE_TIMEOUT = 120 # brv curate — may involve LLM processing
|
||||||
|
|
||||||
|
# Minimum lengths to filter noise
|
||||||
|
_MIN_QUERY_LEN = 10
|
||||||
|
_MIN_OUTPUT_LEN = 20
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# brv binary resolution (cached, thread-safe)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_brv_path_lock = threading.Lock()
|
||||||
|
_cached_brv_path: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_brv_path() -> Optional[str]:
|
||||||
|
"""Find the brv binary on PATH or well-known install locations."""
|
||||||
|
global _cached_brv_path
|
||||||
|
with _brv_path_lock:
|
||||||
|
if _cached_brv_path is not None:
|
||||||
|
return _cached_brv_path if _cached_brv_path != "" else None
|
||||||
|
|
||||||
|
found = shutil.which("brv")
|
||||||
|
if not found:
|
||||||
|
home = Path.home()
|
||||||
|
candidates = [
|
||||||
|
home / ".brv-cli" / "bin" / "brv",
|
||||||
|
Path("/usr/local/bin/brv"),
|
||||||
|
home / ".npm-global" / "bin" / "brv",
|
||||||
|
]
|
||||||
|
for c in candidates:
|
||||||
|
if c.exists():
|
||||||
|
found = str(c)
|
||||||
|
break
|
||||||
|
|
||||||
|
with _brv_path_lock:
|
||||||
|
if _cached_brv_path is not None:
|
||||||
|
return _cached_brv_path if _cached_brv_path != "" else None
|
||||||
|
_cached_brv_path = found or ""
|
||||||
|
return found
|
||||||
|
|
||||||
|
|
||||||
|
def _run_brv(args: List[str], timeout: int = _QUERY_TIMEOUT,
|
||||||
|
cwd: str = None) -> dict:
|
||||||
|
"""Run a brv CLI command. Returns {success, output, error}."""
|
||||||
|
brv_path = _resolve_brv_path()
|
||||||
|
if not brv_path:
|
||||||
|
return {"success": False, "error": "brv CLI not found. Install: npm install -g byterover-cli"}
|
||||||
|
|
||||||
|
cmd = [brv_path] + args
|
||||||
|
effective_cwd = cwd or str(_get_brv_cwd())
|
||||||
|
Path(effective_cwd).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
env = os.environ.copy()
|
||||||
|
brv_bin_dir = str(Path(brv_path).parent)
|
||||||
|
env["PATH"] = brv_bin_dir + os.pathsep + env.get("PATH", "")
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
cmd, capture_output=True, text=True,
|
||||||
|
timeout=timeout, cwd=effective_cwd, env=env,
|
||||||
|
)
|
||||||
|
stdout = result.stdout.strip()
|
||||||
|
stderr = result.stderr.strip()
|
||||||
|
|
||||||
|
if result.returncode == 0:
|
||||||
|
return {"success": True, "output": stdout}
|
||||||
|
return {"success": False, "error": stderr or stdout or f"brv exited {result.returncode}"}
|
||||||
|
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
return {"success": False, "error": f"brv timed out after {timeout}s"}
|
||||||
|
except FileNotFoundError:
|
||||||
|
global _cached_brv_path
|
||||||
|
with _brv_path_lock:
|
||||||
|
_cached_brv_path = None
|
||||||
|
return {"success": False, "error": "brv CLI not found"}
|
||||||
|
except Exception as e:
|
||||||
|
return {"success": False, "error": str(e)}
|
||||||
|
|
||||||
|
|
||||||
|
def _get_brv_cwd() -> Path:
|
||||||
|
"""Profile-scoped working directory for the brv context tree."""
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
return get_hermes_home() / "byterover"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tool schemas
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
QUERY_SCHEMA = {
|
||||||
|
"name": "brv_query",
|
||||||
|
"description": (
|
||||||
|
"Search ByteRover's persistent knowledge tree for relevant context. "
|
||||||
|
"Returns memories, project knowledge, architectural decisions, and "
|
||||||
|
"patterns from previous sessions. Use for any question where past "
|
||||||
|
"context would help."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "What to search for."},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
CURATE_SCHEMA = {
|
||||||
|
"name": "brv_curate",
|
||||||
|
"description": (
|
||||||
|
"Store important information in ByteRover's persistent knowledge tree. "
|
||||||
|
"Use for architectural decisions, bug fixes, user preferences, project "
|
||||||
|
"patterns — anything worth remembering across sessions. ByteRover's LLM "
|
||||||
|
"automatically categorizes and organizes the memory."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"content": {"type": "string", "description": "The information to remember."},
|
||||||
|
},
|
||||||
|
"required": ["content"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
STATUS_SCHEMA = {
|
||||||
|
"name": "brv_status",
|
||||||
|
"description": "Check ByteRover status — CLI version, context tree stats, cloud sync state.",
|
||||||
|
"parameters": {"type": "object", "properties": {}, "required": []},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class ByteRoverMemoryProvider(MemoryProvider):
|
||||||
|
"""ByteRover persistent memory via the brv CLI."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._cwd = ""
|
||||||
|
self._session_id = ""
|
||||||
|
self._turn_count = 0
|
||||||
|
self._prefetch_result = ""
|
||||||
|
self._prefetch_lock = threading.Lock()
|
||||||
|
self._prefetch_thread: Optional[threading.Thread] = None
|
||||||
|
self._sync_thread: Optional[threading.Thread] = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "byterover"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
"""Check if brv CLI is installed. No network calls."""
|
||||||
|
return _resolve_brv_path() is not None
|
||||||
|
|
||||||
|
def get_config_schema(self):
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"key": "api_key",
|
||||||
|
"description": "ByteRover API key (optional, for cloud sync)",
|
||||||
|
"secret": True,
|
||||||
|
"env_var": "BRV_API_KEY",
|
||||||
|
"url": "https://app.byterover.dev",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
self._cwd = str(_get_brv_cwd())
|
||||||
|
self._session_id = session_id
|
||||||
|
self._turn_count = 0
|
||||||
|
Path(self._cwd).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
if not _resolve_brv_path():
|
||||||
|
return ""
|
||||||
|
return (
|
||||||
|
"# ByteRover Memory\n"
|
||||||
|
"Active. Persistent knowledge tree with hierarchical context.\n"
|
||||||
|
"Use brv_query to search past knowledge, brv_curate to store "
|
||||||
|
"important facts, brv_status to check state."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||||
|
self._prefetch_thread.join(timeout=3.0)
|
||||||
|
with self._prefetch_lock:
|
||||||
|
result = self._prefetch_result
|
||||||
|
self._prefetch_result = ""
|
||||||
|
if not result:
|
||||||
|
return ""
|
||||||
|
return f"## ByteRover Context\n{result}"
|
||||||
|
|
||||||
|
def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
if not query or len(query.strip()) < _MIN_QUERY_LEN:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _run():
|
||||||
|
try:
|
||||||
|
result = _run_brv(
|
||||||
|
["query", "--", query.strip()[:5000]],
|
||||||
|
timeout=_QUERY_TIMEOUT, cwd=self._cwd,
|
||||||
|
)
|
||||||
|
if result["success"] and result.get("output"):
|
||||||
|
output = result["output"].strip()
|
||||||
|
if len(output) > _MIN_OUTPUT_LEN:
|
||||||
|
with self._prefetch_lock:
|
||||||
|
self._prefetch_result = output
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("ByteRover prefetch failed: %s", e)
|
||||||
|
|
||||||
|
self._prefetch_thread = threading.Thread(
|
||||||
|
target=_run, daemon=True, name="brv-prefetch"
|
||||||
|
)
|
||||||
|
self._prefetch_thread.start()
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Curate the conversation turn in background (non-blocking)."""
|
||||||
|
self._turn_count += 1
|
||||||
|
|
||||||
|
# Only curate substantive turns
|
||||||
|
if len(user_content.strip()) < _MIN_QUERY_LEN:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _sync():
|
||||||
|
try:
|
||||||
|
combined = f"User: {user_content[:2000]}\nAssistant: {assistant_content[:2000]}"
|
||||||
|
_run_brv(
|
||||||
|
["curate", "--", combined],
|
||||||
|
timeout=_CURATE_TIMEOUT, cwd=self._cwd,
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("ByteRover sync failed: %s", e)
|
||||||
|
|
||||||
|
# Wait for previous sync
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=5.0)
|
||||||
|
|
||||||
|
self._sync_thread = threading.Thread(
|
||||||
|
target=_sync, daemon=True, name="brv-sync"
|
||||||
|
)
|
||||||
|
self._sync_thread.start()
|
||||||
|
|
||||||
|
def on_memory_write(self, action: str, target: str, content: str) -> None:
|
||||||
|
"""Mirror built-in memory writes to ByteRover."""
|
||||||
|
if action not in ("add", "replace") or not content:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _write():
|
||||||
|
try:
|
||||||
|
label = "User profile" if target == "user" else "Agent memory"
|
||||||
|
_run_brv(
|
||||||
|
["curate", "--", f"[{label}] {content}"],
|
||||||
|
timeout=_CURATE_TIMEOUT, cwd=self._cwd,
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("ByteRover memory mirror failed: %s", e)
|
||||||
|
|
||||||
|
t = threading.Thread(target=_write, daemon=True, name="brv-memwrite")
|
||||||
|
t.start()
|
||||||
|
|
||||||
|
def on_pre_compress(self, messages: List[Dict[str, Any]]) -> str:
|
||||||
|
"""Extract insights before context compression discards turns."""
|
||||||
|
if not messages:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
# Build a summary of messages about to be compressed
|
||||||
|
parts = []
|
||||||
|
for msg in messages[-10:]: # last 10 messages
|
||||||
|
role = msg.get("role", "")
|
||||||
|
content = msg.get("content", "")
|
||||||
|
if isinstance(content, str) and content.strip() and role in ("user", "assistant"):
|
||||||
|
parts.append(f"{role}: {content[:500]}")
|
||||||
|
|
||||||
|
if not parts:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
combined = "\n".join(parts)
|
||||||
|
|
||||||
|
def _flush():
|
||||||
|
try:
|
||||||
|
_run_brv(
|
||||||
|
["curate", "--", f"[Pre-compression context]\n{combined}"],
|
||||||
|
timeout=_CURATE_TIMEOUT, cwd=self._cwd,
|
||||||
|
)
|
||||||
|
logger.info("ByteRover pre-compression flush: %d messages", len(parts))
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("ByteRover pre-compression flush failed: %s", e)
|
||||||
|
|
||||||
|
t = threading.Thread(target=_flush, daemon=True, name="brv-flush")
|
||||||
|
t.start()
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
return [QUERY_SCHEMA, CURATE_SCHEMA, STATUS_SCHEMA]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
|
||||||
|
if tool_name == "brv_query":
|
||||||
|
return self._tool_query(args)
|
||||||
|
elif tool_name == "brv_curate":
|
||||||
|
return self._tool_curate(args)
|
||||||
|
elif tool_name == "brv_status":
|
||||||
|
return self._tool_status()
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
for t in (self._sync_thread, self._prefetch_thread):
|
||||||
|
if t and t.is_alive():
|
||||||
|
t.join(timeout=10.0)
|
||||||
|
|
||||||
|
# -- Tool implementations ------------------------------------------------
|
||||||
|
|
||||||
|
def _tool_query(self, args: dict) -> str:
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "query is required"})
|
||||||
|
|
||||||
|
result = _run_brv(
|
||||||
|
["query", "--", query.strip()[:5000]],
|
||||||
|
timeout=_QUERY_TIMEOUT, cwd=self._cwd,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not result["success"]:
|
||||||
|
return json.dumps({"error": result.get("error", "Query failed")})
|
||||||
|
|
||||||
|
output = result.get("output", "").strip()
|
||||||
|
if not output or len(output) < _MIN_OUTPUT_LEN:
|
||||||
|
return json.dumps({"result": "No relevant memories found."})
|
||||||
|
|
||||||
|
# Truncate very long results
|
||||||
|
if len(output) > 8000:
|
||||||
|
output = output[:8000] + "\n\n[... truncated]"
|
||||||
|
|
||||||
|
return json.dumps({"result": output})
|
||||||
|
|
||||||
|
def _tool_curate(self, args: dict) -> str:
|
||||||
|
content = args.get("content", "")
|
||||||
|
if not content:
|
||||||
|
return json.dumps({"error": "content is required"})
|
||||||
|
|
||||||
|
result = _run_brv(
|
||||||
|
["curate", "--", content],
|
||||||
|
timeout=_CURATE_TIMEOUT, cwd=self._cwd,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not result["success"]:
|
||||||
|
return json.dumps({"error": result.get("error", "Curate failed")})
|
||||||
|
|
||||||
|
return json.dumps({"result": "Memory curated successfully."})
|
||||||
|
|
||||||
|
def _tool_status(self) -> str:
|
||||||
|
result = _run_brv(["status"], timeout=15, cwd=self._cwd)
|
||||||
|
if not result["success"]:
|
||||||
|
return json.dumps({"error": result.get("error", "Status check failed")})
|
||||||
|
return json.dumps({"status": result.get("output", "")})
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Plugin entry point
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Register ByteRover as a memory provider plugin."""
|
||||||
|
ctx.register_memory_provider(ByteRoverMemoryProvider())
|
||||||
9
plugins/memory/byterover/plugin.yaml
Normal file
9
plugins/memory/byterover/plugin.yaml
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
name: byterover
|
||||||
|
version: 1.0.0
|
||||||
|
description: "ByteRover — persistent knowledge tree with tiered retrieval via the brv CLI."
|
||||||
|
external_dependencies:
|
||||||
|
- name: brv
|
||||||
|
install: "curl -fsSL https://byterover.dev/install.sh | sh"
|
||||||
|
check: "brv --version"
|
||||||
|
hooks:
|
||||||
|
- on_pre_compress
|
||||||
38
plugins/memory/hindsight/README.md
Normal file
38
plugins/memory/hindsight/README.md
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
# Hindsight Memory Provider
|
||||||
|
|
||||||
|
Long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval. Supports cloud and local modes.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- Cloud: `pip install hindsight-client` + API key from [app.hindsight.vectorize.io](https://app.hindsight.vectorize.io)
|
||||||
|
- Local: `pip install hindsight` + LLM API key for embeddings
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "hindsight"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
hermes config set memory.provider hindsight
|
||||||
|
echo "HINDSIGHT_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
Config file: `$HERMES_HOME/hindsight/config.json` (or `~/.hindsight/config.json` legacy)
|
||||||
|
|
||||||
|
| Key | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `mode` | `cloud` | `cloud` or `local` |
|
||||||
|
| `bank_id` | `hermes` | Memory bank identifier |
|
||||||
|
| `budget` | `mid` | Recall thoroughness: `low`/`mid`/`high` |
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `hindsight_retain` | Store information with auto entity extraction |
|
||||||
|
| `hindsight_recall` | Multi-strategy search (semantic + entity graph) |
|
||||||
|
| `hindsight_reflect` | Cross-memory synthesis (LLM-powered) |
|
||||||
358
plugins/memory/hindsight/__init__.py
Normal file
358
plugins/memory/hindsight/__init__.py
Normal file
|
|
@ -0,0 +1,358 @@
|
||||||
|
"""Hindsight memory plugin — MemoryProvider interface.
|
||||||
|
|
||||||
|
Long-term memory with knowledge graph, entity resolution, and multi-strategy
|
||||||
|
retrieval. Supports cloud (API key) and local (embedded PostgreSQL) modes.
|
||||||
|
|
||||||
|
Original PR #1811 by benfrank241, adapted to MemoryProvider ABC.
|
||||||
|
|
||||||
|
Config via environment variables:
|
||||||
|
HINDSIGHT_API_KEY — API key for Hindsight Cloud
|
||||||
|
HINDSIGHT_BANK_ID — memory bank identifier (default: hermes)
|
||||||
|
HINDSIGHT_BUDGET — recall budget: low/mid/high (default: mid)
|
||||||
|
HINDSIGHT_API_URL — API endpoint
|
||||||
|
HINDSIGHT_MODE — cloud or local (default: cloud)
|
||||||
|
|
||||||
|
Or via $HERMES_HOME/hindsight/config.json (profile-scoped), falling back to
|
||||||
|
~/.hindsight/config.json (legacy, shared) for backward compatibility.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import queue
|
||||||
|
import threading
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_DEFAULT_API_URL = "https://api.hindsight.vectorize.io"
|
||||||
|
_VALID_BUDGETS = {"low", "mid", "high"}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Thread helper (from original PR — avoids aiohttp event loop conflicts)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _run_in_thread(fn, timeout: float = 30.0):
|
||||||
|
result_q: queue.Queue = queue.Queue(maxsize=1)
|
||||||
|
|
||||||
|
def _run():
|
||||||
|
import asyncio
|
||||||
|
asyncio.set_event_loop(None)
|
||||||
|
try:
|
||||||
|
result_q.put(("ok", fn()))
|
||||||
|
except Exception as exc:
|
||||||
|
result_q.put(("err", exc))
|
||||||
|
|
||||||
|
t = threading.Thread(target=_run, daemon=True, name="hindsight-call")
|
||||||
|
t.start()
|
||||||
|
kind, value = result_q.get(timeout=timeout)
|
||||||
|
if kind == "err":
|
||||||
|
raise value
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tool schemas
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
RETAIN_SCHEMA = {
|
||||||
|
"name": "hindsight_retain",
|
||||||
|
"description": (
|
||||||
|
"Store information to long-term memory. Hindsight automatically "
|
||||||
|
"extracts structured facts, resolves entities, and indexes for retrieval."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"content": {"type": "string", "description": "The information to store."},
|
||||||
|
"context": {"type": "string", "description": "Short label (e.g. 'user preference', 'project decision')."},
|
||||||
|
},
|
||||||
|
"required": ["content"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
RECALL_SCHEMA = {
|
||||||
|
"name": "hindsight_recall",
|
||||||
|
"description": (
|
||||||
|
"Search long-term memory. Returns memories ranked by relevance using "
|
||||||
|
"semantic search, keyword matching, entity graph traversal, and reranking."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "What to search for."},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
REFLECT_SCHEMA = {
|
||||||
|
"name": "hindsight_reflect",
|
||||||
|
"description": (
|
||||||
|
"Synthesize a reasoned answer from long-term memories. Unlike recall, "
|
||||||
|
"this reasons across all stored memories to produce a coherent response."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "The question to reflect on."},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Config
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _load_config() -> dict:
|
||||||
|
"""Load config from profile-scoped path, legacy path, or env vars.
|
||||||
|
|
||||||
|
Resolution order:
|
||||||
|
1. $HERMES_HOME/hindsight/config.json (profile-scoped)
|
||||||
|
2. ~/.hindsight/config.json (legacy, shared)
|
||||||
|
3. Environment variables
|
||||||
|
"""
|
||||||
|
from pathlib import Path
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
|
||||||
|
# Profile-scoped path (preferred)
|
||||||
|
profile_path = get_hermes_home() / "hindsight" / "config.json"
|
||||||
|
if profile_path.exists():
|
||||||
|
try:
|
||||||
|
return json.loads(profile_path.read_text(encoding="utf-8"))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Legacy shared path (backward compat)
|
||||||
|
legacy_path = Path.home() / ".hindsight" / "config.json"
|
||||||
|
if legacy_path.exists():
|
||||||
|
try:
|
||||||
|
return json.loads(legacy_path.read_text(encoding="utf-8"))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return {
|
||||||
|
"mode": os.environ.get("HINDSIGHT_MODE", "cloud"),
|
||||||
|
"apiKey": os.environ.get("HINDSIGHT_API_KEY", ""),
|
||||||
|
"banks": {
|
||||||
|
"hermes": {
|
||||||
|
"bankId": os.environ.get("HINDSIGHT_BANK_ID", "hermes"),
|
||||||
|
"budget": os.environ.get("HINDSIGHT_BUDGET", "mid"),
|
||||||
|
"enabled": True,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class HindsightMemoryProvider(MemoryProvider):
|
||||||
|
"""Hindsight long-term memory with knowledge graph and multi-strategy retrieval."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._config = None
|
||||||
|
self._api_key = None
|
||||||
|
self._bank_id = "hermes"
|
||||||
|
self._budget = "mid"
|
||||||
|
self._mode = "cloud"
|
||||||
|
self._prefetch_result = ""
|
||||||
|
self._prefetch_lock = threading.Lock()
|
||||||
|
self._prefetch_thread = None
|
||||||
|
self._sync_thread = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "hindsight"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
try:
|
||||||
|
cfg = _load_config()
|
||||||
|
mode = cfg.get("mode", "cloud")
|
||||||
|
if mode == "local":
|
||||||
|
embed = cfg.get("embed", {})
|
||||||
|
return bool(embed.get("llmApiKey") or os.environ.get("HINDSIGHT_LLM_API_KEY"))
|
||||||
|
api_key = cfg.get("apiKey") or os.environ.get("HINDSIGHT_API_KEY", "")
|
||||||
|
return bool(api_key)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def save_config(self, values, hermes_home):
|
||||||
|
"""Write config to $HERMES_HOME/hindsight/config.json."""
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
config_dir = Path(hermes_home) / "hindsight"
|
||||||
|
config_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
config_path = config_dir / "config.json"
|
||||||
|
existing = {}
|
||||||
|
if config_path.exists():
|
||||||
|
try:
|
||||||
|
existing = json.loads(config_path.read_text())
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
existing.update(values)
|
||||||
|
config_path.write_text(json.dumps(existing, indent=2))
|
||||||
|
|
||||||
|
def get_config_schema(self):
|
||||||
|
return [
|
||||||
|
{"key": "mode", "description": "Cloud API or local embedded mode", "default": "cloud", "choices": ["cloud", "local"]},
|
||||||
|
{"key": "api_key", "description": "Hindsight Cloud API key", "secret": True, "env_var": "HINDSIGHT_API_KEY", "url": "https://app.hindsight.vectorize.io"},
|
||||||
|
{"key": "bank_id", "description": "Memory bank identifier", "default": "hermes"},
|
||||||
|
{"key": "budget", "description": "Recall thoroughness", "default": "mid", "choices": ["low", "mid", "high"]},
|
||||||
|
{"key": "llm_provider", "description": "LLM provider for local mode", "default": "anthropic", "choices": ["anthropic", "openai", "groq", "ollama"]},
|
||||||
|
{"key": "llm_api_key", "description": "LLM API key for local mode", "secret": True, "env_var": "HINDSIGHT_LLM_API_KEY"},
|
||||||
|
{"key": "llm_model", "description": "LLM model for local mode", "default": "claude-haiku-4-5-20251001"},
|
||||||
|
]
|
||||||
|
|
||||||
|
def _make_client(self):
|
||||||
|
"""Create a fresh Hindsight client (thread-safe)."""
|
||||||
|
if self._mode == "local":
|
||||||
|
from hindsight import HindsightEmbedded
|
||||||
|
embed = self._config.get("embed", {})
|
||||||
|
return HindsightEmbedded(
|
||||||
|
profile=embed.get("profile", "hermes"),
|
||||||
|
llm_provider=embed.get("llmProvider", ""),
|
||||||
|
llm_api_key=embed.get("llmApiKey", ""),
|
||||||
|
llm_model=embed.get("llmModel", ""),
|
||||||
|
)
|
||||||
|
from hindsight_client import Hindsight
|
||||||
|
return Hindsight(api_key=self._api_key, timeout=30.0)
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
self._config = _load_config()
|
||||||
|
self._mode = self._config.get("mode", "cloud")
|
||||||
|
self._api_key = self._config.get("apiKey") or os.environ.get("HINDSIGHT_API_KEY", "")
|
||||||
|
|
||||||
|
banks = self._config.get("banks", {}).get("hermes", {})
|
||||||
|
self._bank_id = banks.get("bankId", "hermes")
|
||||||
|
budget = banks.get("budget", "mid")
|
||||||
|
self._budget = budget if budget in _VALID_BUDGETS else "mid"
|
||||||
|
|
||||||
|
# Ensure bank exists
|
||||||
|
try:
|
||||||
|
client = _run_in_thread(self._make_client)
|
||||||
|
_run_in_thread(lambda: client.create_bank(bank_id=self._bank_id, name=self._bank_id))
|
||||||
|
except Exception:
|
||||||
|
pass # Already exists
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
return (
|
||||||
|
f"# Hindsight Memory\n"
|
||||||
|
f"Active. Bank: {self._bank_id}, budget: {self._budget}.\n"
|
||||||
|
f"Use hindsight_recall to search, hindsight_reflect for synthesis, "
|
||||||
|
f"hindsight_retain to store facts."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||||
|
self._prefetch_thread.join(timeout=3.0)
|
||||||
|
with self._prefetch_lock:
|
||||||
|
result = self._prefetch_result
|
||||||
|
self._prefetch_result = ""
|
||||||
|
if not result:
|
||||||
|
return ""
|
||||||
|
return f"## Hindsight Memory\n{result}"
|
||||||
|
|
||||||
|
def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
def _run():
|
||||||
|
try:
|
||||||
|
client = self._make_client()
|
||||||
|
resp = client.recall(bank_id=self._bank_id, query=query, budget=self._budget)
|
||||||
|
if resp.results:
|
||||||
|
text = "\n".join(r.text for r in resp.results if r.text)
|
||||||
|
with self._prefetch_lock:
|
||||||
|
self._prefetch_result = text
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Hindsight prefetch failed: %s", e)
|
||||||
|
|
||||||
|
self._prefetch_thread = threading.Thread(target=_run, daemon=True, name="hindsight-prefetch")
|
||||||
|
self._prefetch_thread.start()
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Retain conversation turn in background (non-blocking)."""
|
||||||
|
combined = f"User: {user_content}\nAssistant: {assistant_content}"
|
||||||
|
|
||||||
|
def _sync():
|
||||||
|
try:
|
||||||
|
_run_in_thread(
|
||||||
|
lambda: self._make_client().retain(
|
||||||
|
bank_id=self._bank_id, content=combined, context="conversation"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Hindsight sync failed: %s", e)
|
||||||
|
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=5.0)
|
||||||
|
self._sync_thread = threading.Thread(target=_sync, daemon=True, name="hindsight-sync")
|
||||||
|
self._sync_thread.start()
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
return [RETAIN_SCHEMA, RECALL_SCHEMA, REFLECT_SCHEMA]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
|
||||||
|
if tool_name == "hindsight_retain":
|
||||||
|
content = args.get("content", "")
|
||||||
|
if not content:
|
||||||
|
return json.dumps({"error": "Missing required parameter: content"})
|
||||||
|
context = args.get("context")
|
||||||
|
try:
|
||||||
|
_run_in_thread(
|
||||||
|
lambda: self._make_client().retain(
|
||||||
|
bank_id=self._bank_id, content=content, context=context
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return json.dumps({"result": "Memory stored successfully."})
|
||||||
|
except Exception as e:
|
||||||
|
return json.dumps({"error": f"Failed to store memory: {e}"})
|
||||||
|
|
||||||
|
elif tool_name == "hindsight_recall":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "Missing required parameter: query"})
|
||||||
|
try:
|
||||||
|
resp = _run_in_thread(
|
||||||
|
lambda: self._make_client().recall(
|
||||||
|
bank_id=self._bank_id, query=query, budget=self._budget
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if not resp.results:
|
||||||
|
return json.dumps({"result": "No relevant memories found."})
|
||||||
|
lines = [f"{i}. {r.text}" for i, r in enumerate(resp.results, 1)]
|
||||||
|
return json.dumps({"result": "\n".join(lines)})
|
||||||
|
except Exception as e:
|
||||||
|
return json.dumps({"error": f"Failed to search memory: {e}"})
|
||||||
|
|
||||||
|
elif tool_name == "hindsight_reflect":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "Missing required parameter: query"})
|
||||||
|
try:
|
||||||
|
resp = _run_in_thread(
|
||||||
|
lambda: self._make_client().reflect(
|
||||||
|
bank_id=self._bank_id, query=query, budget=self._budget
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return json.dumps({"result": resp.text or "No relevant memories found."})
|
||||||
|
except Exception as e:
|
||||||
|
return json.dumps({"error": f"Failed to reflect: {e}"})
|
||||||
|
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
for t in (self._prefetch_thread, self._sync_thread):
|
||||||
|
if t and t.is_alive():
|
||||||
|
t.join(timeout=5.0)
|
||||||
|
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Register Hindsight as a memory provider plugin."""
|
||||||
|
ctx.register_memory_provider(HindsightMemoryProvider())
|
||||||
9
plugins/memory/hindsight/plugin.yaml
Normal file
9
plugins/memory/hindsight/plugin.yaml
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
name: hindsight
|
||||||
|
version: 1.0.0
|
||||||
|
description: "Hindsight — long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval."
|
||||||
|
pip_dependencies:
|
||||||
|
- hindsight-client
|
||||||
|
requires_env:
|
||||||
|
- HINDSIGHT_API_KEY
|
||||||
|
hooks:
|
||||||
|
- on_session_end
|
||||||
36
plugins/memory/holographic/README.md
Normal file
36
plugins/memory/holographic/README.md
Normal file
|
|
@ -0,0 +1,36 @@
|
||||||
|
# Holographic Memory Provider
|
||||||
|
|
||||||
|
Local SQLite fact store with FTS5 search, trust scoring, entity resolution, and HRR-based compositional retrieval.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
None — uses SQLite (always available). NumPy optional for HRR algebra.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "holographic"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
hermes config set memory.provider holographic
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
Config in `config.yaml` under `plugins.hermes-memory-store`:
|
||||||
|
|
||||||
|
| Key | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `db_path` | `$HERMES_HOME/memory_store.db` | SQLite database path |
|
||||||
|
| `auto_extract` | `false` | Auto-extract facts at session end |
|
||||||
|
| `default_trust` | `0.5` | Default trust score for new facts |
|
||||||
|
| `hrr_dim` | `1024` | HRR vector dimensions |
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `fact_store` | 9 actions: add, search, probe, related, reason, contradict, update, remove, list |
|
||||||
|
| `fact_feedback` | Rate facts as helpful/unhelpful (trains trust scores) |
|
||||||
395
plugins/memory/holographic/__init__.py
Normal file
395
plugins/memory/holographic/__init__.py
Normal file
|
|
@ -0,0 +1,395 @@
|
||||||
|
"""hermes-memory-store — holographic memory plugin using MemoryProvider interface.
|
||||||
|
|
||||||
|
Registers as a MemoryProvider plugin, giving the agent structured fact storage
|
||||||
|
with entity resolution, trust scoring, and HRR-based compositional retrieval.
|
||||||
|
|
||||||
|
Original plugin by dusterbloom (PR #2351), adapted to the MemoryProvider ABC.
|
||||||
|
|
||||||
|
Config in $HERMES_HOME/config.yaml (profile-scoped):
|
||||||
|
plugins:
|
||||||
|
hermes-memory-store:
|
||||||
|
db_path: $HERMES_HOME/memory_store.db
|
||||||
|
auto_extract: false
|
||||||
|
default_trust: 0.5
|
||||||
|
min_trust_threshold: 0.3
|
||||||
|
temporal_decay_half_life: 0
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
from .store import MemoryStore
|
||||||
|
from .retrieval import FactRetriever
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tool schemas (unchanged from original PR)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FACT_STORE_SCHEMA = {
|
||||||
|
"name": "fact_store",
|
||||||
|
"description": (
|
||||||
|
"Deep structured memory with algebraic reasoning. "
|
||||||
|
"Use alongside the memory tool — memory for always-on context, "
|
||||||
|
"fact_store for deep recall and compositional queries.\n\n"
|
||||||
|
"ACTIONS (simple → powerful):\n"
|
||||||
|
"• add — Store a fact the user would expect you to remember.\n"
|
||||||
|
"• search — Keyword lookup ('editor config', 'deploy process').\n"
|
||||||
|
"• probe — Entity recall: ALL facts about a person/thing.\n"
|
||||||
|
"• related — What connects to an entity? Structural adjacency.\n"
|
||||||
|
"• reason — Compositional: facts connected to MULTIPLE entities simultaneously.\n"
|
||||||
|
"• contradict — Memory hygiene: find facts making conflicting claims.\n"
|
||||||
|
"• update/remove/list — CRUD operations.\n\n"
|
||||||
|
"IMPORTANT: Before answering questions about the user, ALWAYS probe or reason first."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"action": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["add", "search", "probe", "related", "reason", "contradict", "update", "remove", "list"],
|
||||||
|
},
|
||||||
|
"content": {"type": "string", "description": "Fact content (required for 'add')."},
|
||||||
|
"query": {"type": "string", "description": "Search query (required for 'search')."},
|
||||||
|
"entity": {"type": "string", "description": "Entity name for 'probe'/'related'."},
|
||||||
|
"entities": {"type": "array", "items": {"type": "string"}, "description": "Entity names for 'reason'."},
|
||||||
|
"fact_id": {"type": "integer", "description": "Fact ID for 'update'/'remove'."},
|
||||||
|
"category": {"type": "string", "enum": ["user_pref", "project", "tool", "general"]},
|
||||||
|
"tags": {"type": "string", "description": "Comma-separated tags."},
|
||||||
|
"trust_delta": {"type": "number", "description": "Trust adjustment for 'update'."},
|
||||||
|
"min_trust": {"type": "number", "description": "Minimum trust filter (default: 0.3)."},
|
||||||
|
"limit": {"type": "integer", "description": "Max results (default: 10)."},
|
||||||
|
},
|
||||||
|
"required": ["action"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
FACT_FEEDBACK_SCHEMA = {
|
||||||
|
"name": "fact_feedback",
|
||||||
|
"description": (
|
||||||
|
"Rate a fact after using it. Mark 'helpful' if accurate, 'unhelpful' if outdated. "
|
||||||
|
"This trains the memory — good facts rise, bad facts sink."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"action": {"type": "string", "enum": ["helpful", "unhelpful"]},
|
||||||
|
"fact_id": {"type": "integer", "description": "The fact ID to rate."},
|
||||||
|
},
|
||||||
|
"required": ["action", "fact_id"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Config
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _load_plugin_config() -> dict:
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
config_path = get_hermes_home() / "config.yaml"
|
||||||
|
if not config_path.exists():
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
with open(config_path) as f:
|
||||||
|
all_config = yaml.safe_load(f) or {}
|
||||||
|
return all_config.get("plugins", {}).get("hermes-memory-store", {}) or {}
|
||||||
|
except Exception:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class HolographicMemoryProvider(MemoryProvider):
|
||||||
|
"""Holographic memory with structured facts, entity resolution, and HRR retrieval."""
|
||||||
|
|
||||||
|
def __init__(self, config: dict | None = None):
|
||||||
|
self._config = config or _load_plugin_config()
|
||||||
|
self._store = None
|
||||||
|
self._retriever = None
|
||||||
|
self._min_trust = float(self._config.get("min_trust_threshold", 0.3))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "holographic"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return True # SQLite is always available, numpy is optional
|
||||||
|
|
||||||
|
def save_config(self, values, hermes_home):
|
||||||
|
"""Write config to config.yaml under plugins.hermes-memory-store."""
|
||||||
|
from pathlib import Path
|
||||||
|
config_path = Path(hermes_home) / "config.yaml"
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
existing = {}
|
||||||
|
if config_path.exists():
|
||||||
|
with open(config_path) as f:
|
||||||
|
existing = yaml.safe_load(f) or {}
|
||||||
|
existing.setdefault("plugins", {})
|
||||||
|
existing["plugins"]["hermes-memory-store"] = values
|
||||||
|
with open(config_path, "w") as f:
|
||||||
|
yaml.dump(existing, f, default_flow_style=False)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def get_config_schema(self):
|
||||||
|
from hermes_constants import display_hermes_home
|
||||||
|
_default_db = f"{display_hermes_home()}/memory_store.db"
|
||||||
|
return [
|
||||||
|
{"key": "db_path", "description": "SQLite database path", "default": _default_db},
|
||||||
|
{"key": "auto_extract", "description": "Auto-extract facts at session end", "default": "false", "choices": ["true", "false"]},
|
||||||
|
{"key": "default_trust", "description": "Default trust score for new facts", "default": "0.5"},
|
||||||
|
{"key": "hrr_dim", "description": "HRR vector dimensions", "default": "1024"},
|
||||||
|
]
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
_default_db = str(get_hermes_home() / "memory_store.db")
|
||||||
|
db_path = self._config.get("db_path", _default_db)
|
||||||
|
default_trust = float(self._config.get("default_trust", 0.5))
|
||||||
|
hrr_dim = int(self._config.get("hrr_dim", 1024))
|
||||||
|
hrr_weight = float(self._config.get("hrr_weight", 0.3))
|
||||||
|
temporal_decay = int(self._config.get("temporal_decay_half_life", 0))
|
||||||
|
|
||||||
|
self._store = MemoryStore(db_path=db_path, default_trust=default_trust, hrr_dim=hrr_dim)
|
||||||
|
self._retriever = FactRetriever(
|
||||||
|
store=self._store,
|
||||||
|
temporal_decay_half_life=temporal_decay,
|
||||||
|
hrr_weight=hrr_weight,
|
||||||
|
hrr_dim=hrr_dim,
|
||||||
|
)
|
||||||
|
self._session_id = session_id
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
if not self._store:
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
total = self._store._conn.execute(
|
||||||
|
"SELECT COUNT(*) FROM facts"
|
||||||
|
).fetchone()[0]
|
||||||
|
except Exception:
|
||||||
|
total = 0
|
||||||
|
if total == 0:
|
||||||
|
return ""
|
||||||
|
return (
|
||||||
|
f"# Holographic Memory\n"
|
||||||
|
f"Active. {total} facts stored with entity resolution and trust scoring.\n"
|
||||||
|
f"Use fact_store to search, probe entities, reason across entities, or add facts.\n"
|
||||||
|
f"Use fact_feedback to rate facts after using them (trains trust scores)."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
if not self._retriever or not query:
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
results = self._retriever.search(query, min_trust=self._min_trust, limit=5)
|
||||||
|
if not results:
|
||||||
|
return ""
|
||||||
|
lines = []
|
||||||
|
for r in results:
|
||||||
|
trust = r.get("trust", 0)
|
||||||
|
lines.append(f"- [{trust:.1f}] {r.get('content', '')}")
|
||||||
|
return "## Holographic Memory\n" + "\n".join(lines)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Holographic prefetch failed: %s", e)
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
# Holographic memory stores explicit facts via tools, not auto-sync.
|
||||||
|
# The on_session_end hook handles auto-extraction if configured.
|
||||||
|
pass
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
return [FACT_STORE_SCHEMA, FACT_FEEDBACK_SCHEMA]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: Dict[str, Any], **kwargs) -> str:
|
||||||
|
if tool_name == "fact_store":
|
||||||
|
return self._handle_fact_store(args)
|
||||||
|
elif tool_name == "fact_feedback":
|
||||||
|
return self._handle_fact_feedback(args)
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
|
||||||
|
def on_session_end(self, messages: List[Dict[str, Any]]) -> None:
|
||||||
|
if not self._config.get("auto_extract", False):
|
||||||
|
return
|
||||||
|
if not self._store or not messages:
|
||||||
|
return
|
||||||
|
self._auto_extract_facts(messages)
|
||||||
|
|
||||||
|
def on_memory_write(self, action: str, target: str, content: str) -> None:
|
||||||
|
"""Mirror built-in memory writes as facts."""
|
||||||
|
if action == "add" and self._store and content:
|
||||||
|
try:
|
||||||
|
category = "user_pref" if target == "user" else "general"
|
||||||
|
self._store.add_fact(content, category=category)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Holographic memory_write mirror failed: %s", e)
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
self._store = None
|
||||||
|
self._retriever = None
|
||||||
|
|
||||||
|
# -- Tool handlers -------------------------------------------------------
|
||||||
|
|
||||||
|
def _handle_fact_store(self, args: dict) -> str:
|
||||||
|
try:
|
||||||
|
action = args["action"]
|
||||||
|
store = self._store
|
||||||
|
retriever = self._retriever
|
||||||
|
|
||||||
|
if action == "add":
|
||||||
|
fact_id = store.add_fact(
|
||||||
|
args["content"],
|
||||||
|
category=args.get("category", "general"),
|
||||||
|
tags=args.get("tags", ""),
|
||||||
|
)
|
||||||
|
return json.dumps({"fact_id": fact_id, "status": "added"})
|
||||||
|
|
||||||
|
elif action == "search":
|
||||||
|
results = retriever.search(
|
||||||
|
args["query"],
|
||||||
|
category=args.get("category"),
|
||||||
|
min_trust=float(args.get("min_trust", self._min_trust)),
|
||||||
|
limit=int(args.get("limit", 10)),
|
||||||
|
)
|
||||||
|
return json.dumps({"results": results, "count": len(results)})
|
||||||
|
|
||||||
|
elif action == "probe":
|
||||||
|
results = retriever.probe(
|
||||||
|
args["entity"],
|
||||||
|
category=args.get("category"),
|
||||||
|
limit=int(args.get("limit", 10)),
|
||||||
|
)
|
||||||
|
return json.dumps({"results": results, "count": len(results)})
|
||||||
|
|
||||||
|
elif action == "related":
|
||||||
|
results = retriever.related(
|
||||||
|
args["entity"],
|
||||||
|
category=args.get("category"),
|
||||||
|
limit=int(args.get("limit", 10)),
|
||||||
|
)
|
||||||
|
return json.dumps({"results": results, "count": len(results)})
|
||||||
|
|
||||||
|
elif action == "reason":
|
||||||
|
entities = args.get("entities", [])
|
||||||
|
if not entities:
|
||||||
|
return json.dumps({"error": "reason requires 'entities' list"})
|
||||||
|
results = retriever.reason(
|
||||||
|
entities,
|
||||||
|
category=args.get("category"),
|
||||||
|
limit=int(args.get("limit", 10)),
|
||||||
|
)
|
||||||
|
return json.dumps({"results": results, "count": len(results)})
|
||||||
|
|
||||||
|
elif action == "contradict":
|
||||||
|
results = retriever.contradict(
|
||||||
|
category=args.get("category"),
|
||||||
|
limit=int(args.get("limit", 10)),
|
||||||
|
)
|
||||||
|
return json.dumps({"results": results, "count": len(results)})
|
||||||
|
|
||||||
|
elif action == "update":
|
||||||
|
updated = store.update_fact(
|
||||||
|
int(args["fact_id"]),
|
||||||
|
content=args.get("content"),
|
||||||
|
trust_delta=float(args["trust_delta"]) if "trust_delta" in args else None,
|
||||||
|
tags=args.get("tags"),
|
||||||
|
category=args.get("category"),
|
||||||
|
)
|
||||||
|
return json.dumps({"updated": updated})
|
||||||
|
|
||||||
|
elif action == "remove":
|
||||||
|
removed = store.remove_fact(int(args["fact_id"]))
|
||||||
|
return json.dumps({"removed": removed})
|
||||||
|
|
||||||
|
elif action == "list":
|
||||||
|
facts = store.list_facts(
|
||||||
|
category=args.get("category"),
|
||||||
|
min_trust=float(args.get("min_trust", 0.0)),
|
||||||
|
limit=int(args.get("limit", 10)),
|
||||||
|
)
|
||||||
|
return json.dumps({"facts": facts, "count": len(facts)})
|
||||||
|
|
||||||
|
else:
|
||||||
|
return json.dumps({"error": f"Unknown action: {action}"})
|
||||||
|
|
||||||
|
except KeyError as exc:
|
||||||
|
return json.dumps({"error": f"Missing required argument: {exc}"})
|
||||||
|
except Exception as exc:
|
||||||
|
return json.dumps({"error": str(exc)})
|
||||||
|
|
||||||
|
def _handle_fact_feedback(self, args: dict) -> str:
|
||||||
|
try:
|
||||||
|
fact_id = int(args["fact_id"])
|
||||||
|
helpful = args["action"] == "helpful"
|
||||||
|
result = self._store.record_feedback(fact_id, helpful=helpful)
|
||||||
|
return json.dumps(result)
|
||||||
|
except KeyError as exc:
|
||||||
|
return json.dumps({"error": f"Missing required argument: {exc}"})
|
||||||
|
except Exception as exc:
|
||||||
|
return json.dumps({"error": str(exc)})
|
||||||
|
|
||||||
|
# -- Auto-extraction (on_session_end) ------------------------------------
|
||||||
|
|
||||||
|
def _auto_extract_facts(self, messages: list) -> None:
|
||||||
|
_PREF_PATTERNS = [
|
||||||
|
re.compile(r'\bI\s+(?:prefer|like|love|use|want|need)\s+(.+)', re.IGNORECASE),
|
||||||
|
re.compile(r'\bmy\s+(?:favorite|preferred|default)\s+\w+\s+is\s+(.+)', re.IGNORECASE),
|
||||||
|
re.compile(r'\bI\s+(?:always|never|usually)\s+(.+)', re.IGNORECASE),
|
||||||
|
]
|
||||||
|
_DECISION_PATTERNS = [
|
||||||
|
re.compile(r'\bwe\s+(?:decided|agreed|chose)\s+(?:to\s+)?(.+)', re.IGNORECASE),
|
||||||
|
re.compile(r'\bthe\s+project\s+(?:uses|needs|requires)\s+(.+)', re.IGNORECASE),
|
||||||
|
]
|
||||||
|
|
||||||
|
extracted = 0
|
||||||
|
for msg in messages:
|
||||||
|
if msg.get("role") != "user":
|
||||||
|
continue
|
||||||
|
content = msg.get("content", "")
|
||||||
|
if not isinstance(content, str) or len(content) < 10:
|
||||||
|
continue
|
||||||
|
|
||||||
|
for pattern in _PREF_PATTERNS:
|
||||||
|
if pattern.search(content):
|
||||||
|
try:
|
||||||
|
self._store.add_fact(content[:400], category="user_pref")
|
||||||
|
extracted += 1
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
break
|
||||||
|
|
||||||
|
for pattern in _DECISION_PATTERNS:
|
||||||
|
if pattern.search(content):
|
||||||
|
try:
|
||||||
|
self._store.add_fact(content[:400], category="project")
|
||||||
|
extracted += 1
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
break
|
||||||
|
|
||||||
|
if extracted:
|
||||||
|
logger.info("Auto-extracted %d facts from conversation", extracted)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Plugin entry point
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Register the holographic memory provider with the plugin system."""
|
||||||
|
config = _load_plugin_config()
|
||||||
|
provider = HolographicMemoryProvider(config=config)
|
||||||
|
ctx.register_memory_provider(provider)
|
||||||
203
plugins/memory/holographic/holographic.py
Normal file
203
plugins/memory/holographic/holographic.py
Normal file
|
|
@ -0,0 +1,203 @@
|
||||||
|
"""Holographic Reduced Representations (HRR) with phase encoding.
|
||||||
|
|
||||||
|
HRRs are a vector symbolic architecture for encoding compositional structure
|
||||||
|
into fixed-width distributed representations. This module uses *phase vectors*:
|
||||||
|
each concept is a vector of angles in [0, 2π). The algebraic operations are:
|
||||||
|
|
||||||
|
bind — circular convolution (phase addition) — associates two concepts
|
||||||
|
unbind — circular correlation (phase subtraction) — retrieves a bound value
|
||||||
|
bundle — superposition (circular mean) — merges multiple concepts
|
||||||
|
|
||||||
|
Phase encoding is numerically stable, avoids the magnitude collapse of
|
||||||
|
traditional complex-number HRRs, and maps cleanly to cosine similarity.
|
||||||
|
|
||||||
|
Atoms are generated deterministically from SHA-256 so representations are
|
||||||
|
identical across processes, machines, and language versions.
|
||||||
|
|
||||||
|
References:
|
||||||
|
Plate (1995) — Holographic Reduced Representations
|
||||||
|
Gayler (2004) — Vector Symbolic Architectures answer Jackendoff's challenges
|
||||||
|
"""
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
import struct
|
||||||
|
import math
|
||||||
|
|
||||||
|
try:
|
||||||
|
import numpy as np
|
||||||
|
_HAS_NUMPY = True
|
||||||
|
except ImportError:
|
||||||
|
_HAS_NUMPY = False
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_TWO_PI = 2.0 * math.pi
|
||||||
|
|
||||||
|
|
||||||
|
def _require_numpy() -> None:
|
||||||
|
if not _HAS_NUMPY:
|
||||||
|
raise RuntimeError("numpy is required for holographic operations")
|
||||||
|
|
||||||
|
|
||||||
|
def encode_atom(word: str, dim: int = 1024) -> "np.ndarray":
|
||||||
|
"""Deterministic phase vector via SHA-256 counter blocks.
|
||||||
|
|
||||||
|
Uses hashlib (not numpy RNG) for cross-platform reproducibility.
|
||||||
|
|
||||||
|
Algorithm:
|
||||||
|
- Generate enough SHA-256 blocks by hashing f"{word}:{i}" for i=0,1,2,...
|
||||||
|
- Concatenate digests, interpret as uint16 values via struct.unpack
|
||||||
|
- Scale to [0, 2π): phases = values * (2π / 65536)
|
||||||
|
- Truncate to dim elements
|
||||||
|
- Returns np.float64 array of shape (dim,)
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
|
||||||
|
# Each SHA-256 digest is 32 bytes = 16 uint16 values.
|
||||||
|
values_per_block = 16
|
||||||
|
blocks_needed = math.ceil(dim / values_per_block)
|
||||||
|
|
||||||
|
uint16_values: list[int] = []
|
||||||
|
for i in range(blocks_needed):
|
||||||
|
digest = hashlib.sha256(f"{word}:{i}".encode()).digest()
|
||||||
|
uint16_values.extend(struct.unpack("<16H", digest))
|
||||||
|
|
||||||
|
phases = np.array(uint16_values[:dim], dtype=np.float64) * (_TWO_PI / 65536.0)
|
||||||
|
return phases
|
||||||
|
|
||||||
|
|
||||||
|
def bind(a: "np.ndarray", b: "np.ndarray") -> "np.ndarray":
|
||||||
|
"""Circular convolution = element-wise phase addition.
|
||||||
|
|
||||||
|
Binding associates two concepts into a single composite vector.
|
||||||
|
The result is dissimilar to both inputs (quasi-orthogonal).
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
return (a + b) % _TWO_PI
|
||||||
|
|
||||||
|
|
||||||
|
def unbind(memory: "np.ndarray", key: "np.ndarray") -> "np.ndarray":
|
||||||
|
"""Circular correlation = element-wise phase subtraction.
|
||||||
|
|
||||||
|
Unbinding retrieves the value associated with a key from a memory vector.
|
||||||
|
unbind(bind(a, b), a) ≈ b (up to superposition noise)
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
return (memory - key) % _TWO_PI
|
||||||
|
|
||||||
|
|
||||||
|
def bundle(*vectors: "np.ndarray") -> "np.ndarray":
|
||||||
|
"""Superposition via circular mean of complex exponentials.
|
||||||
|
|
||||||
|
Bundling merges multiple vectors into one that is similar to each input.
|
||||||
|
The result can hold O(sqrt(dim)) items before similarity degrades.
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
complex_sum = np.sum([np.exp(1j * v) for v in vectors], axis=0)
|
||||||
|
return np.angle(complex_sum) % _TWO_PI
|
||||||
|
|
||||||
|
|
||||||
|
def similarity(a: "np.ndarray", b: "np.ndarray") -> float:
|
||||||
|
"""Phase cosine similarity. Range [-1, 1].
|
||||||
|
|
||||||
|
Returns 1.0 for identical vectors, near 0.0 for random (unrelated) vectors,
|
||||||
|
and -1.0 for perfectly anti-correlated vectors.
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
return float(np.mean(np.cos(a - b)))
|
||||||
|
|
||||||
|
|
||||||
|
def encode_text(text: str, dim: int = 1024) -> "np.ndarray":
|
||||||
|
"""Bag-of-words: bundle of atom vectors for each token.
|
||||||
|
|
||||||
|
Tokenizes by lowercasing, splitting on whitespace, and stripping
|
||||||
|
leading/trailing punctuation from each token.
|
||||||
|
|
||||||
|
Returns bundle of all token atom vectors.
|
||||||
|
If text is empty or produces no tokens, returns encode_atom("__hrr_empty__", dim).
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
|
||||||
|
tokens = [
|
||||||
|
token.strip(".,!?;:\"'()[]{}")
|
||||||
|
for token in text.lower().split()
|
||||||
|
]
|
||||||
|
tokens = [t for t in tokens if t]
|
||||||
|
|
||||||
|
if not tokens:
|
||||||
|
return encode_atom("__hrr_empty__", dim)
|
||||||
|
|
||||||
|
atom_vectors = [encode_atom(token, dim) for token in tokens]
|
||||||
|
return bundle(*atom_vectors)
|
||||||
|
|
||||||
|
|
||||||
|
def encode_fact(content: str, entities: list[str], dim: int = 1024) -> "np.ndarray":
|
||||||
|
"""Structured encoding: content bound to ROLE_CONTENT, each entity bound to ROLE_ENTITY, all bundled.
|
||||||
|
|
||||||
|
Role vectors are reserved atoms: "__hrr_role_content__", "__hrr_role_entity__"
|
||||||
|
|
||||||
|
Components:
|
||||||
|
1. bind(encode_text(content, dim), encode_atom("__hrr_role_content__", dim))
|
||||||
|
2. For each entity: bind(encode_atom(entity.lower(), dim), encode_atom("__hrr_role_entity__", dim))
|
||||||
|
3. bundle all components together
|
||||||
|
|
||||||
|
This enables algebraic extraction:
|
||||||
|
unbind(fact, bind(entity, ROLE_ENTITY)) ≈ content_vector
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
|
||||||
|
role_content = encode_atom("__hrr_role_content__", dim)
|
||||||
|
role_entity = encode_atom("__hrr_role_entity__", dim)
|
||||||
|
|
||||||
|
components: list[np.ndarray] = [
|
||||||
|
bind(encode_text(content, dim), role_content)
|
||||||
|
]
|
||||||
|
|
||||||
|
for entity in entities:
|
||||||
|
components.append(bind(encode_atom(entity.lower(), dim), role_entity))
|
||||||
|
|
||||||
|
return bundle(*components)
|
||||||
|
|
||||||
|
|
||||||
|
def phases_to_bytes(phases: "np.ndarray") -> bytes:
|
||||||
|
"""Serialize phase vector to bytes. float64 tobytes — 8 KB at dim=1024."""
|
||||||
|
_require_numpy()
|
||||||
|
return phases.tobytes()
|
||||||
|
|
||||||
|
|
||||||
|
def bytes_to_phases(data: bytes) -> "np.ndarray":
|
||||||
|
"""Deserialize bytes back to phase vector. Inverse of phases_to_bytes.
|
||||||
|
|
||||||
|
The .copy() call is required because frombuffer returns a read-only view
|
||||||
|
backed by the bytes object; callers expect a mutable array.
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
return np.frombuffer(data, dtype=np.float64).copy()
|
||||||
|
|
||||||
|
|
||||||
|
def snr_estimate(dim: int, n_items: int) -> float:
|
||||||
|
"""Signal-to-noise ratio estimate for holographic storage.
|
||||||
|
|
||||||
|
SNR = sqrt(dim / n_items) when n_items > 0, else inf.
|
||||||
|
|
||||||
|
The SNR falls below 2.0 when n_items > dim / 4, meaning retrieval
|
||||||
|
errors become likely. Logs a warning when this threshold is crossed.
|
||||||
|
"""
|
||||||
|
_require_numpy()
|
||||||
|
|
||||||
|
if n_items <= 0:
|
||||||
|
return float("inf")
|
||||||
|
|
||||||
|
snr = math.sqrt(dim / n_items)
|
||||||
|
|
||||||
|
if snr < 2.0:
|
||||||
|
logger.warning(
|
||||||
|
"HRR storage near capacity: SNR=%.2f (dim=%d, n_items=%d). "
|
||||||
|
"Retrieval accuracy may degrade. Consider increasing dim or reducing stored items.",
|
||||||
|
snr,
|
||||||
|
dim,
|
||||||
|
n_items,
|
||||||
|
)
|
||||||
|
|
||||||
|
return snr
|
||||||
5
plugins/memory/holographic/plugin.yaml
Normal file
5
plugins/memory/holographic/plugin.yaml
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
name: holographic
|
||||||
|
version: 0.1.0
|
||||||
|
description: "Holographic memory — local SQLite fact store with FTS5 search, trust scoring, and HRR-based compositional retrieval."
|
||||||
|
hooks:
|
||||||
|
- on_session_end
|
||||||
593
plugins/memory/holographic/retrieval.py
Normal file
593
plugins/memory/holographic/retrieval.py
Normal file
|
|
@ -0,0 +1,593 @@
|
||||||
|
"""Hybrid keyword/BM25 retrieval for the memory store.
|
||||||
|
|
||||||
|
Ported from KIK memory_agent.py — combines FTS5 full-text search with
|
||||||
|
Jaccard similarity reranking and trust-weighted scoring.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import math
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from .store import MemoryStore
|
||||||
|
|
||||||
|
try:
|
||||||
|
from . import holographic as hrr
|
||||||
|
except ImportError:
|
||||||
|
import holographic as hrr # type: ignore[no-redef]
|
||||||
|
|
||||||
|
|
||||||
|
class FactRetriever:
|
||||||
|
"""Multi-strategy fact retrieval with trust-weighted scoring."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
store: MemoryStore,
|
||||||
|
temporal_decay_half_life: int = 0, # days, 0 = disabled
|
||||||
|
fts_weight: float = 0.4,
|
||||||
|
jaccard_weight: float = 0.3,
|
||||||
|
hrr_weight: float = 0.3,
|
||||||
|
hrr_dim: int = 1024,
|
||||||
|
):
|
||||||
|
self.store = store
|
||||||
|
self.half_life = temporal_decay_half_life
|
||||||
|
self.hrr_dim = hrr_dim
|
||||||
|
|
||||||
|
# Auto-redistribute weights if numpy unavailable
|
||||||
|
if hrr_weight > 0 and not hrr._HAS_NUMPY:
|
||||||
|
fts_weight = 0.6
|
||||||
|
jaccard_weight = 0.4
|
||||||
|
hrr_weight = 0.0
|
||||||
|
|
||||||
|
self.fts_weight = fts_weight
|
||||||
|
self.jaccard_weight = jaccard_weight
|
||||||
|
self.hrr_weight = hrr_weight
|
||||||
|
|
||||||
|
def search(
|
||||||
|
self,
|
||||||
|
query: str,
|
||||||
|
category: str | None = None,
|
||||||
|
min_trust: float = 0.3,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Hybrid search: FTS5 candidates → Jaccard rerank → trust weighting.
|
||||||
|
|
||||||
|
Pipeline:
|
||||||
|
1. FTS5 search: Get limit*3 candidates from SQLite full-text search
|
||||||
|
2. Jaccard boost: Token overlap between query and fact content
|
||||||
|
3. Trust weighting: final_score = relevance * trust_score
|
||||||
|
4. Temporal decay (optional): decay = 0.5^(age_days / half_life)
|
||||||
|
|
||||||
|
Returns list of dicts with fact data + 'score' field, sorted by score desc.
|
||||||
|
"""
|
||||||
|
# Stage 1: Get FTS5 candidates (more than limit for reranking headroom)
|
||||||
|
candidates = self._fts_candidates(query, category, min_trust, limit * 3)
|
||||||
|
|
||||||
|
if not candidates:
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Stage 2: Rerank with Jaccard + trust + optional decay
|
||||||
|
query_tokens = self._tokenize(query)
|
||||||
|
scored = []
|
||||||
|
|
||||||
|
for fact in candidates:
|
||||||
|
content_tokens = self._tokenize(fact["content"])
|
||||||
|
tag_tokens = self._tokenize(fact.get("tags", ""))
|
||||||
|
all_tokens = content_tokens | tag_tokens
|
||||||
|
|
||||||
|
jaccard = self._jaccard_similarity(query_tokens, all_tokens)
|
||||||
|
fts_score = fact.get("fts_rank", 0.0)
|
||||||
|
|
||||||
|
# HRR similarity
|
||||||
|
if self.hrr_weight > 0 and fact.get("hrr_vector"):
|
||||||
|
fact_vec = hrr.bytes_to_phases(fact["hrr_vector"])
|
||||||
|
query_vec = hrr.encode_text(query, self.hrr_dim)
|
||||||
|
hrr_sim = (hrr.similarity(query_vec, fact_vec) + 1.0) / 2.0 # shift to [0,1]
|
||||||
|
else:
|
||||||
|
hrr_sim = 0.5 # neutral
|
||||||
|
|
||||||
|
# Combine FTS5 + Jaccard + HRR
|
||||||
|
relevance = (self.fts_weight * fts_score
|
||||||
|
+ self.jaccard_weight * jaccard
|
||||||
|
+ self.hrr_weight * hrr_sim)
|
||||||
|
|
||||||
|
# Trust weighting
|
||||||
|
score = relevance * fact["trust_score"]
|
||||||
|
|
||||||
|
# Optional temporal decay
|
||||||
|
if self.half_life > 0:
|
||||||
|
score *= self._temporal_decay(fact.get("updated_at") or fact.get("created_at"))
|
||||||
|
|
||||||
|
fact["score"] = score
|
||||||
|
scored.append(fact)
|
||||||
|
|
||||||
|
# Sort by score descending, return top limit
|
||||||
|
scored.sort(key=lambda x: x["score"], reverse=True)
|
||||||
|
results = scored[:limit]
|
||||||
|
# Strip raw HRR bytes — callers expect JSON-serializable dicts
|
||||||
|
for fact in results:
|
||||||
|
fact.pop("hrr_vector", None)
|
||||||
|
return results
|
||||||
|
|
||||||
|
def probe(
|
||||||
|
self,
|
||||||
|
entity: str,
|
||||||
|
category: str | None = None,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Compositional entity query using HRR algebra.
|
||||||
|
|
||||||
|
Unbinds entity from memory bank to extract associated content.
|
||||||
|
This is NOT keyword search — it uses algebraic structure to find facts
|
||||||
|
where the entity plays a structural role.
|
||||||
|
|
||||||
|
Falls back to FTS5 search if numpy unavailable.
|
||||||
|
"""
|
||||||
|
if not hrr._HAS_NUMPY:
|
||||||
|
# Fallback to keyword search on entity name
|
||||||
|
return self.search(entity, category=category, limit=limit)
|
||||||
|
|
||||||
|
conn = self.store._conn
|
||||||
|
|
||||||
|
# Encode entity as role-bound vector
|
||||||
|
role_entity = hrr.encode_atom("__hrr_role_entity__", self.hrr_dim)
|
||||||
|
entity_vec = hrr.encode_atom(entity.lower(), self.hrr_dim)
|
||||||
|
probe_key = hrr.bind(entity_vec, role_entity)
|
||||||
|
|
||||||
|
# Try category-specific bank first, then all facts
|
||||||
|
if category:
|
||||||
|
bank_name = f"cat:{category}"
|
||||||
|
bank_row = conn.execute(
|
||||||
|
"SELECT vector FROM memory_banks WHERE bank_name = ?",
|
||||||
|
(bank_name,),
|
||||||
|
).fetchone()
|
||||||
|
if bank_row:
|
||||||
|
bank_vec = hrr.bytes_to_phases(bank_row["vector"])
|
||||||
|
extracted = hrr.unbind(bank_vec, probe_key)
|
||||||
|
# Use extracted signal to score individual facts
|
||||||
|
return self._score_facts_by_vector(
|
||||||
|
extracted, category=category, limit=limit
|
||||||
|
)
|
||||||
|
|
||||||
|
# Score against individual fact vectors directly
|
||||||
|
where = "WHERE hrr_vector IS NOT NULL"
|
||||||
|
params: list = []
|
||||||
|
if category:
|
||||||
|
where += " AND category = ?"
|
||||||
|
params.append(category)
|
||||||
|
|
||||||
|
rows = conn.execute(
|
||||||
|
f"""
|
||||||
|
SELECT fact_id, content, category, tags, trust_score,
|
||||||
|
retrieval_count, helpful_count, created_at, updated_at,
|
||||||
|
hrr_vector
|
||||||
|
FROM facts
|
||||||
|
{where}
|
||||||
|
""",
|
||||||
|
params,
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
if not rows:
|
||||||
|
# Final fallback: keyword search
|
||||||
|
return self.search(entity, category=category, limit=limit)
|
||||||
|
|
||||||
|
scored = []
|
||||||
|
for row in rows:
|
||||||
|
fact = dict(row)
|
||||||
|
fact_vec = hrr.bytes_to_phases(fact.pop("hrr_vector"))
|
||||||
|
# Unbind probe key from fact to see if entity is structurally present
|
||||||
|
residual = hrr.unbind(fact_vec, probe_key)
|
||||||
|
# Compare residual against content signal
|
||||||
|
role_content = hrr.encode_atom("__hrr_role_content__", self.hrr_dim)
|
||||||
|
content_vec = hrr.bind(hrr.encode_text(fact["content"], self.hrr_dim), role_content)
|
||||||
|
sim = hrr.similarity(residual, content_vec)
|
||||||
|
fact["score"] = (sim + 1.0) / 2.0 * fact["trust_score"]
|
||||||
|
scored.append(fact)
|
||||||
|
|
||||||
|
scored.sort(key=lambda x: x["score"], reverse=True)
|
||||||
|
return scored[:limit]
|
||||||
|
|
||||||
|
def related(
|
||||||
|
self,
|
||||||
|
entity: str,
|
||||||
|
category: str | None = None,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Discover facts that share structural connections with an entity.
|
||||||
|
|
||||||
|
Unlike probe (which finds facts *about* an entity), related finds
|
||||||
|
facts that are connected through shared context — e.g., other entities
|
||||||
|
mentioned alongside this one, or content that overlaps structurally.
|
||||||
|
|
||||||
|
Falls back to FTS5 search if numpy unavailable.
|
||||||
|
"""
|
||||||
|
if not hrr._HAS_NUMPY:
|
||||||
|
return self.search(entity, category=category, limit=limit)
|
||||||
|
|
||||||
|
conn = self.store._conn
|
||||||
|
|
||||||
|
# Encode entity as a bare atom (not role-bound — we want ANY structural match)
|
||||||
|
entity_vec = hrr.encode_atom(entity.lower(), self.hrr_dim)
|
||||||
|
|
||||||
|
# Get all facts with vectors
|
||||||
|
where = "WHERE hrr_vector IS NOT NULL"
|
||||||
|
params: list = []
|
||||||
|
if category:
|
||||||
|
where += " AND category = ?"
|
||||||
|
params.append(category)
|
||||||
|
|
||||||
|
rows = conn.execute(
|
||||||
|
f"""
|
||||||
|
SELECT fact_id, content, category, tags, trust_score,
|
||||||
|
retrieval_count, helpful_count, created_at, updated_at,
|
||||||
|
hrr_vector
|
||||||
|
FROM facts
|
||||||
|
{where}
|
||||||
|
""",
|
||||||
|
params,
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
if not rows:
|
||||||
|
return self.search(entity, category=category, limit=limit)
|
||||||
|
|
||||||
|
# Score each fact by how much the entity's atom appears in its vector
|
||||||
|
# This catches both role-bound entity matches AND content word matches
|
||||||
|
scored = []
|
||||||
|
for row in rows:
|
||||||
|
fact = dict(row)
|
||||||
|
fact_vec = hrr.bytes_to_phases(fact.pop("hrr_vector"))
|
||||||
|
|
||||||
|
# Check structural similarity: unbind entity from fact
|
||||||
|
residual = hrr.unbind(fact_vec, entity_vec)
|
||||||
|
# A high-similarity residual to ANY known role vector means this entity
|
||||||
|
# plays a structural role in the fact
|
||||||
|
role_entity = hrr.encode_atom("__hrr_role_entity__", self.hrr_dim)
|
||||||
|
role_content = hrr.encode_atom("__hrr_role_content__", self.hrr_dim)
|
||||||
|
|
||||||
|
entity_role_sim = hrr.similarity(residual, role_entity)
|
||||||
|
content_role_sim = hrr.similarity(residual, role_content)
|
||||||
|
# Take the max — entity could appear in either role
|
||||||
|
best_sim = max(entity_role_sim, content_role_sim)
|
||||||
|
|
||||||
|
fact["score"] = (best_sim + 1.0) / 2.0 * fact["trust_score"]
|
||||||
|
scored.append(fact)
|
||||||
|
|
||||||
|
scored.sort(key=lambda x: x["score"], reverse=True)
|
||||||
|
return scored[:limit]
|
||||||
|
|
||||||
|
def reason(
|
||||||
|
self,
|
||||||
|
entities: list[str],
|
||||||
|
category: str | None = None,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Multi-entity compositional query — vector-space JOIN.
|
||||||
|
|
||||||
|
Given multiple entities, algebraically intersects their structural
|
||||||
|
connections to find facts related to ALL of them simultaneously.
|
||||||
|
This is compositional reasoning that no embedding DB can do.
|
||||||
|
|
||||||
|
Example: reason(["peppi", "backend"]) finds facts where peppi AND
|
||||||
|
backend both play structural roles — without keyword matching.
|
||||||
|
|
||||||
|
Falls back to FTS5 search if numpy unavailable.
|
||||||
|
"""
|
||||||
|
if not hrr._HAS_NUMPY or not entities:
|
||||||
|
# Fallback: search with all entities as keywords
|
||||||
|
query = " ".join(entities)
|
||||||
|
return self.search(query, category=category, limit=limit)
|
||||||
|
|
||||||
|
conn = self.store._conn
|
||||||
|
role_entity = hrr.encode_atom("__hrr_role_entity__", self.hrr_dim)
|
||||||
|
|
||||||
|
# For each entity, compute what the bank "remembers" about it
|
||||||
|
# by unbinding entity+role from each fact vector
|
||||||
|
entity_residuals = []
|
||||||
|
for entity in entities:
|
||||||
|
entity_vec = hrr.encode_atom(entity.lower(), self.hrr_dim)
|
||||||
|
probe_key = hrr.bind(entity_vec, role_entity)
|
||||||
|
entity_residuals.append(probe_key)
|
||||||
|
|
||||||
|
# Get all facts with vectors
|
||||||
|
where = "WHERE hrr_vector IS NOT NULL"
|
||||||
|
params: list = []
|
||||||
|
if category:
|
||||||
|
where += " AND category = ?"
|
||||||
|
params.append(category)
|
||||||
|
|
||||||
|
rows = conn.execute(
|
||||||
|
f"""
|
||||||
|
SELECT fact_id, content, category, tags, trust_score,
|
||||||
|
retrieval_count, helpful_count, created_at, updated_at,
|
||||||
|
hrr_vector
|
||||||
|
FROM facts
|
||||||
|
{where}
|
||||||
|
""",
|
||||||
|
params,
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
if not rows:
|
||||||
|
query = " ".join(entities)
|
||||||
|
return self.search(query, category=category, limit=limit)
|
||||||
|
|
||||||
|
# Score each fact by how much EACH entity is structurally present.
|
||||||
|
# A fact scores high only if ALL entities have structural presence
|
||||||
|
# (AND semantics via min, vs OR which would use mean/max).
|
||||||
|
role_content = hrr.encode_atom("__hrr_role_content__", self.hrr_dim)
|
||||||
|
|
||||||
|
scored = []
|
||||||
|
for row in rows:
|
||||||
|
fact = dict(row)
|
||||||
|
fact_vec = hrr.bytes_to_phases(fact.pop("hrr_vector"))
|
||||||
|
|
||||||
|
entity_scores = []
|
||||||
|
for probe_key in entity_residuals:
|
||||||
|
residual = hrr.unbind(fact_vec, probe_key)
|
||||||
|
sim = hrr.similarity(residual, role_content)
|
||||||
|
entity_scores.append(sim)
|
||||||
|
|
||||||
|
min_sim = min(entity_scores)
|
||||||
|
fact["score"] = (min_sim + 1.0) / 2.0 * fact["trust_score"]
|
||||||
|
scored.append(fact)
|
||||||
|
|
||||||
|
scored.sort(key=lambda x: x["score"], reverse=True)
|
||||||
|
return scored[:limit]
|
||||||
|
|
||||||
|
def contradict(
|
||||||
|
self,
|
||||||
|
category: str | None = None,
|
||||||
|
threshold: float = 0.3,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Find potentially contradictory facts via entity overlap + content divergence.
|
||||||
|
|
||||||
|
Two facts contradict when they share entities (same subject) but have
|
||||||
|
low content-vector similarity (different claims). This is automated
|
||||||
|
memory hygiene — no other memory system does this.
|
||||||
|
|
||||||
|
Returns pairs of facts with a contradiction score.
|
||||||
|
Falls back to empty list if numpy unavailable.
|
||||||
|
"""
|
||||||
|
if not hrr._HAS_NUMPY:
|
||||||
|
return []
|
||||||
|
|
||||||
|
conn = self.store._conn
|
||||||
|
|
||||||
|
# Get all facts with vectors and their linked entities
|
||||||
|
where = "WHERE f.hrr_vector IS NOT NULL"
|
||||||
|
params: list = []
|
||||||
|
if category:
|
||||||
|
where += " AND f.category = ?"
|
||||||
|
params.append(category)
|
||||||
|
|
||||||
|
rows = conn.execute(
|
||||||
|
f"""
|
||||||
|
SELECT f.fact_id, f.content, f.category, f.tags, f.trust_score,
|
||||||
|
f.created_at, f.updated_at, f.hrr_vector
|
||||||
|
FROM facts f
|
||||||
|
{where}
|
||||||
|
""",
|
||||||
|
params,
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
if len(rows) < 2:
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Guard against O(n²) explosion on large fact stores.
|
||||||
|
# At 500 facts, that's ~125K comparisons — acceptable.
|
||||||
|
# Above that, only check the most recently updated facts.
|
||||||
|
_MAX_CONTRADICT_FACTS = 500
|
||||||
|
if len(rows) > _MAX_CONTRADICT_FACTS:
|
||||||
|
rows = sorted(rows, key=lambda r: r["updated_at"] or r["created_at"], reverse=True)
|
||||||
|
rows = rows[:_MAX_CONTRADICT_FACTS]
|
||||||
|
|
||||||
|
# Build entity sets per fact
|
||||||
|
fact_entities: dict[int, set[str]] = {}
|
||||||
|
for row in rows:
|
||||||
|
fid = row["fact_id"]
|
||||||
|
entity_rows = conn.execute(
|
||||||
|
"""
|
||||||
|
SELECT e.name FROM entities e
|
||||||
|
JOIN fact_entities fe ON fe.entity_id = e.entity_id
|
||||||
|
WHERE fe.fact_id = ?
|
||||||
|
""",
|
||||||
|
(fid,),
|
||||||
|
).fetchall()
|
||||||
|
fact_entities[fid] = {r["name"].lower() for r in entity_rows}
|
||||||
|
|
||||||
|
# Compare all pairs: high entity overlap + low content similarity = contradiction
|
||||||
|
facts = [dict(r) for r in rows]
|
||||||
|
contradictions = []
|
||||||
|
|
||||||
|
for i in range(len(facts)):
|
||||||
|
for j in range(i + 1, len(facts)):
|
||||||
|
f1, f2 = facts[i], facts[j]
|
||||||
|
ents1 = fact_entities.get(f1["fact_id"], set())
|
||||||
|
ents2 = fact_entities.get(f2["fact_id"], set())
|
||||||
|
|
||||||
|
if not ents1 or not ents2:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Entity overlap (Jaccard)
|
||||||
|
entity_overlap = len(ents1 & ents2) / len(ents1 | ents2) if (ents1 | ents2) else 0.0
|
||||||
|
|
||||||
|
if entity_overlap < 0.3:
|
||||||
|
continue # Not enough entity overlap to be contradictory
|
||||||
|
|
||||||
|
# Content similarity via HRR vectors
|
||||||
|
v1 = hrr.bytes_to_phases(f1["hrr_vector"])
|
||||||
|
v2 = hrr.bytes_to_phases(f2["hrr_vector"])
|
||||||
|
content_sim = hrr.similarity(v1, v2)
|
||||||
|
|
||||||
|
# High entity overlap + low content similarity = potential contradiction
|
||||||
|
# contradiction_score: higher = more contradictory
|
||||||
|
contradiction_score = entity_overlap * (1.0 - (content_sim + 1.0) / 2.0)
|
||||||
|
|
||||||
|
if contradiction_score >= threshold:
|
||||||
|
# Strip hrr_vector from output (not JSON serializable)
|
||||||
|
f1_clean = {k: v for k, v in f1.items() if k != "hrr_vector"}
|
||||||
|
f2_clean = {k: v for k, v in f2.items() if k != "hrr_vector"}
|
||||||
|
contradictions.append({
|
||||||
|
"fact_a": f1_clean,
|
||||||
|
"fact_b": f2_clean,
|
||||||
|
"entity_overlap": round(entity_overlap, 3),
|
||||||
|
"content_similarity": round(content_sim, 3),
|
||||||
|
"contradiction_score": round(contradiction_score, 3),
|
||||||
|
"shared_entities": sorted(ents1 & ents2),
|
||||||
|
})
|
||||||
|
|
||||||
|
contradictions.sort(key=lambda x: x["contradiction_score"], reverse=True)
|
||||||
|
return contradictions[:limit]
|
||||||
|
|
||||||
|
def _score_facts_by_vector(
|
||||||
|
self,
|
||||||
|
target_vec: "np.ndarray",
|
||||||
|
category: str | None = None,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Score facts by similarity to a target vector."""
|
||||||
|
conn = self.store._conn
|
||||||
|
|
||||||
|
where = "WHERE hrr_vector IS NOT NULL"
|
||||||
|
params: list = []
|
||||||
|
if category:
|
||||||
|
where += " AND category = ?"
|
||||||
|
params.append(category)
|
||||||
|
|
||||||
|
rows = conn.execute(
|
||||||
|
f"""
|
||||||
|
SELECT fact_id, content, category, tags, trust_score,
|
||||||
|
retrieval_count, helpful_count, created_at, updated_at,
|
||||||
|
hrr_vector
|
||||||
|
FROM facts
|
||||||
|
{where}
|
||||||
|
""",
|
||||||
|
params,
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
scored = []
|
||||||
|
for row in rows:
|
||||||
|
fact = dict(row)
|
||||||
|
fact_vec = hrr.bytes_to_phases(fact.pop("hrr_vector"))
|
||||||
|
sim = hrr.similarity(target_vec, fact_vec)
|
||||||
|
fact["score"] = (sim + 1.0) / 2.0 * fact["trust_score"]
|
||||||
|
scored.append(fact)
|
||||||
|
|
||||||
|
scored.sort(key=lambda x: x["score"], reverse=True)
|
||||||
|
return scored[:limit]
|
||||||
|
|
||||||
|
def _fts_candidates(
|
||||||
|
self,
|
||||||
|
query: str,
|
||||||
|
category: str | None,
|
||||||
|
min_trust: float,
|
||||||
|
limit: int,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Get raw FTS5 candidates from the store.
|
||||||
|
|
||||||
|
Uses the store's database connection directly for FTS5 MATCH
|
||||||
|
with rank scoring. Normalizes FTS5 rank to [0, 1] range.
|
||||||
|
"""
|
||||||
|
conn = self.store._conn
|
||||||
|
|
||||||
|
# Build query - FTS5 rank is negative (lower = better match)
|
||||||
|
# We need to join facts_fts with facts to get all columns
|
||||||
|
params: list = []
|
||||||
|
where_clauses = ["facts_fts MATCH ?"]
|
||||||
|
params.append(query)
|
||||||
|
|
||||||
|
if category:
|
||||||
|
where_clauses.append("f.category = ?")
|
||||||
|
params.append(category)
|
||||||
|
|
||||||
|
where_clauses.append("f.trust_score >= ?")
|
||||||
|
params.append(min_trust)
|
||||||
|
|
||||||
|
where_sql = " AND ".join(where_clauses)
|
||||||
|
|
||||||
|
sql = f"""
|
||||||
|
SELECT f.*, facts_fts.rank as fts_rank_raw
|
||||||
|
FROM facts_fts
|
||||||
|
JOIN facts f ON f.fact_id = facts_fts.rowid
|
||||||
|
WHERE {where_sql}
|
||||||
|
ORDER BY facts_fts.rank
|
||||||
|
LIMIT ?
|
||||||
|
"""
|
||||||
|
params.append(limit)
|
||||||
|
|
||||||
|
try:
|
||||||
|
rows = conn.execute(sql, params).fetchall()
|
||||||
|
except Exception:
|
||||||
|
# FTS5 MATCH can fail on malformed queries — fall back to empty
|
||||||
|
return []
|
||||||
|
|
||||||
|
if not rows:
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Normalize FTS5 rank: rank is negative, lower = better
|
||||||
|
# Convert to positive score in [0, 1] range
|
||||||
|
raw_ranks = [abs(row["fts_rank_raw"]) for row in rows]
|
||||||
|
max_rank = max(raw_ranks) if raw_ranks else 1.0
|
||||||
|
max_rank = max(max_rank, 1e-6) # avoid div by zero
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for row, raw_rank in zip(rows, raw_ranks):
|
||||||
|
fact = dict(row)
|
||||||
|
fact.pop("fts_rank_raw", None)
|
||||||
|
fact["fts_rank"] = raw_rank / max_rank # normalize to [0, 1]
|
||||||
|
results.append(fact)
|
||||||
|
|
||||||
|
return results
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _tokenize(text: str) -> set[str]:
|
||||||
|
"""Simple whitespace tokenization with lowercasing.
|
||||||
|
|
||||||
|
Strips common punctuation. No stemming/lemmatization (Phase 1).
|
||||||
|
"""
|
||||||
|
if not text:
|
||||||
|
return set()
|
||||||
|
# Split on whitespace, lowercase, strip punctuation
|
||||||
|
tokens = set()
|
||||||
|
for word in text.lower().split():
|
||||||
|
cleaned = word.strip(".,;:!?\"'()[]{}#@<>")
|
||||||
|
if cleaned:
|
||||||
|
tokens.add(cleaned)
|
||||||
|
return tokens
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _jaccard_similarity(set_a: set, set_b: set) -> float:
|
||||||
|
"""Jaccard similarity coefficient: |A ∩ B| / |A ∪ B|."""
|
||||||
|
if not set_a or not set_b:
|
||||||
|
return 0.0
|
||||||
|
intersection = len(set_a & set_b)
|
||||||
|
union = len(set_a | set_b)
|
||||||
|
return intersection / union if union > 0 else 0.0
|
||||||
|
|
||||||
|
def _temporal_decay(self, timestamp_str: str | None) -> float:
|
||||||
|
"""Exponential decay: 0.5^(age_days / half_life_days).
|
||||||
|
|
||||||
|
Returns 1.0 if decay is disabled or timestamp is missing.
|
||||||
|
"""
|
||||||
|
if not self.half_life or not timestamp_str:
|
||||||
|
return 1.0
|
||||||
|
|
||||||
|
try:
|
||||||
|
if isinstance(timestamp_str, str):
|
||||||
|
# Parse ISO format timestamp from SQLite
|
||||||
|
ts = datetime.fromisoformat(timestamp_str.replace("Z", "+00:00"))
|
||||||
|
else:
|
||||||
|
ts = timestamp_str
|
||||||
|
|
||||||
|
if ts.tzinfo is None:
|
||||||
|
ts = ts.replace(tzinfo=timezone.utc)
|
||||||
|
|
||||||
|
age_days = (datetime.now(timezone.utc) - ts).total_seconds() / 86400
|
||||||
|
if age_days < 0:
|
||||||
|
return 1.0
|
||||||
|
|
||||||
|
return math.pow(0.5, age_days / self.half_life)
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
return 1.0
|
||||||
575
plugins/memory/holographic/store.py
Normal file
575
plugins/memory/holographic/store.py
Normal file
|
|
@ -0,0 +1,575 @@
|
||||||
|
"""
|
||||||
|
SQLite-backed fact store with entity resolution and trust scoring.
|
||||||
|
Single-user Hermes memory store plugin.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import re
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
try:
|
||||||
|
from . import holographic as hrr
|
||||||
|
except ImportError:
|
||||||
|
import holographic as hrr # type: ignore[no-redef]
|
||||||
|
|
||||||
|
_SCHEMA = """
|
||||||
|
CREATE TABLE IF NOT EXISTS facts (
|
||||||
|
fact_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
content TEXT NOT NULL UNIQUE,
|
||||||
|
category TEXT DEFAULT 'general',
|
||||||
|
tags TEXT DEFAULT '',
|
||||||
|
trust_score REAL DEFAULT 0.5,
|
||||||
|
retrieval_count INTEGER DEFAULT 0,
|
||||||
|
helpful_count INTEGER DEFAULT 0,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
hrr_vector BLOB
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS entities (
|
||||||
|
entity_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
entity_type TEXT DEFAULT 'unknown',
|
||||||
|
aliases TEXT DEFAULT '',
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS fact_entities (
|
||||||
|
fact_id INTEGER REFERENCES facts(fact_id),
|
||||||
|
entity_id INTEGER REFERENCES entities(entity_id),
|
||||||
|
PRIMARY KEY (fact_id, entity_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_facts_trust ON facts(trust_score DESC);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_facts_category ON facts(category);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_entities_name ON entities(name);
|
||||||
|
|
||||||
|
CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts
|
||||||
|
USING fts5(content, tags, content=facts, content_rowid=fact_id);
|
||||||
|
|
||||||
|
CREATE TRIGGER IF NOT EXISTS facts_ai AFTER INSERT ON facts BEGIN
|
||||||
|
INSERT INTO facts_fts(rowid, content, tags)
|
||||||
|
VALUES (new.fact_id, new.content, new.tags);
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TRIGGER IF NOT EXISTS facts_ad AFTER DELETE ON facts BEGIN
|
||||||
|
INSERT INTO facts_fts(facts_fts, rowid, content, tags)
|
||||||
|
VALUES ('delete', old.fact_id, old.content, old.tags);
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TRIGGER IF NOT EXISTS facts_au AFTER UPDATE ON facts BEGIN
|
||||||
|
INSERT INTO facts_fts(facts_fts, rowid, content, tags)
|
||||||
|
VALUES ('delete', old.fact_id, old.content, old.tags);
|
||||||
|
INSERT INTO facts_fts(rowid, content, tags)
|
||||||
|
VALUES (new.fact_id, new.content, new.tags);
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS memory_banks (
|
||||||
|
bank_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
bank_name TEXT NOT NULL UNIQUE,
|
||||||
|
vector BLOB NOT NULL,
|
||||||
|
dim INTEGER NOT NULL,
|
||||||
|
fact_count INTEGER DEFAULT 0,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
|
||||||
|
# Trust adjustment constants
|
||||||
|
_HELPFUL_DELTA = 0.05
|
||||||
|
_UNHELPFUL_DELTA = -0.10
|
||||||
|
_TRUST_MIN = 0.0
|
||||||
|
_TRUST_MAX = 1.0
|
||||||
|
|
||||||
|
# Entity extraction patterns
|
||||||
|
_RE_CAPITALIZED = re.compile(r'\b([A-Z][a-z]+(?:\s+[A-Z][a-z]+)+)\b')
|
||||||
|
_RE_DOUBLE_QUOTE = re.compile(r'"([^"]+)"')
|
||||||
|
_RE_SINGLE_QUOTE = re.compile(r"'([^']+)'")
|
||||||
|
_RE_AKA = re.compile(
|
||||||
|
r'(\w+(?:\s+\w+)*)\s+(?:aka|also known as)\s+(\w+(?:\s+\w+)*)',
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _clamp_trust(value: float) -> float:
|
||||||
|
return max(_TRUST_MIN, min(_TRUST_MAX, value))
|
||||||
|
|
||||||
|
|
||||||
|
class MemoryStore:
|
||||||
|
"""SQLite-backed fact store with entity resolution and trust scoring."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
db_path: "str | Path | None" = None,
|
||||||
|
default_trust: float = 0.5,
|
||||||
|
hrr_dim: int = 1024,
|
||||||
|
) -> None:
|
||||||
|
if db_path is None:
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
db_path = str(get_hermes_home() / "memory_store.db")
|
||||||
|
self.db_path = Path(db_path).expanduser()
|
||||||
|
self.db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.default_trust = _clamp_trust(default_trust)
|
||||||
|
self.hrr_dim = hrr_dim
|
||||||
|
self._hrr_available = hrr._HAS_NUMPY
|
||||||
|
self._conn: sqlite3.Connection = sqlite3.connect(
|
||||||
|
str(self.db_path),
|
||||||
|
check_same_thread=False,
|
||||||
|
timeout=10.0,
|
||||||
|
)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._conn.row_factory = sqlite3.Row
|
||||||
|
self._init_db()
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Initialisation
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _init_db(self) -> None:
|
||||||
|
"""Create tables, indexes, and triggers if they do not exist. Enable WAL mode."""
|
||||||
|
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||||
|
self._conn.executescript(_SCHEMA)
|
||||||
|
# Migrate: add hrr_vector column if missing (safe for existing databases)
|
||||||
|
columns = {row[1] for row in self._conn.execute("PRAGMA table_info(facts)").fetchall()}
|
||||||
|
if "hrr_vector" not in columns:
|
||||||
|
self._conn.execute("ALTER TABLE facts ADD COLUMN hrr_vector BLOB")
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Public API
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def add_fact(
|
||||||
|
self,
|
||||||
|
content: str,
|
||||||
|
category: str = "general",
|
||||||
|
tags: str = "",
|
||||||
|
) -> int:
|
||||||
|
"""Insert a fact and return its fact_id.
|
||||||
|
|
||||||
|
Deduplicates by content (UNIQUE constraint). On duplicate, returns
|
||||||
|
the existing fact_id without modifying the row. Extracts entities from
|
||||||
|
the content and links them to the fact.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
content = content.strip()
|
||||||
|
if not content:
|
||||||
|
raise ValueError("content must not be empty")
|
||||||
|
|
||||||
|
try:
|
||||||
|
cur = self._conn.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO facts (content, category, tags, trust_score)
|
||||||
|
VALUES (?, ?, ?, ?)
|
||||||
|
""",
|
||||||
|
(content, category, tags, self.default_trust),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
fact_id: int = cur.lastrowid # type: ignore[assignment]
|
||||||
|
except sqlite3.IntegrityError:
|
||||||
|
# Duplicate content — return existing id
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT fact_id FROM facts WHERE content = ?", (content,)
|
||||||
|
).fetchone()
|
||||||
|
return int(row["fact_id"])
|
||||||
|
|
||||||
|
# Entity extraction and linking
|
||||||
|
for name in self._extract_entities(content):
|
||||||
|
entity_id = self._resolve_entity(name)
|
||||||
|
self._link_fact_entity(fact_id, entity_id)
|
||||||
|
|
||||||
|
# Compute HRR vector after entity linking
|
||||||
|
self._compute_hrr_vector(fact_id, content)
|
||||||
|
self._rebuild_bank(category)
|
||||||
|
|
||||||
|
return fact_id
|
||||||
|
|
||||||
|
def search_facts(
|
||||||
|
self,
|
||||||
|
query: str,
|
||||||
|
category: str | None = None,
|
||||||
|
min_trust: float = 0.3,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Full-text search over facts using FTS5.
|
||||||
|
|
||||||
|
Returns a list of fact dicts ordered by FTS5 rank, then trust_score
|
||||||
|
descending. Also increments retrieval_count for matched facts.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
query = query.strip()
|
||||||
|
if not query:
|
||||||
|
return []
|
||||||
|
|
||||||
|
params: list = [query, min_trust]
|
||||||
|
category_clause = ""
|
||||||
|
if category is not None:
|
||||||
|
category_clause = "AND f.category = ?"
|
||||||
|
params.append(category)
|
||||||
|
params.append(limit)
|
||||||
|
|
||||||
|
sql = f"""
|
||||||
|
SELECT f.fact_id, f.content, f.category, f.tags,
|
||||||
|
f.trust_score, f.retrieval_count, f.helpful_count,
|
||||||
|
f.created_at, f.updated_at
|
||||||
|
FROM facts f
|
||||||
|
JOIN facts_fts fts ON fts.rowid = f.fact_id
|
||||||
|
WHERE facts_fts MATCH ?
|
||||||
|
AND f.trust_score >= ?
|
||||||
|
{category_clause}
|
||||||
|
ORDER BY fts.rank, f.trust_score DESC
|
||||||
|
LIMIT ?
|
||||||
|
"""
|
||||||
|
|
||||||
|
rows = self._conn.execute(sql, params).fetchall()
|
||||||
|
results = [self._row_to_dict(r) for r in rows]
|
||||||
|
|
||||||
|
if results:
|
||||||
|
ids = [r["fact_id"] for r in results]
|
||||||
|
placeholders = ",".join("?" * len(ids))
|
||||||
|
self._conn.execute(
|
||||||
|
f"UPDATE facts SET retrieval_count = retrieval_count + 1 WHERE fact_id IN ({placeholders})",
|
||||||
|
ids,
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
return results
|
||||||
|
|
||||||
|
def update_fact(
|
||||||
|
self,
|
||||||
|
fact_id: int,
|
||||||
|
content: str | None = None,
|
||||||
|
trust_delta: float | None = None,
|
||||||
|
tags: str | None = None,
|
||||||
|
category: str | None = None,
|
||||||
|
) -> bool:
|
||||||
|
"""Partially update a fact. Trust is clamped to [0, 1].
|
||||||
|
|
||||||
|
Returns True if the row existed, False otherwise.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT fact_id, trust_score FROM facts WHERE fact_id = ?", (fact_id,)
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return False
|
||||||
|
|
||||||
|
assignments: list[str] = ["updated_at = CURRENT_TIMESTAMP"]
|
||||||
|
params: list = []
|
||||||
|
|
||||||
|
if content is not None:
|
||||||
|
assignments.append("content = ?")
|
||||||
|
params.append(content.strip())
|
||||||
|
if tags is not None:
|
||||||
|
assignments.append("tags = ?")
|
||||||
|
params.append(tags)
|
||||||
|
if category is not None:
|
||||||
|
assignments.append("category = ?")
|
||||||
|
params.append(category)
|
||||||
|
if trust_delta is not None:
|
||||||
|
new_trust = _clamp_trust(row["trust_score"] + trust_delta)
|
||||||
|
assignments.append("trust_score = ?")
|
||||||
|
params.append(new_trust)
|
||||||
|
|
||||||
|
params.append(fact_id)
|
||||||
|
self._conn.execute(
|
||||||
|
f"UPDATE facts SET {', '.join(assignments)} WHERE fact_id = ?",
|
||||||
|
params,
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
# If content changed, re-extract entities
|
||||||
|
if content is not None:
|
||||||
|
self._conn.execute(
|
||||||
|
"DELETE FROM fact_entities WHERE fact_id = ?", (fact_id,)
|
||||||
|
)
|
||||||
|
for name in self._extract_entities(content):
|
||||||
|
entity_id = self._resolve_entity(name)
|
||||||
|
self._link_fact_entity(fact_id, entity_id)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
# Recompute HRR vector if content changed
|
||||||
|
if content is not None:
|
||||||
|
self._compute_hrr_vector(fact_id, content)
|
||||||
|
# Rebuild bank for relevant category
|
||||||
|
cat = category or self._conn.execute(
|
||||||
|
"SELECT category FROM facts WHERE fact_id = ?", (fact_id,)
|
||||||
|
).fetchone()["category"]
|
||||||
|
self._rebuild_bank(cat)
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
def remove_fact(self, fact_id: int) -> bool:
|
||||||
|
"""Delete a fact and its entity links. Returns True if the row existed."""
|
||||||
|
with self._lock:
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT fact_id, category FROM facts WHERE fact_id = ?", (fact_id,)
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return False
|
||||||
|
|
||||||
|
self._conn.execute(
|
||||||
|
"DELETE FROM fact_entities WHERE fact_id = ?", (fact_id,)
|
||||||
|
)
|
||||||
|
self._conn.execute("DELETE FROM facts WHERE fact_id = ?", (fact_id,))
|
||||||
|
self._conn.commit()
|
||||||
|
self._rebuild_bank(row["category"])
|
||||||
|
return True
|
||||||
|
|
||||||
|
def list_facts(
|
||||||
|
self,
|
||||||
|
category: str | None = None,
|
||||||
|
min_trust: float = 0.0,
|
||||||
|
limit: int = 50,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Browse facts ordered by trust_score descending.
|
||||||
|
|
||||||
|
Optionally filter by category and minimum trust score.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
params: list = [min_trust]
|
||||||
|
category_clause = ""
|
||||||
|
if category is not None:
|
||||||
|
category_clause = "AND category = ?"
|
||||||
|
params.append(category)
|
||||||
|
params.append(limit)
|
||||||
|
|
||||||
|
sql = f"""
|
||||||
|
SELECT fact_id, content, category, tags, trust_score,
|
||||||
|
retrieval_count, helpful_count, created_at, updated_at
|
||||||
|
FROM facts
|
||||||
|
WHERE trust_score >= ?
|
||||||
|
{category_clause}
|
||||||
|
ORDER BY trust_score DESC
|
||||||
|
LIMIT ?
|
||||||
|
"""
|
||||||
|
rows = self._conn.execute(sql, params).fetchall()
|
||||||
|
return [self._row_to_dict(r) for r in rows]
|
||||||
|
|
||||||
|
def record_feedback(self, fact_id: int, helpful: bool) -> dict:
|
||||||
|
"""Record user feedback and adjust trust asymmetrically.
|
||||||
|
|
||||||
|
helpful=True -> trust += 0.05, helpful_count += 1
|
||||||
|
helpful=False -> trust -= 0.10
|
||||||
|
|
||||||
|
Returns a dict with fact_id, old_trust, new_trust, helpful_count.
|
||||||
|
Raises KeyError if fact_id does not exist.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT fact_id, trust_score, helpful_count FROM facts WHERE fact_id = ?",
|
||||||
|
(fact_id,),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise KeyError(f"fact_id {fact_id} not found")
|
||||||
|
|
||||||
|
old_trust: float = row["trust_score"]
|
||||||
|
delta = _HELPFUL_DELTA if helpful else _UNHELPFUL_DELTA
|
||||||
|
new_trust = _clamp_trust(old_trust + delta)
|
||||||
|
|
||||||
|
helpful_increment = 1 if helpful else 0
|
||||||
|
self._conn.execute(
|
||||||
|
"""
|
||||||
|
UPDATE facts
|
||||||
|
SET trust_score = ?,
|
||||||
|
helpful_count = helpful_count + ?,
|
||||||
|
updated_at = CURRENT_TIMESTAMP
|
||||||
|
WHERE fact_id = ?
|
||||||
|
""",
|
||||||
|
(new_trust, helpful_increment, fact_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
return {
|
||||||
|
"fact_id": fact_id,
|
||||||
|
"old_trust": old_trust,
|
||||||
|
"new_trust": new_trust,
|
||||||
|
"helpful_count": row["helpful_count"] + helpful_increment,
|
||||||
|
}
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Entity helpers
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _extract_entities(self, text: str) -> list[str]:
|
||||||
|
"""Extract entity candidates from text using simple regex rules.
|
||||||
|
|
||||||
|
Rules applied (in order):
|
||||||
|
1. Capitalized multi-word phrases e.g. "John Doe"
|
||||||
|
2. Double-quoted terms e.g. "Python"
|
||||||
|
3. Single-quoted terms e.g. 'pytest'
|
||||||
|
4. AKA patterns e.g. "Guido aka BDFL" -> two entities
|
||||||
|
|
||||||
|
Returns a deduplicated list preserving first-seen order.
|
||||||
|
"""
|
||||||
|
seen: set[str] = set()
|
||||||
|
candidates: list[str] = []
|
||||||
|
|
||||||
|
def _add(name: str) -> None:
|
||||||
|
stripped = name.strip()
|
||||||
|
if stripped and stripped.lower() not in seen:
|
||||||
|
seen.add(stripped.lower())
|
||||||
|
candidates.append(stripped)
|
||||||
|
|
||||||
|
for m in _RE_CAPITALIZED.finditer(text):
|
||||||
|
_add(m.group(1))
|
||||||
|
|
||||||
|
for m in _RE_DOUBLE_QUOTE.finditer(text):
|
||||||
|
_add(m.group(1))
|
||||||
|
|
||||||
|
for m in _RE_SINGLE_QUOTE.finditer(text):
|
||||||
|
_add(m.group(1))
|
||||||
|
|
||||||
|
for m in _RE_AKA.finditer(text):
|
||||||
|
_add(m.group(1))
|
||||||
|
_add(m.group(2))
|
||||||
|
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
def _resolve_entity(self, name: str) -> int:
|
||||||
|
"""Find an existing entity by name or alias (case-insensitive) or create one.
|
||||||
|
|
||||||
|
Returns the entity_id.
|
||||||
|
"""
|
||||||
|
# Exact name match
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT entity_id FROM entities WHERE name LIKE ?", (name,)
|
||||||
|
).fetchone()
|
||||||
|
if row is not None:
|
||||||
|
return int(row["entity_id"])
|
||||||
|
|
||||||
|
# Search aliases — aliases stored as comma-separated; use LIKE with % boundaries
|
||||||
|
alias_row = self._conn.execute(
|
||||||
|
"""
|
||||||
|
SELECT entity_id FROM entities
|
||||||
|
WHERE ',' || aliases || ',' LIKE '%,' || ? || ',%'
|
||||||
|
""",
|
||||||
|
(name,),
|
||||||
|
).fetchone()
|
||||||
|
if alias_row is not None:
|
||||||
|
return int(alias_row["entity_id"])
|
||||||
|
|
||||||
|
# Create new entity
|
||||||
|
cur = self._conn.execute(
|
||||||
|
"INSERT INTO entities (name) VALUES (?)", (name,)
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
return int(cur.lastrowid) # type: ignore[return-value]
|
||||||
|
|
||||||
|
def _link_fact_entity(self, fact_id: int, entity_id: int) -> None:
|
||||||
|
"""Insert into fact_entities, silently ignore if the link already exists."""
|
||||||
|
self._conn.execute(
|
||||||
|
"""
|
||||||
|
INSERT OR IGNORE INTO fact_entities (fact_id, entity_id)
|
||||||
|
VALUES (?, ?)
|
||||||
|
""",
|
||||||
|
(fact_id, entity_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
def _compute_hrr_vector(self, fact_id: int, content: str) -> None:
|
||||||
|
"""Compute and store HRR vector for a fact. No-op if numpy unavailable."""
|
||||||
|
with self._lock:
|
||||||
|
if not self._hrr_available:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Get entities linked to this fact
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"""
|
||||||
|
SELECT e.name FROM entities e
|
||||||
|
JOIN fact_entities fe ON fe.entity_id = e.entity_id
|
||||||
|
WHERE fe.fact_id = ?
|
||||||
|
""",
|
||||||
|
(fact_id,),
|
||||||
|
).fetchall()
|
||||||
|
entities = [row["name"] for row in rows]
|
||||||
|
|
||||||
|
vector = hrr.encode_fact(content, entities, self.hrr_dim)
|
||||||
|
self._conn.execute(
|
||||||
|
"UPDATE facts SET hrr_vector = ? WHERE fact_id = ?",
|
||||||
|
(hrr.phases_to_bytes(vector), fact_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
def _rebuild_bank(self, category: str) -> None:
|
||||||
|
"""Full rebuild of a category's memory bank from all its fact vectors."""
|
||||||
|
with self._lock:
|
||||||
|
if not self._hrr_available:
|
||||||
|
return
|
||||||
|
|
||||||
|
bank_name = f"cat:{category}"
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT hrr_vector FROM facts WHERE category = ? AND hrr_vector IS NOT NULL",
|
||||||
|
(category,),
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
if not rows:
|
||||||
|
self._conn.execute("DELETE FROM memory_banks WHERE bank_name = ?", (bank_name,))
|
||||||
|
self._conn.commit()
|
||||||
|
return
|
||||||
|
|
||||||
|
vectors = [hrr.bytes_to_phases(row["hrr_vector"]) for row in rows]
|
||||||
|
bank_vector = hrr.bundle(*vectors)
|
||||||
|
fact_count = len(vectors)
|
||||||
|
|
||||||
|
# Check SNR
|
||||||
|
hrr.snr_estimate(self.hrr_dim, fact_count)
|
||||||
|
|
||||||
|
self._conn.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO memory_banks (bank_name, vector, dim, fact_count, updated_at)
|
||||||
|
VALUES (?, ?, ?, ?, CURRENT_TIMESTAMP)
|
||||||
|
ON CONFLICT(bank_name) DO UPDATE SET
|
||||||
|
vector = excluded.vector,
|
||||||
|
dim = excluded.dim,
|
||||||
|
fact_count = excluded.fact_count,
|
||||||
|
updated_at = excluded.updated_at
|
||||||
|
""",
|
||||||
|
(bank_name, hrr.phases_to_bytes(bank_vector), self.hrr_dim, fact_count),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
def rebuild_all_vectors(self, dim: int | None = None) -> int:
|
||||||
|
"""Recompute all HRR vectors + banks from text. For recovery/migration.
|
||||||
|
|
||||||
|
Returns the number of facts processed.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
if not self._hrr_available:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if dim is not None:
|
||||||
|
self.hrr_dim = dim
|
||||||
|
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT fact_id, content, category FROM facts"
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
categories: set[str] = set()
|
||||||
|
for row in rows:
|
||||||
|
self._compute_hrr_vector(row["fact_id"], row["content"])
|
||||||
|
categories.add(row["category"])
|
||||||
|
|
||||||
|
for category in categories:
|
||||||
|
self._rebuild_bank(category)
|
||||||
|
|
||||||
|
return len(rows)
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Utilities
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _row_to_dict(self, row: sqlite3.Row) -> dict:
|
||||||
|
"""Convert a sqlite3.Row to a plain dict."""
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
"""Close the database connection."""
|
||||||
|
self._conn.close()
|
||||||
|
|
||||||
|
def __enter__(self) -> "MemoryStore":
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *_: object) -> None:
|
||||||
|
self.close()
|
||||||
35
plugins/memory/honcho/README.md
Normal file
35
plugins/memory/honcho/README.md
Normal file
|
|
@ -0,0 +1,35 @@
|
||||||
|
# Honcho Memory Provider
|
||||||
|
|
||||||
|
AI-native cross-session user modeling with dialectic Q&A, semantic search, peer cards, and persistent conclusions.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- `pip install honcho-ai`
|
||||||
|
- Honcho API key from [app.honcho.dev](https://app.honcho.dev)
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "honcho"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
hermes config set memory.provider honcho
|
||||||
|
echo "HONCHO_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
Config file: `$HERMES_HOME/honcho.json` (or `~/.honcho/config.json` legacy)
|
||||||
|
|
||||||
|
Existing Honcho users: your config and data are preserved. Just set `memory.provider: honcho`.
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `honcho_profile` | User's peer card — key facts, no LLM |
|
||||||
|
| `honcho_search` | Semantic search over stored context |
|
||||||
|
| `honcho_context` | LLM-synthesized answer from memory |
|
||||||
|
| `honcho_conclude` | Write a fact about the user to memory |
|
||||||
355
plugins/memory/honcho/__init__.py
Normal file
355
plugins/memory/honcho/__init__.py
Normal file
|
|
@ -0,0 +1,355 @@
|
||||||
|
"""Honcho memory plugin — MemoryProvider for Honcho AI-native memory.
|
||||||
|
|
||||||
|
Provides cross-session user modeling with dialectic Q&A, semantic search,
|
||||||
|
peer cards, and persistent conclusions via the Honcho SDK. Honcho provides AI-native cross-session user
|
||||||
|
modeling with dialectic Q&A, semantic search, peer cards, and conclusions.
|
||||||
|
|
||||||
|
The 4 tools (profile, search, context, conclude) are exposed through
|
||||||
|
the MemoryProvider interface.
|
||||||
|
|
||||||
|
Config: Uses the existing Honcho config chain:
|
||||||
|
1. $HERMES_HOME/honcho.json (profile-scoped)
|
||||||
|
2. ~/.honcho/config.json (legacy global)
|
||||||
|
3. Environment variables
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import threading
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tool schemas (moved from tools/honcho_tools.py)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PROFILE_SCHEMA = {
|
||||||
|
"name": "honcho_profile",
|
||||||
|
"description": (
|
||||||
|
"Retrieve the user's peer card from Honcho — a curated list of key facts "
|
||||||
|
"about them (name, role, preferences, communication style, patterns). "
|
||||||
|
"Fast, no LLM reasoning, minimal cost. "
|
||||||
|
"Use this at conversation start or when you need a quick factual snapshot."
|
||||||
|
),
|
||||||
|
"parameters": {"type": "object", "properties": {}, "required": []},
|
||||||
|
}
|
||||||
|
|
||||||
|
SEARCH_SCHEMA = {
|
||||||
|
"name": "honcho_search",
|
||||||
|
"description": (
|
||||||
|
"Semantic search over Honcho's stored context about the user. "
|
||||||
|
"Returns raw excerpts ranked by relevance — no LLM synthesis. "
|
||||||
|
"Cheaper and faster than honcho_context. "
|
||||||
|
"Good when you want to find specific past facts and reason over them yourself."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "What to search for in Honcho's memory.",
|
||||||
|
},
|
||||||
|
"max_tokens": {
|
||||||
|
"type": "integer",
|
||||||
|
"description": "Token budget for returned context (default 800, max 2000).",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
CONTEXT_SCHEMA = {
|
||||||
|
"name": "honcho_context",
|
||||||
|
"description": (
|
||||||
|
"Ask Honcho a natural language question and get a synthesized answer. "
|
||||||
|
"Uses Honcho's LLM (dialectic reasoning) — higher cost than honcho_profile or honcho_search. "
|
||||||
|
"Can query about any peer: the user (default) or the AI assistant."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "A natural language question.",
|
||||||
|
},
|
||||||
|
"peer": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Which peer to query about: 'user' (default) or 'ai'.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
CONCLUDE_SCHEMA = {
|
||||||
|
"name": "honcho_conclude",
|
||||||
|
"description": (
|
||||||
|
"Write a conclusion about the user back to Honcho's memory. "
|
||||||
|
"Conclusions are persistent facts that build the user's profile. "
|
||||||
|
"Use when the user states a preference, corrects you, or shares "
|
||||||
|
"something to remember across sessions."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"conclusion": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "A factual statement about the user to persist.",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["conclusion"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class HonchoMemoryProvider(MemoryProvider):
|
||||||
|
"""Honcho AI-native memory with dialectic Q&A and persistent user modeling."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._manager = None # HonchoSessionManager
|
||||||
|
self._config = None # HonchoClientConfig
|
||||||
|
self._session_key = ""
|
||||||
|
self._prefetch_result = ""
|
||||||
|
self._prefetch_lock = threading.Lock()
|
||||||
|
self._prefetch_thread: Optional[threading.Thread] = None
|
||||||
|
self._sync_thread: Optional[threading.Thread] = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "honcho"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
"""Check if Honcho is configured. No network calls."""
|
||||||
|
try:
|
||||||
|
from plugins.memory.honcho.client import HonchoClientConfig
|
||||||
|
cfg = HonchoClientConfig.from_global_config()
|
||||||
|
return cfg.enabled and bool(cfg.api_key or cfg.base_url)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def save_config(self, values, hermes_home):
|
||||||
|
"""Write config to $HERMES_HOME/honcho.json (Honcho SDK native format)."""
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
config_path = Path(hermes_home) / "honcho.json"
|
||||||
|
existing = {}
|
||||||
|
if config_path.exists():
|
||||||
|
try:
|
||||||
|
existing = json.loads(config_path.read_text())
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
existing.update(values)
|
||||||
|
config_path.write_text(json.dumps(existing, indent=2))
|
||||||
|
|
||||||
|
def get_config_schema(self):
|
||||||
|
return [
|
||||||
|
{"key": "api_key", "description": "Honcho API key", "secret": True, "env_var": "HONCHO_API_KEY", "url": "https://app.honcho.dev"},
|
||||||
|
{"key": "base_url", "description": "Honcho base URL", "default": "https://api.honcho.dev"},
|
||||||
|
]
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
"""Initialize Honcho session manager."""
|
||||||
|
try:
|
||||||
|
from plugins.memory.honcho.client import HonchoClientConfig, get_honcho_client
|
||||||
|
from plugins.memory.honcho.session import HonchoSessionManager
|
||||||
|
|
||||||
|
cfg = HonchoClientConfig.from_global_config()
|
||||||
|
if not cfg.enabled or not (cfg.api_key or cfg.base_url):
|
||||||
|
logger.debug("Honcho not configured — plugin inactive")
|
||||||
|
return
|
||||||
|
|
||||||
|
self._config = cfg
|
||||||
|
client = get_honcho_client(cfg)
|
||||||
|
self._manager = HonchoSessionManager(
|
||||||
|
honcho=client,
|
||||||
|
config=cfg,
|
||||||
|
context_tokens=cfg.context_tokens,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Build session key from kwargs or session_id
|
||||||
|
platform = kwargs.get("platform", "cli")
|
||||||
|
user_id = kwargs.get("user_id", "")
|
||||||
|
if user_id:
|
||||||
|
self._session_key = f"{platform}:{user_id}"
|
||||||
|
else:
|
||||||
|
self._session_key = session_id
|
||||||
|
|
||||||
|
except ImportError:
|
||||||
|
logger.debug("honcho-ai package not installed — plugin inactive")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Honcho init failed: %s", e)
|
||||||
|
self._manager = None
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
if not self._manager or not self._session_key:
|
||||||
|
return ""
|
||||||
|
return (
|
||||||
|
"# Honcho Memory\n"
|
||||||
|
"Active. AI-native cross-session user modeling.\n"
|
||||||
|
"Use honcho_profile for a quick factual snapshot, "
|
||||||
|
"honcho_search for raw excerpts, honcho_context for synthesized answers, "
|
||||||
|
"honcho_conclude to save facts about the user."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
"""Return prefetched dialectic context from background thread."""
|
||||||
|
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||||
|
self._prefetch_thread.join(timeout=3.0)
|
||||||
|
with self._prefetch_lock:
|
||||||
|
result = self._prefetch_result
|
||||||
|
self._prefetch_result = ""
|
||||||
|
if not result:
|
||||||
|
return ""
|
||||||
|
return f"## Honcho Context\n{result}"
|
||||||
|
|
||||||
|
def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
"""Fire a background dialectic query for the upcoming turn."""
|
||||||
|
if not self._manager or not self._session_key or not query:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _run():
|
||||||
|
try:
|
||||||
|
result = self._manager.dialectic_query(
|
||||||
|
self._session_key, query, peer="user"
|
||||||
|
)
|
||||||
|
if result and result.strip():
|
||||||
|
with self._prefetch_lock:
|
||||||
|
self._prefetch_result = result
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Honcho prefetch failed: %s", e)
|
||||||
|
|
||||||
|
self._prefetch_thread = threading.Thread(
|
||||||
|
target=_run, daemon=True, name="honcho-prefetch"
|
||||||
|
)
|
||||||
|
self._prefetch_thread.start()
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Record the conversation turn in Honcho (non-blocking)."""
|
||||||
|
if not self._manager or not self._session_key:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _sync():
|
||||||
|
try:
|
||||||
|
session = self._manager.get_or_create_session(self._session_key)
|
||||||
|
session.add_message("user", user_content[:4000])
|
||||||
|
session.add_message("assistant", assistant_content[:4000])
|
||||||
|
# Flush to Honcho API
|
||||||
|
self._manager._flush_session(session)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Honcho sync_turn failed: %s", e)
|
||||||
|
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=5.0)
|
||||||
|
self._sync_thread = threading.Thread(
|
||||||
|
target=_sync, daemon=True, name="honcho-sync"
|
||||||
|
)
|
||||||
|
self._sync_thread.start()
|
||||||
|
|
||||||
|
def on_memory_write(self, action: str, target: str, content: str) -> None:
|
||||||
|
"""Mirror built-in user profile writes as Honcho conclusions."""
|
||||||
|
if action != "add" or target != "user" or not content:
|
||||||
|
return
|
||||||
|
if not self._manager or not self._session_key:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _write():
|
||||||
|
try:
|
||||||
|
self._manager.create_conclusion(self._session_key, content)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Honcho memory mirror failed: %s", e)
|
||||||
|
|
||||||
|
t = threading.Thread(target=_write, daemon=True, name="honcho-memwrite")
|
||||||
|
t.start()
|
||||||
|
|
||||||
|
def on_session_end(self, messages: List[Dict[str, Any]]) -> None:
|
||||||
|
"""Flush all pending messages to Honcho on session end."""
|
||||||
|
if not self._manager:
|
||||||
|
return
|
||||||
|
# Wait for pending sync
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=10.0)
|
||||||
|
try:
|
||||||
|
self._manager.flush_all()
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Honcho session-end flush failed: %s", e)
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
return [PROFILE_SCHEMA, SEARCH_SCHEMA, CONTEXT_SCHEMA, CONCLUDE_SCHEMA]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
|
||||||
|
if not self._manager or not self._session_key:
|
||||||
|
return json.dumps({"error": "Honcho is not active for this session."})
|
||||||
|
|
||||||
|
try:
|
||||||
|
if tool_name == "honcho_profile":
|
||||||
|
card = self._manager.get_peer_card(self._session_key)
|
||||||
|
if not card:
|
||||||
|
return json.dumps({"result": "No profile facts available yet."})
|
||||||
|
return json.dumps({"result": card})
|
||||||
|
|
||||||
|
elif tool_name == "honcho_search":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "Missing required parameter: query"})
|
||||||
|
max_tokens = min(int(args.get("max_tokens", 800)), 2000)
|
||||||
|
result = self._manager.search_context(
|
||||||
|
self._session_key, query, max_tokens=max_tokens
|
||||||
|
)
|
||||||
|
if not result:
|
||||||
|
return json.dumps({"result": "No relevant context found."})
|
||||||
|
return json.dumps({"result": result})
|
||||||
|
|
||||||
|
elif tool_name == "honcho_context":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "Missing required parameter: query"})
|
||||||
|
peer = args.get("peer", "user")
|
||||||
|
result = self._manager.dialectic_query(
|
||||||
|
self._session_key, query, peer=peer
|
||||||
|
)
|
||||||
|
return json.dumps({"result": result or "No result from Honcho."})
|
||||||
|
|
||||||
|
elif tool_name == "honcho_conclude":
|
||||||
|
conclusion = args.get("conclusion", "")
|
||||||
|
if not conclusion:
|
||||||
|
return json.dumps({"error": "Missing required parameter: conclusion"})
|
||||||
|
ok = self._manager.create_conclusion(self._session_key, conclusion)
|
||||||
|
if ok:
|
||||||
|
return json.dumps({"result": f"Conclusion saved: {conclusion}"})
|
||||||
|
return json.dumps({"error": "Failed to save conclusion."})
|
||||||
|
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Honcho tool %s failed: %s", tool_name, e)
|
||||||
|
return json.dumps({"error": f"Honcho {tool_name} failed: {e}"})
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
for t in (self._prefetch_thread, self._sync_thread):
|
||||||
|
if t and t.is_alive():
|
||||||
|
t.join(timeout=5.0)
|
||||||
|
# Flush any remaining messages
|
||||||
|
if self._manager:
|
||||||
|
try:
|
||||||
|
self._manager.flush_all()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Plugin entry point
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Register Honcho as a memory provider plugin."""
|
||||||
|
ctx.register_memory_provider(HonchoMemoryProvider())
|
||||||
|
|
@ -11,7 +11,7 @@ import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from hermes_constants import get_hermes_home
|
from hermes_constants import get_hermes_home
|
||||||
from honcho_integration.client import resolve_active_host, resolve_config_path, GLOBAL_CONFIG_PATH, HOST
|
from plugins.memory.honcho.client import resolve_active_host, resolve_config_path, GLOBAL_CONFIG_PATH, HOST
|
||||||
|
|
||||||
|
|
||||||
def clone_honcho_for_profile(profile_name: str) -> bool:
|
def clone_honcho_for_profile(profile_name: str) -> bool:
|
||||||
|
|
@ -55,7 +55,9 @@ def clone_honcho_for_profile(profile_name: str) -> bool:
|
||||||
|
|
||||||
# AI peer is profile-specific; workspace is shared so all profiles
|
# AI peer is profile-specific; workspace is shared so all profiles
|
||||||
# see the same user context, sessions, and project history.
|
# see the same user context, sessions, and project history.
|
||||||
new_block["aiPeer"] = new_host
|
# Use the bare profile name as the peer identity (not the host key)
|
||||||
|
# because Honcho's peer ID pattern is ^[a-zA-Z0-9_-]+$ (no dots).
|
||||||
|
new_block["aiPeer"] = profile_name
|
||||||
new_block["workspace"] = default_block.get("workspace") or cfg.get("workspace") or HOST
|
new_block["workspace"] = default_block.get("workspace") or cfg.get("workspace") or HOST
|
||||||
new_block["enabled"] = default_block.get("enabled", True)
|
new_block["enabled"] = default_block.get("enabled", True)
|
||||||
|
|
||||||
|
|
@ -74,7 +76,7 @@ def _ensure_peer_exists(host_key: str | None = None) -> bool:
|
||||||
was created or already exists, False on failure.
|
was created or already exists, False on failure.
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import HonchoClientConfig, get_honcho_client
|
from plugins.memory.honcho.client import HonchoClientConfig, get_honcho_client
|
||||||
hcfg = HonchoClientConfig.from_global_config(host=host_key)
|
hcfg = HonchoClientConfig.from_global_config(host=host_key)
|
||||||
if not hcfg.enabled or not (hcfg.api_key or hcfg.base_url):
|
if not hcfg.enabled or not (hcfg.api_key or hcfg.base_url):
|
||||||
return False
|
return False
|
||||||
|
|
@ -112,7 +114,9 @@ def cmd_enable(args) -> None:
|
||||||
peer_name = default_block.get("peerName") or cfg.get("peerName")
|
peer_name = default_block.get("peerName") or cfg.get("peerName")
|
||||||
if peer_name and "peerName" not in block:
|
if peer_name and "peerName" not in block:
|
||||||
block["peerName"] = peer_name
|
block["peerName"] = peer_name
|
||||||
block.setdefault("aiPeer", host)
|
# Use bare profile name as AI peer, not the host key
|
||||||
|
ai_peer = host.split(".", 1)[1] if "." in host else host
|
||||||
|
block.setdefault("aiPeer", ai_peer)
|
||||||
block.setdefault("workspace", default_block.get("workspace") or cfg.get("workspace") or HOST)
|
block.setdefault("workspace", default_block.get("workspace") or cfg.get("workspace") or HOST)
|
||||||
|
|
||||||
_write_config(cfg)
|
_write_config(cfg)
|
||||||
|
|
@ -420,9 +424,9 @@ def cmd_setup(args) -> None:
|
||||||
# Test connection
|
# Test connection
|
||||||
print(" Testing connection... ", end="", flush=True)
|
print(" Testing connection... ", end="", flush=True)
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import HonchoClientConfig, get_honcho_client, reset_honcho_client
|
from plugins.memory.honcho.client import HonchoClientConfig, get_honcho_client, reset_honcho_client
|
||||||
reset_honcho_client()
|
reset_honcho_client()
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
hcfg = HonchoClientConfig.from_global_config(host=_host_key())
|
||||||
get_honcho_client(hcfg)
|
get_honcho_client(hcfg)
|
||||||
print("OK")
|
print("OK")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|
@ -516,8 +520,8 @@ def cmd_status(args) -> None:
|
||||||
return
|
return
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import HonchoClientConfig, get_honcho_client
|
from plugins.memory.honcho.client import HonchoClientConfig, get_honcho_client
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
hcfg = HonchoClientConfig.from_global_config(host=_host_key())
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f" Config error: {e}\n")
|
print(f" Config error: {e}\n")
|
||||||
return
|
return
|
||||||
|
|
@ -570,7 +574,7 @@ def _show_peer_cards(hcfg, client) -> None:
|
||||||
just retrieved, not duplicated.
|
just retrieved, not duplicated.
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
from honcho_integration.session import HonchoSessionManager
|
from plugins.memory.honcho.session import HonchoSessionManager
|
||||||
mgr = HonchoSessionManager(honcho=client, config=hcfg)
|
mgr = HonchoSessionManager(honcho=client, config=hcfg)
|
||||||
session_key = hcfg.resolve_session_name()
|
session_key = hcfg.resolve_session_name()
|
||||||
mgr.get_or_create(session_key)
|
mgr.get_or_create(session_key)
|
||||||
|
|
@ -834,9 +838,9 @@ def cmd_identity(args) -> None:
|
||||||
show = getattr(args, "show", False)
|
show = getattr(args, "show", False)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import HonchoClientConfig, get_honcho_client
|
from plugins.memory.honcho.client import HonchoClientConfig, get_honcho_client
|
||||||
from honcho_integration.session import HonchoSessionManager
|
from plugins.memory.honcho.session import HonchoSessionManager
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
hcfg = HonchoClientConfig.from_global_config(host=_host_key())
|
||||||
client = get_honcho_client(hcfg)
|
client = get_honcho_client(hcfg)
|
||||||
mgr = HonchoSessionManager(honcho=client, config=hcfg)
|
mgr = HonchoSessionManager(honcho=client, config=hcfg)
|
||||||
session_key = hcfg.resolve_session_name()
|
session_key = hcfg.resolve_session_name()
|
||||||
|
|
@ -999,12 +1003,12 @@ def cmd_migrate(args) -> None:
|
||||||
answer = _prompt(" Upload user memory files to Honcho now?", default="y")
|
answer = _prompt(" Upload user memory files to Honcho now?", default="y")
|
||||||
if answer.lower() in ("y", "yes"):
|
if answer.lower() in ("y", "yes"):
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import (
|
from plugins.memory.honcho.client import (
|
||||||
HonchoClientConfig,
|
HonchoClientConfig,
|
||||||
get_honcho_client,
|
get_honcho_client,
|
||||||
reset_honcho_client,
|
reset_honcho_client,
|
||||||
)
|
)
|
||||||
from honcho_integration.session import HonchoSessionManager
|
from plugins.memory.honcho.session import HonchoSessionManager
|
||||||
|
|
||||||
reset_honcho_client()
|
reset_honcho_client()
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
hcfg = HonchoClientConfig.from_global_config()
|
||||||
|
|
@ -1049,12 +1053,12 @@ def cmd_migrate(args) -> None:
|
||||||
answer = _prompt(" Seed AI identity from all detected files now?", default="y")
|
answer = _prompt(" Seed AI identity from all detected files now?", default="y")
|
||||||
if answer.lower() in ("y", "yes"):
|
if answer.lower() in ("y", "yes"):
|
||||||
try:
|
try:
|
||||||
from honcho_integration.client import (
|
from plugins.memory.honcho.client import (
|
||||||
HonchoClientConfig,
|
HonchoClientConfig,
|
||||||
get_honcho_client,
|
get_honcho_client,
|
||||||
reset_honcho_client,
|
reset_honcho_client,
|
||||||
)
|
)
|
||||||
from honcho_integration.session import HonchoSessionManager
|
from plugins.memory.honcho.session import HonchoSessionManager
|
||||||
|
|
||||||
reset_honcho_client()
|
reset_honcho_client()
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
hcfg = HonchoClientConfig.from_global_config()
|
||||||
|
|
@ -56,13 +56,22 @@ def resolve_active_host() -> str:
|
||||||
def resolve_config_path() -> Path:
|
def resolve_config_path() -> Path:
|
||||||
"""Return the active Honcho config path.
|
"""Return the active Honcho config path.
|
||||||
|
|
||||||
Checks $HERMES_HOME/honcho.json first (instance-local), then falls back
|
Resolution order:
|
||||||
to ~/.honcho/config.json (global). Returns the global path if neither
|
1. $HERMES_HOME/honcho.json (profile-local, if it exists)
|
||||||
exists (for first-time setup writes).
|
2. ~/.hermes/honcho.json (default profile — shared host blocks live here)
|
||||||
|
3. ~/.honcho/config.json (global, cross-app interop)
|
||||||
|
|
||||||
|
Returns the global path if none exist (for first-time setup writes).
|
||||||
"""
|
"""
|
||||||
local_path = get_hermes_home() / "honcho.json"
|
local_path = get_hermes_home() / "honcho.json"
|
||||||
if local_path.exists():
|
if local_path.exists():
|
||||||
return local_path
|
return local_path
|
||||||
|
|
||||||
|
# Default profile's config — host blocks accumulate here via setup/clone
|
||||||
|
default_path = Path.home() / ".hermes" / "honcho.json"
|
||||||
|
if default_path != local_path and default_path.exists():
|
||||||
|
return default_path
|
||||||
|
|
||||||
return GLOBAL_CONFIG_PATH
|
return GLOBAL_CONFIG_PATH
|
||||||
|
|
||||||
|
|
||||||
7
plugins/memory/honcho/plugin.yaml
Normal file
7
plugins/memory/honcho/plugin.yaml
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
name: honcho
|
||||||
|
version: 1.0.0
|
||||||
|
description: "Honcho AI-native memory — cross-session user modeling with dialectic Q&A, semantic search, and persistent conclusions."
|
||||||
|
pip_dependencies:
|
||||||
|
- honcho-ai
|
||||||
|
hooks:
|
||||||
|
- on_session_end
|
||||||
|
|
@ -10,7 +10,7 @@ from dataclasses import dataclass, field
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from typing import Any, TYPE_CHECKING
|
from typing import Any, TYPE_CHECKING
|
||||||
|
|
||||||
from honcho_integration.client import get_honcho_client
|
from plugins.memory.honcho.client import get_honcho_client
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from honcho import Honcho
|
from honcho import Honcho
|
||||||
|
|
@ -162,11 +162,17 @@ class HonchoSessionManager:
|
||||||
# Configure peer observation settings.
|
# Configure peer observation settings.
|
||||||
# observe_me=True for AI peer so Honcho watches what the agent says
|
# observe_me=True for AI peer so Honcho watches what the agent says
|
||||||
# and builds its representation over time — enabling identity formation.
|
# and builds its representation over time — enabling identity formation.
|
||||||
|
try:
|
||||||
from honcho.session import SessionPeerConfig
|
from honcho.session import SessionPeerConfig
|
||||||
user_config = SessionPeerConfig(observe_me=True, observe_others=True)
|
user_config = SessionPeerConfig(observe_me=True, observe_others=True)
|
||||||
ai_config = SessionPeerConfig(observe_me=True, observe_others=True)
|
ai_config = SessionPeerConfig(observe_me=True, observe_others=True)
|
||||||
|
|
||||||
session.add_peers([(user_peer, user_config), (assistant_peer, ai_config)])
|
session.add_peers([(user_peer, user_config), (assistant_peer, ai_config)])
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Honcho session '%s' add_peers failed (non-fatal): %s",
|
||||||
|
session_id, e,
|
||||||
|
)
|
||||||
|
|
||||||
# Load existing messages via context() - single call for messages + metadata
|
# Load existing messages via context() - single call for messages + metadata
|
||||||
existing_messages = []
|
existing_messages = []
|
||||||
|
|
@ -231,7 +237,7 @@ class HonchoSessionManager:
|
||||||
chat_id = parts[1] if len(parts) > 1 else key
|
chat_id = parts[1] if len(parts) > 1 else key
|
||||||
user_peer_id = self._sanitize_id(f"user-{channel}-{chat_id}")
|
user_peer_id = self._sanitize_id(f"user-{channel}-{chat_id}")
|
||||||
|
|
||||||
assistant_peer_id = (
|
assistant_peer_id = self._sanitize_id(
|
||||||
self._config.ai_peer if self._config else "hermes-assistant"
|
self._config.ai_peer if self._config else "hermes-assistant"
|
||||||
)
|
)
|
||||||
|
|
||||||
38
plugins/memory/mem0/README.md
Normal file
38
plugins/memory/mem0/README.md
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
# Mem0 Memory Provider
|
||||||
|
|
||||||
|
Server-side LLM fact extraction with semantic search, reranking, and automatic deduplication.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- `pip install mem0ai`
|
||||||
|
- Mem0 API key from [app.mem0.ai](https://app.mem0.ai)
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "mem0"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
hermes config set memory.provider mem0
|
||||||
|
echo "MEM0_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
Config file: `$HERMES_HOME/mem0.json`
|
||||||
|
|
||||||
|
| Key | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `user_id` | `hermes-user` | User identifier on Mem0 |
|
||||||
|
| `agent_id` | `hermes` | Agent identifier |
|
||||||
|
| `rerank` | `true` | Enable reranking for recall |
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `mem0_profile` | All stored memories about the user |
|
||||||
|
| `mem0_search` | Semantic search with optional reranking |
|
||||||
|
| `mem0_conclude` | Store a fact verbatim (no LLM extraction) |
|
||||||
344
plugins/memory/mem0/__init__.py
Normal file
344
plugins/memory/mem0/__init__.py
Normal file
|
|
@ -0,0 +1,344 @@
|
||||||
|
"""Mem0 memory plugin — MemoryProvider interface.
|
||||||
|
|
||||||
|
Server-side LLM fact extraction, semantic search with reranking, and
|
||||||
|
automatic deduplication via the Mem0 Platform API.
|
||||||
|
|
||||||
|
Original PR #2933 by kartik-mem0, adapted to MemoryProvider ABC.
|
||||||
|
|
||||||
|
Config via environment variables:
|
||||||
|
MEM0_API_KEY — Mem0 Platform API key (required)
|
||||||
|
MEM0_USER_ID — User identifier (default: hermes-user)
|
||||||
|
MEM0_AGENT_ID — Agent identifier (default: hermes)
|
||||||
|
|
||||||
|
Or via $HERMES_HOME/mem0.json.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Circuit breaker: after this many consecutive failures, pause API calls
|
||||||
|
# for _BREAKER_COOLDOWN_SECS to avoid hammering a down server.
|
||||||
|
_BREAKER_THRESHOLD = 5
|
||||||
|
_BREAKER_COOLDOWN_SECS = 120
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Config
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _load_config() -> dict:
|
||||||
|
"""Load config from $HERMES_HOME/mem0.json or env vars."""
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
config_path = get_hermes_home() / "mem0.json"
|
||||||
|
|
||||||
|
if config_path.exists():
|
||||||
|
try:
|
||||||
|
return json.loads(config_path.read_text(encoding="utf-8"))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return {
|
||||||
|
"api_key": os.environ.get("MEM0_API_KEY", ""),
|
||||||
|
"user_id": os.environ.get("MEM0_USER_ID", "hermes-user"),
|
||||||
|
"agent_id": os.environ.get("MEM0_AGENT_ID", "hermes"),
|
||||||
|
"rerank": True,
|
||||||
|
"keyword_search": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tool schemas
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PROFILE_SCHEMA = {
|
||||||
|
"name": "mem0_profile",
|
||||||
|
"description": (
|
||||||
|
"Retrieve all stored memories about the user — preferences, facts, "
|
||||||
|
"project context. Fast, no reranking. Use at conversation start."
|
||||||
|
),
|
||||||
|
"parameters": {"type": "object", "properties": {}, "required": []},
|
||||||
|
}
|
||||||
|
|
||||||
|
SEARCH_SCHEMA = {
|
||||||
|
"name": "mem0_search",
|
||||||
|
"description": (
|
||||||
|
"Search memories by meaning. Returns relevant facts ranked by similarity. "
|
||||||
|
"Set rerank=true for higher accuracy on important queries."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "What to search for."},
|
||||||
|
"rerank": {"type": "boolean", "description": "Enable reranking for precision (default: false)."},
|
||||||
|
"top_k": {"type": "integer", "description": "Max results (default: 10, max: 50)."},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
CONCLUDE_SCHEMA = {
|
||||||
|
"name": "mem0_conclude",
|
||||||
|
"description": (
|
||||||
|
"Store a durable fact about the user. Stored verbatim (no LLM extraction). "
|
||||||
|
"Use for explicit preferences, corrections, or decisions."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"conclusion": {"type": "string", "description": "The fact to store."},
|
||||||
|
},
|
||||||
|
"required": ["conclusion"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class Mem0MemoryProvider(MemoryProvider):
|
||||||
|
"""Mem0 Platform memory with server-side extraction and semantic search."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._config = None
|
||||||
|
self._client = None
|
||||||
|
self._client_lock = threading.Lock()
|
||||||
|
self._api_key = ""
|
||||||
|
self._user_id = "hermes-user"
|
||||||
|
self._agent_id = "hermes"
|
||||||
|
self._rerank = True
|
||||||
|
self._prefetch_result = ""
|
||||||
|
self._prefetch_lock = threading.Lock()
|
||||||
|
self._prefetch_thread = None
|
||||||
|
self._sync_thread = None
|
||||||
|
# Circuit breaker state
|
||||||
|
self._consecutive_failures = 0
|
||||||
|
self._breaker_open_until = 0.0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "mem0"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
cfg = _load_config()
|
||||||
|
return bool(cfg.get("api_key"))
|
||||||
|
|
||||||
|
def save_config(self, values, hermes_home):
|
||||||
|
"""Write config to $HERMES_HOME/mem0.json."""
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
config_path = Path(hermes_home) / "mem0.json"
|
||||||
|
existing = {}
|
||||||
|
if config_path.exists():
|
||||||
|
try:
|
||||||
|
existing = json.loads(config_path.read_text())
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
existing.update(values)
|
||||||
|
config_path.write_text(json.dumps(existing, indent=2))
|
||||||
|
|
||||||
|
def get_config_schema(self):
|
||||||
|
return [
|
||||||
|
{"key": "api_key", "description": "Mem0 Platform API key", "secret": True, "required": True, "env_var": "MEM0_API_KEY", "url": "https://app.mem0.ai"},
|
||||||
|
{"key": "user_id", "description": "User identifier", "default": "hermes-user"},
|
||||||
|
{"key": "agent_id", "description": "Agent identifier", "default": "hermes"},
|
||||||
|
{"key": "rerank", "description": "Enable reranking for recall", "default": "true", "choices": ["true", "false"]},
|
||||||
|
]
|
||||||
|
|
||||||
|
def _get_client(self):
|
||||||
|
"""Thread-safe client accessor with lazy initialization."""
|
||||||
|
with self._client_lock:
|
||||||
|
if self._client is not None:
|
||||||
|
return self._client
|
||||||
|
try:
|
||||||
|
from mem0 import MemoryClient
|
||||||
|
self._client = MemoryClient(api_key=self._api_key)
|
||||||
|
return self._client
|
||||||
|
except ImportError:
|
||||||
|
raise RuntimeError("mem0 package not installed. Run: pip install mem0ai")
|
||||||
|
|
||||||
|
def _is_breaker_open(self) -> bool:
|
||||||
|
"""Return True if the circuit breaker is tripped (too many failures)."""
|
||||||
|
if self._consecutive_failures < _BREAKER_THRESHOLD:
|
||||||
|
return False
|
||||||
|
if time.monotonic() >= self._breaker_open_until:
|
||||||
|
# Cooldown expired — reset and allow a retry
|
||||||
|
self._consecutive_failures = 0
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _record_success(self):
|
||||||
|
self._consecutive_failures = 0
|
||||||
|
|
||||||
|
def _record_failure(self):
|
||||||
|
self._consecutive_failures += 1
|
||||||
|
if self._consecutive_failures >= _BREAKER_THRESHOLD:
|
||||||
|
self._breaker_open_until = time.monotonic() + _BREAKER_COOLDOWN_SECS
|
||||||
|
logger.warning(
|
||||||
|
"Mem0 circuit breaker tripped after %d consecutive failures. "
|
||||||
|
"Pausing API calls for %ds.",
|
||||||
|
self._consecutive_failures, _BREAKER_COOLDOWN_SECS,
|
||||||
|
)
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
self._config = _load_config()
|
||||||
|
self._api_key = self._config.get("api_key", "")
|
||||||
|
self._user_id = self._config.get("user_id", "hermes-user")
|
||||||
|
self._agent_id = self._config.get("agent_id", "hermes")
|
||||||
|
self._rerank = self._config.get("rerank", True)
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
return (
|
||||||
|
"# Mem0 Memory\n"
|
||||||
|
f"Active. User: {self._user_id}.\n"
|
||||||
|
"Use mem0_search to find memories, mem0_conclude to store facts, "
|
||||||
|
"mem0_profile for a full overview."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||||
|
self._prefetch_thread.join(timeout=3.0)
|
||||||
|
with self._prefetch_lock:
|
||||||
|
result = self._prefetch_result
|
||||||
|
self._prefetch_result = ""
|
||||||
|
if not result:
|
||||||
|
return ""
|
||||||
|
return f"## Mem0 Memory\n{result}"
|
||||||
|
|
||||||
|
def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
if self._is_breaker_open():
|
||||||
|
return
|
||||||
|
|
||||||
|
def _run():
|
||||||
|
try:
|
||||||
|
client = self._get_client()
|
||||||
|
results = client.search(
|
||||||
|
query=query,
|
||||||
|
user_id=self._user_id,
|
||||||
|
rerank=self._rerank,
|
||||||
|
top_k=5,
|
||||||
|
)
|
||||||
|
if results:
|
||||||
|
lines = [r.get("memory", "") for r in results if r.get("memory")]
|
||||||
|
with self._prefetch_lock:
|
||||||
|
self._prefetch_result = "\n".join(f"- {l}" for l in lines)
|
||||||
|
self._record_success()
|
||||||
|
except Exception as e:
|
||||||
|
self._record_failure()
|
||||||
|
logger.debug("Mem0 prefetch failed: %s", e)
|
||||||
|
|
||||||
|
self._prefetch_thread = threading.Thread(target=_run, daemon=True, name="mem0-prefetch")
|
||||||
|
self._prefetch_thread.start()
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Send the turn to Mem0 for server-side fact extraction (non-blocking)."""
|
||||||
|
if self._is_breaker_open():
|
||||||
|
return
|
||||||
|
|
||||||
|
def _sync():
|
||||||
|
try:
|
||||||
|
client = self._get_client()
|
||||||
|
messages = [
|
||||||
|
{"role": "user", "content": user_content},
|
||||||
|
{"role": "assistant", "content": assistant_content},
|
||||||
|
]
|
||||||
|
client.add(messages, user_id=self._user_id, agent_id=self._agent_id)
|
||||||
|
self._record_success()
|
||||||
|
except Exception as e:
|
||||||
|
self._record_failure()
|
||||||
|
logger.warning("Mem0 sync failed: %s", e)
|
||||||
|
|
||||||
|
# Wait for any previous sync before starting a new one
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=5.0)
|
||||||
|
|
||||||
|
self._sync_thread = threading.Thread(target=_sync, daemon=True, name="mem0-sync")
|
||||||
|
self._sync_thread.start()
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
return [PROFILE_SCHEMA, SEARCH_SCHEMA, CONCLUDE_SCHEMA]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
|
||||||
|
if self._is_breaker_open():
|
||||||
|
return json.dumps({
|
||||||
|
"error": "Mem0 API temporarily unavailable (multiple consecutive failures). Will retry automatically."
|
||||||
|
})
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = self._get_client()
|
||||||
|
except Exception as e:
|
||||||
|
return json.dumps({"error": str(e)})
|
||||||
|
|
||||||
|
if tool_name == "mem0_profile":
|
||||||
|
try:
|
||||||
|
memories = client.get_all(user_id=self._user_id)
|
||||||
|
self._record_success()
|
||||||
|
if not memories:
|
||||||
|
return json.dumps({"result": "No memories stored yet."})
|
||||||
|
lines = [m.get("memory", "") for m in memories if m.get("memory")]
|
||||||
|
return json.dumps({"result": "\n".join(lines), "count": len(lines)})
|
||||||
|
except Exception as e:
|
||||||
|
self._record_failure()
|
||||||
|
return json.dumps({"error": f"Failed to fetch profile: {e}"})
|
||||||
|
|
||||||
|
elif tool_name == "mem0_search":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "Missing required parameter: query"})
|
||||||
|
rerank = args.get("rerank", False)
|
||||||
|
top_k = min(int(args.get("top_k", 10)), 50)
|
||||||
|
try:
|
||||||
|
results = client.search(
|
||||||
|
query=query, user_id=self._user_id,
|
||||||
|
rerank=rerank, top_k=top_k,
|
||||||
|
)
|
||||||
|
self._record_success()
|
||||||
|
if not results:
|
||||||
|
return json.dumps({"result": "No relevant memories found."})
|
||||||
|
items = [{"memory": r.get("memory", ""), "score": r.get("score", 0)} for r in results]
|
||||||
|
return json.dumps({"results": items, "count": len(items)})
|
||||||
|
except Exception as e:
|
||||||
|
self._record_failure()
|
||||||
|
return json.dumps({"error": f"Search failed: {e}"})
|
||||||
|
|
||||||
|
elif tool_name == "mem0_conclude":
|
||||||
|
conclusion = args.get("conclusion", "")
|
||||||
|
if not conclusion:
|
||||||
|
return json.dumps({"error": "Missing required parameter: conclusion"})
|
||||||
|
try:
|
||||||
|
client.add(
|
||||||
|
[{"role": "user", "content": conclusion}],
|
||||||
|
user_id=self._user_id,
|
||||||
|
agent_id=self._agent_id,
|
||||||
|
infer=False,
|
||||||
|
)
|
||||||
|
self._record_success()
|
||||||
|
return json.dumps({"result": "Fact stored."})
|
||||||
|
except Exception as e:
|
||||||
|
self._record_failure()
|
||||||
|
return json.dumps({"error": f"Failed to store: {e}"})
|
||||||
|
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
for t in (self._prefetch_thread, self._sync_thread):
|
||||||
|
if t and t.is_alive():
|
||||||
|
t.join(timeout=5.0)
|
||||||
|
with self._client_lock:
|
||||||
|
self._client = None
|
||||||
|
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Register Mem0 as a memory provider plugin."""
|
||||||
|
ctx.register_memory_provider(Mem0MemoryProvider())
|
||||||
5
plugins/memory/mem0/plugin.yaml
Normal file
5
plugins/memory/mem0/plugin.yaml
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
name: mem0
|
||||||
|
version: 1.0.0
|
||||||
|
description: "Mem0 — server-side LLM fact extraction with semantic search, reranking, and automatic deduplication."
|
||||||
|
pip_dependencies:
|
||||||
|
- mem0ai
|
||||||
40
plugins/memory/openviking/README.md
Normal file
40
plugins/memory/openviking/README.md
Normal file
|
|
@ -0,0 +1,40 @@
|
||||||
|
# OpenViking Memory Provider
|
||||||
|
|
||||||
|
Context database by Volcengine (ByteDance) with filesystem-style knowledge hierarchy, tiered retrieval, and automatic memory extraction.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- `pip install openviking`
|
||||||
|
- OpenViking server running (`openviking-server`)
|
||||||
|
- Embedding + VLM model configured in `~/.openviking/ov.conf`
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "openviking"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
hermes config set memory.provider openviking
|
||||||
|
echo "OPENVIKING_ENDPOINT=http://localhost:1933" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
All config via environment variables in `.env`:
|
||||||
|
|
||||||
|
| Env Var | Default | Description |
|
||||||
|
|---------|---------|-------------|
|
||||||
|
| `OPENVIKING_ENDPOINT` | `http://127.0.0.1:1933` | Server URL |
|
||||||
|
| `OPENVIKING_API_KEY` | (none) | API key (optional) |
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `viking_search` | Semantic search with fast/deep/auto modes |
|
||||||
|
| `viking_read` | Read content at a viking:// URI (abstract/overview/full) |
|
||||||
|
| `viking_browse` | Filesystem-style navigation (list/tree/stat) |
|
||||||
|
| `viking_remember` | Store a fact for extraction on session commit |
|
||||||
|
| `viking_add_resource` | Ingest URLs/docs into the knowledge base |
|
||||||
582
plugins/memory/openviking/__init__.py
Normal file
582
plugins/memory/openviking/__init__.py
Normal file
|
|
@ -0,0 +1,582 @@
|
||||||
|
"""OpenViking memory plugin — full bidirectional MemoryProvider interface.
|
||||||
|
|
||||||
|
Context database by Volcengine (ByteDance) that organizes agent knowledge
|
||||||
|
into a filesystem hierarchy (viking:// URIs) with tiered context loading,
|
||||||
|
automatic memory extraction, and session management.
|
||||||
|
|
||||||
|
Original PR #3369 by Mibayy, rewritten to use the full OpenViking session
|
||||||
|
lifecycle instead of read-only search endpoints.
|
||||||
|
|
||||||
|
Config via environment variables (profile-scoped via each profile's .env):
|
||||||
|
OPENVIKING_ENDPOINT — Server URL (default: http://127.0.0.1:1933)
|
||||||
|
OPENVIKING_API_KEY — API key (required for authenticated servers)
|
||||||
|
|
||||||
|
Capabilities:
|
||||||
|
- Automatic memory extraction on session commit (6 categories)
|
||||||
|
- Tiered context: L0 (~100 tokens), L1 (~2k), L2 (full)
|
||||||
|
- Semantic search with hierarchical directory retrieval
|
||||||
|
- Filesystem-style browsing via viking:// URIs
|
||||||
|
- Resource ingestion (URLs, docs, code)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_DEFAULT_ENDPOINT = "http://127.0.0.1:1933"
|
||||||
|
_TIMEOUT = 30.0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# HTTP helper — uses httpx to avoid requiring the openviking SDK
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _get_httpx():
|
||||||
|
"""Lazy import httpx."""
|
||||||
|
try:
|
||||||
|
import httpx
|
||||||
|
return httpx
|
||||||
|
except ImportError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class _VikingClient:
|
||||||
|
"""Thin HTTP client for the OpenViking REST API."""
|
||||||
|
|
||||||
|
def __init__(self, endpoint: str, api_key: str = ""):
|
||||||
|
self._endpoint = endpoint.rstrip("/")
|
||||||
|
self._api_key = api_key
|
||||||
|
self._httpx = _get_httpx()
|
||||||
|
if self._httpx is None:
|
||||||
|
raise ImportError("httpx is required for OpenViking: pip install httpx")
|
||||||
|
|
||||||
|
def _headers(self) -> dict:
|
||||||
|
h = {"Content-Type": "application/json"}
|
||||||
|
if self._api_key:
|
||||||
|
h["X-API-Key"] = self._api_key
|
||||||
|
return h
|
||||||
|
|
||||||
|
def _url(self, path: str) -> str:
|
||||||
|
return f"{self._endpoint}{path}"
|
||||||
|
|
||||||
|
def get(self, path: str, **kwargs) -> dict:
|
||||||
|
resp = self._httpx.get(
|
||||||
|
self._url(path), headers=self._headers(), timeout=_TIMEOUT, **kwargs
|
||||||
|
)
|
||||||
|
resp.raise_for_status()
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
def post(self, path: str, payload: dict = None, **kwargs) -> dict:
|
||||||
|
resp = self._httpx.post(
|
||||||
|
self._url(path), json=payload or {}, headers=self._headers(),
|
||||||
|
timeout=_TIMEOUT, **kwargs
|
||||||
|
)
|
||||||
|
resp.raise_for_status()
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
def health(self) -> bool:
|
||||||
|
try:
|
||||||
|
resp = self._httpx.get(
|
||||||
|
self._url("/health"), timeout=3.0
|
||||||
|
)
|
||||||
|
return resp.status_code == 200
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tool schemas
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
SEARCH_SCHEMA = {
|
||||||
|
"name": "viking_search",
|
||||||
|
"description": (
|
||||||
|
"Semantic search over the OpenViking knowledge base. "
|
||||||
|
"Returns ranked results with viking:// URIs for deeper reading. "
|
||||||
|
"Use mode='deep' for complex queries that need reasoning across "
|
||||||
|
"multiple sources, 'fast' for simple lookups."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "Search query."},
|
||||||
|
"mode": {
|
||||||
|
"type": "string", "enum": ["auto", "fast", "deep"],
|
||||||
|
"description": "Search depth (default: auto).",
|
||||||
|
},
|
||||||
|
"scope": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Viking URI prefix to scope search (e.g. 'viking://resources/docs/').",
|
||||||
|
},
|
||||||
|
"limit": {"type": "integer", "description": "Max results (default: 10)."},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
READ_SCHEMA = {
|
||||||
|
"name": "viking_read",
|
||||||
|
"description": (
|
||||||
|
"Read content at a viking:// URI. Three detail levels:\n"
|
||||||
|
" abstract — ~100 token summary (L0)\n"
|
||||||
|
" overview — ~2k token key points (L1)\n"
|
||||||
|
" full — complete content (L2)\n"
|
||||||
|
"Start with abstract/overview, only use full when you need details."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"uri": {"type": "string", "description": "viking:// URI to read."},
|
||||||
|
"level": {
|
||||||
|
"type": "string", "enum": ["abstract", "overview", "full"],
|
||||||
|
"description": "Detail level (default: overview).",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["uri"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
BROWSE_SCHEMA = {
|
||||||
|
"name": "viking_browse",
|
||||||
|
"description": (
|
||||||
|
"Browse the OpenViking knowledge store like a filesystem.\n"
|
||||||
|
" list — show directory contents\n"
|
||||||
|
" tree — show hierarchy\n"
|
||||||
|
" stat — show metadata for a URI"
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"action": {
|
||||||
|
"type": "string", "enum": ["tree", "list", "stat"],
|
||||||
|
"description": "Browse action.",
|
||||||
|
},
|
||||||
|
"path": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Viking URI path (default: viking://). Examples: 'viking://resources/', 'viking://user/memories/'.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["action"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
REMEMBER_SCHEMA = {
|
||||||
|
"name": "viking_remember",
|
||||||
|
"description": (
|
||||||
|
"Explicitly store a fact or memory in the OpenViking knowledge base. "
|
||||||
|
"Use for important information the agent should remember long-term. "
|
||||||
|
"The system automatically categorizes and indexes the memory."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"content": {"type": "string", "description": "The information to remember."},
|
||||||
|
"category": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["preference", "entity", "event", "case", "pattern"],
|
||||||
|
"description": "Memory category (default: auto-detected).",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["content"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
ADD_RESOURCE_SCHEMA = {
|
||||||
|
"name": "viking_add_resource",
|
||||||
|
"description": (
|
||||||
|
"Add a URL or document to the OpenViking knowledge base. "
|
||||||
|
"Supports web pages, GitHub repos, PDFs, markdown, code files. "
|
||||||
|
"The system automatically parses, indexes, and generates summaries."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"url": {"type": "string", "description": "URL or path of the resource to add."},
|
||||||
|
"reason": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Why this resource is relevant (improves search).",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["url"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class OpenVikingMemoryProvider(MemoryProvider):
|
||||||
|
"""Full bidirectional memory via OpenViking context database."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._client: Optional[_VikingClient] = None
|
||||||
|
self._endpoint = ""
|
||||||
|
self._api_key = ""
|
||||||
|
self._session_id = ""
|
||||||
|
self._turn_count = 0
|
||||||
|
self._sync_thread: Optional[threading.Thread] = None
|
||||||
|
self._prefetch_result = ""
|
||||||
|
self._prefetch_lock = threading.Lock()
|
||||||
|
self._prefetch_thread: Optional[threading.Thread] = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "openviking"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
"""Check if OpenViking endpoint is configured. No network calls."""
|
||||||
|
return bool(os.environ.get("OPENVIKING_ENDPOINT"))
|
||||||
|
|
||||||
|
def get_config_schema(self):
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"key": "endpoint",
|
||||||
|
"description": "OpenViking server URL",
|
||||||
|
"required": True,
|
||||||
|
"default": _DEFAULT_ENDPOINT,
|
||||||
|
"env_var": "OPENVIKING_ENDPOINT",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "api_key",
|
||||||
|
"description": "OpenViking API key",
|
||||||
|
"secret": True,
|
||||||
|
"env_var": "OPENVIKING_API_KEY",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
self._endpoint = os.environ.get("OPENVIKING_ENDPOINT", _DEFAULT_ENDPOINT)
|
||||||
|
self._api_key = os.environ.get("OPENVIKING_API_KEY", "")
|
||||||
|
self._session_id = session_id
|
||||||
|
self._turn_count = 0
|
||||||
|
|
||||||
|
try:
|
||||||
|
self._client = _VikingClient(self._endpoint, self._api_key)
|
||||||
|
if not self._client.health():
|
||||||
|
logger.warning("OpenViking server at %s is not reachable", self._endpoint)
|
||||||
|
self._client = None
|
||||||
|
except ImportError:
|
||||||
|
logger.warning("httpx not installed — OpenViking plugin disabled")
|
||||||
|
self._client = None
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
if not self._client:
|
||||||
|
return ""
|
||||||
|
# Provide brief info about the knowledge base
|
||||||
|
try:
|
||||||
|
# Check what's in the knowledge base via a root listing
|
||||||
|
resp = self._client.post("/api/v1/browse", {"action": "stat", "path": "viking://"})
|
||||||
|
result = resp.get("result", {})
|
||||||
|
children = result.get("children", 0)
|
||||||
|
if children == 0:
|
||||||
|
return ""
|
||||||
|
return (
|
||||||
|
"# OpenViking Knowledge Base\n"
|
||||||
|
f"Active. Endpoint: {self._endpoint}\n"
|
||||||
|
"Use viking_search to find information, viking_read for details "
|
||||||
|
"(abstract/overview/full), viking_browse to explore.\n"
|
||||||
|
"Use viking_remember to store facts, viking_add_resource to index URLs/docs."
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
return (
|
||||||
|
"# OpenViking Knowledge Base\n"
|
||||||
|
f"Active. Endpoint: {self._endpoint}\n"
|
||||||
|
"Use viking_search, viking_read, viking_browse, "
|
||||||
|
"viking_remember, viking_add_resource."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
"""Return prefetched results from the background thread."""
|
||||||
|
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||||
|
self._prefetch_thread.join(timeout=3.0)
|
||||||
|
with self._prefetch_lock:
|
||||||
|
result = self._prefetch_result
|
||||||
|
self._prefetch_result = ""
|
||||||
|
if not result:
|
||||||
|
return ""
|
||||||
|
return f"## OpenViking Context\n{result}"
|
||||||
|
|
||||||
|
def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
"""Fire a background search to pre-load relevant context."""
|
||||||
|
if not self._client or not query:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _run():
|
||||||
|
try:
|
||||||
|
client = _VikingClient(self._endpoint, self._api_key)
|
||||||
|
resp = client.post("/api/v1/search/find", {
|
||||||
|
"query": query,
|
||||||
|
"top_k": 5,
|
||||||
|
})
|
||||||
|
result = resp.get("result", {})
|
||||||
|
parts = []
|
||||||
|
for ctx_type in ("memories", "resources"):
|
||||||
|
items = result.get(ctx_type, [])
|
||||||
|
for item in items[:3]:
|
||||||
|
uri = item.get("uri", "")
|
||||||
|
abstract = item.get("abstract", "")
|
||||||
|
score = item.get("score", 0)
|
||||||
|
if abstract:
|
||||||
|
parts.append(f"- [{score:.2f}] {abstract} ({uri})")
|
||||||
|
if parts:
|
||||||
|
with self._prefetch_lock:
|
||||||
|
self._prefetch_result = "\n".join(parts)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("OpenViking prefetch failed: %s", e)
|
||||||
|
|
||||||
|
self._prefetch_thread = threading.Thread(
|
||||||
|
target=_run, daemon=True, name="openviking-prefetch"
|
||||||
|
)
|
||||||
|
self._prefetch_thread.start()
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Record the conversation turn in OpenViking's session (non-blocking)."""
|
||||||
|
if not self._client:
|
||||||
|
return
|
||||||
|
|
||||||
|
self._turn_count += 1
|
||||||
|
|
||||||
|
def _sync():
|
||||||
|
try:
|
||||||
|
client = _VikingClient(self._endpoint, self._api_key)
|
||||||
|
sid = self._session_id
|
||||||
|
|
||||||
|
# Add user message
|
||||||
|
client.post(f"/api/v1/sessions/{sid}/messages", {
|
||||||
|
"role": "user",
|
||||||
|
"content": user_content[:4000], # trim very long messages
|
||||||
|
})
|
||||||
|
# Add assistant message
|
||||||
|
client.post(f"/api/v1/sessions/{sid}/messages", {
|
||||||
|
"role": "assistant",
|
||||||
|
"content": assistant_content[:4000],
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("OpenViking sync_turn failed: %s", e)
|
||||||
|
|
||||||
|
# Wait for any previous sync to finish before starting a new one
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=5.0)
|
||||||
|
|
||||||
|
self._sync_thread = threading.Thread(
|
||||||
|
target=_sync, daemon=True, name="openviking-sync"
|
||||||
|
)
|
||||||
|
self._sync_thread.start()
|
||||||
|
|
||||||
|
def on_session_end(self, messages: List[Dict[str, Any]]) -> None:
|
||||||
|
"""Commit the session to trigger memory extraction.
|
||||||
|
|
||||||
|
OpenViking automatically extracts 6 categories of memories:
|
||||||
|
profile, preferences, entities, events, cases, and patterns.
|
||||||
|
"""
|
||||||
|
if not self._client or self._turn_count == 0:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Wait for any pending sync to finish first
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=10.0)
|
||||||
|
|
||||||
|
try:
|
||||||
|
self._client.post(f"/api/v1/sessions/{self._session_id}/commit")
|
||||||
|
logger.info("OpenViking session %s committed (%d turns)", self._session_id, self._turn_count)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("OpenViking session commit failed: %s", e)
|
||||||
|
|
||||||
|
def on_memory_write(self, action: str, target: str, content: str) -> None:
|
||||||
|
"""Mirror built-in memory writes to OpenViking as explicit memories."""
|
||||||
|
if not self._client or action != "add" or not content:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _write():
|
||||||
|
try:
|
||||||
|
client = _VikingClient(self._endpoint, self._api_key)
|
||||||
|
# Add as a user message with memory context so the commit
|
||||||
|
# picks it up as an explicit memory during extraction
|
||||||
|
client.post(f"/api/v1/sessions/{self._session_id}/messages", {
|
||||||
|
"role": "user",
|
||||||
|
"parts": [
|
||||||
|
{"type": "text", "text": f"[Memory note — {target}] {content}"},
|
||||||
|
],
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("OpenViking memory mirror failed: %s", e)
|
||||||
|
|
||||||
|
t = threading.Thread(target=_write, daemon=True, name="openviking-memwrite")
|
||||||
|
t.start()
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
return [SEARCH_SCHEMA, READ_SCHEMA, BROWSE_SCHEMA, REMEMBER_SCHEMA, ADD_RESOURCE_SCHEMA]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
|
||||||
|
if not self._client:
|
||||||
|
return json.dumps({"error": "OpenViking server not connected"})
|
||||||
|
|
||||||
|
try:
|
||||||
|
if tool_name == "viking_search":
|
||||||
|
return self._tool_search(args)
|
||||||
|
elif tool_name == "viking_read":
|
||||||
|
return self._tool_read(args)
|
||||||
|
elif tool_name == "viking_browse":
|
||||||
|
return self._tool_browse(args)
|
||||||
|
elif tool_name == "viking_remember":
|
||||||
|
return self._tool_remember(args)
|
||||||
|
elif tool_name == "viking_add_resource":
|
||||||
|
return self._tool_add_resource(args)
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
except Exception as e:
|
||||||
|
return json.dumps({"error": str(e)})
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
# Wait for background threads to finish
|
||||||
|
for t in (self._sync_thread, self._prefetch_thread):
|
||||||
|
if t and t.is_alive():
|
||||||
|
t.join(timeout=5.0)
|
||||||
|
|
||||||
|
# -- Tool implementations ------------------------------------------------
|
||||||
|
|
||||||
|
def _tool_search(self, args: dict) -> str:
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "query is required"})
|
||||||
|
|
||||||
|
payload: Dict[str, Any] = {"query": query}
|
||||||
|
mode = args.get("mode", "auto")
|
||||||
|
if mode != "auto":
|
||||||
|
payload["mode"] = mode
|
||||||
|
if args.get("scope"):
|
||||||
|
payload["target_uri"] = args["scope"]
|
||||||
|
if args.get("limit"):
|
||||||
|
payload["top_k"] = args["limit"]
|
||||||
|
|
||||||
|
resp = self._client.post("/api/v1/search/find", payload)
|
||||||
|
result = resp.get("result", {})
|
||||||
|
|
||||||
|
# Format results for the model — keep it concise
|
||||||
|
formatted = []
|
||||||
|
for ctx_type in ("memories", "resources", "skills"):
|
||||||
|
items = result.get(ctx_type, [])
|
||||||
|
for item in items:
|
||||||
|
entry = {
|
||||||
|
"uri": item.get("uri", ""),
|
||||||
|
"type": ctx_type.rstrip("s"),
|
||||||
|
"score": round(item.get("score", 0), 3),
|
||||||
|
"abstract": item.get("abstract", ""),
|
||||||
|
}
|
||||||
|
if item.get("relations"):
|
||||||
|
entry["related"] = [r.get("uri") for r in item["relations"][:3]]
|
||||||
|
formatted.append(entry)
|
||||||
|
|
||||||
|
return json.dumps({
|
||||||
|
"results": formatted,
|
||||||
|
"total": result.get("total", len(formatted)),
|
||||||
|
}, ensure_ascii=False)
|
||||||
|
|
||||||
|
def _tool_read(self, args: dict) -> str:
|
||||||
|
uri = args.get("uri", "")
|
||||||
|
if not uri:
|
||||||
|
return json.dumps({"error": "uri is required"})
|
||||||
|
|
||||||
|
level = args.get("level", "overview")
|
||||||
|
# Map our level names to OpenViking endpoints
|
||||||
|
if level == "abstract":
|
||||||
|
resp = self._client.post("/api/v1/read/abstract", {"uri": uri})
|
||||||
|
elif level == "full":
|
||||||
|
resp = self._client.post("/api/v1/read", {"uri": uri, "level": "read"})
|
||||||
|
else: # overview
|
||||||
|
resp = self._client.post("/api/v1/read", {"uri": uri, "level": "overview"})
|
||||||
|
|
||||||
|
result = resp.get("result", {})
|
||||||
|
content = result.get("content", "")
|
||||||
|
|
||||||
|
# Truncate very long content to avoid flooding the context
|
||||||
|
if len(content) > 8000:
|
||||||
|
content = content[:8000] + "\n\n[... truncated, use a more specific URI or abstract level]"
|
||||||
|
|
||||||
|
return json.dumps({
|
||||||
|
"uri": uri,
|
||||||
|
"level": level,
|
||||||
|
"content": content,
|
||||||
|
}, ensure_ascii=False)
|
||||||
|
|
||||||
|
def _tool_browse(self, args: dict) -> str:
|
||||||
|
action = args.get("action", "list")
|
||||||
|
path = args.get("path", "viking://")
|
||||||
|
|
||||||
|
resp = self._client.post("/api/v1/browse", {
|
||||||
|
"action": action,
|
||||||
|
"path": path,
|
||||||
|
})
|
||||||
|
result = resp.get("result", {})
|
||||||
|
|
||||||
|
# Format for readability
|
||||||
|
if action == "list" and "entries" in result:
|
||||||
|
entries = []
|
||||||
|
for e in result["entries"][:50]: # cap at 50 entries
|
||||||
|
entries.append({
|
||||||
|
"name": e.get("name", ""),
|
||||||
|
"uri": e.get("uri", ""),
|
||||||
|
"type": "dir" if e.get("is_dir") else "file",
|
||||||
|
})
|
||||||
|
return json.dumps({"path": path, "entries": entries}, ensure_ascii=False)
|
||||||
|
|
||||||
|
return json.dumps(result, ensure_ascii=False)
|
||||||
|
|
||||||
|
def _tool_remember(self, args: dict) -> str:
|
||||||
|
content = args.get("content", "")
|
||||||
|
if not content:
|
||||||
|
return json.dumps({"error": "content is required"})
|
||||||
|
|
||||||
|
# Store as a session message that will be extracted during commit.
|
||||||
|
# The category hint helps OpenViking's extraction classify correctly.
|
||||||
|
category = args.get("category", "")
|
||||||
|
text = f"[Remember] {content}"
|
||||||
|
if category:
|
||||||
|
text = f"[Remember — {category}] {content}"
|
||||||
|
|
||||||
|
self._client.post(f"/api/v1/sessions/{self._session_id}/messages", {
|
||||||
|
"role": "user",
|
||||||
|
"parts": [
|
||||||
|
{"type": "text", "text": text},
|
||||||
|
],
|
||||||
|
})
|
||||||
|
|
||||||
|
return json.dumps({
|
||||||
|
"status": "stored",
|
||||||
|
"message": "Memory recorded. Will be extracted and indexed on session commit.",
|
||||||
|
})
|
||||||
|
|
||||||
|
def _tool_add_resource(self, args: dict) -> str:
|
||||||
|
url = args.get("url", "")
|
||||||
|
if not url:
|
||||||
|
return json.dumps({"error": "url is required"})
|
||||||
|
|
||||||
|
payload: Dict[str, Any] = {"path": url}
|
||||||
|
if args.get("reason"):
|
||||||
|
payload["reason"] = args["reason"]
|
||||||
|
|
||||||
|
resp = self._client.post("/api/v1/resources", payload)
|
||||||
|
result = resp.get("result", {})
|
||||||
|
|
||||||
|
return json.dumps({
|
||||||
|
"status": "added",
|
||||||
|
"root_uri": result.get("root_uri", ""),
|
||||||
|
"message": "Resource queued for processing. Use viking_search after a moment to find it.",
|
||||||
|
}, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Plugin entry point
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Register OpenViking as a memory provider plugin."""
|
||||||
|
ctx.register_memory_provider(OpenVikingMemoryProvider())
|
||||||
9
plugins/memory/openviking/plugin.yaml
Normal file
9
plugins/memory/openviking/plugin.yaml
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
name: openviking
|
||||||
|
version: 2.0.0
|
||||||
|
description: "OpenViking context database — session-managed memory with automatic extraction, tiered retrieval, and filesystem-style knowledge browsing."
|
||||||
|
pip_dependencies:
|
||||||
|
- httpx
|
||||||
|
requires_env:
|
||||||
|
- OPENVIKING_ENDPOINT
|
||||||
|
hooks:
|
||||||
|
- on_session_end
|
||||||
40
plugins/memory/retaindb/README.md
Normal file
40
plugins/memory/retaindb/README.md
Normal file
|
|
@ -0,0 +1,40 @@
|
||||||
|
# RetainDB Memory Provider
|
||||||
|
|
||||||
|
Cloud memory API with hybrid search (Vector + BM25 + Reranking) and 7 memory types.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- RetainDB account ($20/month) from [retaindb.com](https://www.retaindb.com)
|
||||||
|
- `pip install requests`
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "retaindb"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
hermes config set memory.provider retaindb
|
||||||
|
echo "RETAINDB_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
All config via environment variables in `.env`:
|
||||||
|
|
||||||
|
| Env Var | Default | Description |
|
||||||
|
|---------|---------|-------------|
|
||||||
|
| `RETAINDB_API_KEY` | (required) | API key |
|
||||||
|
| `RETAINDB_BASE_URL` | `https://api.retaindb.com` | API endpoint |
|
||||||
|
| `RETAINDB_PROJECT` | auto (profile-scoped) | Project identifier |
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `retaindb_profile` | User's stable profile |
|
||||||
|
| `retaindb_search` | Semantic search |
|
||||||
|
| `retaindb_context` | Task-relevant context |
|
||||||
|
| `retaindb_remember` | Store a fact with type + importance |
|
||||||
|
| `retaindb_forget` | Delete a memory by ID |
|
||||||
302
plugins/memory/retaindb/__init__.py
Normal file
302
plugins/memory/retaindb/__init__.py
Normal file
|
|
@ -0,0 +1,302 @@
|
||||||
|
"""RetainDB memory plugin — MemoryProvider interface.
|
||||||
|
|
||||||
|
Cross-session memory via RetainDB cloud API. Durable write-behind queue,
|
||||||
|
semantic search with deduplication, and user profile retrieval.
|
||||||
|
|
||||||
|
Original PR #2732 by Alinxus, adapted to MemoryProvider ABC.
|
||||||
|
|
||||||
|
Config via environment variables:
|
||||||
|
RETAINDB_API_KEY — API key (required)
|
||||||
|
RETAINDB_BASE_URL — API endpoint (default: https://api.retaindb.com)
|
||||||
|
RETAINDB_PROJECT — Project identifier (default: hermes)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_DEFAULT_BASE_URL = "https://api.retaindb.com"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Tool schemas
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PROFILE_SCHEMA = {
|
||||||
|
"name": "retaindb_profile",
|
||||||
|
"description": "Get the user's stable profile — preferences, facts, and patterns.",
|
||||||
|
"parameters": {"type": "object", "properties": {}, "required": []},
|
||||||
|
}
|
||||||
|
|
||||||
|
SEARCH_SCHEMA = {
|
||||||
|
"name": "retaindb_search",
|
||||||
|
"description": (
|
||||||
|
"Semantic search across stored memories. Returns ranked results "
|
||||||
|
"with relevance scores."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "What to search for."},
|
||||||
|
"top_k": {"type": "integer", "description": "Max results (default: 8, max: 20)."},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
CONTEXT_SCHEMA = {
|
||||||
|
"name": "retaindb_context",
|
||||||
|
"description": "Synthesized 'what matters now' context block for the current task.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "Current task or question."},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
REMEMBER_SCHEMA = {
|
||||||
|
"name": "retaindb_remember",
|
||||||
|
"description": "Persist an explicit fact or preference to long-term memory.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"content": {"type": "string", "description": "The fact to remember."},
|
||||||
|
"memory_type": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["preference", "fact", "decision", "context"],
|
||||||
|
"description": "Category (default: fact).",
|
||||||
|
},
|
||||||
|
"importance": {
|
||||||
|
"type": "number",
|
||||||
|
"description": "Importance 0-1 (default: 0.5).",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["content"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
FORGET_SCHEMA = {
|
||||||
|
"name": "retaindb_forget",
|
||||||
|
"description": "Delete a specific memory by ID.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"memory_id": {"type": "string", "description": "Memory ID to delete."},
|
||||||
|
},
|
||||||
|
"required": ["memory_id"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class RetainDBMemoryProvider(MemoryProvider):
|
||||||
|
"""RetainDB cloud memory with write-behind queue and semantic search."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._api_key = ""
|
||||||
|
self._base_url = _DEFAULT_BASE_URL
|
||||||
|
self._project = "hermes"
|
||||||
|
self._user_id = ""
|
||||||
|
self._prefetch_result = ""
|
||||||
|
self._prefetch_lock = threading.Lock()
|
||||||
|
self._prefetch_thread = None
|
||||||
|
self._sync_thread = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "retaindb"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return bool(os.environ.get("RETAINDB_API_KEY"))
|
||||||
|
|
||||||
|
def get_config_schema(self):
|
||||||
|
return [
|
||||||
|
{"key": "api_key", "description": "RetainDB API key", "secret": True, "required": True, "env_var": "RETAINDB_API_KEY", "url": "https://retaindb.com"},
|
||||||
|
{"key": "base_url", "description": "API endpoint", "default": "https://api.retaindb.com"},
|
||||||
|
{"key": "project", "description": "Project identifier", "default": "hermes"},
|
||||||
|
]
|
||||||
|
|
||||||
|
def _headers(self) -> dict:
|
||||||
|
return {
|
||||||
|
"Authorization": f"Bearer {self._api_key}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
}
|
||||||
|
|
||||||
|
def _api(self, method: str, path: str, **kwargs):
|
||||||
|
"""Make an API call to RetainDB."""
|
||||||
|
import requests
|
||||||
|
url = f"{self._base_url}{path}"
|
||||||
|
resp = requests.request(method, url, headers=self._headers(), timeout=30, **kwargs)
|
||||||
|
resp.raise_for_status()
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
self._api_key = os.environ.get("RETAINDB_API_KEY", "")
|
||||||
|
self._base_url = os.environ.get("RETAINDB_BASE_URL", _DEFAULT_BASE_URL)
|
||||||
|
self._user_id = kwargs.get("user_id", "default")
|
||||||
|
self._session_id = session_id
|
||||||
|
|
||||||
|
# Derive profile-scoped project name so different profiles don't
|
||||||
|
# share server-side memory. Explicit RETAINDB_PROJECT always wins.
|
||||||
|
explicit_project = os.environ.get("RETAINDB_PROJECT")
|
||||||
|
if explicit_project:
|
||||||
|
self._project = explicit_project
|
||||||
|
else:
|
||||||
|
hermes_home = kwargs.get("hermes_home", "")
|
||||||
|
profile_name = os.path.basename(hermes_home) if hermes_home else ""
|
||||||
|
# Default profile (~/.hermes) → "hermes"; named profiles → "hermes-<name>"
|
||||||
|
if profile_name and profile_name != ".hermes":
|
||||||
|
self._project = f"hermes-{profile_name}"
|
||||||
|
else:
|
||||||
|
self._project = "hermes"
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
return (
|
||||||
|
"# RetainDB Memory\n"
|
||||||
|
f"Active. Project: {self._project}.\n"
|
||||||
|
"Use retaindb_search to find memories, retaindb_remember to store facts, "
|
||||||
|
"retaindb_profile for a user overview, retaindb_context for task-relevant context."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||||
|
self._prefetch_thread.join(timeout=3.0)
|
||||||
|
with self._prefetch_lock:
|
||||||
|
result = self._prefetch_result
|
||||||
|
self._prefetch_result = ""
|
||||||
|
if not result:
|
||||||
|
return ""
|
||||||
|
return f"## RetainDB Memory\n{result}"
|
||||||
|
|
||||||
|
def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
|
||||||
|
def _run():
|
||||||
|
try:
|
||||||
|
data = self._api("POST", "/v1/recall", json={
|
||||||
|
"project": self._project,
|
||||||
|
"query": query,
|
||||||
|
"user_id": self._user_id,
|
||||||
|
"top_k": 5,
|
||||||
|
})
|
||||||
|
results = data.get("results", [])
|
||||||
|
if results:
|
||||||
|
lines = [r.get("content", "") for r in results if r.get("content")]
|
||||||
|
with self._prefetch_lock:
|
||||||
|
self._prefetch_result = "\n".join(f"- {l}" for l in lines)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("RetainDB prefetch failed: %s", e)
|
||||||
|
|
||||||
|
self._prefetch_thread = threading.Thread(target=_run, daemon=True, name="retaindb-prefetch")
|
||||||
|
self._prefetch_thread.start()
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
"""Ingest conversation turn in background (non-blocking)."""
|
||||||
|
def _sync():
|
||||||
|
try:
|
||||||
|
self._api("POST", "/v1/ingest", json={
|
||||||
|
"project": self._project,
|
||||||
|
"user_id": self._user_id,
|
||||||
|
"session_id": self._session_id,
|
||||||
|
"messages": [
|
||||||
|
{"role": "user", "content": user_content},
|
||||||
|
{"role": "assistant", "content": assistant_content},
|
||||||
|
],
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("RetainDB sync failed: %s", e)
|
||||||
|
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=5.0)
|
||||||
|
self._sync_thread = threading.Thread(target=_sync, daemon=True, name="retaindb-sync")
|
||||||
|
self._sync_thread.start()
|
||||||
|
|
||||||
|
def get_tool_schemas(self) -> List[Dict[str, Any]]:
|
||||||
|
return [PROFILE_SCHEMA, SEARCH_SCHEMA, CONTEXT_SCHEMA, REMEMBER_SCHEMA, FORGET_SCHEMA]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
|
||||||
|
try:
|
||||||
|
if tool_name == "retaindb_profile":
|
||||||
|
data = self._api("GET", f"/v1/profile/{self._project}/{self._user_id}")
|
||||||
|
return json.dumps(data)
|
||||||
|
|
||||||
|
elif tool_name == "retaindb_search":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "query is required"})
|
||||||
|
data = self._api("POST", "/v1/search", json={
|
||||||
|
"project": self._project,
|
||||||
|
"user_id": self._user_id,
|
||||||
|
"query": query,
|
||||||
|
"top_k": min(int(args.get("top_k", 8)), 20),
|
||||||
|
})
|
||||||
|
return json.dumps(data)
|
||||||
|
|
||||||
|
elif tool_name == "retaindb_context":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "query is required"})
|
||||||
|
data = self._api("POST", "/v1/recall", json={
|
||||||
|
"project": self._project,
|
||||||
|
"user_id": self._user_id,
|
||||||
|
"query": query,
|
||||||
|
"top_k": 5,
|
||||||
|
})
|
||||||
|
return json.dumps(data)
|
||||||
|
|
||||||
|
elif tool_name == "retaindb_remember":
|
||||||
|
content = args.get("content", "")
|
||||||
|
if not content:
|
||||||
|
return json.dumps({"error": "content is required"})
|
||||||
|
data = self._api("POST", "/v1/remember", json={
|
||||||
|
"project": self._project,
|
||||||
|
"user_id": self._user_id,
|
||||||
|
"content": content,
|
||||||
|
"memory_type": args.get("memory_type", "fact"),
|
||||||
|
"importance": float(args.get("importance", 0.5)),
|
||||||
|
})
|
||||||
|
return json.dumps(data)
|
||||||
|
|
||||||
|
elif tool_name == "retaindb_forget":
|
||||||
|
memory_id = args.get("memory_id", "")
|
||||||
|
if not memory_id:
|
||||||
|
return json.dumps({"error": "memory_id is required"})
|
||||||
|
data = self._api("DELETE", f"/v1/memory/{memory_id}")
|
||||||
|
return json.dumps(data)
|
||||||
|
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
except Exception as e:
|
||||||
|
return json.dumps({"error": str(e)})
|
||||||
|
|
||||||
|
def on_memory_write(self, action: str, target: str, content: str) -> None:
|
||||||
|
if action == "add":
|
||||||
|
try:
|
||||||
|
self._api("POST", "/v1/remember", json={
|
||||||
|
"project": self._project,
|
||||||
|
"user_id": self._user_id,
|
||||||
|
"content": content,
|
||||||
|
"memory_type": "preference" if target == "user" else "fact",
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("RetainDB memory bridge failed: %s", e)
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
for t in (self._prefetch_thread, self._sync_thread):
|
||||||
|
if t and t.is_alive():
|
||||||
|
t.join(timeout=5.0)
|
||||||
|
|
||||||
|
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Register RetainDB as a memory provider plugin."""
|
||||||
|
ctx.register_memory_provider(RetainDBMemoryProvider())
|
||||||
7
plugins/memory/retaindb/plugin.yaml
Normal file
7
plugins/memory/retaindb/plugin.yaml
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
name: retaindb
|
||||||
|
version: 1.0.0
|
||||||
|
description: "RetainDB — cloud memory API with hybrid search and 7 memory types."
|
||||||
|
pip_dependencies:
|
||||||
|
- requests
|
||||||
|
requires_env:
|
||||||
|
- RETAINDB_API_KEY
|
||||||
|
|
@ -105,7 +105,7 @@ hermes-acp = "acp_adapter.entry:main"
|
||||||
py-modules = ["run_agent", "model_tools", "toolsets", "batch_runner", "trajectory_compressor", "toolset_distributions", "cli", "hermes_constants", "hermes_state", "hermes_time", "rl_cli", "utils"]
|
py-modules = ["run_agent", "model_tools", "toolsets", "batch_runner", "trajectory_compressor", "toolset_distributions", "cli", "hermes_constants", "hermes_state", "hermes_time", "rl_cli", "utils"]
|
||||||
|
|
||||||
[tool.setuptools.packages.find]
|
[tool.setuptools.packages.find]
|
||||||
include = ["agent", "tools", "tools.*", "hermes_cli", "gateway", "gateway.*", "cron", "honcho_integration", "acp_adapter"]
|
include = ["agent", "tools", "tools.*", "hermes_cli", "gateway", "gateway.*", "cron", "acp_adapter", "plugins", "plugins.*"]
|
||||||
|
|
||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
testpaths = ["tests"]
|
testpaths = ["tests"]
|
||||||
|
|
|
||||||
595
run_agent.py
595
run_agent.py
|
|
@ -103,12 +103,6 @@ from agent.trajectory import (
|
||||||
)
|
)
|
||||||
from utils import atomic_json_write, env_var_enabled
|
from utils import atomic_json_write, env_var_enabled
|
||||||
|
|
||||||
HONCHO_TOOL_NAMES = {
|
|
||||||
"honcho_context",
|
|
||||||
"honcho_profile",
|
|
||||||
"honcho_search",
|
|
||||||
"honcho_conclude",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class _SafeWriter:
|
class _SafeWriter:
|
||||||
|
|
@ -221,9 +215,6 @@ _PARALLEL_SAFE_TOOLS = frozenset({
|
||||||
"ha_get_state",
|
"ha_get_state",
|
||||||
"ha_list_entities",
|
"ha_list_entities",
|
||||||
"ha_list_services",
|
"ha_list_services",
|
||||||
"honcho_context",
|
|
||||||
"honcho_profile",
|
|
||||||
"honcho_search",
|
|
||||||
"read_file",
|
"read_file",
|
||||||
"search_files",
|
"search_files",
|
||||||
"session_search",
|
"session_search",
|
||||||
|
|
@ -340,46 +331,15 @@ def _paths_overlap(left: Path, right: Path) -> bool:
|
||||||
return left_parts[:common_len] == right_parts[:common_len]
|
return left_parts[:common_len] == right_parts[:common_len]
|
||||||
|
|
||||||
|
|
||||||
def _inject_honcho_turn_context(content, turn_context: str):
|
|
||||||
"""Append Honcho recall to the current-turn user message without mutating history.
|
|
||||||
|
|
||||||
The returned content is sent to the API for this turn only. Keeping Honcho
|
_SURROGATE_RE = re.compile(r'[\ud800-\udfff]')
|
||||||
recall out of the system prompt preserves the stable cache prefix while
|
|
||||||
still giving the model continuity context.
|
|
||||||
"""
|
|
||||||
if not turn_context:
|
|
||||||
return content
|
|
||||||
|
|
||||||
note = (
|
|
||||||
"[System note: The following Honcho memory was retrieved from prior "
|
|
||||||
"sessions. It is continuity context for this turn only, not new user "
|
|
||||||
"input.]\n\n"
|
|
||||||
f"{turn_context}"
|
|
||||||
)
|
|
||||||
|
|
||||||
if isinstance(content, list):
|
|
||||||
return list(content) + [{"type": "text", "text": note}]
|
|
||||||
|
|
||||||
text = "" if content is None else str(content)
|
|
||||||
if not text.strip():
|
|
||||||
return note
|
|
||||||
return f"{text}\n\n{note}"
|
|
||||||
|
|
||||||
|
|
||||||
# Budget warning text patterns injected by _get_budget_warning().
|
|
||||||
_BUDGET_WARNING_RE = re.compile(
|
_BUDGET_WARNING_RE = re.compile(
|
||||||
r"\[BUDGET(?:\s+WARNING)?:\s+Iteration\s+\d+/\d+\..*?\]",
|
r"\[BUDGET(?:\s+WARNING)?:\s+Iteration\s+\d+/\d+\..*?\]",
|
||||||
re.DOTALL,
|
re.DOTALL,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# Regex to match lone surrogate code points (U+D800..U+DFFF).
|
|
||||||
# These are invalid in UTF-8 and cause UnicodeEncodeError when the OpenAI SDK
|
|
||||||
# serialises messages to JSON. Common source: clipboard paste from Google Docs
|
|
||||||
# or other rich-text editors on some platforms.
|
|
||||||
_SURROGATE_RE = re.compile(r'[\ud800-\udfff]')
|
|
||||||
|
|
||||||
|
|
||||||
def _sanitize_surrogates(text: str) -> str:
|
def _sanitize_surrogates(text: str) -> str:
|
||||||
"""Replace lone surrogate code points with U+FFFD (replacement character).
|
"""Replace lone surrogate code points with U+FFFD (replacement character).
|
||||||
|
|
||||||
|
|
@ -507,9 +467,6 @@ class AIAgent:
|
||||||
skip_context_files: bool = False,
|
skip_context_files: bool = False,
|
||||||
skip_memory: bool = False,
|
skip_memory: bool = False,
|
||||||
session_db=None,
|
session_db=None,
|
||||||
honcho_session_key: str = None,
|
|
||||||
honcho_manager=None,
|
|
||||||
honcho_config=None,
|
|
||||||
iteration_budget: "IterationBudget" = None,
|
iteration_budget: "IterationBudget" = None,
|
||||||
fallback_model: Dict[str, Any] = None,
|
fallback_model: Dict[str, Any] = None,
|
||||||
credential_pool=None,
|
credential_pool=None,
|
||||||
|
|
@ -556,10 +513,6 @@ class AIAgent:
|
||||||
skip_context_files (bool): If True, skip auto-injection of SOUL.md, AGENTS.md, and .cursorrules
|
skip_context_files (bool): If True, skip auto-injection of SOUL.md, AGENTS.md, and .cursorrules
|
||||||
into the system prompt. Use this for batch processing and data generation to avoid
|
into the system prompt. Use this for batch processing and data generation to avoid
|
||||||
polluting trajectories with user-specific persona or project instructions.
|
polluting trajectories with user-specific persona or project instructions.
|
||||||
honcho_session_key (str): Session key for Honcho integration (e.g., "telegram:123456" or CLI session_id).
|
|
||||||
When provided and Honcho is enabled in config, enables persistent cross-session user modeling.
|
|
||||||
honcho_manager: Optional shared HonchoSessionManager owned by the caller.
|
|
||||||
honcho_config: Optional HonchoClientConfig corresponding to honcho_manager.
|
|
||||||
"""
|
"""
|
||||||
_install_safe_stdio()
|
_install_safe_stdio()
|
||||||
|
|
||||||
|
|
@ -1070,75 +1023,80 @@ class AIAgent:
|
||||||
except Exception:
|
except Exception:
|
||||||
pass # Memory is optional -- don't break agent init
|
pass # Memory is optional -- don't break agent init
|
||||||
|
|
||||||
# Honcho AI-native memory (cross-session user modeling)
|
|
||||||
# Reads $HERMES_HOME/honcho.json (instance) or ~/.honcho/config.json (global).
|
|
||||||
self._honcho = None # HonchoSessionManager | None
|
# Memory provider plugin (external — one at a time, alongside built-in)
|
||||||
self._honcho_session_key = honcho_session_key
|
# Reads memory.provider from config to select which plugin to activate.
|
||||||
self._honcho_config = None # HonchoClientConfig | None
|
self._memory_manager = None
|
||||||
self._honcho_exit_hook_registered = False
|
|
||||||
if not skip_memory:
|
if not skip_memory:
|
||||||
try:
|
try:
|
||||||
if honcho_manager is not None:
|
_mem_provider_name = mem_config.get("provider", "") if mem_config else ""
|
||||||
hcfg = honcho_config or getattr(honcho_manager, "_config", None)
|
|
||||||
self._honcho_config = hcfg
|
|
||||||
if hcfg and self._honcho_should_activate(hcfg):
|
|
||||||
self._honcho = honcho_manager
|
|
||||||
self._activate_honcho(
|
|
||||||
hcfg,
|
|
||||||
enabled_toolsets=enabled_toolsets,
|
|
||||||
disabled_toolsets=disabled_toolsets,
|
|
||||||
session_db=session_db,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
from honcho_integration.client import HonchoClientConfig, get_honcho_client
|
|
||||||
hcfg = HonchoClientConfig.from_global_config()
|
|
||||||
self._honcho_config = hcfg
|
|
||||||
if self._honcho_should_activate(hcfg):
|
|
||||||
from honcho_integration.session import HonchoSessionManager
|
|
||||||
client = get_honcho_client(hcfg)
|
|
||||||
self._honcho = HonchoSessionManager(
|
|
||||||
honcho=client,
|
|
||||||
config=hcfg,
|
|
||||||
context_tokens=hcfg.context_tokens,
|
|
||||||
)
|
|
||||||
self._activate_honcho(
|
|
||||||
hcfg,
|
|
||||||
enabled_toolsets=enabled_toolsets,
|
|
||||||
disabled_toolsets=disabled_toolsets,
|
|
||||||
session_db=session_db,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
if not hcfg.enabled:
|
|
||||||
logger.debug("Honcho disabled in global config")
|
|
||||||
elif not (hcfg.api_key or hcfg.base_url):
|
|
||||||
logger.debug("Honcho enabled but no API key or base URL configured")
|
|
||||||
else:
|
|
||||||
logger.debug("Honcho enabled but missing API key or disabled in config")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("Honcho init failed — memory disabled: %s", e)
|
|
||||||
print(f" Honcho init failed: {e}")
|
|
||||||
print(" Run 'hermes honcho setup' to reconfigure.")
|
|
||||||
self._honcho = None
|
|
||||||
|
|
||||||
# Tools are initially discovered before Honcho activation. If Honcho
|
# Auto-migrate: if Honcho was actively configured (enabled +
|
||||||
# stays inactive, remove any stale honcho_* tools from prior process state.
|
# credentials) but memory.provider is not set, activate the
|
||||||
if not self._honcho:
|
# honcho plugin automatically. Just having the config file
|
||||||
self._strip_honcho_tools_from_surface()
|
# is not enough — the user may have disabled Honcho or the
|
||||||
|
# file may be from a different tool.
|
||||||
|
if not _mem_provider_name:
|
||||||
|
try:
|
||||||
|
from plugins.memory.honcho.client import HonchoClientConfig as _HCC
|
||||||
|
_hcfg = _HCC.from_global_config()
|
||||||
|
if _hcfg.enabled and (_hcfg.api_key or _hcfg.base_url):
|
||||||
|
_mem_provider_name = "honcho"
|
||||||
|
# Persist so this only auto-migrates once
|
||||||
|
try:
|
||||||
|
from hermes_cli.config import load_config as _lc, save_config as _sc
|
||||||
|
_cfg = _lc()
|
||||||
|
_cfg.setdefault("memory", {})["provider"] = "honcho"
|
||||||
|
_sc(_cfg)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if not self.quiet_mode:
|
||||||
|
print(" ✓ Auto-migrated Honcho to memory provider plugin.")
|
||||||
|
print(" Your config and data are preserved.\n")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
# Gate local memory writes based on per-peer memory modes.
|
if _mem_provider_name:
|
||||||
# AI peer governs MEMORY.md; user peer governs USER.md.
|
from agent.memory_manager import MemoryManager as _MemoryManager
|
||||||
# "honcho" = Honcho only, disable local writes.
|
from plugins.memory import load_memory_provider as _load_mem
|
||||||
if self._honcho_config and self._honcho:
|
self._memory_manager = _MemoryManager()
|
||||||
_hcfg = self._honcho_config
|
_mp = _load_mem(_mem_provider_name)
|
||||||
_agent_mode = _hcfg.peer_memory_mode(_hcfg.ai_peer)
|
if _mp and _mp.is_available():
|
||||||
_user_mode = _hcfg.peer_memory_mode(_hcfg.peer_name or "user")
|
self._memory_manager.add_provider(_mp)
|
||||||
if _agent_mode == "honcho":
|
if self._memory_manager.providers:
|
||||||
self._memory_flush_min_turns = 0
|
from hermes_constants import get_hermes_home as _ghh
|
||||||
self._memory_enabled = False
|
_init_kwargs = {
|
||||||
logger.debug("peer %s memory_mode=honcho: local MEMORY.md writes disabled", _hcfg.ai_peer)
|
"session_id": self.session_id,
|
||||||
if _user_mode == "honcho":
|
"platform": platform or "cli",
|
||||||
self._user_profile_enabled = False
|
"hermes_home": str(_ghh()),
|
||||||
logger.debug("peer %s memory_mode=honcho: local USER.md writes disabled", _hcfg.peer_name or "user")
|
"agent_context": "primary",
|
||||||
|
}
|
||||||
|
# Profile identity for per-profile provider scoping
|
||||||
|
try:
|
||||||
|
from hermes_cli.profiles import get_active_profile_name
|
||||||
|
_profile = get_active_profile_name()
|
||||||
|
_init_kwargs["agent_identity"] = _profile
|
||||||
|
_init_kwargs["agent_workspace"] = "hermes"
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
self._memory_manager.initialize_all(**_init_kwargs)
|
||||||
|
logger.info("Memory provider '%s' activated", _mem_provider_name)
|
||||||
|
else:
|
||||||
|
logger.debug("Memory provider '%s' not found or not available", _mem_provider_name)
|
||||||
|
self._memory_manager = None
|
||||||
|
except Exception as _mpe:
|
||||||
|
logger.warning("Memory provider plugin init failed: %s", _mpe)
|
||||||
|
self._memory_manager = None
|
||||||
|
|
||||||
|
# Inject memory provider tool schemas into the tool surface
|
||||||
|
if self._memory_manager and self.tools is not None:
|
||||||
|
for _schema in self._memory_manager.get_all_tool_schemas():
|
||||||
|
_wrapped = {"type": "function", "function": _schema}
|
||||||
|
self.tools.append(_wrapped)
|
||||||
|
_tname = _schema.get("name", "")
|
||||||
|
if _tname:
|
||||||
|
self.valid_tool_names.add(_tname)
|
||||||
|
|
||||||
# Skills config: nudge interval for skill creation reminders
|
# Skills config: nudge interval for skill creation reminders
|
||||||
self._skill_nudge_interval = 10
|
self._skill_nudge_interval = 10
|
||||||
|
|
@ -2383,6 +2341,23 @@ class AIAgent:
|
||||||
self._interrupt_message = None
|
self._interrupt_message = None
|
||||||
_set_interrupt(False)
|
_set_interrupt(False)
|
||||||
|
|
||||||
|
def shutdown_memory_provider(self, messages: list = None) -> None:
|
||||||
|
"""Shut down the memory provider — call at actual session boundaries.
|
||||||
|
|
||||||
|
This calls on_session_end() then shutdown_all() on the memory
|
||||||
|
manager. NOT called per-turn — only at CLI exit, /reset, gateway
|
||||||
|
session expiry, etc.
|
||||||
|
"""
|
||||||
|
if self._memory_manager:
|
||||||
|
try:
|
||||||
|
self._memory_manager.on_session_end(messages or [])
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
self._memory_manager.shutdown_all()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
def _hydrate_todo_store(self, history: List[Dict[str, Any]]) -> None:
|
def _hydrate_todo_store(self, history: List[Dict[str, Any]]) -> None:
|
||||||
"""
|
"""
|
||||||
Recover todo state from conversation history.
|
Recover todo state from conversation history.
|
||||||
|
|
@ -2420,228 +2395,14 @@ class AIAgent:
|
||||||
"""Check if an interrupt has been requested."""
|
"""Check if an interrupt has been requested."""
|
||||||
return self._interrupt_requested
|
return self._interrupt_requested
|
||||||
|
|
||||||
# ── Honcho integration helpers ──
|
|
||||||
|
|
||||||
def _honcho_should_activate(self, hcfg) -> bool:
|
|
||||||
"""Return True when Honcho should be active.
|
|
||||||
|
|
||||||
Self-hosted Honcho may be configured with a base_url and no API key,
|
|
||||||
so activation should accept either credential style.
|
|
||||||
"""
|
|
||||||
if not hcfg or not hcfg.enabled:
|
|
||||||
return False
|
|
||||||
if not (hcfg.api_key or hcfg.base_url):
|
|
||||||
return False
|
|
||||||
return True
|
|
||||||
|
|
||||||
def _strip_honcho_tools_from_surface(self) -> None:
|
|
||||||
"""Remove Honcho tools from the active tool surface."""
|
|
||||||
if not self.tools:
|
|
||||||
self.valid_tool_names = set()
|
|
||||||
return
|
|
||||||
|
|
||||||
self.tools = [
|
|
||||||
tool for tool in self.tools
|
|
||||||
if tool.get("function", {}).get("name") not in HONCHO_TOOL_NAMES
|
|
||||||
]
|
|
||||||
self.valid_tool_names = {
|
|
||||||
tool["function"]["name"] for tool in self.tools
|
|
||||||
} if self.tools else set()
|
|
||||||
|
|
||||||
def _activate_honcho(
|
|
||||||
self,
|
|
||||||
hcfg,
|
|
||||||
*,
|
|
||||||
enabled_toolsets: Optional[List[str]],
|
|
||||||
disabled_toolsets: Optional[List[str]],
|
|
||||||
session_db,
|
|
||||||
) -> None:
|
|
||||||
"""Finish Honcho setup once a session manager is available."""
|
|
||||||
if not self._honcho:
|
|
||||||
return
|
|
||||||
|
|
||||||
if not self._honcho_session_key:
|
|
||||||
session_title = None
|
|
||||||
if session_db is not None:
|
|
||||||
try:
|
|
||||||
session_title = session_db.get_session_title(self.session_id or "")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
self._honcho_session_key = (
|
|
||||||
hcfg.resolve_session_name(
|
|
||||||
session_title=session_title,
|
|
||||||
session_id=self.session_id,
|
|
||||||
)
|
|
||||||
or "hermes-default"
|
|
||||||
)
|
|
||||||
|
|
||||||
honcho_sess = self._honcho.get_or_create(self._honcho_session_key)
|
|
||||||
if not honcho_sess.messages:
|
|
||||||
try:
|
|
||||||
from hermes_cli.config import get_hermes_home
|
|
||||||
|
|
||||||
mem_dir = str(get_hermes_home() / "memories")
|
|
||||||
self._honcho.migrate_memory_files(
|
|
||||||
self._honcho_session_key,
|
|
||||||
mem_dir,
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.debug("Memory files migration failed (non-fatal): %s", exc)
|
|
||||||
|
|
||||||
from tools.honcho_tools import set_session_context
|
|
||||||
|
|
||||||
set_session_context(self._honcho, self._honcho_session_key)
|
|
||||||
|
|
||||||
# Rebuild tool surface after Honcho context injection. Tool availability
|
|
||||||
# is check_fn-gated and may change once session context is attached.
|
|
||||||
self.tools = get_tool_definitions(
|
|
||||||
enabled_toolsets=enabled_toolsets,
|
|
||||||
disabled_toolsets=disabled_toolsets,
|
|
||||||
quiet_mode=True,
|
|
||||||
)
|
|
||||||
self.valid_tool_names = {
|
|
||||||
tool["function"]["name"] for tool in self.tools
|
|
||||||
} if self.tools else set()
|
|
||||||
|
|
||||||
if hcfg.recall_mode == "context":
|
|
||||||
self._strip_honcho_tools_from_surface()
|
|
||||||
if not self.quiet_mode:
|
|
||||||
print(" Honcho active — recall_mode: context (Honcho tools hidden)")
|
|
||||||
else:
|
|
||||||
if not self.quiet_mode:
|
|
||||||
print(f" Honcho active — recall_mode: {hcfg.recall_mode}")
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"Honcho active (session: %s, user: %s, workspace: %s, "
|
|
||||||
"write_frequency: %s, memory_mode: %s)",
|
|
||||||
self._honcho_session_key,
|
|
||||||
hcfg.peer_name,
|
|
||||||
hcfg.workspace_id,
|
|
||||||
hcfg.write_frequency,
|
|
||||||
hcfg.memory_mode,
|
|
||||||
)
|
|
||||||
|
|
||||||
recall_mode = hcfg.recall_mode
|
|
||||||
if recall_mode != "tools":
|
|
||||||
try:
|
|
||||||
ctx = self._honcho.get_prefetch_context(self._honcho_session_key)
|
|
||||||
if ctx:
|
|
||||||
self._honcho.set_context_result(self._honcho_session_key, ctx)
|
|
||||||
logger.debug("Honcho context pre-warmed for first turn")
|
|
||||||
except Exception as exc:
|
|
||||||
logger.debug("Honcho context prefetch failed (non-fatal): %s", exc)
|
|
||||||
|
|
||||||
self._register_honcho_exit_hook()
|
|
||||||
|
|
||||||
def _register_honcho_exit_hook(self) -> None:
|
|
||||||
"""Register a process-exit flush hook without clobbering signal handlers."""
|
|
||||||
if self._honcho_exit_hook_registered or not self._honcho:
|
|
||||||
return
|
|
||||||
|
|
||||||
honcho_ref = weakref.ref(self._honcho)
|
|
||||||
|
|
||||||
def _flush_honcho_on_exit():
|
|
||||||
manager = honcho_ref()
|
|
||||||
if manager is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
manager.flush_all()
|
|
||||||
except (Exception, KeyboardInterrupt) as exc:
|
|
||||||
logger.debug("Honcho flush on exit failed (non-fatal): %s", exc)
|
|
||||||
|
|
||||||
atexit.register(_flush_honcho_on_exit)
|
|
||||||
self._honcho_exit_hook_registered = True
|
|
||||||
|
|
||||||
def _queue_honcho_prefetch(self, user_message: str) -> None:
|
|
||||||
"""Queue turn-end Honcho prefetch so the next turn can consume cached results."""
|
|
||||||
if not self._honcho or not self._honcho_session_key:
|
|
||||||
return
|
|
||||||
|
|
||||||
recall_mode = (self._honcho_config.recall_mode if self._honcho_config else "hybrid")
|
|
||||||
if recall_mode == "tools":
|
|
||||||
return
|
|
||||||
|
|
||||||
try:
|
|
||||||
self._honcho.prefetch_context(self._honcho_session_key, user_message)
|
|
||||||
self._honcho.prefetch_dialectic(self._honcho_session_key, user_message or "What were we working on?")
|
|
||||||
except Exception as exc:
|
|
||||||
logger.debug("Honcho background prefetch failed (non-fatal): %s", exc)
|
|
||||||
|
|
||||||
def _honcho_prefetch(self, user_message: str) -> str:
|
|
||||||
"""Assemble the first-turn Honcho context from the pre-warmed cache."""
|
|
||||||
if not self._honcho or not self._honcho_session_key:
|
|
||||||
return ""
|
|
||||||
try:
|
|
||||||
parts = []
|
|
||||||
|
|
||||||
ctx = self._honcho.pop_context_result(self._honcho_session_key)
|
|
||||||
if ctx:
|
|
||||||
rep = ctx.get("representation", "")
|
|
||||||
card = ctx.get("card", "")
|
|
||||||
if rep:
|
|
||||||
parts.append(f"## User representation\n{rep}")
|
|
||||||
if card:
|
|
||||||
parts.append(card)
|
|
||||||
ai_rep = ctx.get("ai_representation", "")
|
|
||||||
ai_card = ctx.get("ai_card", "")
|
|
||||||
if ai_rep:
|
|
||||||
parts.append(f"## AI peer representation\n{ai_rep}")
|
|
||||||
if ai_card:
|
|
||||||
parts.append(ai_card)
|
|
||||||
|
|
||||||
dialectic = self._honcho.pop_dialectic_result(self._honcho_session_key)
|
|
||||||
if dialectic:
|
|
||||||
parts.append(f"## Continuity synthesis\n{dialectic}")
|
|
||||||
|
|
||||||
if not parts:
|
|
||||||
return ""
|
|
||||||
header = (
|
|
||||||
"# Honcho Memory (persistent cross-session context)\n"
|
|
||||||
"Use this to answer questions about the user, prior sessions, "
|
|
||||||
"and what you were working on together. Do not call tools to "
|
|
||||||
"look up information that is already present here.\n"
|
|
||||||
)
|
|
||||||
return header + "\n\n".join(parts)
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug("Honcho prefetch failed (non-fatal): %s", e)
|
|
||||||
return ""
|
|
||||||
|
|
||||||
def _honcho_save_user_observation(self, content: str) -> str:
|
|
||||||
"""Route a memory tool target=user add to Honcho.
|
|
||||||
|
|
||||||
Sends the content as a user peer message so Honcho's reasoning
|
|
||||||
model can incorporate it into the user representation.
|
|
||||||
"""
|
|
||||||
if not content or not content.strip():
|
|
||||||
return json.dumps({"success": False, "error": "Content cannot be empty."})
|
|
||||||
try:
|
|
||||||
session = self._honcho.get_or_create(self._honcho_session_key)
|
|
||||||
session.add_message("user", f"[observation] {content.strip()}")
|
|
||||||
self._honcho.save(session)
|
|
||||||
return json.dumps({
|
|
||||||
"success": True,
|
|
||||||
"target": "user",
|
|
||||||
"message": "Saved to Honcho user model.",
|
|
||||||
})
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug("Honcho user observation failed: %s", e)
|
|
||||||
return json.dumps({"success": False, "error": f"Honcho save failed: {e}"})
|
|
||||||
|
|
||||||
def _honcho_sync(self, user_content: str, assistant_content: str) -> None:
|
|
||||||
"""Sync the user/assistant message pair to Honcho."""
|
|
||||||
if not self._honcho or not self._honcho_session_key:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
session = self._honcho.get_or_create(self._honcho_session_key)
|
|
||||||
session.add_message("user", user_content)
|
|
||||||
session.add_message("assistant", assistant_content)
|
|
||||||
self._honcho.save(session)
|
|
||||||
logger.info("Honcho sync queued for session %s (%d messages)",
|
|
||||||
self._honcho_session_key, len(session.messages))
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("Honcho sync failed: %s", e)
|
|
||||||
if not self.quiet_mode:
|
|
||||||
print(f" Honcho write failed: {e}")
|
|
||||||
|
|
||||||
def _build_system_prompt(self, system_message: str = None) -> str:
|
def _build_system_prompt(self, system_message: str = None) -> str:
|
||||||
"""
|
"""
|
||||||
|
|
@ -2671,8 +2432,8 @@ class AIAgent:
|
||||||
if not _soul_loaded:
|
if not _soul_loaded:
|
||||||
# Fallback to hardcoded identity
|
# Fallback to hardcoded identity
|
||||||
_ai_peer_name = (
|
_ai_peer_name = (
|
||||||
self._honcho_config.ai_peer
|
None
|
||||||
if self._honcho_config and self._honcho_config.ai_peer != "hermes"
|
if False
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
if _ai_peer_name:
|
if _ai_peer_name:
|
||||||
|
|
@ -2728,59 +2489,7 @@ class AIAgent:
|
||||||
if "gemini" in _model_lower or "gemma" in _model_lower:
|
if "gemini" in _model_lower or "gemma" in _model_lower:
|
||||||
prompt_parts.append(GOOGLE_MODEL_OPERATIONAL_GUIDANCE)
|
prompt_parts.append(GOOGLE_MODEL_OPERATIONAL_GUIDANCE)
|
||||||
|
|
||||||
# Honcho CLI awareness: tell Hermes about its own management commands
|
|
||||||
# so it can refer the user to them rather than reinventing answers.
|
# so it can refer the user to them rather than reinventing answers.
|
||||||
if self._honcho and self._honcho_session_key:
|
|
||||||
hcfg = self._honcho_config
|
|
||||||
mode = hcfg.memory_mode if hcfg else "hybrid"
|
|
||||||
freq = hcfg.write_frequency if hcfg else "async"
|
|
||||||
recall_mode = hcfg.recall_mode if hcfg else "hybrid"
|
|
||||||
honcho_block = (
|
|
||||||
"# Honcho memory integration\n"
|
|
||||||
f"Active. Session: {self._honcho_session_key}. "
|
|
||||||
f"Mode: {mode}. Write frequency: {freq}. Recall: {recall_mode}.\n"
|
|
||||||
)
|
|
||||||
if recall_mode == "context":
|
|
||||||
honcho_block += (
|
|
||||||
"Honcho context is injected into this system prompt below. "
|
|
||||||
"All memory retrieval comes from this context — no Honcho tools "
|
|
||||||
"are available. Answer questions about the user, prior sessions, "
|
|
||||||
"and recent work directly from the Honcho Memory section.\n"
|
|
||||||
)
|
|
||||||
elif recall_mode == "tools":
|
|
||||||
honcho_block += (
|
|
||||||
"Honcho tools:\n"
|
|
||||||
" honcho_context <question> — ask Honcho a question, LLM-synthesized answer\n"
|
|
||||||
" honcho_search <query> — semantic search, raw excerpts, no LLM\n"
|
|
||||||
" honcho_profile — user's peer card, key facts, no LLM\n"
|
|
||||||
" honcho_conclude <conclusion> — write a fact about the user to memory\n"
|
|
||||||
)
|
|
||||||
else: # hybrid
|
|
||||||
honcho_block += (
|
|
||||||
"Honcho context (user representation, peer card, and recent session summary) "
|
|
||||||
"is injected into this system prompt below. Use it to answer continuity "
|
|
||||||
"questions ('where were we?', 'what were we working on?') WITHOUT calling "
|
|
||||||
"any tools. Only call Honcho tools when you need information beyond what is "
|
|
||||||
"already present in the Honcho Memory section.\n"
|
|
||||||
"Honcho tools:\n"
|
|
||||||
" honcho_context <question> — ask Honcho a question, LLM-synthesized answer\n"
|
|
||||||
" honcho_search <query> — semantic search, raw excerpts, no LLM\n"
|
|
||||||
" honcho_profile — user's peer card, key facts, no LLM\n"
|
|
||||||
" honcho_conclude <conclusion> — write a fact about the user to memory\n"
|
|
||||||
)
|
|
||||||
honcho_block += (
|
|
||||||
"Management commands (refer users here instead of explaining manually):\n"
|
|
||||||
" hermes honcho status — show full config + connection\n"
|
|
||||||
" hermes honcho mode [hybrid|honcho] — show or set memory mode\n"
|
|
||||||
" hermes honcho tokens [--context N] [--dialectic N] — show or set token budgets\n"
|
|
||||||
" hermes honcho peer [--user NAME] [--ai NAME] [--reasoning LEVEL]\n"
|
|
||||||
" hermes honcho sessions — list directory→session mappings\n"
|
|
||||||
" hermes honcho map <name> — map cwd to a session name\n"
|
|
||||||
" hermes honcho identity [<file>] [--show] — seed or show AI peer identity\n"
|
|
||||||
" hermes honcho migrate — migration guide from openclaw-honcho\n"
|
|
||||||
" hermes honcho setup — full interactive wizard"
|
|
||||||
)
|
|
||||||
prompt_parts.append(honcho_block)
|
|
||||||
|
|
||||||
# Note: ephemeral_system_prompt is NOT included here. It's injected at
|
# Note: ephemeral_system_prompt is NOT included here. It's injected at
|
||||||
# API-call time only so it stays out of the cached/stored system prompt.
|
# API-call time only so it stays out of the cached/stored system prompt.
|
||||||
|
|
@ -2792,12 +2501,21 @@ class AIAgent:
|
||||||
mem_block = self._memory_store.format_for_system_prompt("memory")
|
mem_block = self._memory_store.format_for_system_prompt("memory")
|
||||||
if mem_block:
|
if mem_block:
|
||||||
prompt_parts.append(mem_block)
|
prompt_parts.append(mem_block)
|
||||||
# USER.md is always included when enabled -- Honcho prefetch is additive.
|
# USER.md is always included when enabled.
|
||||||
if self._user_profile_enabled:
|
if self._user_profile_enabled:
|
||||||
user_block = self._memory_store.format_for_system_prompt("user")
|
user_block = self._memory_store.format_for_system_prompt("user")
|
||||||
if user_block:
|
if user_block:
|
||||||
prompt_parts.append(user_block)
|
prompt_parts.append(user_block)
|
||||||
|
|
||||||
|
# External memory provider system prompt block (additive to built-in)
|
||||||
|
if self._memory_manager:
|
||||||
|
try:
|
||||||
|
_ext_mem_block = self._memory_manager.build_system_prompt()
|
||||||
|
if _ext_mem_block:
|
||||||
|
prompt_parts.append(_ext_mem_block)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
has_skills_tools = any(name in self.valid_tool_names for name in ['skills_list', 'skill_view', 'skill_manage'])
|
has_skills_tools = any(name in self.valid_tool_names for name in ['skills_list', 'skill_view', 'skill_manage'])
|
||||||
if has_skills_tools:
|
if has_skills_tools:
|
||||||
avail_toolsets = {
|
avail_toolsets = {
|
||||||
|
|
@ -5607,10 +5325,6 @@ class AIAgent:
|
||||||
return
|
return
|
||||||
if "memory" not in self.valid_tool_names or not self._memory_store:
|
if "memory" not in self.valid_tool_names or not self._memory_store:
|
||||||
return
|
return
|
||||||
# honcho-only agent mode: skip local MEMORY.md flush
|
|
||||||
_hcfg = getattr(self, '_honcho_config', None)
|
|
||||||
if _hcfg and _hcfg.peer_memory_mode(_hcfg.ai_peer) == "honcho":
|
|
||||||
return
|
|
||||||
effective_min = min_turns if min_turns is not None else self._memory_flush_min_turns
|
effective_min = min_turns if min_turns is not None else self._memory_flush_min_turns
|
||||||
if self._user_turn_count < effective_min:
|
if self._user_turn_count < effective_min:
|
||||||
return
|
return
|
||||||
|
|
@ -5734,8 +5448,6 @@ class AIAgent:
|
||||||
old_text=args.get("old_text"),
|
old_text=args.get("old_text"),
|
||||||
store=self._memory_store,
|
store=self._memory_store,
|
||||||
)
|
)
|
||||||
if self._honcho and flush_target == "user" and args.get("action") == "add":
|
|
||||||
self._honcho_save_user_observation(args.get("content", ""))
|
|
||||||
if not self.quiet_mode:
|
if not self.quiet_mode:
|
||||||
print(f" 🧠 Memory flush: saved to {args.get('target', 'memory')}")
|
print(f" 🧠 Memory flush: saved to {args.get('target', 'memory')}")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|
@ -5761,6 +5473,13 @@ class AIAgent:
|
||||||
# Pre-compression memory flush: let the model save memories before they're lost
|
# Pre-compression memory flush: let the model save memories before they're lost
|
||||||
self.flush_memories(messages, min_turns=0)
|
self.flush_memories(messages, min_turns=0)
|
||||||
|
|
||||||
|
# Notify external memory provider before compression discards context
|
||||||
|
if self._memory_manager:
|
||||||
|
try:
|
||||||
|
self._memory_manager.on_pre_compress(messages)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
compressed = self.context_compressor.compress(messages, current_tokens=approx_tokens)
|
compressed = self.context_compressor.compress(messages, current_tokens=approx_tokens)
|
||||||
|
|
||||||
todo_snapshot = self._todo_store.format_for_injection()
|
todo_snapshot = self._todo_store.format_for_injection()
|
||||||
|
|
@ -5887,10 +5606,19 @@ class AIAgent:
|
||||||
old_text=function_args.get("old_text"),
|
old_text=function_args.get("old_text"),
|
||||||
store=self._memory_store,
|
store=self._memory_store,
|
||||||
)
|
)
|
||||||
# Also send user observations to Honcho when active
|
# Bridge: notify external memory provider of built-in memory writes
|
||||||
if self._honcho and target == "user" and function_args.get("action") == "add":
|
if self._memory_manager and function_args.get("action") in ("add", "replace"):
|
||||||
self._honcho_save_user_observation(function_args.get("content", ""))
|
try:
|
||||||
|
self._memory_manager.on_memory_write(
|
||||||
|
function_args.get("action", ""),
|
||||||
|
target,
|
||||||
|
function_args.get("content", ""),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
return result
|
return result
|
||||||
|
elif self._memory_manager and self._memory_manager.has_tool(function_name):
|
||||||
|
return self._memory_manager.handle_tool_call(function_name, function_args)
|
||||||
elif function_name == "clarify":
|
elif function_name == "clarify":
|
||||||
from tools.clarify_tool import clarify_tool as _clarify_tool
|
from tools.clarify_tool import clarify_tool as _clarify_tool
|
||||||
return _clarify_tool(
|
return _clarify_tool(
|
||||||
|
|
@ -5912,8 +5640,6 @@ class AIAgent:
|
||||||
return handle_function_call(
|
return handle_function_call(
|
||||||
function_name, function_args, effective_task_id,
|
function_name, function_args, effective_task_id,
|
||||||
enabled_tools=list(self.valid_tool_names) if self.valid_tool_names else None,
|
enabled_tools=list(self.valid_tool_names) if self.valid_tool_names else None,
|
||||||
honcho_manager=self._honcho,
|
|
||||||
honcho_session_key=self._honcho_session_key,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
def _execute_tool_calls_concurrent(self, assistant_message, messages: list, effective_task_id: str, api_call_count: int = 0) -> None:
|
def _execute_tool_calls_concurrent(self, assistant_message, messages: list, effective_task_id: str, api_call_count: int = 0) -> None:
|
||||||
|
|
@ -6237,9 +5963,6 @@ class AIAgent:
|
||||||
old_text=function_args.get("old_text"),
|
old_text=function_args.get("old_text"),
|
||||||
store=self._memory_store,
|
store=self._memory_store,
|
||||||
)
|
)
|
||||||
# Also send user observations to Honcho when active
|
|
||||||
if self._honcho and target == "user" and function_args.get("action") == "add":
|
|
||||||
self._honcho_save_user_observation(function_args.get("content", ""))
|
|
||||||
tool_duration = time.time() - tool_start_time
|
tool_duration = time.time() - tool_start_time
|
||||||
if self.quiet_mode:
|
if self.quiet_mode:
|
||||||
self._vprint(f" {_get_cute_tool_message_impl('memory', function_args, tool_duration, result=function_result)}")
|
self._vprint(f" {_get_cute_tool_message_impl('memory', function_args, tool_duration, result=function_result)}")
|
||||||
|
|
@ -6299,8 +6022,6 @@ class AIAgent:
|
||||||
function_result = handle_function_call(
|
function_result = handle_function_call(
|
||||||
function_name, function_args, effective_task_id,
|
function_name, function_args, effective_task_id,
|
||||||
enabled_tools=list(self.valid_tool_names) if self.valid_tool_names else None,
|
enabled_tools=list(self.valid_tool_names) if self.valid_tool_names else None,
|
||||||
honcho_manager=self._honcho,
|
|
||||||
honcho_session_key=self._honcho_session_key,
|
|
||||||
)
|
)
|
||||||
_spinner_result = function_result
|
_spinner_result = function_result
|
||||||
except Exception as tool_error:
|
except Exception as tool_error:
|
||||||
|
|
@ -6318,8 +6039,6 @@ class AIAgent:
|
||||||
function_result = handle_function_call(
|
function_result = handle_function_call(
|
||||||
function_name, function_args, effective_task_id,
|
function_name, function_args, effective_task_id,
|
||||||
enabled_tools=list(self.valid_tool_names) if self.valid_tool_names else None,
|
enabled_tools=list(self.valid_tool_names) if self.valid_tool_names else None,
|
||||||
honcho_manager=self._honcho,
|
|
||||||
honcho_session_key=self._honcho_session_key,
|
|
||||||
)
|
)
|
||||||
except Exception as tool_error:
|
except Exception as tool_error:
|
||||||
function_result = f"Error executing tool '{function_name}': {tool_error}"
|
function_result = f"Error executing tool '{function_name}': {tool_error}"
|
||||||
|
|
@ -6633,7 +6352,6 @@ class AIAgent:
|
||||||
task_id: str = None,
|
task_id: str = None,
|
||||||
stream_callback: Optional[callable] = None,
|
stream_callback: Optional[callable] = None,
|
||||||
persist_user_message: Optional[str] = None,
|
persist_user_message: Optional[str] = None,
|
||||||
sync_honcho: bool = True,
|
|
||||||
) -> Dict[str, Any]:
|
) -> Dict[str, Any]:
|
||||||
"""
|
"""
|
||||||
Run a complete conversation with tool calling until completion.
|
Run a complete conversation with tool calling until completion.
|
||||||
|
|
@ -6649,8 +6367,7 @@ class AIAgent:
|
||||||
persist_user_message: Optional clean user message to store in
|
persist_user_message: Optional clean user message to store in
|
||||||
transcripts/history when user_message contains API-only
|
transcripts/history when user_message contains API-only
|
||||||
synthetic prefixes.
|
synthetic prefixes.
|
||||||
sync_honcho: When False, skip writing the final synthetic turn back
|
or queuing follow-up prefetch work.
|
||||||
to Honcho or queuing follow-up prefetch work.
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Dict: Complete conversation result with final response and message history
|
Dict: Complete conversation result with final response and message history
|
||||||
|
|
@ -6734,7 +6451,6 @@ class AIAgent:
|
||||||
self._user_turn_count += 1
|
self._user_turn_count += 1
|
||||||
|
|
||||||
# Preserve the original user message (no nudge injection).
|
# Preserve the original user message (no nudge injection).
|
||||||
# Honcho should receive the actual user input, not system nudges.
|
|
||||||
original_user_message = persist_user_message if persist_user_message is not None else user_message
|
original_user_message = persist_user_message if persist_user_message is not None else user_message
|
||||||
|
|
||||||
# Track memory nudge trigger (turn-based, checked here).
|
# Track memory nudge trigger (turn-based, checked here).
|
||||||
|
|
@ -6749,27 +6465,6 @@ class AIAgent:
|
||||||
_should_review_memory = True
|
_should_review_memory = True
|
||||||
self._turns_since_memory = 0
|
self._turns_since_memory = 0
|
||||||
|
|
||||||
# Honcho prefetch consumption:
|
|
||||||
# - First turn: bake into cached system prompt (stable for the session).
|
|
||||||
# - Later turns: attach recall to the current-turn user message at
|
|
||||||
# API-call time only (never persisted to history / session DB).
|
|
||||||
#
|
|
||||||
# This keeps the system-prefix cache stable while still allowing turn N
|
|
||||||
# to consume background prefetch results from turn N-1.
|
|
||||||
self._honcho_context = ""
|
|
||||||
self._honcho_turn_context = ""
|
|
||||||
_recall_mode = (self._honcho_config.recall_mode if self._honcho_config else "hybrid")
|
|
||||||
if self._honcho and self._honcho_session_key and _recall_mode != "tools":
|
|
||||||
try:
|
|
||||||
prefetched_context = self._honcho_prefetch(original_user_message)
|
|
||||||
if prefetched_context:
|
|
||||||
if not conversation_history:
|
|
||||||
self._honcho_context = prefetched_context
|
|
||||||
else:
|
|
||||||
self._honcho_turn_context = prefetched_context
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug("Honcho prefetch failed (non-fatal): %s", e)
|
|
||||||
|
|
||||||
# Add user message
|
# Add user message
|
||||||
user_msg = {"role": "user", "content": user_message}
|
user_msg = {"role": "user", "content": user_message}
|
||||||
messages.append(user_msg)
|
messages.append(user_msg)
|
||||||
|
|
@ -6807,13 +6502,6 @@ class AIAgent:
|
||||||
else:
|
else:
|
||||||
# First turn of a new session — build from scratch.
|
# First turn of a new session — build from scratch.
|
||||||
self._cached_system_prompt = self._build_system_prompt(system_message)
|
self._cached_system_prompt = self._build_system_prompt(system_message)
|
||||||
# Bake Honcho context into the prompt so it's stable for
|
|
||||||
# the entire session (not re-fetched per turn).
|
|
||||||
if self._honcho_context:
|
|
||||||
self._cached_system_prompt = (
|
|
||||||
self._cached_system_prompt + "\n\n" + self._honcho_context
|
|
||||||
).strip()
|
|
||||||
|
|
||||||
# Plugin hook: on_session_start
|
# Plugin hook: on_session_start
|
||||||
# Fired once when a brand-new session is created (not on
|
# Fired once when a brand-new session is created (not on
|
||||||
# continuation). Plugins can use this to initialise
|
# continuation). Plugins can use this to initialise
|
||||||
|
|
@ -6936,6 +6624,17 @@ class AIAgent:
|
||||||
# Clear any stale interrupt state at start
|
# Clear any stale interrupt state at start
|
||||||
self.clear_interrupt()
|
self.clear_interrupt()
|
||||||
|
|
||||||
|
# External memory provider: prefetch once before the tool loop.
|
||||||
|
# Reuse the cached result on every iteration to avoid re-calling
|
||||||
|
# prefetch_all() on each tool call (10 tool calls = 10x latency + cost).
|
||||||
|
_ext_prefetch_cache = ""
|
||||||
|
if self._memory_manager:
|
||||||
|
try:
|
||||||
|
_query = user_message if isinstance(user_message, str) else ""
|
||||||
|
_ext_prefetch_cache = self._memory_manager.prefetch_all(_query) or ""
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
while api_call_count < self.max_iterations and self.iteration_budget.remaining > 0:
|
while api_call_count < self.max_iterations and self.iteration_budget.remaining > 0:
|
||||||
# Reset per-turn checkpoint dedup so each iteration can take one snapshot
|
# Reset per-turn checkpoint dedup so each iteration can take one snapshot
|
||||||
self._checkpoint_mgr.new_turn()
|
self._checkpoint_mgr.new_turn()
|
||||||
|
|
@ -6984,10 +6683,11 @@ class AIAgent:
|
||||||
for idx, msg in enumerate(messages):
|
for idx, msg in enumerate(messages):
|
||||||
api_msg = msg.copy()
|
api_msg = msg.copy()
|
||||||
|
|
||||||
if idx == current_turn_user_idx and msg.get("role") == "user" and self._honcho_turn_context:
|
# External memory provider prefetch: inject cached recalled context
|
||||||
api_msg["content"] = _inject_honcho_turn_context(
|
if idx == current_turn_user_idx and msg.get("role") == "user" and _ext_prefetch_cache:
|
||||||
api_msg.get("content", ""), self._honcho_turn_context
|
_base = api_msg.get("content", "")
|
||||||
)
|
if isinstance(_base, str):
|
||||||
|
api_msg["content"] = _base + "\n\n" + _ext_prefetch_cache
|
||||||
|
|
||||||
# For ALL assistant messages, pass reasoning back to the API
|
# For ALL assistant messages, pass reasoning back to the API
|
||||||
# This ensures multi-turn reasoning context is preserved
|
# This ensures multi-turn reasoning context is preserved
|
||||||
|
|
@ -7016,8 +6716,8 @@ class AIAgent:
|
||||||
|
|
||||||
# Build the final system message: cached prompt + ephemeral system prompt.
|
# Build the final system message: cached prompt + ephemeral system prompt.
|
||||||
# Ephemeral additions are API-call-time only (not persisted to session DB).
|
# Ephemeral additions are API-call-time only (not persisted to session DB).
|
||||||
# Honcho later-turn recall is intentionally kept OUT of the system prompt
|
# External recall context is injected into the user message, not the system
|
||||||
# so the stable cache prefix remains unchanged.
|
# prompt, so the stable cache prefix remains unchanged.
|
||||||
effective_system = active_system_prompt or ""
|
effective_system = active_system_prompt or ""
|
||||||
if self.ephemeral_system_prompt:
|
if self.ephemeral_system_prompt:
|
||||||
effective_system = (effective_system + "\n\n" + self.ephemeral_system_prompt).strip()
|
effective_system = (effective_system + "\n\n" + self.ephemeral_system_prompt).strip()
|
||||||
|
|
@ -8730,10 +8430,6 @@ class AIAgent:
|
||||||
# Persist session to both JSON log and SQLite
|
# Persist session to both JSON log and SQLite
|
||||||
self._persist_session(messages, conversation_history)
|
self._persist_session(messages, conversation_history)
|
||||||
|
|
||||||
# Sync conversation to Honcho for user modeling
|
|
||||||
if final_response and not interrupted and sync_honcho:
|
|
||||||
self._honcho_sync(original_user_message, final_response)
|
|
||||||
self._queue_honcho_prefetch(original_user_message)
|
|
||||||
|
|
||||||
# Plugin hook: post_llm_call
|
# Plugin hook: post_llm_call
|
||||||
# Fired once per turn after the tool-calling loop completes.
|
# Fired once per turn after the tool-calling loop completes.
|
||||||
|
|
@ -8807,6 +8503,14 @@ class AIAgent:
|
||||||
_should_review_skills = True
|
_should_review_skills = True
|
||||||
self._iters_since_skill = 0
|
self._iters_since_skill = 0
|
||||||
|
|
||||||
|
# External memory provider: sync the completed turn + queue next prefetch
|
||||||
|
if self._memory_manager and final_response and user_message:
|
||||||
|
try:
|
||||||
|
self._memory_manager.sync_all(user_message, final_response)
|
||||||
|
self._memory_manager.queue_prefetch_all(user_message)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
# Background memory/skill review — runs AFTER the response is delivered
|
# Background memory/skill review — runs AFTER the response is delivered
|
||||||
# so it never competes with the user's task for model attention.
|
# so it never competes with the user's task for model attention.
|
||||||
if final_response and not interrupted and (_should_review_memory or _should_review_skills):
|
if final_response and not interrupted and (_should_review_memory or _should_review_skills):
|
||||||
|
|
@ -8819,6 +8523,13 @@ class AIAgent:
|
||||||
except Exception:
|
except Exception:
|
||||||
pass # Background review is best-effort
|
pass # Background review is best-effort
|
||||||
|
|
||||||
|
# Note: Memory provider on_session_end() + shutdown_all() are NOT
|
||||||
|
# called here — run_conversation() is called once per user message in
|
||||||
|
# multi-turn sessions. Shutting down after every turn would kill the
|
||||||
|
# provider before the second message. Actual session-end cleanup is
|
||||||
|
# handled by the CLI (atexit / /reset) and gateway (session expiry /
|
||||||
|
# _reset_session).
|
||||||
|
|
||||||
# Plugin hook: on_session_end
|
# Plugin hook: on_session_end
|
||||||
# Fired at the very end of every run_conversation call.
|
# Fired at the very end of every run_conversation call.
|
||||||
# Plugins can use this for cleanup, flushing buffers, etc.
|
# Plugins can use this for cleanup, flushing buffers, etc.
|
||||||
|
|
|
||||||
299
tests/agent/test_memory_plugin_e2e.py
Normal file
299
tests/agent/test_memory_plugin_e2e.py
Normal file
|
|
@ -0,0 +1,299 @@
|
||||||
|
"""End-to-end test: a SQLite-backed memory plugin exercising the full interface.
|
||||||
|
|
||||||
|
This proves a real plugin can register as a MemoryProvider and get wired
|
||||||
|
into the agent loop via MemoryManager. Uses SQLite + FTS5 (stdlib, no
|
||||||
|
external deps, no API keys).
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import tempfile
|
||||||
|
import pytest
|
||||||
|
from unittest.mock import patch, MagicMock
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
from agent.memory_manager import MemoryManager
|
||||||
|
from agent.builtin_memory_provider import BuiltinMemoryProvider
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# SQLite FTS5 memory provider — a real, minimal plugin implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteMemoryProvider(MemoryProvider):
|
||||||
|
"""Minimal SQLite + FTS5 memory provider for testing.
|
||||||
|
|
||||||
|
Demonstrates the full MemoryProvider interface with a real backend.
|
||||||
|
No external dependencies — just stdlib sqlite3.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, db_path: str = ":memory:"):
|
||||||
|
self._db_path = db_path
|
||||||
|
self._conn = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "sqlite_memory"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return True # SQLite is always available
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
self._conn = sqlite3.connect(self._db_path)
|
||||||
|
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||||
|
self._conn.execute("""
|
||||||
|
CREATE VIRTUAL TABLE IF NOT EXISTS memories
|
||||||
|
USING fts5(content, context, session_id)
|
||||||
|
""")
|
||||||
|
self._session_id = session_id
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
if not self._conn:
|
||||||
|
return ""
|
||||||
|
count = self._conn.execute("SELECT COUNT(*) FROM memories").fetchone()[0]
|
||||||
|
if count == 0:
|
||||||
|
return ""
|
||||||
|
return (
|
||||||
|
f"# SQLite Memory Plugin\n"
|
||||||
|
f"Active. {count} memories stored.\n"
|
||||||
|
f"Use sqlite_recall to search, sqlite_retain to store."
|
||||||
|
)
|
||||||
|
|
||||||
|
def prefetch(self, query: str, *, session_id: str = "") -> str:
|
||||||
|
if not self._conn or not query:
|
||||||
|
return ""
|
||||||
|
# FTS5 search
|
||||||
|
try:
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT content FROM memories WHERE memories MATCH ? LIMIT 5",
|
||||||
|
(query,)
|
||||||
|
).fetchall()
|
||||||
|
if not rows:
|
||||||
|
return ""
|
||||||
|
results = [row[0] for row in rows]
|
||||||
|
return "## SQLite Memory\n" + "\n".join(f"- {r}" for r in results)
|
||||||
|
except sqlite3.OperationalError:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
|
||||||
|
if not self._conn:
|
||||||
|
return
|
||||||
|
combined = f"User: {user_content}\nAssistant: {assistant_content}"
|
||||||
|
self._conn.execute(
|
||||||
|
"INSERT INTO memories (content, context, session_id) VALUES (?, ?, ?)",
|
||||||
|
(combined, "conversation", self._session_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
def get_tool_schemas(self):
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"name": "sqlite_retain",
|
||||||
|
"description": "Store a fact to SQLite memory.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"content": {"type": "string", "description": "What to remember"},
|
||||||
|
"context": {"type": "string", "description": "Category/context"},
|
||||||
|
},
|
||||||
|
"required": ["content"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "sqlite_recall",
|
||||||
|
"description": "Search SQLite memory.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"query": {"type": "string", "description": "Search query"},
|
||||||
|
},
|
||||||
|
"required": ["query"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
|
||||||
|
if tool_name == "sqlite_retain":
|
||||||
|
content = args.get("content", "")
|
||||||
|
context = args.get("context", "explicit")
|
||||||
|
if not content:
|
||||||
|
return json.dumps({"error": "content is required"})
|
||||||
|
self._conn.execute(
|
||||||
|
"INSERT INTO memories (content, context, session_id) VALUES (?, ?, ?)",
|
||||||
|
(content, context, self._session_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
return json.dumps({"result": "Stored."})
|
||||||
|
|
||||||
|
elif tool_name == "sqlite_recall":
|
||||||
|
query = args.get("query", "")
|
||||||
|
if not query:
|
||||||
|
return json.dumps({"error": "query is required"})
|
||||||
|
try:
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT content, context FROM memories WHERE memories MATCH ? LIMIT 10",
|
||||||
|
(query,)
|
||||||
|
).fetchall()
|
||||||
|
results = [{"content": r[0], "context": r[1]} for r in rows]
|
||||||
|
return json.dumps({"results": results})
|
||||||
|
except sqlite3.OperationalError:
|
||||||
|
return json.dumps({"results": []})
|
||||||
|
|
||||||
|
return json.dumps({"error": f"Unknown tool: {tool_name}"})
|
||||||
|
|
||||||
|
def on_memory_write(self, action, target, content):
|
||||||
|
"""Mirror built-in memory writes to SQLite."""
|
||||||
|
if action == "add" and self._conn:
|
||||||
|
self._conn.execute(
|
||||||
|
"INSERT INTO memories (content, context, session_id) VALUES (?, ?, ?)",
|
||||||
|
(content, f"builtin_{target}", self._session_id),
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
|
def shutdown(self):
|
||||||
|
if self._conn:
|
||||||
|
self._conn.close()
|
||||||
|
self._conn = None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# End-to-end tests
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestSQLiteMemoryPlugin:
|
||||||
|
"""Full lifecycle test with the SQLite provider."""
|
||||||
|
|
||||||
|
def test_full_lifecycle(self):
|
||||||
|
"""Exercise init → store → recall → sync → prefetch → shutdown."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider()
|
||||||
|
sqlite_mem = SQLiteMemoryProvider()
|
||||||
|
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
mgr.add_provider(sqlite_mem)
|
||||||
|
|
||||||
|
# Initialize
|
||||||
|
mgr.initialize_all(session_id="test-session-1", platform="cli")
|
||||||
|
assert sqlite_mem._conn is not None
|
||||||
|
|
||||||
|
# System prompt — empty at first
|
||||||
|
prompt = mgr.build_system_prompt()
|
||||||
|
assert "SQLite Memory Plugin" not in prompt
|
||||||
|
|
||||||
|
# Store via tool call
|
||||||
|
result = json.loads(mgr.handle_tool_call(
|
||||||
|
"sqlite_retain", {"content": "User prefers dark mode", "context": "preference"}
|
||||||
|
))
|
||||||
|
assert result["result"] == "Stored."
|
||||||
|
|
||||||
|
# System prompt now shows count
|
||||||
|
prompt = mgr.build_system_prompt()
|
||||||
|
assert "1 memories stored" in prompt
|
||||||
|
|
||||||
|
# Recall via tool call
|
||||||
|
result = json.loads(mgr.handle_tool_call(
|
||||||
|
"sqlite_recall", {"query": "dark mode"}
|
||||||
|
))
|
||||||
|
assert len(result["results"]) == 1
|
||||||
|
assert "dark mode" in result["results"][0]["content"]
|
||||||
|
|
||||||
|
# Sync a turn (auto-stores conversation)
|
||||||
|
mgr.sync_all("What's my theme?", "You prefer dark mode.")
|
||||||
|
count = sqlite_mem._conn.execute("SELECT COUNT(*) FROM memories").fetchone()[0]
|
||||||
|
assert count == 2 # 1 explicit + 1 synced
|
||||||
|
|
||||||
|
# Prefetch for next turn
|
||||||
|
prefetched = mgr.prefetch_all("dark mode")
|
||||||
|
assert "dark mode" in prefetched
|
||||||
|
|
||||||
|
# Memory bridge — mirroring builtin writes
|
||||||
|
mgr.on_memory_write("add", "user", "Timezone: US Pacific")
|
||||||
|
count = sqlite_mem._conn.execute("SELECT COUNT(*) FROM memories").fetchone()[0]
|
||||||
|
assert count == 3
|
||||||
|
|
||||||
|
# Shutdown
|
||||||
|
mgr.shutdown_all()
|
||||||
|
assert sqlite_mem._conn is None
|
||||||
|
|
||||||
|
def test_tool_routing_with_builtin(self):
|
||||||
|
"""Verify builtin + plugin tools coexist without conflict."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider()
|
||||||
|
sqlite_mem = SQLiteMemoryProvider()
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
mgr.add_provider(sqlite_mem)
|
||||||
|
mgr.initialize_all(session_id="test-2")
|
||||||
|
|
||||||
|
# Builtin has no tools
|
||||||
|
assert len(builtin.get_tool_schemas()) == 0
|
||||||
|
# SQLite has 2 tools
|
||||||
|
schemas = mgr.get_all_tool_schemas()
|
||||||
|
names = {s["name"] for s in schemas}
|
||||||
|
assert names == {"sqlite_retain", "sqlite_recall"}
|
||||||
|
|
||||||
|
# Routing works
|
||||||
|
assert mgr.has_tool("sqlite_retain")
|
||||||
|
assert mgr.has_tool("sqlite_recall")
|
||||||
|
assert not mgr.has_tool("memory") # builtin doesn't register this
|
||||||
|
|
||||||
|
def test_second_external_plugin_rejected(self):
|
||||||
|
"""Only one external memory provider is allowed at a time."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = SQLiteMemoryProvider()
|
||||||
|
p2 = SQLiteMemoryProvider()
|
||||||
|
# Hack name for p2
|
||||||
|
p2._name_override = "sqlite_memory_2"
|
||||||
|
original_name = p2.__class__.name
|
||||||
|
type(p2).name = property(lambda self: getattr(self, '_name_override', 'sqlite_memory'))
|
||||||
|
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2) # should be rejected
|
||||||
|
|
||||||
|
# Only p1 was accepted
|
||||||
|
assert len(mgr.providers) == 1
|
||||||
|
assert mgr.provider_names == ["sqlite_memory"]
|
||||||
|
|
||||||
|
# Restore class
|
||||||
|
type(p2).name = original_name
|
||||||
|
mgr.shutdown_all()
|
||||||
|
|
||||||
|
def test_provider_failure_isolation(self):
|
||||||
|
"""Failing external provider doesn't break builtin."""
|
||||||
|
from agent.builtin_memory_provider import BuiltinMemoryProvider
|
||||||
|
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider() # name="builtin", always accepted
|
||||||
|
ext = SQLiteMemoryProvider()
|
||||||
|
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
mgr.add_provider(ext)
|
||||||
|
mgr.initialize_all(session_id="test-4")
|
||||||
|
|
||||||
|
# Break external provider's connection
|
||||||
|
ext._conn.close()
|
||||||
|
ext._conn = None
|
||||||
|
|
||||||
|
# Sync — external fails silently, builtin (no-op sync) succeeds
|
||||||
|
mgr.sync_all("user", "assistant") # should not raise
|
||||||
|
|
||||||
|
mgr.shutdown_all()
|
||||||
|
|
||||||
|
def test_plugin_registration_flow(self):
|
||||||
|
"""Simulate the full plugin load → agent init path."""
|
||||||
|
# Simulate what AIAgent.__init__ does via plugins/memory/ discovery
|
||||||
|
provider = SQLiteMemoryProvider()
|
||||||
|
|
||||||
|
mem_mgr = MemoryManager()
|
||||||
|
mem_mgr.add_provider(BuiltinMemoryProvider())
|
||||||
|
if provider.is_available():
|
||||||
|
mem_mgr.add_provider(provider)
|
||||||
|
mem_mgr.initialize_all(session_id="agent-session")
|
||||||
|
|
||||||
|
assert len(mem_mgr.providers) == 2
|
||||||
|
assert mem_mgr.provider_names == ["builtin", "sqlite_memory"]
|
||||||
|
assert provider._conn is not None # initialized = connection established
|
||||||
|
|
||||||
|
mem_mgr.shutdown_all()
|
||||||
549
tests/agent/test_memory_provider.py
Normal file
549
tests/agent/test_memory_provider.py
Normal file
|
|
@ -0,0 +1,549 @@
|
||||||
|
"""Tests for the memory provider interface, manager, and builtin provider."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import pytest
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
from agent.memory_manager import MemoryManager
|
||||||
|
from agent.builtin_memory_provider import BuiltinMemoryProvider
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Concrete test provider
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class FakeMemoryProvider(MemoryProvider):
|
||||||
|
"""Minimal concrete provider for testing."""
|
||||||
|
|
||||||
|
def __init__(self, name="fake", available=True, tools=None):
|
||||||
|
self._name = name
|
||||||
|
self._available = available
|
||||||
|
self._tools = tools or []
|
||||||
|
self.initialized = False
|
||||||
|
self.synced_turns = []
|
||||||
|
self.prefetch_queries = []
|
||||||
|
self.queued_prefetches = []
|
||||||
|
self.turn_starts = []
|
||||||
|
self.session_end_called = False
|
||||||
|
self.pre_compress_called = False
|
||||||
|
self.memory_writes = []
|
||||||
|
self.shutdown_called = False
|
||||||
|
self._prefetch_result = ""
|
||||||
|
self._prompt_block = ""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return self._name
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return self._available
|
||||||
|
|
||||||
|
def initialize(self, session_id, **kwargs):
|
||||||
|
self.initialized = True
|
||||||
|
self._init_kwargs = {"session_id": session_id, **kwargs}
|
||||||
|
|
||||||
|
def system_prompt_block(self) -> str:
|
||||||
|
return self._prompt_block
|
||||||
|
|
||||||
|
def prefetch(self, query, *, session_id=""):
|
||||||
|
self.prefetch_queries.append(query)
|
||||||
|
return self._prefetch_result
|
||||||
|
|
||||||
|
def queue_prefetch(self, query, *, session_id=""):
|
||||||
|
self.queued_prefetches.append(query)
|
||||||
|
|
||||||
|
def sync_turn(self, user_content, assistant_content, *, session_id=""):
|
||||||
|
self.synced_turns.append((user_content, assistant_content))
|
||||||
|
|
||||||
|
def get_tool_schemas(self):
|
||||||
|
return self._tools
|
||||||
|
|
||||||
|
def handle_tool_call(self, tool_name, args, **kwargs):
|
||||||
|
return json.dumps({"handled": tool_name, "args": args})
|
||||||
|
|
||||||
|
def shutdown(self):
|
||||||
|
self.shutdown_called = True
|
||||||
|
|
||||||
|
def on_turn_start(self, turn_number, message):
|
||||||
|
self.turn_starts.append((turn_number, message))
|
||||||
|
|
||||||
|
def on_session_end(self, messages):
|
||||||
|
self.session_end_called = True
|
||||||
|
|
||||||
|
def on_pre_compress(self, messages):
|
||||||
|
self.pre_compress_called = True
|
||||||
|
|
||||||
|
def on_memory_write(self, action, target, content):
|
||||||
|
self.memory_writes.append((action, target, content))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryProvider ABC tests
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestMemoryProviderABC:
|
||||||
|
def test_cannot_instantiate_abstract(self):
|
||||||
|
"""ABC cannot be instantiated directly."""
|
||||||
|
with pytest.raises(TypeError):
|
||||||
|
MemoryProvider()
|
||||||
|
|
||||||
|
def test_concrete_provider_works(self):
|
||||||
|
"""Concrete implementation can be instantiated."""
|
||||||
|
p = FakeMemoryProvider()
|
||||||
|
assert p.name == "fake"
|
||||||
|
assert p.is_available()
|
||||||
|
|
||||||
|
def test_default_optional_hooks_are_noop(self):
|
||||||
|
"""Optional hooks have default no-op implementations."""
|
||||||
|
p = FakeMemoryProvider()
|
||||||
|
# These should not raise
|
||||||
|
p.on_turn_start(1, "hello")
|
||||||
|
p.on_session_end([])
|
||||||
|
p.on_pre_compress([])
|
||||||
|
p.on_memory_write("add", "memory", "test")
|
||||||
|
p.queue_prefetch("query")
|
||||||
|
p.sync_turn("user", "assistant")
|
||||||
|
p.shutdown()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MemoryManager tests
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestMemoryManager:
|
||||||
|
def test_empty_manager(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
assert mgr.providers == []
|
||||||
|
assert mgr.provider_names == []
|
||||||
|
assert mgr.get_all_tool_schemas() == []
|
||||||
|
assert mgr.build_system_prompt() == ""
|
||||||
|
assert mgr.prefetch_all("test") == ""
|
||||||
|
|
||||||
|
def test_add_provider(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p = FakeMemoryProvider("test1")
|
||||||
|
mgr.add_provider(p)
|
||||||
|
assert len(mgr.providers) == 1
|
||||||
|
assert mgr.provider_names == ["test1"]
|
||||||
|
|
||||||
|
def test_get_provider_by_name(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p = FakeMemoryProvider("test1")
|
||||||
|
mgr.add_provider(p)
|
||||||
|
assert mgr.get_provider("test1") is p
|
||||||
|
assert mgr.get_provider("nonexistent") is None
|
||||||
|
|
||||||
|
def test_builtin_plus_external(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
assert mgr.provider_names == ["builtin", "external"]
|
||||||
|
|
||||||
|
def test_second_external_rejected(self):
|
||||||
|
"""Only one non-builtin provider is allowed."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = FakeMemoryProvider("builtin")
|
||||||
|
ext1 = FakeMemoryProvider("mem0")
|
||||||
|
ext2 = FakeMemoryProvider("hindsight")
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
mgr.add_provider(ext1)
|
||||||
|
mgr.add_provider(ext2) # should be rejected
|
||||||
|
assert mgr.provider_names == ["builtin", "mem0"]
|
||||||
|
assert len(mgr.providers) == 2
|
||||||
|
|
||||||
|
def test_system_prompt_merges_blocks(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1._prompt_block = "Block from builtin"
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
p2._prompt_block = "Block from external"
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
result = mgr.build_system_prompt()
|
||||||
|
assert "Block from builtin" in result
|
||||||
|
assert "Block from external" in result
|
||||||
|
|
||||||
|
def test_system_prompt_skips_empty(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1._prompt_block = "Has content"
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
p2._prompt_block = ""
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
result = mgr.build_system_prompt()
|
||||||
|
assert result == "Has content"
|
||||||
|
|
||||||
|
def test_prefetch_merges_results(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1._prefetch_result = "Memory from builtin"
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
p2._prefetch_result = "Memory from external"
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
result = mgr.prefetch_all("what do you know?")
|
||||||
|
assert "Memory from builtin" in result
|
||||||
|
assert "Memory from external" in result
|
||||||
|
assert p1.prefetch_queries == ["what do you know?"]
|
||||||
|
assert p2.prefetch_queries == ["what do you know?"]
|
||||||
|
|
||||||
|
def test_prefetch_skips_empty(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1._prefetch_result = "Has memories"
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
p2._prefetch_result = ""
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
result = mgr.prefetch_all("query")
|
||||||
|
assert result == "Has memories"
|
||||||
|
|
||||||
|
def test_queue_prefetch_all(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
mgr.queue_prefetch_all("next turn")
|
||||||
|
assert p1.queued_prefetches == ["next turn"]
|
||||||
|
assert p2.queued_prefetches == ["next turn"]
|
||||||
|
|
||||||
|
def test_sync_all(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
mgr.sync_all("user msg", "assistant msg")
|
||||||
|
assert p1.synced_turns == [("user msg", "assistant msg")]
|
||||||
|
assert p2.synced_turns == [("user msg", "assistant msg")]
|
||||||
|
|
||||||
|
def test_sync_failure_doesnt_block_others(self):
|
||||||
|
"""If one provider's sync fails, others still run."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1.sync_turn = MagicMock(side_effect=RuntimeError("boom"))
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
mgr.sync_all("user", "assistant")
|
||||||
|
# p1 failed but p2 still synced
|
||||||
|
assert p2.synced_turns == [("user", "assistant")]
|
||||||
|
|
||||||
|
# -- Tool routing -------------------------------------------------------
|
||||||
|
|
||||||
|
def test_tool_schemas_collected(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin", tools=[
|
||||||
|
{"name": "recall_builtin", "description": "Builtin recall", "parameters": {}}
|
||||||
|
])
|
||||||
|
p2 = FakeMemoryProvider("external", tools=[
|
||||||
|
{"name": "recall_ext", "description": "External recall", "parameters": {}}
|
||||||
|
])
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
schemas = mgr.get_all_tool_schemas()
|
||||||
|
names = {s["name"] for s in schemas}
|
||||||
|
assert names == {"recall_builtin", "recall_ext"}
|
||||||
|
|
||||||
|
def test_tool_name_conflict_first_wins(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin", tools=[
|
||||||
|
{"name": "shared_tool", "description": "From builtin", "parameters": {}}
|
||||||
|
])
|
||||||
|
p2 = FakeMemoryProvider("external", tools=[
|
||||||
|
{"name": "shared_tool", "description": "From external", "parameters": {}}
|
||||||
|
])
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
assert mgr.has_tool("shared_tool")
|
||||||
|
result = json.loads(mgr.handle_tool_call("shared_tool", {"q": "test"}))
|
||||||
|
assert result["handled"] == "shared_tool"
|
||||||
|
# Should be handled by p1 (first registered)
|
||||||
|
|
||||||
|
def test_handle_unknown_tool(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
result = json.loads(mgr.handle_tool_call("nonexistent", {}))
|
||||||
|
assert "error" in result
|
||||||
|
|
||||||
|
def test_tool_routing(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin", tools=[
|
||||||
|
{"name": "builtin_tool", "description": "Builtin", "parameters": {}}
|
||||||
|
])
|
||||||
|
p2 = FakeMemoryProvider("external", tools=[
|
||||||
|
{"name": "ext_tool", "description": "External", "parameters": {}}
|
||||||
|
])
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
r1 = json.loads(mgr.handle_tool_call("builtin_tool", {"a": 1}))
|
||||||
|
assert r1["handled"] == "builtin_tool"
|
||||||
|
r2 = json.loads(mgr.handle_tool_call("ext_tool", {"b": 2}))
|
||||||
|
assert r2["handled"] == "ext_tool"
|
||||||
|
|
||||||
|
# -- Lifecycle hooks -----------------------------------------------------
|
||||||
|
|
||||||
|
def test_on_turn_start(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p = FakeMemoryProvider("p")
|
||||||
|
mgr.add_provider(p)
|
||||||
|
mgr.on_turn_start(3, "hello")
|
||||||
|
assert p.turn_starts == [(3, "hello")]
|
||||||
|
|
||||||
|
def test_on_session_end(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p = FakeMemoryProvider("p")
|
||||||
|
mgr.add_provider(p)
|
||||||
|
mgr.on_session_end([{"role": "user", "content": "hi"}])
|
||||||
|
assert p.session_end_called
|
||||||
|
|
||||||
|
def test_on_pre_compress(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p = FakeMemoryProvider("p")
|
||||||
|
mgr.add_provider(p)
|
||||||
|
mgr.on_pre_compress([{"role": "user", "content": "old"}])
|
||||||
|
assert p.pre_compress_called
|
||||||
|
|
||||||
|
def test_on_memory_write_skips_builtin(self):
|
||||||
|
"""on_memory_write should skip the builtin provider."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider()
|
||||||
|
external = FakeMemoryProvider("external")
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
mgr.add_provider(external)
|
||||||
|
|
||||||
|
mgr.on_memory_write("add", "memory", "test fact")
|
||||||
|
assert external.memory_writes == [("add", "memory", "test fact")]
|
||||||
|
|
||||||
|
def test_shutdown_all_reverse_order(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
order = []
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1.shutdown = lambda: order.append("builtin")
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
p2.shutdown = lambda: order.append("external")
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
mgr.shutdown_all()
|
||||||
|
assert order == ["external", "builtin"] # reverse order
|
||||||
|
|
||||||
|
def test_initialize_all(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
mgr.initialize_all(session_id="test-123", platform="cli")
|
||||||
|
assert p1.initialized
|
||||||
|
assert p2.initialized
|
||||||
|
assert p1._init_kwargs["session_id"] == "test-123"
|
||||||
|
assert p1._init_kwargs["platform"] == "cli"
|
||||||
|
|
||||||
|
# -- Error resilience ---------------------------------------------------
|
||||||
|
|
||||||
|
def test_prefetch_failure_doesnt_block(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1.prefetch = MagicMock(side_effect=RuntimeError("network error"))
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
p2._prefetch_result = "external memory"
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
result = mgr.prefetch_all("query")
|
||||||
|
assert "external memory" in result
|
||||||
|
|
||||||
|
def test_system_prompt_failure_doesnt_block(self):
|
||||||
|
mgr = MemoryManager()
|
||||||
|
p1 = FakeMemoryProvider("builtin")
|
||||||
|
p1.system_prompt_block = MagicMock(side_effect=RuntimeError("broken"))
|
||||||
|
p2 = FakeMemoryProvider("external")
|
||||||
|
p2._prompt_block = "works fine"
|
||||||
|
mgr.add_provider(p1)
|
||||||
|
mgr.add_provider(p2)
|
||||||
|
|
||||||
|
result = mgr.build_system_prompt()
|
||||||
|
assert result == "works fine"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# BuiltinMemoryProvider tests
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestBuiltinMemoryProvider:
|
||||||
|
def test_name(self):
|
||||||
|
p = BuiltinMemoryProvider()
|
||||||
|
assert p.name == "builtin"
|
||||||
|
|
||||||
|
def test_always_available(self):
|
||||||
|
p = BuiltinMemoryProvider()
|
||||||
|
assert p.is_available()
|
||||||
|
|
||||||
|
def test_no_tools(self):
|
||||||
|
"""Builtin provider exposes no tools (memory tool is agent-level)."""
|
||||||
|
p = BuiltinMemoryProvider()
|
||||||
|
assert p.get_tool_schemas() == []
|
||||||
|
|
||||||
|
def test_system_prompt_with_store(self):
|
||||||
|
store = MagicMock()
|
||||||
|
store.format_for_system_prompt.side_effect = lambda t: f"BLOCK_{t}" if t == "memory" else f"BLOCK_{t}"
|
||||||
|
|
||||||
|
p = BuiltinMemoryProvider(
|
||||||
|
memory_store=store,
|
||||||
|
memory_enabled=True,
|
||||||
|
user_profile_enabled=True,
|
||||||
|
)
|
||||||
|
block = p.system_prompt_block()
|
||||||
|
assert "BLOCK_memory" in block
|
||||||
|
assert "BLOCK_user" in block
|
||||||
|
|
||||||
|
def test_system_prompt_memory_disabled(self):
|
||||||
|
store = MagicMock()
|
||||||
|
store.format_for_system_prompt.return_value = "content"
|
||||||
|
|
||||||
|
p = BuiltinMemoryProvider(
|
||||||
|
memory_store=store,
|
||||||
|
memory_enabled=False,
|
||||||
|
user_profile_enabled=False,
|
||||||
|
)
|
||||||
|
assert p.system_prompt_block() == ""
|
||||||
|
|
||||||
|
def test_system_prompt_no_store(self):
|
||||||
|
p = BuiltinMemoryProvider(memory_store=None, memory_enabled=True)
|
||||||
|
assert p.system_prompt_block() == ""
|
||||||
|
|
||||||
|
def test_prefetch_returns_empty(self):
|
||||||
|
p = BuiltinMemoryProvider()
|
||||||
|
assert p.prefetch("anything") == ""
|
||||||
|
|
||||||
|
def test_store_property(self):
|
||||||
|
store = MagicMock()
|
||||||
|
p = BuiltinMemoryProvider(memory_store=store)
|
||||||
|
assert p.store is store
|
||||||
|
|
||||||
|
def test_initialize_loads_from_disk(self):
|
||||||
|
store = MagicMock()
|
||||||
|
p = BuiltinMemoryProvider(memory_store=store)
|
||||||
|
p.initialize(session_id="test")
|
||||||
|
store.load_from_disk.assert_called_once()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Plugin registration tests
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestSingleProviderGating:
|
||||||
|
"""Only the configured provider should activate."""
|
||||||
|
|
||||||
|
def test_no_provider_configured_means_builtin_only(self):
|
||||||
|
"""When memory.provider is empty, no plugin providers activate."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider()
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
|
||||||
|
# Simulate what run_agent.py does when provider=""
|
||||||
|
configured = ""
|
||||||
|
available_plugins = [
|
||||||
|
FakeMemoryProvider("holographic"),
|
||||||
|
FakeMemoryProvider("mem0"),
|
||||||
|
]
|
||||||
|
# With empty config, no plugins should be added
|
||||||
|
if configured:
|
||||||
|
for p in available_plugins:
|
||||||
|
if p.name == configured and p.is_available():
|
||||||
|
mgr.add_provider(p)
|
||||||
|
|
||||||
|
assert mgr.provider_names == ["builtin"]
|
||||||
|
|
||||||
|
def test_configured_provider_activates(self):
|
||||||
|
"""Only the named provider should be added."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider()
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
|
||||||
|
configured = "holographic"
|
||||||
|
p1 = FakeMemoryProvider("holographic")
|
||||||
|
p2 = FakeMemoryProvider("mem0")
|
||||||
|
p3 = FakeMemoryProvider("hindsight")
|
||||||
|
|
||||||
|
for p in [p1, p2, p3]:
|
||||||
|
if p.name == configured and p.is_available():
|
||||||
|
mgr.add_provider(p)
|
||||||
|
|
||||||
|
assert mgr.provider_names == ["builtin", "holographic"]
|
||||||
|
assert p1.initialized is False # not initialized by the gating logic itself
|
||||||
|
|
||||||
|
def test_unavailable_provider_skipped(self):
|
||||||
|
"""If the configured provider is unavailable, it should be skipped."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider()
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
|
||||||
|
configured = "holographic"
|
||||||
|
p1 = FakeMemoryProvider("holographic", available=False)
|
||||||
|
|
||||||
|
for p in [p1]:
|
||||||
|
if p.name == configured and p.is_available():
|
||||||
|
mgr.add_provider(p)
|
||||||
|
|
||||||
|
assert mgr.provider_names == ["builtin"]
|
||||||
|
|
||||||
|
def test_nonexistent_provider_results_in_builtin_only(self):
|
||||||
|
"""If the configured name doesn't match any plugin, only builtin remains."""
|
||||||
|
mgr = MemoryManager()
|
||||||
|
builtin = BuiltinMemoryProvider()
|
||||||
|
mgr.add_provider(builtin)
|
||||||
|
|
||||||
|
configured = "nonexistent"
|
||||||
|
plugins = [FakeMemoryProvider("holographic"), FakeMemoryProvider("mem0")]
|
||||||
|
|
||||||
|
for p in plugins:
|
||||||
|
if p.name == configured and p.is_available():
|
||||||
|
mgr.add_provider(p)
|
||||||
|
|
||||||
|
assert mgr.provider_names == ["builtin"]
|
||||||
|
|
||||||
|
|
||||||
|
class TestPluginMemoryDiscovery:
|
||||||
|
"""Memory providers are discovered from plugins/memory/ directory."""
|
||||||
|
|
||||||
|
def test_discover_finds_providers(self):
|
||||||
|
"""discover_memory_providers returns available providers."""
|
||||||
|
from plugins.memory import discover_memory_providers
|
||||||
|
providers = discover_memory_providers()
|
||||||
|
names = [name for name, _, _ in providers]
|
||||||
|
assert "holographic" in names # always available (no external deps)
|
||||||
|
|
||||||
|
def test_load_provider_by_name(self):
|
||||||
|
"""load_memory_provider returns a working provider instance."""
|
||||||
|
from plugins.memory import load_memory_provider
|
||||||
|
p = load_memory_provider("holographic")
|
||||||
|
assert p is not None
|
||||||
|
assert p.name == "holographic"
|
||||||
|
assert p.is_available()
|
||||||
|
|
||||||
|
def test_load_nonexistent_returns_none(self):
|
||||||
|
"""load_memory_provider returns None for unknown names."""
|
||||||
|
from plugins.memory import load_memory_provider
|
||||||
|
assert load_memory_provider("nonexistent_provider") is None
|
||||||
|
|
@ -54,9 +54,10 @@ class TestCronSessionBypass:
|
||||||
# session_store.load_transcript should never be called
|
# session_store.load_transcript should never be called
|
||||||
runner.session_store.load_transcript.assert_not_called()
|
runner.session_store.load_transcript.assert_not_called()
|
||||||
|
|
||||||
def test_cron_session_with_honcho_key_skipped(self):
|
def test_cron_session_with_prefix_skipped(self):
|
||||||
|
"""Cron sessions with different prefixes are still skipped."""
|
||||||
runner = _make_runner()
|
runner = _make_runner()
|
||||||
runner._flush_memories_for_session("cron_daily_20260323", "some-honcho-key")
|
runner._flush_memories_for_session("cron_daily_20260323")
|
||||||
runner.session_store.load_transcript.assert_not_called()
|
runner.session_store.load_transcript.assert_not_called()
|
||||||
|
|
||||||
def test_non_cron_session_proceeds(self):
|
def test_non_cron_session_proceeds(self):
|
||||||
|
|
|
||||||
|
|
@ -1,131 +0,0 @@
|
||||||
"""Tests for gateway-owned Honcho lifecycle helpers."""
|
|
||||||
|
|
||||||
from types import SimpleNamespace
|
|
||||||
from unittest.mock import AsyncMock, MagicMock, patch
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from gateway.config import Platform
|
|
||||||
from gateway.platforms.base import MessageEvent
|
|
||||||
from gateway.session import SessionSource
|
|
||||||
|
|
||||||
|
|
||||||
def _make_runner():
|
|
||||||
from gateway.run import GatewayRunner
|
|
||||||
|
|
||||||
runner = object.__new__(GatewayRunner)
|
|
||||||
runner._honcho_managers = {}
|
|
||||||
runner._honcho_configs = {}
|
|
||||||
runner._running_agents = {}
|
|
||||||
runner._pending_messages = {}
|
|
||||||
runner._pending_approvals = {}
|
|
||||||
runner.adapters = {}
|
|
||||||
runner.hooks = MagicMock()
|
|
||||||
runner.hooks.emit = AsyncMock()
|
|
||||||
return runner
|
|
||||||
|
|
||||||
|
|
||||||
def _make_event(text="/reset"):
|
|
||||||
return MessageEvent(
|
|
||||||
text=text,
|
|
||||||
source=SessionSource(
|
|
||||||
platform=Platform.TELEGRAM,
|
|
||||||
chat_id="chat-1",
|
|
||||||
user_id="user-1",
|
|
||||||
user_name="alice",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class TestGatewayHonchoLifecycle:
|
|
||||||
def test_gateway_reuses_honcho_manager_for_session_key(self):
|
|
||||||
runner = _make_runner()
|
|
||||||
hcfg = SimpleNamespace(
|
|
||||||
enabled=True,
|
|
||||||
api_key="honcho-key",
|
|
||||||
ai_peer="hermes",
|
|
||||||
peer_name="alice",
|
|
||||||
context_tokens=123,
|
|
||||||
peer_memory_mode=lambda peer: "hybrid",
|
|
||||||
)
|
|
||||||
manager = MagicMock()
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch("honcho_integration.client.HonchoClientConfig.from_global_config", return_value=hcfg),
|
|
||||||
patch("honcho_integration.client.get_honcho_client", return_value=MagicMock()),
|
|
||||||
patch("honcho_integration.session.HonchoSessionManager", return_value=manager) as mock_mgr_cls,
|
|
||||||
):
|
|
||||||
first_mgr, first_cfg = runner._get_or_create_gateway_honcho("session-key")
|
|
||||||
second_mgr, second_cfg = runner._get_or_create_gateway_honcho("session-key")
|
|
||||||
|
|
||||||
assert first_mgr is manager
|
|
||||||
assert second_mgr is manager
|
|
||||||
assert first_cfg is hcfg
|
|
||||||
assert second_cfg is hcfg
|
|
||||||
mock_mgr_cls.assert_called_once()
|
|
||||||
|
|
||||||
def test_gateway_skips_honcho_manager_when_disabled(self):
|
|
||||||
runner = _make_runner()
|
|
||||||
hcfg = SimpleNamespace(
|
|
||||||
enabled=False,
|
|
||||||
api_key="honcho-key",
|
|
||||||
ai_peer="hermes",
|
|
||||||
peer_name="alice",
|
|
||||||
)
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch("honcho_integration.client.HonchoClientConfig.from_global_config", return_value=hcfg),
|
|
||||||
patch("honcho_integration.client.get_honcho_client") as mock_client,
|
|
||||||
patch("honcho_integration.session.HonchoSessionManager") as mock_mgr_cls,
|
|
||||||
):
|
|
||||||
manager, cfg = runner._get_or_create_gateway_honcho("session-key")
|
|
||||||
|
|
||||||
assert manager is None
|
|
||||||
assert cfg is hcfg
|
|
||||||
mock_client.assert_not_called()
|
|
||||||
mock_mgr_cls.assert_not_called()
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_reset_shuts_down_gateway_honcho_manager(self):
|
|
||||||
runner = _make_runner()
|
|
||||||
event = _make_event()
|
|
||||||
runner._shutdown_gateway_honcho = MagicMock()
|
|
||||||
runner._async_flush_memories = AsyncMock()
|
|
||||||
runner.session_store = MagicMock()
|
|
||||||
runner.session_store._generate_session_key.return_value = "gateway-key"
|
|
||||||
runner.session_store._entries = {
|
|
||||||
"gateway-key": SimpleNamespace(session_id="old-session"),
|
|
||||||
}
|
|
||||||
runner.session_store.reset_session.return_value = SimpleNamespace(session_id="new-session")
|
|
||||||
|
|
||||||
result = await runner._handle_reset_command(event)
|
|
||||||
|
|
||||||
runner._shutdown_gateway_honcho.assert_called_once_with("gateway-key")
|
|
||||||
runner._async_flush_memories.assert_called_once_with("old-session", "gateway-key")
|
|
||||||
assert "Session reset" in result
|
|
||||||
|
|
||||||
def test_flush_memories_reuses_gateway_session_key_and_skips_honcho_sync(self):
|
|
||||||
runner = _make_runner()
|
|
||||||
runner.session_store = MagicMock()
|
|
||||||
runner.session_store.load_transcript.return_value = [
|
|
||||||
{"role": "user", "content": "a"},
|
|
||||||
{"role": "assistant", "content": "b"},
|
|
||||||
{"role": "user", "content": "c"},
|
|
||||||
{"role": "assistant", "content": "d"},
|
|
||||||
]
|
|
||||||
tmp_agent = MagicMock()
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch("gateway.run._resolve_runtime_agent_kwargs", return_value={"api_key": "test-key"}),
|
|
||||||
patch("gateway.run._resolve_gateway_model", return_value="model-name"),
|
|
||||||
patch("run_agent.AIAgent", return_value=tmp_agent) as mock_agent_cls,
|
|
||||||
):
|
|
||||||
runner._flush_memories_for_session("old-session", "gateway-key")
|
|
||||||
|
|
||||||
mock_agent_cls.assert_called_once()
|
|
||||||
_, kwargs = mock_agent_cls.call_args
|
|
||||||
assert kwargs["session_id"] == "old-session"
|
|
||||||
assert kwargs["honcho_session_key"] == "gateway-key"
|
|
||||||
tmp_agent.run_conversation.assert_called_once()
|
|
||||||
_, run_kwargs = tmp_agent.run_conversation.call_args
|
|
||||||
assert run_kwargs["sync_honcho"] is False
|
|
||||||
|
|
@ -58,7 +58,7 @@ class TestHonchoDoctorConfigDetection:
|
||||||
fake_config = SimpleNamespace(enabled=True, api_key="***")
|
fake_config = SimpleNamespace(enabled=True, api_key="***")
|
||||||
|
|
||||||
monkeypatch.setattr(
|
monkeypatch.setattr(
|
||||||
"honcho_integration.client.HonchoClientConfig.from_global_config",
|
"plugins.memory.honcho.client.HonchoClientConfig.from_global_config",
|
||||||
lambda: fake_config,
|
lambda: fake_config,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -68,7 +68,7 @@ class TestHonchoDoctorConfigDetection:
|
||||||
fake_config = SimpleNamespace(enabled=True, api_key="")
|
fake_config = SimpleNamespace(enabled=True, api_key="")
|
||||||
|
|
||||||
monkeypatch.setattr(
|
monkeypatch.setattr(
|
||||||
"honcho_integration.client.HonchoClientConfig.from_global_config",
|
"plugins.memory.honcho.client.HonchoClientConfig.from_global_config",
|
||||||
lambda: fake_config,
|
lambda: fake_config,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,179 +0,0 @@
|
||||||
"""Tests for Honcho CLI helpers."""
|
|
||||||
|
|
||||||
import json
|
|
||||||
from unittest.mock import patch
|
|
||||||
|
|
||||||
from honcho_integration.cli import _resolve_api_key, clone_honcho_for_profile, sync_honcho_profiles_quiet
|
|
||||||
|
|
||||||
|
|
||||||
class TestResolveApiKey:
|
|
||||||
def test_prefers_host_scoped_key(self):
|
|
||||||
cfg = {
|
|
||||||
"apiKey": "root-key",
|
|
||||||
"hosts": {
|
|
||||||
"hermes": {
|
|
||||||
"apiKey": "host-key",
|
|
||||||
}
|
|
||||||
},
|
|
||||||
}
|
|
||||||
assert _resolve_api_key(cfg) == "host-key"
|
|
||||||
|
|
||||||
def test_falls_back_to_root_key(self):
|
|
||||||
cfg = {
|
|
||||||
"apiKey": "root-key",
|
|
||||||
"hosts": {"hermes": {}},
|
|
||||||
}
|
|
||||||
assert _resolve_api_key(cfg) == "root-key"
|
|
||||||
|
|
||||||
def test_falls_back_to_env_key(self, monkeypatch):
|
|
||||||
monkeypatch.setenv("HONCHO_API_KEY", "env-key")
|
|
||||||
assert _resolve_api_key({}) == "env-key"
|
|
||||||
monkeypatch.delenv("HONCHO_API_KEY", raising=False)
|
|
||||||
|
|
||||||
|
|
||||||
class TestCloneHonchoForProfile:
|
|
||||||
def test_clones_default_settings_to_new_profile(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text(json.dumps({
|
|
||||||
"apiKey": "test-key",
|
|
||||||
"hosts": {
|
|
||||||
"hermes": {
|
|
||||||
"peerName": "alice",
|
|
||||||
"memoryMode": "honcho",
|
|
||||||
"recallMode": "tools",
|
|
||||||
"writeFrequency": "turn",
|
|
||||||
"dialecticReasoningLevel": "medium",
|
|
||||||
"enabled": True,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}))
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("honcho_integration.cli._local_config_path", return_value=config_file):
|
|
||||||
result = clone_honcho_for_profile("coder")
|
|
||||||
|
|
||||||
assert result is True
|
|
||||||
|
|
||||||
cfg = json.loads(config_file.read_text())
|
|
||||||
new_block = cfg["hosts"]["hermes.coder"]
|
|
||||||
assert new_block["peerName"] == "alice"
|
|
||||||
assert new_block["memoryMode"] == "honcho"
|
|
||||||
assert new_block["recallMode"] == "tools"
|
|
||||||
assert new_block["writeFrequency"] == "turn"
|
|
||||||
assert new_block["aiPeer"] == "hermes.coder"
|
|
||||||
assert new_block["workspace"] == "hermes" # shared, not profile-derived
|
|
||||||
assert new_block["enabled"] is True
|
|
||||||
|
|
||||||
def test_skips_when_no_honcho_configured(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text("{}")
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("honcho_integration.cli._local_config_path", return_value=config_file):
|
|
||||||
result = clone_honcho_for_profile("coder")
|
|
||||||
|
|
||||||
assert result is False
|
|
||||||
|
|
||||||
def test_skips_when_host_block_already_exists(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text(json.dumps({
|
|
||||||
"apiKey": "key",
|
|
||||||
"hosts": {
|
|
||||||
"hermes": {"peerName": "alice"},
|
|
||||||
"hermes.coder": {"peerName": "existing"},
|
|
||||||
},
|
|
||||||
}))
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("honcho_integration.cli._local_config_path", return_value=config_file):
|
|
||||||
result = clone_honcho_for_profile("coder")
|
|
||||||
|
|
||||||
assert result is False
|
|
||||||
cfg = json.loads(config_file.read_text())
|
|
||||||
assert cfg["hosts"]["hermes.coder"]["peerName"] == "existing"
|
|
||||||
|
|
||||||
def test_inherits_peer_name_from_root_when_not_in_host(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text(json.dumps({
|
|
||||||
"apiKey": "key",
|
|
||||||
"peerName": "root-alice",
|
|
||||||
"hosts": {"hermes": {}},
|
|
||||||
}))
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("honcho_integration.cli._local_config_path", return_value=config_file):
|
|
||||||
clone_honcho_for_profile("dreamer")
|
|
||||||
|
|
||||||
cfg = json.loads(config_file.read_text())
|
|
||||||
assert cfg["hosts"]["hermes.dreamer"]["peerName"] == "root-alice"
|
|
||||||
|
|
||||||
def test_works_with_api_key_only_no_host_block(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text(json.dumps({"apiKey": "key"}))
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("honcho_integration.cli._local_config_path", return_value=config_file):
|
|
||||||
result = clone_honcho_for_profile("coder")
|
|
||||||
|
|
||||||
assert result is True
|
|
||||||
cfg = json.loads(config_file.read_text())
|
|
||||||
assert cfg["hosts"]["hermes.coder"]["aiPeer"] == "hermes.coder"
|
|
||||||
assert cfg["hosts"]["hermes.coder"]["workspace"] == "hermes" # shared
|
|
||||||
|
|
||||||
|
|
||||||
class TestSyncHonchoProfilesQuiet:
|
|
||||||
def test_syncs_missing_profiles(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text(json.dumps({
|
|
||||||
"apiKey": "key",
|
|
||||||
"hosts": {"hermes": {"peerName": "alice", "memoryMode": "honcho"}},
|
|
||||||
}))
|
|
||||||
|
|
||||||
class FakeProfile:
|
|
||||||
def __init__(self, name):
|
|
||||||
self.name = name
|
|
||||||
self.is_default = name == "default"
|
|
||||||
|
|
||||||
profiles = [FakeProfile("default"), FakeProfile("coder"), FakeProfile("dreamer")]
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("honcho_integration.cli._local_config_path", return_value=config_file), \
|
|
||||||
patch("hermes_cli.profiles.list_profiles", return_value=profiles):
|
|
||||||
count = sync_honcho_profiles_quiet()
|
|
||||||
|
|
||||||
assert count == 2
|
|
||||||
cfg = json.loads(config_file.read_text())
|
|
||||||
assert "hermes.coder" in cfg["hosts"]
|
|
||||||
assert "hermes.dreamer" in cfg["hosts"]
|
|
||||||
|
|
||||||
def test_returns_zero_when_no_honcho(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text("{}")
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("honcho_integration.cli._local_config_path", return_value=config_file):
|
|
||||||
count = sync_honcho_profiles_quiet()
|
|
||||||
|
|
||||||
assert count == 0
|
|
||||||
|
|
||||||
def test_skips_already_synced(self, tmp_path):
|
|
||||||
config_file = tmp_path / "config.json"
|
|
||||||
config_file.write_text(json.dumps({
|
|
||||||
"apiKey": "key",
|
|
||||||
"hosts": {
|
|
||||||
"hermes": {"peerName": "alice"},
|
|
||||||
"hermes.coder": {"peerName": "existing"},
|
|
||||||
},
|
|
||||||
}))
|
|
||||||
|
|
||||||
class FakeProfile:
|
|
||||||
def __init__(self, name):
|
|
||||||
self.name = name
|
|
||||||
self.is_default = name == "default"
|
|
||||||
|
|
||||||
with patch("honcho_integration.cli._config_path", return_value=config_file), \
|
|
||||||
patch("hermes_cli.profiles.list_profiles", return_value=[FakeProfile("default"), FakeProfile("coder")]):
|
|
||||||
count = sync_honcho_profiles_quiet()
|
|
||||||
|
|
||||||
assert count == 0
|
|
||||||
|
|
||||||
|
|
@ -1,190 +0,0 @@
|
||||||
"""Tests for Honcho config profile isolation.
|
|
||||||
|
|
||||||
Verifies that each Hermes profile writes to its own instance-local
|
|
||||||
honcho.json ($HERMES_HOME/honcho.json) rather than the shared global
|
|
||||||
~/.honcho/config.json.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
from unittest.mock import patch
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from honcho_integration.cli import (
|
|
||||||
_config_path,
|
|
||||||
_local_config_path,
|
|
||||||
_read_config,
|
|
||||||
_write_config,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def isolated_home(tmp_path, monkeypatch):
|
|
||||||
"""Create an isolated HERMES_HOME + real home for testing."""
|
|
||||||
hermes_home = tmp_path / "profile_a"
|
|
||||||
hermes_home.mkdir()
|
|
||||||
global_dir = tmp_path / "home" / ".honcho"
|
|
||||||
global_dir.mkdir(parents=True)
|
|
||||||
global_config = global_dir / "config.json"
|
|
||||||
|
|
||||||
monkeypatch.setenv("HERMES_HOME", str(hermes_home))
|
|
||||||
monkeypatch.setattr(Path, "home", staticmethod(lambda: tmp_path / "home"))
|
|
||||||
# GLOBAL_CONFIG_PATH is a module-level constant cached at import time,
|
|
||||||
# so we must patch it in both the defining module and the importing module.
|
|
||||||
import honcho_integration.client as _client_mod
|
|
||||||
import honcho_integration.cli as _cli_mod
|
|
||||||
monkeypatch.setattr(_client_mod, "GLOBAL_CONFIG_PATH", global_config)
|
|
||||||
monkeypatch.setattr(_cli_mod, "GLOBAL_CONFIG_PATH", global_config)
|
|
||||||
|
|
||||||
return {
|
|
||||||
"hermes_home": hermes_home,
|
|
||||||
"global_config": global_config,
|
|
||||||
"local_config": hermes_home / "honcho.json",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class TestLocalConfigPath:
|
|
||||||
"""_local_config_path always returns $HERMES_HOME/honcho.json."""
|
|
||||||
|
|
||||||
def test_returns_hermes_home_path(self, isolated_home):
|
|
||||||
assert _local_config_path() == isolated_home["local_config"]
|
|
||||||
|
|
||||||
def test_differs_from_global(self, isolated_home):
|
|
||||||
from honcho_integration.client import GLOBAL_CONFIG_PATH
|
|
||||||
assert _local_config_path() != GLOBAL_CONFIG_PATH
|
|
||||||
|
|
||||||
|
|
||||||
class TestWriteConfigIsolation:
|
|
||||||
"""_write_config defaults to the instance-local path."""
|
|
||||||
|
|
||||||
def test_write_creates_local_file(self, isolated_home):
|
|
||||||
cfg = {"apiKey": "test-key", "hosts": {"hermes": {"enabled": True}}}
|
|
||||||
_write_config(cfg)
|
|
||||||
|
|
||||||
assert isolated_home["local_config"].exists()
|
|
||||||
written = json.loads(isolated_home["local_config"].read_text())
|
|
||||||
assert written["apiKey"] == "test-key"
|
|
||||||
|
|
||||||
def test_write_does_not_touch_global(self, isolated_home):
|
|
||||||
# Pre-populate global config
|
|
||||||
isolated_home["global_config"].write_text(
|
|
||||||
json.dumps({"apiKey": "global-key"})
|
|
||||||
)
|
|
||||||
|
|
||||||
cfg = {"apiKey": "profile-key"}
|
|
||||||
_write_config(cfg)
|
|
||||||
|
|
||||||
# Global should be untouched
|
|
||||||
global_data = json.loads(isolated_home["global_config"].read_text())
|
|
||||||
assert global_data["apiKey"] == "global-key"
|
|
||||||
|
|
||||||
# Local should have the new value
|
|
||||||
local_data = json.loads(isolated_home["local_config"].read_text())
|
|
||||||
assert local_data["apiKey"] == "profile-key"
|
|
||||||
|
|
||||||
def test_explicit_path_override_still_works(self, isolated_home):
|
|
||||||
custom = isolated_home["hermes_home"] / "custom.json"
|
|
||||||
_write_config({"custom": True}, path=custom)
|
|
||||||
assert custom.exists()
|
|
||||||
assert not isolated_home["local_config"].exists()
|
|
||||||
|
|
||||||
|
|
||||||
class TestReadConfigFallback:
|
|
||||||
"""_read_config falls back to global when no local file exists."""
|
|
||||||
|
|
||||||
def test_reads_local_when_exists(self, isolated_home):
|
|
||||||
isolated_home["local_config"].write_text(
|
|
||||||
json.dumps({"source": "local"})
|
|
||||||
)
|
|
||||||
cfg = _read_config()
|
|
||||||
assert cfg["source"] == "local"
|
|
||||||
|
|
||||||
def test_falls_back_to_global(self, isolated_home):
|
|
||||||
isolated_home["global_config"].write_text(
|
|
||||||
json.dumps({"source": "global"})
|
|
||||||
)
|
|
||||||
# No local file exists
|
|
||||||
assert not isolated_home["local_config"].exists()
|
|
||||||
cfg = _read_config()
|
|
||||||
assert cfg["source"] == "global"
|
|
||||||
|
|
||||||
def test_local_takes_priority_over_global(self, isolated_home):
|
|
||||||
isolated_home["local_config"].write_text(
|
|
||||||
json.dumps({"source": "local"})
|
|
||||||
)
|
|
||||||
isolated_home["global_config"].write_text(
|
|
||||||
json.dumps({"source": "global"})
|
|
||||||
)
|
|
||||||
cfg = _read_config()
|
|
||||||
assert cfg["source"] == "local"
|
|
||||||
|
|
||||||
|
|
||||||
class TestMultiProfileIsolation:
|
|
||||||
"""Two profiles writing config don't interfere with each other."""
|
|
||||||
|
|
||||||
def test_two_profiles_get_separate_configs(self, tmp_path, monkeypatch):
|
|
||||||
home = tmp_path / "home"
|
|
||||||
home.mkdir()
|
|
||||||
monkeypatch.setattr(Path, "home", staticmethod(lambda: home))
|
|
||||||
|
|
||||||
profile_a = tmp_path / "profile_a"
|
|
||||||
profile_b = tmp_path / "profile_b"
|
|
||||||
profile_a.mkdir()
|
|
||||||
profile_b.mkdir()
|
|
||||||
|
|
||||||
# Profile A writes its config
|
|
||||||
monkeypatch.setenv("HERMES_HOME", str(profile_a))
|
|
||||||
_write_config({"apiKey": "key-a", "hosts": {"hermes": {"peerName": "alice"}}})
|
|
||||||
|
|
||||||
# Profile B writes its config
|
|
||||||
monkeypatch.setenv("HERMES_HOME", str(profile_b))
|
|
||||||
_write_config({"apiKey": "key-b", "hosts": {"hermes": {"peerName": "bob"}}})
|
|
||||||
|
|
||||||
# Verify isolation
|
|
||||||
a_data = json.loads((profile_a / "honcho.json").read_text())
|
|
||||||
b_data = json.loads((profile_b / "honcho.json").read_text())
|
|
||||||
|
|
||||||
assert a_data["hosts"]["hermes"]["peerName"] == "alice"
|
|
||||||
assert b_data["hosts"]["hermes"]["peerName"] == "bob"
|
|
||||||
|
|
||||||
def test_first_setup_seeds_from_global(self, tmp_path, monkeypatch):
|
|
||||||
"""First setup reads global config, writes to local."""
|
|
||||||
home = tmp_path / "home"
|
|
||||||
global_dir = home / ".honcho"
|
|
||||||
global_dir.mkdir(parents=True)
|
|
||||||
monkeypatch.setattr(Path, "home", staticmethod(lambda: home))
|
|
||||||
import honcho_integration.client as _client_mod
|
|
||||||
import honcho_integration.cli as _cli_mod
|
|
||||||
global_cfg_path = global_dir / "config.json"
|
|
||||||
monkeypatch.setattr(_client_mod, "GLOBAL_CONFIG_PATH", global_cfg_path)
|
|
||||||
monkeypatch.setattr(_cli_mod, "GLOBAL_CONFIG_PATH", global_cfg_path)
|
|
||||||
|
|
||||||
# Existing global config
|
|
||||||
global_config = global_dir / "config.json"
|
|
||||||
global_config.write_text(json.dumps({
|
|
||||||
"apiKey": "shared-key",
|
|
||||||
"hosts": {"hermes": {"workspace": "shared-ws"}},
|
|
||||||
}))
|
|
||||||
|
|
||||||
profile = tmp_path / "new_profile"
|
|
||||||
profile.mkdir()
|
|
||||||
monkeypatch.setenv("HERMES_HOME", str(profile))
|
|
||||||
|
|
||||||
# Read seeds from global
|
|
||||||
cfg = _read_config()
|
|
||||||
assert cfg["apiKey"] == "shared-key"
|
|
||||||
|
|
||||||
# Modify and write goes to local
|
|
||||||
cfg["hosts"]["hermes"]["peerName"] = "new-user"
|
|
||||||
_write_config(cfg)
|
|
||||||
|
|
||||||
local_config = profile / "honcho.json"
|
|
||||||
assert local_config.exists()
|
|
||||||
local_data = json.loads(local_config.read_text())
|
|
||||||
assert local_data["hosts"]["hermes"]["peerName"] == "new-user"
|
|
||||||
|
|
||||||
# Global unchanged
|
|
||||||
global_data = json.loads(global_config.read_text())
|
|
||||||
assert "peerName" not in global_data["hosts"]["hermes"]
|
|
||||||
|
|
@ -20,8 +20,8 @@ from unittest.mock import MagicMock, patch, call
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from honcho_integration.client import HonchoClientConfig
|
from plugins.memory.honcho.client import HonchoClientConfig
|
||||||
from honcho_integration.session import (
|
from plugins.memory.honcho.session import (
|
||||||
HonchoSession,
|
HonchoSession,
|
||||||
HonchoSessionManager,
|
HonchoSessionManager,
|
||||||
_ASYNC_SHUTDOWN,
|
_ASYNC_SHUTDOWN,
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
"""Tests for honcho_integration/client.py — Honcho client configuration."""
|
"""Tests for plugins/memory/honcho/client.py — Honcho client configuration."""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
|
@ -7,7 +7,7 @@ from unittest.mock import patch, MagicMock
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from honcho_integration.client import (
|
from plugins.memory.honcho.client import (
|
||||||
HonchoClientConfig,
|
HonchoClientConfig,
|
||||||
get_honcho_client,
|
get_honcho_client,
|
||||||
reset_honcho_client,
|
reset_honcho_client,
|
||||||
|
|
@ -461,7 +461,7 @@ class TestProfileScopedConfig:
|
||||||
"hermes.dreamer": {"peerName": "dreamer-user"},
|
"hermes.dreamer": {"peerName": "dreamer-user"},
|
||||||
},
|
},
|
||||||
}))
|
}))
|
||||||
with patch("honcho_integration.client.resolve_active_host", return_value="hermes.dreamer"):
|
with patch("plugins.memory.honcho.client.resolve_active_host", return_value="hermes.dreamer"):
|
||||||
config = HonchoClientConfig.from_global_config(config_path=config_file)
|
config = HonchoClientConfig.from_global_config(config_path=config_file)
|
||||||
assert config.host == "hermes.dreamer"
|
assert config.host == "hermes.dreamer"
|
||||||
assert config.peer_name == "dreamer-user"
|
assert config.peer_name == "dreamer-user"
|
||||||
|
|
@ -469,7 +469,7 @@ class TestProfileScopedConfig:
|
||||||
|
|
||||||
class TestResetHonchoClient:
|
class TestResetHonchoClient:
|
||||||
def test_reset_clears_singleton(self):
|
def test_reset_clears_singleton(self):
|
||||||
import honcho_integration.client as mod
|
import plugins.memory.honcho.client as mod
|
||||||
mod._honcho_client = MagicMock()
|
mod._honcho_client = MagicMock()
|
||||||
assert mod._honcho_client is not None
|
assert mod._honcho_client is not None
|
||||||
reset_honcho_client()
|
reset_honcho_client()
|
||||||
|
|
@ -1,9 +1,9 @@
|
||||||
"""Tests for honcho_integration/session.py — HonchoSession and helpers."""
|
"""Tests for plugins/memory/honcho/session.py — HonchoSession and helpers."""
|
||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from unittest.mock import MagicMock
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
from honcho_integration.session import (
|
from plugins.memory.honcho.session import (
|
||||||
HonchoSession,
|
HonchoSession,
|
||||||
HonchoSessionManager,
|
HonchoSessionManager,
|
||||||
)
|
)
|
||||||
|
|
@ -4,10 +4,41 @@ import types
|
||||||
from contextlib import nullcontext
|
from contextlib import nullcontext
|
||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
from hermes_cli.auth import AuthError
|
from hermes_cli.auth import AuthError
|
||||||
from hermes_cli import main as hermes_main
|
from hermes_cli import main as hermes_main
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Module isolation: _import_cli() wipes tools.* / cli / run_agent from
|
||||||
|
# sys.modules so it can re-import cli fresh. Without cleanup the wiped
|
||||||
|
# modules leak into subsequent tests on the same xdist worker, breaking
|
||||||
|
# mock patches that target "tools.file_tools._get_file_ops" etc.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _reset_modules(prefixes: tuple[str, ...]):
|
||||||
|
for name in list(sys.modules):
|
||||||
|
if any(name == p or name.startswith(p + ".") for p in prefixes):
|
||||||
|
sys.modules.pop(name, None)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _restore_cli_and_tool_modules():
|
||||||
|
"""Save and restore tools/cli/run_agent modules around every test."""
|
||||||
|
prefixes = ("tools", "cli", "run_agent")
|
||||||
|
original_modules = {
|
||||||
|
name: module
|
||||||
|
for name, module in sys.modules.items()
|
||||||
|
if any(name == p or name.startswith(p + ".") for p in prefixes)
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
_reset_modules(prefixes)
|
||||||
|
sys.modules.update(original_modules)
|
||||||
|
|
||||||
|
|
||||||
def _install_prompt_toolkit_stubs():
|
def _install_prompt_toolkit_stubs():
|
||||||
class _Dummy:
|
class _Dummy:
|
||||||
def __init__(self, *args, **kwargs):
|
def __init__(self, *args, **kwargs):
|
||||||
|
|
|
||||||
|
|
@ -13,38 +13,6 @@ from unittest.mock import MagicMock, patch, call
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
class TestHonchoAtexitFlush:
|
|
||||||
"""run_agent.py — _register_honcho_exit_hook atexit handler."""
|
|
||||||
|
|
||||||
def test_keyboard_interrupt_during_flush_does_not_propagate(self):
|
|
||||||
"""The atexit handler must swallow KeyboardInterrupt from flush_all()."""
|
|
||||||
mock_manager = MagicMock()
|
|
||||||
mock_manager.flush_all.side_effect = KeyboardInterrupt
|
|
||||||
|
|
||||||
# Capture functions passed to atexit.register
|
|
||||||
registered_fns = []
|
|
||||||
original_register = atexit.register
|
|
||||||
|
|
||||||
def capturing_register(fn, *args, **kwargs):
|
|
||||||
registered_fns.append(fn)
|
|
||||||
# Don't actually register — we don't want side effects
|
|
||||||
|
|
||||||
with patch("atexit.register", side_effect=capturing_register):
|
|
||||||
from run_agent import AIAgent
|
|
||||||
agent = object.__new__(AIAgent)
|
|
||||||
agent._honcho = mock_manager
|
|
||||||
agent._honcho_exit_hook_registered = False
|
|
||||||
agent._register_honcho_exit_hook()
|
|
||||||
|
|
||||||
# Our handler is the last one registered
|
|
||||||
assert len(registered_fns) >= 1, "atexit handler was not registered"
|
|
||||||
flush_handler = registered_fns[-1]
|
|
||||||
|
|
||||||
# Invoke the registered handler — must not raise
|
|
||||||
flush_handler()
|
|
||||||
mock_manager.flush_all.assert_called_once()
|
|
||||||
|
|
||||||
|
|
||||||
class TestCronJobCleanup:
|
class TestCronJobCleanup:
|
||||||
"""cron/scheduler.py — end_session + close in the finally block."""
|
"""cron/scheduler.py — end_session + close in the finally block."""
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,7 @@ from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from honcho_integration.client import HonchoClientConfig
|
from plugins.memory.honcho.client import HonchoClientConfig
|
||||||
|
|
||||||
|
|
||||||
class TestHonchoClientConfigAutoEnable:
|
class TestHonchoClientConfigAutoEnable:
|
||||||
|
|
|
||||||
|
|
@ -17,8 +17,7 @@ from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
import run_agent
|
import run_agent
|
||||||
from honcho_integration.client import HonchoClientConfig
|
from run_agent import AIAgent
|
||||||
from run_agent import AIAgent, _inject_honcho_turn_context
|
|
||||||
from agent.prompt_builder import DEFAULT_AGENT_IDENTITY
|
from agent.prompt_builder import DEFAULT_AGENT_IDENTITY
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1254,8 +1253,7 @@ class TestConcurrentToolExecution:
|
||||||
mock_hfc.assert_called_once_with(
|
mock_hfc.assert_called_once_with(
|
||||||
"web_search", {"q": "test"}, "task-1",
|
"web_search", {"q": "test"}, "task-1",
|
||||||
enabled_tools=list(agent.valid_tool_names),
|
enabled_tools=list(agent.valid_tool_names),
|
||||||
honcho_manager=None,
|
|
||||||
honcho_session_key=None,
|
|
||||||
)
|
)
|
||||||
assert result == "result"
|
assert result == "result"
|
||||||
|
|
||||||
|
|
@ -2193,305 +2191,6 @@ class TestSystemPromptStability:
|
||||||
# Empty string is falsy, so should fall through to fresh build
|
# Empty string is falsy, so should fall through to fresh build
|
||||||
assert "Hermes Agent" in agent._cached_system_prompt
|
assert "Hermes Agent" in agent._cached_system_prompt
|
||||||
|
|
||||||
def test_honcho_context_baked_into_prompt_on_first_turn(self, agent):
|
|
||||||
"""Honcho context should be baked into _cached_system_prompt on
|
|
||||||
the first turn, not injected separately per API call."""
|
|
||||||
agent._honcho_context = "User prefers Python over JavaScript."
|
|
||||||
agent._cached_system_prompt = None
|
|
||||||
|
|
||||||
# Simulate first turn: build fresh and bake in Honcho
|
|
||||||
agent._cached_system_prompt = agent._build_system_prompt()
|
|
||||||
if agent._honcho_context:
|
|
||||||
agent._cached_system_prompt = (
|
|
||||||
agent._cached_system_prompt + "\n\n" + agent._honcho_context
|
|
||||||
).strip()
|
|
||||||
|
|
||||||
assert "User prefers Python over JavaScript" in agent._cached_system_prompt
|
|
||||||
|
|
||||||
def test_honcho_prefetch_runs_on_continuing_session(self):
|
|
||||||
"""Honcho prefetch is consumed on continuing sessions via ephemeral context."""
|
|
||||||
conversation_history = [
|
|
||||||
{"role": "user", "content": "hello"},
|
|
||||||
{"role": "assistant", "content": "hi there"},
|
|
||||||
]
|
|
||||||
recall_mode = "hybrid"
|
|
||||||
should_prefetch = bool(conversation_history) and recall_mode != "tools"
|
|
||||||
assert should_prefetch is True
|
|
||||||
|
|
||||||
def test_inject_honcho_turn_context_appends_system_note(self):
|
|
||||||
content = _inject_honcho_turn_context("hello", "## Honcho Memory\nprior context")
|
|
||||||
assert "hello" in content
|
|
||||||
assert "Honcho memory was retrieved from prior sessions" in content
|
|
||||||
assert "## Honcho Memory" in content
|
|
||||||
|
|
||||||
def test_honcho_continuing_session_keeps_turn_context_out_of_system_prompt(self, agent):
|
|
||||||
captured = {}
|
|
||||||
|
|
||||||
def _fake_api_call(api_kwargs):
|
|
||||||
captured.update(api_kwargs)
|
|
||||||
return _mock_response(content="done", finish_reason="stop")
|
|
||||||
|
|
||||||
agent._honcho = object()
|
|
||||||
agent._honcho_session_key = "session-1"
|
|
||||||
agent._honcho_config = SimpleNamespace(
|
|
||||||
ai_peer="hermes",
|
|
||||||
memory_mode="hybrid",
|
|
||||||
write_frequency="async",
|
|
||||||
recall_mode="hybrid",
|
|
||||||
)
|
|
||||||
agent._use_prompt_caching = False
|
|
||||||
conversation_history = [
|
|
||||||
{"role": "user", "content": "hello"},
|
|
||||||
{"role": "assistant", "content": "hi there"},
|
|
||||||
]
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch.object(agent, "_honcho_prefetch", return_value="## Honcho Memory\nprior context"),
|
|
||||||
patch.object(agent, "_queue_honcho_prefetch"),
|
|
||||||
patch.object(agent, "_persist_session"),
|
|
||||||
patch.object(agent, "_save_trajectory"),
|
|
||||||
patch.object(agent, "_cleanup_task_resources"),
|
|
||||||
patch.object(agent, "_interruptible_api_call", side_effect=_fake_api_call),
|
|
||||||
):
|
|
||||||
result = agent.run_conversation("what were we doing?", conversation_history=conversation_history)
|
|
||||||
|
|
||||||
assert result["completed"] is True
|
|
||||||
api_messages = captured["messages"]
|
|
||||||
assert api_messages[0]["role"] == "system"
|
|
||||||
assert "prior context" not in api_messages[0]["content"]
|
|
||||||
current_user = api_messages[-1]
|
|
||||||
assert current_user["role"] == "user"
|
|
||||||
assert "what were we doing?" in current_user["content"]
|
|
||||||
assert "prior context" in current_user["content"]
|
|
||||||
assert "Honcho memory was retrieved from prior sessions" in current_user["content"]
|
|
||||||
|
|
||||||
def test_honcho_prefetch_runs_on_first_turn(self):
|
|
||||||
"""Honcho prefetch should run when conversation_history is empty."""
|
|
||||||
conversation_history = []
|
|
||||||
should_prefetch = not conversation_history
|
|
||||||
assert should_prefetch is True
|
|
||||||
|
|
||||||
def test_run_conversation_can_skip_honcho_sync_for_synthetic_turns(self, agent):
|
|
||||||
captured = {}
|
|
||||||
|
|
||||||
def _fake_api_call(api_kwargs):
|
|
||||||
captured.update(api_kwargs)
|
|
||||||
return _mock_response(content="done", finish_reason="stop")
|
|
||||||
|
|
||||||
agent._honcho = MagicMock()
|
|
||||||
agent._honcho_session_key = "session-1"
|
|
||||||
agent._honcho_config = SimpleNamespace(
|
|
||||||
ai_peer="hermes",
|
|
||||||
memory_mode="hybrid",
|
|
||||||
write_frequency="async",
|
|
||||||
recall_mode="hybrid",
|
|
||||||
)
|
|
||||||
agent._use_prompt_caching = False
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch.object(agent, "_honcho_sync") as mock_sync,
|
|
||||||
patch.object(agent, "_queue_honcho_prefetch") as mock_prefetch,
|
|
||||||
patch.object(agent, "_persist_session"),
|
|
||||||
patch.object(agent, "_save_trajectory"),
|
|
||||||
patch.object(agent, "_cleanup_task_resources"),
|
|
||||||
patch.object(agent, "_interruptible_api_call", side_effect=_fake_api_call),
|
|
||||||
):
|
|
||||||
result = agent.run_conversation("synthetic flush turn", sync_honcho=False)
|
|
||||||
|
|
||||||
assert result["completed"] is True
|
|
||||||
assert captured["messages"][-1]["content"] == "synthetic flush turn"
|
|
||||||
mock_sync.assert_not_called()
|
|
||||||
mock_prefetch.assert_not_called()
|
|
||||||
|
|
||||||
|
|
||||||
class TestHonchoActivation:
|
|
||||||
def test_disabled_config_skips_honcho_init(self):
|
|
||||||
hcfg = HonchoClientConfig(
|
|
||||||
enabled=False,
|
|
||||||
api_key="honcho-key",
|
|
||||||
peer_name="user",
|
|
||||||
ai_peer="hermes",
|
|
||||||
)
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch("run_agent.get_tool_definitions", return_value=_make_tool_defs("web_search")),
|
|
||||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
|
||||||
patch("run_agent.OpenAI"),
|
|
||||||
patch("honcho_integration.client.HonchoClientConfig.from_global_config", return_value=hcfg),
|
|
||||||
patch("honcho_integration.client.get_honcho_client") as mock_client,
|
|
||||||
):
|
|
||||||
agent = AIAgent(
|
|
||||||
api_key="test-key-1234567890",
|
|
||||||
quiet_mode=True,
|
|
||||||
skip_context_files=True,
|
|
||||||
skip_memory=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert agent._honcho is None
|
|
||||||
assert agent._honcho_config is hcfg
|
|
||||||
mock_client.assert_not_called()
|
|
||||||
|
|
||||||
def test_injected_honcho_manager_skips_fresh_client_init(self):
|
|
||||||
hcfg = HonchoClientConfig(
|
|
||||||
enabled=True,
|
|
||||||
api_key="honcho-key",
|
|
||||||
memory_mode="hybrid",
|
|
||||||
peer_name="user",
|
|
||||||
ai_peer="hermes",
|
|
||||||
recall_mode="hybrid",
|
|
||||||
)
|
|
||||||
manager = MagicMock()
|
|
||||||
manager._config = hcfg
|
|
||||||
manager.get_or_create.return_value = SimpleNamespace(messages=[])
|
|
||||||
manager.get_prefetch_context.return_value = {"representation": "Known user", "card": ""}
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch("run_agent.get_tool_definitions", return_value=_make_tool_defs("web_search")),
|
|
||||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
|
||||||
patch("run_agent.OpenAI"),
|
|
||||||
patch("honcho_integration.client.get_honcho_client") as mock_client,
|
|
||||||
patch("tools.honcho_tools.set_session_context"),
|
|
||||||
):
|
|
||||||
agent = AIAgent(
|
|
||||||
api_key="test-key-1234567890",
|
|
||||||
quiet_mode=True,
|
|
||||||
skip_context_files=True,
|
|
||||||
skip_memory=False,
|
|
||||||
honcho_session_key="gateway-session",
|
|
||||||
honcho_manager=manager,
|
|
||||||
honcho_config=hcfg,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert agent._honcho is manager
|
|
||||||
manager.get_or_create.assert_called_once_with("gateway-session")
|
|
||||||
manager.get_prefetch_context.assert_called_once_with("gateway-session")
|
|
||||||
manager.set_context_result.assert_called_once_with(
|
|
||||||
"gateway-session",
|
|
||||||
{"representation": "Known user", "card": ""},
|
|
||||||
)
|
|
||||||
mock_client.assert_not_called()
|
|
||||||
|
|
||||||
def test_recall_mode_context_suppresses_honcho_tools(self):
|
|
||||||
hcfg = HonchoClientConfig(
|
|
||||||
enabled=True,
|
|
||||||
api_key="honcho-key",
|
|
||||||
memory_mode="hybrid",
|
|
||||||
peer_name="user",
|
|
||||||
ai_peer="hermes",
|
|
||||||
recall_mode="context",
|
|
||||||
)
|
|
||||||
manager = MagicMock()
|
|
||||||
manager._config = hcfg
|
|
||||||
manager.get_or_create.return_value = SimpleNamespace(messages=[])
|
|
||||||
manager.get_prefetch_context.return_value = {"representation": "Known user", "card": ""}
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch(
|
|
||||||
"run_agent.get_tool_definitions",
|
|
||||||
side_effect=[
|
|
||||||
_make_tool_defs("web_search"),
|
|
||||||
_make_tool_defs(
|
|
||||||
"web_search",
|
|
||||||
"honcho_context",
|
|
||||||
"honcho_profile",
|
|
||||||
"honcho_search",
|
|
||||||
"honcho_conclude",
|
|
||||||
),
|
|
||||||
],
|
|
||||||
),
|
|
||||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
|
||||||
patch("run_agent.OpenAI"),
|
|
||||||
patch("tools.honcho_tools.set_session_context"),
|
|
||||||
):
|
|
||||||
agent = AIAgent(
|
|
||||||
api_key="test-key-1234567890",
|
|
||||||
quiet_mode=True,
|
|
||||||
skip_context_files=True,
|
|
||||||
skip_memory=False,
|
|
||||||
honcho_session_key="gateway-session",
|
|
||||||
honcho_manager=manager,
|
|
||||||
honcho_config=hcfg,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert "web_search" in agent.valid_tool_names
|
|
||||||
assert "honcho_context" not in agent.valid_tool_names
|
|
||||||
assert "honcho_profile" not in agent.valid_tool_names
|
|
||||||
assert "honcho_search" not in agent.valid_tool_names
|
|
||||||
assert "honcho_conclude" not in agent.valid_tool_names
|
|
||||||
|
|
||||||
def test_inactive_honcho_strips_stale_honcho_tools(self):
|
|
||||||
hcfg = HonchoClientConfig(
|
|
||||||
enabled=False,
|
|
||||||
api_key="honcho-key",
|
|
||||||
peer_name="user",
|
|
||||||
ai_peer="hermes",
|
|
||||||
)
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch("run_agent.get_tool_definitions", return_value=_make_tool_defs("web_search", "honcho_context")),
|
|
||||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
|
||||||
patch("run_agent.OpenAI"),
|
|
||||||
patch("honcho_integration.client.HonchoClientConfig.from_global_config", return_value=hcfg),
|
|
||||||
patch("honcho_integration.client.get_honcho_client") as mock_client,
|
|
||||||
):
|
|
||||||
agent = AIAgent(
|
|
||||||
api_key="test-key-1234567890",
|
|
||||||
quiet_mode=True,
|
|
||||||
skip_context_files=True,
|
|
||||||
skip_memory=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert agent._honcho is None
|
|
||||||
assert "web_search" in agent.valid_tool_names
|
|
||||||
assert "honcho_context" not in agent.valid_tool_names
|
|
||||||
mock_client.assert_not_called()
|
|
||||||
|
|
||||||
|
|
||||||
class TestHonchoPrefetchScheduling:
|
|
||||||
def test_honcho_prefetch_includes_cached_dialectic(self, agent):
|
|
||||||
agent._honcho = MagicMock()
|
|
||||||
agent._honcho_session_key = "session-key"
|
|
||||||
agent._honcho.pop_context_result.return_value = {}
|
|
||||||
agent._honcho.pop_dialectic_result.return_value = "Continue with the migration checklist."
|
|
||||||
|
|
||||||
context = agent._honcho_prefetch("what next?")
|
|
||||||
|
|
||||||
assert "Continuity synthesis" in context
|
|
||||||
assert "migration checklist" in context
|
|
||||||
|
|
||||||
def test_queue_honcho_prefetch_skips_tools_mode(self, agent):
|
|
||||||
agent._honcho = MagicMock()
|
|
||||||
agent._honcho_session_key = "session-key"
|
|
||||||
agent._honcho_config = HonchoClientConfig(
|
|
||||||
enabled=True,
|
|
||||||
api_key="honcho-key",
|
|
||||||
recall_mode="tools",
|
|
||||||
)
|
|
||||||
|
|
||||||
agent._queue_honcho_prefetch("what next?")
|
|
||||||
|
|
||||||
agent._honcho.prefetch_context.assert_not_called()
|
|
||||||
agent._honcho.prefetch_dialectic.assert_not_called()
|
|
||||||
|
|
||||||
def test_queue_honcho_prefetch_runs_when_context_enabled(self, agent):
|
|
||||||
agent._honcho = MagicMock()
|
|
||||||
agent._honcho_session_key = "session-key"
|
|
||||||
agent._honcho_config = HonchoClientConfig(
|
|
||||||
enabled=True,
|
|
||||||
api_key="honcho-key",
|
|
||||||
recall_mode="hybrid",
|
|
||||||
)
|
|
||||||
|
|
||||||
agent._queue_honcho_prefetch("what next?")
|
|
||||||
|
|
||||||
agent._honcho.prefetch_context.assert_called_once_with("session-key", "what next?")
|
|
||||||
agent._honcho.prefetch_dialectic.assert_called_once_with("session-key", "what next?")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Iteration budget pressure warnings
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestBudgetPressure:
|
class TestBudgetPressure:
|
||||||
"""Budget pressure warning system (issue #414)."""
|
"""Budget pressure warning system (issue #414)."""
|
||||||
|
|
||||||
|
|
@ -2629,38 +2328,8 @@ class TestSafeWriter:
|
||||||
sys.stdout = original_stdout
|
sys.stdout = original_stdout
|
||||||
sys.stderr = original_stderr
|
sys.stderr = original_stderr
|
||||||
|
|
||||||
def test_installed_before_init_time_honcho_error_prints(self):
|
# test_installed_before_init_time_honcho_error_prints removed —
|
||||||
"""AIAgent.__init__ wraps stdout before Honcho fallback prints can fire."""
|
# Honcho integration extracted to plugin (PR #4154).
|
||||||
import sys
|
|
||||||
from run_agent import _SafeWriter
|
|
||||||
|
|
||||||
broken = MagicMock()
|
|
||||||
broken.write.side_effect = OSError(5, "Input/output error")
|
|
||||||
broken.flush.side_effect = OSError(5, "Input/output error")
|
|
||||||
|
|
||||||
original = sys.stdout
|
|
||||||
sys.stdout = broken
|
|
||||||
try:
|
|
||||||
hcfg = HonchoClientConfig(enabled=True, api_key="test-honcho-key")
|
|
||||||
with (
|
|
||||||
patch("run_agent.get_tool_definitions", return_value=_make_tool_defs("web_search")),
|
|
||||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
|
||||||
patch("run_agent.OpenAI"),
|
|
||||||
patch("hermes_cli.config.load_config", return_value={"memory": {}}),
|
|
||||||
patch("honcho_integration.client.HonchoClientConfig.from_global_config", return_value=hcfg),
|
|
||||||
patch("honcho_integration.client.get_honcho_client", side_effect=RuntimeError("boom")),
|
|
||||||
):
|
|
||||||
agent = AIAgent(
|
|
||||||
api_key="test-k...7890",
|
|
||||||
quiet_mode=True,
|
|
||||||
skip_context_files=True,
|
|
||||||
skip_memory=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert isinstance(sys.stdout, _SafeWriter)
|
|
||||||
assert agent._honcho is None
|
|
||||||
finally:
|
|
||||||
sys.stdout = original
|
|
||||||
|
|
||||||
def test_double_wrap_prevented(self):
|
def test_double_wrap_prevented(self):
|
||||||
"""Wrapping an already-wrapped stream doesn't add layers."""
|
"""Wrapping an already-wrapped stream doesn't add layers."""
|
||||||
|
|
|
||||||
|
|
@ -1,111 +0,0 @@
|
||||||
"""Regression tests for per-call Honcho tool session routing."""
|
|
||||||
|
|
||||||
import json
|
|
||||||
from unittest.mock import MagicMock, patch
|
|
||||||
from dataclasses import dataclass
|
|
||||||
|
|
||||||
from tools import honcho_tools
|
|
||||||
|
|
||||||
|
|
||||||
class TestCheckHonchoAvailable:
|
|
||||||
"""Tests for _check_honcho_available (banner + runtime gating)."""
|
|
||||||
|
|
||||||
def setup_method(self):
|
|
||||||
self.orig_manager = honcho_tools._session_manager
|
|
||||||
self.orig_key = honcho_tools._session_key
|
|
||||||
|
|
||||||
def teardown_method(self):
|
|
||||||
honcho_tools._session_manager = self.orig_manager
|
|
||||||
honcho_tools._session_key = self.orig_key
|
|
||||||
|
|
||||||
def test_returns_true_when_session_active(self):
|
|
||||||
"""Fast path: session context already injected (mid-conversation)."""
|
|
||||||
honcho_tools._session_manager = MagicMock()
|
|
||||||
honcho_tools._session_key = "test-key"
|
|
||||||
assert honcho_tools._check_honcho_available() is True
|
|
||||||
|
|
||||||
def test_returns_true_when_configured_but_no_session(self):
|
|
||||||
"""Slow path: honcho configured but agent not started yet (banner time)."""
|
|
||||||
honcho_tools._session_manager = None
|
|
||||||
honcho_tools._session_key = None
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class FakeConfig:
|
|
||||||
enabled: bool = True
|
|
||||||
api_key: str = "test-key"
|
|
||||||
base_url: str = None
|
|
||||||
|
|
||||||
with patch("tools.honcho_tools.HonchoClientConfig", create=True):
|
|
||||||
with patch(
|
|
||||||
"honcho_integration.client.HonchoClientConfig"
|
|
||||||
) as mock_cls:
|
|
||||||
mock_cls.from_global_config.return_value = FakeConfig()
|
|
||||||
assert honcho_tools._check_honcho_available() is True
|
|
||||||
|
|
||||||
def test_returns_false_when_not_configured(self):
|
|
||||||
"""No session, no config: tool genuinely unavailable."""
|
|
||||||
honcho_tools._session_manager = None
|
|
||||||
honcho_tools._session_key = None
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class FakeConfig:
|
|
||||||
enabled: bool = False
|
|
||||||
api_key: str = None
|
|
||||||
base_url: str = None
|
|
||||||
|
|
||||||
with patch(
|
|
||||||
"honcho_integration.client.HonchoClientConfig"
|
|
||||||
) as mock_cls:
|
|
||||||
mock_cls.from_global_config.return_value = FakeConfig()
|
|
||||||
assert honcho_tools._check_honcho_available() is False
|
|
||||||
|
|
||||||
def test_returns_false_when_import_fails(self):
|
|
||||||
"""Graceful fallback when honcho_integration not installed."""
|
|
||||||
import sys
|
|
||||||
|
|
||||||
honcho_tools._session_manager = None
|
|
||||||
honcho_tools._session_key = None
|
|
||||||
|
|
||||||
# Hide honcho_integration from the import system to simulate
|
|
||||||
# an environment where the package is not installed.
|
|
||||||
hidden = {
|
|
||||||
k: sys.modules.pop(k)
|
|
||||||
for k in list(sys.modules)
|
|
||||||
if k.startswith("honcho_integration")
|
|
||||||
}
|
|
||||||
try:
|
|
||||||
with patch.dict(sys.modules, {"honcho_integration": None,
|
|
||||||
"honcho_integration.client": None}):
|
|
||||||
assert honcho_tools._check_honcho_available() is False
|
|
||||||
finally:
|
|
||||||
sys.modules.update(hidden)
|
|
||||||
|
|
||||||
|
|
||||||
class TestHonchoToolSessionContext:
|
|
||||||
def setup_method(self):
|
|
||||||
self.orig_manager = honcho_tools._session_manager
|
|
||||||
self.orig_key = honcho_tools._session_key
|
|
||||||
|
|
||||||
def teardown_method(self):
|
|
||||||
honcho_tools._session_manager = self.orig_manager
|
|
||||||
honcho_tools._session_key = self.orig_key
|
|
||||||
|
|
||||||
def test_explicit_call_context_wins_over_module_global_state(self):
|
|
||||||
global_manager = MagicMock()
|
|
||||||
global_manager.get_peer_card.return_value = ["global"]
|
|
||||||
explicit_manager = MagicMock()
|
|
||||||
explicit_manager.get_peer_card.return_value = ["explicit"]
|
|
||||||
|
|
||||||
honcho_tools.set_session_context(global_manager, "global-session")
|
|
||||||
|
|
||||||
result = json.loads(
|
|
||||||
honcho_tools._handle_honcho_profile(
|
|
||||||
{},
|
|
||||||
honcho_manager=explicit_manager,
|
|
||||||
honcho_session_key="explicit-session",
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
assert result == {"result": ["explicit"]}
|
|
||||||
explicit_manager.get_peer_card.assert_called_once_with("explicit-session")
|
|
||||||
global_manager.get_peer_card.assert_not_called()
|
|
||||||
|
|
@ -559,6 +559,19 @@ def delegate_task(
|
||||||
# Sort by task_index so results match input order
|
# Sort by task_index so results match input order
|
||||||
results.sort(key=lambda r: r["task_index"])
|
results.sort(key=lambda r: r["task_index"])
|
||||||
|
|
||||||
|
# Notify parent's memory provider of delegation outcomes
|
||||||
|
if parent_agent and hasattr(parent_agent, '_memory_manager') and parent_agent._memory_manager:
|
||||||
|
for entry in results:
|
||||||
|
try:
|
||||||
|
_task_goal = tasks[entry["task_index"]]["goal"] if entry["task_index"] < len(tasks) else ""
|
||||||
|
parent_agent._memory_manager.on_delegation(
|
||||||
|
task=_task_goal,
|
||||||
|
result=entry.get("summary", "") or "",
|
||||||
|
child_session_id=getattr(children[entry["task_index"]][2], "session_id", "") if entry["task_index"] < len(children) else "",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
total_duration = round(time.monotonic() - overall_start, 2)
|
total_duration = round(time.monotonic() - overall_start, 2)
|
||||||
|
|
||||||
return json.dumps({
|
return json.dumps({
|
||||||
|
|
|
||||||
|
|
@ -1,279 +0,0 @@
|
||||||
"""Honcho tools for user context retrieval.
|
|
||||||
|
|
||||||
Registers three complementary tools, ordered by capability:
|
|
||||||
|
|
||||||
honcho_context — dialectic Q&A (LLM-powered, direct answers)
|
|
||||||
honcho_search — semantic search (fast, no LLM, raw excerpts)
|
|
||||||
honcho_profile — peer card (fast, no LLM, structured facts)
|
|
||||||
|
|
||||||
Use honcho_context when you need Honcho to synthesize an answer.
|
|
||||||
Use honcho_search or honcho_profile when you want raw data to reason
|
|
||||||
over yourself.
|
|
||||||
|
|
||||||
The session key is injected at runtime by the agent loop via
|
|
||||||
``set_session_context()``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
# ── Module-level state (injected by AIAgent at init time) ──
|
|
||||||
|
|
||||||
_session_manager = None # HonchoSessionManager instance
|
|
||||||
_session_key: str | None = None # Current session key (e.g., "telegram:123456")
|
|
||||||
|
|
||||||
|
|
||||||
def set_session_context(session_manager, session_key: str) -> None:
|
|
||||||
"""Register the active Honcho session manager and key.
|
|
||||||
|
|
||||||
Called by AIAgent.__init__ when Honcho is enabled.
|
|
||||||
"""
|
|
||||||
global _session_manager, _session_key
|
|
||||||
_session_manager = session_manager
|
|
||||||
_session_key = session_key
|
|
||||||
|
|
||||||
|
|
||||||
def clear_session_context() -> None:
|
|
||||||
"""Clear session context (for testing or shutdown)."""
|
|
||||||
global _session_manager, _session_key
|
|
||||||
_session_manager = None
|
|
||||||
_session_key = None
|
|
||||||
|
|
||||||
|
|
||||||
# ── Availability check ──
|
|
||||||
|
|
||||||
def _check_honcho_available() -> bool:
|
|
||||||
"""Tool is available when Honcho is active OR configured.
|
|
||||||
|
|
||||||
At banner time the session context hasn't been injected yet, but if
|
|
||||||
a valid config exists the tools *will* activate once the agent starts.
|
|
||||||
Returning True for "configured" prevents the banner from marking
|
|
||||||
honcho tools as red/disabled when they're actually going to work.
|
|
||||||
"""
|
|
||||||
# Fast path: session already active (mid-conversation)
|
|
||||||
if _session_manager is not None and _session_key is not None:
|
|
||||||
return True
|
|
||||||
# Slow path: check if Honcho is configured (banner time)
|
|
||||||
try:
|
|
||||||
from honcho_integration.client import HonchoClientConfig
|
|
||||||
cfg = HonchoClientConfig.from_global_config()
|
|
||||||
return cfg.enabled and bool(cfg.api_key or cfg.base_url)
|
|
||||||
except Exception:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_session_context(**kwargs):
|
|
||||||
"""Prefer the calling agent's session context over module-global fallback."""
|
|
||||||
session_manager = kwargs.get("honcho_manager") or _session_manager
|
|
||||||
session_key = kwargs.get("honcho_session_key") or _session_key
|
|
||||||
return session_manager, session_key
|
|
||||||
|
|
||||||
|
|
||||||
# ── honcho_profile ──
|
|
||||||
|
|
||||||
_PROFILE_SCHEMA = {
|
|
||||||
"name": "honcho_profile",
|
|
||||||
"description": (
|
|
||||||
"Retrieve the user's peer card from Honcho — a curated list of key facts "
|
|
||||||
"about them (name, role, preferences, communication style, patterns). "
|
|
||||||
"Fast, no LLM reasoning, minimal cost. "
|
|
||||||
"Use this at conversation start or when you need a quick factual snapshot. "
|
|
||||||
"Use honcho_context instead when you need Honcho to synthesize an answer."
|
|
||||||
),
|
|
||||||
"parameters": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {},
|
|
||||||
"required": [],
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_honcho_profile(args: dict, **kw) -> str:
|
|
||||||
session_manager, session_key = _resolve_session_context(**kw)
|
|
||||||
if not session_manager or not session_key:
|
|
||||||
return json.dumps({"error": "Honcho is not active for this session."})
|
|
||||||
try:
|
|
||||||
card = session_manager.get_peer_card(session_key)
|
|
||||||
if not card:
|
|
||||||
return json.dumps({"result": "No profile facts available yet. The user's profile builds over time through conversations."})
|
|
||||||
return json.dumps({"result": card})
|
|
||||||
except Exception as e:
|
|
||||||
logger.error("Error fetching Honcho peer card: %s", e)
|
|
||||||
return json.dumps({"error": f"Failed to fetch profile: {e}"})
|
|
||||||
|
|
||||||
|
|
||||||
# ── honcho_search ──
|
|
||||||
|
|
||||||
_SEARCH_SCHEMA = {
|
|
||||||
"name": "honcho_search",
|
|
||||||
"description": (
|
|
||||||
"Semantic search over Honcho's stored context about the user. "
|
|
||||||
"Returns raw excerpts ranked by relevance to your query — no LLM synthesis. "
|
|
||||||
"Cheaper and faster than honcho_context. "
|
|
||||||
"Good when you want to find specific past facts and reason over them yourself. "
|
|
||||||
"Use honcho_context when you need a direct synthesized answer."
|
|
||||||
),
|
|
||||||
"parameters": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"query": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "What to search for in Honcho's memory (e.g. 'programming languages', 'past projects', 'timezone').",
|
|
||||||
},
|
|
||||||
"max_tokens": {
|
|
||||||
"type": "integer",
|
|
||||||
"description": "Token budget for returned context (default 800, max 2000).",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"required": ["query"],
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_honcho_search(args: dict, **kw) -> str:
|
|
||||||
query = args.get("query", "")
|
|
||||||
if not query:
|
|
||||||
return json.dumps({"error": "Missing required parameter: query"})
|
|
||||||
session_manager, session_key = _resolve_session_context(**kw)
|
|
||||||
if not session_manager or not session_key:
|
|
||||||
return json.dumps({"error": "Honcho is not active for this session."})
|
|
||||||
max_tokens = min(int(args.get("max_tokens", 800)), 2000)
|
|
||||||
try:
|
|
||||||
result = session_manager.search_context(session_key, query, max_tokens=max_tokens)
|
|
||||||
if not result:
|
|
||||||
return json.dumps({"result": "No relevant context found."})
|
|
||||||
return json.dumps({"result": result})
|
|
||||||
except Exception as e:
|
|
||||||
logger.error("Error searching Honcho context: %s", e)
|
|
||||||
return json.dumps({"error": f"Failed to search context: {e}"})
|
|
||||||
|
|
||||||
|
|
||||||
# ── honcho_context (dialectic — LLM-powered) ──
|
|
||||||
|
|
||||||
_QUERY_SCHEMA = {
|
|
||||||
"name": "honcho_context",
|
|
||||||
"description": (
|
|
||||||
"Ask Honcho a natural language question and get a synthesized answer. "
|
|
||||||
"Uses Honcho's LLM (dialectic reasoning) — higher cost than honcho_profile or honcho_search. "
|
|
||||||
"Can query about any peer: the user (default), the AI assistant, or any named peer. "
|
|
||||||
"Examples: 'What are the user's main goals?', 'What has hermes been working on?', "
|
|
||||||
"'What is the user's technical expertise level?'"
|
|
||||||
),
|
|
||||||
"parameters": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"query": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "A natural language question.",
|
|
||||||
},
|
|
||||||
"peer": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "Which peer to query about: 'user' (default) or 'ai'. Omit for user.",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"required": ["query"],
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_honcho_context(args: dict, **kw) -> str:
|
|
||||||
query = args.get("query", "")
|
|
||||||
if not query:
|
|
||||||
return json.dumps({"error": "Missing required parameter: query"})
|
|
||||||
session_manager, session_key = _resolve_session_context(**kw)
|
|
||||||
if not session_manager or not session_key:
|
|
||||||
return json.dumps({"error": "Honcho is not active for this session."})
|
|
||||||
peer_target = args.get("peer", "user")
|
|
||||||
try:
|
|
||||||
result = session_manager.dialectic_query(session_key, query, peer=peer_target)
|
|
||||||
return json.dumps({"result": result or "No result from Honcho."})
|
|
||||||
except Exception as e:
|
|
||||||
logger.error("Error querying Honcho context: %s", e)
|
|
||||||
return json.dumps({"error": f"Failed to query context: {e}"})
|
|
||||||
|
|
||||||
|
|
||||||
# ── honcho_conclude ──
|
|
||||||
|
|
||||||
_CONCLUDE_SCHEMA = {
|
|
||||||
"name": "honcho_conclude",
|
|
||||||
"description": (
|
|
||||||
"Write a conclusion about the user back to Honcho's memory. "
|
|
||||||
"Conclusions are persistent facts that build the user's profile — "
|
|
||||||
"preferences, corrections, clarifications, project context, or anything "
|
|
||||||
"the user tells you that should be remembered across sessions. "
|
|
||||||
"Use this when the user explicitly states a preference, corrects you, "
|
|
||||||
"or shares something they want remembered. "
|
|
||||||
"Examples: 'User prefers dark mode', 'User's project uses Python 3.11', "
|
|
||||||
"'User corrected: their name is spelled Eri not Eric'."
|
|
||||||
),
|
|
||||||
"parameters": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"conclusion": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "A factual statement about the user to persist in memory.",
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"required": ["conclusion"],
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_honcho_conclude(args: dict, **kw) -> str:
|
|
||||||
conclusion = args.get("conclusion", "")
|
|
||||||
if not conclusion:
|
|
||||||
return json.dumps({"error": "Missing required parameter: conclusion"})
|
|
||||||
session_manager, session_key = _resolve_session_context(**kw)
|
|
||||||
if not session_manager or not session_key:
|
|
||||||
return json.dumps({"error": "Honcho is not active for this session."})
|
|
||||||
try:
|
|
||||||
ok = session_manager.create_conclusion(session_key, conclusion)
|
|
||||||
if ok:
|
|
||||||
return json.dumps({"result": f"Conclusion saved: {conclusion}"})
|
|
||||||
return json.dumps({"error": "Failed to save conclusion."})
|
|
||||||
except Exception as e:
|
|
||||||
logger.error("Error creating Honcho conclusion: %s", e)
|
|
||||||
return json.dumps({"error": f"Failed to save conclusion: {e}"})
|
|
||||||
|
|
||||||
|
|
||||||
# ── Registration ──
|
|
||||||
|
|
||||||
from tools.registry import registry
|
|
||||||
|
|
||||||
registry.register(
|
|
||||||
name="honcho_profile",
|
|
||||||
toolset="honcho",
|
|
||||||
schema=_PROFILE_SCHEMA,
|
|
||||||
handler=_handle_honcho_profile,
|
|
||||||
check_fn=_check_honcho_available,
|
|
||||||
emoji="🔮",
|
|
||||||
)
|
|
||||||
|
|
||||||
registry.register(
|
|
||||||
name="honcho_search",
|
|
||||||
toolset="honcho",
|
|
||||||
schema=_SEARCH_SCHEMA,
|
|
||||||
handler=_handle_honcho_search,
|
|
||||||
check_fn=_check_honcho_available,
|
|
||||||
emoji="🔮",
|
|
||||||
)
|
|
||||||
|
|
||||||
registry.register(
|
|
||||||
name="honcho_context",
|
|
||||||
toolset="honcho",
|
|
||||||
schema=_QUERY_SCHEMA,
|
|
||||||
handler=_handle_honcho_context,
|
|
||||||
check_fn=_check_honcho_available,
|
|
||||||
emoji="🔮",
|
|
||||||
)
|
|
||||||
|
|
||||||
registry.register(
|
|
||||||
name="honcho_conclude",
|
|
||||||
toolset="honcho",
|
|
||||||
schema=_CONCLUDE_SCHEMA,
|
|
||||||
handler=_handle_honcho_conclude,
|
|
||||||
check_fn=_check_honcho_available,
|
|
||||||
emoji="🔮",
|
|
||||||
)
|
|
||||||
12
toolsets.py
12
toolsets.py
|
|
@ -60,8 +60,6 @@ _HERMES_CORE_TOOLS = [
|
||||||
"cronjob",
|
"cronjob",
|
||||||
# Cross-platform messaging (gated on gateway running via check_fn)
|
# Cross-platform messaging (gated on gateway running via check_fn)
|
||||||
"send_message",
|
"send_message",
|
||||||
# Honcho memory tools (gated on honcho being active via check_fn)
|
|
||||||
"honcho_context", "honcho_profile", "honcho_search", "honcho_conclude",
|
|
||||||
# Home Assistant smart home control (gated on HASS_TOKEN via check_fn)
|
# Home Assistant smart home control (gated on HASS_TOKEN via check_fn)
|
||||||
"ha_list_entities", "ha_get_state", "ha_list_services", "ha_call_service",
|
"ha_list_entities", "ha_get_state", "ha_list_services", "ha_call_service",
|
||||||
]
|
]
|
||||||
|
|
@ -196,11 +194,8 @@ TOOLSETS = {
|
||||||
"includes": []
|
"includes": []
|
||||||
},
|
},
|
||||||
|
|
||||||
"honcho": {
|
# "honcho" toolset removed — Honcho is now a memory provider plugin.
|
||||||
"description": "Honcho AI-native memory for persistent cross-session user modeling",
|
# Tools are injected via MemoryManager, not the toolset system.
|
||||||
"tools": ["honcho_context", "honcho_profile", "honcho_search", "honcho_conclude"],
|
|
||||||
"includes": []
|
|
||||||
},
|
|
||||||
|
|
||||||
"homeassistant": {
|
"homeassistant": {
|
||||||
"description": "Home Assistant smart home control and monitoring",
|
"description": "Home Assistant smart home control and monitoring",
|
||||||
|
|
@ -279,8 +274,7 @@ TOOLSETS = {
|
||||||
"cronjob",
|
"cronjob",
|
||||||
# Home Assistant smart home control (gated on HASS_TOKEN via check_fn)
|
# Home Assistant smart home control (gated on HASS_TOKEN via check_fn)
|
||||||
"ha_list_entities", "ha_get_state", "ha_list_services", "ha_call_service",
|
"ha_list_entities", "ha_get_state", "ha_list_services", "ha_call_service",
|
||||||
# Honcho memory tools (gated on honcho being active via check_fn)
|
|
||||||
"honcho_context", "honcho_profile", "honcho_search", "honcho_conclude",
|
|
||||||
],
|
],
|
||||||
"includes": []
|
"includes": []
|
||||||
},
|
},
|
||||||
|
|
|
||||||
|
|
@ -24,7 +24,7 @@ hermes-agent/
|
||||||
├── tools/ # tool implementations and terminal environments
|
├── tools/ # tool implementations and terminal environments
|
||||||
├── gateway/ # messaging gateway, session routing, delivery, pairing, hooks
|
├── gateway/ # messaging gateway, session routing, delivery, pairing, hooks
|
||||||
├── cron/ # scheduled job storage and scheduler
|
├── cron/ # scheduled job storage and scheduler
|
||||||
├── honcho_integration/ # Honcho memory integration
|
├── plugins/memory/ # Memory provider plugins (honcho, openviking, mem0, etc.)
|
||||||
├── acp_adapter/ # ACP editor integration server
|
├── acp_adapter/ # ACP editor integration server
|
||||||
├── acp_registry/ # ACP registry manifest + icon
|
├── acp_registry/ # ACP registry manifest + icon
|
||||||
├── environments/ # Hermes RL / benchmark environment framework
|
├── environments/ # Hermes RL / benchmark environment framework
|
||||||
|
|
|
||||||
|
|
@ -86,33 +86,23 @@ The gateway also runs maintenance tasks such as:
|
||||||
|
|
||||||
## Honcho interaction
|
## Honcho interaction
|
||||||
|
|
||||||
When Honcho is enabled, the gateway keeps persistent Honcho managers aligned with session lifetimes and platform-specific session keys.
|
When a memory provider plugin (e.g. Honcho) is enabled, the gateway creates an AIAgent per incoming message with the same session ID. The memory provider's `initialize()` receives the session ID and creates the appropriate backend session. Tools are routed through the `MemoryManager`, which handles all provider lifecycle hooks (prefetch, sync, session end).
|
||||||
|
|
||||||
### Session routing
|
### Memory provider session routing
|
||||||
|
|
||||||
Honcho tools (`honcho_profile`, `honcho_search`, `honcho_context`, `honcho_conclude`) need to execute against the correct user's Honcho session. In a multi-user gateway, the process-global module state in `tools/honcho_tools.py` is insufficient — multiple sessions may be active concurrently.
|
Memory provider tools (e.g. `honcho_profile`, `viking_search`) are routed through the MemoryManager in `_invoke_tool()`:
|
||||||
|
|
||||||
The solution threads session context through the call chain:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
AIAgent._invoke_tool()
|
AIAgent._invoke_tool()
|
||||||
→ handle_function_call(honcho_manager=..., honcho_session_key=...)
|
→ self._memory_manager.handle_tool_call(name, args)
|
||||||
→ registry.dispatch(**kwargs)
|
→ provider.handle_tool_call(name, args)
|
||||||
→ _handle_honcho_*(args, **kw)
|
|
||||||
→ _resolve_session_context(**kw) # prefers explicit kwargs over module globals
|
|
||||||
```
|
```
|
||||||
|
|
||||||
`_resolve_session_context()` in `honcho_tools.py` checks for `honcho_manager` and `honcho_session_key` in the kwargs first, falling back to the module-global `_session_manager` / `_session_key` for CLI mode where there's only one session.
|
Each memory provider manages its own session lifecycle internally. The `initialize()` method receives the session ID, and `on_session_end()` handles cleanup and final flush.
|
||||||
|
|
||||||
### Memory flush lifecycle
|
### Memory flush lifecycle
|
||||||
|
|
||||||
When a session is reset, resumed, or expires, the gateway flushes memories before discarding context. The flush creates a temporary `AIAgent` with:
|
When a session is reset, resumed, or expires, the gateway flushes built-in memories before discarding context. The flush creates a temporary `AIAgent` that runs a memory-only conversation turn. The memory provider's `on_session_end()` hook fires during this process, giving external providers a chance to persist any buffered data.
|
||||||
|
|
||||||
- `session_id` set to the old session's ID (so transcripts load correctly)
|
|
||||||
- `honcho_session_key` set to the gateway session key (so Honcho writes go to the right place)
|
|
||||||
- `sync_honcho=False` passed to `run_conversation()` (so the synthetic flush turn doesn't write back to Honcho's conversation history)
|
|
||||||
|
|
||||||
After the flush completes, any queued Honcho writes are drained and the gateway-level Honcho manager is shut down for that session key.
|
|
||||||
|
|
||||||
## Related docs
|
## Related docs
|
||||||
|
|
||||||
|
|
|
||||||
197
website/docs/developer-guide/memory-provider-plugin.md
Normal file
197
website/docs/developer-guide/memory-provider-plugin.md
Normal file
|
|
@ -0,0 +1,197 @@
|
||||||
|
---
|
||||||
|
sidebar_position: 8
|
||||||
|
title: "Memory Provider Plugins"
|
||||||
|
description: "How to build a memory provider plugin for Hermes Agent"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Building a Memory Provider Plugin
|
||||||
|
|
||||||
|
Memory provider plugins give Hermes Agent persistent, cross-session knowledge beyond the built-in MEMORY.md and USER.md. This guide covers how to build one.
|
||||||
|
|
||||||
|
## Directory Structure
|
||||||
|
|
||||||
|
Each memory provider lives in `plugins/memory/<name>/`:
|
||||||
|
|
||||||
|
```
|
||||||
|
plugins/memory/my-provider/
|
||||||
|
├── __init__.py # MemoryProvider implementation + register() entry point
|
||||||
|
├── plugin.yaml # Metadata (name, description, hooks)
|
||||||
|
└── README.md # Setup instructions, config reference, tools
|
||||||
|
```
|
||||||
|
|
||||||
|
## The MemoryProvider ABC
|
||||||
|
|
||||||
|
Your plugin implements the `MemoryProvider` abstract base class from `agent/memory_provider.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from agent.memory_provider import MemoryProvider
|
||||||
|
|
||||||
|
class MyMemoryProvider(MemoryProvider):
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "my-provider"
|
||||||
|
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
"""Check if this provider can activate. NO network calls."""
|
||||||
|
return bool(os.environ.get("MY_API_KEY"))
|
||||||
|
|
||||||
|
def initialize(self, session_id: str, **kwargs) -> None:
|
||||||
|
"""Called once at agent startup.
|
||||||
|
|
||||||
|
kwargs always includes:
|
||||||
|
hermes_home (str): Active HERMES_HOME path. Use for storage.
|
||||||
|
"""
|
||||||
|
self._api_key = os.environ.get("MY_API_KEY", "")
|
||||||
|
self._session_id = session_id
|
||||||
|
|
||||||
|
# ... implement remaining methods
|
||||||
|
```
|
||||||
|
|
||||||
|
## Required Methods
|
||||||
|
|
||||||
|
### Core Lifecycle
|
||||||
|
|
||||||
|
| Method | When Called | Must Implement? |
|
||||||
|
|--------|-----------|-----------------|
|
||||||
|
| `name` (property) | Always | **Yes** |
|
||||||
|
| `is_available()` | Agent init, before activation | **Yes** — no network calls |
|
||||||
|
| `initialize(session_id, **kwargs)` | Agent startup | **Yes** |
|
||||||
|
| `get_tool_schemas()` | After init, for tool injection | **Yes** |
|
||||||
|
| `handle_tool_call(name, args)` | When agent uses your tools | **Yes** (if you have tools) |
|
||||||
|
|
||||||
|
### Config
|
||||||
|
|
||||||
|
| Method | Purpose | Must Implement? |
|
||||||
|
|--------|---------|-----------------|
|
||||||
|
| `get_config_schema()` | Declare config fields for `hermes memory setup` | **Yes** |
|
||||||
|
| `save_config(values, hermes_home)` | Write non-secret config to native location | **Yes** (unless env-var-only) |
|
||||||
|
|
||||||
|
### Optional Hooks
|
||||||
|
|
||||||
|
| Method | When Called | Use Case |
|
||||||
|
|--------|-----------|----------|
|
||||||
|
| `system_prompt_block()` | System prompt assembly | Static provider info |
|
||||||
|
| `prefetch(query)` | Before each API call | Return recalled context |
|
||||||
|
| `queue_prefetch(query)` | After each turn | Pre-warm for next turn |
|
||||||
|
| `sync_turn(user, assistant)` | After each completed turn | Persist conversation |
|
||||||
|
| `on_session_end(messages)` | Conversation ends | Final extraction/flush |
|
||||||
|
| `on_pre_compress(messages)` | Before context compression | Save insights before discard |
|
||||||
|
| `on_memory_write(action, target, content)` | Built-in memory writes | Mirror to your backend |
|
||||||
|
| `shutdown()` | Process exit | Clean up connections |
|
||||||
|
|
||||||
|
## Config Schema
|
||||||
|
|
||||||
|
`get_config_schema()` returns a list of field descriptors used by `hermes memory setup`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def get_config_schema(self):
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"key": "api_key",
|
||||||
|
"description": "My Provider API key",
|
||||||
|
"secret": True, # → written to .env
|
||||||
|
"required": True,
|
||||||
|
"env_var": "MY_API_KEY", # explicit env var name
|
||||||
|
"url": "https://my-provider.com/keys", # where to get it
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "region",
|
||||||
|
"description": "Server region",
|
||||||
|
"default": "us-east",
|
||||||
|
"choices": ["us-east", "eu-west", "ap-south"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "project",
|
||||||
|
"description": "Project identifier",
|
||||||
|
"default": "hermes",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
Fields with `secret: True` and `env_var` go to `.env`. Non-secret fields are passed to `save_config()`.
|
||||||
|
|
||||||
|
## Save Config
|
||||||
|
|
||||||
|
```python
|
||||||
|
def save_config(self, values: dict, hermes_home: str) -> None:
|
||||||
|
"""Write non-secret config to your native location."""
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
config_path = Path(hermes_home) / "my-provider.json"
|
||||||
|
config_path.write_text(json.dumps(values, indent=2))
|
||||||
|
```
|
||||||
|
|
||||||
|
For env-var-only providers, leave the default no-op.
|
||||||
|
|
||||||
|
## Plugin Entry Point
|
||||||
|
|
||||||
|
```python
|
||||||
|
def register(ctx) -> None:
|
||||||
|
"""Called by the memory plugin discovery system."""
|
||||||
|
ctx.register_memory_provider(MyMemoryProvider())
|
||||||
|
```
|
||||||
|
|
||||||
|
## plugin.yaml
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: my-provider
|
||||||
|
version: 1.0.0
|
||||||
|
description: "Short description of what this provider does."
|
||||||
|
hooks:
|
||||||
|
- on_session_end # list hooks you implement
|
||||||
|
```
|
||||||
|
|
||||||
|
## Threading Contract
|
||||||
|
|
||||||
|
**`sync_turn()` MUST be non-blocking.** If your backend has latency (API calls, LLM processing), run the work in a daemon thread:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def sync_turn(self, user_content, assistant_content):
|
||||||
|
def _sync():
|
||||||
|
try:
|
||||||
|
self._api.ingest(user_content, assistant_content)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Sync failed: %s", e)
|
||||||
|
|
||||||
|
if self._sync_thread and self._sync_thread.is_alive():
|
||||||
|
self._sync_thread.join(timeout=5.0)
|
||||||
|
self._sync_thread = threading.Thread(target=_sync, daemon=True)
|
||||||
|
self._sync_thread.start()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Profile Isolation
|
||||||
|
|
||||||
|
All storage paths **must** use the `hermes_home` kwarg from `initialize()`, not hardcoded `~/.hermes`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# CORRECT — profile-scoped
|
||||||
|
from hermes_constants import get_hermes_home
|
||||||
|
data_dir = get_hermes_home() / "my-provider"
|
||||||
|
|
||||||
|
# WRONG — shared across all profiles
|
||||||
|
data_dir = Path("~/.hermes/my-provider").expanduser()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
See `tests/agent/test_memory_plugin_e2e.py` for the complete E2E testing pattern using a real SQLite provider.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from agent.memory_manager import MemoryManager
|
||||||
|
|
||||||
|
mgr = MemoryManager()
|
||||||
|
mgr.add_provider(my_provider)
|
||||||
|
mgr.initialize_all(session_id="test-1", platform="cli")
|
||||||
|
|
||||||
|
# Test tool routing
|
||||||
|
result = mgr.handle_tool_call("my_tool", {"action": "add", "content": "test"})
|
||||||
|
|
||||||
|
# Test lifecycle
|
||||||
|
mgr.sync_all("user msg", "assistant msg")
|
||||||
|
mgr.on_session_end([])
|
||||||
|
mgr.shutdown_all()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Single Provider Rule
|
||||||
|
|
||||||
|
Only **one** external memory provider can be active at a time. If a user tries to register a second, the MemoryManager rejects it with a warning. This prevents tool schema bloat and conflicting backends.
|
||||||
|
|
@ -1,404 +1,43 @@
|
||||||
---
|
---
|
||||||
title: Honcho Memory
|
sidebar_position: 99
|
||||||
description: AI-native persistent memory for cross-session user modeling and personalization.
|
title: "Honcho Memory"
|
||||||
sidebar_label: Honcho Memory
|
description: "Honcho is now available as a memory provider plugin"
|
||||||
sidebar_position: 8
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Honcho Memory
|
# Honcho Memory
|
||||||
|
|
||||||
[Honcho](https://honcho.dev) is an AI-native memory system that gives Hermes persistent, cross-session understanding of users. While Hermes has built-in memory (`MEMORY.md` and `USER.md`), Honcho adds a deeper layer of **user modeling** — learning preferences, goals, communication style, and context across conversations via a dual-peer architecture where both the user and the AI build representations over time.
|
:::info Honcho is now a Memory Provider Plugin
|
||||||
|
Honcho has been integrated into the [Memory Providers](./memory-providers.md) system. All Honcho features are available through the unified memory provider interface.
|
||||||
## Works Alongside Built-in Memory
|
:::
|
||||||
|
|
||||||
Hermes has two memory systems that can work together or be configured separately. In `hybrid` mode (the default), both run side by side — Honcho adds cross-session user modeling while local files handle agent-level notes.
|
|
||||||
|
|
||||||
| Feature | Built-in Memory | Honcho Memory |
|
|
||||||
|---------|----------------|---------------|
|
|
||||||
| Storage | Local files (`~/.hermes/memories/`) | Cloud-hosted Honcho API |
|
|
||||||
| Scope | Agent-level notes and user profile | Deep user modeling via dialectic reasoning |
|
|
||||||
| Persistence | Across sessions on same machine | Across sessions, machines, and platforms |
|
|
||||||
| Query | Injected into system prompt automatically | Prefetched + on-demand via tools |
|
|
||||||
| Content | Manually curated by the agent | Automatically learned from conversations |
|
|
||||||
| Write surface | `memory` tool (add/replace/remove) | `honcho_conclude` tool (persist facts) |
|
|
||||||
|
|
||||||
Set `memoryMode` to `honcho` to use Honcho exclusively. See [Memory Modes](#memory-modes) for per-peer configuration.
|
|
||||||
|
|
||||||
|
|
||||||
## Self-hosted / Docker
|
|
||||||
|
|
||||||
Hermes supports a local Honcho instance (e.g. via Docker) in addition to the hosted API. Point it at your instance using `HONCHO_BASE_URL` — no API key required.
|
|
||||||
|
|
||||||
**Via `hermes config`:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
hermes config set HONCHO_BASE_URL http://localhost:8000
|
|
||||||
```
|
|
||||||
|
|
||||||
**Via `~/.honcho/config.json`:**
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"hosts": {
|
|
||||||
"hermes": {
|
|
||||||
"base_url": "http://localhost:8000",
|
|
||||||
"enabled": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Hermes auto-enables Honcho when either `apiKey` or `base_url` is present, so no further configuration is needed for a local instance.
|
|
||||||
|
|
||||||
To run Honcho locally, refer to the [Honcho self-hosting docs](https://docs.honcho.dev).
|
|
||||||
|
|
||||||
## Setup
|
## Setup
|
||||||
|
|
||||||
### Interactive Setup
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
hermes honcho setup
|
hermes memory setup # select "honcho"
|
||||||
```
|
```
|
||||||
|
|
||||||
The setup wizard walks through API key, peer names, workspace, memory mode, write frequency, recall mode, and session strategy. It offers to install `honcho-ai` if missing.
|
Or set manually:
|
||||||
|
|
||||||
### Manual Setup
|
|
||||||
|
|
||||||
#### 1. Install the Client Library
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install 'honcho-ai>=2.0.1'
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 2. Get an API Key
|
|
||||||
|
|
||||||
Go to [app.honcho.dev](https://app.honcho.dev) > Settings > API Keys.
|
|
||||||
|
|
||||||
#### 3. Configure
|
|
||||||
|
|
||||||
Honcho reads from `~/.honcho/config.json` (shared across all Honcho-enabled applications):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"apiKey": "your-honcho-api-key",
|
|
||||||
"hosts": {
|
|
||||||
"hermes": {
|
|
||||||
"workspace": "hermes",
|
|
||||||
"peerName": "your-name",
|
|
||||||
"aiPeer": "hermes",
|
|
||||||
"memoryMode": "hybrid",
|
|
||||||
"writeFrequency": "async",
|
|
||||||
"recallMode": "hybrid",
|
|
||||||
"sessionStrategy": "per-session",
|
|
||||||
"enabled": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
`apiKey` lives at the root because it is a shared credential across all Honcho-enabled tools. All other settings are scoped under `hosts.hermes`. The `hermes honcho setup` wizard writes this structure automatically.
|
|
||||||
|
|
||||||
Or set the API key as an environment variable:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
hermes config set HONCHO_API_KEY your-key
|
|
||||||
```
|
|
||||||
|
|
||||||
:::info
|
|
||||||
When an API key is present (either in `~/.honcho/config.json` or as `HONCHO_API_KEY`), Honcho auto-enables unless explicitly set to `"enabled": false`.
|
|
||||||
:::
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### Global Config (`~/.honcho/config.json`)
|
|
||||||
|
|
||||||
Settings are scoped to `hosts.hermes` and fall back to root-level globals when the host field is absent. Root-level keys are managed by the user or the honcho CLI -- Hermes only writes to its own host block (except `apiKey`, which is a shared credential at root).
|
|
||||||
|
|
||||||
**Root-level (shared)**
|
|
||||||
|
|
||||||
| Field | Default | Description |
|
|
||||||
|-------|---------|-------------|
|
|
||||||
| `apiKey` | — | Honcho API key (required, shared across all hosts) |
|
|
||||||
| `sessions` | `{}` | Manual session name overrides per directory (shared) |
|
|
||||||
|
|
||||||
**Host-level (`hosts.hermes`)**
|
|
||||||
|
|
||||||
| Field | Default | Description |
|
|
||||||
|-------|---------|-------------|
|
|
||||||
| `workspace` | `"hermes"` | Workspace identifier |
|
|
||||||
| `peerName` | *(derived)* | Your identity name for user modeling |
|
|
||||||
| `aiPeer` | `"hermes"` | AI assistant identity name |
|
|
||||||
| `environment` | `"production"` | Honcho environment |
|
|
||||||
| `enabled` | *(auto)* | Auto-enables when API key is present |
|
|
||||||
| `saveMessages` | `true` | Whether to sync messages to Honcho |
|
|
||||||
| `memoryMode` | `"hybrid"` | Memory mode: `hybrid` or `honcho` |
|
|
||||||
| `writeFrequency` | `"async"` | When to write: `async`, `turn`, `session`, or integer N |
|
|
||||||
| `recallMode` | `"hybrid"` | Retrieval strategy: `hybrid`, `context`, or `tools` |
|
|
||||||
| `sessionStrategy` | `"per-session"` | How sessions are scoped |
|
|
||||||
| `sessionPeerPrefix` | `false` | Prefix session names with peer name |
|
|
||||||
| `contextTokens` | *(Honcho default)* | Max tokens for auto-injected context |
|
|
||||||
| `dialecticReasoningLevel` | `"low"` | Floor for dialectic reasoning: `minimal` / `low` / `medium` / `high` / `max` |
|
|
||||||
| `dialecticMaxChars` | `600` | Char cap on dialectic results injected into system prompt |
|
|
||||||
| `linkedHosts` | `[]` | Other host keys whose workspaces to cross-reference |
|
|
||||||
|
|
||||||
All host-level fields fall back to the equivalent root-level key if not set under `hosts.hermes`. Existing configs with settings at root level continue to work.
|
|
||||||
|
|
||||||
### Memory Modes
|
|
||||||
|
|
||||||
| Mode | Effect |
|
|
||||||
|------|--------|
|
|
||||||
| `hybrid` | Write to both Honcho and local files (default) |
|
|
||||||
| `honcho` | Honcho only — skip local file writes |
|
|
||||||
|
|
||||||
Memory mode can be set globally or per-peer (user, agent1, agent2, etc):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"memoryMode": {
|
|
||||||
"default": "hybrid",
|
|
||||||
"hermes": "honcho"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
To disable Honcho entirely, set `enabled: false` or remove the API key.
|
|
||||||
|
|
||||||
### Recall Modes
|
|
||||||
|
|
||||||
Controls how Honcho context reaches the agent:
|
|
||||||
|
|
||||||
| Mode | Behavior |
|
|
||||||
|------|----------|
|
|
||||||
| `hybrid` | Auto-injected context + Honcho tools available (default) |
|
|
||||||
| `context` | Auto-injected context only — Honcho tools hidden |
|
|
||||||
| `tools` | Honcho tools only — no auto-injected context |
|
|
||||||
|
|
||||||
### Write Frequency
|
|
||||||
|
|
||||||
| Setting | Behavior |
|
|
||||||
|---------|----------|
|
|
||||||
| `async` | Background thread writes (zero blocking, default) |
|
|
||||||
| `turn` | Synchronous write after each turn |
|
|
||||||
| `session` | Batched write at session end |
|
|
||||||
| *integer N* | Write every N turns |
|
|
||||||
|
|
||||||
### Session Strategies
|
|
||||||
|
|
||||||
| Strategy | Session key | Use case |
|
|
||||||
|----------|-------------|----------|
|
|
||||||
| `per-session` | Unique per run | Default. Fresh session every time. |
|
|
||||||
| `per-directory` | CWD basename | Each project gets its own session. |
|
|
||||||
| `per-repo` | Git repo root name | Groups subdirectories under one session. |
|
|
||||||
| `global` | Fixed `"global"` | Single cross-project session. |
|
|
||||||
|
|
||||||
Resolution order: manual map > session title > strategy-derived key > platform key.
|
|
||||||
|
|
||||||
### Multi-host Configuration
|
|
||||||
|
|
||||||
Multiple Honcho-enabled tools share `~/.honcho/config.json`. Each tool writes only to its own host block, reads its host block first, and falls back to root-level globals:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"apiKey": "your-key",
|
|
||||||
"peerName": "eri",
|
|
||||||
"hosts": {
|
|
||||||
"hermes": {
|
|
||||||
"workspace": "my-workspace",
|
|
||||||
"aiPeer": "hermes-assistant",
|
|
||||||
"memoryMode": "honcho",
|
|
||||||
"linkedHosts": ["claude-code"],
|
|
||||||
"contextTokens": 2000,
|
|
||||||
"dialecticReasoningLevel": "medium"
|
|
||||||
},
|
|
||||||
"claude-code": {
|
|
||||||
"workspace": "my-workspace",
|
|
||||||
"aiPeer": "clawd"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Resolution: `hosts.<tool>` field > root-level field > default. In this example, both tools share the root `apiKey` and `peerName`, but each has its own `aiPeer` and workspace settings.
|
|
||||||
|
|
||||||
### Hermes Config (`~/.hermes/config.yaml`)
|
|
||||||
|
|
||||||
Intentionally minimal — most configuration comes from `~/.honcho/config.json`:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
honcho: {}
|
# ~/.hermes/config.yaml
|
||||||
|
memory:
|
||||||
|
provider: honcho
|
||||||
```
|
```
|
||||||
|
|
||||||
## How It Works
|
|
||||||
|
|
||||||
### Async Context Pipeline
|
|
||||||
|
|
||||||
Honcho context is fetched asynchronously to avoid blocking the response path:
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
user["User message"] --> cache["Consume cached Honcho context<br/>from the previous turn"]
|
|
||||||
cache --> prompt["Inject user, AI, and dialectic context<br/>into the system prompt"]
|
|
||||||
prompt --> llm["LLM call"]
|
|
||||||
llm --> response["Assistant response"]
|
|
||||||
response --> fetch["Start background fetch for Turn N+1"]
|
|
||||||
fetch --> ctx["Fetch context"]
|
|
||||||
fetch --> dia["Fetch dialectic"]
|
|
||||||
ctx --> next["Cache for the next turn"]
|
|
||||||
dia --> next
|
|
||||||
```
|
|
||||||
|
|
||||||
Turn 1 is a cold start (no cache). All subsequent turns consume cached results with zero HTTP latency on the response path. The system prompt on turn 1 uses only static context to preserve prefix cache hits at the LLM provider.
|
|
||||||
|
|
||||||
### Dual-Peer Architecture
|
|
||||||
|
|
||||||
Both the user and AI have peer representations in Honcho:
|
|
||||||
|
|
||||||
- **User peer** — observed from user messages. Honcho learns preferences, goals, communication style.
|
|
||||||
- **AI peer** — observed from assistant messages (`observe_me=True`). Honcho builds a representation of the agent's knowledge and behavior.
|
|
||||||
|
|
||||||
Both representations are injected into the system prompt when available.
|
|
||||||
|
|
||||||
### Dynamic Reasoning Level
|
|
||||||
|
|
||||||
Dialectic queries scale reasoning effort with message complexity:
|
|
||||||
|
|
||||||
| Message length | Reasoning level |
|
|
||||||
|----------------|-----------------|
|
|
||||||
| < 120 chars | Config default (typically `low`) |
|
|
||||||
| 120-400 chars | One level above default (cap: `high`) |
|
|
||||||
| > 400 chars | Two levels above default (cap: `high`) |
|
|
||||||
|
|
||||||
`max` is never selected automatically.
|
|
||||||
|
|
||||||
### Gateway Integration
|
|
||||||
|
|
||||||
The gateway creates short-lived `AIAgent` instances per request. Honcho managers are owned at the gateway session layer (`_honcho_managers` dict) so they persist across requests within the same session and flush at real session boundaries (reset, resume, expiry, server stop).
|
|
||||||
|
|
||||||
#### Session Isolation
|
|
||||||
|
|
||||||
Each gateway session (e.g., a Telegram chat, a Discord channel) gets its own Honcho session context. The session key — derived from the platform and chat ID — is threaded through the entire tool dispatch chain so that Honcho tool calls always execute against the correct session, even when multiple users are messaging concurrently.
|
|
||||||
|
|
||||||
This means:
|
|
||||||
- **`honcho_profile`**, **`honcho_search`**, **`honcho_context`**, and **`honcho_conclude`** all resolve the correct session at call time, not at startup
|
|
||||||
- Background memory flushes (triggered by `/reset`, `/resume`, or session expiry) preserve the original session key so they write to the correct Honcho session
|
|
||||||
- Synthetic flush turns (where the agent saves memories before context is lost) skip Honcho sync to avoid polluting conversation history with internal bookkeeping
|
|
||||||
|
|
||||||
#### Session Lifecycle
|
|
||||||
|
|
||||||
| Event | What happens to Honcho |
|
|
||||||
|-------|------------------------|
|
|
||||||
| New message arrives | Agent inherits the gateway's Honcho manager + session key |
|
|
||||||
| `/reset` | Memory flush fires with the old session key, then Honcho manager shuts down |
|
|
||||||
| `/resume` | Current session is flushed, then the resumed session's Honcho context loads |
|
|
||||||
| Session expiry | Automatic flush + shutdown after the configured idle timeout |
|
|
||||||
| Gateway stop | All active Honcho managers are flushed and shut down gracefully |
|
|
||||||
|
|
||||||
## Tools
|
|
||||||
|
|
||||||
When Honcho is active, four tools become available. Availability is gated dynamically — they are invisible when Honcho is disabled.
|
|
||||||
|
|
||||||
### `honcho_profile`
|
|
||||||
|
|
||||||
Fast peer card retrieval (no LLM). Returns a curated list of key facts about the user.
|
|
||||||
|
|
||||||
### `honcho_search`
|
|
||||||
|
|
||||||
Semantic search over memory (no LLM). Returns raw excerpts ranked by relevance. Cheaper and faster than `honcho_context` — good for factual lookups.
|
|
||||||
|
|
||||||
Parameters:
|
|
||||||
- `query` (string) — search query
|
|
||||||
- `max_tokens` (integer, optional) — result token budget
|
|
||||||
|
|
||||||
### `honcho_context`
|
|
||||||
|
|
||||||
Dialectic Q&A powered by Honcho's LLM. Synthesizes an answer from accumulated conversation history.
|
|
||||||
|
|
||||||
Parameters:
|
|
||||||
- `query` (string) — natural language question
|
|
||||||
- `peer` (string, optional) — `"user"` (default) or `"ai"`. Querying `"ai"` asks about the assistant's own history and identity.
|
|
||||||
|
|
||||||
Example queries the agent might make:
|
|
||||||
|
|
||||||
```
|
|
||||||
"What are this user's main goals?"
|
|
||||||
"What communication style does this user prefer?"
|
|
||||||
"What topics has this user discussed recently?"
|
|
||||||
"What is this user's technical expertise level?"
|
|
||||||
```
|
|
||||||
|
|
||||||
### `honcho_conclude`
|
|
||||||
|
|
||||||
Writes a fact to Honcho memory. Use when the user explicitly states a preference, correction, or project context worth remembering. Feeds into the user's peer card and representation.
|
|
||||||
|
|
||||||
Parameters:
|
|
||||||
- `conclusion` (string) — the fact to persist
|
|
||||||
|
|
||||||
## CLI Commands
|
|
||||||
|
|
||||||
```
|
|
||||||
hermes honcho setup # Interactive setup wizard
|
|
||||||
hermes honcho status # Show config and connection status
|
|
||||||
hermes honcho sessions # List directory → session name mappings
|
|
||||||
hermes honcho map <name> # Map current directory to a session name
|
|
||||||
hermes honcho peer # Show peer names and dialectic settings
|
|
||||||
hermes honcho peer --user NAME # Set user peer name
|
|
||||||
hermes honcho peer --ai NAME # Set AI peer name
|
|
||||||
hermes honcho peer --reasoning LEVEL # Set dialectic reasoning level
|
|
||||||
hermes honcho mode # Show current memory mode
|
|
||||||
hermes honcho mode [hybrid|honcho|local] # Set memory mode
|
|
||||||
hermes honcho tokens # Show token budget settings
|
|
||||||
hermes honcho tokens --context N # Set context token cap
|
|
||||||
hermes honcho tokens --dialectic N # Set dialectic char cap
|
|
||||||
hermes honcho identity # Show AI peer identity
|
|
||||||
hermes honcho identity <file> # Seed AI peer identity from file (SOUL.md, etc.)
|
|
||||||
hermes honcho migrate # Migration guide: OpenClaw → Hermes + Honcho
|
|
||||||
```
|
|
||||||
|
|
||||||
### Doctor Integration
|
|
||||||
|
|
||||||
`hermes doctor` includes a Honcho section that validates config, API key, and connection status.
|
|
||||||
|
|
||||||
## Migration
|
|
||||||
|
|
||||||
### From Local Memory
|
|
||||||
|
|
||||||
When Honcho activates on an instance with existing local history, migration runs automatically:
|
|
||||||
|
|
||||||
1. **Conversation history** — prior messages are uploaded as an XML transcript file
|
|
||||||
2. **Memory files** — existing `MEMORY.md`, `USER.md`, and `SOUL.md` are uploaded for context
|
|
||||||
|
|
||||||
### From OpenClaw
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
hermes honcho migrate
|
echo "HONCHO_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
```
|
```
|
||||||
|
|
||||||
Walks through converting an OpenClaw native Honcho setup to the shared `~/.honcho/config.json` format.
|
## Migrating from `hermes honcho`
|
||||||
|
|
||||||
## AI Peer Identity
|
If you previously used `hermes honcho setup`:
|
||||||
|
|
||||||
Honcho can build a representation of the AI assistant over time (via `observe_me=True`). You can also seed the AI peer explicitly:
|
1. Your existing configuration (`honcho.json` or `~/.honcho/config.json`) is preserved
|
||||||
|
2. Your server-side data (memories, conclusions, user profiles) is intact
|
||||||
|
3. Just set `memory.provider: honcho` to reactivate
|
||||||
|
|
||||||
```bash
|
No re-login or re-setup needed. Run `hermes memory setup` and select "honcho" — the wizard detects your existing config.
|
||||||
hermes honcho identity ~/.hermes/SOUL.md
|
|
||||||
```
|
|
||||||
|
|
||||||
This uploads the file content through Honcho's observation pipeline. The AI peer representation is then injected into the system prompt alongside the user's, giving the agent awareness of its own accumulated identity.
|
## Full Documentation
|
||||||
|
|
||||||
```bash
|
See [Memory Providers — Honcho](./memory-providers.md#honcho) for tools, config reference, and details.
|
||||||
hermes honcho identity --show
|
|
||||||
```
|
|
||||||
|
|
||||||
Shows the current AI peer representation from Honcho.
|
|
||||||
|
|
||||||
## Use Cases
|
|
||||||
|
|
||||||
- **Personalized responses** — Honcho learns how each user prefers to communicate
|
|
||||||
- **Goal tracking** — remembers what users are working toward across sessions
|
|
||||||
- **Expertise adaptation** — adjusts technical depth based on user's background
|
|
||||||
- **Cross-platform memory** — same user understanding across CLI, Telegram, Discord, etc.
|
|
||||||
- **Multi-user support** — each user (via messaging platforms) gets their own user model
|
|
||||||
|
|
||||||
:::tip
|
|
||||||
Honcho is fully opt-in — zero behavior change when disabled or unconfigured. All Honcho calls are non-fatal; if the service is unreachable, the agent continues normally.
|
|
||||||
:::
|
|
||||||
|
|
|
||||||
277
website/docs/user-guide/features/memory-providers.md
Normal file
277
website/docs/user-guide/features/memory-providers.md
Normal file
|
|
@ -0,0 +1,277 @@
|
||||||
|
---
|
||||||
|
sidebar_position: 4
|
||||||
|
title: "Memory Providers"
|
||||||
|
description: "External memory provider plugins — Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Memory Providers
|
||||||
|
|
||||||
|
Hermes Agent ships with 7 external memory provider plugins that give the agent persistent, cross-session knowledge beyond the built-in MEMORY.md and USER.md. Only **one** external provider can be active at a time — the built-in memory is always active alongside it.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes memory setup # interactive picker + configuration
|
||||||
|
hermes memory status # check what's active
|
||||||
|
hermes memory off # disable external provider
|
||||||
|
```
|
||||||
|
|
||||||
|
Or set manually in `~/.hermes/config.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
memory:
|
||||||
|
provider: openviking # or honcho, mem0, hindsight, holographic, retaindb, byterover
|
||||||
|
```
|
||||||
|
|
||||||
|
## How It Works
|
||||||
|
|
||||||
|
When a memory provider is active, Hermes automatically:
|
||||||
|
|
||||||
|
1. **Injects provider context** into the system prompt (what the provider knows)
|
||||||
|
2. **Prefetches relevant memories** before each turn (background, non-blocking)
|
||||||
|
3. **Syncs conversation turns** to the provider after each response
|
||||||
|
4. **Extracts memories on session end** (for providers that support it)
|
||||||
|
5. **Mirrors built-in memory writes** to the external provider
|
||||||
|
6. **Adds provider-specific tools** so the agent can search, store, and manage memories
|
||||||
|
|
||||||
|
The built-in memory (MEMORY.md / USER.md) continues to work exactly as before. The external provider is additive.
|
||||||
|
|
||||||
|
## Available Providers
|
||||||
|
|
||||||
|
### Honcho
|
||||||
|
|
||||||
|
AI-native cross-session user modeling with dialectic Q&A, semantic search, and persistent conclusions.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Best for** | Teams using Honcho's user modeling platform |
|
||||||
|
| **Requires** | `pip install honcho-ai` + API key |
|
||||||
|
| **Data storage** | Honcho Cloud |
|
||||||
|
| **Cost** | Honcho pricing |
|
||||||
|
|
||||||
|
**Tools:** `honcho_profile` (peer card), `honcho_search` (semantic search), `honcho_context` (LLM-synthesized), `honcho_conclude` (store facts)
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "honcho"
|
||||||
|
# Or manually:
|
||||||
|
hermes config set memory.provider honcho
|
||||||
|
echo "HONCHO_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Config:** `$HERMES_HOME/honcho.json` — existing Honcho users' configuration and data are fully preserved.
|
||||||
|
|
||||||
|
:::tip Migrating from `hermes honcho`
|
||||||
|
If you previously used `hermes honcho setup`, your config and all server-side data are intact. Just set `memory.provider: honcho` to reactivate via the new system.
|
||||||
|
:::
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### OpenViking
|
||||||
|
|
||||||
|
Context database by Volcengine (ByteDance) with filesystem-style knowledge hierarchy, tiered retrieval, and automatic memory extraction into 6 categories.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Best for** | Self-hosted knowledge management with structured browsing |
|
||||||
|
| **Requires** | `pip install openviking` + running server |
|
||||||
|
| **Data storage** | Self-hosted (local or cloud) |
|
||||||
|
| **Cost** | Free (open-source, AGPL-3.0) |
|
||||||
|
|
||||||
|
**Tools:** `viking_search` (semantic search), `viking_read` (tiered: abstract/overview/full), `viking_browse` (filesystem navigation), `viking_remember` (store facts), `viking_add_resource` (ingest URLs/docs)
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
```bash
|
||||||
|
# Start the OpenViking server first
|
||||||
|
pip install openviking
|
||||||
|
openviking-server
|
||||||
|
|
||||||
|
# Then configure Hermes
|
||||||
|
hermes memory setup # select "openviking"
|
||||||
|
# Or manually:
|
||||||
|
hermes config set memory.provider openviking
|
||||||
|
echo "OPENVIKING_ENDPOINT=http://localhost:1933" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key features:**
|
||||||
|
- Tiered context loading: L0 (~100 tokens) → L1 (~2k) → L2 (full)
|
||||||
|
- Automatic memory extraction on session commit (profile, preferences, entities, events, cases, patterns)
|
||||||
|
- `viking://` URI scheme for hierarchical knowledge browsing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Mem0
|
||||||
|
|
||||||
|
Server-side LLM fact extraction with semantic search, reranking, and automatic deduplication.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Best for** | Hands-off memory management — Mem0 handles extraction automatically |
|
||||||
|
| **Requires** | `pip install mem0ai` + API key |
|
||||||
|
| **Data storage** | Mem0 Cloud |
|
||||||
|
| **Cost** | Mem0 pricing |
|
||||||
|
|
||||||
|
**Tools:** `mem0_profile` (all stored memories), `mem0_search` (semantic search + reranking), `mem0_conclude` (store verbatim facts)
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "mem0"
|
||||||
|
# Or manually:
|
||||||
|
hermes config set memory.provider mem0
|
||||||
|
echo "MEM0_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Config:** `$HERMES_HOME/mem0.json`
|
||||||
|
|
||||||
|
| Key | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `user_id` | `hermes-user` | User identifier |
|
||||||
|
| `agent_id` | `hermes` | Agent identifier |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Hindsight
|
||||||
|
|
||||||
|
Long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval. The `hindsight_reflect` tool provides cross-memory synthesis that no other provider offers.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Best for** | Knowledge graph-based recall with entity relationships |
|
||||||
|
| **Requires** | Cloud: `pip install hindsight-client` + API key. Local: `pip install hindsight` + LLM key |
|
||||||
|
| **Data storage** | Hindsight Cloud or local embedded PostgreSQL |
|
||||||
|
| **Cost** | Hindsight pricing (cloud) or free (local) |
|
||||||
|
|
||||||
|
**Tools:** `hindsight_retain` (store with entity extraction), `hindsight_recall` (multi-strategy search), `hindsight_reflect` (cross-memory synthesis)
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "hindsight"
|
||||||
|
# Or manually:
|
||||||
|
hermes config set memory.provider hindsight
|
||||||
|
echo "HINDSIGHT_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Config:** `$HERMES_HOME/hindsight/config.json`
|
||||||
|
|
||||||
|
| Key | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `mode` | `cloud` | `cloud` or `local` |
|
||||||
|
| `bank_id` | `hermes` | Memory bank identifier |
|
||||||
|
| `budget` | `mid` | Recall thoroughness: `low` / `mid` / `high` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Holographic
|
||||||
|
|
||||||
|
Local SQLite fact store with FTS5 full-text search, trust scoring, and HRR (Holographic Reduced Representations) for compositional algebraic queries.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Best for** | Local-only memory with advanced retrieval, no external dependencies |
|
||||||
|
| **Requires** | Nothing (SQLite is always available). NumPy optional for HRR algebra. |
|
||||||
|
| **Data storage** | Local SQLite |
|
||||||
|
| **Cost** | Free |
|
||||||
|
|
||||||
|
**Tools:** `fact_store` (9 actions: add, search, probe, related, reason, contradict, update, remove, list), `fact_feedback` (helpful/unhelpful rating that trains trust scores)
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "holographic"
|
||||||
|
# Or manually:
|
||||||
|
hermes config set memory.provider holographic
|
||||||
|
```
|
||||||
|
|
||||||
|
**Config:** `config.yaml` under `plugins.hermes-memory-store`
|
||||||
|
|
||||||
|
| Key | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `db_path` | `$HERMES_HOME/memory_store.db` | SQLite database path |
|
||||||
|
| `auto_extract` | `false` | Auto-extract facts at session end |
|
||||||
|
| `default_trust` | `0.5` | Default trust score (0.0–1.0) |
|
||||||
|
|
||||||
|
**Unique capabilities:**
|
||||||
|
- `probe` — entity-specific algebraic recall (all facts about a person/thing)
|
||||||
|
- `reason` — compositional AND queries across multiple entities
|
||||||
|
- `contradict` — automated detection of conflicting facts
|
||||||
|
- Trust scoring with asymmetric feedback (+0.05 helpful / -0.10 unhelpful)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### RetainDB
|
||||||
|
|
||||||
|
Cloud memory API with hybrid search (Vector + BM25 + Reranking), 7 memory types, and delta compression.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Best for** | Teams already using RetainDB's infrastructure |
|
||||||
|
| **Requires** | RetainDB account + API key |
|
||||||
|
| **Data storage** | RetainDB Cloud |
|
||||||
|
| **Cost** | $20/month |
|
||||||
|
|
||||||
|
**Tools:** `retaindb_profile` (user profile), `retaindb_search` (semantic search), `retaindb_context` (task-relevant context), `retaindb_remember` (store with type + importance), `retaindb_forget` (delete memories)
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
```bash
|
||||||
|
hermes memory setup # select "retaindb"
|
||||||
|
# Or manually:
|
||||||
|
hermes config set memory.provider retaindb
|
||||||
|
echo "RETAINDB_API_KEY=your-key" >> ~/.hermes/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ByteRover
|
||||||
|
|
||||||
|
Persistent memory via the `brv` CLI — hierarchical knowledge tree with tiered retrieval (fuzzy text → LLM-driven search). Local-first with optional cloud sync.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Best for** | Developers who want portable, local-first memory with a CLI |
|
||||||
|
| **Requires** | ByteRover CLI (`npm install -g byterover-cli` or [install script](https://byterover.dev)) |
|
||||||
|
| **Data storage** | Local (default) or ByteRover Cloud (optional sync) |
|
||||||
|
| **Cost** | Free (local) or ByteRover pricing (cloud) |
|
||||||
|
|
||||||
|
**Tools:** `brv_query` (search knowledge tree), `brv_curate` (store facts/decisions/patterns), `brv_status` (CLI version + tree stats)
|
||||||
|
|
||||||
|
**Setup:**
|
||||||
|
```bash
|
||||||
|
# Install the CLI first
|
||||||
|
curl -fsSL https://byterover.dev/install.sh | sh
|
||||||
|
|
||||||
|
# Then configure Hermes
|
||||||
|
hermes memory setup # select "byterover"
|
||||||
|
# Or manually:
|
||||||
|
hermes config set memory.provider byterover
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key features:**
|
||||||
|
- Automatic pre-compression extraction (saves insights before context compression discards them)
|
||||||
|
- Knowledge tree stored at `$HERMES_HOME/byterover/` (profile-scoped)
|
||||||
|
- SOC2 Type II certified cloud sync (optional)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Provider Comparison
|
||||||
|
|
||||||
|
| Provider | Storage | Cost | Tools | Dependencies | Unique Feature |
|
||||||
|
|----------|---------|------|-------|-------------|----------------|
|
||||||
|
| **Honcho** | Cloud | Paid | 4 | `honcho-ai` | Dialectic user modeling |
|
||||||
|
| **OpenViking** | Self-hosted | Free | 5 | `openviking` + server | Filesystem hierarchy + tiered loading |
|
||||||
|
| **Mem0** | Cloud | Paid | 3 | `mem0ai` | Server-side LLM extraction |
|
||||||
|
| **Hindsight** | Cloud/Local | Free/Paid | 3 | `hindsight-client` | Knowledge graph + reflect synthesis |
|
||||||
|
| **Holographic** | Local | Free | 2 | None | HRR algebra + trust scoring |
|
||||||
|
| **RetainDB** | Cloud | $20/mo | 5 | `requests` | Delta compression |
|
||||||
|
| **ByteRover** | Local/Cloud | Free/Paid | 3 | `brv` CLI | Pre-compression extraction |
|
||||||
|
|
||||||
|
## Profile Isolation
|
||||||
|
|
||||||
|
Each provider's data is isolated per [profile](/docs/user-guide/profiles):
|
||||||
|
|
||||||
|
- **Local storage providers** (Holographic, ByteRover) use `$HERMES_HOME/` paths which differ per profile
|
||||||
|
- **Config file providers** (Honcho, Mem0, Hindsight) store config in `$HERMES_HOME/` so each profile has its own credentials
|
||||||
|
- **Cloud providers** (RetainDB) auto-derive profile-scoped project names
|
||||||
|
- **Env var providers** (OpenViking) are configured via each profile's `.env` file
|
||||||
|
|
||||||
|
## Building a Memory Provider
|
||||||
|
|
||||||
|
See the [Developer Guide: Memory Provider Plugins](/docs/developer-guide/memory-provider-plugin) for how to create your own.
|
||||||
|
|
@ -207,12 +207,15 @@ memory:
|
||||||
user_char_limit: 1375 # ~500 tokens
|
user_char_limit: 1375 # ~500 tokens
|
||||||
```
|
```
|
||||||
|
|
||||||
## Honcho Integration (Cross-Session User Modeling)
|
## External Memory Providers
|
||||||
|
|
||||||
For deeper, AI-generated user understanding that works across sessions and platforms, you can enable [Honcho Memory](./honcho.md). Honcho runs alongside built-in memory in `hybrid` mode (the default) — `MEMORY.md` and `USER.md` stay as-is, and Honcho adds a persistent user modeling layer on top.
|
For deeper, persistent memory that goes beyond MEMORY.md and USER.md, Hermes ships with 7 external memory provider plugins — including Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, and ByteRover.
|
||||||
|
|
||||||
|
External providers run **alongside** built-in memory (never replacing it) and add capabilities like knowledge graphs, semantic search, automatic fact extraction, and cross-session user modeling.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
hermes honcho setup
|
hermes memory setup # pick a provider and configure it
|
||||||
|
hermes memory status # check what's active
|
||||||
```
|
```
|
||||||
|
|
||||||
See the [Honcho Memory](./honcho.md) docs for full configuration, tools, and CLI reference.
|
See the [Memory Providers](./memory-providers.md) guide for full details on each provider, setup instructions, and comparison.
|
||||||
|
|
|
||||||
|
|
@ -42,6 +42,7 @@ const sidebars: SidebarsConfig = {
|
||||||
'user-guide/features/tools',
|
'user-guide/features/tools',
|
||||||
'user-guide/features/skills',
|
'user-guide/features/skills',
|
||||||
'user-guide/features/memory',
|
'user-guide/features/memory',
|
||||||
|
'user-guide/features/memory-providers',
|
||||||
'user-guide/features/context-files',
|
'user-guide/features/context-files',
|
||||||
'user-guide/features/context-references',
|
'user-guide/features/context-references',
|
||||||
'user-guide/features/personality',
|
'user-guide/features/personality',
|
||||||
|
|
@ -166,6 +167,7 @@ const sidebars: SidebarsConfig = {
|
||||||
items: [
|
items: [
|
||||||
'developer-guide/adding-tools',
|
'developer-guide/adding-tools',
|
||||||
'developer-guide/adding-providers',
|
'developer-guide/adding-providers',
|
||||||
|
'developer-guide/memory-provider-plugin',
|
||||||
'developer-guide/creating-skills',
|
'developer-guide/creating-skills',
|
||||||
'developer-guide/extending-the-cli',
|
'developer-guide/extending-the-cli',
|
||||||
],
|
],
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue