mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-06-24 10:52:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
hermes-agent/tests/plugins/memory/test_mem0_v3.py
Kartik 2e779d11a0
feat(mem0): v3 API, OSS mode, update/delete tools, telemetry & review fixes (#15624) 
* fix: update to version 3 endpoints and adding update and delete tool

* chore: removing the test md file

* fix: prevent circuit breaker on client errors in Mem0 provider

* chore: add telemetry for platform version

* feat: add OSS mode support to Mem0 memory provider

* chore: bump mem0ai dependency to >=2.0.1 in memory plugin

* refactor: enhance dependency checks and embedder config in mem0 backend

* refactor: adjust fact storage message for OSS mode

* refactor: expand user paths, add collection recreation on dimension change for Qdrant

* fix(mem0): make MEM0_USER_ID override gateway-native ids and tag writes with channel

When MEM0_USER_ID was configured (env or mem0.json), the gateway-native id
from kwargs (Telegram numeric id, Discord snowflake, ...) still won, so the
same human ended up under different user_ids per channel and memories never
merged across CLI / Telegram / Slack / Discord. Mirrors openclaw's cfg.userId
pattern: configured override wins, gateway-native id is the fallback.

The legacy "hermes-user" placeholder default written by the setup wizard is
treated as unset to avoid silently bucketing every gateway user together.

Also tag every write with metadata.channel (cli/telegram/discord/...) so the
dashboard can offer per-channel filtered views without coupling identity to
the channel; document the read/write filter asymmetry as intentional
(reads scope to user_id only for cross-agent recall).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: improve Mem0 memory provider backend, pagination, config, and error handling

* refactor: update mem0 telemetry code, docs, and bump version

* fix(mem0): make get_config_schema() return unified schema with mode-aware required flag

Schema always includes api_key field so picker shows "API key / local" for
both modes. In OSS mode api_key.required=False so status won't mislead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: improve mem0 telemetry, add env var key and OSS mode detection

* chore: bump mem0ai lower bound to 2.0.4 (latest SDK release)

* refactor: set telemetry sample rate to 1.0 and update docs for opt‑out

* fix(mem0): resolve 15 correctness, thread-safety, and resource bugs

Thread safety:
- Protect circuit breaker counters with _breaker_lock (race between
  prefetch/sync daemon threads and main thread)
- Wrap sync_turn thread creation in _sync_lock; skip if previous sync
  is still alive after 5 s join to prevent duplicate memory ingestion
- Guard _schedule_flush timer creation under _queue_lock (TOCTOU race)
- Capture local `backend` reference in prefetch/sync closures so
  shutdown() nulling self._backend cannot crash in-flight threads

Correctness:
- Fix bool("false")==True for rerank param; parse string values explicitly
- Guard page/top_k with max(1,...) and move int() inside try blocks
- Fix fact_count=0 always in OSS mode (Memory.add returns list, not dict)
- Fix prefetch() not clearing result when thread still alive after timeout
- Fix atexit.register accumulating on repeated initialize() calls

Backend / setup:
- Handle Qdrant named-vector collections in _recreate_collection_if_dims_changed
  (vectors is a dict; .size access raised AttributeError, swallowed silently)
- Wrap QdrantClient and psycopg2 conn/cursor in try/finally to prevent leaks
- Resolve ollama_bin at top of _ensure_ollama; use it for ollama pull
- Fix embedder key lookup when LLM provider has no env_var (e.g. ollama)

Also: remove _telemetry_enabled cache (env var check is cheap), bump
required mem0ai to >=2.0.7, minor README wording fix.

* fix(mem0): fix brittle qdrant path test + add telemetry sample-rate docs

- Replace generator-throw lambda with a proper def in
  test_qdrant_path_not_writable; use tmp_path instead of a hardcoded
  /nonexistent path so the test is root-safe
- Add MEM0_TELEMETRY_SAMPLE_RATE to memory-providers.md (was only
  in the plugin README, not the user-guide docs)

* revert: remove MEM0_TELEMETRY_SAMPLE_RATE from user-guide docs

* refactor: remove telemetry from mem0 plugin and update documentation

* fix(mem0): set stdin=DEVNULL on setup subprocess calls

The TUI stdin guard (scripts/check_subprocess_stdin.py) requires every
subprocess call in plugin code to set stdin= so it can't inherit the
gateway's JSON-RPC stdin fd. Muzzle the docker/ollama calls in the OSS
setup wizard with stdin=subprocess.DEVNULL (none need interactive input).
Also covers the docker-inspect call the linter's regex misses.

---------

Co-authored-by: chaithanyak42 <chaithanya.kumar42a@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-22 12:30:47 +00:00

463 lines
19 KiB
Python
Raw Blame History

"""Tests for Mem0 v3 API — new tool names, paginated responses, update/delete tools."""
import json
import pytest
from plugins.memory.mem0 import Mem0MemoryProvider
class FakeBackend:
"""Fake Mem0Backend for provider-level tests."""
def __init__(self, search_results=None, all_results=None):
self._search_results = search_results or []
self._all_results = all_results or {"results": [], "count": 0}
self.captured = []
def search(self, query, *, filters, top_k=10, rerank=True):
self.captured.append(("search", query, {"filters": filters, "top_k": top_k, "rerank": rerank}))
return self._search_results
def get_all(self, *, filters, page=1, page_size=100):
self.captured.append(("get_all", {"filters": filters, "page": page, "page_size": page_size}))
return self._all_results
def add(self, messages, *, user_id, agent_id, infer=False, metadata=None):
self.captured.append((
"add",
messages,
{"user_id": user_id, "agent_id": agent_id, "infer": infer, "metadata": metadata},
))
return {"status": "PENDING", "event_id": "evt-test-123"}
def update(self, memory_id, text):
self.captured.append(("update", memory_id, text))
return {"result": "Memory updated.", "memory_id": memory_id}
def delete(self, memory_id):
self.captured.append(("delete", memory_id))
return {"result": "Memory deleted.", "memory_id": memory_id}
class TestMem0V3Tools:
"""Test v3 tool names and response handling."""
def _make_provider(self, monkeypatch, backend):
provider = Mem0MemoryProvider()
provider.initialize("test-session")
provider._user_id = "u123"
provider._agent_id = "hermes"
provider._backend = backend
return provider
def test_list_returns_paginated_with_ids(self, monkeypatch):
backend = FakeBackend(all_results={
"count": 2,
"results": [
{"id": "mem-1", "memory": "alpha"},
{"id": "mem-2", "memory": "beta"},
]
})
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_list", {}))
assert result["count"] == 2
assert result["results"][0]["id"] == "mem-1"
assert result["results"][0]["memory"] == "alpha"
def test_list_pagination_params(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
provider.handle_tool_call("mem0_list", {"page": 2, "page_size": 50})
assert backend.captured[0][1]["page"] == 2
assert backend.captured[0][1]["page_size"] == 50
def test_list_empty(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_list", {}))
assert result["result"] == "No memories stored yet."
def test_search_returns_ids(self, monkeypatch):
backend = FakeBackend(search_results=[{"id": "mem-1", "memory": "foo", "score": 0.9}])
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_search", {"query": "test"}))
assert result["results"][0]["id"] == "mem-1"
def test_search_uses_filters(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
provider.handle_tool_call("mem0_search", {"query": "hello", "top_k": 3})
assert backend.captured[0][2]["filters"] == {"user_id": "u123"}
assert backend.captured[0][2]["top_k"] == 3
def test_search_rerank_default_true(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
provider.handle_tool_call("mem0_search", {"query": "test"})
assert backend.captured[0][2]["rerank"] is True
def test_search_rerank_override_false(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
provider.handle_tool_call("mem0_search", {"query": "test", "rerank": False})
assert backend.captured[0][2]["rerank"] is False
def test_add_uses_content_param(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_add", {"content": "user likes dark mode"}))
assert len(backend.captured) == 1
call = backend.captured[0]
assert call[2]["infer"] is False
assert call[2]["user_id"] == "u123"
assert call[2]["agent_id"] == "hermes"
assert "event_id" in result
def test_add_returns_event_id(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_add", {"content": "test"}))
assert result["event_id"] == "evt-test-123"
def test_add_missing_content(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_add", {}))
assert "error" in result
def test_old_tool_names_return_unknown(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_profile", {}))
assert "error" in result
result = json.loads(provider.handle_tool_call("mem0_conclude", {}))
assert "error" in result
class TestMem0UpdateDelete:
def _make_provider(self, monkeypatch, backend):
provider = Mem0MemoryProvider()
provider.initialize("test-session")
provider._user_id = "u123"
provider._agent_id = "hermes"
provider._backend = backend
return provider
def test_update_calls_sdk(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call(
"mem0_update", {"memory_id": "mem-1", "text": "updated fact"}
))
assert backend.captured[0][1] == "mem-1"
assert backend.captured[0][2] == "updated fact"
assert result["result"] == "Memory updated."
assert result["memory_id"] == "mem-1"
def test_update_missing_memory_id(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_update", {"text": "no id"}))
assert "error" in result
def test_update_missing_text(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_update", {"memory_id": "mem-1"}))
assert "error" in result
def test_delete_calls_sdk(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call(
"mem0_delete", {"memory_id": "mem-1"}
))
assert backend.captured[0][1] == "mem-1"
assert result["result"] == "Memory deleted."
def test_delete_missing_memory_id(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_delete", {}))
assert "error" in result
class TestMem0ErrorHandling:
def _make_provider(self, monkeypatch, backend):
provider = Mem0MemoryProvider()
provider.initialize("test-session")
provider._user_id = "u123"
provider._agent_id = "hermes"
provider._backend = backend
return provider
def test_update_404_no_circuit_breaker(self, monkeypatch):
backend = FakeBackend()
backend.update = lambda mid, text: (_ for _ in ()).throw(Exception("404 Not Found"))
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call(
"mem0_update", {"memory_id": "bad-id", "text": "x"}
))
assert "error" in result
assert provider._consecutive_failures == 0
def test_delete_404_no_circuit_breaker(self, monkeypatch):
backend = FakeBackend()
backend.delete = lambda mid: (_ for _ in ()).throw(Exception("404 not found"))
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call(
"mem0_delete", {"memory_id": "bad-id"}
))
assert "error" in result
assert provider._consecutive_failures == 0
def test_update_validation_error_no_circuit_breaker(self, monkeypatch):
"""ValidationError (bad UUID format) should not trip circuit breaker."""
class ValidationError(Exception):
pass
backend = FakeBackend()
backend.update = lambda mid, text: (_ for _ in ()).throw(
ValidationError('{"error":"memory_id should be a valid UUID"}')
)
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call(
"mem0_update", {"memory_id": "not-a-uuid", "text": "x"}
))
assert "error" in result
assert provider._consecutive_failures == 0
def test_delete_validation_error_no_circuit_breaker(self, monkeypatch):
class ValidationError(Exception):
pass
backend = FakeBackend()
backend.delete = lambda mid: (_ for _ in ()).throw(
ValidationError('{"error":"memory_id should be a valid UUID"}')
)
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call(
"mem0_delete", {"memory_id": "not-a-uuid"}
))
assert "error" in result
assert provider._consecutive_failures == 0
def test_update_5xx_trips_circuit_breaker(self, monkeypatch):
backend = FakeBackend()
backend.update = lambda mid, text: (_ for _ in ()).throw(Exception("500 Internal Server Error"))
provider = self._make_provider(monkeypatch, backend)
provider.handle_tool_call("mem0_update", {"memory_id": "mem-1", "text": "x"})
assert provider._consecutive_failures == 1
class TestMem0V3Internal:
def _make_provider(self, monkeypatch, backend):
provider = Mem0MemoryProvider()
provider.initialize("test-session")
provider._user_id = "u123"
provider._agent_id = "hermes"
provider._backend = backend
return provider
def test_sync_turn_explicit_kwargs(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
provider.sync_turn("user said", "assistant replied", session_id="s1")
provider._sync_thread.join(timeout=2)
assert len(backend.captured) == 1
call = backend.captured[0]
assert call[2]["user_id"] == "u123"
assert call[2]["agent_id"] == "hermes"
assert call[2]["infer"] is True
def test_old_tool_names_return_unknown(self, monkeypatch):
backend = FakeBackend()
provider = self._make_provider(monkeypatch, backend)
result = json.loads(provider.handle_tool_call("mem0_profile", {}))
assert "error" in result
result = json.loads(provider.handle_tool_call("mem0_conclude", {}))
assert "error" in result
class TestMem0V3Config:
def test_tool_schemas_five_tools(self):
provider = Mem0MemoryProvider()
schemas = provider.get_tool_schemas()
names = [s["name"] for s in schemas]
assert names == ["mem0_list", "mem0_search", "mem0_add", "mem0_update", "mem0_delete"]
def test_system_prompt_new_tool_names(self):
provider = Mem0MemoryProvider()
provider._user_id = "test"
block = provider.system_prompt_block()
assert "mem0_search" in block
assert "mem0_add" in block
assert "mem0_list" in block
assert "mem0_update" in block
assert "mem0_delete" in block
assert "mem0_profile" not in block
assert "mem0_conclude" not in block
def test_system_prompt_shows_platform_mode(self):
provider = Mem0MemoryProvider()
provider._user_id = "test"
provider._mode = "platform"
block = provider.system_prompt_block()
assert "platform" in block
assert "Rerank" in block
def test_system_prompt_shows_oss_mode(self):
provider = Mem0MemoryProvider()
provider._user_id = "test"
provider._mode = "oss"
block = provider.system_prompt_block()
assert "OSS" in block
assert "Rerank" not in block
def test_search_schema_has_rerank(self):
"""rerank property available in SEARCH_SCHEMA for platform mode."""
provider = Mem0MemoryProvider()
schemas = provider.get_tool_schemas()
search = next(s for s in schemas if s["name"] == "mem0_search")
assert "rerank" in search["parameters"]["properties"]
assert search["parameters"]["properties"]["rerank"]["type"] == "boolean"
class TestMem0ModeSwitch:
def test_default_mode_is_platform(self, monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
monkeypatch.setenv("MEM0_API_KEY", "test-key")
provider = Mem0MemoryProvider()
provider.initialize("test")
assert provider._mode == "platform"
def test_missing_mode_key_defaults_platform(self, monkeypatch, tmp_path):
"""Backward compat: old mem0.json without mode key works."""
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
config_path = tmp_path / "mem0.json"
config_path.write_text('{"user_id": "old-user"}')
monkeypatch.setenv("MEM0_API_KEY", "test-key")
provider = Mem0MemoryProvider()
provider.initialize("test")
assert provider._mode == "platform"
assert provider._user_id == "old-user"
def test_is_available_platform_needs_key(self, monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
monkeypatch.delenv("MEM0_API_KEY", raising=False)
provider = Mem0MemoryProvider()
assert provider.is_available() is False
def test_is_available_oss_needs_vector(self, monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
config_path = tmp_path / "mem0.json"
config_path.write_text('{"mode": "oss", "oss": {"vector_store": {"provider": "qdrant"}}}')
provider = Mem0MemoryProvider()
assert provider.is_available() is True
def test_is_available_oss_no_vector(self, monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
config_path = tmp_path / "mem0.json"
config_path.write_text('{"mode": "oss", "oss": {}}')
provider = Mem0MemoryProvider()
assert provider.is_available() is False
def test_tool_schemas_unchanged(self):
provider = Mem0MemoryProvider()
schemas = provider.get_tool_schemas()
names = [s["name"] for s in schemas]
assert names == ["mem0_list", "mem0_search", "mem0_add", "mem0_update", "mem0_delete"]
def test_system_prompt_includes_mode(self):
provider = Mem0MemoryProvider()
provider._user_id = "test"
provider._mode = "oss"
block = provider.system_prompt_block()
assert "mem0_search" in block
assert "mem0_list" in block
assert "OSS" in block
class TestMem0UserIdResolution:
"""user_id resolution: configured override > gateway-native id > placeholder.
Same human across CLI / Telegram / Discord / Slack / etc. should map to
the same memory store when MEM0_USER_ID is set, and only fall back to the
gateway-native id when it isn't.
"""
def _provider(self, monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
monkeypatch.setenv("MEM0_API_KEY", "test-key")
provider = Mem0MemoryProvider()
# Skip backend instantiation — we only care about identity resolution.
provider._create_backend = lambda: None # type: ignore[method-assign]
return provider
def test_env_override_beats_gateway_native_id(self, monkeypatch, tmp_path):
monkeypatch.setenv("MEM0_USER_ID", "ryan@example.com")
provider = self._provider(monkeypatch, tmp_path)
provider.initialize("test", user_id="123456789", platform="telegram")
assert provider._user_id == "ryan@example.com"
def test_file_override_beats_gateway_native_id(self, monkeypatch, tmp_path):
monkeypatch.delenv("MEM0_USER_ID", raising=False)
(tmp_path / "mem0.json").write_text('{"user_id": "ryan@example.com"}')
provider = self._provider(monkeypatch, tmp_path)
provider.initialize("test", user_id="123456789", platform="telegram")
assert provider._user_id == "ryan@example.com"
def test_unset_falls_back_to_gateway_native_id(self, monkeypatch, tmp_path):
monkeypatch.delenv("MEM0_USER_ID", raising=False)
provider = self._provider(monkeypatch, tmp_path)
provider.initialize("test", user_id="123456789", platform="telegram")
assert provider._user_id == "123456789"
def test_unset_and_no_kwargs_falls_back_to_default(self, monkeypatch, tmp_path):
monkeypatch.delenv("MEM0_USER_ID", raising=False)
provider = self._provider(monkeypatch, tmp_path)
provider.initialize("test")
assert provider._user_id == "hermes-user"
def test_legacy_placeholder_in_config_does_not_override_kwargs(self, monkeypatch, tmp_path):
# Setup wizard historically wrote {"user_id": "hermes-user"} as the
# suggested default. Treat that placeholder as unset so users on
# gateways still get gateway-native ids — not silent collisions.
monkeypatch.delenv("MEM0_USER_ID", raising=False)
(tmp_path / "mem0.json").write_text('{"user_id": "hermes-user"}')
provider = self._provider(monkeypatch, tmp_path)
provider.initialize("test", user_id="123456789", platform="telegram")
assert provider._user_id == "123456789"
class TestMem0WriteMetadata:
"""Writes carry metadata.channel so per-channel filtered views are possible
without coupling identity to the channel.
"""
def _make_provider(self, channel: str = "cli"):
provider = Mem0MemoryProvider()
provider._user_id = "u123"
provider._agent_id = "hermes"
provider._channel = channel
provider._backend = FakeBackend()
return provider
def test_add_tool_passes_channel_metadata(self):
provider = self._make_provider("telegram")
provider.handle_tool_call("mem0_add", {"content": "user likes dark mode"})
call = provider._backend.captured[-1]
assert call[2]["metadata"] == {"channel": "telegram"}
def test_sync_turn_passes_channel_metadata(self):
provider = self._make_provider("discord")
provider.sync_turn("hi", "hello", session_id="s")
# sync_turn fires a daemon thread; wait for it.
if provider._sync_thread:
provider._sync_thread.join(timeout=5.0)
adds = [c for c in provider._backend.captured if c[0] == "add"]
assert adds, "expected an add call from sync_turn"
assert adds[-1][2]["metadata"] == {"channel": "discord"}
Reference in a new issue View git blame Copy permalink