hermes-agent/plugins/memory/mem0/_oss_providers.py
Kartik 2e779d11a0
feat(mem0): v3 API, OSS mode, update/delete tools, telemetry & review fixes (#15624)
* fix: update to version 3 endpoints and adding update and delete tool

* chore: removing the test md file

* fix: prevent circuit breaker on client errors in Mem0 provider

* chore: add telemetry for platform version

* feat: add OSS mode support to Mem0 memory provider

* chore: bump mem0ai dependency to >=2.0.1 in memory plugin

* refactor: enhance dependency checks and embedder config in mem0 backend

* refactor: adjust fact storage message for OSS mode

* refactor: expand user paths, add collection recreation on dimension change for Qdrant

* fix(mem0): make MEM0_USER_ID override gateway-native ids and tag writes with channel

When MEM0_USER_ID was configured (env or mem0.json), the gateway-native id
from kwargs (Telegram numeric id, Discord snowflake, ...) still won, so the
same human ended up under different user_ids per channel and memories never
merged across CLI / Telegram / Slack / Discord. Mirrors openclaw's cfg.userId
pattern: configured override wins, gateway-native id is the fallback.

The legacy "hermes-user" placeholder default written by the setup wizard is
treated as unset to avoid silently bucketing every gateway user together.

Also tag every write with metadata.channel (cli/telegram/discord/...) so the
dashboard can offer per-channel filtered views without coupling identity to
the channel; document the read/write filter asymmetry as intentional
(reads scope to user_id only for cross-agent recall).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: improve Mem0 memory provider backend, pagination, config, and error handling

* refactor: update mem0 telemetry code, docs, and bump version

* fix(mem0): make get_config_schema() return unified schema with mode-aware required flag

Schema always includes api_key field so picker shows "API key / local" for
both modes. In OSS mode api_key.required=False so status won't mislead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: improve mem0 telemetry, add env var key and OSS mode detection

* chore: bump mem0ai lower bound to 2.0.4 (latest SDK release)

* refactor: set telemetry sample rate to 1.0 and update docs for opt‑out

* fix(mem0): resolve 15 correctness, thread-safety, and resource bugs

Thread safety:
- Protect circuit breaker counters with _breaker_lock (race between
  prefetch/sync daemon threads and main thread)
- Wrap sync_turn thread creation in _sync_lock; skip if previous sync
  is still alive after 5 s join to prevent duplicate memory ingestion
- Guard _schedule_flush timer creation under _queue_lock (TOCTOU race)
- Capture local `backend` reference in prefetch/sync closures so
  shutdown() nulling self._backend cannot crash in-flight threads

Correctness:
- Fix bool("false")==True for rerank param; parse string values explicitly
- Guard page/top_k with max(1,...) and move int() inside try blocks
- Fix fact_count=0 always in OSS mode (Memory.add returns list, not dict)
- Fix prefetch() not clearing result when thread still alive after timeout
- Fix atexit.register accumulating on repeated initialize() calls

Backend / setup:
- Handle Qdrant named-vector collections in _recreate_collection_if_dims_changed
  (vectors is a dict; .size access raised AttributeError, swallowed silently)
- Wrap QdrantClient and psycopg2 conn/cursor in try/finally to prevent leaks
- Resolve ollama_bin at top of _ensure_ollama; use it for ollama pull
- Fix embedder key lookup when LLM provider has no env_var (e.g. ollama)

Also: remove _telemetry_enabled cache (env var check is cheap), bump
required mem0ai to >=2.0.7, minor README wording fix.

* fix(mem0): fix brittle qdrant path test + add telemetry sample-rate docs

- Replace generator-throw lambda with a proper def in
  test_qdrant_path_not_writable; use tmp_path instead of a hardcoded
  /nonexistent path so the test is root-safe
- Add MEM0_TELEMETRY_SAMPLE_RATE to memory-providers.md (was only
  in the plugin README, not the user-guide docs)

* revert: remove MEM0_TELEMETRY_SAMPLE_RATE from user-guide docs

* refactor: remove telemetry from mem0 plugin and update documentation

* fix(mem0): set stdin=DEVNULL on setup subprocess calls

The TUI stdin guard (scripts/check_subprocess_stdin.py) requires every
subprocess call in plugin code to set stdin= so it can't inherit the
gateway's JSON-RPC stdin fd. Muzzle the docker/ollama calls in the OSS
setup wizard with stdin=subprocess.DEVNULL (none need interactive input).
Also covers the docker-inspect call the linter's regex misses.

---------

Co-authored-by: chaithanyak42 <chaithanya.kumar42a@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-22 12:30:47 +00:00

84 lines
2.5 KiB
Python

"""OSS provider definitions for LLM, embedder, and vector store."""
from __future__ import annotations
import os
from typing import Any
LLM_PROVIDERS: dict[str, dict[str, Any]] = {
"openai": {
"label": "OpenAI",
"needs_key": True,
"env_var": "OPENAI_API_KEY",
"default_model": "gpt-5-mini",
},
"ollama": {
"label": "Ollama (local)",
"needs_key": False,
"default_model": "llama3.1:8b",
"default_url": "http://localhost:11434",
"pip_dep": "ollama",
},
}
EMBEDDER_PROVIDERS: dict[str, dict[str, Any]] = {
"openai": {
"label": "OpenAI",
"needs_key": True,
"env_var": "OPENAI_API_KEY",
"default_model": "text-embedding-3-small",
"dims": 1536,
},
"ollama": {
"label": "Ollama (local)",
"needs_key": False,
"default_model": "nomic-embed-text",
"default_url": "http://localhost:11434",
"dims": 768,
"pip_dep": "ollama",
},
}
VECTOR_PROVIDERS: dict[str, dict[str, Any]] = {
"qdrant": {
"label": "Qdrant",
"default_config": {"path": os.path.expanduser("~/.hermes/mem0_qdrant")},
"pip_dep": "qdrant-client",
},
"pgvector": {
"label": "PGVector",
"default_config": {"host": "localhost", "port": 5432, "user": os.getenv("USER", "postgres"), "dbname": "postgres"},
"pip_dep": "psycopg2-binary",
},
}
KNOWN_DIMS: dict[str, int] = {
"text-embedding-3-small": 1536,
"text-embedding-3-large": 3072,
"text-embedding-ada-002": 1536,
"nomic-embed-text": 768,
}
def validate_oss_config(oss_config: dict) -> list[str]:
"""Validate an OSS config dict. Returns list of error strings (empty = valid)."""
errors: list[str] = []
for section, registry in [("llm", LLM_PROVIDERS), ("embedder", EMBEDDER_PROVIDERS),
("vector_store", VECTOR_PROVIDERS)]:
block = oss_config.get(section)
if not block or not isinstance(block, dict):
errors.append(f"Missing required section: {section}")
continue
provider_id = block.get("provider", "")
if provider_id not in registry:
valid = ", ".join(registry.keys())
errors.append(f"Unknown {section} provider '{provider_id}'. Valid: {valid}")
vs = oss_config.get("vector_store", {})
if vs.get("provider") == "pgvector":
cfg = vs.get("config", {})
if not cfg.get("user"):
errors.append("PGVector requires 'user' in vector_store.config")
return errors