mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-05-09 03:11:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
hermes-agent/plugins/memory/hindsight
History
Teknium cc38282b04 feat(cross-platform): psutil for PID/process management + Windows footgun checker 
## Why

Hermes supports Linux, macOS, and native Windows, but the codebase grew up
POSIX-first and has accumulated patterns that silently break (or worse,
silently kill!) on Windows:

- `os.kill(pid, 0)` as a liveness probe — on Windows this maps to
  CTRL_C_EVENT and broadcasts Ctrl+C to the target's entire console
  process group (bpo-14484, open since 2012).
- `os.killpg` — doesn't exist on Windows at all (AttributeError).
- `os.setsid` / `os.getuid` / `os.geteuid` — same.
- `signal.SIGKILL` / `signal.SIGHUP` / `signal.SIGUSR1` — module-attr
  errors at runtime on Windows.
- `open(path)` / `open(path, "r")` without explicit encoding= — inherits
  the platform default, which is cp1252/mbcs on Windows (UTF-8 on POSIX),
  causing mojibake round-tripping between hosts.
- `wmic` — removed from Windows 10 21H1+.

This commit does three things:

1. Makes `psutil` a core dependency and migrates critical callsites to it.
2. Adds a grep-based CI gate (`scripts/check-windows-footguns.py`) that
   blocks new instances of any of the above patterns.
3. Fixes every existing instance in the codebase so the baseline is clean.

## What changed

### 1. psutil as a core dependency (pyproject.toml)

Added `psutil>=5.9.0,<8` to core deps. psutil is the canonical
cross-platform answer for "is this PID alive" and "kill this process
tree" — its `pid_exists()` uses `OpenProcess + GetExitCodeProcess` on
Windows (NOT a signal call), and its `Process.children(recursive=True)`
+ `.kill()` combo replaces `os.killpg()` portably.

### 2. `gateway/status.py::_pid_exists`

Rewrote to call `psutil.pid_exists()` first, falling back to the
hand-rolled ctypes `OpenProcess + WaitForSingleObject` dance on Windows
(and `os.kill(pid, 0)` on POSIX) only if psutil is somehow missing —
e.g. during the scaffold phase of a fresh install before pip finishes.

### 3. `os.killpg` migration to psutil (7 callsites, 5 files)

- `tools/code_execution_tool.py`
- `tools/process_registry.py`
- `tools/tts_tool.py`
- `tools/environments/local.py` (3 sites kept as-is, suppressed with
  `# windows-footgun: ok` — the pgid semantics psutil can't replicate,
  and the calls are already Windows-guarded at the outer branch)
- `gateway/platforms/whatsapp.py`

### 4. `scripts/check-windows-footguns.py` (NEW, 500 lines)

Grep-based checker with 11 rules covering every Windows cross-platform
footgun we've hit so far:

1. `os.kill(pid, 0)` — the silent killer
2. `os.setsid` without guard
3. `os.killpg` (recommends psutil)
4. `os.getuid` / `os.geteuid` / `os.getgid`
5. `os.fork`
6. `signal.SIGKILL`
7. `signal.SIGHUP/SIGUSR1/SIGUSR2/SIGALRM/SIGCHLD/SIGPIPE/SIGQUIT`
8. `subprocess` shebang script invocation
9. `wmic` without `shutil.which` guard
10. Hardcoded `~/Desktop` (OneDrive trap)
11. `asyncio.add_signal_handler` without try/except
12. `open()` without `encoding=` on text mode

Features:
- Triple-quoted-docstring aware (won't flag prose inside docstrings)
- Trailing-comment aware (won't flag mentions in `# os.kill(pid, 0)` comments)
- Guard-hint aware (skips lines with `hasattr(os, ...)`,
  `shutil.which(...)`, `if platform.system() != 'Windows'`, etc.)
- Inline suppression with `# windows-footgun: ok — <reason>`
- `--list` to print all rules with fixes
- `--all` / `--diff <ref>` / staged-files (default) modes
- Scans 380 files in under 2 seconds

### 5. CI integration

A GitHub Actions workflow that runs the checker on every PR and push is
staged at `/tmp/hermes-stash/windows-footguns.yml` — not included in this
commit because the GH token on the push machine lacks `workflow` scope.
A maintainer with `workflow` permissions should add it as
`.github/workflows/windows-footguns.yml` in a follow-up. Content:

```yaml
name: Windows footgun check
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: {python-version: "3.11"}
      - run: python scripts/check-windows-footguns.py --all
```

### 6. CONTRIBUTING.md — "Cross-Platform Compatibility" expansion

Expanded from 5 to 16 rules, each with message, example, and fix.
Recommends psutil as the preferred API for PID / process-tree operations.

### 7. Baseline cleanup (91 → 0 findings)

- 14 `open()` sites → added `encoding='utf-8'` (internal logs/caches) or
  `encoding='utf-8-sig'` (user-editable files that Notepad may BOM)
- 23 POSIX-only callsites in systemd helpers, pty_bridge, and plugin
  tool subprocess management → annotated with
  `# windows-footgun: ok — <reason>`
- 7 `os.killpg` sites → migrated to psutil (see §3 above)

## Verification

```
$ python scripts/check-windows-footguns.py --all
✓ No Windows footguns found (380 file(s) scanned).

$ python -c "from gateway.status import _pid_exists; import os
> print('self:', _pid_exists(os.getpid())); print('bogus:', _pid_exists(999999))"
self: True
bogus: False
```

Proof-of-repro that `os.kill(pid, 0)` was actually killing processes
before this fix — see commit `1cbe39914` and bpo-14484. This commit
removes the last hand-rolled ctypes path from the hot liveness-check
path and defers to the best-maintained cross-platform answer.
2026-05-08 14:27:40 -07:00
..
__init__.py feat(cross-platform): psutil for PID/process management + Windows footgun checker 2026-05-08 14:27:40 -07:00
plugin.yaml feat(hindsight): feature parity, setup wizard, and config improvements 2026-04-08 23:54:15 -07:00
README.md feat(hindsight): optional bank_id_template for per-agent / per-user banks 2026-04-24 03:38:17 -07:00

README.md

Hindsight Memory Provider

Long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval. Supports cloud, local embedded, and local external modes.

Requirements

  • Cloud: API key from ui.hindsight.vectorize.io
  • Local Embedded: API key for a supported LLM provider (OpenAI, Anthropic, Gemini, Groq, OpenRouter, MiniMax, Ollama, or any OpenAI-compatible endpoint). Embeddings and reranking run locally — no additional API keys needed.
  • Local External: A running Hindsight instance (Docker or self-hosted) reachable over HTTP.

Setup

hermes memory setup    # select "hindsight"

The setup wizard will install dependencies automatically via uv and walk you through configuration.

Or manually (cloud mode with defaults):

hermes config set memory.provider hindsight
echo "HINDSIGHT_API_KEY=your-key" >> ~/.hermes/.env

Cloud

Connects to the Hindsight Cloud API. Requires an API key from ui.hindsight.vectorize.io.

Local Embedded

Hermes spins up a local Hindsight daemon with built-in PostgreSQL. Requires an LLM API key for memory extraction and synthesis. The daemon starts automatically in the background on first use and stops after 5 minutes of inactivity.

Supports any OpenAI-compatible LLM endpoint (llama.cpp, vLLM, LM Studio, etc.) — pick openai_compatible as the provider and enter the base URL.

Daemon startup logs: ~/.hermes/logs/hindsight-embed.log Daemon runtime logs: ~/.hindsight/profiles/<profile>.log

To open the Hindsight web UI (local embedded mode only):

hindsight-embed -p hermes ui start

Local External

Points the plugin at an existing Hindsight instance you're already running (Docker, self-hosted, etc.). No daemon management — just a URL and an optional API key.

Config

Config file: ~/.hermes/hindsight/config.json

Connection

Key Default Description
mode cloud cloud, local_embedded, or local_external
api_url https://api.hindsight.vectorize.io API URL (cloud and local_external modes)

Memory Bank

Key Default Description
bank_id hermes Memory bank name (static fallback used when bank_id_template is unset or resolves empty)
bank_id_template — Optional template to derive the bank name dynamically. Placeholders: {profile}, {workspace}, {platform}, {user}, {session}. Example: hermes-{profile} isolates memory per active Hermes profile. Empty placeholders collapse cleanly (e.g. hermes-{user} with no user becomes hermes).
bank_mission — Reflect mission (identity/framing for reflect reasoning). Applied via Banks API.
bank_retain_mission — Retain mission (steers what gets extracted). Applied via Banks API.

Recall

Key Default Description
recall_budget mid Recall thoroughness: low / mid / high
recall_prefetch_method recall Auto-recall method: recall (raw facts) or reflect (LLM synthesis)
recall_max_tokens 4096 Maximum tokens for recall results
recall_max_input_chars 800 Maximum input query length for auto-recall
recall_prompt_preamble — Custom preamble for recalled memories in context
recall_tags — Tags to filter when searching memories
recall_tags_match any Tag matching mode: any / all / any_strict / all_strict
auto_recall true Automatically recall memories before each turn

Retain

Key Default Description
auto_retain true Automatically retain conversation turns
retain_async true Process retain asynchronously on the Hindsight server
retain_every_n_turns 1 Retain every N turns (1 = every turn)
retain_context conversation between Hermes Agent and the User Context label for retained memories
retain_tags — Default tags applied to retained memories; merged with per-call tool tags
retain_source — Optional metadata.source attached to retained memories
retain_user_prefix User Label used before user turns in auto-retained transcripts
retain_assistant_prefix Assistant Label used before assistant turns in auto-retained transcripts

Integration

Key Default Description
memory_mode hybrid How memories are integrated into the agent

memory_mode:

  • hybrid — automatic context injection + tools available to the LLM
  • context — automatic injection only, no tools exposed
  • tools — tools only, no automatic injection

Local Embedded LLM

Key Default Description
llm_provider openai openai, anthropic, gemini, groq, openrouter, minimax, ollama, lmstudio, openai_compatible
llm_model per-provider Model name (e.g. gpt-4o-mini, qwen/qwen3.5-9b)
llm_base_url — Endpoint URL for openai_compatible (e.g. http://192.168.1.10:8080/v1)

The LLM API key is stored in ~/.hermes/.env as HINDSIGHT_LLM_API_KEY.

Tools

Available in hybrid and tools memory modes:

Tool Description
hindsight_retain Store information with auto entity extraction; supports optional per-call tags
hindsight_recall Multi-strategy search (semantic + entity graph)
hindsight_reflect Cross-memory synthesis (LLM-powered)

Environment Variables

Variable Description
HINDSIGHT_API_KEY API key for Hindsight Cloud
HINDSIGHT_LLM_API_KEY LLM API key for local mode
HINDSIGHT_API_LLM_BASE_URL LLM Base URL for local mode (e.g. OpenRouter)
HINDSIGHT_API_URL Override API endpoint
HINDSIGHT_BANK_ID Override bank name
HINDSIGHT_BUDGET Override recall budget
HINDSIGHT_MODE Override mode (cloud, local_embedded, local_external)

Client Version

Requires hindsight-client >= 0.4.22. The plugin auto-upgrades on session start if an older version is detected.