mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-30 01:41:43 +00:00
42cc905c13
Opt-in Langfuse tracing for Hermes conversations — LLM calls, tool usage, usage/cost breakdown per span. Hooks into pre/post_api_request, pre/post_llm_call, pre/post_tool_call. SDK is optional; missing SDK or credentials renders the plugin inert. Salvaged from PR #16845 by @kshitijk4poor, who wrote the plugin (~875 LOC, 6 hooks, Langfuse usage-details/cost-details normalization, read_file payload summarization). Salvage scope (why this isn't PR #16845 as-authored): - Lives at plugins/observability/langfuse/ (standalone kind, opt-in via plugins.enabled) instead of a new parallel optional-plugins/ directory. Standalone bundled plugins are already opt-in — only their plugin.yaml is scanned at startup; the Python module is not imported unless the user enables it. The premise of optional-plugins/ (avoid import cost for users who don't want it) is already solved by the existing plugin system. - Dropped the triple activation gate (plugins.enabled + plugins.langfuse.enabled + HERMES_LANGFUSE_ENABLED). The Hermes plugin system's own enable/disable is authoritative; runtime credentials gate whether the hook actually traces. - Rewrote _is_enabled() → cached _get_langfuse() with an _INIT_FAILED sentinel. The original called hermes_cli.config.load_config() from every hook invocation (full yaml parse + deep merge + env expansion on every pre/post_tool_call, potentially 100+ times per turn). The cached version reads env once and returns the cached client or None on every subsequent call with zero further work. - hermes tools → Langfuse Observability post-setup adds observability/langfuse to plugins.enabled directly (via _save_enabled_set) instead of going through an install-copy flow. Enable: hermes tools # interactive hermes plugins enable observability/langfuse # manual Required env (set by `hermes tools` or in ~/.hermes/.env): HERMES_LANGFUSE_PUBLIC_KEY HERMES_LANGFUSE_SECRET_KEY HERMES_LANGFUSE_BASE_URL # optional Co-authored-by: kshitijk4poor <kshitijk4poor@gmail.com>
53 lines
1.3 KiB
Markdown
53 lines
1.3 KiB
Markdown
# Langfuse Observability Plugin
|
|
|
|
This plugin ships bundled with Hermes but is **opt-in** — it only loads when
|
|
you explicitly enable it.
|
|
|
|
## Enable
|
|
|
|
Pick one:
|
|
|
|
```bash
|
|
# Interactive: walks you through credentials + SDK install + enable
|
|
hermes tools # → Langfuse Observability
|
|
|
|
# Manual
|
|
pip install langfuse
|
|
hermes plugins enable observability/langfuse
|
|
```
|
|
|
|
## Required credentials
|
|
|
|
Set these in `~/.hermes/.env` (or via `hermes tools`):
|
|
|
|
```bash
|
|
HERMES_LANGFUSE_PUBLIC_KEY=pk-lf-...
|
|
HERMES_LANGFUSE_SECRET_KEY=sk-lf-...
|
|
HERMES_LANGFUSE_BASE_URL=https://cloud.langfuse.com # or your self-hosted URL
|
|
```
|
|
|
|
Without the SDK or credentials the hooks no-op silently — the plugin fails
|
|
open.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
hermes plugins list # observability/langfuse should show "enabled"
|
|
hermes chat -q "hello" # then check Langfuse for a "Hermes turn" trace
|
|
```
|
|
|
|
## Optional tuning
|
|
|
|
```bash
|
|
HERMES_LANGFUSE_ENV=production # environment tag
|
|
HERMES_LANGFUSE_RELEASE=v1.0.0 # release tag
|
|
HERMES_LANGFUSE_SAMPLE_RATE=0.5 # sample 50% of traces
|
|
HERMES_LANGFUSE_MAX_CHARS=12000 # max chars per field (default: 12000)
|
|
HERMES_LANGFUSE_DEBUG=true # verbose plugin logging
|
|
```
|
|
|
|
## Disable
|
|
|
|
```bash
|
|
hermes plugins disable observability/langfuse
|
|
```
|