mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
feat(telemetry): local-first telemetry & observability
Add a built-in telemetry system that records what the agent does — workflows,
model calls, tool calls, errors — to the local machine, powers `/insights`, and
can export to an operator-chosen destination. Default-on locally; nothing leaves
the machine unless the user exports it or opts into the aggregate plane.
Three planes with a hard wall between them:
- local: full-fidelity observability (real model/provider/tool names), on by
default, never leaves the machine.
- aggregate: opt-in metadata, default off. No uploader ships — consent is
recorded via telemetry.consent_state, and `preview` shows what would be
produced, computed locally.
- trajectories: full message content, opt-in, exported only to the operator's
own destination.
Mechanism:
- Bundled `telemetry` plugin registers observational lifecycle hooks
(on_session_start / post_api_request / post_tool_call / on_session_finalize).
No core call sites are edited; hooks already carry the data.
- Fire-and-forget emitter: emit() returns in microseconds, never blocks or
raises into a model/tool call. A daemon thread writes events to an
append-only JSONL log and the tel_* tables in state.db (its own sqlite
connection, separate from SessionDB).
- tel_runs / tel_model_calls / tel_tool_calls live in the declarative
SCHEMA_SQL and are reconciled automatically; SCHEMA_VERSION 16 -> 17.
- metrics derives rollups for /usage and /insights; rollup builds per-run
summaries for `hermes telemetry preview`.
Consent is config, not a parallel command surface. The config file is the root
of trust: set telemetry.consent_state with `hermes config set`, or pin any
telemetry.* key (including allow_aggregate) via managed scope, which overrides
the user's value per key. `hermes telemetry` exposes only what config cannot:
status (report), preview (query), and export.
Export:
- exporter_bulk writes telemetry (and, when the trajectories plane is enabled,
session content) to ndjson/json.
- otlp_exporter streams spans to a configured OpenTelemetry Collector over
OTLP/HTTP. The SDK is an optional extra (hermes-agent[otlp]), lazily
installed via tools.lazy_deps on first use.
- Secrets are always redacted on every export path
(redact_sensitive_text(force=True)); content export is gated by the
trajectories plane, and PII scrubbing follows telemetry.content_redaction.
OTLP auth headers reference environment variable names, never inline values.
No outbound emission to Nous. The aggregate uploader is intentionally not built.
This commit is contained in:
parent
64131bf975
commit
ccfa079252
37 changed files with 3920 additions and 4 deletions
|
|
@ -204,6 +204,23 @@ def _env_enabled(name: str) -> bool:
|
|||
return env_var_enabled(name)
|
||||
|
||||
|
||||
def _telemetry_local_enabled() -> bool:
|
||||
"""True when the local telemetry plane is enabled (telemetry.local, default true).
|
||||
|
||||
Gates auto-loading of the bundled ``telemetry`` plugin. Reads config defensively
|
||||
so a malformed/missing telemetry section defaults to on (the design default).
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.config import load_config
|
||||
config = load_config()
|
||||
tel = cfg_get(config, "telemetry", default={})
|
||||
if not isinstance(tel, dict):
|
||||
return True
|
||||
return bool(tel.get("local", True))
|
||||
except Exception:
|
||||
return True
|
||||
|
||||
|
||||
def _get_disabled_plugins() -> set:
|
||||
"""Read the disabled plugins list from config.yaml.
|
||||
|
||||
|
|
@ -1326,6 +1343,24 @@ class PluginManager:
|
|||
self._load_plugin(manifest)
|
||||
continue
|
||||
|
||||
# The bundled telemetry plugin auto-loads when the local plane is
|
||||
# enabled (telemetry.local, default true). It's observational
|
||||
# (lifecycle hooks -> local event log), writes nothing to the
|
||||
# network, and powers /usage and /insights, so it should be on out
|
||||
# of the box without an opt-in. A user who sets telemetry.local:
|
||||
# false opts out and it is skipped like any disabled plugin.
|
||||
if (
|
||||
manifest.source == "bundled"
|
||||
and (manifest.key or manifest.name) == "telemetry"
|
||||
):
|
||||
if _telemetry_local_enabled():
|
||||
self._load_plugin(manifest)
|
||||
else:
|
||||
loaded = LoadedPlugin(manifest=manifest, enabled=False)
|
||||
loaded.error = "disabled (telemetry.local is false)"
|
||||
self._plugins[lookup_key] = loaded
|
||||
continue
|
||||
|
||||
# Everything else (standalone, user-installed backends,
|
||||
# entry-point plugins) is opt-in via plugins.enabled.
|
||||
# Accept both the path-derived key and the legacy bare name
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue