mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
feat(telemetry): local-first telemetry & observability
Add a built-in telemetry system that records what the agent does — workflows,
model calls, tool calls, errors — to the local machine, powers `/insights`, and
can export to an operator-chosen destination. Default-on locally; nothing leaves
the machine unless the user exports it or opts into the aggregate plane.
Three planes with a hard wall between them:
- local: full-fidelity observability (real model/provider/tool names), on by
default, never leaves the machine.
- aggregate: opt-in metadata, default off. No uploader ships — consent is
recorded via telemetry.consent_state, and `preview` shows what would be
produced, computed locally.
- trajectories: full message content, opt-in, exported only to the operator's
own destination.
Mechanism:
- Bundled `telemetry` plugin registers observational lifecycle hooks
(on_session_start / post_api_request / post_tool_call / on_session_finalize).
No core call sites are edited; hooks already carry the data.
- Fire-and-forget emitter: emit() returns in microseconds, never blocks or
raises into a model/tool call. A daemon thread writes events to an
append-only JSONL log and the tel_* tables in state.db (its own sqlite
connection, separate from SessionDB).
- tel_runs / tel_model_calls / tel_tool_calls live in the declarative
SCHEMA_SQL and are reconciled automatically; SCHEMA_VERSION 16 -> 17.
- metrics derives rollups for /usage and /insights; rollup builds per-run
summaries for `hermes telemetry preview`.
Consent is config, not a parallel command surface. The config file is the root
of trust: set telemetry.consent_state with `hermes config set`, or pin any
telemetry.* key (including allow_aggregate) via managed scope, which overrides
the user's value per key. `hermes telemetry` exposes only what config cannot:
status (report), preview (query), and export.
Export:
- exporter_bulk writes telemetry (and, when the trajectories plane is enabled,
session content) to ndjson/json.
- otlp_exporter streams spans to a configured OpenTelemetry Collector over
OTLP/HTTP. The SDK is an optional extra (hermes-agent[otlp]), lazily
installed via tools.lazy_deps on first use.
- Secrets are always redacted on every export path
(redact_sensitive_text(force=True)); content export is gated by the
trajectories plane, and PII scrubbing follows telemetry.content_redaction.
OTLP auth headers reference environment variable names, never inline values.
No outbound emission to Nous. The aggregate uploader is intentionally not built.
This commit is contained in:
parent
64131bf975
commit
ccfa079252
37 changed files with 3920 additions and 4 deletions
|
|
@ -1159,6 +1159,49 @@ display:
|
|||
# # Routing/delivery still uses the original values internally.
|
||||
# redact_pii: false
|
||||
|
||||
# =============================================================================
|
||||
# Telemetry & Observability
|
||||
# =============================================================================
|
||||
# Three planes with a hard wall between them:
|
||||
# local — full-fidelity observability you own. Default ON.
|
||||
# aggregate — opt-in metadata to Nous (no uploader ships yet). Default OFF.
|
||||
# trajectories — content trajectories for training. Separate consent (later).
|
||||
#
|
||||
# The local plane records real values (actual models, providers, tool names) —
|
||||
# your own data, on your machine. The aggregate plane is opt-in and has no
|
||||
# uploader today; if one ships it would summarize at that egress boundary.
|
||||
#
|
||||
# Enterprise locking: any telemetry.* key can be pinned by an administrator via
|
||||
# the managed-scope layer (/etc/hermes/config.yaml), which wins over the user's
|
||||
# value. To hard-forbid egress on a locked-down deployment, pin
|
||||
# `telemetry.allow_aggregate: false` there.
|
||||
# telemetry:
|
||||
# # Local plane: event log + SQLite index in state.db. Never leaves your
|
||||
# # machine unless you export it or opt into the aggregate plane.
|
||||
# local: true
|
||||
# # Hard gate for the aggregate plane. When false, the aggregate plane is off
|
||||
# # regardless of consent_state. Pin false via managed scope to forbid egress.
|
||||
# allow_aggregate: true
|
||||
# # Aggregate-plane consent (the opt-in). No uploader ships yet.
|
||||
# # unknown (no choice — never uploads) | local (declined) | aggregate (opted in)
|
||||
# consent_state: unknown
|
||||
# # Stable install id (aggregate plane only). Empty = mint on first use;
|
||||
# # clear it to rotate.
|
||||
# install_id: ""
|
||||
# # Local event-log retention before rotation (days).
|
||||
# retention_days: 90
|
||||
# # Keep secret redaction on even at full local capture.
|
||||
# redact_secrets: true
|
||||
# # Content redaction for exports / support bundles: none | pii.
|
||||
# content_redaction: none
|
||||
# # Exporters. The OTLP exporter sends spans to a configured Collector endpoint;
|
||||
# # header values reference environment variable names, not inline secrets.
|
||||
# export:
|
||||
# otlp:
|
||||
# enabled: false
|
||||
# endpoint: null
|
||||
# headers_env: {} # e.g. {Authorization: MY_OTLP_TOKEN_ENVVAR}
|
||||
|
||||
# =============================================================================
|
||||
# Shell-script hooks
|
||||
# =============================================================================
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue