feat(telemetry): local-first telemetry & observability

Add a built-in telemetry system that records what the agent does — workflows,
model calls, tool calls, errors — to the local machine, powers `/insights`, and
can export to an operator-chosen destination. Default-on locally; nothing leaves
the machine unless the user exports it or opts into the aggregate plane.

Three planes with a hard wall between them:
  - local: full-fidelity observability (real model/provider/tool names), on by
    default, never leaves the machine.
  - aggregate: opt-in metadata, default off. No uploader ships — consent is
    recorded via telemetry.consent_state, and `preview` shows what would be
    produced, computed locally.
  - trajectories: full message content, opt-in, exported only to the operator's
    own destination.

Mechanism:
  - Bundled `telemetry` plugin registers observational lifecycle hooks
    (on_session_start / post_api_request / post_tool_call / on_session_finalize).
    No core call sites are edited; hooks already carry the data.
  - Fire-and-forget emitter: emit() returns in microseconds, never blocks or
    raises into a model/tool call. A daemon thread writes events to an
    append-only JSONL log and the tel_* tables in state.db (its own sqlite
    connection, separate from SessionDB).
  - tel_runs / tel_model_calls / tel_tool_calls live in the declarative
    SCHEMA_SQL and are reconciled automatically; SCHEMA_VERSION 16 -> 17.
  - metrics derives rollups for /usage and /insights; rollup builds per-run
    summaries for `hermes telemetry preview`.

Consent is config, not a parallel command surface. The config file is the root
of trust: set telemetry.consent_state with `hermes config set`, or pin any
telemetry.* key (including allow_aggregate) via managed scope, which overrides
the user's value per key. `hermes telemetry` exposes only what config cannot:
status (report), preview (query), and export.

Export:
  - exporter_bulk writes telemetry (and, when the trajectories plane is enabled,
    session content) to ndjson/json.
  - otlp_exporter streams spans to a configured OpenTelemetry Collector over
    OTLP/HTTP. The SDK is an optional extra (hermes-agent[otlp]), lazily
    installed via tools.lazy_deps on first use.
  - Secrets are always redacted on every export path
    (redact_sensitive_text(force=True)); content export is gated by the
    trajectories plane, and PII scrubbing follows telemetry.content_redaction.
    OTLP auth headers reference environment variable names, never inline values.

No outbound emission to Nous. The aggregate uploader is intentionally not built.
This commit is contained in:
emozilla 2026-06-24 02:14:02 -04:00
parent 64131bf975
commit ccfa079252
37 changed files with 3920 additions and 4 deletions

View file

@ -116,6 +116,15 @@ LAZY_DEPS: dict[str, tuple[str, ...]] = {
# ─── Image generation backends ─────────────────────────────────────────
"image.fal": ("fal-client==0.13.1",),
# ─── Observability ─────────────────────────────────────────────────────
# OTLP telemetry export. Lazily installed on first use of
# `hermes telemetry export --otlp`. Tracks the `otlp` extra in
# pyproject.toml — bump both together.
"export.otlp": (
"opentelemetry-sdk==1.30.0",
"opentelemetry-exporter-otlp-proto-http==1.30.0",
),
# ─── Memory providers ──────────────────────────────────────────────────
"memory.honcho": ("honcho-ai==2.0.1",),
"memory.hindsight": ("hindsight-client==0.6.1",),