mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
docs(telemetry): align observability docs with the trimmed schema
Match the docs to the code after the dead-schema cut and span layer:
- List the actual tel_* tables (runs, spans, model_calls, tool_calls,
error_events) instead of a vague "indexed tel_* tables".
- Add a "Traces and spans" section: a run = one session, each call is a child
span under the run root in tel_spans keyed by span_id, reconstructable as a
connected run -> calls tree. Note subagent cross-run lineage isn't recorded.
- Fix stale "tool failure rates by category" -> "by tool" (categories were
removed; insights groups by raw tool name).
- OTLP: state plainly that events export as per-event spans and the tel_spans
parent/timing linkage isn't reconstructed into connected SpanContexts yet,
matching the exporter's own docstring.
- README: "telemetry plane" -> "telemetry system" (stale rename miss); mention
spans.
Config reference verified to match DEFAULT_CONFIG exactly (9 keys).
This commit is contained in:
parent
edac3a7bae
commit
fdb5ae012a
2 changed files with 25 additions and 8 deletions
|
|
@ -307,8 +307,8 @@ nested agent work or security lifecycle events.
|
|||
|
||||
## Existing Consumers
|
||||
|
||||
The bundled **`telemetry`** plugin is the built-in local-first telemetry plane: it
|
||||
records runs, model calls, and tool calls to a local event log + `state.db`, powers
|
||||
The bundled **`telemetry`** plugin is the built-in local-first telemetry system: it
|
||||
records runs, spans, model calls, and tool calls to a local event log + `state.db`, powers
|
||||
`hermes insights`, and supports export to your own file or OpenTelemetry Collector. It
|
||||
is on by default and never sends data to Nous unless you opt in. See
|
||||
[`telemetry.md`](./telemetry.md) for the full feature, the `hermes telemetry` commands,
|
||||
|
|
|
|||
|
|
@ -36,12 +36,25 @@ Local telemetry is implemented as a bundled `telemetry` plugin that listens to H
|
|||
lifecycle hooks (model calls, tool calls, session start/finalize) and writes events to:
|
||||
|
||||
- an append-only JSONL log at `~/.hermes/telemetry/events.jsonl` (the source of truth)
|
||||
- indexed `tel_*` tables in `state.db` (for fast queries and rollups)
|
||||
- indexed `tel_*` tables in `state.db` (a rebuildable index for fast queries):
|
||||
`tel_runs`, `tel_spans`, `tel_model_calls`, `tel_tool_calls`, `tel_error_events`
|
||||
|
||||
Writes are fire-and-forget on a background thread: telemetry can never block, slow, or
|
||||
fail a model call or tool call. If local telemetry is disabled (`telemetry.local: false`)
|
||||
the plugin does not load at all.
|
||||
|
||||
### Traces and spans
|
||||
|
||||
A **run** is one session (from `on_session_start` to `on_session_finalize`). Each run gets
|
||||
a root span in `tel_spans`, and every model and tool call within it is recorded as a child
|
||||
span (timing + `parent_span_id` = the run's root) keyed by the same `span_id` as its
|
||||
detail row in `tel_model_calls` / `tel_tool_calls`. So a run reconstructs as a connected
|
||||
`run -> calls` tree, ordered by `start_ns` and joinable to the per-call detail — the shape
|
||||
a trace viewer or any OpenTelemetry backend can render.
|
||||
|
||||
Call spans are timed from the measured latency/duration the hooks report; cross-run
|
||||
subagent lineage (linking a delegated child run to its parent) is not yet recorded.
|
||||
|
||||
### Seeing your local data
|
||||
|
||||
```bash
|
||||
|
|
@ -50,7 +63,7 @@ hermes telemetry status # settings, consent, export posture, local data volum
|
|||
```
|
||||
|
||||
The `insights` Observability section shows workflow counts and success rate, duration
|
||||
p50/p95, tool failure rates by category, provider/model mix, and cache hit rate —
|
||||
p50/p95, tool failure rates by tool, provider/model mix, and cache hit rate —
|
||||
all computed locally with exact values.
|
||||
|
||||
## `hermes telemetry` commands
|
||||
|
|
@ -146,10 +159,14 @@ Set the referenced environment variable, then run the export:
|
|||
hermes telemetry export --otlp # drain current telemetry to your collector
|
||||
```
|
||||
|
||||
The token value lives only in the environment variable named by `headers_env`. Span
|
||||
attributes are structural by default. The config holds the *name* of an environment
|
||||
variable rather than the secret itself; the value is read at export time and is never
|
||||
written to config or logged.
|
||||
The token value lives only in the environment variable named by `headers_env`. The
|
||||
config holds the *name* of an environment variable rather than the secret itself; the
|
||||
value is read at export time and is never written to config or logged.
|
||||
|
||||
Each telemetry event is exported as an OTel span carrying its recorded attributes
|
||||
(provider, model, tokens, duration, etc.). The `tel_spans` timing/parent linkage is not
|
||||
yet reconstructed into connected OTel `SpanContext`s, so spans currently arrive as
|
||||
independent records rather than a connected trace tree; that projection is planned.
|
||||
|
||||
## Redaction
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue