docs: document Hermes Relay and the desktop feature wave

- New page user-guide/messaging/relay.md: what Relay is, enrollment
  (hermes gateway enroll), capability handshake (media, native
  approval/clarify, threads, typing), config keys, troubleshooting —
  derived from gateway/relay/ + gateway_enroll.py (PRs #48147 #48242
  #71300 #71363 #71404 #71624 #69721).
- desktop.md: artifacts viewer, timeline rail, find-in-page,
  multi-window, git review/worktrees, multi-terminal + persistence,
  theme import, rebindable shortcuts, quick entry, context-usage
  popover, keep-awake, command palette, Hermes Cloud mode, memory
  graph — all verified against apps/desktop/ source.
This commit is contained in:
Teknium 2026-07-29 00:55:15 -07:00
parent 355b376225
commit 068d7812a4
2 changed files with 238 additions and 2 deletions

View file

@ -45,12 +45,16 @@ The center of the app. You get:
- **Drag-and-drop files** anywhere in the chat area to attach them to your next message.
- **A right-hand preview rail** — render web pages, files, and tool outputs side by side while you keep chatting.
- **Composer history and queue editing** — press the up/down arrow keys in an empty composer to recall and reuse previous prompts, and edit messages you've queued up before they're sent. Pressing Stop (or Esc) while turns are queued pauses the queue and expands it above the composer; resume it from there, or send, edit, and delete individual entries.
- **A conversation timeline rail** — long chats get a slim rail of markers along the edge of the transcript, one per prompt. Hover it to pop open the list of prompts, click one to jump straight to that point in the conversation. (It appears once the chat has a handful of turns.)
- **Find in page** — press **Cmd/Ctrl+F** to open a find bar that searches the rendered chat transcript. Enter / Shift+Enter (or Cmd/Ctrl+G / Cmd/Ctrl+Shift+G while the bar is open) step through matches; Esc closes it.
#### Status bar
The bar along the bottom of the chat shows live session state and exposes quick controls without opening Settings:
- **Per-session YOLO toggle** — flip YOLO on or off for just this session (matching the TUI). YOLO bypasses the dangerous-command approval prompts, so know what you're turning off — see [Security → YOLO Mode](./security.md#yolo-mode).
- **Context-usage meter** — a live "% full" meter of the session's context window. Click it to open the **Context Usage** popover with a token breakdown by category (system prompt, tool definitions, skills, memory, rules, MCP, subagent definitions, and the conversation itself) so you can see exactly what's eating the window before compression kicks in.
- **Customizable items** — right-click the status bar (**Show in status bar**) to choose what appears: the context meter, workspace, model, approvals, turn/session timers, terminal, Command Center, backend version, and more — or hide the bar entirely (**Cmd/Ctrl+Shift+S** toggles it).
Chatting against a Hermes instance on another machine instead of the bundled local backend? See [Connecting to a remote backend](#connecting-to-a-remote-backend) below — and for the full picture of how the remote-hosted dashboard connection works (the auth gate, the `/api/ws` chat socket, and WebSocket close-code triage), see [Web Dashboard → Connecting Hermes Desktop to a remote backend](./features/web-dashboard.md#connecting-hermes-desktop-to-a-remote-backend).
@ -84,6 +88,41 @@ The model picker lives in the **composer**, just left of the microphone. Click i
Explore and preview the working directory without leaving the app — useful for following along as the agent reads, writes, and edits files. Set the initial project directory with `hermes desktop --cwd <path>` (or the `HERMES_DESKTOP_CWD` environment variable).
### Artifacts
The **Artifacts** view collects what your sessions generate — **images, files, and links** — into one searchable, browsable gallery. Open it from the sidebar, the command palette (**Artifacts — Browse generated outputs**), or a `nav.artifacts` shortcut you bind yourself. It indexes recent session outputs automatically; every artifact shows which session produced it with a jump back to that chat, and images and files open in a preview with download / open-in-browser / copy actions.
### Windows, tabs & panes
The app is built for working on several things at once:
- **Tabs****Cmd/Ctrl+T** opens a new session tab; **Ctrl+Tab** / **Ctrl+Shift+Tab** cycle sessions, and **Ctrl+1…9** jump to a recent session by position. **Cmd/Ctrl+W** closes the focused tab and **Cmd/Ctrl+Shift+T** reopens the last closed one.
- **Multiple windows****Cmd/Ctrl+Shift+N** opens a new window, and any session can be popped out via its context menu (**New window**) or from the command palette. A popped-out window renders that single chat without the global sidebar — handy for parking a long-running session on another monitor. Live agent output streams into every window showing the session.
- **Panes****Cmd/Ctrl+B** toggles the left sidebar, **Cmd/Ctrl+J** the right one, and **Cmd/Ctrl+\\** swaps which side the sidebars sit on.
### Terminal
A real terminal lives in the right sidebar, next to the file browser:
- **Ctrl+`** shows the terminal (opening one if none exist); **Ctrl+Shift+`** spawns an additional one. Multiple terminals stack in a tab rail — **Ctrl+Shift+↓/↑** walk between them, **Ctrl+Shift+W** closes the active one.
- **Shells persist while hidden.** Closing or hiding the panel doesn't kill your shell — every open terminal stays mounted with its scrollback and running processes intact until you explicitly close it.
- **Add to chat** — select terminal output and send it into the composer as context for your next message.
### Git review & worktrees
For sessions running inside a Git repository, the app has a built-in source-control surface:
- **Review pane****Cmd/Ctrl+G** toggles the working-tree review pane: branch and ahead/behind status, changed files (list or tree view), and diffs scoped to **Uncommitted**, **Branch**, or **Last turn** (just what the agent changed in its most recent turn). Stage/unstage files, revert changes, write a commit message (or **Generate commit message**), then **Commit** or **Commit & Push** — and **Create PR** via the GitHub CLI (`gh`), or hand the whole thing to the agent with **Ask Hermes to open PR**. You can also create and switch branches from here.
- **Worktrees****Cmd/Ctrl+Shift+B** (or **New worktree** on a project in the sidebar) creates a Git worktree on a new branch so an agent can work on a parallel copy of the repo without touching your checkout. Worktrees show up as their own lanes under the project; removing one offers to delete the worktree directory (the branch stays) or just hide the lane and leave it on disk, with a force option when it has uncommitted changes.
### Memory Graph
The **Memory Graph** (command palette → *Memory Graph*, or the status-bar item) is an interactive map of what Hermes has learned for you — skills and memories laid out as a zoomable node graph with a timeline, filterable by **All / Used / Learned**. A share control exports the map layout as a compact code you can paste to someone else (layout only — none of your memory or skill text is included) and imports codes the same way.
### Quick Entry
Quick Entry is a small always-available composer summoned by a **global hotkey from anywhere on your system** — fire off a prompt without switching to (or even opening) the main window. Enable it in **Settings → Advanced → Quick Entry**; the default shortcut is **Ctrl/Cmd+Shift+Space** and you can set your own (it needs at least one modifier). If another app already owns the chord, the settings row tells you so you can pick a different one.
### Voice
Talk to Hermes and hear it back, the same [voice mode](./features/voice-mode.md) available elsewhere. On macOS the OS will prompt once for microphone access.
@ -97,6 +136,8 @@ Manage providers, models, tools, and credentials from a real UI instead of editi
- **xAI Grok OAuth** — Grok is a first-class OAuth provider in the launcher; sign in through the browser flow like the other OAuth providers.
- **Tool-backend installs from the GUI** — run a tool backend's post-setup install steps directly from the app instead of dropping to a terminal.
- **Auxiliary-model warning** — if you switch the main model to a new provider while auxiliary tasks (titling, summarization, and similar helpers) are still pinned to another provider, the app warns you so you don't unknowingly split work across two providers.
- **VS Code Marketplace themes** — beyond the built-in theme presets, the appearance settings include a live VS Code Marketplace search: pick any color theme and the app downloads, converts, and installs it as a desktop theme. The same importer is available from the command palette (*Install theme*), and imported themes can be removed again from the appearance settings.
- **Keep computer awake****Settings → Advanced → Keep computer awake** stops the machine from sleeping so long or overnight agent runs keep going (the display can still dim). This is a per-computer setting.
First-run onboarding has been redesigned on a unified overlay design system, and you can pick **Choose provider later** to skip provider setup and get into the app first.
@ -105,6 +146,7 @@ First-run onboarding has been redesigned on a unified overlay design system, and
The app also surfaces the broader Hermes management surface so you don't have to drop to a terminal:
- **Skills** — browse, install, and manage [skills](./features/skills.md).
- **Memory graph (Star Map)** — type `/journey` (aliases `/learning`, `/memory-graph`) in chat to open an interactive constellation of learned skills and memories over time, with a playback scrubber. Nodes can be edited or deleted right from the panel (skills are archived, memories removed). See [Learning Journey](./features/memory.md#learning-journey-journey).
- **Cron** — view and manage [scheduled jobs](../reference/cli-commands.md#hermes-cron).
- **Profiles** — switch between [Hermes profiles](./profiles.md) (isolated config/skills/sessions).
- **Messaging** — set up gateway channels.
@ -112,8 +154,8 @@ The app also surfaces the broader Hermes management surface so you don't have to
### Keyboard & navigation
- **Command palette** — press **Cmd+K** (Ctrl+K on Windows/Linux) to jump to actions and navigate the app from the keyboard.
- **Rebindable shortcuts**a shortcuts panel in Settings lets you remap the app's keyboard shortcuts to your own keys.
- **Command palette** — press **Cmd+K** or **Cmd+P** (Ctrl+K / Ctrl+P on Windows/Linux) to jump to actions and navigate the app from the keyboard: open any page or settings section, jump to a session by title or id, switch model/theme/color mode, spawn a terminal, restart the gateway, update Hermes, and more.
- **Rebindable shortcuts****Settings → Keyboard Shortcuts** (or **Cmd/Ctrl+/**) opens the shortcuts panel where you can remap almost every binding — profile switching, session navigation, view toggles, and any shortcuts contributed by desktop plugins. Duplicate assignments are flagged as conflicts. A few defaults worth knowing: **Cmd/Ctrl+N** new session, **Cmd/Ctrl+.** Command Center, **Cmd/Ctrl+,** Settings, **Cmd/Ctrl+Shift+F** search sessions, **Cmd/Ctrl+19** switch profiles, **Shift+X** toggle light/dark.
- **Custom zoom shortcuts** — zoom the interface in half-step increments for finer control over text size.
- **UI language switcher** — change the app's interface language in-app, including Simplified Chinese (zh-Hans).
@ -168,6 +210,13 @@ The packaged app ships the Electron shell and a native React chat surface. On fi
By default the app starts and manages its own **local** backend. You can instead point it at a Hermes backend running on another machine — a VPS, a home server, or a Mini behind Tailscale.
**Settings → Gateway → Connection mode** offers the alternatives to the local gateway:
- **Remote gateway** — enter the URL of a `hermes serve` backend you run yourself and sign in. This is the mode the rest of this section walks through.
- **Hermes Cloud** — sign in once to Hermes Cloud and pick from the agents on your account; no URL to paste. The app discovers your agents (with an organization picker if your account spans several orgs), and connecting to one switches the session over automatically. The status bar shows the cloud connection while it's active.
Connection modes are configured **per profile** — a per-profile override can point one profile at a remote or cloud backend while others stay local (**Use default gateway** removes an override).
:::info The remote backend is a running `hermes serve` process
"Remote backend" means a **`hermes serve`** server running on the remote machine — that is the process the desktop app connects to. Nothing in this section works unless that backend is actually up and reachable. The desktop app does not start it for you; you (or a `systemd` service) keep `hermes serve` running on the remote host, and the app attaches to it. If you also use messaging channels (Telegram, Discord, etc.), the **gateway** is a *separate* long-running process you start independently — see the note after the setup steps.
:::

View file

@ -0,0 +1,187 @@
---
sidebar_position: 30
title: "Hermes Relay"
description: "Connect Hermes to messaging platforms through a relay connector that owns the platform credentials — enrollment, capabilities, config, and troubleshooting"
---
# Hermes Relay (Connector)
:::warning Experimental
Relay is **experimental**. The wire contract, auth scheme, and configuration
may change without a deprecation cycle while the system is being validated.
:::
Hermes Relay is not a chat platform itself — it is a **connector system** that
lets your gateway front one or more real messaging platforms (Discord,
Telegram, Slack, WhatsApp, …) **without holding any platform credentials**. A
separate service, the *connector*, owns the platform bot tokens and sockets.
Your gateway dials **out** to the connector over a single authenticated
WebSocket, receives a capability descriptor at handshake, and then exchanges
normalized message events (inbound) and actions (outbound) over that socket.
Key properties:
- **Outbound-only networking.** The gateway never opens an inbound port.
Inbound messages ride back down the same WebSocket the gateway dialed, so
relay works behind NAT and on hosts with no public IP.
- **No platform secrets on the gateway.** Bot tokens live on the connector.
Auth-gated platform media URLs are re-hosted connector-side, so platform
credentials never cross the wire.
- **Platform-agnostic.** The gateway learns what the fronted platform can do
(message length limits, markdown dialect, edit/thread/streaming support, and
the exact set of supported operations) from the handshake descriptor, not
from hardcoded platform logic.
The formal gateway ⇄ connector interface lives in the repository at
`docs/relay-connector-contract.md`.
## When to use Relay
Relay is for deployments where a hosted or shared connector service manages
the platform side — for example multi-tenant hosting where one shared bot
fronts many users' agents, or setups where you don't want bot tokens on the
gateway machine. If you run your own bots directly, use the native platform
adapters ([Telegram](/user-guide/messaging/telegram),
[Discord](/user-guide/messaging/discord), etc.) instead.
## Enrollment
A self-hosted gateway authenticates to the connector with a per-gateway
secret. `hermes gateway enroll` redeems a **single-use enrollment token**
(minted by the connector when your tenant's route is provisioned and delivered
with your gateway config) for that secret:
```bash
hermes gateway enroll \
--token <enrollment-token> \
--connector-url wss://connector.example.com/relay
```
What it does:
1. Resolves a fresh Nous Portal access token from your existing login
(`~/.hermes/auth.json`) — this proves which Nous org (tenant) you own. If
`gateway.idp.token_url` is configured, a generic OAuth2 client-credentials
token from your own IdP is used instead (the air-gapped / self-hosted-IdP
path, no Nous Portal involved).
2. POSTs the enrollment token and a gateway id to the connector's
`/relay/enroll` endpoint over TLS.
3. The connector verifies the token (signature, single-use, tenant match),
mints a per-gateway secret plus a per-tenant delivery key, and returns them
once.
4. Persists the credentials into `~/.hermes/.env`:
`GATEWAY_RELAY_ID`, `GATEWAY_RELAY_SECRET`, `GATEWAY_RELAY_DELIVERY_KEY`
(plus `GATEWAY_RELAY_URL` / `GATEWAY_RELAY_WAKE_URL` when supplied).
Restart the gateway afterwards to pick up the new environment.
Flags:
| Flag | Description |
|------|-------------|
| `--token` | The single-use enrollment token. Also settable via `GATEWAY_RELAY_ENROLL_TOKEN`. |
| `--connector-url` | Connector base or relay URL (`wss://…/relay` or `https://…`). Also settable via `GATEWAY_RELAY_URL` or `gateway.relay_url` in `config.yaml`. |
| `--gateway-id` | Stable id for this gateway instance (used for kill-switch granularity). Defaults to `gw-<hostname>`. |
| `--wake-url` | Optional reachable URL the connector pokes (payload-free GET) to wake this gateway when buffered work arrives while it is idle. Persisted as `GATEWAY_RELAY_WAKE_URL`. Without it the gateway still drains buffered messages whenever it next reconnects. |
:::note Managed installs
`hermes gateway enroll` refuses to run in managed/hosted installs — there the
hosting platform provisions the relay secret directly into the container
environment.
:::
## Configuration
Relay activates when a connector relay URL is configured — there is no
separate feature flag. Deployments that don't set it are unaffected.
| Setting | Where | Meaning |
|---------|-------|---------|
| `GATEWAY_RELAY_URL` | env (`~/.hermes/.env`) | Connector relay WebSocket URL. Presence enables the relay platform. |
| `gateway.relay_url` | `config.yaml` | Same as above, config-file form (env takes precedence). |
| `GATEWAY_RELAY_ID` | env | This gateway instance's id (written by `enroll`). |
| `GATEWAY_RELAY_SECRET` | env | Per-gateway secret authenticating the WebSocket upgrade (written by `enroll`). |
| `GATEWAY_RELAY_DELIVERY_KEY` | env | Per-tenant delivery key (written by `enroll`; retained for forward-compat). |
| `GATEWAY_RELAY_WAKE_URL` / `gateway.relay_wake_url` | env / `config.yaml` | Optional wake-poke target for idle/suspended gateways. |
| `GATEWAY_RELAY_PLATFORMS` | env | Comma-separated list of platforms this gateway fronts over one connection (e.g. `discord,telegram`). Usually stamped by the deployment/orchestrator. |
| `GATEWAY_RELAY_BOT_IDS` | env | JSON map of per-platform bot identities, e.g. `{"discord": {"botId": "…"}}`. Paired with `GATEWAY_RELAY_PLATFORMS`. |
| `gateway.idp.token_url` | `config.yaml` | When set, enrollment/provisioning authenticates via generic OAuth2 client-credentials against your own IdP instead of Nous Portal. |
## Supported capabilities
What actually works over a relay connection is negotiated at handshake: the
connector advertises a `supported_ops` list, and the gateway only uses an
operation the connector explicitly advertises (older connectors fall back to a
legacy `send`/`edit`/`typing`/`follow_up` set). Per-platform capability flags
(edit-based streaming, threads, draft streaming, markdown dialect, message
length limit) also come from the handshake descriptor. Subject to that
negotiation, the relay supports:
- **Text messages and streaming** — sends, replies, and progressive
edit-based streaming when the fronted platform supports message editing;
otherwise output degrades to one message per segment.
- **Media, both directions** — outbound images, voice, audio, video, and
documents are uploaded to the connector (or referenced by public URL) and
delivered through each platform's native upload lane, with captions.
Inbound attachments are localized to files for the agent; auth-gated
platform URLs are re-hosted connector-side so platform credentials never
reach the gateway. Media re-hosts are capped at 25 MB and expire (~1 hour).
- **Native interactive prompts** — exec approvals, confirmations, and clarify
questions render with **native platform controls** (Discord buttons,
Telegram inline keyboards, Slack Block Kit actions, WhatsApp button/list
messages) instead of numbered-text fallbacks. Button presses come back as
authenticated prompt responses from the actual clicking user, so the
gateway's authorization gates apply exactly as to a typed reply. Prompt
expiry is enforced gateway-side.
- **Reaction ack lifecycle** — the bot's processing-status reactions
(👀 while working, ✅/❌ on completion) work over the relay. Reactions are
best-effort: a failed reaction never fails a turn.
- **Thread lifecycle** — creating handoff threads and renaming threads
(including LLM-titled semantic renames) through platform-abstract
`thread_create` / `thread_rename` operations, with a no-clobber guard so a
human's manual rename wins. Availability depends on the platform (e.g.
Slack threads can't be renamed; WhatsApp has no threads).
- **Typing indicators** — the gateway egresses typing (and stop-typing) through
the connector while processing.
- **Chat metadata**`get_chat_info` lookups are proxied to the connector
when advertised.
- **Buffered delivery and wake** — when the gateway goes idle or disconnects,
the connector buffers inbound messages durably and replays them in order on
reconnect (ack-gated, no loss or duplication). If a wake URL is registered,
the connector pokes it when buffered work arrives for a sleeping gateway.
Multi-platform fronting is supported: one gateway can front several platforms
(e.g. Discord *and* Telegram) over a single relay connection, with each
outbound message tagged for the platform it targets.
## Troubleshooting
**Enrollment fails with 401** — the connector could not verify your identity
token. Re-login with `hermes auth add nous` (or `hermes setup`) and retry.
**Enrollment fails with 403** — the enrollment token is invalid, expired,
already used, or belongs to a different tenant. Enrollment tokens are
single-use; request a fresh one from whoever provisioned your tenant route.
**"Could not reach the connector"** — check the connector URL. You can paste
either the `wss://…/relay` dial URL or the `https://…` base URL; the CLI maps
between them automatically.
**`enroll` refuses to run** — you are in a managed/hosted install, where the
relay secret is provisioned by the hosting platform. Self-enrollment is only
for self-hosted gateways.
**Relay platform shows as disabled after it previously worked** — a WebSocket
close with code 4401 *after* a successful handshake means the gateway's secret
was revoked (e.g. the instance was deprovisioned). The gateway deliberately
stops reconnecting and reports relay as disabled rather than retrying. A 4401
*before* any successful handshake is treated as a transient
not-yet-provisioned race and retried normally.
**Nothing changed after enrolling** — the gateway reads `GATEWAY_RELAY_*` at
startup. Restart it (`hermes gateway restart`).
**A feature (buttons, media, threads…) silently degrades to plain text** — the
connector for your platform did not advertise that operation in its handshake
`supported_ops`. The gateway intentionally falls back to the text behavior
rather than sending an op the connector can't handle.