hermes-agent/website/docs/user-guide/tui.md
brooklyn! a7cd254c29
feat(tui): mouse_tracking DEC mode presets (salvage of #26681) (#30084)
* feat(tui): make display.mouse_tracking pick which DEC modes to enable

Previously the boolean flag was all-or-nothing across modes 1000+1002+1003+1006.
Inside tmux, mode 1003 (any-motion) makes every mouse cross of the prompt row
fire a clipboard probe that surfaces as "No image in clipboard" — sometimes
dozens in a row. Disabling tracking entirely killed scroll-wheel scrolling too,
since tmux's own scrollback is preempted by the alt-screen TUI.

`display.mouse_tracking` (and `/mouse <preset>`) now accepts `off | wheel |
buttons | all` in addition to the legacy booleans. `wheel` is 1000+1006:
scroll wheel + click only, no drag, no hover — the tmux-friendly subset.
`buttons` adds 1002 for drag-to-select. `all` (= legacy `true`) keeps the
hover-driven UI (scrollbar paginate-on-hover, link mouseenter, etc.).

* fix(tui): repaint + sync mouse mode when display.mouse_tracking changes

Two interacting bugs left the TUI blank when `display.mouse_tracking`
switched at runtime (config edit, /mouse <preset>):

1. AlternateScreen's effect re-runs on every `mouseTracking` change,
   tearing down and re-entering the alt screen. After re-entry, ink's
   frame buffers are reset by `resetFramesForAltScreen()` but nothing
   schedules the follow-up render — the alt screen sits blank until
   some other state change happens to trigger one. Add a
   `scheduleRender()` in `setAltScreenActive`'s active=true branch so
   the freshly-entered alt screen gets a full repaint immediately.

2. `setAltScreenActive` early-returns when `active` hasn't changed,
   which silently drops a `mouseTracking` change if the cleanup→setup
   pair somehow leaves `altScreenActive` already true. Call
   `setAltScreenMouseTracking` explicitly from the AlternateScreen
   effect so the in-memory mode and terminal DECSET sequence stay in
   sync regardless of how `setAltScreenActive` resolved (the call is a
   no-op when the mode is unchanged).

* fix(tui): address copilot review #4341269705

- tui_gateway/server.py: drop the never-referenced _MOUSE_TRACKING_MODES
  frozenset (comment #3284802434). _MOUSE_TRACKING_ALIASES already
  centralizes the canonical preset set via its values; the separate
  constant added no behavior.
- tests/test_tui_gateway_server.py: update the existing
  test_config_mouse_uses_documented_key_with_legacy_fallback to assert
  the new preset strings ('all'/'off' instead of 'on'/'off',
  display.mouse_tracking persisted as 'all' instead of True) and add
  test_config_mouse_accepts_preset_strings_and_aliases covering /mouse
  set with wheel/click/unknown (comment #3284802453). The on/off legacy
  config.set return shape was an implementation detail of the boolean
  flag, not a stable API — the slash command, gateway help text, and
  docs all advertise the preset values now.
- ui-tui/packages/hermes-ink/src/ink/ink.tsx: schedule a render at the
  end of reenterAltScreen() (comment #3284802461). Mirrors the same fix
  in setAltScreenActive() from ece0a2f4c — without it, SIGCONT/resize
  self-heal/stdin-gap re-entry leaves the alt screen blank because
  every caller returns early after invoking us.

* fix(tui): address copilot review #4341308478 round 2

- ui-tui/src/config/env.ts (comment #3284837577): the precedence
  comment was misleading. Actual behavior on origin/main is
  HERMES_TUI_MOUSE_TRACKING (explicit override) > Termux default >
  HERMES_TUI_DISABLE_MOUSE legacy kill-switch. This is preserved from
  main; the only change here was the wrong comment that claimed
  DISABLE_MOUSE kept kill-switch semantics. Rewrote the comment block
  to document the actual precedence ladder.
- tui_gateway/server.py /mouse set (comment #3284837607): replaced
  'str(value or "").strip().lower()' with the explicit None idiom
  already used for /indicator, so programmatic callers can pass 0 /
  False and have them route through _MOUSE_TRACKING_ALIASES → 'off'
  instead of collapsing to '' and triggering the toggle path.
- ui-tui/packages/hermes-ink/src/ink/components/AlternateScreen.tsx
  (comment #3284837620): always prepend DISABLE_MOUSE_TRACKING before
  enableMouseTrackingFor(...) on mount. Otherwise selecting
  'wheel'/'buttons' from a state where DEC 1003 was already asserted
  (crash, another app, debugger) would silently leave hover on. Also
  unconditionally DISABLE on unmount so a crash mid-mount can't leak
  DEC modes back to the host shell.

* chore(release): map nat@nthrow.io to @nthrow for #26681 salvage

* fix(tui): drop redundant setAltScreenMouseTracking in AlternateScreen

Copilot review #4341356637 (comment #3284880417). The explicit
setAltScreenMouseTracking(mouseTracking) after setAltScreenActive(true,
mouseTracking) was defensive paranoia added in the previous fix commit
that's not actually reachable in practice:

- React's cleanup always runs before the next setup, so on any prop
  change (mouseTracking or writeRaw) the cleanup sets active=false
  first. Setup then sees active was false and applies the new mode
  via setAltScreenActive without early-returning.
- On the impossible 'active stayed true' path, the writeRaw above has
  already sent DISABLE_MOUSE_TRACKING + enableMouseTrackingFor(newMode)
  to the terminal, so the in-memory mode would lag but the visible
  state is already correct.

Removing the redundant call means a single DEC sequence per mount.
If the 'active stayed true' path ever manifests in practice, the
right fix is in setAltScreenActive (track mode regardless of the
active early-return), not here.

* fix(tui): always DISABLE before enableMouseTrackingFor in ink.tsx

Copilot review #4341379994 (comments #3284900825, #3284900840,
#3284900852). Three remaining call sites in ink.tsx still re-enabled
mouse tracking without first sending DISABLE_MOUSE_TRACKING:

- handleResize alt-screen recovery (line ~577)
- reassertTerminalModes stdin-gap re-assertion (line ~1351)
- reenterAltScreen SIGCONT/resize/stdin-gap self-heal (line ~1408)

For 'wheel'/'buttons' presets, omitting DISABLE leaves any externally-
asserted DEC 1003 (other apps, prior crash, tmux state) still active
and the hover-free preset silently has hover on. DISABLE_MOUSE_TRACKING
is idempotent and safe to send unconditionally — it resets all four
modes. Matches the pattern already in setAltScreenMouseTracking and
the AlternateScreen mount path.

* fix(tui): always DISABLE before enableMouseTrackingFor in exitAlternateScreen

Copilot review #4341452823 (comment #3284959762). exitAlternateScreen()
was the last call site in ink.tsx still re-enabling mouse tracking
without DISABLE first. Editors (vim/nvim/less) and tmux can leave
DEC 1003 hover asserted across the handoff back; without DISABLE,
'wheel'/'buttons' presets silently kept hover on after the editor
quit. Now all five enableMouseTrackingFor() call sites in ink.tsx
prepend DISABLE_MOUSE_TRACKING — handleResize, reassertTerminalModes,
reenterAltScreen, setAltScreenMouseTracking, exitAlternateScreen.

* fix(tui): add defensive default to enableMouseTrackingFor switch

Copilot review #4341485231 (comment #3284979323). TS exhaustive switch
returns string per the type system, but a JS caller / corrupted config
/ hot-reload-in-dev could reach the function with an unknown value at
runtime. Without a default, that path returns undefined which then
concatenates as the literal string 'undefined' into the terminal byte
stream — visibly garbling output. Treat unknown as 'off' (no DEC
sequences) so the worst case is silent input loss rather than a
wrecked screen.

---------

Co-authored-by: Nat Thrower <nat@nthrow.io>
2026-05-21 20:25:52 -05:00

15 KiB

sidebar_position title description
2 TUI Launch the modern terminal UI for Hermes — mouse-friendly, rich overlays, and non-blocking input.

TUI

The TUI is the modern front-end for Hermes — a terminal UI backed by the same Python runtime as the Classic CLI. Same agent, same sessions, same slash commands; a cleaner, more responsive surface for interacting with them.

It's the recommended way to run Hermes interactively.

Launch

# Launch the TUI
hermes --tui

# Resume the latest TUI session (falls back to the latest classic session)
hermes --tui -c
hermes --tui --continue

# Resume a specific session by ID or title
hermes --tui -r 20260409_000000_aa11bb
hermes --tui --resume "my t0p session"

# Run source directly — skips the prebuild step (for TUI contributors)
hermes --tui --dev

You can also enable it via env var:

export HERMES_TUI=1
hermes          # now uses the TUI
hermes chat     # same

The classic CLI remains available as the default. Anything documented in CLI Interface — slash commands, quick commands, skill preloading, personalities, multi-line input, interrupts — works in the TUI identically.

Why the TUI

  • Instant first frame — the banner paints before the app finishes loading, so the terminal never feels frozen while Hermes is starting.
  • Non-blocking input — type and queue messages before the session is ready. Your first prompt sends the moment the agent comes online.
  • Rich overlays — model picker, session picker, approval and clarification prompts all render as modal panels rather than inline flows.
  • Live session panel — tools and skills fill in progressively as they initialize.
  • Mouse-friendly selection — drag to highlight with a uniform background instead of SGR inverse. Copy with your terminal's normal copy gesture.
  • Alternate-screen rendering — differential updates mean no flicker when streaming, no scrollback clutter after you quit.
  • Composer affordances — inline paste-collapse for long snippets, Cmd+V / Ctrl+V text paste with clipboard-image fallback, bracketed-paste safety, and image/file-path attachment normalization.

Same skins and personalities apply. Switch mid-session with /skin ares, /personality pirate, and the UI repaints live. See Skins & Themes for the full list of customizable keys and which ones apply to classic vs TUI — the TUI honors the banner palette, UI colors, prompt glyph/color, session display, completion menu, selection bg, tool_prefix, and help_header.

Collapsible banner sections

The TUI startup banner groups runtime info into four collapsible sections, each rendered with a / chevron next to the section title:

Section Default state
Tools Open
Skills Collapsed
System Prompt Collapsed
MCP Servers Collapsed

Click anywhere on a section header (or its chevron) to toggle it. The Tools list opens by default because it's the most-checked section at session start; Skills, System Prompt, and MCP Servers collapse by default so the banner stays compact even when you've installed dozens of skills or wired up many MCP servers. State is local to the banner instance, so the next launch resets to the defaults.

Requirements

  • Node.js ≥ 20 — the TUI runs as a subprocess launched from the Python CLI. hermes doctor verifies this.
  • TTY — like the classic CLI, piping stdin or running in non-interactive environments falls back to single-query mode.

On first launch Hermes installs the TUI's Node dependencies into ui-tui/node_modules (one-time, a few seconds). Subsequent launches are fast. If you pull a new Hermes version, the TUI bundle is rebuilt automatically when sources are newer than the dist.

External prebuild

Distributions that ship a prebuilt bundle (Nix, system packages) can point Hermes at it:

export HERMES_TUI_DIR=/path/to/prebuilt/ui-tui
hermes --tui

The directory must contain dist/entry.js.

Keybindings

Keybindings match the Classic CLI exactly. The only behavioral differences:

  • Mouse drag highlights text with a uniform selection background.
  • Cmd+V / Ctrl+V first tries normal text paste, then falls back to OSC52/native clipboard reads, and finally image attach when the clipboard or pasted payload resolves to an image.
  • /terminal-setup installs local VS Code / Cursor / Windsurf terminal bindings for better Cmd+Enter and undo/redo parity on macOS.
  • Slash autocompletion opens as a floating panel with descriptions, not an inline dropdown.
  • Ctrl+X — when a queued message is highlighted (sent while the agent was still running), delete it from the queue. Esc cancels editing and unhighlights without deleting.
  • Ctrl+G / Ctrl+X Ctrl+E — open the current input buffer in $EDITOR for multi-line / long-prompt composition; save-and-exit sends the contents back as the prompt.

Slash commands

All slash commands work unchanged. A few are TUI-owned — they produce richer output or render as overlays rather than inline panels:

Command TUI behavior
/help Overlay with categorized commands, arrow-key navigable
/sessions Modal session picker — preview, title, token totals, resume inline
/model Modal model picker grouped by provider, with cost hints
/skin Live preview — theme change applies as you browse
/details Toggle verbose tool-call details (global or per-section)
/usage Rich token / cost / context panel
/agents (alias /tasks) Observability overlay — live subagent tree with kill/pause controls, per-branch cost / token / file rollups, turn-by-turn history
/reload Re-reads ~/.hermes/.env into the running TUI process so newly added API keys take effect without a restart
/mouse [on|off|toggle|wheel|buttons|all] Pick a mouse tracking preset at runtime (also persists to display.mouse_tracking in config.yaml). wheel (1000+1006) keeps scroll-wheel scrolling without the hover events that make tmux spam "No image in clipboard" over the prompt row; buttons adds drag-to-select; all is the default with hover-driven UI.

Every other slash command (including installed skills, quick commands, and personality toggles) works identically to the classic CLI. See Slash Commands Reference.

LaTeX math rendering

The TUI's markdown pipeline renders LaTeX math inline: $E = mc^2$ and $$\frac{a}{b}$$ render as Unicode-formatted math instead of the raw TeX source. Works for inline and block math; unsupported syntax falls back to showing the literal TeX wrapped in a code span so it remains copyable.

This is always-on — nothing to configure. Classic CLI keeps the raw TeX.

Light-terminal detection

The TUI auto-detects light terminals and swaps to the light theme accordingly. Detection works in three layers:

  1. HERMES_TUI_THEME env var — highest priority. Values: light, dark, or a raw 6-char background hex (e.g. ffffff, 1a1a2e).
  2. COLORFGBG env var — the classic "what's my background color?" hint used by xterm-derived terminals.
  3. Terminal background probe via OSC 11 — works on modern terminals (Ghostty, Warp, iTerm2, WezTerm, Kitty) that don't set COLORFGBG.

If you want the light theme permanently regardless of terminal:

export HERMES_TUI_THEME=light

Busy indicator styles

The status-bar busy indicator is pluggable — the default rotates Hermes' kawaii face palette every 2.5 seconds during agent work. Pick a different style via config or the /indicator slash command:

display:
  tui_status_indicator: kaomoji   # kaomoji | emoji | unicode | ascii

Or in-session: /indicator emoji (etc.). Styles ship with matched glyph widths so the rest of the status bar doesn't jitter on rotation.

Auto-resume

By default, hermes --tui starts a fresh session each launch. To re-attach to the most recent TUI session automatically (useful when your terminal or SSH connection drops unexpectedly), opt in:

export HERMES_TUI_RESUME=1          # most-recent TUI session
# or:
export HERMES_TUI_RESUME=<session-id>   # specific session

Unset the variable or pass --resume <id> explicitly to override on a per-launch basis.

Status line

The TUI's status line tracks agent state in real time:

Status Meaning
starting agent… Session ID is live; tools and skills still coming online. You can type — messages queue and send when ready.
ready Agent is idle, accepting input.
thinking… / running… Agent is reasoning or running a tool.
interrupted Current turn was cancelled; press Enter to send again.
forging session… / resuming… Initial connect or --resume handshake.

The per-skin status-bar colors and thresholds are shared with the classic CLI — see Skins for customization.

The status line also shows:

  • Working directory with git branch~/projects/hermes-agent (docs/two-week-gap-sweep). The branch suffix updates when you git checkout in a side terminal (mtime-cached) so the TUI reflects your actual active branch, not whatever it was at launch.
  • Per-prompt elapsed time⏱ 12s/3m 45s while the turn is running (live), frozen to ⏲ 32s / 3m 45s after the turn completes. First number is time since last user message; second is total session duration. Resets on every new prompt.
  • 🗜️ N — number of times the running session has been auto-compressed. Appears once the first compression fires.
  • ▶ N — number of /background tasks currently running in this session. Appears whenever at least one task is in flight.
  • ⚠ YOLO — visible warning whenever YOLO mode is on (hermes --yolo, /yolo, or HERMES_YOLO_MODE=1). The same badge also appears in the startup banner so you cannot launch an auto-approving session without noticing.

Configuration

The TUI respects all standard Hermes config: ~/.hermes/config.yaml, profiles, personalities, skins, quick commands, credential pools, memory providers, tool/skill enablement. No TUI-specific config file exists.

A handful of keys tune the TUI surface specifically:

display:
  skin: default              # any built-in or custom skin
  personality: helpful
  details_mode: collapsed    # hidden | collapsed | expanded — global accordion default
  sections:                  # optional: per-section overrides (any subset)
    thinking: expanded       # always open
    tools: expanded          # always open
    activity: collapsed      # opt back IN to the activity panel (hidden by default)
  mouse_tracking: all        # off | wheel | buttons | all (or true/false for back-compat).
                             #   wheel   — 1000+1006 (scroll + click; no drag, no hover —
                             #             recommended inside tmux to silence the prompt-row
                             #             "No image in clipboard" spam from hover events)
                             #   buttons — adds 1002 for terminal-side drag selection
                             #   all     — adds 1003 for hover (scrollbar paginate-on-hover,
                             #             link mouseenter, etc.)

Runtime toggles:

  • /details [hidden|collapsed|expanded|cycle] — set the global mode
  • /details <section> [hidden|collapsed|expanded|reset] — override one section (sections: thinking, tools, subagents, activity)

Default visibility

The TUI ships with opinionated per-section defaults that stream the turn as a live transcript instead of a wall of chevrons:

  • thinkingexpanded. Reasoning streams inline as the model emits it.
  • toolsexpanded. Tool calls and their results render open.
  • subagents — falls through to the global details_mode (collapsed under chevron by default — stays quiet until a delegation actually happens).
  • activityhidden. Ambient meta (gateway hints, terminal-parity nudges, background notifications) is noise for most day-to-day use. Tool failures still render inline on the failing tool row; ambient errors/warnings surface via a floating-alert backstop when every panel is hidden.

Per-section overrides take precedence over both the section default and the global details_mode. To reshape the layout:

  • display.sections.thinking: collapsed — put thinking back under a chevron
  • display.sections.tools: collapsed — put tool calls back under a chevron
  • display.sections.activity: collapsed — opt the activity panel back in
  • /details <section> <mode> at runtime

Anything set explicitly in display.sections wins over the defaults, so existing configs keep working unchanged.

Sessions

Sessions are shared between the TUI and the classic CLI — both write to the same ~/.hermes/state.db. You can start a session in one, resume in the other. The session picker surfaces sessions from both sources, with a source tag.

See Sessions for lifecycle, search, compression, and export.

Attaching to a running gateway

By default the TUI spawns its own in-process gateway, so each TUI instance is self-contained. If you already have a long-lived gateway running (e.g. hermes gateway run in tmux, or the systemd / launchd service), you can point the TUI at that gateway instead — the TUI then becomes a thin client and shares state with every other surface (messaging platforms, web dashboard, other TUI sessions) that's attached to the same gateway.

Set the websocket URL via env before launching:

export HERMES_TUI_GATEWAY_URL="ws://localhost:8765/api/ws?token=<auth-token>"
hermes --tui

The token comes from the gateway's API auth configuration (see API Server). When the env var is set, the TUI:

  • Skips spawning a local gateway entirely — no duplicate platform adapters, no port conflicts.
  • Routes every action (slash commands, image attach, browser progress, voice events, …) over the websocket to the shared gateway.
  • Reconnects automatically if the gateway URL rotates (new token) between requests.

This is the same channel the web dashboard's embedded TUI uses (see Web Dashboard) — one gateway, many clients.

Reverting to the classic CLI

Launching hermes (without --tui) stays on the classic CLI. To make a machine prefer the TUI, set HERMES_TUI=1 in your shell profile. To go back, unset it.

If the TUI fails to launch (no Node, missing bundle, TTY issue), Hermes prints a diagnostic and falls back — rather than leaving you stuck.

See also