feat(tui): mouse_tracking DEC mode presets (salvage of #26681) (#30084)

* feat(tui): make display.mouse_tracking pick which DEC modes to enable Previously the boolean flag was all-or-nothing across modes 1000+1002+1003+1006. Inside tmux, mode 1003 (any-motion) makes every mouse cross of the prompt row fire a clipboard probe that surfaces as "No image in clipboard" — sometimes dozens in a row. Disabling tracking entirely killed scroll-wheel scrolling too, since tmux's own scrollback is preempted by the alt-screen TUI. `display.mouse_tracking` (and `/mouse <preset>`) now accepts `off | wheel | buttons | all` in addition to the legacy booleans. `wheel` is 1000+1006: scroll wheel + click only, no drag, no hover — the tmux-friendly subset. `buttons` adds 1002 for drag-to-select. `all` (= legacy `true`) keeps the hover-driven UI (scrollbar paginate-on-hover, link mouseenter, etc.). * fix(tui): repaint + sync mouse mode when display.mouse_tracking changes Two interacting bugs left the TUI blank when `display.mouse_tracking` switched at runtime (config edit, /mouse <preset>): 1. AlternateScreen's effect re-runs on every `mouseTracking` change, tearing down and re-entering the alt screen. After re-entry, ink's frame buffers are reset by `resetFramesForAltScreen()` but nothing schedules the follow-up render — the alt screen sits blank until some other state change happens to trigger one. Add a `scheduleRender()` in `setAltScreenActive`'s active=true branch so the freshly-entered alt screen gets a full repaint immediately. 2. `setAltScreenActive` early-returns when `active` hasn't changed, which silently drops a `mouseTracking` change if the cleanup→setup pair somehow leaves `altScreenActive` already true. Call `setAltScreenMouseTracking` explicitly from the AlternateScreen effect so the in-memory mode and terminal DECSET sequence stay in sync regardless of how `setAltScreenActive` resolved (the call is a no-op when the mode is unchanged). * fix(tui): address copilot review #4341269705 - tui_gateway/server.py: drop the never-referenced _MOUSE_TRACKING_MODES frozenset (comment #3284802434). _MOUSE_TRACKING_ALIASES already centralizes the canonical preset set via its values; the separate constant added no behavior. - tests/test_tui_gateway_server.py: update the existing test_config_mouse_uses_documented_key_with_legacy_fallback to assert the new preset strings ('all'/'off' instead of 'on'/'off', display.mouse_tracking persisted as 'all' instead of True) and add test_config_mouse_accepts_preset_strings_and_aliases covering /mouse set with wheel/click/unknown (comment #3284802453). The on/off legacy config.set return shape was an implementation detail of the boolean flag, not a stable API — the slash command, gateway help text, and docs all advertise the preset values now. - ui-tui/packages/hermes-ink/src/ink/ink.tsx: schedule a render at the end of reenterAltScreen() (comment #3284802461). Mirrors the same fix in setAltScreenActive() from ece0a2f4c — without it, SIGCONT/resize self-heal/stdin-gap re-entry leaves the alt screen blank because every caller returns early after invoking us. * fix(tui): address copilot review #4341308478 round 2 - ui-tui/src/config/env.ts (comment #3284837577): the precedence comment was misleading. Actual behavior on origin/main is HERMES_TUI_MOUSE_TRACKING (explicit override) > Termux default > HERMES_TUI_DISABLE_MOUSE legacy kill-switch. This is preserved from main; the only change here was the wrong comment that claimed DISABLE_MOUSE kept kill-switch semantics. Rewrote the comment block to document the actual precedence ladder. - tui_gateway/server.py /mouse set (comment #3284837607): replaced 'str(value or "").strip().lower()' with the explicit None idiom already used for /indicator, so programmatic callers can pass 0 / False and have them route through _MOUSE_TRACKING_ALIASES → 'off' instead of collapsing to '' and triggering the toggle path. - ui-tui/packages/hermes-ink/src/ink/components/AlternateScreen.tsx (comment #3284837620): always prepend DISABLE_MOUSE_TRACKING before enableMouseTrackingFor(...) on mount. Otherwise selecting 'wheel'/'buttons' from a state where DEC 1003 was already asserted (crash, another app, debugger) would silently leave hover on. Also unconditionally DISABLE on unmount so a crash mid-mount can't leak DEC modes back to the host shell. * chore(release): map nat@nthrow.io to @nthrow for #26681 salvage * fix(tui): drop redundant setAltScreenMouseTracking in AlternateScreen Copilot review #4341356637 (comment #3284880417). The explicit setAltScreenMouseTracking(mouseTracking) after setAltScreenActive(true, mouseTracking) was defensive paranoia added in the previous fix commit that's not actually reachable in practice: - React's cleanup always runs before the next setup, so on any prop change (mouseTracking or writeRaw) the cleanup sets active=false first. Setup then sees active was false and applies the new mode via setAltScreenActive without early-returning. - On the impossible 'active stayed true' path, the writeRaw above has already sent DISABLE_MOUSE_TRACKING + enableMouseTrackingFor(newMode) to the terminal, so the in-memory mode would lag but the visible state is already correct. Removing the redundant call means a single DEC sequence per mount. If the 'active stayed true' path ever manifests in practice, the right fix is in setAltScreenActive (track mode regardless of the active early-return), not here. * fix(tui): always DISABLE before enableMouseTrackingFor in ink.tsx Copilot review #4341379994 (comments #3284900825, #3284900840, #3284900852). Three remaining call sites in ink.tsx still re-enabled mouse tracking without first sending DISABLE_MOUSE_TRACKING: - handleResize alt-screen recovery (line ~577) - reassertTerminalModes stdin-gap re-assertion (line ~1351) - reenterAltScreen SIGCONT/resize/stdin-gap self-heal (line ~1408) For 'wheel'/'buttons' presets, omitting DISABLE leaves any externally- asserted DEC 1003 (other apps, prior crash, tmux state) still active and the hover-free preset silently has hover on. DISABLE_MOUSE_TRACKING is idempotent and safe to send unconditionally — it resets all four modes. Matches the pattern already in setAltScreenMouseTracking and the AlternateScreen mount path. * fix(tui): always DISABLE before enableMouseTrackingFor in exitAlternateScreen Copilot review #4341452823 (comment #3284959762). exitAlternateScreen() was the last call site in ink.tsx still re-enabling mouse tracking without DISABLE first. Editors (vim/nvim/less) and tmux can leave DEC 1003 hover asserted across the handoff back; without DISABLE, 'wheel'/'buttons' presets silently kept hover on after the editor quit. Now all five enableMouseTrackingFor() call sites in ink.tsx prepend DISABLE_MOUSE_TRACKING — handleResize, reassertTerminalModes, reenterAltScreen, setAltScreenMouseTracking, exitAlternateScreen. * fix(tui): add defensive default to enableMouseTrackingFor switch Copilot review #4341485231 (comment #3284979323). TS exhaustive switch returns string per the type system, but a JS caller / corrupted config / hot-reload-in-dev could reach the function with an unknown value at runtime. Without a default, that path returns undefined which then concatenates as the literal string 'undefined' into the terminal byte stream — visibly garbling output. Treat unknown as 'off' (no DEC sequences) so the worst case is silent input loss rather than a wrecked screen. --------- Co-authored-by: Nat Thrower <nat@nthrow.io>
2026-06-06 07:51:53 +00:00 · 2026-05-21 20:25:52 -05:00 · 2026-05-21 20:25:52 -05:00 · a7cd254c29
commit a7cd254c29
parent 4d58e48cdb
14 changed files with 399 additions and 88 deletions
--- a/ui-tui/src/tests/useConfigSync.test.ts
+++ b/ui-tui/src/tests/useConfigSync.test.ts
@ -77,13 +77,26 @@ describe('applyDisplay', () => {
    const setBell = vi.fn()

    applyDisplay({ config: { display: { mouse_tracking: false } } }, setBell)
-    expect($uiState.get().mouseTracking).toBe(false)
+    expect($uiState.get().mouseTracking).toBe('off')

    applyDisplay({ config: { display: { mouse_tracking: true, tui_mouse: false } } }, setBell)
-    expect($uiState.get().mouseTracking).toBe(true)
+    expect($uiState.get().mouseTracking).toBe('all')

    applyDisplay({ config: { display: { tui_mouse: false } } }, setBell)
-    expect($uiState.get().mouseTracking).toBe(false)
+    expect($uiState.get().mouseTracking).toBe('off')
+  })
+
+  it('threads mouse_tracking presets through to $uiState', () => {
+    const setBell = vi.fn()
+
+    applyDisplay({ config: { display: { mouse_tracking: 'wheel' } } }, setBell)
+    expect($uiState.get().mouseTracking).toBe('wheel')
+
+    applyDisplay({ config: { display: { mouse_tracking: 'buttons' } } }, setBell)
+    expect($uiState.get().mouseTracking).toBe('buttons')
+
+    applyDisplay({ config: { display: { mouse_tracking: 'all' } } }, setBell)
+    expect($uiState.get().mouseTracking).toBe('all')
  })

  it('parses display.sections into per-section overrides', () => {
@ -183,15 +196,30 @@ describe('normalizeStatusBar', () => {
 })

 describe('normalizeMouseTracking', () => {
-  it('defaults on and prefers canonical mouse_tracking over legacy tui_mouse', () => {
-    expect(normalizeMouseTracking({})).toBe(true)
-    expect(normalizeMouseTracking({ mouse_tracking: false })).toBe(false)
-    expect(normalizeMouseTracking({ mouse_tracking: 0 })).toBe(false)
-    expect(normalizeMouseTracking({ mouse_tracking: 'off' })).toBe(false)
-    expect(normalizeMouseTracking({ mouse_tracking: 'false' })).toBe(false)
-    expect(normalizeMouseTracking({ mouse_tracking: null, tui_mouse: false })).toBe(true)
-    expect(normalizeMouseTracking({ mouse_tracking: true, tui_mouse: false })).toBe(true)
-    expect(normalizeMouseTracking({ tui_mouse: false })).toBe(false)
+  it('defaults to all and prefers canonical mouse_tracking over legacy tui_mouse', () => {
+    expect(normalizeMouseTracking({})).toBe('all')
+    expect(normalizeMouseTracking({ mouse_tracking: false })).toBe('off')
+    expect(normalizeMouseTracking({ mouse_tracking: 0 })).toBe('off')
+    expect(normalizeMouseTracking({ mouse_tracking: 'off' })).toBe('off')
+    expect(normalizeMouseTracking({ mouse_tracking: 'false' })).toBe('off')
+    expect(normalizeMouseTracking({ mouse_tracking: null, tui_mouse: false })).toBe('all')
+    expect(normalizeMouseTracking({ mouse_tracking: true, tui_mouse: false })).toBe('all')
+    expect(normalizeMouseTracking({ tui_mouse: false })).toBe('off')
+  })
+
+  it('accepts preset strings (wheel/buttons/all) and their aliases', () => {
+    expect(normalizeMouseTracking({ mouse_tracking: 'wheel' })).toBe('wheel')
+    expect(normalizeMouseTracking({ mouse_tracking: 'scroll' })).toBe('wheel')
+    expect(normalizeMouseTracking({ mouse_tracking: 'buttons' })).toBe('buttons')
+    expect(normalizeMouseTracking({ mouse_tracking: 'click' })).toBe('buttons')
+    expect(normalizeMouseTracking({ mouse_tracking: 'all' })).toBe('all')
+    expect(normalizeMouseTracking({ mouse_tracking: 'full' })).toBe('all')
+    expect(normalizeMouseTracking({ mouse_tracking: 'on' })).toBe('all')
+    expect(normalizeMouseTracking({ mouse_tracking: ' WHEEL ' })).toBe('wheel')
+  })
+
+  it('falls back to all for unknown strings', () => {
+    expect(normalizeMouseTracking({ mouse_tracking: 'rainbows' })).toBe('all')
  })
 })

--- a/ui-tui/src/app/interfaces.ts
+++ b/ui-tui/src/app/interfaces.ts
@ -1,4 +1,4 @@
-import type { ScrollBoxHandle } from '@hermes/ink'
+import type { MouseTrackingMode, ScrollBoxHandle } from '@hermes/ink'
 import type { MutableRefObject, ReactNode, RefObject, SetStateAction } from 'react'

 import type { PasteEvent } from '../components/textInput.js'
@ -104,7 +104,7 @@ export interface UiState {
  detailsModeCommandOverride: boolean
  info: null | SessionInfo
  inlineDiffs: boolean
-  mouseTracking: boolean
+  mouseTracking: MouseTrackingMode
  sections: SectionVisibility
  showCost: boolean
  showReasoning: boolean
@ -351,7 +351,7 @@ export interface AppLayoutTranscriptProps {
 export interface AppLayoutProps {
  actions: AppLayoutActions
  composer: AppLayoutComposerProps
-  mouseTracking: boolean
+  mouseTracking: MouseTrackingMode
  progress: AppLayoutProgressProps
  status: AppLayoutStatusProps
  transcript: AppLayoutTranscriptProps
--- a/ui-tui/src/app/slash/commands/core.ts
+++ b/ui-tui/src/app/slash/commands/core.ts
@ -1,9 +1,9 @@
-import { forceRedraw } from '@hermes/ink'
+import { forceRedraw, type MouseTrackingMode } from '@hermes/ink'

 import { NO_CONFIRM_DESTRUCTIVE } from '../../../config/env.js'
 import { dailyFortune, randomFortune } from '../../../content/fortunes.js'
 import { HOTKEYS } from '../../../content/hotkeys.js'
-import { SECTION_NAMES, isSectionName, nextDetailsMode, parseDetailsMode } from '../../../domain/details.js'
+import { isSectionName, nextDetailsMode, parseDetailsMode, SECTION_NAMES } from '../../../domain/details.js'
 import type {
  ConfigGetValueResponse,
  ConfigSetResponse,
@ -44,6 +44,30 @@ const flagFromArg = (arg: string, current: boolean): boolean | null => {
  return null
 }

+// `/mouse` toggles between full tracking and off when called bare so the
+// old binary muscle-memory still works. Explicit presets (wheel / buttons /
+// all) target the tmux-friendly hover-free subsets.
+const MOUSE_MODE_ALIASES: Record<string, MouseTrackingMode> = {
+  all: 'all',
+  any: 'all',
+  button: 'buttons',
+  buttons: 'buttons',
+  click: 'buttons',
+  full: 'all',
+  off: 'off',
+  on: 'all',
+  scroll: 'wheel',
+  wheel: 'wheel'
+}
+
+const mouseModeFromArg = (arg: string, current: MouseTrackingMode): MouseTrackingMode | null => {
+  if (!arg || arg.trim().toLowerCase() === 'toggle') {
+    return current === 'off' ? 'all' : 'off'
+  }
+
+  return MOUSE_MODE_ALIASES[arg.trim().toLowerCase()] ?? null
+}
+
 const RESET_WORDS = new Set(['reset', 'clear', 'default'])
 const CYCLE_WORDS = new Set(['cycle', 'toggle'])

@ -105,20 +129,20 @@ export const coreCommands: SlashCommand[] = [

  {
    aliases: ['scroll'],
-    help: 'toggle mouse/wheel tracking [on|off|toggle]',
+    help: 'set mouse tracking preset [on|off|toggle|wheel|buttons|all]',
    name: 'mouse',
    run: (arg, ctx) => {
      const current = ctx.ui.mouseTracking
-      const next = flagFromArg(arg, current)
+      const next = mouseModeFromArg(arg, current)

      if (next === null) {
-        return ctx.transcript.sys('usage: /mouse [on|off|toggle]')
+        return ctx.transcript.sys('usage: /mouse [on|off|toggle|wheel|buttons|all]')
      }

      patchUiState({ mouseTracking: next })
-      ctx.gateway.rpc<ConfigSetResponse>('config.set', { key: 'mouse', value: next ? 'on' : 'off' }).catch(() => {})
+      ctx.gateway.rpc<ConfigSetResponse>('config.set', { key: 'mouse', value: next }).catch(() => {})

-      queueMicrotask(() => ctx.transcript.sys(`mouse tracking ${next ? 'on' : 'off'}`))
+      queueMicrotask(() => ctx.transcript.sys(`mouse tracking ${next}`))
    }
  },

--- a/ui-tui/src/app/useConfigSync.ts
+++ b/ui-tui/src/app/useConfigSync.ts
@ -1,3 +1,4 @@
+import type { MouseTrackingMode } from '@hermes/ink'
 import { useEffect, useRef } from 'react'

 import { resolveDetailsMode, resolveSections } from '../domain/details.js'
@ -9,8 +10,8 @@ import type {
 } from '../gatewayTypes.js'
 import {
  DEFAULT_VOICE_RECORD_KEY,
-  parseVoiceRecordKey,
-  type ParsedVoiceRecordKey
+  type ParsedVoiceRecordKey,
+  parseVoiceRecordKey
 } from '../lib/platform.js'
 import { asRpcResult } from '../lib/rpc.js'

@ -68,16 +69,57 @@ export const normalizeIndicatorStyle = (raw: unknown): IndicatorStyle => {
 }

 const FALSEY_MOUSE = new Set(['0', 'false', 'no', 'off'])
+const TRUTHY_MOUSE_ALL = new Set(['1', 'true', 'yes', 'on', 'all', 'full', 'any'])
 const hasOwn = (obj: object, key: PropertyKey) => Object.prototype.hasOwnProperty.call(obj, key)

-export const normalizeMouseTracking = (display: { mouse_tracking?: unknown; tui_mouse?: unknown }): boolean => {
+// `display.mouse_tracking` accepts boolean (`true` ⇒ all modes, `false` ⇒ off)
+// for back-compat, plus the string presets `off|wheel|buttons|all` (aliases:
+// `on`/`full`/`any`/`1`/`true`/... → `all`; `0`/`false`/`no`/`off` → `off`).
+// `wheel` enables 1000+1006 — scroll wheel + click only, no drag or hover,
+// which silences tmux's "No image in clipboard" spam over the prompt row.
+// `buttons` adds 1002 so terminal-side text selection drags still register.
+// Legacy `tui_mouse` is honored only if `mouse_tracking` is absent.
+export const normalizeMouseTracking = (display: {
+  mouse_tracking?: unknown
+  tui_mouse?: unknown
+}): MouseTrackingMode => {
  const raw = hasOwn(display, 'mouse_tracking') ? display.mouse_tracking : display.tui_mouse

  if (raw === false || raw === 0) {
-    return false
+    return 'off'
  }

-  return typeof raw === 'string' ? !FALSEY_MOUSE.has(raw.trim().toLowerCase()) : true
+  if (raw === true || raw === undefined || raw === null) {
+    return 'all'
+  }
+
+  if (typeof raw === 'number') {
+    return 'all'
+  }
+
+  if (typeof raw !== 'string') {
+    return 'all'
+  }
+
+  const v = raw.trim().toLowerCase()
+
+  if (FALSEY_MOUSE.has(v)) {
+    return 'off'
+  }
+
+  if (TRUTHY_MOUSE_ALL.has(v)) {
+    return 'all'
+  }
+
+  if (v === 'wheel' || v === 'scroll') {
+    return 'wheel'
+  }
+
+  if (v === 'buttons' || v === 'button' || v === 'click') {
+    return 'buttons'
+  }
+
+  return 'all'
 }

 const MTIME_POLL_MS = 5000
@ -114,6 +156,7 @@ export async function hydrateFullConfig(
 ): Promise<ConfigFullResponse | null> {
  const cfg = await quietRpc<ConfigFullResponse>(gw, 'config.get', { key: 'full' })
  applyDisplay(cfg, setBell, setVoiceRecordKey)
+
  return cfg
 }

@ -125,6 +168,7 @@ export const applyDisplay = (
  const d = cfg?.config?.display ?? {}

  setBell(!!d.bell_on_complete)
+
  // Only push the voice record key when the RPC actually returned a
  // config payload. ``quietRpc()`` collapses failures to ``null``; if we
  // reset the cached shortcut on every null we would clobber a custom
@ -135,6 +179,7 @@ export const applyDisplay = (
  if (setVoiceRecordKey && cfg) {
    setVoiceRecordKey(_voiceRecordKeyFromConfig(cfg))
  }
+
  patchUiState({
    busyInputMode: normalizeBusyInputMode(d.busy_input_mode),
    compact: !!d.tui_compact,
--- a/ui-tui/src/config/env.ts
+++ b/ui-tui/src/config/env.ts
@ -1,3 +1,4 @@
+import type { MouseTrackingMode } from '@hermes/ink'
 import { isTermuxTuiMode } from '../lib/termux.js'

 const truthy = (v?: string) => /^(?:1|true|yes|on)$/i.test((v ?? '').trim())
@ -27,13 +28,24 @@ export const STARTUP_RESUME_ID = (process.env.HERMES_TUI_RESUME ?? '').trim()
 export const STARTUP_QUERY = (process.env.HERMES_TUI_QUERY ?? '').trim()
 export const STARTUP_IMAGE = (process.env.HERMES_TUI_IMAGE ?? '').trim()

+// Mouse tracking mode resolution at startup. Per-mode selection (off|wheel|
+// buttons|all) lives in display.mouse_tracking in config.yaml — these env
+// vars only set the boot-time default before that config is applied.
+//
+// Precedence (highest first):
+//
+// - HERMES_TUI_MOUSE_TRACKING (truthy/falsy) explicitly overrides everything.
+//   This is the "force a value" knob and intentionally beats the legacy
+//   kill-switch and the Termux default.
+// - HERMES_TUI_DISABLE_MOUSE=1 forces mouse off — the legacy kill switch.
+// - On Termux the default is mouse off so touch selection isn't intercepted
+//   by terminal mouse protocols. Desktop defaults to 'all' to preserve prior
+//   behavior.
 const mouseTrackingOverride = parseToggle(process.env.HERMES_TUI_MOUSE_TRACKING)
 const mouseTrackingDisabledLegacy = truthy(process.env.HERMES_TUI_DISABLE_MOUSE)
-// Mobile selection UX: on Termux default mouse tracking OFF so touch selection
-// is less likely to be intercepted by terminal mouse protocols. Desktop keeps
-// prior behavior unless explicitly overridden.
-export const MOUSE_TRACKING =
+const resolvedBootMouseEnabled =
  mouseTrackingOverride ?? (TERMUX_TUI_MODE ? false : !mouseTrackingDisabledLegacy)
+export const MOUSE_TRACKING: MouseTrackingMode = resolvedBootMouseEnabled ? 'all' : 'off'

 export const NO_CONFIRM_DESTRUCTIVE = truthy(process.env.HERMES_TUI_NO_CONFIRM)