Commit graph

1912 commits

Author SHA1 Message Date
Teknium
7e7f7d3059
Merge pull request #70509 from NousResearch/hermes/hermes-29661bf6
feat(voice): on-device wake words with open-vocabulary phrases and multi-profile voice routing
2026-07-28 17:58:33 -07:00
brooklyn!
392f1b9477
Merge pull request #73705 from NousResearch/bb/terminal-clipboard
feat(desktop): copy and paste in the GUI terminal
2026-07-28 19:58:31 -05:00
Brooklyn Nicholson
a7e3536a70 fix(desktop): align the composer status lane with the surface
The lane is `inset-x-0`, which resolves against the composer root's
padding box, while the surface and the underside strip sit in its
content box — so the pill strip hung 5px further left than both.

Measured: pills 321.83, chip 326.83, surface 326.83.
2026-07-28 19:42:35 -05:00
brooklyn!
ded2314910
Merge pull request #73711 from NousResearch/bb/composer-micro-actions
Composer contribution seams: micro-action pills and an underside strip
2026-07-28 19:28:42 -05:00
brooklyn!
6cccef6c10
Merge pull request #73710 from NousResearch/bb/triage-large-remote-attach
fix(desktop): allow large remote attachments (supersedes #66555)
2026-07-28 19:28:01 -05:00
brooklyn!
9b6d91045c
Merge pull request #73704 from NousResearch/bb/slash-skills-by-usage
Rank the slash menu by the skills you actually use
2026-07-28 19:20:49 -05:00
Brooklyn Nicholson
254aeda122 fix(desktop): dedicated reader for large remote attachments
Keep Settings-configurable preview/image loads on readFileDataUrl, and
route remote non-image attach through a 256 MiB IPC so uploads are not
stuck on the 16 MiB default after #73221.

Co-authored-by: Börje <borje@dqsverige.se>
2026-07-28 19:20:14 -05:00
Brooklyn Nicholson
1e355116a1 fix(desktop): keep composer chrome out of the pop-out gesture
The pill strips live inside the composer root, so their box sits within
the drag region and their gaps read as grab area. Exclude anything
marked composer-no-drag, matching how buttons and menu items are already
excluded.
2026-07-28 19:20:07 -05:00
Brooklyn Nicholson
9b01c74f00 feat(desktop): render the micro-action and underside strips
Pills pin to the top of the composer's overlay lane, outside the status
card and outside its scroller, so nothing stacks above them and a long
todo list can't scroll them away. The underside slot sits below the
surface. Both share one grid constant and one parent, so their left
edges match by construction rather than by matching numbers in two
files.

The strips take pointer events while their empty space falls through to
the pop-out drag region, which keeps the composer draggable by the band
its badges live in.
2026-07-28 19:20:00 -05:00
Brooklyn Nicholson
1cefad5523 feat(desktop): composer.microActions + composer.underside contribution areas
Two new seams on the composer, both through the existing contribution
registry: a data area whose providers resolve badge descriptors per
session, and a render area for a chrome-free strip below the surface.
Core registers nothing in either — they stay empty until something
contributes.

Providers resolve from live session context rather than registering
static entries, so a contribution can be conditional without the
reactive when() the registry deliberately doesn't offer.
2026-07-28 19:19:52 -05:00
Brooklyn Nicholson
807dc0c45f feat(desktop): copy and paste in the GUI terminal
xterm paints to a canvas, so its selection is not a DOM selection: the Edit
menu's Copy and the right-click Copy both call webContents.copy(), find
nothing, and copy nothing. On macOS the menu also swallows the Cmd+C
accelerator before the renderer sees it. That left Ctrl+C as the only key that
did anything, and xterm correctly forwards it to the PTY as SIGINT.

Mirror the selection into xterm's hidden helper textarea (the mechanism xterm
already uses for Linux middle-click paste) so the OS sees a real selection and
every platform copy path works, and add explicit chords on top: Cmd+C/Cmd+V on
macOS, Ctrl+Shift+C/V elsewhere, matching VS Code. Plain Ctrl+C copies only
when text is selected -- the behavior Windows Terminal and Tabby ship -- and
stays SIGINT otherwise, so interrupting a process never breaks.

Reads go through a new hermes:readClipboard IPC handler for the same reason
writes already do: the renderer's clipboard API throws whenever the document
isn't focused.
2026-07-28 19:15:04 -05:00
Brooklyn Nicholson
959d2993ab feat(desktop): rank the slash menu by the skills you actually use
The `/` popover listed skills alphabetically, so on a 200-skill install
the ones invoked daily sat below a wall of skills that shipped with
Hermes and were never opened — /research-paper-writing (never used)
outranked /research (60 invocations).

Sort the Skills section most-used first, A-Z within a tie, and on a bare
`/` drop bundled skills with no recorded activity. Typing a query keeps
every match: a search that hides a result is broken, so a typed `/re`
only reorders.
2026-07-28 19:14:46 -05:00
hermes-seaeye[bot]
9dc191d437
fmt(js): npm run fix on merge (#73699)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-07-29 00:09:37 +00:00
Brooklyn Nicholson
7d5b92cdfb fix(desktop): bump session list order on user send
Recents were hard-sorted by started_at and only refreshed last_active after
turn complete — so reviving an old thread stayed buried until the assistant
finished. Stamp last_active on prompt seed, keep it monotonic across mid-turn
refreshes, and sort agent recents by activity.
2026-07-28 18:53:33 -05:00
brooklyn!
6c2a3b4bb2
Merge pull request #73673 from NousResearch/bb/desktop-eventbus
Desktop: event-driven live sync — gateway change broadcasts replace the always-on polls
2026-07-28 17:39:40 -05:00
Brooklyn Nicholson
fa8fb82d0e test(gateway): change-watcher behavior contracts; align status-snapshot test with 60s cadence
Real temp-HERMES_HOME tests for _broadcast_watched_changes: silent seed,
cron/sessions signature moves, the 2s sessions floor's trailing edge, the
pet signature staying 'off' without a renderable pet, meta payload on
pet.changed, and a broken probe never killing the pass. Part of #73618.
2026-07-28 17:21:58 -05:00
Brooklyn Nicholson
c14ae3796e feat(desktop): fs-watch the plugin dir + demote the status snapshot
- watchDirectory IPC (same registry/channel as the preview file watchers)
  replaces the disk-plugin door's 5s readdir poll; older shells without
  the capability keep the poll, which self-upgrades to the watch once the
  plugins dir exists.
- Status snapshot: 15s → 60s, skips round-trips while hidden, and
  refreshes immediately on visibilitychange so re-focus never shows stale
  health.

Part of #73618.
2026-07-28 17:21:58 -05:00
Brooklyn Nicholson
fe64cafa33 feat(desktop): event-driven live sync — change broadcasts replace the always-on polls
One controller (store/live-sync.ts, the workspace-events twin for gateway
state): gateway-event.ts routes pet.changed / cron.changed /
sessions.changed into tick atoms, gated on the active profile like
skin.changed; gateway.ready's change_events flag arms them; a gateway wipe
resets the capability.

Consumers subscribe to ticks and demote their polls to slow backstops
(legacy cadence is kept verbatim against older backends):

- session.active_list 1.5s → sessions.changed + 30s backstop
- cron list 30s → cron.changed + 5min backstop
- messaging lists 10s → trailing sessions.changed refresh + legacy poll
  only when events are unavailable
- open messaging transcript 5s → sessions.changed + 30s backstop
- pet.info 3s/15s → pet.changed, NO timer on event-capable backends
  (users with no pet used to poll hardest); enabled=false broadcasts
  clear the mascot with zero round-trips
- cron runs page/peek 8s → cron.changed + 60s backstop

Idle renderer traffic drops from ~89 req/min to the backstops (~5/min).
Part of #73618.
2026-07-28 17:21:58 -05:00
brooklyn!
43ddefd08f
Merge pull request #73675 from NousResearch/bb/edit-models-hover
fix(desktop): match Edit Models row hover to the model picker
2026-07-28 17:19:44 -05:00
brooklyn!
f9d7bca252
fix(desktop): overlay scrollbars on conversation code blocks (#73670)
The app-wide `.scrollbar-dt *` webkit rules force classic always-on
gutters on every descendant scroller. Opt code-card scroll surfaces out
via `.scrollbar-overlay` so Electron keeps platform overlay bars (fade
in on scroll, no reserved track).
2026-07-28 22:13:54 +00:00
Brooklyn Nicholson
6d83a79f93 fix(desktop): match Edit Models row hover to the model picker 2026-07-28 17:13:20 -05:00
brooklyn!
ed5fd3503d
fix(desktop): clear surface height vars from the attached surface, not a detached node (#73638)
When the status stack collapses (last subagent/background/queue item
finishes), React removes the stack div before the layout-effect cleanup
runs. Resolving the surface via closest('[data-chat-surface]') from that
now-detached node falls back to the document root, so the stale measured
height was cleared from the wrong element and stayed on the surface —
inflating --thread-last-message-clearance (the thread's bottom padding)
until the next publish. Same latent hazard in the composer's unmount
cleanup.

Capture the surface root while the node is still attached and clear from
it directly.
2026-07-28 17:07:22 -05:00
hermes-seaeye[bot]
66e20786d0
fmt(js): npm run fix on merge (#73666)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-07-28 21:55:38 +00:00
Teknium
03af12c5c9 style(desktop): blank line between node: and vitest import groups
perfectionist/sort-imports requires spacing between builtin and
external import groups.
2026-07-28 14:47:36 -07:00
Teknium
20e0b26c4e test(desktop): run venv-blocker-scan tests under vitest
CI's electron vitest project collects electron/**/*.test.ts; the
node:test-based suite reported 'No test suite found'. Import
describe/it from vitest like sibling electron tests.
2026-07-28 14:47:36 -07:00
Teknium
1dd1c449bb fix(desktop): run venv-blocker scan async off the main-process event loop
The preflight's execFileSync froze the Electron UI event loop for up to
15s while psutil scanned the full process table. Convert
scanVenvBlockers to async execFile and await it in applyUpdates; tests
updated to async DI stubs. Also adds trailing newlines.
2026-07-28 14:47:36 -07:00
atakan g
d210500b54 fix(desktop): surface external venv update blockers 2026-07-28 14:47:36 -07:00
Siddharth Balyan
3cedac00b7
fix(credits): reintroduce grant_spent, gated on an in-session crossing (#73634)
* Revert "fix(credits): remove the 'Grant spent · $X top-up left' notice"

This reverts commit 5dc6a14c14.

* fix(credits): reintroduce grant_spent behind an in-session crossing gate

The removed notice nagged because its condition is a steady state for
accounts living on top-up, the latch is per-session, and the cold-start seed
runs the policy at every session open — every session re-announced
'Grant spent · $X top-up left'. Reintroduce it gated so only a session that
WATCHES the grant run out announces:

- seen_grant_unspent crossing gate, mirroring seen_below_90: opens when the
  session observes the grant meaningfully unspent (>= GRANT_UNSPENT_MIN_MICROS,
  1 cent — portal-seeded states derive micros from float dollars and can carry
  sub-cent residue where headers report exactly spent). Seeds never prime it.
- The gate guards only the show branch and is consumed by the announcement —
  one announcement per crossing. Header flicker (used_fraction None and back)
  clears the sticky line but cannot re-announce; a renewal re-opens the gate.
- new_credits_latch() centralizes the latch shape (agent build, lazy re-init,
  and the policy test helper all build through it).
- Tests: steady-state-open stays silent (seed + policy seams), live crossing
  announces once, flicker/renewal/residual cases locked; the rendering test
  primes the gate and asserts its leg count so a gate regression cannot
  silently shrink coverage.
2026-07-28 21:36:23 +00:00
atakan g
fa9217542d fix(desktop): preserve local model provider endpoint 2026-07-28 14:17:27 -07:00
Brooklyn Nicholson
aa0465a309 fix(desktop): clear surface height vars from the attached surface, not a detached node
When the status stack collapses (last subagent/background/queue item
finishes), React removes the stack div before the layout-effect cleanup
runs. Resolving the surface via closest('[data-chat-surface]') from that
now-detached node falls back to the document root, so the stale measured
height was cleared from the wrong element and stayed on the surface —
inflating --thread-last-message-clearance (the thread's bottom padding)
until the next publish. Same latent hazard in the composer's unmount
cleanup.

Capture the surface root while the node is still attached and clear from
it directly.
2026-07-28 15:34:15 -05:00
Teknium
f5ff8fa4f8
Merge remote-tracking branch 'origin/main' into wake-toggle-config 2026-07-28 13:24:23 -07:00
brooklyn!
c8cdeb435f
fix(desktop): allow renderer camera capture (#73558)
* fix(desktop): allow camera capture through the permission handlers

The session permission hooks were written for the voice composer and denied
video outright, so any renderer getUserMedia({video}) failed with
NotAllowedError before the OS was ever consulted.

Rename isAudioCapturePermission to isMediaCapturePermission and accept video
alongside audio in both the request and check handlers. The OS capture
permission still applies, so the user keeps a real allow/deny.

* build(desktop): declare camera usage for signed macOS builds

A hardened-runtime build needs the camera entitlement and an
NSCameraUsageDescription string, or the packaged app is killed on first
camera access instead of prompting.
2026-07-28 14:50:52 -05:00
Teknium
0cf58de85e
Merge remote-tracking branch 'origin/main' into wake-toggle-config
# Conflicts:
#	tests/test_tui_gateway_server.py
#	tui_gateway/server.py
2026-07-28 12:37:35 -07:00
atakan g
89490ae373 fix(desktop): honor gitignore across Windows path casing 2026-07-28 12:36:35 -07:00
Atakan
15317e1fd7 fix(desktop): upload Windows attachments to Linux backends 2026-07-28 12:36:01 -07:00
atakan g
d4221b2736 fix(desktop): validate WebSocket token before backend ready 2026-07-28 12:35:57 -07:00
Teknium
eb8d88f33a
fix(desktop): await wake-word mic release before opening the voice-chat mic
'Clicked voice chat but it never starts listening' — a mic-device
contention race. Starting a voice conversation fires wake.pause (to free
the mic from the wake-word listener) and opens the conversation's own mic
in two separate effects, both keyed on voiceConversationActive with no
ordering between them. wake.pause was fire-and-forget, so getUserMedia
often raced the wake listener's stream teardown (which joins a reader
thread + closes the device in a finally). On Windows the capture device is
effectively single-owner, so opening it while wake still held it failed and
the conversation never started listening.

Wake-word-initiated starts don't hit this: the backend's _on_detect calls
pause_listening synchronously before emitting wake.detected, so the mic is
already free by the time the frontend opens it. Only the button/hotkey path
raced — matching the report.

Fix: pauseWakeForVoice now returns an awaitable barrier for the in-flight
wake.pause round-trip, and useVoiceConversation awaits a new beforeMicOpen
hook (wired to that barrier) right before handle.start(), re-checking
enabled/muted/busy/idle after the wait. No behavior change when the wake
word isn't running (barrier is null → no wait).

tsc + eslint clean, 39 desktop voice/wake tests green.
2026-07-28 12:16:54 -07:00
Teknium
f9d7be82fb
fix(desktop): speak the first reply of a wake-started voice session
Diagnosed from a Windows desktop's logs: the backend TTS was generating
and saving the reply mp3 every turn (provider: openai), so synthesis was
fine — the DESKTOP wasn't playing the first reply. Root cause is Chromium's
autoplay policy: audio (HTMLAudioElement.play() and AudioContext) is
suspended until the frame sees a user gesture. A voice conversation started
by the 'Hey Hermes' wake word has no preceding click, so the first reply's
play() was rejected with NotAllowedError (silently swallowed) and only
turn 2+ spoke. Manual voice-start worked because the button click WAS the
gesture — exactly the 'first message in a new voice session is silent,
manual start is fine' report.

Fix (defense in depth):
- chatWindowWebPreferences sets autoplayPolicy: 'no-user-gesture-required'
  on every chat window (primary + secondary), so a deliberately-launched
  native app never gates audio on a gesture. Primary fix.
- voice-playback.ts resumes a suspended AudioContext on stream 'start' and
  retries HTMLAudioElement.play() once via an unlock context after a
  NotAllowedError — covers the dashboard-embedded surface that doesn't get
  the Electron policy.

Tests: 16 electron session-window tests (incl. new autoplay assertion),
tsc + eslint clean.
2026-07-28 12:01:27 -07:00
Teknium
43e1b7f474 style: eslint padding fix in profile-scope test (salvage follow-up) 2026-07-28 11:58:03 -07:00
Teknium
e27997d63f fix(desktop): route streaming TTS through the active profile backend
The /api/audio/speak-stream WebSocket URL was always minted for the primary
backend with no profile param, so streaming read-aloud used the default
profile's TTS provider even when another profile was active. Mint the
ticket for the active profile's connection and carry ?profile= (same seam
as /api/pty), via a read-only getApiRequestProfile() accessor.
2026-07-28 11:58:03 -07:00
Tom Brautlacht
c95f04501d fix(desktop): skip markdown tables in speech output 2026-07-28 11:58:03 -07:00
HarpStar
f02d41cb21 fix(desktop): summarize code blocks for read aloud 2026-07-28 11:58:03 -07:00
webtecnica
7c398a5c8e fix(desktop): prevent voice playback from auto-starting next sentence after stop (#70955) 2026-07-28 11:58:03 -07:00
Max Hsu
181511473e fix(desktop): render video/audio image-markdown as media, not broken <img>
Generated media often arrives as image markdown (![clip](clip.mp4)). The chat
markdown renderer maps the img element to MarkdownImage, which renders a raw
<img>; for a video/audio source the browser cannot paint it and shows a
broken-image icon even though the file is valid and plays fine. Images worked
only because <img src=...png> is valid markup.

Route video/audio sources in MarkdownImage to the existing MediaAttachment,
which already picks the correct <video>/<audio> element (streaming protocol +
open-externally fallback) by media kind. The intended MEDIA:-tag -> #media:
link path is unaffected; this only fixes the bare image-markdown case.

Since #57944 the image path is built on hooks (async src resolution, failure
and loading states), so the routing check cannot be a plain early return
inside it -- it would have to sit after every hook call and would still fire
an image resolve for media never rendered as an image. MarkdownImage is
therefore a thin hookless router in front of MarkdownImageContent, which is
the previous body unchanged.

The regression tests join the existing markdown-text.media.test.tsx added by
#57944 rather than replacing it.

Fixes #40896

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018vU45SEvcxLkVeyyEE9gJ2
2026-07-28 11:58:03 -07:00
Ares4Tech
3e47efeb25 fix(desktop): stream remote media through gateway 2026-07-28 11:58:03 -07:00
Bryan Roller
3a1efa5e30 [verified] fix(desktop): scope TTS requests to active profile 2026-07-28 11:58:03 -07:00
Bartok9
a2b0c314ae fix(desktop): scope audio endpoints to the active profile
transcribeAudio/speakText/getElevenLabsVoices called the backend without
profileScoped(), so for a non-default profile they hit the default backend
config instead of the active profile - playback used the wrong TTS/voice
even though settings were saved to the active profile config.

Fixes #53441
2026-07-28 11:58:03 -07:00
Brooklyn Nicholson
eaf30a4de7 build(desktop): declare camera usage for signed macOS builds
A hardened-runtime build needs the camera entitlement and an
NSCameraUsageDescription string, or the packaged app is killed on first
camera access instead of prompting.
2026-07-28 13:40:44 -05:00
Brooklyn Nicholson
47cb4ea1fe fix(desktop): allow camera capture through the permission handlers
The session permission hooks were written for the voice composer and denied
video outright, so any renderer getUserMedia({video}) failed with
NotAllowedError before the OS was ever consulted.

Rename isAudioCapturePermission to isMediaCapturePermission and accept video
alongside audio in both the request and check handlers. The OS capture
permission still applies, so the user keeps a real allow/deny.
2026-07-28 13:40:44 -05:00
hermes-seaeye[bot]
9cf1227d4e
fmt(js): npm run fix on merge (#73555)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-07-28 18:38:33 +00:00