Commit graph

19083 commits

Author SHA1 Message Date
Carlos Diosdado
bc4dcb1b02 feat(tts): Gemini SSE + xAI WebSocket streaming providers, tts.streaming.provider knob, docs + E2E tests
Salvaged from PR #47588 and rebased onto the post-campaign streaming core:
the StreamingTTSProvider ABC/registry and the ElevenLabs/OpenAI streamers
already live on main (tools/tts_streaming.py), so this ports the pieces
main lacked:

- GeminiStreamer: streamGenerateContent?alt=sse -> base64 PCM chunks
  (24 kHz mono int16), reusing main's DEFAULT_GEMINI_TTS_* constants.
- XAIStreamer: WebSocket wss://api.x.ai/v1/tts -> binary PCM frames,
  async->sync bridged via the _collect_async test seam.
- tts.streaming.provider config knob: pin one streamer, or 'auto' to
  walk the priority list elevenlabs -> gemini -> openai -> xai. Unset
  keeps the never-swap-the-user's-voice default.
- docs/streaming-tts.md: architecture, capability matrix, how to add
  a provider.
- Unit tests for the knob, SSE parsing, and the WS bridge; key-gated
  E2E tests (skipped without credentials).

Refs: #47588
2026-07-28 22:31:40 -07:00
Carl Taylor
3a4aa2f8e6 feat(gateway): streaming TTS adapter contract and consumer (#60671)
Add an opt-in streaming-audio adapter seam to BasePlatformAdapter so
voice-capable gateway platforms (LiveKit, Discord voice, future adapters)
can consume LLM output as streaming PCM audio before the full response
completes, dropping perceived voice latency from ~2-3.5s to ~500-800ms.

Adapter contract (gateway/platforms/base.py):
- AudioFormat dataclass: declared sample_rate, channels, sample_width
- StreamingTTSHandle: opaque handle with audible/aborted flags
- supports_streaming_tts / begin_streaming_tts / write_streaming_tts
  / finish_streaming_tts / abort_streaming_tts
- All default to unsupported/no-op so existing adapters are source-compatible
- Per-turn _streaming_tts_completed_chats set suppresses duplicate whole-file
  auto-TTS when streaming succeeded; cleared after turn completion

Gateway consumer (gateway/streaming_tts_consumer.py):
- StreamingTTSConsumer: bridges sync agent deltas to async adapter audio sink
- Uses existing SentenceChunker (no competing parser)
- Thread-safe bounded queue; on_delta never blocks the agent worker thread
- Resolves configured streaming provider via resolve_streaming_provider()
- Serialises clause playback in order; flushes tail on completion
- Pre-audio failure: completed=False (falls back to whole-file TTS)
- Post-audio failure: completed=True, partial=True (no replay from start)
- Abort is idempotent; late chunks silently dropped
- Per-turn state isolated across concurrent chats

Gateway integration (gateway/run.py):
- message_type parameter threaded through _run_agent -> _run_agent_inner
- StreamingTTSConsumer created when voice input + auto-TTS + provider active
- Delta callback teed to both text stream consumer and TTS consumer
- TTS-only delta callback installed when text streaming is off
- finish() called from executor; wait_complete() in async context after
- Barge-in aborts the consumer at all three interrupt detection points
- Runner-level _send_voice_reply suppressed when streaming TTS completed

Tests (tests/gateway/test_streaming_tts_consumer.py):
- 15 focused tests: adapter defaults, lifecycle, ordered chunks,
  unsupported/No-streamer fallback, abort idempotency, late-chunk drop,
  pre/post-audio failure, concurrent-turn isolation, think-block suppression,
  queue backpressure

Does not touch desktop/TUI code or add config flags. Plugin TTS provider
stream() metadata gap (#47896) is explicitly out of scope — built-in
ElevenLabs/OpenAI PCM streamers are the first consumers.

Refs: #60671, #47896
2026-07-28 22:31:40 -07:00
kshitijk4poor
02eff547a8 chore: add AUTHOR_MAP entry for kingrubic (nnqbao@gmail.com) 2026-07-29 10:56:19 +05:30
kshitijk4poor
b8cdb698d1 fix: extend pool-credential resolution to Bedrock API-key flow
Sibling fix for #65977 — _model_flow_bedrock_api_key used only
get_env_value for AWS_BEARER_TOKEN_BEDROCK, missing pool-backed
keys. Now uses _resolve_api_key_provider_secret like the other
flows.
2026-07-29 10:56:19 +05:30
Bao
c4d9fbacb3 fix(cli): honor pooled credentials in model wizard
Signed-off-by: Bao <nnqbao@gmail.com>
2026-07-29 10:56:19 +05:30
Teknium
5940026245 test(photon): drain detached fatal-notification task in zombie watchdog test
The lifecycle cluster made fatal notifications detached; the watchdog test
still asserted synchronously.
2026-07-28 22:22:42 -07:00
Teknium
f2364f1f81 test(photon): copy sidecar helper modules into the spectrum-patch fixture
index.mjs now imports sibling .mjs helpers (send-format, stream-staleness);
the fixture copied only index.mjs so the sidecar died on module resolution
before reaching the health endpoint.
2026-07-28 22:22:42 -07:00
Teknium
dcace573da fix(photon): rework zombie-stream watchdog for spectrum-ts 8 with strict probe semantics
Maintainer rework of #45580 (issue #54036) on top of the contributor's
cherry-pick, which targeted spectrum-ts 3.1.0 while main pins 8.0.0:

Sidecar (primary detection, new):
- stream-staleness.mjs: pure decision rules, executable under node.
  * classifyProbeRejection: only a not-found-shaped rejection of the
    synthetic-id read counts as a completed round-trip (ALIVE); any other
    rejection is INCONCLUSIVE — never alive. The original /probe treated
    ANY rejection as alive, which was too loose.
  * shouldProbe: probe only after 10+ min of stream silence (configurable
    via PHOTON_STREAM_SILENCE_PROBE_MS; <=0 disables) with a cooldown.
  * isZombieSuspect: zombie only on silence past threshold AND a
    probe-proven live channel. Silence alone NEVER degrades (shared lines
    can be quiet for hours); inconclusive probes NEVER degrade (network
    may be down — the iterator will throw and the re-subscribe loop
    recovers on its own).
- index.mjs: track last inbound-iterator yield (noteInboundYield), run a
  30s watchdog tick, and on a confirmed zombie feed markStreamDegraded ->
  the existing exit-75 restart path. /healthz gains a stream.staleness
  block (silentForMs, threshold, lastProbeOutcome, zombieSuspected).
  /probe reworked to strict semantics: 200 only on a proven round-trip,
  503 with outcome hung|inconclusive otherwise.

Adapter (second layer, reworked):
- _probe_once returns tri-state alive|hung|inconclusive; only a hung
  sidecar HTTP call counts toward the respawn counter — inconclusive
  resets nothing and triggers nothing.
- default probe_interval_seconds 60 -> 600 (conservative; avoid restart
  storms on quiet lines).
- _monitor_sidecar_health surfaces zombieSuspected from /healthz as a
  warning; the fatal UPSTREAM_STREAM_DEGRADED path is unchanged and fires
  when the sidecar escalates.

Tests: test_zombie_stream_watchdog.py executes the real node decision
module and drives the adapter against mocked /healthz responses;
test_presence_watchdog.py updated for the tri-state probe.

Also adds contributor mappings for nickkarhan (#53283) and vaibhavjnf
(#45580).
2026-07-28 22:22:42 -07:00
Teknium
87fe75fde4 test(photon): replace source-grep URL-routing tests with behavior tests
Follow-up to the URL markdown fix: extract the /send builder decision into
sidecar/send-format.mjs and rewrite test_url_send_path.py to execute the real
module under node (format+text in -> chosen builder out) instead of regex-
grepping index.mjs source, which is a banned test pattern in this repo.
2026-07-28 22:22:42 -07:00
vaibhavjnf
709dd3282f fix(photon): recover inbound after half-open ("zombie") gRPC stream
spectrum-ts's live-stream consumer (consumeLive in spectrum-ts 3.x) only
reconnects when its inbound async iterator throws or ends. A half-open
("zombie") gRPC socket — where the TCP connection stays ESTABLISHED but the
peer is gone (NAT idle-timeout, network blip, laptop sleep) — makes the
iterator hang forever: no error, no end. The SDK exposes no gRPC keepalive
knob (createClient takes only {address, tls, token}; grpc.keepalive_time_ms
defaults to -1 = pings off), so the inbound stream silently dies and stays
dead until the gateway is restarted. Symptom: the agent's iMessage line goes
"online but deaf" — Photon's cloud-side fallback answers users with "the agent
isn't online right now" and inbound never reaches the gateway.

Fix, entirely in the code we own (no SDK fork):

- Sidecar gains a POST /probe endpoint that drives a cheap unary read
  (space.getMessage on a synthetic id) over the SAME gRPC channel the inbound
  stream uses. A live channel round-trips in ms (server returns not-found,
  which is success for liveness); a zombie hangs. It sends nothing to any user
  and creates no chat (space.get is local in shared/dedicated mode; only the
  message read touches the wire).

- The adapter runs a presence watchdog: it probes on an interval, skips the
  probe when natural inbound traffic already proved liveness within the
  window, and after N consecutive failed probes respawns the sidecar — a fresh
  Spectrum() re-subscribes the stream and re-registers presence. Successful
  probes double as application-level keepalive, helping prevent the zombie
  from forming at all. Respawn is lock-guarded against double-spawn and the
  watchdog is torn down cleanly on disconnect.

Behavioural settings live in config.yaml (extra), bridged to env per the
.env-is-secrets-only convention:
  probe_interval_seconds (60), probe_timeout_seconds (10),
  probe_max_failures (3). A non-positive interval disables the watchdog.

Tests: tests/plugins/platforms/photon/test_presence_watchdog.py covers config
resolution, the disable switch, probe alive/dead(500)/timeout/no-client, the
core N-failures->one-respawn detection, success-resets-failures, stop-then-
start respawn ordering, and lock-guarding — all without spawning Node or
hitting the network.

Contributed by Vaibhav Sharma (X: @vabbyshabby).
2026-07-28 22:22:42 -07:00
Nick K
2f4462fcaa fix(photon): send markdown messages with URLs as text 2026-07-28 22:22:42 -07:00
Teknium
402286fcff fix(photon): tolerate SDK builds without the iMessage effect surface
imessage.effect.message crashed the sidecar at import against SDK stubs/
builds lacking the effect surface (caught by the patch-failure health test).
Optional-chain with {} fallback; /send-effect rejects cleanly instead.
2026-07-28 22:10:57 -07:00
huntsyea
cf550c0863 feat(photon): support rich link previews 2026-07-28 22:10:57 -07:00
Teknium
324f310263 refactor(photon): reuse the /send-poll primitive from #43665 for poll clarify
Follow-up to the #48194 pick: it was written before #43665 landed and
re-added its own /send-poll sidecar route and poll import. Collapse the
duplicates:

- keep #43665's /send-poll route (>=2 trimmed string options) as the
  single sidecar implementation; drop #48194's variant
- drop the duplicated poll import in the sidecar destructure
- make adapter.send_poll() a thin wrapper over _sidecar_send_poll(), the
  one /send-poll client (shared with the poll-backed clarify path), and
  align its validation to the sidecar's >=2-options contract
2026-07-28 22:10:57 -07:00
vaibhavjnf
fe95194c59 feat(photon): render multiple-choice clarify as a native iMessage poll
The `clarify` tool's multiple-choice prompts flattened to a numbered text
list on Photon/iMessage, even though iMessage has a native poll bubble and
spectrum-ts already exposes it via the `poll()` content builder. Two gaps
caused the flattening:

  * Outbound: the sidecar only had `/send` (text); there was no way to send
    a poll, so the base adapter's numbered-text fallback was used.
  * Inbound: `normalizeContent()` handled only text/attachment/voice, so a
    poll vote (`poll_option`) was dropped on the floor ("[Photon content
    type not handled: poll_option]") and never resolved the clarify.

Fix, end to end:

  * Sidecar: import `poll` from spectrum-ts; add a `/send-poll` route
    (`space.send(poll(title, ...options))`); serialize inbound `poll_option`
    (the vote: chosen title + selected bool) and `poll` content in
    `normalizeContent()`.
  * Adapter: override `send_clarify` — for choices, send a native poll via
    `_sidecar_send_poll` and call `mark_awaiting_text` so the gateway's
    existing pending-clarify text-intercept resolves the answer; open-ended
    clarifies keep the plain-text path. Inbound `poll_option` selections are
    dispatched as a plain-text MessageEvent carrying the chosen option
    (deselections / empty votes are dropped). If the poll send fails (an
    older sidecar without `/send-poll`, or a send error) it falls back to the
    numbered-text clarify, so nothing regresses on a half-upgraded restart.

No new model tool, no new env var, no core change — the capability lives at
the platform edge. The poll vote reuses the existing clarify text-intercept
resolution path, so no new gateway resolution mechanism is introduced.

Tests: tests/plugins/platforms/photon/test_poll_clarify.py — inbound vote ->
choice text, deselection/empty-vote dropped, send_clarify sends a poll +
enables text-capture, open-ended stays text, and poll-failure falls back to
the text list. Full photon suite green.

Contributed by Vaibhav Sharma (X: @vabbyshabby).
2026-07-28 22:10:57 -07:00
Hermes Agent
06150f70af feat(photon): add native message effects 2026-07-28 22:10:57 -07:00
Hermes Agent
077c583c75 feat(photon): add native poll sending 2026-07-28 22:10:57 -07:00
Teknium
95d303138b test(photon): fix PlatformConfig kwargs in target_not_allowed standalone test 2026-07-28 21:45:52 -07:00
Teknium
ed02df6ddc chore: add contributor mappings for photon lifecycle salvage train 2026-07-28 21:45:52 -07:00
Teknium
e68f3fd825 fix(photon): harden structured sidecar error classes + target_not_allowed
Maintainer follow-up to the #51193 salvage:

- _send_with_retry: permanent classes (auth_or_config, target_not_allowed)
  now short-circuit BEFORE the unconditional plain-text fallback resend,
  including when a retry attempt surfaces one — no more double-sends of
  permanently-failing requests.
- sidecar classifySidecarError: new structured code target_not_allowed for
  Spectrum's 'Target not allowed for this project' AuthenticationError
  (shared/free-tier lines cannot initiate outbound sends to new targets).
  Classification applies to every handler sharing the catch-all
  serverError path (/send, /send-attachment, /react, /typing, ...).
- _standalone_send now parses the structured error body too (it reads
  sidecar responses independently of _sidecar_call) and returns
  error_class/retryable alongside the message.
- target_not_allowed maps to a canonical user-facing message in both
  paths; raw upstream error text never leaks through the structured code.

Closes the actionable halves of #50971, #51897, #52794.
2026-07-28 21:45:52 -07:00
SeoYeonKim
91c4c6f9d2 Preserve Photon sidecar retry semantics
Photon's Node sidecar intentionally hides raw handler exceptions, but the Python adapter still needs a safe failure class and retryability bit so delivery retries do not collapse into an opaque generic 500.

Constraint: Sidecar responses must not leak raw stack traces or private exception text

Rejected: Retry every internal sidecar error | masks permanent auth/config failures

Confidence: high

Scope-risk: narrow

Directive: Keep sidecar error text generic; extend safe error classes instead of exposing raw SDK failures

Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon/test_overflow_recovery.py -q

Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon -q

Tested: uv run ruff check plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py

Tested: python3 -m py_compile plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py

Tested: node --check plugins/platforms/photon/sidecar/index.mjs

Tested: git diff --check

Tested: python3 scripts/check-windows-footguns.py --diff origin/main

Not-tested: Live Photon/Spectrum delivery against a real iMessage account

Related: #50971
2026-07-28 21:45:52 -07:00
Frowtek
c0ff5e9169 fix(photon): run the Spectrum patch spawn off the gateway event loop
`PhotonAdapter._start_sidecar` is `async`, but it ran the Spectrum
mixed-attachment patch script with a bare `subprocess.run(...)`: it spawns
node and *waits* for it, with `timeout=10`. Executed inline that holds the
shared gateway event loop for the whole window, so no other platform's
messages, heartbeats, or sessions are serviced until it returns.

The same function already establishes this exact invariant twenty lines
above, where the stale-dependency reinstall hops to a worker thread:

    # Runs off the event loop so a cold install can't freeze every other
    # platform's traffic.
    if _sidecar_deps_stale():
        await asyncio.to_thread(_reinstall_sidecar_deps)

The patch spawn never got the same treatment. It is not startup-only
either — `_start_sidecar` is called from `connect()`, which takes
`is_reconnect`, so an ordinary Photon reconnect (network blip, sidecar
death) re-runs it and stalls a live gateway that is actively serving
Discord/Telegram/Slack traffic.

Dispatch it via `asyncio.to_thread` like its sibling. Same off-the-loop
class as the inbound-image decision (#66688) and the cron-fire verifier.

Adds a regression test asserting the spawn executes on a worker thread
rather than the loop thread.
2026-07-28 21:45:52 -07:00
Frowtek
a90a2b3c3c fix(photon): inspect port listeners off the gateway event loop
`_reap_stale_sidecar` is `async`, but it identified the processes holding
the sidecar port with two blocking helpers called inline:

* `_find_listener_pids` -> `subprocess.run(["lsof", ...], timeout=5.0)`
* `_pid_is_sidecar` -> `subprocess.run(["ps", ...], timeout=5.0)`, once
  per candidate pid

so the inspection can hold the shared gateway loop for 5 + 5·N seconds
while nothing else on it is serviced. It only runs once the /healthz probe
finds something already listening — the orphaned-sidecar recovery path —
and `_reap_stale_sidecar` is awaited from `_start_sidecar`, which runs on
every reconnect (`connect(is_reconnect=True)`). The stall therefore lands
on a live gateway that is still serving every other platform, right when a
crashed sidecar has already left an orphan behind.

Move the whole inspection to one `asyncio.to_thread` hop (one hop rather
than N+1 round trips). The reaping semantics are untouched: SIGTERM for
verified orphans, SIGKILL escalation, and both foreign-listener
RuntimeErrors behave exactly as before.

Same off-the-loop class as the inbound-image decision (#66688) and the
cron-fire verifier.

Adds a regression test asserting both the lsof lookup and the per-pid ps
check execute on a worker thread rather than the loop thread.
2026-07-28 21:45:52 -07:00
Teknium
1b7db6e037 fix(photon): guard inbound-task cancel against self-cancellation in disconnect()
Follow-up widening of the #73170 pattern: apply the same
asyncio.current_task() guard used for the health task (and now the
supervisor task in _stop_sidecar) to the inbound-task cancel path in
disconnect(), so an inbound task that triggers disconnect() cannot
cancel-and-await itself.
2026-07-28 21:45:52 -07:00
ygd58
74939d2bfe fix(photon): stop sidecar-crash fatal handler from cancelling its own supervisor task
Fixes #73159.

When the Photon sidecar exits unexpectedly, _supervise_sidecar()
(running as self._sidecar_supervisor_task) correctly detects
SIDECAR_CRASHED and calls self._notify_fatal_error(). The Gateway's
fatal-error handler answers that by calling adapter.disconnect(),
which calls _stop_sidecar() -- from INSIDE the very task that's
currently executing this whole chain.

_stop_sidecar()'s cleanup unconditionally cancelled
self._sidecar_supervisor_task. Cancelling the currently-running task
raises CancelledError at its own next await point (inside
_notify_fatal_error() or _stop_sidecar() itself). Since
asyncio.CancelledError inherits from BaseException (not Exception),
the Gateway's `except Exception` guards around the fatal-error handler
don't catch it -- the handler aborts before ever reaching the
"queue platform for background reconnection" step. Photon then stays
permanently in `retrying` state until the whole process is manually
restarted, even though detection worked correctly and the underlying
transient upstream outage had long since recovered.

Fix: in _stop_sidecar()'s cleanup, check whether
self._sidecar_supervisor_task is asyncio.current_task() before
cancelling it. A task cannot legally cancel itself in any useful way
anyway (the cancellation only takes effect at its own next await,
which is exactly the corruption described above) -- when we're inside
the supervisor's own call stack, it's already in the process of
finishing on its own once _notify_fatal_error() returns, so skip the
cancel and just clear the reference.

This mirrors an existing precedent in the same file: disconnect()
already guards its OTHER task-cancellation (self._sidecar_health_task)
the same way (`if task is not asyncio.current_task()`), just not the
supervisor task in _stop_sidecar().

Per the issue's own note that existing tests mock out
_notify_fatal_error() entirely (so this integration chain was never
exercised), added two tests that drive the REAL chain: one runs
_supervise_sidecar() as an actual asyncio task with a real
_notify_fatal_error() that calls the real disconnect() -> _stop_sidecar(),
confirming the task completes without CancelledError and that the
post-disconnect reconnect-queue step actually executes; a second
confirms the OTHER call path (external cleanup, a different task)
still correctly cancels a running supervisor exactly as before.
Reverting only the adapter.py fix (keeping the new test) reproduces
the exact CancelledError from the bug report, confirming this is a
genuine regression test.

2 new tests pass; 113/113 in the full tests/plugins/platforms/photon/
directory (no regression to sidecar lifecycle, overflow recovery, or
health-monitoring behavior).
2026-07-28 21:45:52 -07:00
Spark (DGX)
3c7cff843d fix(photon): dispatch fatal-error notification from a detached task
Follow-up to #69112. That PR hardened the shared gateway dispatch path in
gateway/run.py against the *caller* being cancelled. A second, self-referential
cancellation specific to PhotonAdapter sits one layer underneath it and survived
that fix.

PhotonAdapter is the only platform adapter that awaits _notify_fatal_error()
inline, on the same task that detected the fault. Both _monitor_sidecar_health
and _supervise_sidecar run as self._sidecar_health_task /
self._sidecar_supervisor_task, and the notification routes into
GatewayRunner._handle_adapter_fatal_error_impl, which tears the adapter down via
_safe_adapter_disconnect -> disconnect(). disconnect() then cancels
self._sidecar_health_task and awaits it -- which, when the health task is what
raised the notification, means disconnect() cancels its own caller several
plain-await frames up.

disconnect()'s `task is not asyncio.current_task()` guard does not catch this.
The current task where that guard evaluates is the wrapper
_await_adapter_cleanup_with_timeout creates around disconnect() via
asyncio.ensure_future, not the health task further up the chain, so the guard
passes and the cancel lands.

CancelledError stopped subclassing Exception in Python 3.8, so the
`except Exception` that wrapped the inline notify call never saw it. The health
task died silently mid-handoff: no log line, no "exception never retrieved"
warning (cancellation is normal asyncio), and no retry. The platform stayed
stranded until the gateway was restarted by hand.

Fix: dispatch the notification onto a new task, the same pattern
DiscordAdapter._handle_bot_task_done already uses for this reason. disconnect()
can then cancel the health/supervisor task freely without that cancellation
reaching the code still running the handoff, so the handoff always reaches the
reconnect queue. Both Photon fatal call sites are converted: the health-poll
path (observed wedging) and the sidecar-crash path (same shape, not yet
observed). gateway/run.py is untouched.

Observed twice on a self-hosted gateway, ~4h38m and ~52min of silent inbound
outage, both cleared only by a manual restart, both post-dating #69112's merge.
In each case the fatal log line appears and `queued for background reconnection`
never does.

Tests: new tests/plugins/platforms/photon/test_fatal_notify_self_cancel.py
covers the self-cancellation (fails with CancelledError without this change),
that the dispatch does not block its caller, that a failing notification warns
rather than raising, and a source guard against reintroducing either inline
await. Two assertions in test_overflow_recovery.py that drove these coroutines
directly now drain pending tasks before asserting delivery, since the
notification is deliberately no longer awaited inline.

Prepared with agent assistance and reviewed before submission.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 21:45:52 -07:00
yoma
ef61fd7ade fix(photon): keep sidecar alive if spectrum patch fails 2026-07-28 21:45:52 -07:00
Nick Edson
201be3e546 fix(gateway): reserve retrying Photon listener ownership 2026-07-28 21:45:52 -07:00
Nick Edson
c608a6937a fix(gateway): release failed Photon listener claims 2026-07-28 21:45:52 -07:00
Nick Edson
a908c62d28 fix(gateway): guard Photon sidecar listener collisions 2026-07-28 21:45:52 -07:00
Nick Edson
cf19ac8ff3 fix(gateway): prevent duplicate Photon sidecar storms 2026-07-28 21:45:52 -07:00
Teknium
88ff722f94 docs(api-server): document profile-bound HTTP auth from #72285
The multiplexed listener now rejects the default API_SERVER_KEY on
/p/<profile>/ prefixes (fail-closed per-profile keys). Add the
multi-profile routing section with an explicit breaking-change callout
for the next release notes.
2026-07-28 21:45:44 -07:00
Teknium
41233e19c6 fix(gateway): forward failure_reason through the empty-response return path
Sibling of #64686: _run_agent's empty-final_response early return dropped
failure_reason (and #64686 only fixed the non-empty path), so downstream
consumers (TUI billing surface, transient-failure persistence) lost the
structured reason exactly when a failed run produced no text.

Also hardens the two BasePlatformAdapter identity checks (edit_message /
delete_message) with getattr so duck-typed adapters without the attribute
mean 'capability absent', not AttributeError — this was crashing the
send_progress_messages path for minimal adapters and test fakes.
2026-07-28 21:45:44 -07:00
Teknium
d4ff566232 fix(model-set): persist base_url/api_key on auxiliary slot assignments
Sibling of #65254 (main-slot endpoint preservation): the auxiliary scope of
POST /api/model/set dropped the request's base_url/api_key on the floor, so
an aux slot pinned to a custom/local endpoint silently depended on
model.base_url — and broke the moment the main slot switched away and
cleared it. The aux resolver already reads auxiliary.<task>.base_url/api_key
(_resolve_task_provider_model); this persists them.

Desktop side: setAuxiliaryToMain / applyAuxiliaryDraft now carry the
user-defined provider's api_url as base_url, mirroring applyMainModel.
2026-07-28 21:45:44 -07:00
Teknium
07e931fcb4 feat(buzz): WebSocket inbound transport — NIP-42 auth, live DM discovery, poll fallback
Consolidates the native-transport half of PR #73636 by @ScaleLeanChris
onto the merged adapter: persistent NIP-42-authenticated Nostr WebSocket
subscription as the default inbound path (transport=auto|websocket|poll),
kind-44100 membership events for live DM discovery, since-timestamp
resume on reconnect with bounded exponential backoff, and automatic
fallback to CLI polling when the WS can't be established. Events route
through the same _handle_event() pipeline as the poll loop, so de-dupe,
mention gating, p-tag DM latching, and allow-lists behave identically on
both transports. Outbound stays on the CLI (one-shot sends never race a
WS auth handshake — his design).

E2E verified against a real in-process websockets relay: NIP-42
challenge -> signed kind-22242 AUTH (event id re-derived server-side) ->
REQ subscription -> EVENT dispatch -> clean disconnect.

Co-authored-by: ScaleLeanChris <chris@scalelean.com>
2026-07-28 21:45:34 -07:00
ScaleLeanChris
21c7b806a3 feat(buzz): dependency-free Nostr signing module (NIP-42 auth, BIP-340)
Extracted verbatim from PR #73636 by @ScaleLeanChris — nsec/hex key
decoding, secp256k1 point math, BIP-340 Schnorr signing, and NIP-42
AUTH event construction with optional NIP-OA owner attestation tag.
2026-07-28 21:45:34 -07:00
Teknium
ceaa7880ee fix(photon): un-shadow the U+FFFC deferred-wait handler; fix stale sidecar-deps fixture
Two independent cross-PR collisions red on main (slice 8/8):

1. fd4f756492 (salvaged from stale #54514) added an early 'drop U+FFFC
   placeholder' return at the top of _dispatch_inbound — written before
   the deferred-wait handler (6b91b50c6e/afab7ed46e) existed further
   down the same function. The early return shadowed it: _pending_fffc
   never populated, no attachment-timeout tracking, 4 tests red. Remove
   the duplicate block; the deferred handler already drops the
   placeholder AND tracks/cancels/warns.

2. 9cf2046081 tightened sidecar_deps_installed() to require
   node_modules/spectrum-ts, but test_runtime_record's _patch_spawn
   fixture still created only bare node_modules/ — 2 tests red. Mirror
   a real completed install.

tests/plugins/platforms/photon: 164/164 after; 158/164 before.
2026-07-28 21:28:25 -07:00
Teknium
0f64557c06 fix(wake): coerce dead onnx->tflite on macOS ARM64; clear stale voice turn-timeout
Two follow-ups from the voice PR (#70509).

1. macOS ARM64 onnx migration. Existing users who pinned
   openwakeword.inference_framework=onnx before the tflite fix landed kept a
   wake word that arms but never fires (ONNX's embedding model is broken on
   Apple Silicon, upstream #336). New resolve_inference_framework() honors an
   explicit framework everywhere ONNX actually works, but coerces the one
   provably-dead combination (explicit onnx + macOS ARM64) to tflite with a
   one-time warning. No config mutation; empty still falls back to the platform
   default. Both read sites (engine init + requirements check) route through
   the shared resolver.

2. Voice turn-timeout leak. Each listen cycle reassigned turnTimeoutRef
   without clearing the prior 60s timer, so a stale timer from an earlier
   cycle could fire handleTurn() mid-way through a later listen — after enough
   idle re-listens this wedged the loop into a non-re-arming state (the
   'voice chat deactivates after ~a minute' report). Clear before re-arm.

Tests: 64 wake tests (added onnx-coercion / intel-kept / tflite-kept /
empty-default cases; updated the stale 'explicit onnx kept on ARM64' test
that encoded the old broken behavior), 39 desktop voice/wake vitest, tsc +
eslint clean.
2026-07-28 19:40:25 -07:00
brooklyn!
533d633ab9
Merge pull request #73774 from NousResearch/bb/desktop-messaging-icon-churn
perf(desktop): memoize PlatformAvatar + StatusDot (messaging icon churn)
2026-07-28 21:16:19 -05:00
hermes-seaeye[bot]
8a342a0002
fmt(js): npm run fix on merge (#73780)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-07-29 02:10:16 +00:00
brooklyn!
40a53ca031
Merge pull request #73698 from NousResearch/bb/desktop-overlay-perf
perf(desktop): kill sidebar + overlay render churn from hot store subscriptions
2026-07-28 21:01:26 -05:00
Brooklyn Nicholson
d76c7f5409 perf(desktop): memoize PlatformAvatar + StatusDot to kill messaging churn
The sidebar's messaging section renders one PlatformAvatar (labelIcon) and
StatusDot per platform group. The sidebar re-renders on every streaming tick
($sessions/$workingSessionIds/$messagingSessions churn), and both leaves were
unmemoized — so every platform's avatar + dot re-rendered on each delta even
though their props (platformId/tone/className) never changed.

- PlatformAvatar: memo(forwardRef(...)) — pure fn of platformId/name/class/style
- StatusDot: memo() — pure fn of tone/class

Measured (Messaging open, settled, 2 sessions streaming, 3s window):
  before: 96 wasted (PlatformAvatar 32, StatusDot 32, + brand icons)
  after:  0 wasted

Both are shared primitives; the memo also helps every other consumer
(session rows, gateway menu, session tiles) that renders them under a hot
parent.
2026-07-28 20:51:04 -05:00
hermes-seaeye[bot]
ada389004f
fmt(js): npm run fix on merge (#73770)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-07-29 01:49:23 +00:00
brooklyn!
f088fa5070
Merge pull request #73764 from NousResearch/bb/message-age-fallback
fix(desktop): stop message age falling back to the 1970 epoch ("20663d ago")
2026-07-28 20:40:18 -05:00
Brooklyn Nicholson
50d0de0358 fix(desktop): satisfy lint — import order, unused import, exhaustive-deps
- command-center/command-palette/skills: import ordering + drop unused
  HermesGateway type
- settings: wrap openSubView/openProviderView/openKeysView in useCallback so
  the navGroups memo deps are honest and stable
- command-center: add setSection to navGroups memo deps
2026-07-28 20:38:04 -05:00
Brooklyn Nicholson
64bdbd7f3c fix(desktop): stop message age falling back to the 1970 epoch ("20663d ago") 2026-07-28 20:33:25 -05:00
Brooklyn Nicholson
4acc162712 chore: overlay churn measurement probes (sweep + median A/B) 2026-07-28 20:26:58 -05:00
Brooklyn Nicholson
6fc01f8ab8 perf(desktop): memoize sidebar rows + status dots + artifact cells
The sidebar stays mounted beneath every overlay/page, and it subscribes to
$sessions + $workingSessionIds — both tick on every streaming token. An
unmemoized SidebarSessionRow re-rendered the whole list (Codicon, labels,
status dots) on each delta, and that churn bled into every overlay opened on
top: Cron, Profiles, Agents, Starmap, Webhooks, Command Center, Settings.

- SidebarSessionRow: memo() with a custom comparator that ignores the pure
  id-forwarding callbacks (fresh closures by design) and compares only the
  data that changes what the row paints. Rows bail out while siblings stream.
- SessionStatusDot: the 5 $...SessionIds arrays now read via useStoreSelector
  returning this session's boolean, so a dot repaints only when ITS OWN
  membership flips, not on every array tick.
- Artifacts: stable cellCtx (useMemo) + memoized Primary/Location/Session
  cells so a link-title fetch on one row stops re-rendering the whole table.

Measured before/after (2s idle, sessions streaming), sidebar-fed overlays:
  Cron 407->~30 wasted, Profiles 732->~80, Agents 132->9, Starmap 188->56,
  Webhooks 154->22.
2026-07-28 20:23:10 -05:00
Teknium
76173ba8cd lint: suppress windows-footgun false positive on POSIX-gated os.kill probe 2026-07-28 18:21:01 -07:00
Teknium
f041c95b7f fix(send_message): pass photon DM chat GUIDs through as explicit targets
'photon:any;-;+1555...' targets matched no parser pattern, so
_handle_send bounced them off the channel directory and failed
resolution even though the adapter accepts the GUID verbatim (the
react handler already passed them through). Recognize the DM chat
GUID shape (mirrors the adapter's _DM_CHAT_GUID_RE) in
_parse_target_ref for photon only.
2026-07-28 18:21:01 -07:00