Streamed responses no longer insert real newlines at terminal width —
logical lines are emitted whole and the terminal soft-wraps them, so
highlight-copy rejoins the full line (emulators only keep linebreaks
the app actually printed). This is the CLI equivalent of the TUI's
selection copy, which reads logical source lines from its screen
buffer. TTFT perception is preserved by mirroring the unfinished
line's tail into the spinner status text instead of chunk-printing.
/copy now prefers OSC 52 when running over SSH (SSH_CONNECTION /
SSH_TTY / SSH_CLIENT) — native tools there write the REMOTE clipboard,
which is never what the user wants. The CLI's OSC 52 writer also gains
tmux/screen DCS passthrough wrapping, mirroring the TUI's
wrapForMultiplexer. Fixes#31528 for the CLI surface.
Sabotage-verified: restoring the old chunk emitter fails 3 of the new
tests (hard-wrap detection, spinner mirror, unbreakable-run split).
Premise check on live main: barge-in machinery EXISTS for the per-turn
STREAMING pipeline only — cli.py chat() arms _voice_barge_in_monitor and
tui_gateway _tts_stream_begin arms _tts_stream_barge_in_monitor. What was
actually broken for spoken interruptions:
1. CLI whole-file fallback (_voice_speak_response_async — used whenever
streaming TTS cannot start: sounddevice missing, requirement probe
fails): NO monitor was ever armed, so talking over the reply did
nothing. Now arms _voice_barge_in_monitor in continuous voice mode.
2. Gateway fallback speak (tts_queue None → speak_text thread) and the
voice.tts RPC (desktop-triggered speech): speak_text ran bare, and
its internal streaming dispatch created a PRIVATE stop event nothing
could reach — uninterruptible even by stop_playback(). New
_speak_text_with_barge() runs the same barge monitor beside the speak
thread; hermes_cli.voice.speak_text/_speak_text_streaming accept an
external stop_event so a barge cuts the streaming pipeline too.
Stop-phrase handling and voice.transcript submission are inherited
from the shared monitor (merged #73933 behavior preserved).
3. False barge during TTS (the reason interruption "worked" then
self-cancelled or fired randomly): salvaged PR #71083 by @beardedeagle
(previous commit, kept authorship) — rolling-window VAD floor, 8x
multiplier, 4000-RMS trigger ceiling, barge_in_grace_seconds (2s)
before the mic opens, min-floor clamp. barge_in_grace_seconds is now
documented in DEFAULT_CONFIG.
Desktop spoken barge (renderer mic via voice-barge-in.ts) already covers
both its live-stream and fallback speech paths — verified, no change.
Long thinking/tool stretches in a voice conversation are dead air — the
user cannot tell whether the agent is alive. New: quiet, repeating soft
bubble blips while the agent works and no speech audio is flowing.
- tools/voice_mode.py: numpy-synthesized blips (no binary assets) — two
alternating low pitches (G4/E4) with pitch glide + smooth attack/decay
envelopes, ~0.8-1.2s randomized spacing, volume = voice.beep_volume * 0.5.
start_thinking_sound(should_play=...) / stop_thinking_sound() daemon-loop
lifecycle; macOS-TCC-safe (sounddevice output gated there → silent skip,
no per-second afplay churn). New mark_audio_output_active()/
is_audio_output_active() ref-count wraps play_audio_file and the
streaming OutputStream sentence writes so "audio is flowing" is accurate.
- Config: voice.thinking_sound (default true) off-switch.
- cli.py: starts when a voice-mode turn begins, per-blip gate skips while
TTS speaks / mic records / barge capture owns the mic; stopped in the
chat() finally on every exit path.
- tui_gateway/server.py: same lifecycle around _run_prompt_submit turns
(voice mode on), gated on is_audio_output_active + continuous capture.
- Desktop: renderer owns voice-conversation audio, so a matching WebAudio
implementation (src/lib/thinking-sound.ts, same envelope/pitches) runs
while conversation status === "thinking"; honors voice.thinking_sound
(via config store) and the shared sound-mute toggle; stops instantly on
speaking/listening/end.
One owner for the wording: voice_stop_hint() in tools/voice_mode.py —
sources the phrase from voice.stop_phrases (first entry) so a custom
phrase renders correctly, and returns "" when the feature is disabled
(stop_phrases: []) so no surface shows a hint.
- CLI: printed in /voice on output (style-matched dim notice).
- TUI: voice.toggle action=on now carries stop_hint; the Ink client
renders it in the "Voice mode enabled" block (older gateways omit
the field — no hint, no crash).
- Desktop: the renderer voice loop never touches tools/voice_mode.py,
so the phrase is read from config (voice.stop_phrases → $voiceStopPhrase
store, seeded in use-hermes-config) and shown as an info toast when a
voice conversation starts. i18n: en/ja/zh/zh-hant/ar.
The continuous-voice no-speech counter (3 strikes -> voice off) counted
every silent capture cycle unconditionally. During a long agent turn
(thinking/tool-calling for minutes) or while TTS is speaking, the user
is CORRECTLY silent — those cycles ended the voice chat under them.
- hermes_cli/voice.py: new set_voice_busy_probe() seam + _voice_activity_held()
(TTS-playing via the existing _tts_playing Event, agent-busy via the
registered probe). Both the continuous-loop strike path and the
force-transcribe single-shot strike path skip counting while held.
Fail-open: a broken probe counts cycles as before.
- tui_gateway/server.py: registers _any_session_running() as the probe
on voice.record start (voice is process-global; any running session holds).
- cli.py: classic CLI strike path skips counting while _agent_running
or TTS playback is in flight.
Stop phrase and barge-in still work during the hold (own paths).
Includes a fixture fix for the #71083 cherry-pick: the fake tools.tts_tool
module needs _load_tts_config (main's tts_streaming imports it).
Replace one-shot VAD calibration with a rolling deque window that
continuously recalibrates the noise floor throughout TTS playback,
preventing false barge-in triggers from stale calibration. Add a
grace period before VAD activates so TTS playback establishes first.
Suppress duplicate text rendering when token streaming is enabled.
Mirror the barge-in and TTS stream stop logic to the TUI gateway path
so both CLI and gateway use the same VAD semantics.
Rolling-window VAD:
- 90th percentile of rolling window (~3s) for noise floor
- 8x multiplier (was 5x) for TTS volume variation headroom
- 4000 RMS trigger ceiling so genuine speech can still trip
- min_floor clamped to SILENCE_RMS_THRESHOLD * 2
- sustained_ms=1000, calibration_ms=800
Barge-in grace period (barge_in_grace_seconds, default 2.0s):
Delays VAD activation so TTS playback establishes before the mic opens.
Duplicate render suppression:
When streaming_enabled, pass display_callback=None to
stream_tts_to_speaker so the token stream is the sole display path.
TUI gateway mirror:
Mirror _tts_stream_stop and _tts_stream_barge_in_monitor changes to
tui_gateway/server.py so both code paths use the same VAD parameters,
grace period, and TTS CUT diagnostic logging.
Profile-scoped session DB and MoA progress events in tui_gateway/server.py
were necessitated by the TTS pipeline changes affecting session state
and event routing.
Normal-exit flag and TTS CUT diagnostic logging at all cut paths.
Regression tests:
- test_quiet_then_loud_playback_does_not_trip
- test_8x_multiplier_absorbs_tts_volume_spikes
- test_trigger_ceiling_lets_genuine_speech_trip
- test_silence_calibration_does_not_false_trip_on_tts
- test_tts_stream_stop_latches_interruption_for_next_turn
- test_tts_stream_stop_after_natural_finish_does_not_latch
- Profile-scoped session DB tests (10 tests)
Fixes#73973.
A finite one-shot whose dispatch was claimed (claim_dispatch increments
repeat.completed BEFORE execution) but whose run died before mark_job_run
was left permanently wedged: completed==times, last_run_at null, state
'scheduled'. The run-claim TTL blocked re-dispatch, and once it expired
the dispatch-limit guard silently removed the job with no output and no
error.
Two complementary fixes:
- cron/scheduler.py run_one_job: the outer handler now catches
BaseException, not just Exception. The inner run_job handler re-raises
CancelledError/KeyboardInterrupt/SystemExit after agent teardown, and
none of those are Exception subclasses, so the outer 'except Exception'
missed them and mark_job_run(False) was never called. Failures are now
recorded first (mark_job_run + finish_execution, each independently
guarded), then non-Exception BaseExceptions are re-raised to preserve
teardown semantics. Plain Exceptions keep the existing behavior
(recorded, return False, no re-raise). Empty str(e) (bare
CancelledError) falls back to the exception class name.
- cron/jobs.py: when either removal site (claim_dispatch or the
get_due_jobs dispatch-limit guard) drops a one-shot whose claimed run
never completed (last_run_at null), _write_wedged_oneshot_diagnostic
now writes an operator-visible .md into cron/output/<job_id>/ instead
of vanishing silently. Best-effort: diagnostics can never break the
removal. No diagnostic when last_run_at is set (normal completion
race).
Issue #65773: run_one_job installs a <home>/.env secret scope around every
job; before c758ded6d (#69057) an installed scope was authoritative even
with multiplexing off, so provider keys injected only via the process
environment (container env vars, systemd Environment=) resolved to empty
inside cron and every provider call went out with the no-key-required
placeholder -> HTTP 401, while interactive turns kept working.
The fix landed in agent/secret_scope.py (scope-miss fallthrough to
os.environ when multiplex is off) with unit tests at that layer only.
These two tests pin the end-to-end contract where the bug actually
surfaced - cron's run_one_job:
- env-injected key resolves during run_job with multiplex OFF (fails on
pre-fix code, verified by mutation against c758ded6d~1)
- .env value still wins when both sources define the key (precedence)
Implementation-agnostic: passes whether the fix is the get_secret
fallthrough (main today) or a multiplex guard at the installation site
(the approach in #65801/#65802/#73037).
The gateway's /reset cleanup path called shutdown_memory_provider without
first draining the memory manager's serialized background write worker.
shutdown_all only gives that worker a bounded (~5s) drain and abandons
whatever is still queued past it, so a /reset could silently drop writes
the session had already handed off -- the next session then loaded
stale MEMORY.md.
The CLI exit path already drains via MemoryManager.flush_pending before
shutdown; this PR pins the same contract on the gateway cleanup path.
Cleanup now calls agent._memory_manager.flush_pending(timeout=10) before
the existing shutdown_memory_provider step. The flush is best-effort:
a flush failure must never block teardown, so it is wrapped in
try/except and the existing shutdown path remains the fallback.
Closes#73297
- extract _commit_registry/_note_refresh_failure shared by the background
worker and foreground stage-4 (identical 4-step success + failure paths
were duplicated); worker now commits under _models_dev_fetch_lock so a
failing background refresh can never re-arm the backoff immediately
after a successful force_refresh committed (unsynchronized-write race)
- add should_clear_context_pin_async to hermes_cli/route_identity.py
(matching the get_model_context_length_async precedent) and use it at
the 4 async gateway sites instead of inline asyncio.to_thread wraps;
the sync _format_session_info site keeps the sync call (already
off-loop via its callers' to_thread)
- test the background-refresh success path (the PR's primary new
behavior): disk saved, mem cache swapped, backoff cleared, in_flight
reset — mutation-checked
- replace the race-prone spin-wait on _models_dev_refresh_in_flight with
a named-thread join in the backoff test
The branched call shape in get_provider_info/get_provider is deliberate:
~69 test sites across tests/hermes_cli and tests/gateway monkeypatch
fetch_models_dev (and get_provider_info) with zero/single-arg lambdas.
Passing allow_network= unconditionally broke 5 tests in CI slices 2/3/7.
Documented the constraint inline.
- _mark_stale_cache_grace only moves cache_time forward so a completed
background refresh is never rewound to a 5-minute grace window
- clear _models_dev_refresh_in_flight if Thread.start() raises so a
one-off thread-exhaustion failure doesn't disable refresh forever
- move empty-registry validation into _fetch_models_dev_from_network
(was duplicated in the background worker and the foreground fetch)
- pass allow_network through as a plain kwarg in get_provider_info and
hermes_cli.providers.get_provider instead of the branched call shape
- refresh the stale module docstring (no bundled snapshot exists; the
resolution order now describes stale-serve + background refresh)
Follow-ups on top of @DonutsDelivery's salvaged reaper commit (#64141):
- create_from_config() now parses lsp.idle_timeout (invalid values fall
back to DEFAULT_IDLE_TIMEOUT) — previously the constructor knob was
unreachable from config.yaml (config exposure adapted from #36892 by
@0xbWy and #68091 by @9miya20)
- canonical default declared in hermes_cli DEFAULT_CONFIG so config
discovery surfaces the knob (per sweeper review note on #64980)
- reaper loop survives transient sweep errors instead of dying and
silently re-opening the leak (gap flagged in #68091 review)
- eventlog.log_reaped(): one INFO line per sweep + clears the
log_active announce cache so respawns re-announce at INFO
- docs: replace the stale 'no idle-timeout reaper' paragraph with the
new lifecycle description + config reference
- tests: reuse-refresh protection (the regression teknium's sweeper
requested on #64141), reaper-survives-error, config propagation,
invalid-value fallback, DEFAULT_CONFIG/manager-constant sync
memories/MEMORY.md is the "§"-delimited store written by MemoryStore, not a
markdown document. parse_existing_memory_entries() fell back to
extract_markdown_entries() -- the *source* parser for CLAUDE.md / AGENTS.md --
whenever the destination held no delimiter, which is exactly the case for a
single-entry store or one that was hand-edited or shell-appended. That
extractor skips fenced code blocks, skips table rows, splits a block into one
entry per bullet and reflows paragraphs. The shredded result was then written
straight back over the user's store and reported as "Imported", with no backup
to recover from.
Parse the destination the way MemoryStore._parse_entries does: split on
ENTRY_DELIMITER only, so a store with no delimiter is one intact entry.
extract_markdown_entries() is unchanged and still used on the sources, where
it is correct.
Also restore the safety net the port dropped. The openclaw migration script
this module was ported from calls maybe_backup(destination) before rewriting a
memory store; the port did not. Snapshot the store to <name>.bak.<unix_ts>
(same naming as MemoryStore._backup_drifted_file), refuse to rewrite when the
snapshot fails, and write via temp file + atomic rename so an interrupted
import cannot leave a truncated store and a symlinked MEMORY.md stays a
symlink.
The identical fallback lives in openclaw_to_hermes.py, where it is reached
from migrate_memory() (memories/MEMORY.md and memories/USER.md) and
migrate_daily_memory(); fixed there too.
Follow-up for salvaged PRs #73400 + #73372:
- Move _preserve_agent_history_on_shutdown logic into
gateway/shutdown_flush.py as flush_agent_history_to_file()
- Reuse the hardened _write_payload (atomic writes, fsync, private
permissions, UUID filenames) from #73372 instead of the plain
open()/write() from #73400
- Replace os.environ.get('HERMES_HOME') with get_hermes_home() for
profile-safe path resolution
- Update tests to test the real function directly
- Net: removes 37 lines from gateway/run.py, unifies both fix paths
in a single module with consistent atomic-write guarantees
The previous attempt (#73171) snapshotted GatewayRunner._pending_messages,
which on current main has no writers (commit f6736ced8 removed its write
path; interrupt delivery uses adapter._pending_messages instead). The live
container is the per-agent agent._session_messages, flushed via
_flush_messages_to_session_db. When that flush raises (FTS/SQLite corruption,
the disk=0/memory=N state from #72680), the in-memory transcript is lost when
the process exits.
Retarget the preservation to the real path: in _finalize_shutdown_agents, wrap
the _flush call; on exception, dump agent._session_messages to an external JSON
recovery snapshot under $HERMES_HOME/shutdown-recovery/ (tagged issue=#72680)
so an operator can salvage it after repairing state.db. The dump is fully
guarded (non-fatal) so shutdown never blocks on a best-effort backup.
This directly addresses the reviewer note on #73171: retarget to the actual
cached-agent history (agent._session_messages) and prove a stale DB + shutdown
leaves a recoverable transcript.
Regression tests: tests/gateway/test_session_messages_shutdown_preserve.py
- flush raises -> recovery file written with session_id + messages
- healthy flush -> no recovery file
- write error -> non-fatal, no raise
Fixes#72680
The old test asserted _warm_gateway_module was fire-and-forget (startup
completes in << SLOW_SECONDS). PR #73291 intentionally reversed this:
the import now runs synchronously before the lifespan yield because
run_in_executor didn't release the GIL on Windows + Python 3.11.
Updated the test to assert startup blocks for >= SLOW_SECONDS.
STT previously had no configuration surface outside hand-editing
config.yaml — no category in the hermes tools picker, no provider
matrix in the GUI capabilities tab, no status line in hermes setup.
- TOOL_CATEGORIES['stt']: 7 provider rows (Local Whisper, Nous
Subscription managed, OpenAI, Groq, xAI, ElevenLabs Scribe,
DeepInfra) with key prompts, badges, and post-setup hooks
- stt_provider marker wired through _write_provider_config,
_configure_provider, _reconfigure_provider, and
_is_provider_active — GUI and CLI share one write path
(apply_provider_selection)
- STT model picker (_configure_stt_model + STT_MODEL_CATALOG) runs
after provider pick: local sizes, Groq whisper family, OpenAI
whisper-1/gpt-4o-*/gpt-transcribe, ElevenLabs scribe (model_id key)
- faster_whisper post-setup hook auto-installs the local backend;
registered in _POST_SETUP_READY
- stt is CONFIG-ONLY (_CONFIG_ONLY_TOOLSETS): it ships no tool
schemas, so it is excluded from the per-platform enable checklist;
the GUI toolset toggle writes stt.enabled instead of
platform_toolsets
- hermes setup shows a Speech-to-Text status line per provider
- Mistral row omitted (mistralai PyPI quarantine), mirroring the
dashboard stt.provider options
Tests: tests/hermes_cli/test_stt_picker.py (20 cases) incl. invariant
checks against agent.transcription_registry builtins and the runtime
OPENAI_MODELS/GROQ_MODELS sets.
Follow-up on the #53205 salvage: replace bare is_file() probes of the
managed (~/.hermes/node[/bin]) and legacy (node_modules/.bin) locations
with shutil.which(..., path=dir) so Windows resolves the executable
.cmd shim instead of the extensionless POSIX script — the same miss
class fixed for _has_agent_browser() in #73932. Also covers the
Windows managed layout where the binary sits in node/ directly.
`hermes acp --setup-browser` installs agent-browser into the Hermes-managed
node prefix (~/.hermes/node/bin/agent-browser), which isn't necessarily on
PATH. doctor only checked PROJECT_ROOT/node_modules and PATH (shutil.which),
so it false-negatived with "agent-browser not installed" even though the
binary was present and runnable. Mirror dep_ensure._has_hermes_agent_browser()
by also checking HERMES_HOME/node/bin and the legacy
HERMES_HOME/node_modules/.bin path, each gated by agent_browser_runnable().
Tested with tests/hermes_cli/test_doctor.py (added positive + not-runnable
cases) and pytest tests/hermes_cli/test_doctor.py -q (66 passed).
The exact kwargs snapshot broke when VAD hardening added keys — the
test's real contract is 'null stt.local: must not crash or force
language/prompt'. Baseline kwargs are pinned by the dedicated suite.
Local faster-whisper called model.transcribe with bare {'beam_size': 5}:
no VAD, cross-window conditioning on, no confidence filtering. Pure
silence produced hallucinated tokens (E2E: 5s anullsrc WAV -> 'You',
no_speech_prob=0.705) and noisy clips could produce runs of junk, often
in other languages.
Three-layer class fix, one shared owner for every local-whisper call
site (build_local_transcribe_kwargs):
1. Silero VAD filter (bundled with faster-whisper) on by default —
silence never reaches the model. stt.local.vad: false restores the
raw behavior for music/ambient transcription.
stt.local.vad_min_silence_ms tunes chunk splitting (default 500).
2. condition_on_previous_text=False — one hallucinated token can no
longer seed a self-reinforcing run; negligible cost for
voice-note-length audio.
3. Segment confidence gate (_join_confident_segments): drop a segment
only when no_speech_prob > 0.6 AND avg_logprob < -1.0 (openai-whisper's
own heuristic shape; both must hit so quiet-but-real speech survives).
Config: stt.local.no_speech_prob_threshold / logprob_threshold.
The WHISPER_HALLUCINATIONS blocklist in voice_mode.py stays as
last-resort defense but should now almost never fire.
E2E (real faster-whisper 'base', CPU int8):
silence.wav before 'You' -> after ''
noise.wav before '' -> after ''
speech.wav before/after 'Hello World, this is a test of the
transcription system.' (unchanged)
Docs (EN + zh-Hans), DEFAULT_CONFIG, cli-config.yaml.example updated;
19 unit tests (kwargs contract, off-switch, confidence gate incl.
quiet-speech survival, _transcribe_local wiring), sabotage-verified.
Saying OR typing a configured stop phrase (voice.stop_phrases, default
"stop") now ends the voice chat everywhere, not just classic CLI PTT:
- hermes_cli/voice.py: new explicit on_stop_phrase callback through
start_continuous/stop_continuous. The force-transcribe path previously
DISCARDED the stop phrase silently — with auto_restart=False the client
re-arms the next capture, so the conversation never ended. Both halt
paths now fire on_stop_phrase (fallback: on_silent_limit for legacy
callers) as user intent, distinct from the no-speech timeout.
- tui_gateway/server.py: voice.record wires on_stop_phrase and emits
voice.transcript {stop_phrase: true} after flipping HERMES_VOICE(_TTS)
off and stopping streaming TTS — same teardown as /voice off. The TTS
barge-in monitor stop-checks its transcript too. prompt.submit consumes
a TYPED bare stop phrase at the server-side choke point when voice mode
is on (returns {voice_stopped: true}, no turn starts).
- ui-tui: voice.transcript {stop_phrase} ends voice mode with a clear
'voice chat ended' notice (distinct from the no-speech-limit message);
submitPrompt releases the busy latch on a consumed voice_stopped reply.
- cli.py: _typed_voice_stop in process_loop — typing a bare stop phrase
while voice mode/continuous is active ends voice mode instead of
sending 'stop' to the agent; typed 'stop' outside voice mode is
unchanged. Voice transcripts skip the check (already stop-checked).
- desktop: interceptsTypedVoiceStop — the composer's onSubmit ends the
live voice conversation (same path as clicking end on the pill) when a
bare stop command is typed with no attachments; renderer-owned loop, so
handled client-side like the existing spoken isVoiceStopCommand.
- tools/voice_mode.py: transcribe_recording never lets the Whisper
hallucination filter swallow a configured stop phrase (e.g. 'bye'
configured as a stop phrase is both a hallucination-blocklist entry and
a stop phrase — stop-phrase check now wins).
Tests: continuous-loop signal (sabotage-verified), force-transcribe stop
signal + legacy fallback, hallucination-filter ordering, typed-stop CLI
unit tests (voice on/off/longer text), prompt.submit typed-stop gateway
tests, TUI vitest for stop_phrase event handling, desktop vitest for the
typed-stop interceptor.
Streamed response text carried a 4-space _STREAM_PAD indent and the
final-response Rich Panel used padding=(1, 4), so every line selected
out of the terminal came with leading whitespace. Both now render
flush-left (pad empty, panel padding=(1, 0)); the table-realignment
width budgets were widened to match.
/copy now writes the ORIGINAL message text through native clipboard
tools (pbcopy / PowerShell Set-Clipboard via base64 / wl-copy / xclip /
xsel — same fallback chain as the TUI's writeClipboardText), falling
back to OSC 52 only when no native backend succeeds. This is the
TUI-equivalent answer to soft-wrap mangling: the clipboard gets the raw
text, not the rendered layout.
_has_agent_browser()'s new managed-Node rung calls
shutil.which('agent-browser', path=...); tests that monkeypatch
shutil.which globally with 1-arg lambdas raised TypeError when their
code path reached the browser readiness probe (test_post_setup_gating,
test_setup_model_provider).
Three GUI Capabilities-tab defects reported on Windows:
1. Browser rows stuck on 'Setup required' after a successful setup run.
Root causes, all in the readiness probe (not the installer):
- _has_agent_browser() never searched the Hermes-managed Node dir
(%LOCALAPPDATA%/hermes/node / $HERMES_HOME/node/bin) where the
Windows install lands, and probed node_modules/.bin/agent-browser
as the extensionless POSIX shim, which fails exec on Windows
(WinError 193) — now resolved via PATHEXT-aware shutil.which
against both rungs, mirroring _find_agent_browser().
- Cloud rows (Nous Subscription Browser Use, Browserbase, Browser
Use, Firecrawl) declared post_setup: agent_browser, whose
readiness gate requires a LOCAL Chromium build the cloud never
uses — switched to the cloud-scoped 'browserbase' hook (CLI-only).
- _agent_browser_installed() could read browser_tool's stale cached
'Chromium missing' result from before the install ran in the
spawned post-setup process — cache now dropped before probing so
the pill flips to Ready right after a successful run.
2. No way to tell which backend is active, and clicking a row to read
its details silently rewrote config. Row click now only
expands/collapses; activation is an explicit 'Use this backend'
button, the active row carries an 'Active' pill, and the expanded
active row says 'This is your active backend'.
3. OpenAI TTS showed one model and one voice. The options were always
defined but rendered through a native <datalist>, which filters by
the field's current value — a field already set to a valid option
suggested only itself. Replaced with a real combobox (Input +
dropdown) that lists every option, and voice suggestions now track
the selected model per the OpenAI TTS docs: tts-1/tts-1-hd = 9
voices, gpt-4o-mini-tts = 13 (adds ballad, verse, marin, cedar).
The Skills Hub 'Marketplace' tab showed a single useless entry: Anthropic
changed .claude-plugin/marketplace.json to bundle-shaped plugins whose
source is './', so all plugins collapsed to one identifier pointing at the
repo root, and the second marketplace repo (aiskillstore/marketplace) is
gone (404). Everything in anthropics/skills is already surfaced by the
GitHub tap as the Anthropic tab, making this source fully redundant.
Removes ClaudeMarketplaceSource and all wiring: source router, index
builder (crawl + floors + sort order + rate-limit messaging), extract
labels/install/URL mapping, hub UI tab, web server labels, CLI limits,
docs (en + zh), the legacy index-cache snapshot, and test fixtures.
Stale skills-index entries with source 'claude-marketplace' still install
fine: HermesIndexSource fetches via resolved GitHub paths generically.
Replace 20 lines of manual os.open/O_EXCL/fdopen/fsync/os.replace with the
existing atomic_json_write() from utils.py, which is already used by 6+
modules and handles temp-file creation, fsync, atomic replace, mode
control, and owner preservation. The only novel helper (_fsync_directory)
is retained — atomic_json_write does not do directory fsync.
Update test_flush_write_failure_leaves_no_recovery_file to monkeypatch
utils.os.replace (the new call path) instead of gateway.shutdown_flush.os.replace.
test_default_bind_serves_both_families asserts an IPv6 listening socket
exists, which is the environment's capability, not the adapter's — CI
runners with IPv6 disabled would flake. Probe an actual ::1 bind (not
just socket.has_ipv6, a compile-time constant) and skip cleanly when
the host has no usable IPv6 stack.
The LINE adapter's webhook server defaulted to host="0.0.0.0", which
binds IPv4 ONLY. On IPv6-only private networks — notably Fly.io 6PN,
where the hosted edge router reverse-proxies LINE ingest to
<app>.internal:8646 over an fdaa: IPv6 address — nothing is listening
on the dialed address: connection refused → customer-visible 502 when
LINE's console verifies the webhook (NS-603).
This is the same bug the generic webhook adapter fixed in d542894ad;
the LINE adapter was never updated to match. Fix mirrors that commit:
- DEFAULT_HOST = None → asyncio/aiohttp create_server binds one socket
per address family (v4 + v6), regardless of the bindv6only sysctl.
"::" is NOT a valid substitute — Fly machines set bindv6only=1, so
it would yield an IPv6-only socket and break IPv4 loopback probes.
- Empty-string host collapses to None; LINE_HOST/extra.host still pin
a specific bind address.
- reuse_address=False scoped to macOS only (BSD wildcard-socket
traffic-splitting footgun), mirroring 9420ad946.
- The three outbound-media guards compared webhook_host == "0.0.0.0"
by string equality; extracted to _missing_public_url() which treats
None/0.0.0.0/::/"" as "no fetchable hostname" so the LINE_PUBLIC_URL
requirement still fires under the new default. _media_url() falls
back to 127.0.0.1 instead of interpolating 'None' into URLs.
Tests: dual-stack default decision table (default None, empty→None,
pinned preserved, LINE_HOST override), behavioural both-families bind
proof via runner.addresses, and the media public-URL guard matrix.
88 passed, ruff clean.
Companion router fix (hermes-agent-router) routes /webhooks/line and
/line/webhook|/line/media to :8646 over 6PN; both are needed for
end-to-end hosted LINE delivery.
Fixes NS-603
show_status now reads get_nous_auth_status_local(); the test_status.py
mocks still patched the old live-resolve entry point, so the patched
dict was never consumed (CI slice 8/8 red on the salvage PR).
Follow-up widening of the /api/status fix: add get_nous_auth_status_local(),
a refresh-free auth-store snapshot (local invoke-JWT decode only), and use it
on the read-only display surfaces that previously called
get_nous_auth_status() -> resolve_nous_runtime_credentials() -> live OAuth
refresh POST:
- hermes_cli/status.py (hermes status auth-provider panel)
- hermes_cli/doctor.py (hermes doctor auth-provider checks)
- hermes_cli/portal_cli.py (hermes portal status display)
- hermes_cli/web_server.py /api/portal endpoint and the accounts-tab
provider card dispatcher (_resolve_provider_status nous branch)
Action paths (login flows, portal operations needing a live credential)
keep using get_nous_auth_status(). Part of NS-592.
resolve_sidecar_dir() probes the filesystem (touch/unlink) and can mirror
sidecar files to HERMES_HOME. Doing that as a module-import side effect
meant plugin discovery, `hermes --help`, and test collection all paid a
filesystem probe (and possibly a mirror copy) just for importing the
photon adapter or CLI.
Convert _SIDECAR_DIR/_NPM_ERROR_LOG in adapter.py and cli.py to lazy
cached accessors (_sidecar_dir()/_npm_error_log()); resolution now
happens on first actual use. Existing tests that monkeypatch the
_SIDECAR_DIR module global keep working — the accessors honor a
non-None value. Adds a regression test proving import performs no
resolution.
The Photon iMessage sidecar needs node_modules under
plugins/platforms/photon/sidecar/, but hosted/managed images keep the
whole install tree under an immutable /opt/hermes — every install and
self-heal path (setup CLI, stale-deps reinstall, cold install) died on
EROFS, and hosted users have no shell to work around it.
Three-layer fix, mirroring the WhatsApp bridge resolver pattern:
1. Bake the deps into the image. The Dockerfile now runs npm ci for the
sidecar in the layer-cached dependency stage (deterministic installs
from the committed lockfile; the postinstall spectrum-ts patch runs
at build time). Hosted happy path needs no runtime install at all.
2. New sidecar_paths.resolve_sidecar_dir() decides where the sidecar
runs from: PHOTON_SIDECAR_DIR override > writable source dir (dev
installs, unchanged) > read-only dir with baked fresh deps (managed
image) > mirror to $HERMES_HOME/photon/sidecar (writable data
volume) when deps are missing or stale in a read-only tree. The
mirror refreshes changed source files on image updates while
keeping node_modules, so the existing lockfile-staleness self-heal
works there.
3. connect() can now cold-install: _start_sidecar() runs the bounded
npm ci bootstrap when node_modules is missing instead of raising
immediately, and check_requirements() reports available when a
self-install is possible (npm present + writable resolved dir) so
the gateway actually creates the adapter on hosted instances. A
failed bootstrap still raises the actionable error, which connect()
surfaces as the retryable SIDECAR_FAILED fatal state on the
dashboard.
Tests: resolver decision table (env override, in-place, mirror,
refresh, fail-open), cold-install lifecycle paths, and a Dockerfile
contract test guarding the baked-deps + no-chown invariants.
Fixes NS-606.
The ownership guard let a sentinel with pid=None pass as self-owned, so
an exiting life could clobber evidence of unknown provenance with a
clean-exit claim. Tighten: only rewrite when the sentinel pid matches
os.getpid() exactly; pid=None (or malformed) is left untouched. Adds
tests for the pid=None no-op and the own-pid rewrite paths.
Hosted agents that die uncleanly (kernel OOM kill, SIGKILL, whole-VM
death) leave no trace: shutdown_forensics only covers graceful signals,
gateway-exit-diag.log only covers exit paths that actually run, and the
VM reboot wipes dmesg before anyone can capture it. NS-608 (BlueAtlas
hourly crash cycle, July 12-15) took days of manual log correlation to
classify because nothing recorded 'the previous life ended violently'.
Add gateway/lifecycle_ledger.py — a sentinel state machine persisted to
<HERMES_HOME>/state/gateway.lifecycle.json:
- start_gateway() claims the sentinel (phase=running) right after the
PID-file/runtime-lock claim, and reports any prior life that never
reached an exit path as gateway.previous_unclean_exit in
gateway-exit-diag.log + a WARNING log line.
- Every exit funnel marks the sentinel exited with a reason:
_exit_after_graceful_shutdown (graceful_shutdown), the shutdown
watchdog (shutdown_watchdog), and the loop-liveness watchdog
(loop_liveness_watchdog).
- Ownership-guarded for --replace takeovers: a live matching owner is
never reported dead, and the old life cannot clobber the
replacement's freshly claimed sentinel on its way out.
The 30s loop heartbeat now embeds a cheap /proc memory sample (own RSS,
MemAvailable, swap used) so every unclean-death report carries a
'memory N seconds before death' snapshot; the detector flags
suspected_oom when the last sample shows <64MiB or <5% available.
container-boot.log lines gain prior_exit=clean|unclean|unknown per
profile, stamping unclean container deaths into the volume-persisted
boot log where support can grep for them.
Tests: tests/gateway/test_lifecycle_ledger.py (16 cases) + 4 new
container-boot annotation cases. Existing watchdog/forensics/boot
suites all green; ruff clean.
Adds gpt-transcribe (OpenAI's new file-transcription model, $0.0045/min)
to the OpenAI STT provider:
- OPENAI_MODELS set: gpt-transcribe is recognized so provider
auto-correction keeps it on OpenAI and rejects it on Groq
- Language hint wiring: gpt-transcribe replaces the singular
'language' field with a 'languages' list; the API rejects the legacy
field, so the hint is sent via extra_body {languages: [..]}
- Config comment (DEFAULT_CONFIG), cli-config.yaml.example, desktop
settings enum, and docs (en + zh-Hans) updated
- Tests: model pass-through, languages-list hint shape, legacy singular
hint preserved for gpt-4o-transcribe, Groq auto-correction
gpt-live-transcribe (realtime WebSocket, $0.017/min) is NOT wired here:
the file-based STT pipeline has no realtime session path; it belongs in
a future realtime/voice-mode integration.
_install_dependencies now routes through tools.lazy_deps.install_specs
(NS-605); the force-reinstall test still stubbed
hermes_cli.tools_config._pip_install, so its spy list stayed empty
(CI slice 3/8 red on the salvage PR).
Widen NS-605 to the two remaining direct-install sites in the hindsight
plugin, which still shelled out to 'uv pip install --python
sys.executable' and therefore failed (EROFS/EACCES) on immutable hosted
images with sealed venvs, and lost packages on redeploy:
- post_setup dependency install (~L835): install_specs() with ok /
blocked-reason / stderr handling matching honcho/mem0.
- initialize()-time hindsight-client auto-upgrade (~L1240):
install_specs(); blocked installs log the gate reason with the manual
command instead of a raw subprocess error, and init proceeds.
Audited every other memory plugin (supermemory, byterover, holographic,
openviking, retaindb) for direct pip/uv install subprocess calls: none
remain — their deps flow through plugin.yaml pip_dependencies or
lazy_deps.ensure().
Tests: TestClientAutoUpgradeRoutesThroughLazyDeps — upgrade goes through
install_specs with the exact spec (regression guard asserts no
subprocess.run), blocked upgrade is non-fatal and surfaces the gate
reason. Updated TestPostSetupEnvEncoding stubs to the new install path.
Installing a memory provider (Honcho, mem0, hindsight, ...) from the
dashboard Plugins page failed on hosted deployments with a permission
error: the setup endpoint shelled out to
`uv pip install --python sys.executable`, which targets the sealed
read-only venv under /opt/hermes (immutable hosted image, NS-579/#49113).
The correct mechanism already exists: tools/lazy_deps.py redirects
installs to the writable durable target on the data volume
(HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages) when the venv is
sealed (HERMES_DISABLE_LAZY_INSTALLS=1), appends the target to the END
of sys.path (core venv always wins collisions), and constrains shared
deps to core-venv versions. The dashboard installer simply never used
it.
Fix:
- tools/lazy_deps.py: new public install_specs() — installs arbitrary
manifest-declared pip specs through the same environment routing as
ensure(): venv-scoped by default, durable-target on sealed images,
refused with an actionable reason when gated off (config kill switch
or sealed venv without a target — never surfaces raw EROFS/EACCES).
Specs are validated with _spec_is_safe(); post-install it invalidates
import/metadata caches so availability rechecks in the same process
see the new packages without a restart. Never raises.
- hermes_cli/web_server.py: _install_memory_provider_pip_dependencies
now calls install_specs() instead of building its own uv/pip
subprocess. Blocked installs surface the gate reason in the setup
results; the response's status block reflects post-install
availability (stale 'missing deps' state clears immediately).
- hermes_cli/memory_setup.py, plugins/memory/honcho/cli.py,
plugins/memory/mem0/_setup.py: CLI setup wizards routed through
install_specs() too — same sealed-venv failure mode, same fix.
No hosted setup path writes to /opt/hermes anymore; provider discovery
and installation now use the same environment (sys.path activation is
shared with the lazy-install bootstrap in hermes_bootstrap).
Tests:
- tests/tools/test_lazy_deps.py: TestInstallSpecs — gating matrix
(sealed+no-target blocked with immutable-deployment reason, config
kill switch, sealed+target proceeds), spec-safety rejection before
any subprocess, venv-scoped vs --target command display, failure
stderr passthrough, never-raises contract.
- tests/hermes_cli/test_web_server.py: setup endpoint routes pip
through lazy_deps (regression guard asserts no direct 'pip install'
subprocess), blocked-reason surfacing, same-response availability
recheck clears stale missing state.
Fixes NS-605 (Plain T-1111).
