- gateway/relay/media.py: RelayMediaClient for the connector's /relay/media
plane (upload local files → re-host reference for send_media; download
re-hosted inbound attachments → local temp paths). Same connector base URL
the WS dials, same per-gateway signed bearer as the upgrade (auth.py) —
no new configuration. stdlib urllib in a thread executor (no new deps);
25MB cap mirroring the connector's MEDIA_MAX_BYTES.
- RelayAdapter: send_image / send_image_file / send_voice / send_video /
send_document overrides route through ONE send_media op (media by
reference: local paths upload first, public URLs pass through). Gated on
supported_ops advertising send_media — legacy connectors keep today's
text fallbacks; connector declines/failed uploads degrade the same way.
Scope/user egress discriminators ride metadata exactly like send.
- Inbound: _localize_inbound_media downloads each media_urls entry to a
local temp path (native-adapter parity — vision/file tools consume
paths); dead re-host refs are dropped, public URLs survive a missing
client. Best-effort, never blocks handle_message.
- docs/relay-connector-contract.md §4: send_media op row + media
ingress/egress semantics (replaces the 'deferred to a later revision'
note). Additive within contract_version 1.
- tests: tests/gateway/relay/test_relay_media.py (15) — kind mapping,
upload-first path handling, op gating (explicit + legacy-empty),
decline/upload-failure fallbacks, scope metadata, inbound localization
matrix, client URL derivation/credential gating. Stub connector grew a
canned send_media result.
Cross-repo pair: gateway-gateway 'Phase 2 media parity' PR (re-host plane +
four ingress lanes + four platform send_media senders).
Follow-up for salvaged PR #70615 — the 2-button smart_deny case
(Allow Once + Deny only) was exercised by an existing test but only
at the flat-label level, not asserting the row pairing. Adds the
missing row-structure assertion using the same capture pattern as
the 4-button and 3-button tests.
Follow-up for salvaged PR #69380. The snippet
'export -p | grep -vE ... > $tmp.$BASHPID || true' attaches the redirect to
the grep pipeline segment, so $BASHPID expands inside grep's pipeline
subshell — a DIFFERENT pid than the parent shell that expands the follow-up
'mv $tmp' operand. The dump landed in an orphaned temp file, mv failed
silently (2>/dev/null), and the shared snapshot never updated again:
exported env / venv activation stopped persisting between commands
(tests/tools/test_local_shell_init.py TestSnapshotEndToEnd caught it).
Wrap the pipeline in a brace group and attach the redirect to the group so
the expansion happens in the current shell, matching mv's expansion. Also
rename the shell-init probe var HERMES_SESSION_ENV_PROBE ->
HERMES_STICKY_ENV_PROBE: it matched the HERMES_SESSION_ prefix the salvaged
fix now intentionally strips from snapshots, and the snippet-shape test is
updated to pin the brace-group contract.
A single long-lived backend serves many sessions through one
_active_environments['default'] LocalEnvironment (the messaging gateway, TUI,
and desktop/web dashboard all collapse the terminal to 'default'). That
environment persists a bash session snapshot file and sources it before every
command. 'export -p' dumped the FIRST session's HERMES_SESSION_ID into the
snapshot, so every LATER session sourced that stale value and its
'echo $HERMES_SESSION_ID' reported a FOREIGN session's id — overriding the
correct per-command Popen env injected by _inject_session_context_env.
Confirmed on staging-fp: session A read its own id, session B (same backend)
read A's id. Reproduced locally with two threads sharing one LocalEnvironment;
the snapshot held 'declare -x HERMES_SESSION_ID=...' from the first session.
Fix: strip the per-session bridged vars (HERMES_SESSION_* / HERMES_UI_SESSION_ID
/ HERMES_CRON_AUTO_DELIVER_*, i.e. gateway.session_context._VAR_MAP prefixes)
from the snapshot at both dump sites in base.py. They are re-injected fresh on
every command, so a snapshot should carry only the user's own shell state, not
Hermes' per-turn session identity.
Complements the _set_session_context session_id fix: that ensures the ContextVar
carries the right id; this ensures the shared snapshot can't override it with a
neighbour's. Adds tests/tools/test_snapshot_session_id_leak.py (regex unit +
real two-session LocalEnvironment integration) and updates the export-shape
assertions in test_base_environment.py for the new 'export -p | grep' dump.
_set_session_context() called set_session_vars() without session_id, so the
HERMES_SESSION_ID contextvar was set to "" (explicitly empty) on every
prompt.submit / session bind. The subprocess-env bridge
(_inject_session_context_env) treats an explicit "" as authoritative and does
NOT fall back to os.environ, so terminal/execute_code commands in a
dashboard/TUI/web session saw an empty $HERMES_SESSION_ID — even though
agent_init had populated it via set_current_session_id().
Every other set_session_vars() call site (gateway/run.py, cron, api_server)
passes session_id; tui_gateway was the only one that dropped it, affecting all
three of its frontends (Ink TUI, desktop, web).
Fix: derive the live id in the existing _sessions lookup loop
(agent.session_id, falling back to session_key — same derivation used at
session finalize) and pass it through to set_session_vars.
Adds tests/tui_gateway/test_session_id_injection.py covering agent id,
session_key fallback, and unknown/ephemeral key.
Reviewer egilewski identified that the 5s lock timeout fell open:
quarantine_zeroed_state_db() logged 'proceeding without the cross-
process lock' and continued to re-check + rename state.db. A slow or
paused startup that still owns the lock can overlap this fallback and
the two processes can again act on the same live file.
Fix: fail closed — return None without moving the file when the lock
cannot be acquired within 5s. Log an error with recovery guidance
(restore from state-snapshots).
Test: test_quarantine_fails_closed_when_lock_held holds the cross-
process lock from a background thread, calls quarantine, and asserts
it returns None without moving the zeroed file.
Reviewer egilewski identified two recovery-loss paths:
Path A — quarantine race (hermes_state.py):
SessionDB checks state.db before quarantine_zeroed_state_db() without a
shared cross-process lock, and Path.rename() may replace an existing
destination. Two writable startups can therefore move the first
instance's newly created database over the .zeroed-*.bak, erase the
original damaged-file evidence, and replace the live database with
another empty one.
Fix: add a cross-process lock (msvcrt on Windows, fcntl on POSIX)
around quarantine_zeroed_state_db() with a 5s bounded timeout. Under
the lock: re-check is_zeroed_state_db (another process may have already
quarantined it and created a fresh DB), use a PID-suffixed unique
destination, and non-clobbering rename with counter fallback.
Path B — size-cap pruning gap (hermes_cli/backup.py):
_too_large() runs before failed-database tracking. With keep=1 and a
size cap, an oversized state.db is omitted while failed_dbs stays empty,
so automatic pruning deletes the older complete snapshot that may
contain the only recoverable database.
Fix: track oversized DB files in a new oversized_skipped list (both in
the directory walk and top-level file loop). The manifest now records
oversized_skipped. Pruning is suppressed when failed_dbs or
oversized_skipped is non-empty, preserving the older complete snapshot
as recovery source.
Tests:
- test_concurrent_quarantine_no_clobber: two threads racing on the
same zeroed state.db — verifies quarantine backup survives with
original bytes and live DB is valid.
- test_oversized_db_suppresses_pruning: keep=1 + oversized state.db
verifies the older complete snapshot is not pruned.
All 33 tests pass (3 zeroed_state_db + 25 TestQuickSnapshot + 5
quarantine_forensic_logging).
Hardening for the Windows zeroed-state.db class (#68474):
- Surface critical stdout when pre-update/quick snapshot cannot copy a
present *.db (was log-only; update still looked successful).
- Detect all-NUL SQLite header on SessionDB open, quarantine the bytes,
and open a fresh DB with recovery guidance to state-snapshots.
Does not claim storage-stack root cause.
The #71119 integrity gate rejected CORRECT ARM64 desktop rebuilds on
Windows-on-ARM (#69179 follow-up): platform.machine() reports the PROCESS
architecture, and the update chain runs an x64 hermes-setup.exe / x64
Python under ARM64 emulation, so the gate compared the ARM64 Hermes.exe
against a phantom 'AMD64 host' and aborted the update.
_windows_native_machine() now asks the OS via IsWow64Process2 (whose
nativeMachine is truthful from emulated processes), falling back to
PROCESSOR_ARCHITEW6432/PROCESSOR_ARCHITECTURE (pre-1511 Win10) and then
platform.machine(). All three consumers — the integrity gate, the
packaged-exe arch preference, and backup validation — go through it.
Sabotage-verified: the three new regression tests fail against the old
process-arch behavior and pass with the fix.
Builds on @ms-alan's label fix: the negative figure had a second cause
that relabelling alone leaves in place.
`hermes sessions optimize-storage` reported "reclaimed -3820.1 MB" on a
database that had in fact shrunk 60% (25069 MB -> 9975 MB). Both figures
came from os.path.getsize(). In WAL mode a VACUUM's rewrite lands in the
-wal file, and the checkpoint that folds it back is REFUSED
(SQLITE_BUSY) while any other connection holds a read-mark — e.g. the
live gateway. So the main file still carried its pre-VACUUM size AND kept
growing while the command ran, making the after-figure larger than the
before-figure. The TRUNCATE checkpoint in close() lands after the caller
has already measured and printed.
- hermes_state.py: add SessionDB.logical_size_bytes() — page_count *
page_size, the size the file settles at once the WAL is folded back.
Correct immediately, readers or not; returns None when the connection
is gone so callers fall back to stat().
- hermes_state.py: best-effort TRUNCATE checkpoint after the optimize
VACUUM so the file settles promptly when nothing else holds the DB.
Documented as insufficient alone (busy under a live gateway) so the
stat() approach is not reintroduced.
- hermes_cli/main.py: both `optimize-storage` and `optimize` report via
logical_size_bytes(); fixing only the reported command would have left
the same bug class next door.
- hermes_cli/main.py: lift the contributor's inline label to a
module-level _size_delta_label() shared by both sites, so the "grew by"
wording is consistent and unit-testable without reading source.
The two halves are complementary: page accounting removes the phantom
negative, and "grew by" still covers a genuine increase when concurrent
session writes outweigh what the rebuild freed.
Verified: sabotaging logical_size_bytes() back to stat() fails the new
test with a 22 MB overstatement, reproducing the report. The test asserts
its own precondition (stat() must actually be lagging) so it cannot
silently stop exercising the bug.
Tests: 464 passed (test_hermes_state.py + the new label tests).
Co-authored-by: chenbin <h-chenbin@voyah.com.cn>
GitHub's current signup rules forbid consecutive hyphens, but legacy
accounts with them exist and are valid (Roger--Han, hit live during the
July 24 sweep — the mapping had to be written by hand). Accept any
alphanumeric/hyphen login that doesn't start or end with a hyphen.
#71184 (stream resume) upgraded agent-build-failure delivery from a bare
'error' event to a terminal message.complete frame (status=error,
recoverable) so failed turns replay on resume. The #71140 test pinned the
old event shape and broke on main where the two merged within the hour.
Contract unchanged: build failure must reach the client visibly.
Leg 2 of #63078: prompt.submit returns {"status":"streaming"} immediately and
runs _start_agent_build + _wait_agent(timeout=30s) behind it. The deferred
build (MCP discovery with per-server retry backoff, synchronous model-metadata
HTTP, skills scanning) routinely outlives 30s on cold starts; on timeout
run_after_agent_ready emitted an error EVENT and returned without ever calling
_run_prompt_submit — the user's first message was permanently discarded while
the build finished successfully in the background. The desktop's optimistic
row eventually cleared with no visible error: the blank first session.
New _wait_agent_for_prompt replaces the flat cliff for the deferred prompt
path only (_sess()'s RPC-blocking _wait_agent keeps its 30s contract):
- The pending prompt stays attached to the (already off-RPC) run thread and
is delivered the moment the still-running build completes — a slow build is
no longer message loss.
- The wait runs in 5s slices so a cancel (session.interrupt / churn) is
honored promptly; the cancelled path returns None and defers to the
caller's cancel branch (the #65567 emit) for user-visible messaging.
- Past 30s the client gets ONE keyed notification.show ('Still starting the
agent…', key=agent-build-slow, desktop toast / TUI status bar), cleared on
delivery — patient, never silent.
- Permanent failure only when the build itself fails: agent_error set at
ready, the build thread died without signalling ready (fail fast via the
new _agent_build_thread handle instead of sitting out the cap on a corpse),
or the bounded cap expired on a genuinely hung build. The cap defaults to
600s and is tunable via agent.build_wait_timeout in config.yaml (no new
env vars); the error message states the message was not sent.
Tests: slow-build delivery with zero error events; the keyed progress notice
shown once and cleared; build-failure surfacing exactly one error event with
the real reason; dead-thread fail-fast; cancel honored mid-wait; config
override + fallback semantics; cap expiry message. The compute-host fallback
test stubs the new waiter alongside _wait_agent.
run_after_agent_ready() silently returned when _turn_cancel_requested or
running=False was set during lazy agent startup, leaving the Desktop with
a {"status":"streaming"} reply that never produced a message.start or
error event. The _wait_agent error branch 6 lines above already emits;
this mirrors it so the client can surface feedback instead of hanging.
This is the server-side half of #63078 — the client-side drift guard is
addressed by #64327, but even with that fix the server-side cancel race
still silently dropped the turn.
Adds two regression tests that capture _emit (the existing sibling test
mocked it to a no-op, so it could not catch the silent drop).
Closes#63078
The post-update state.db integrity guard called verify_sqlite_integrity()
with max_bytes=0, which disables the size ceiling and forces a full
PRAGMA integrity_check. That pragma walks every b-tree page in the file,
so its cost scales with database size — measured on a real 30 GB state.db:
143.5s with a cold page cache (worse under an update's memory pressure),
with zero output on screen. The update looks hung right after
"✓ Code updated!" and a CPU sits pegged.
Multi-GB session databases are normal for heavy users, so a
size-unbounded check is never an acceptable default on the update path.
- verify_sqlite_integrity(): max_bytes now defaults to
DEFAULT_INTEGRITY_CHECK_MAX_BYTES (2 GiB) instead of 0. max_bytes=0
remains the explicit opt-in for a full scan.
- The oversized path no longer degrades to a header-only check: it adds a
constant-time structural probe (read-only open + schema_version +
sqlite_master read) so the malformed-schema class is still caught, not
just the #68474 zeroed-file signature.
- Drop the explicit max_bytes=0 at the post-update guard and in
copy_db_and_verify() so both inherit the bounded default.
Measured on the reporter's real 30 GB state.db: 143.5s cold → 0.001s,
still valid=True. Corruption detection verified at multi-GB scale for
both classes (zeroed header, malformed schema) — both still fail closed.
Tests: default-is-bounded invariant, oversized probe catches malformed
schema, max_bytes=0 still forces the full check.
The renderer's session-state cache is memory-only and the backend's
inflight snapshot dies with the backend process, so nothing survived a
full app or machine death mid-turn: reopening the session showed the
transcript up to the last committed turn and silently dropped everything
the crashed turn had streamed.
While a turn runs, the visible tail (user prompt + streamed assistant
rows, tool calls included) is now journaled to localStorage — throttled
off the delta-flush hot path, bounded (24 entries / 7 days), cleared the
moment the turn settles. Session resume folds the journaled tail back
onto the restored transcript. When the backend also has a live text-only
inflight projection for the same turn, the journal overlays its richer
structure onto that row (longer text wins, base row id kept so live
deltas keep landing) instead of treating it as caught up — the ordering
defect that dropped locally recorded tool progress in the original PR.
Co-authored-by: Omar Baradei <omar@kostudios.io>
A turn that ended in error cleared inflight_turn and emitted its terminal
frame in the same breath. If the client was disconnected during that window
(the exact case for a failure like a network drop), the frame went to the
detached drop-transport and the in-memory state was already gone — the
desktop reconnected to a session with no trace of the failure.
Failed turns now retain a compact error snapshot (user prompt, partial
assistant text, error, recoverable) that session.resume's inflight payload
carries to a reconnecting client. Covers all three loss sites: the
returned-error result path, the turn exception path (which now closes with
the same status:"error" message.complete frame shape instead of a bare
error event), and agent-init failure. The snapshot lives until the next
turn starts or the session closes; _run_prompt_submit replaces a retained
error leftover instead of appending onto it.
Co-authored-by: Reza Sayar <rsayar@uvic.ca>
A real APPLICATION_COMMAND interaction forwarded over the relay arrived
slash-less: _discord_interaction_to_event set text = data['name'] ("new",
not "/new"), MessageType.TEXT, and dropped options entirely — so a
registered /new dispatched as plain chat instead of a command
(MessageEvent.is_command() is text.startswith("/")).
Port the connector's Slack slash-command precedent (normalizeSlackCommand
builds `${command} ${args}`.trim() with a leading slash and explicit
command type): for type-2 interactions build "/" + name, append rendered
options space-separated (scalar options contribute their value, matching
the native adapter's f"/model {name}" shape; SUB_COMMAND/
SUB_COMMAND_GROUP contribute their name then recurse into nested
options), and set MessageType.COMMAND. Type-3 (custom_id) and other
interaction types are unchanged. This implements the interaction->command
sub-design previously flagged as deferred in the _on_passthrough
docstring.
Companion connector fix in gateway-gateway: fix(relay): strip own-mention
prefix so addressed slash commands dispatch.
Session previews are the first 60 characters of the first user message, so
persisting the @image: directives ahead of the caption labelled the session
with a truncated file path in the sidebar, session switcher, and command
palette. Clients lift the refs out of the body line by line, so moving them
after the caption changes nothing about how the turn renders.
A turn routed to a model that takes pixels directly sends `content` as a
parts list, and the session store deliberately ignores a plain-string
persist override for a list payload — a text override must not erase a
turn's image summary. So the override was dropped for every user on a
vision-capable main model, and the durable row kept only the caption plus a
literal `[Image attached at: ...]` / `[screenshot]`, which the renderer
cannot turn back into an image. Only vision-preprocessed (text-mode) turns
were actually fixed.
Mirror the shape instead: swap the text part for the `@image:` ref form and
keep the image parts, so the model still has the pixels for the rest of the
session, and drop the `[screenshot]` stand-in on the way into the bubble
when a ref was lifted from the same message.
The unquoted alternative in the directive pattern is `\S+`, so a ref built
by string interpolation truncates at the first space and strands the tail
as loose text next to a broken thumbnail. Composer images live in the app's
userData dir, which on macOS is `~/Library/Application Support/<App>/` — so
every pasted or dropped image hit this.
Adds format_reference_value next to REFERENCE_PATTERN, mirroring
formatRefValue in the desktop's directive-text.tsx, and covers the
round-trip through the parser.
The desktop gateway passed the vision-enriched, model-only message text
(carrying an `image_url:<path>` hint) straight into run_conversation as
the persisted user turn. The renderer only parses `@image:<path>`, so it
could not rebuild the attachment from history: after a restart the image
was gone and only the caption survived, and on a live session switch the
warm cache disagreed with the authoritative text and the frontend
"rescued" the image by appending it after the caption.
run_conversation already supports persist_user_message for exactly this
"what the model sees" vs "what gets stored" split; it was simply never
wired up for the attachment path.
Bug-class coverage for both fixes: the full catalogue survives Save, context
lengths are preserved, the key never lands in config.yaml on either write
path, blank clears it, a pre-fix plaintext key migrates while a ${VAR}
template is left alone, two endpoints on one host keep separate credentials,
and an IP-derived name is still a valid POSIX env var.
The two delete tests asserted on the plaintext mirror; they now assert the
same invariants against the credential reference.
The desktop self-update chain (Desktop -> hermes-setup --update ->
hermes update -> hermes desktop --build-only -> relaunch) rebuilds
Hermes.exe on the user's machine and declared success on bare file
EXISTENCE. A truncated PE (corrupt cached Electron zip / interrupted
extraction or rcedit rewrite / full disk) or a wrong-architecture
unpacked tree therefore shipped as the 'updated' app, which Windows
refuses to load with 'This app can't run on your computer'
(此应用无法在你的电脑上运行) — and the previous working build had
already been wiped by before-pack.mjs, leaving nothing to fall back to.
Fix, in three parts:
- hermes_cli/main.py: post-build integrity gate on Windows
(_ensure_desktop_exe_launchable). Parses the PE header of the freshly
built Hermes.exe — MZ/PE magic, section-table completeness vs file
size (catches truncation), and COFF machine vs the host arch (catches
arm64/x64 mixups). On failure it purges the (likely corrupt) cached
Electron zip, invalidates the content-hash build stamp so the
updater's retry-once genuinely re-downloads and rebuilds, restores
the previous build from the .bak tree when one exists (keeping the
corrupt tree as .corrupt for diagnostics), tells the user the update
was aborted and their old version kept, and exits nonzero.
_desktop_packaged_executable also now prefers a host-loadable PE over
pure newest-mtime when multiple win-*-unpacked trees coexist.
- apps/desktop/scripts/before-pack.mjs: on win32, the previous unpacked
tree is preserved as <appOutDir>.bak (only when it holds the product
exe — partial/corrupt trees still get the plain wipe) instead of
being destroyed, providing the rollback material for the gate above.
Non-Windows behavior is unchanged.
- Behavior-contract tests: tests/hermes_cli/test_desktop_exe_integrity.py
(23 tests — synthetic PE fixtures for truncation/non-PE/arch-mismatch,
rollback semantics, and the build-only exit contract) and 6 new vitest
cases in before-pack.test.mjs for the .bak preservation rules.
Progresses #69179
The strict cold-start readiness gate (#67498) means adapter.connect() no
longer returns True until the mocked start_polling records a successful
getUpdates round trip for its generation. Update the conflict-suite
Application mocks accordingly:
- fake_start_polling side effects call
adapter._record_polling_progress(adapter._polling_generation) on the
initial connect (retry generations intentionally do NOT auto-progress
where a test asserts the conflict count survives an unproven retry).
- _build_polling_app takes the adapter so its start_polling mock can
record progress.
Without this, the cold connects in these tests wait out the full 60s
readiness deadline and fail — which is exactly the fail-closed behavior
the gate is supposed to provide when polling shows no progress.
Follow-up hardening for the salvaged #69240 readiness gate (#67498):
- _start_polling_once now returns its (generation, progress_event) pair
so the strict cold-start gate binds to exactly the generation it
started, instead of re-reading self._polling_progress_event which a
concurrent recovery task may have replaced with a newer generation's
event (the G1/G2 race flagged in the #69240 review).
- Strict cold start no longer schedules background polling recovery: a
polling error during the readiness wait is captured by a strict
callback and fails the connect attempt immediately with a loud
OSError, so GatewayRunner disposes the partial adapter and retries
with a fresh one — no more waiting out the full readiness deadline on
a generation that already errored, and no G2-on-partial-app healing.
- After readiness is proven the strict callback delegates every later
polling error to the real background-recovery callback, preserving
the existing degraded/reconnect semantics for the polling lifetime.
- The readiness-timeout error message now states the deadline and that
the gateway will retry with a fresh adapter (loud failure, not a
silent wait).
- Regression tests: current-generation progress connects; a polling
error during strict cold start fails fast without scheduling
background recovery (the #67498 idle-threads shape); stale-generation
progress is rejected.
Progresses #67498
Give Telegram a 180s default outer connect budget so cold polling can prove getUpdates readiness. Preserve the 30s default for other platforms and all explicit config/env overrides.\n\nRefs #67498
Use wall deadlines for deleteWebhook and start_polling, then fail cold startup unless getUpdates proves progress. This lets the gateway discard partial PTB state and retry with a fresh adapter.\n\nRefs #67498
Follow-up to the cherry-picked #69063: egilewski's review found that the
_dir_has_any_entry(parent) guard treats ANY entry in the mount point's
parent as proof the volume is attached — but unmounting exposes the
UNDERLAY directory's own files (e.g. a .keep placeholder), so a populated
underlying mount-point dir still classified the project as an orphan and
deleted its ref/index/metadata. Reproduced on both main and the PR head.
Attachment evidence is now positive instead of circumstantial:
* _volume_evidence() records the parent directory's (st_dev, st_ino)
identity in the project's metadata while the workdir is observably
live (at _register_project/_touch_project time). A mount point
resolves to the mounted filesystem's root while attached and to the
underlay directory after detach — same path, different directory,
different identity.
* _workdir_is_observably_gone() now requires the parent visible at
prune time to match that recorded identity before the populated-parent
check can classify an orphan. A mismatch means a different directory
(the underlay) is showing through — a detached volume, not an
observed deletion.
* Metadata without a recorded identity (written by older versions) is
never orphan-classified — unsure never deletes; the retention/stale
rule still reclaims genuinely abandoned projects off last_touch.
* The frozen pre-v2 layout has no metadata channel for the identity, so
it keeps the structural checks only (require_parent_identity=False).
* A failed evidence probe on re-registration preserves the previously
recorded identity — stale evidence can only make pruning MORE
conservative.
Windows: st_dev/st_ino of 0 (filesystems without file IDs, some network
shares) is treated as "no evidence recorded", which falls into the
conservative never-orphan path. os.path.ismount and Path.stat are
cross-platform; no POSIX-only calls added.
tests/tools/test_checkpoint_manager.py: adds egilewski's exact
regression (checkpoint history for mnt/volume/project, detach exposes
mnt/volume/.keep, prune with orphan deletion enabled → NOT deleted;
fails on the bare cherry-pick, passes with this fix), plus
no-recorded-identity conservatism and probe-failure identity
preservation. His absent-parent/empty-parent/retention/genuine-deletion/
live-project controls all still pass.
Reported-by: egilewski (review on #69063)
Addresses @egilewski's review: the parent-directory check still deleted
checkpoint history for the most common unmount layout.
Detaching storage removes the parent outright in some layouts
(`/Volumes/Ext/proj` on macOS, `/media/<user>/<label>/proj`), which the first
commit handles. But in the classic static layout — `/mnt/volume/proj`, an
fstab entry, a container bind-mount — unmounting removes the contents and
leaves the mount point behind as an empty directory. `parent.is_dir()` is then
true, the project is absent, and the startup sweep deletes its ref, index and
metadata: exactly the case this PR set out to protect.
Reproduced against the real predicate before this commit:
mount root vanished (macOS) -> False ok
empty surviving mount point -> True <-- history deleted
really deleted (siblings) -> True ok
An empty parent carries no information: it looks identical whether the volume
was detached or the project was deleted. So require the parent to actually say
something — it holds some other entry (we observed a populated directory that
does not contain the project), or it is itself a live mount point (the volume
is attached right now and demonstrably does not hold the project).
The cost is that a project deleted out of an otherwise-empty parent is no
longer reclaimed by the orphan rule. It is not leaked: the retention rule
reads `last_touch` rather than probing the filesystem and still collects it,
so reclamation is deferred, not lost. That is the right direction for a
predicate whose false positive destroys a user's restore points unattended.
`_dir_has_any_entry` stops at the first entry via `os.scandir` instead of
materializing a listing, since a project root can hold a large tree.
tests/tools/test_checkpoint_manager.py: `test_surviving_empty_mountpoint_
keeps_its_checkpoints` pins the reviewed case, and `test_empty_parent_project_
is_still_reclaimed_by_retention` pins the deferral above so the safety valve
cannot silently regress into a leak. Both fail on the previous commit. The
real-orphan control now seeds a sibling so it exercises a populated parent
rather than the ambiguous empty one. 80 passed in the checkpoint suite; the 2
remaining failures (`TestGitEnvIsolation`, `TestClearFunctions`) fail
identically on clean main.
Orphan pruning decides a project is gone from a single probe:
if delete_orphans and (not workdir or not Path(workdir).exists()):
reason = "orphan"
then deletes its ref, index, and metadata — the project's entire checkpoint
history. `Path.exists()` is False for a deleted directory, but it is equally
False for one whose storage is not attached right now: an unplugged external
drive, a share behind a downed VPN, a bind-mount absent from this container,
an offline Windows mapped drive. The project is fine; only our view of it is.
This is not an opt-in maintenance command. `maybe_auto_prune_checkpoints`
runs unattended at startup from both `cli.py` and `gateway/run.py`, with
`delete_orphans=True` by default. So starting Hermes once while the drive is
unplugged silently destroys the restore points for every project on it — the
one thing checkpoints exist to provide, and there is nothing to restore from
afterwards.
Reproduced against the real store: a project registered under an unmounted
path and one on local disk, then a startup prune —
prune: {'scanned': 2, 'deleted_orphan': 1}
unreachable project index still on disk: False
The legacy pre-v2 branch has the same flaw plus a second one: a
`HERMES_WORKDIR` marker that exists but cannot be read leaves `workdir = None`,
which the same condition treats as an orphan. Failing to read a file is not
evidence that a project was deleted.
Require corroboration before deleting: the workdir's parent must be present,
so its absence is something we actually observed. A missing parent means the
volume is not there and we know nothing, so the entry is left alone — and an
unreadable marker never deletes at all. Genuinely abandoned projects are still
reclaimed, both by the unchanged orphan path (parent present, project gone)
and by the retention/stale rule, which runs off `last_touch` rather than a
filesystem probe.
tests/tools/test_checkpoint_manager.py: a project whose whole mount disappears
keeps its history; controls prove a genuinely deleted project is still pruned
and a live project is untouched. The data-loss test fails on main; both
controls pass there. 81 passed across the checkpoint suites (2 failures in
test_checkpoint_manager.py are pre-existing and fail identically on clean
main).
export_session() reads through get_messages(), so before the read fix an
already-serialized string went straight back into _insert_message_rows() and
got re-dumped — an export/import round trip permanently corrupted the row.
Guard the three write paths the same way tool_calls already is: parse a
string argument before storing it, and drop metadata that isn't an object
rather than persisting something no reader can use.
Co-authored-by: xxxigm <xxxigm@users.noreply.github.com>
Co-authored-by: aml1973 <aml1973@users.noreply.github.com>
get_messages(), get_messages_around() and get_anchored_view() returned the
raw display_metadata column instead of the dict every caller expects. The
desktop paints a resumed transcript from the REST prefetch, which reads
through get_messages(), so any session holding an async_delegation_complete
event failed resume with "Cannot use 'in' operator to search for
'task_count'" — on every such session, not just corrupted ones.
Route all four read paths through one shared codec that also unwraps rows
carrying a second JSON layer, so sessions already broken on disk recover on
read rather than needing a migration.
Co-authored-by: Studio729 <Studio729@users.noreply.github.com>
Co-authored-by: aml1973 <aml1973@users.noreply.github.com>
Co-authored-by: xxxigm <xxxigm@users.noreply.github.com>
/api/status is the only public liveness route, and its handler loads the
gateway config, probes gateway health, and counts sessions before it can
answer. That work is wrong for a readiness probe: a caller that only needs
to know the process is up pays for a cold plugin import tree.
Add /api/health, which returns process liveness, version, and the auth-gate
shape and touches nothing else.
The bundled office skills (#68595) read user documents and agent-authored
payloads with the locale-default codec:
- docx/powerpoint validators/base.py opened OOXML part XML in text mode
before handing it to lxml. On Windows (cp1251/GBK) the bytes decode to
mojibake that lxml then parses, so validation runs against silently
corrupted document text; on locales where the UTF-8 bytes don't decode
the validator crashes with UnicodeDecodeError instead of validating.
Opening as bytes lets lxml honor the encoding declared in the XML prolog.
- The pdf form scripts (fill_fillable_fields, fill_pdf_form_with_annotations,
create_validation_image, check_bounding_boxes) read the fields JSON the
agent authors — UTF-8 by construction — with the locale codec, so
non-ASCII form values (any Cyrillic/CJK/accented input) either crash or
get written into the user's PDF as mojibake. The json.dump writers use
ensure_ascii=True and were already safe; only the readers needed pinning.
Adds a contract test asserting every document/payload reader is
locale-independent, plus a live regression test that runs
check_bounding_boxes.py on a non-ASCII fields.json under a forced
non-UTF-8 locale — it fails without the fix on both POSIX (C locale)
and Windows (cp1251 chokes on the 0x98 byte of U+2018).
_setup_worktree read both files with the locale default encoding. On a
cp1251/GBK Windows machine a UTF-8 include list either decodes to
mojibake paths (non-ASCII entries silently not copied) or raises
UnicodeDecodeError, which the enclosing handler logs at DEBUG and
swallows — no include is copied at all, so the worktree starts without
.env/keys and the agent breaks invisibly. A Notepad BOM likewise glues
to the first include entry on every platform, and to the first
.gitignore line, defeating the '.worktrees/' membership check and
appending a duplicate entry on each run.
Read both files with utf-8-sig + errors=replace, matching the canonical
.env readers in hermes_cli/config.py (utf-8-sig because Notepad adds a
BOM) and the UTF-8 append this same block already performs on
.gitignore.
Regression tests exercise the real cli._setup_worktree: the two BOM
tests fail without the fix on any platform, the non-ASCII include test
additionally reproduces the Windows locale failure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_read_hub_installed_names() reads ~/.hermes/skills/.hub/lock.json with a
strict utf-8 decode. Hub skill descriptions can carry Windows-1252
typographic bytes (em-dash 0x97, smart quotes, bullets) as single high
bytes; read_text(encoding="utf-8") then raises UnicodeDecodeError, which
is a ValueError sibling not caught by the function's
except (OSError, json.JSONDecodeError). It escapes and 500s the whole
/api/skills endpoint, blanking the desktop Skills panel.
Decode with errors="replace" so the offending byte degrades to U+FFFD
and the structurally valid JSON — and every other skill — stays readable.
Fixes#68053
`_render_distribution_plan` reads the target profile's `.env` to decide
whether a required env var is already set (so it doesn't nag the user),
using `Path.read_text()` with no encoding. Two bugs:
1. `Path.read_text()` defaults to the system locale (cp1251/GBK on Windows),
which raises `UnicodeDecodeError` on any non-ASCII byte. The surrounding
`except OSError` does NOT catch that — `UnicodeDecodeError` is a
`ValueError` — so a mis-encoded `.env` aborts the entire install preview.
2. Even on a UTF-8 locale, a Notepad-added BOM prefixes the first key
(`KEY`), so the very first required env var is mis-reported as
"needs setting" when it is actually present.
`.env` is written as UTF-8 everywhere in the codebase. Read it as
`utf-8-sig` (tolerates the BOM) and also catch `UnicodeDecodeError` so a
genuinely un-decodable file skips the pre-check instead of crashing.
Regression tests: a BOM-prefixed `.env` whose first key must still read as
"set", and an invalid-UTF-8 `.env` that must not abort the preview.
Follow-up to review feedback:
- mem0 _prompt_api_key read .env with the locale default, so a Notepad
BOM hid the first key from the masked current-value lookup; read it
with utf-8-sig + errors=replace like the canonical readers in
hermes_cli/config.py.
- hindsight _load_simple_env used plain utf-8; it also parses the Hermes
.env during post_setup, where a BOM stuck to the first key. Switch to
utf-8-sig + errors=replace.
- Add hindsight regressions: BOM key matching in _load_simple_env and in
the cloud post_setup writer, plus non-ASCII round-trip preservation,
and a mem0 regression for the BOM'd masked-key lookup. The BOM tests
fail without the fix on any platform.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mem0 and hindsight memory-provider setup routines round-trip the
user's ~/.hermes/.env: they read existing lines, update the keys they
manage, and rewrite the whole file preserving every other line verbatim.
Both used env_path.read_text() / write_text() with no encoding.
read_text()/write_text() with no encoding fall back to the system locale
(cp1252/GBK on Windows), so on a non-UTF-8 host the preserved lines get
mangled or the call crashes on any non-ASCII value, and — because the
reader never strips a BOM — a Notepad-edited .env makes the first key
fail the in-place match and get duplicated instead of updated.
Match the canonical .env readers in hermes_cli/config.py: read with
encoding='utf-8-sig' (BOM-tolerant) and write with encoding='utf-8'.
mem0/_setup.py already pins utf-8 for mem0.json, so this just aligns the
.env path in the same file. Fixes both memory plugins in one class fix.
Adds regression tests: a BOM'd .env updates the first key in place
(locale-independent, fails without the fix) and non-ASCII existing lines
survive the round-trip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses hermes-sweeper review on PR #54866: the installer runs
tools/skills_sync.py as a child python.exe whose PYTHONIOENCODING /
PYTHONUTF8 the scoped install.ps1 block sets, but there was no
regression test for this child-Python UTF-8 path. The existing
test_child_process_inherits_utf8_mode covers a different (bootstrap
entry-point) flow.
Add TestSkillsSyncUtf8Guard: three subprocess tests that import
skills_sync (triggering its import-time stdout/stderr reconfigure)
and assert the checkmark/up-arrow glyphs the script prints at
tools/skills_sync.py:596,675 emit valid UTF-8 and exit 0 even when
the child env is left unset or explicitly hostile (gbk). A third
test proves the guard is load-bearing by reproducing the crash
without it.
Also keep the new install.ps1 comment ASCII-only (the checkmark
spelled out as U+2713) per the file's PS 5.1 parser-compatibility
contract at scripts/install.ps1:79-80; the literal glyph in the
comment violated that contract.
Several file-I/O call sites still use open() / Path.read_text() /
Path.write_text() without an explicit encoding, so they fall back to
the platform default. On Windows CN/JP/KR locales (GBK/CP932/CP949)
any non-ASCII byte in a config/state/user-content file raises
UnicodeDecodeError or UnicodeEncodeError and crashes the caller.
to the remaining hot paths:
- agent/copilot_acp_client.py: fs/read_text_file and fs/write_text_file
(Copilot's read_file / write_file tools,
directly reported in #18637 bug 2)
- agent/model_metadata.py: context-length YAML cache load + two
save sites (context probing is on the
call path of every model invocation)
- agent/nous_rate_guard.py: cross-session rate-limit JSON state
(read + atomic write via os.fdopen)
- cron/scheduler.py: user config.yaml read in run_job
- gateway/delivery.py: cron output writes for AI-generated
content, very likely non-ASCII
yaml.dump call sites also gain allow_unicode=True so the emitted
YAML preserves non-ASCII chars as-is instead of emitting \u escape
sequences.
Adds regression tests that monkeypatch builtins.open / Path.read_text
/ Path.write_text to simulate a GBK locale: each test raises
UnicodeDecodeError / UnicodeEncodeError unless the caller explicitly
passes encoding='utf-8'. Verified that the tests fail on main and
pass with this change, on Linux as well as on Windows.
Refs #18637
