Staging re-test (2026-07-31, post-74482 image roll): auto-created
threads still stuck on their initial titles; connector telemetry shows
zero thread_rename ops. Root cause is an ordering flaw in the 74482
consume path: BOTH the title-callback registration gate and the
schedule gate read the send-result feedback cache
(_relay_auto_thread_info) — but registration runs BEFORE delivery on
the non-streaming lane, and the auto-title thread races delivery even
when registration survives. The cache read can only succeed AFTER the
connector answers the send, so the rename lane deterministically
disqualified itself on the title turn.
Fix — decide shape early, facts late:
- New _is_relay_discord_channel_lane: SHAPE-only predicate (relay
Discord channel event, no thread) used by the registration and
schedule gates; no cache read before delivery.
- _rename_discord_auto_thread_for_session_title: on the relay lane,
poll the adapter's feedback cache (0.5s ticks, ≤10s) — delivery is
typically right behind the title. True miss (connector didn't
auto-thread: policy off, DM, send failed) no-ops exactly as before.
Tests: shape-gate matrix; late-arriving feedback -> rename fires with
only_if_current_name guard; never-arriving feedback -> no-op. Relay
suite 174 passed.
Follow-up to PR #70238. Remove 'free-response channel' and
'authorization' jargon from the model-facing prompt. Collapse the
triple-negative 'do not ask / do not reject / do not stay silent'
into a single directive. ~70 tokens vs ~175 in the contributor's
version, same semantics.
Salvaged from #57016 by @lEWFkRAD:
- cli.py: handle file:///C:/... drive-letter URIs on nt (strip the
leading slash urlparse leaves); join Termux example paths with literal
forward slashes so hints stay POSIX on Windows.
- gateway/status.py + hermes_cli/gateway.py: normalize backslashes to
forward slashes before the HERMES_HOME substring match so separator
style cannot defeat profile ownership detection.
- hermes_cli/banner.py: cprint degrades to plain print when
prompt_toolkit has no console (NoConsoleScreenBufferError on
redirected/absent Windows stdout).
- hermes_cli/browser_connect.py: posixpath.join for WSL /mnt/c/... bases
(os.path.join would emit backslashes on nt).
- Test hardening: symlink skip-guards, USERPROFILE alongside HOME for
ntpath.expanduser, SIGKILL absence skipif fixed via monkeypatch,
drive-letter URI / separator-normalization / banner-fallback coverage.
Dropped from the original PR: tests/cli/conftest.py fixture and the
AppSession _output monkeypatch — main's merged tests/cli/conftest.py
already handles that prompt_toolkit pollution.
The file-local telegram mock in test_dm_topics.py installed unconditionally
(no __file__ guard), registered a separate string-valued telegram.constants
module, and force-popped the adapter — poisoning the session for any later
telegram test in the same process (assert 'MARKDOWN_V2' in "'MarkdownV2'").
Fix at the source:
- conftest: _FakeEnumMember(str) with PTB-faithful str()==value and
repr()==<ChatType.X: 'x'>, satisfying both repr assertions and the
adapter's str(chat.type) normalization; the same object is bound to
mod.ParseMode and mod.constants.ParseMode.
- test_dm_topics.py: delete the divergent local mock installer; import the
shared conftest one.
- release.py: mailmap entry for the author.
Verified: the 5-failure cluster repro (dm_topics + slash_confirm +
approval_buttons + model_picker + network_reconnect + telegram_format in one
process) goes 83/83 green (3x); full tests/gateway single-process run drops
10 -> 5 failed, the remainder being pre-existing discord order-dep failures
out of scope here.
Salvaged from #68873. Credit to @liuhao1024 for the earliest root-cause
diagnosis of this str-enum mock class in PR #33875, two months earlier.
Fixes the telegram-mock order-dependent flake cluster.
Salvaged from PR #73982 by Alexander Russell (@AlexxRussell) — the
collector half only. _collect_history_media_paths used only
_TOOL_MEDIA_RE, which misses quoted and spaced paths that the delivery
pipeline's extract_media grammar accepts; run text content through the
same extractor so the surviving dedup consumers (auto-append lane and
bare-path filter) see every path that could actually have been
delivered.
The PR's other halves (post-stream dedup snapshot plumbing, canonical
path comparison in _deliver_media_from_response, queued-followup
snapshot union) are moot after #74495 removed the post-stream history
filter entirely.
DEFAULT_DB_PATH in hermes_state.py is computed at import time, freezing
the developer's real ~/.hermes even when a test fixture (or runtime
profile switch) later redirects HERMES_HOME. Any default SessionDB() —
e.g. gateway SessionStore — then opened the real state.db.
Add _default_db_path(): resolves get_hermes_home() fresh at call time,
while a deliberately re-pointed DEFAULT_DB_PATH (the established
monkeypatch escape hatch) still wins via an import-time snapshot
comparison, preserving existing test behavior. SessionDB.__init__ and
session_search's requirement check now use the resolver; explicit
db_path arguments are untouched.
Reimplemented from PR #11875 by @JorkeyLiu (original diff predates the
hermes_state rewrite); regression test ported and modernized.
tui_gateway/server.py freezes _hermes_home at import time, so
_launch_configured_cwd() reads the developer's real config.yaml even
under the per-test HERMES_HOME redirect. Any absolute terminal.cwd in
the real config made _completion_cwd() ignore monkeypatch.chdir and
broke the completion tests on a pristine checkout.
Patch _launch_configured_cwd to None in the autouse _reset_fuzzy_cache
fixture and add a regression test pinning that _completion_cwd resolves
via os.getcwd() under tests.
Salvaged from PR #70148 by @smfworks (rebased onto the pruned suite).
The restart-routing, systemd-support, and subprocess-HOME tests asserted
branch behavior but left part of the real probe surface unmocked, so they
fail when the suite itself runs inside a container (self-hosted CI) or a
launchd-descended shell:
- /restart routing tests: the handler also consults the real /.dockerenv —
extract the inline probe to gateway.restart.is_container_restart_context()
(patchable seam, no behavior change) and pin it False; scrub ALL four
supervisor env markers (ambient XPC_SERVICE_NAME on macOS flipped one).
- supports_systemd_services tests: pin shutil.which('systemctl') and
is_container() so the test asserts the branch, not the host.
- copilot ACP real-HOME test: pin is_container() (auto mode prefers profile
home in containers) and scrub ambient HERMES_REAL_HOME/TERMINAL_HOME_MODE.
97 tests green on macOS dev box AND inside a docker CI runner container.
Co-authored-by: Kyzcreig <9063726+Kyzcreig@users.noreply.github.com>
Manual port of @jethac's #63522 onto the pruned tree: 3 of the original
6 fixed-budget poll sites survive (3-concurrent-agents wait, session-A/B
routing wait, two-session queue wait). Replaced each 2.5-5s hard-ceiling
poll loop with the PR's _wait_until(predicate) helper — generous 30s
ceiling reached only on genuine failure, instant return on the green
path, assert with message instead of silent fall-through.
Dropped: the 3 hunks targeting pruned code, and the unrelated
scripts/release.py mailmap hunk (AUTHOR_MAP is frozen; contributor
mapping handled via contributors/emails/).
Companion to the cherry-picked #74158 fix (non-streaming path):
- gateway/run.py: remove the identical history-dedup filter from
_deliver_media_from_response — the post-stream rescan is explicit-only
by design (#20834), so every MEDIA tag it finds is a deliberate
attachment request; drop the now-unused history_media_paths parameter
and its call-site plumbing.
- gateway/platforms/base.py: log suppressed bare local file paths on the
surviving local_files history dedup (#73771 observability ask).
- tests: focused regression file covering explicit resend delivery on
both lanes, current-turn tool-echo poisoning, surviving bare-path
dedup + its log line, and the upstream auto-append dedup invariant.
Fixes#73771
Two relay-lane bugs from live Discord staging testing (2026-07-29):
1. TTS audio never attached over relay (any platform, any lane).
_history_media_paths_for_session excluded only the trailing assistant
entry from the persisted transcript when building the delivered-media
dedup set. The agent persists rows as it produces them, so THIS turn's
text_to_speech tool result (media_tag JSON) was already in the
transcript at delivery time — the fresh TTS path deduped against
itself and extract_media's attachment was silently stripped
(response_delivery_dropped for a MEDIA-tag-only reply; fly logs show
the exact signature). Fix: exclude everything from the last USER
message onward (the current turn); prior-turn dedup unchanged.
Affects every platform adapter (native + relay) on the non-streaming
delivery path — the streaming path passes explicit history and was
unaffected.
2. Connector-auto-created threads never got the LLM session-title
rename. The title fires on the FIRST exchange, whose source is the
PARENT channel event — the thread didn't exist at ingest, so the
Phase 4 auto-thread markers can't be present and
_is_discord_auto_thread_lane never matches on the relay title turn
(initial titles worked; semantic renames never happened; staging
telemetry shows zero thread_rename ops ever sent). Fix: consume the
connector's new send-result feedback (paired gateway-gateway PR —
contract §SendResult thread_id/auto_thread_name, additive):
RelayAdapter.send() caches (thread_id, initial_name) per chat
(bounded 256), run.py's title-callback registration + rename lane
read it back and pass initial_name as only_if_current_name so the
human-rename-wins guard holds on the relay lane too. Native marker
path unchanged; connectors that don't stamp the fields degrade to
exactly the old behavior.
Tests: 3 new (send-result feedback capture, absence, bound) in
test_relay_threads.py; 3 new in test_history_media_current_turn.py
(current-turn TTS not deduped, prior-turn still deduped, no-user-row
fallback). Relay suite 144 passed.
Telegram clients split messages above 4096 chars into multiple updates. A
long '/queue <prompt>' paste arrives as a COMMAND chunk near the limit plus
plain TEXT continuation chunk(s). _handle_command dispatched the command
chunk immediately, so the continuation landed as a separate plain message
that interrupted the running agent instead of being queued.
Near-limit (>= _SPLIT_THRESHOLD) command chunks now route through the same
text-batching pipeline used for split plain-text messages, merging the
continuation before dispatch. Short commands (/stop, /approve, ...) keep the
immediate path and are never delayed.
The gateway keeps one PairingStore per served profile, but every
`/api/pairing` endpoint built the global one. An operator managing a named
profile saw the wrong pending list, and approving wrote a grant into a
whitelist their running gateway never consults — the user stays locked out
while the UI shows them as approved.
`_pairing_store(profile)` now resolves per profile and validates the name
(400/404 on an unknown one). No `_profile_scope` needed: PairingStore
resolves the profile's home itself, so nothing process-global is swapped
across an await.
Both GUIs had to change to match. The listing rides the query param — for
the dashboard that meant deleting `pairing` from the "machine-global, must
NOT be rewritten" exclusion list, a comment this change makes false. The
mutating endpoints read the profile off the BODY, which no query-param
rewrite reaches, so approve/revoke send it explicitly on both surfaces.
The per-profile pairing isolation added self._dir and scoped every
per-file path helper (_pending_path, _approved_path) to it, but
_all_platforms still enumerated the module-global PAIRING_DIR. For a
profile-scoped PairingStore, list_approved/list_pending/clear_pending
therefore operated on the GLOBAL platform set while loading each
platform's file from the PROFILE dir — so list_approved() returned []
for a user that is_approved() confirmed as approved, a silent divergence
between the authz surface and the list/inspect/clear surface.
Route discovery through self._dir. Byte-identical for the global store
(self._dir == PAIRING_DIR when no profile is set); only the buggy
profile-scoped case changes. self._dir is guaranteed to exist (__init__
mkdirs it).
Follow-up hardening on the request-id grant path.
approve_request took the same lockout treatment as approve_code: gated by
it, and recording a miss toward it. But the two paths defend different
things. The lockout exists to stop guessing at the 8-char code space over a
messaging channel; a request id is only ever obtained by an admin already
authenticated to the store, so a miss means the row they clicked went stale.
Counting those let a handful of clicks on a stale list lock the operator out
of `hermes pairing approve` for an hour — the GUI DoSing the CLI.
Also drops the `code`/`code_hash_prefix` compat fields from list_pending.
The hash prefix is what admin surfaces mistook for an approvable code in the
first place, and re-exporting the request id under the old `code` key just
preserves the ambiguity; both consumers in the tree read `request_id` now.
The 16-hex sniffing that had been copy-pasted into the CLI and the endpoint
(where a chained conditional consulted it against the wrong field) moves to
one owner, PairingStore.looks_like_request_id.
The endpoint no longer reports a 429 on the request-id path, where lockout
can't apply — a stale id surfaced as a bogus "locked out" while the platform
sat locked for something else entirely.
approve_code()'s success path never cleared _failures:{platform}. The
counter is incremented on every non-matching code, persisted in
_rate_limits.json, and only ever reset to 0 when it reaches
MAX_FAILED_ATTEMPTS (firing the lockout). So it counts failures over the
gateway's entire lifetime, not consecutive ones.
An owner who mistypes a pairing code on a handful of separate occasions
— each time immediately retyping it correctly and successfully pairing —
accumulates those isolated typos. A later single fresh typo then hits
MAX_FAILED_ATTEMPTS and locks the whole platform out for an hour, at
which point _is_locked_out gates approve_code and even the *correct*
code is rejected.
Reset the counter on a successful approval, matching standard
brute-force-guard semantics (the counter tracks consecutive failures).
This does not weaken protection: an attacker cannot produce a success
without a valid code, and 5 consecutive wrong attempts still lock out.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Remove tests/-shadowing sys.path.insert(dirname/'..') from 11 test files:
it prepended the tests/ dir itself to sys.path, so 'import agent' /
'import hermes_cli' resolved to the test packages and collection died
with ModuleNotFoundError depending on import order (2 files failed in
every full-suite run; 9 more were latent).
- Patch call_llm in 5 context-compressor tests that called compress()
unmocked: each burned ~50s attempting live LLM traffic through the
relay before falling back (572s file — the slowest in the suite, and
flaky under the 300s per-file timeout). File now runs in ~5s.
- agent/redact.py: fix two catastrophically-backtracking regexes hit by
the compressor's redaction pass on large payloads —
_STRICT_URL_USERINFO_RE anchors on the mandatory '//' (optional-scheme
prefix backtracked O(n^2): ~55s on a 320KB payload, now sub-ms;
output-equivalence fuzz-verified on 20k random strings), and the
_CFG_DOTTED_RE/_CFG_ANCHORED_RE subs gain an exact linear keyword
pre-gate so secret-free text skips the quadratic pattern entirely.
- tests/gateway/test_feishu.py: version-guard the extra_ua_tags SDK
signature check; the repo pins lark-oapi==1.6.8 but stale local
installs (1.5.3) fail the assertion — skip below the pin.
- tests/tools/test_managed_browserbase_and_modal.py: stub
agent.redact + agent.credential_persistence in the fake agent package
(empty __path__ blocks all real agent.* imports added since the fake
was written).
- tests/gateway/test_startup_restart_race.py: raise wait_for timeouts
2s -> 30s; 2s wall-clock on a loaded 40-worker box flaked in the
baseline run (passes instantly when the box is quiet).
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
- Remove tests/-shadowing sys.path.insert(dirname/'..') from 11 test files:
it prepended the tests/ dir itself to sys.path, so 'import agent' /
'import hermes_cli' resolved to the test packages and collection died
with ModuleNotFoundError depending on import order (2 files failed in
every full-suite run; 9 more were latent).
- Patch call_llm in 5 context-compressor tests that called compress()
unmocked: each burned ~50s attempting live LLM traffic through the
relay before falling back (572s file — the slowest in the suite, and
flaky under the 300s per-file timeout). File now runs in ~5s.
- agent/redact.py: fix two catastrophically-backtracking regexes hit by
the compressor's redaction pass on large payloads —
_STRICT_URL_USERINFO_RE anchors on the mandatory '//' (optional-scheme
prefix backtracked O(n^2): ~55s on a 320KB payload, now sub-ms;
output-equivalence fuzz-verified on 20k random strings), and the
_CFG_DOTTED_RE/_CFG_ANCHORED_RE subs gain an exact linear keyword
pre-gate so secret-free text skips the quadratic pattern entirely.
- tests/gateway/test_feishu.py: version-guard the extra_ua_tags SDK
signature check; the repo pins lark-oapi==1.6.8 but stale local
installs (1.5.3) fail the assertion — skip below the pin.
- tests/tools/test_managed_browserbase_and_modal.py: stub
agent.redact + agent.credential_persistence in the fake agent package
(empty __path__ blocks all real agent.* imports added since the fake
was written).
- tests/gateway/test_startup_restart_race.py: raise wait_for timeouts
2s -> 30s; 2s wall-clock on a loaded 40-worker box flaked in the
baseline run (passes instantly when the box is quiet).
The earlier scrub covered adapter.py; nine internal QA-campaign tracker IDs
remained in the two new test files, including a module docstring and an
assertion message. They mean nothing to a future reader — describe the
behavior instead. Comments only, no assertion changes.
Both relay Slack knobs read their value through bool(), while the native
adapter they mirror uses str(raw).strip().lower() in {"1","true","yes","on"}.
A YAML-quoted string diverges:
dm_top_level_threads_as_sessions: "false" → relay True, native False
Non-empty strings are truthy, so the escape hatch is silently ignored in
exactly the shape an operator writes to switch it OFF. reply_in_thread has the
same defect and gates reply placement, session keying and run.py's progress
resolver, so one quoted "false" misfires three ways.
Route both through a shared _coerce_flag mirroring native's predicate. Real
booleans pass through untouched; None falls back to the default. Contract §8
documents the accepted spellings.
Tests: both knobs parametrized over the true/false spellings native accepts,
plus the absent-key default.
The DM thread-anchor contract was resolved only in send(). _send_media() —
backing send_image, send_image_file, send_voice, send_video and send_document
— passed reply_to straight to the frame and never touched metadata, so
attachments egressing through the same connector-side Slack sender got both
failure shapes this branch set out to remove:
flat mode → reply_to survives, the image threads UNDER the user's DM
message (the original reported symptom)
thread mode → no metadata.thread_id, and threadTs() never reads reply_to,
so the image lands in the home channel instead of the
per-message thread
Both are reachable: gateway/run.py delivers agent artifacts through
send_voice/send_document.
Extract the three steps that must always happen together (mode gate, mirrored
reply_to_message_id strip, metadata promotion) into
_apply_slack_thread_anchor and route BOTH lanes through it, so text and media
cannot drift again. The media lane copies caller metadata rather than mutating
it — these helpers are called in loops with a shared mapping.
Also fold send_typing/stop_typing's duplicated status-anchor blocks into
_with_status_thread_anchor. They had already drifted (stop_typing omitted the
platform check) and the clear must target the thread the heartbeat set or the
status line sticks until Slack's own timeout.
Tests: media lane pinned in both modes plus the channel and
no-caller-mutation cases; verified as real by reverting the fix and watching
them fail.
The gateway's /reset cleanup path called shutdown_memory_provider without
first draining the memory manager's serialized background write worker.
shutdown_all only gives that worker a bounded (~5s) drain and abandons
whatever is still queued past it, so a /reset could silently drop writes
the session had already handed off -- the next session then loaded
stale MEMORY.md.
The CLI exit path already drains via MemoryManager.flush_pending before
shutdown; this PR pins the same contract on the gateway cleanup path.
Cleanup now calls agent._memory_manager.flush_pending(timeout=10) before
the existing shutdown_memory_provider step. The flush is best-effort:
a flush failure must never block teardown, so it is wrapped in
try/except and the existing shutdown path remains the fallback.
Closes#73297
Follow-up for salvaged PRs #73400 + #73372:
- Move _preserve_agent_history_on_shutdown logic into
gateway/shutdown_flush.py as flush_agent_history_to_file()
- Reuse the hardened _write_payload (atomic writes, fsync, private
permissions, UUID filenames) from #73372 instead of the plain
open()/write() from #73400
- Replace os.environ.get('HERMES_HOME') with get_hermes_home() for
profile-safe path resolution
- Update tests to test the real function directly
- Net: removes 37 lines from gateway/run.py, unifies both fix paths
in a single module with consistent atomic-write guarantees
The previous attempt (#73171) snapshotted GatewayRunner._pending_messages,
which on current main has no writers (commit f6736ced8 removed its write
path; interrupt delivery uses adapter._pending_messages instead). The live
container is the per-agent agent._session_messages, flushed via
_flush_messages_to_session_db. When that flush raises (FTS/SQLite corruption,
the disk=0/memory=N state from #72680), the in-memory transcript is lost when
the process exits.
Retarget the preservation to the real path: in _finalize_shutdown_agents, wrap
the _flush call; on exception, dump agent._session_messages to an external JSON
recovery snapshot under $HERMES_HOME/shutdown-recovery/ (tagged issue=#72680)
so an operator can salvage it after repairing state.db. The dump is fully
guarded (non-fatal) so shutdown never blocks on a best-effort backup.
This directly addresses the reviewer note on #73171: retarget to the actual
cached-agent history (agent._session_messages) and prove a stale DB + shutdown
leaves a recoverable transcript.
Regression tests: tests/gateway/test_session_messages_shutdown_preserve.py
- flush raises -> recovery file written with session_id + messages
- healthy flush -> no recovery file
- write error -> non-fatal, no raise
Fixes#72680
Replace 20 lines of manual os.open/O_EXCL/fdopen/fsync/os.replace with the
existing atomic_json_write() from utils.py, which is already used by 6+
modules and handles temp-file creation, fsync, atomic replace, mode
control, and owner preservation. The only novel helper (_fsync_directory)
is retained — atomic_json_write does not do directory fsync.
Update test_flush_write_failure_leaves_no_recovery_file to monkeypatch
utils.os.replace (the new call path) instead of gateway.shutdown_flush.os.replace.
test_default_bind_serves_both_families asserts an IPv6 listening socket
exists, which is the environment's capability, not the adapter's — CI
runners with IPv6 disabled would flake. Probe an actual ::1 bind (not
just socket.has_ipv6, a compile-time constant) and skip cleanly when
the host has no usable IPv6 stack.
