Commit graph

2362 commits

Author SHA1 Message Date
Fraser Humphries
4dc9fb8008 fix(gateway): exclude current-turn media from dedup 2026-07-29 19:11:05 -07:00
Jorkey Liu
05afea65f4 fix(session): resolve default state DB path at call time
DEFAULT_DB_PATH in hermes_state.py is computed at import time, freezing
the developer's real ~/.hermes even when a test fixture (or runtime
profile switch) later redirects HERMES_HOME. Any default SessionDB() —
e.g. gateway SessionStore — then opened the real state.db.

Add _default_db_path(): resolves get_hermes_home() fresh at call time,
while a deliberately re-pointed DEFAULT_DB_PATH (the established
monkeypatch escape hatch) still wins via an import-time snapshot
comparison, preserving existing test behavior. SessionDB.__init__ and
session_search's requirement check now use the resolver; explicit
db_path arguments are untouched.

Reimplemented from PR #11875 by @JorkeyLiu (original diff predates the
hermes_state rewrite); regression test ported and modernized.
2026-07-29 18:55:10 -07:00
Jasmine Naderi
3e54a366e7 fix(test): patch _launch_configured_cwd in completion tests for hermeticity (#70041)
tui_gateway/server.py freezes _hermes_home at import time, so
_launch_configured_cwd() reads the developer's real config.yaml even
under the per-test HERMES_HOME redirect. Any absolute terminal.cwd in
the real config made _completion_cwd() ignore monkeypatch.chdir and
broke the completion tests on a pristine checkout.

Patch _launch_configured_cwd to None in the autouse _reset_fuzzy_cache
fixture and add a regression test pinning that _completion_cwd resolves
via os.getcwd() under tests.

Salvaged from PR #70148 by @smfworks (rebased onto the pruned suite).
2026-07-29 18:55:10 -07:00
Kyzcreig
bcec6c8d39 fix(test): hermetic env-detection tests — pin container/supervisor/HOME probes (#422)
The restart-routing, systemd-support, and subprocess-HOME tests asserted
branch behavior but left part of the real probe surface unmocked, so they
fail when the suite itself runs inside a container (self-hosted CI) or a
launchd-descended shell:

- /restart routing tests: the handler also consults the real /.dockerenv —
  extract the inline probe to gateway.restart.is_container_restart_context()
  (patchable seam, no behavior change) and pin it False; scrub ALL four
  supervisor env markers (ambient XPC_SERVICE_NAME on macOS flipped one).
- supports_systemd_services tests: pin shutil.which('systemctl') and
  is_container() so the test asserts the branch, not the host.
- copilot ACP real-HOME test: pin is_container() (auto mode prefers profile
  home in containers) and scrub ambient HERMES_REAL_HOME/TERMINAL_HOME_MODE.

97 tests green on macOS dev box AND inside a docker CI runner container.

Co-authored-by: Kyzcreig <9063726+Kyzcreig@users.noreply.github.com>
2026-07-29 18:55:10 -07:00
jethac
2c37b1af25 test(approval): event-based waits for blocking-approval E2E polls (PR #63522 port)
Manual port of @jethac's #63522 onto the pruned tree: 3 of the original
6 fixed-budget poll sites survive (3-concurrent-agents wait, session-A/B
routing wait, two-session queue wait). Replaced each 2.5-5s hard-ceiling
poll loop with the PR's _wait_until(predicate) helper — generous 30s
ceiling reached only on genuine failure, instant return on the green
path, assert with message instead of silent fall-through.

Dropped: the 3 hunks targeting pruned code, and the unrelated
scripts/release.py mailmap hunk (AUTHOR_MAP is frozen; contributor
mapping handled via contributors/emails/).
2026-07-29 18:55:10 -07:00
Teknium
646761c783 fix(gateway): widen explicit-MEDIA resend fix to the streaming path + log bare-path suppression
Companion to the cherry-picked #74158 fix (non-streaming path):

- gateway/run.py: remove the identical history-dedup filter from
  _deliver_media_from_response — the post-stream rescan is explicit-only
  by design (#20834), so every MEDIA tag it finds is a deliberate
  attachment request; drop the now-unused history_media_paths parameter
  and its call-site plumbing.
- gateway/platforms/base.py: log suppressed bare local file paths on the
  surviving local_files history dedup (#73771 observability ask).
- tests: focused regression file covering explicit resend delivery on
  both lanes, current-turn tool-echo poisoning, surviving bare-path
  dedup + its log line, and the upstream auto-append dedup invariant.

Fixes #73771
2026-07-29 18:33:38 -07:00
Ben Barclay
d26983e485
fix(gateway): relay TTS attachments + semantic auto-thread rename on the title turn (#74482)
Two relay-lane bugs from live Discord staging testing (2026-07-29):

1. TTS audio never attached over relay (any platform, any lane).
   _history_media_paths_for_session excluded only the trailing assistant
   entry from the persisted transcript when building the delivered-media
   dedup set. The agent persists rows as it produces them, so THIS turn's
   text_to_speech tool result (media_tag JSON) was already in the
   transcript at delivery time — the fresh TTS path deduped against
   itself and extract_media's attachment was silently stripped
   (response_delivery_dropped for a MEDIA-tag-only reply; fly logs show
   the exact signature). Fix: exclude everything from the last USER
   message onward (the current turn); prior-turn dedup unchanged.
   Affects every platform adapter (native + relay) on the non-streaming
   delivery path — the streaming path passes explicit history and was
   unaffected.

2. Connector-auto-created threads never got the LLM session-title
   rename. The title fires on the FIRST exchange, whose source is the
   PARENT channel event — the thread didn't exist at ingest, so the
   Phase 4 auto-thread markers can't be present and
   _is_discord_auto_thread_lane never matches on the relay title turn
   (initial titles worked; semantic renames never happened; staging
   telemetry shows zero thread_rename ops ever sent). Fix: consume the
   connector's new send-result feedback (paired gateway-gateway PR —
   contract §SendResult thread_id/auto_thread_name, additive):
   RelayAdapter.send() caches (thread_id, initial_name) per chat
   (bounded 256), run.py's title-callback registration + rename lane
   read it back and pass initial_name as only_if_current_name so the
   human-rename-wins guard holds on the relay lane too. Native marker
   path unchanged; connectors that don't stamp the fields degrade to
   exactly the old behavior.

Tests: 3 new (send-result feedback capture, absence, bound) in
test_relay_threads.py; 3 new in test_history_media_current_turn.py
(current-turn TTS not deduped, prior-turn still deduped, no-user-row
fallback). Relay suite 144 passed.
2026-07-29 18:00:18 -07:00
Teknium
240afd0b70 fix(telegram): batch near-limit command chunks so split /queue pastes don't orphan their continuation
Telegram clients split messages above 4096 chars into multiple updates. A
long '/queue <prompt>' paste arrives as a COMMAND chunk near the limit plus
plain TEXT continuation chunk(s). _handle_command dispatched the command
chunk immediately, so the continuation landed as a separate plain message
that interrupted the running agent instead of being queued.

Near-limit (>= _SPLIT_THRESHOLD) command chunks now route through the same
text-batching pipeline used for split plain-text messages, merging the
continuation before dispatch. Short commands (/stop, /approve, ...) keep the
immediate path and are never delayed.
2026-07-29 17:22:52 -07:00
brooklyn!
08e428ac8b
Merge pull request #74446 from NousResearch/bb/desktop-pairing
feat(pairing): profile-correct approvals, and a desktop surface to do them from
2026-07-29 19:04:36 -05:00
teknium1
4b33e5663b refactor: config auto-migration support floor at v12 + deprecated shim retirement 2026-07-29 16:44:31 -07:00
Brooklyn Nicholson
f174c0b6bb fix(pairing): scope the approve/revoke endpoints to a profile
The gateway keeps one PairingStore per served profile, but every
`/api/pairing` endpoint built the global one. An operator managing a named
profile saw the wrong pending list, and approving wrote a grant into a
whitelist their running gateway never consults — the user stays locked out
while the UI shows them as approved.

`_pairing_store(profile)` now resolves per profile and validates the name
(400/404 on an unknown one). No `_profile_scope` needed: PairingStore
resolves the profile's home itself, so nothing process-global is swapped
across an await.

Both GUIs had to change to match. The listing rides the query param — for
the dashboard that meant deleting `pairing` from the "machine-global, must
NOT be rewritten" exclusion list, a comment this change makes false. The
mutating endpoints read the profile off the BODY, which no query-param
rewrite reaches, so approve/revoke send it explicitly on both surfaces.
2026-07-29 18:43:50 -05:00
wgu9
a6397c379b fix(gateway): align multiplex pairing stores
Co-authored-by: x7peeps <x7peeps@users.noreply.github.com>
2026-07-29 18:43:50 -05:00
briandevans
75657f89c4 test(gateway): patch hermes_constants.get_hermes_home so profile store scopes to the mocked home 2026-07-29 18:43:49 -05:00
briandevans
4cbd46545e fix(gateway): scope pairing platform discovery to the profile dir
The per-profile pairing isolation added self._dir and scoped every
per-file path helper (_pending_path, _approved_path) to it, but
_all_platforms still enumerated the module-global PAIRING_DIR. For a
profile-scoped PairingStore, list_approved/list_pending/clear_pending
therefore operated on the GLOBAL platform set while loading each
platform's file from the PROFILE dir — so list_approved() returned []
for a user that is_approved() confirmed as approved, a silent divergence
between the authz surface and the list/inspect/clear surface.

Route discovery through self._dir. Byte-identical for the global store
(self._dir == PAIRING_DIR when no profile is set); only the buggy
profile-scoped case changes. self._dir is guaranteed to exist (__init__
mkdirs it).
2026-07-29 18:43:49 -05:00
teknium1
1a3a9de630 refactor(gateway): extract run_sync onto TurnRunner (completes the TurnContext seam; AST-identical body) 2026-07-29 16:21:46 -07:00
Brooklyn Nicholson
37d0766ba5 fix(pairing): keep GUI approvals off the code brute-force lockout
Follow-up hardening on the request-id grant path.

approve_request took the same lockout treatment as approve_code: gated by
it, and recording a miss toward it. But the two paths defend different
things. The lockout exists to stop guessing at the 8-char code space over a
messaging channel; a request id is only ever obtained by an admin already
authenticated to the store, so a miss means the row they clicked went stale.
Counting those let a handful of clicks on a stale list lock the operator out
of `hermes pairing approve` for an hour — the GUI DoSing the CLI.

Also drops the `code`/`code_hash_prefix` compat fields from list_pending.
The hash prefix is what admin surfaces mistook for an approvable code in the
first place, and re-exporting the request id under the old `code` key just
preserves the ambiguity; both consumers in the tree read `request_id` now.
The 16-hex sniffing that had been copy-pasted into the CLI and the endpoint
(where a chained conditional consulted it against the wrong field) moves to
one owner, PairingStore.looks_like_request_id.

The endpoint no longer reports a 429 on the request-id path, where lockout
can't apply — a stale id surfaced as a bogus "locked out" while the platform
sat locked for something else entirely.
2026-07-29 17:51:40 -05:00
solyanviktor-star
c352a322b0 fix(pairing): reset the failed-approval counter on a successful approval
approve_code()'s success path never cleared _failures:{platform}. The
counter is incremented on every non-matching code, persisted in
_rate_limits.json, and only ever reset to 0 when it reaches
MAX_FAILED_ATTEMPTS (firing the lockout). So it counts failures over the
gateway's entire lifetime, not consecutive ones.

An owner who mistypes a pairing code on a handful of separate occasions
— each time immediately retyping it correctly and successfully pairing —
accumulates those isolated typos. A later single fresh typo then hits
MAX_FAILED_ATTEMPTS and locks the whole platform out for an hour, at
which point _is_locked_out gates approve_code and even the *correct*
code is rejected.

Reset the counter on a successful approval, matching standard
brute-force-guard semantics (the counter tracks consecutive failures).
This does not weaken protection: an attacker cannot produce a success
without a valid code, and 5 consecutive wrong attempts still lock out.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 17:49:33 -05:00
Flownium
774d92fbfa fix: approve listed pairing requests 2026-07-29 17:49:33 -05:00
Teknium
7c5a98d888
Merge remote-tracking branch 'origin/main' into tests/prune-low-value
# Conflicts:
#	tests/run_agent/test_conversation_fallback_state.py
2026-07-29 15:24:14 -07:00
Teknium
fc551f9691
Merge pull request #72103 from victor-kyriazakos/feat/relay-slack-blockkit-native-parity
fix(relay): Slack DM-root prompts + flat-DM edit-streaming (native _resolve_thread_ts parity)
2026-07-29 15:13:42 -07:00
Teknium
28524adb0e fix(tests): eliminate flaky/broken tests — shadow sys.path inserts, unmocked network in compressor tests, stale-SDK feishu pin guard, quadratic redact regexes
- Remove tests/-shadowing sys.path.insert(dirname/'..') from 11 test files:
  it prepended the tests/ dir itself to sys.path, so 'import agent' /
  'import hermes_cli' resolved to the test packages and collection died
  with ModuleNotFoundError depending on import order (2 files failed in
  every full-suite run; 9 more were latent).
- Patch call_llm in 5 context-compressor tests that called compress()
  unmocked: each burned ~50s attempting live LLM traffic through the
  relay before falling back (572s file — the slowest in the suite, and
  flaky under the 300s per-file timeout). File now runs in ~5s.
- agent/redact.py: fix two catastrophically-backtracking regexes hit by
  the compressor's redaction pass on large payloads —
  _STRICT_URL_USERINFO_RE anchors on the mandatory '//' (optional-scheme
  prefix backtracked O(n^2): ~55s on a 320KB payload, now sub-ms;
  output-equivalence fuzz-verified on 20k random strings), and the
  _CFG_DOTTED_RE/_CFG_ANCHORED_RE subs gain an exact linear keyword
  pre-gate so secret-free text skips the quadratic pattern entirely.
- tests/gateway/test_feishu.py: version-guard the extra_ua_tags SDK
  signature check; the repo pins lark-oapi==1.6.8 but stale local
  installs (1.5.3) fail the assertion — skip below the pin.
- tests/tools/test_managed_browserbase_and_modal.py: stub
  agent.redact + agent.credential_persistence in the fake agent package
  (empty __path__ blocks all real agent.* imports added since the fake
  was written).
- tests/gateway/test_startup_restart_race.py: raise wait_for timeouts
  2s -> 30s; 2s wall-clock on a loaded 40-worker box flaked in the
  baseline run (passes instantly when the box is quiet).
2026-07-29 15:12:28 -07:00
Teknium
40ec417881
Merge remote-tracking branch 'origin/main' into tests/prune-low-value
# Conflicts:
#	tests/hermes_cli/test_install_cua_driver.py
#	tests/run_agent/test_codex_app_server_integration.py
#	tests/test_tui_gateway_server.py
#	tests/tools/test_computer_use_delivery_ladder.py
#	tests/tools/test_zombie_process_cleanup.py
2026-07-29 14:12:25 -07:00
Teknium
39975613b1
test: prune wave 2 + speed fixes — 28,106 → 19,757 test functions, suite wall 315s → 294s
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.

Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
  (DEFAULT_CONFIG smart-approval leaked in) — pinned approval
  mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
  silently probed live provider catalogs (~2s/test) — stubbed
  cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
  shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
  MagicMock agents — interrupt.side_effect now clears _running_agents
  (22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
  (24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
  patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
  5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
  with rationale comment — the watchdog-dump assertion needs the loop
  blocked well past deadline+grace under parallel load (flaked once in
  the 40-worker verification run at a 0.2s margin).

Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
2026-07-29 13:39:40 -07:00
teknium1
a022229566 refactor(gateway): TurnContext/TurnRunner seam — extract _run_agent_inner nested closures (byte-identical bodies) 2026-07-29 13:34:03 -07:00
Teknium
6b81590c55
test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions
Systematic prune per AGENTS.md test policy, one pass over every major
test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli,
cron, tui_gateway, honcho/openviking, root-level):

- DELETE: source-reading tests (read_text/getsource on prod files),
  change-detector tests (exact catalog counts, model-name snapshots,
  config version literals), mock-echo tests (assert a mock returns what
  it was told), assertion-free/trivial tests, near-duplicate
  parametrizations (boundaries + one representative kept), async/sync
  twin duplicates, cosmetic within-file variations.
- KEEP (mandatory): security/redaction/approval guards, message-role
  alternation invariants, prompt-caching/deterministic-call-id
  invariants, issue-number regression tests (deduped), E2E tests.
- 6 test files deleted outright (script-style/no-assert or fully
  redundant); conftest.py, fakes/, fixtures/ untouched.
- tests/acp/conftest.py added: autouse fixture stubs the live
  models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server
  tests performed on every session create — test_server.py 147s → 3.4s,
  and the tests are now genuinely hermetic.
- Sleep-based slowness shrunk where safe (codex_ttfb_watchdog,
  compression_concurrent_fork, etc.); no wall-clock assertion tightened.

Verification: full hermetic suite via scripts/run_tests.sh —
2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall
(baseline: 583s wall, 13,564s subprocess CPU).
2026-07-29 13:10:23 -07:00
Teknium
3dd8059a05
fix(tests): eliminate flaky/broken tests — shadow sys.path inserts, unmocked network in compressor tests, stale-SDK feishu pin guard, quadratic redact regexes
- Remove tests/-shadowing sys.path.insert(dirname/'..') from 11 test files:
  it prepended the tests/ dir itself to sys.path, so 'import agent' /
  'import hermes_cli' resolved to the test packages and collection died
  with ModuleNotFoundError depending on import order (2 files failed in
  every full-suite run; 9 more were latent).
- Patch call_llm in 5 context-compressor tests that called compress()
  unmocked: each burned ~50s attempting live LLM traffic through the
  relay before falling back (572s file — the slowest in the suite, and
  flaky under the 300s per-file timeout). File now runs in ~5s.
- agent/redact.py: fix two catastrophically-backtracking regexes hit by
  the compressor's redaction pass on large payloads —
  _STRICT_URL_USERINFO_RE anchors on the mandatory '//' (optional-scheme
  prefix backtracked O(n^2): ~55s on a 320KB payload, now sub-ms;
  output-equivalence fuzz-verified on 20k random strings), and the
  _CFG_DOTTED_RE/_CFG_ANCHORED_RE subs gain an exact linear keyword
  pre-gate so secret-free text skips the quadratic pattern entirely.
- tests/gateway/test_feishu.py: version-guard the extra_ua_tags SDK
  signature check; the repo pins lark-oapi==1.6.8 but stale local
  installs (1.5.3) fail the assertion — skip below the pin.
- tests/tools/test_managed_browserbase_and_modal.py: stub
  agent.redact + agent.credential_persistence in the fake agent package
  (empty __path__ blocks all real agent.* imports added since the fake
  was written).
- tests/gateway/test_startup_restart_race.py: raise wait_for timeouts
  2s -> 30s; 2s wall-clock on a loaded 40-worker box flaked in the
  baseline run (passes instantly when the box is quiet).
2026-07-29 12:18:07 -07:00
Ben Barclay
b521fd9dc9 docs(relay): finish the QA-N scrub in the new relay test files
The earlier scrub covered adapter.py; nine internal QA-campaign tracker IDs
remained in the two new test files, including a module docstring and an
assertion message. They mean nothing to a future reader — describe the
behavior instead. Comments only, no assertion changes.
2026-07-29 12:03:01 -07:00
Ben Barclay
0dc293f4f7 fix(relay): coerce Slack behavior flags exactly as the native adapter does
Both relay Slack knobs read their value through bool(), while the native
adapter they mirror uses str(raw).strip().lower() in {"1","true","yes","on"}.
A YAML-quoted string diverges:

    dm_top_level_threads_as_sessions: "false"   → relay True, native False

Non-empty strings are truthy, so the escape hatch is silently ignored in
exactly the shape an operator writes to switch it OFF. reply_in_thread has the
same defect and gates reply placement, session keying and run.py's progress
resolver, so one quoted "false" misfires three ways.

Route both through a shared _coerce_flag mirroring native's predicate. Real
booleans pass through untouched; None falls back to the default. Contract §8
documents the accepted spellings.

Tests: both knobs parametrized over the true/false spellings native accepts,
plus the absent-key default.
2026-07-29 12:03:01 -07:00
Ben Barclay
33833e232e fix(relay): apply the Slack thread anchor on the media lane too
The DM thread-anchor contract was resolved only in send(). _send_media() —
backing send_image, send_image_file, send_voice, send_video and send_document
— passed reply_to straight to the frame and never touched metadata, so
attachments egressing through the same connector-side Slack sender got both
failure shapes this branch set out to remove:

  flat mode   → reply_to survives, the image threads UNDER the user's DM
                message (the original reported symptom)
  thread mode → no metadata.thread_id, and threadTs() never reads reply_to,
                so the image lands in the home channel instead of the
                per-message thread

Both are reachable: gateway/run.py delivers agent artifacts through
send_voice/send_document.

Extract the three steps that must always happen together (mode gate, mirrored
reply_to_message_id strip, metadata promotion) into
_apply_slack_thread_anchor and route BOTH lanes through it, so text and media
cannot drift again. The media lane copies caller metadata rather than mutating
it — these helpers are called in loops with a shared mapping.

Also fold send_typing/stop_typing's duplicated status-anchor blocks into
_with_status_thread_anchor. They had already drifted (stop_typing omitted the
platform check) and the clear must target the thread the heartbeat set or the
status line sticks until Slack's own timeout.

Tests: media lane pinned in both modes plus the channel and
no-caller-mutation cases; verified as real by reverting the fix and watching
them fail.
2026-07-29 12:03:00 -07:00
teknium1
bd93ccb890 refactor(gateway): shared fence-aware markdown chunker core (yuanbao-derived) + canonical table-row splitter 2026-07-29 11:53:59 -07:00
Gille
6d292fd5eb fix(discord): retry slash sync after failed fingerprint 2026-07-29 11:33:38 -07:00
teknium1
1f45ff9e8a refactor(gateway): shared exec-approval/picker formatting cores in base adapter
- base._format_exec_approval(command, description, smart_denied): shared
  header/fence/reason/smart-deny assembly driven by _EA_* template attrs and
  an _ea_escape() hook; base._format_choice_page(options, page, per_page):
  shared pagination core returning (page_options, meta) incl. the
  ' (N-M of T)' page_info suffix; base._truncate_preview: the shared
  truncate-with-ellipsis idiom.
- telegram (HTML attrs + _html.escape hook), feishu (card markdown attrs),
  matrix (head-only; local reaction-legend tail) rewired; telegram's
  provider/model keyboard pagination and slash-confirm preview use the
  shared cores. All user-visible strings byte-identical (parity-tested).
- slack/discord/teams left untouched: their formatting interleaves
  platform-specific budget arithmetic (Slack 3000-char section budget
  subtraction, Discord mention-prefix + dual content/embed budgets, Teams
  adaptive-card blocks) beyond template params.
- tests/gateway/test_interactive_prompt_base.py covers the cores + parity.
2026-07-29 11:19:16 -07:00
teknium1
bf15259e33 refactor(gateway): shared media-cache mime dispatch for adapter downloads (per-adapter overrides preserve historical mappings) 2026-07-29 10:14:59 -07:00
victor-kyriazakos
1773752c8c Merge remote-tracking branch 'origin/main' into feat/gateway-health-diagnostics-monitoring
# Conflicts:
#	uv.lock
2026-07-29 15:37:14 +00:00
Enough1122
5cc5c58e01 fix(gateway): flush pending memory writes before session teardown (#73297)
The gateway's /reset cleanup path called shutdown_memory_provider without
first draining the memory manager's serialized background write worker.
shutdown_all only gives that worker a bounded (~5s) drain and abandons
whatever is still queued past it, so a /reset could silently drop writes
the session had already handed off -- the next session then loaded
stale MEMORY.md.

The CLI exit path already drains via MemoryManager.flush_pending before
shutdown; this PR pins the same contract on the gateway cleanup path.

Cleanup now calls agent._memory_manager.flush_pending(timeout=10) before
the existing shutdown_memory_provider step. The flush is best-effort:
a flush failure must never block teardown, so it is wrapped in
try/except and the existing shutdown path remains the fallback.

Closes #73297
2026-07-29 17:21:55 +05:30
StellarisW
8c50aaceb6 fix(gateway): keep models.dev refreshes off event loop 2026-07-29 17:13:48 +05:30
kshitijk4poor
23e44a2843 fix: consolidate agent-history preservation into shutdown_flush module
Follow-up for salvaged PRs #73400 + #73372:
- Move _preserve_agent_history_on_shutdown logic into
  gateway/shutdown_flush.py as flush_agent_history_to_file()
- Reuse the hardened _write_payload (atomic writes, fsync, private
  permissions, UUID filenames) from #73372 instead of the plain
  open()/write() from #73400
- Replace os.environ.get('HERMES_HOME') with get_hermes_home() for
  profile-safe path resolution
- Update tests to test the real function directly
- Net: removes 37 lines from gateway/run.py, unifies both fix paths
  in a single module with consistent atomic-write guarantees
2026-07-29 16:45:35 +05:30
Baophan00
40837e2dd0 Fix #72680 (retargeted): preserve agent._session_messages on shutdown flush failure
The previous attempt (#73171) snapshotted GatewayRunner._pending_messages,
which on current main has no writers (commit f6736ced8 removed its write
path; interrupt delivery uses adapter._pending_messages instead). The live
container is the per-agent agent._session_messages, flushed via
_flush_messages_to_session_db. When that flush raises (FTS/SQLite corruption,
the disk=0/memory=N state from #72680), the in-memory transcript is lost when
the process exits.

Retarget the preservation to the real path: in _finalize_shutdown_agents, wrap
the _flush call; on exception, dump agent._session_messages to an external JSON
recovery snapshot under $HERMES_HOME/shutdown-recovery/ (tagged issue=#72680)
so an operator can salvage it after repairing state.db. The dump is fully
guarded (non-fatal) so shutdown never blocks on a best-effort backup.

This directly addresses the reviewer note on #73171: retarget to the actual
cached-agent history (agent._session_messages) and prove a stale DB + shutdown
leaves a recoverable transcript.

Regression tests: tests/gateway/test_session_messages_shutdown_preserve.py
- flush raises -> recovery file written with session_id + messages
- healthy flush -> no recovery file
- write error -> non-fatal, no raise

Fixes #72680
2026-07-29 16:45:35 +05:30
kshitij
720cdd1d14 refactor: use atomic_json_write instead of hand-rolled _write_payload
Replace 20 lines of manual os.open/O_EXCL/fdopen/fsync/os.replace with the
existing atomic_json_write() from utils.py, which is already used by 6+
modules and handles temp-file creation, fsync, atomic replace, mode
control, and owner preservation. The only novel helper (_fsync_directory)
is retained — atomic_json_write does not do directory fsync.

Update test_flush_write_failure_leaves_no_recovery_file to monkeypatch
utils.os.replace (the new call path) instead of gateway.shutdown_flush.os.replace.
2026-07-29 11:16:09 +05:30
deacon-botdoctor
72024950cf fix(gateway): harden shutdown message flush 2026-07-29 11:16:09 +05:30
Teknium
cf1e3585b6 test(line): skip dual-stack both-families assertion on IPv4-only hosts
test_default_bind_serves_both_families asserts an IPv6 listening socket
exists, which is the environment's capability, not the adapter's — CI
runners with IPv6 disabled would flake. Probe an actual ::1 bind (not
just socket.has_ipv6, a compile-time constant) and skip cleanly when
the host has no usable IPv6 stack.
2026-07-28 22:42:41 -07:00
Shannon Sands
e24bb0b42f fix(line): dual-stack webhook bind — 0.0.0.0 default unreachable over IPv6-only networks
The LINE adapter's webhook server defaulted to host="0.0.0.0", which
binds IPv4 ONLY. On IPv6-only private networks — notably Fly.io 6PN,
where the hosted edge router reverse-proxies LINE ingest to
<app>.internal:8646 over an fdaa: IPv6 address — nothing is listening
on the dialed address: connection refused → customer-visible 502 when
LINE's console verifies the webhook (NS-603).

This is the same bug the generic webhook adapter fixed in d542894ad;
the LINE adapter was never updated to match. Fix mirrors that commit:

- DEFAULT_HOST = None → asyncio/aiohttp create_server binds one socket
  per address family (v4 + v6), regardless of the bindv6only sysctl.
  "::" is NOT a valid substitute — Fly machines set bindv6only=1, so
  it would yield an IPv6-only socket and break IPv4 loopback probes.
- Empty-string host collapses to None; LINE_HOST/extra.host still pin
  a specific bind address.
- reuse_address=False scoped to macOS only (BSD wildcard-socket
  traffic-splitting footgun), mirroring 9420ad946.
- The three outbound-media guards compared webhook_host == "0.0.0.0"
  by string equality; extracted to _missing_public_url() which treats
  None/0.0.0.0/::/"" as "no fetchable hostname" so the LINE_PUBLIC_URL
  requirement still fires under the new default. _media_url() falls
  back to 127.0.0.1 instead of interpolating 'None' into URLs.

Tests: dual-stack default decision table (default None, empty→None,
pinned preserved, LINE_HOST override), behavioural both-families bind
proof via runner.addresses, and the media public-URL guard matrix.
88 passed, ruff clean.

Companion router fix (hermes-agent-router) routes /webhooks/line and
/line/webhook|/line/media to :8646 over 6PN; both are needed for
end-to-end hosted LINE delivery.

Fixes NS-603
2026-07-28 22:42:41 -07:00
Teknium
a65494ed00 fix(gateway): treat pid=None lifecycle sentinel as unknown ownership in mark_exited
The ownership guard let a sentinel with pid=None pass as self-owned, so
an exiting life could clobber evidence of unknown provenance with a
clean-exit claim. Tighten: only rewrite when the sentinel pid matches
os.getpid() exactly; pid=None (or malformed) is left untouched. Adds
tests for the pid=None no-op and the own-pid rewrite paths.
2026-07-28 22:41:09 -07:00
Shannon Sands
9c76c133b7 fix(gateway): detect and report unclean shutdowns via lifecycle ledger (NS-608)
Hosted agents that die uncleanly (kernel OOM kill, SIGKILL, whole-VM
death) leave no trace: shutdown_forensics only covers graceful signals,
gateway-exit-diag.log only covers exit paths that actually run, and the
VM reboot wipes dmesg before anyone can capture it. NS-608 (BlueAtlas
hourly crash cycle, July 12-15) took days of manual log correlation to
classify because nothing recorded 'the previous life ended violently'.

Add gateway/lifecycle_ledger.py — a sentinel state machine persisted to
<HERMES_HOME>/state/gateway.lifecycle.json:

- start_gateway() claims the sentinel (phase=running) right after the
  PID-file/runtime-lock claim, and reports any prior life that never
  reached an exit path as gateway.previous_unclean_exit in
  gateway-exit-diag.log + a WARNING log line.
- Every exit funnel marks the sentinel exited with a reason:
  _exit_after_graceful_shutdown (graceful_shutdown), the shutdown
  watchdog (shutdown_watchdog), and the loop-liveness watchdog
  (loop_liveness_watchdog).
- Ownership-guarded for --replace takeovers: a live matching owner is
  never reported dead, and the old life cannot clobber the
  replacement's freshly claimed sentinel on its way out.

The 30s loop heartbeat now embeds a cheap /proc memory sample (own RSS,
MemAvailable, swap used) so every unclean-death report carries a
'memory N seconds before death' snapshot; the detector flags
suspected_oom when the last sample shows <64MiB or <5% available.

container-boot.log lines gain prior_exit=clean|unclean|unknown per
profile, stamping unclean container deaths into the volume-persisted
boot log where support can grep for them.

Tests: tests/gateway/test_lifecycle_ledger.py (16 cases) + 4 new
container-boot annotation cases. Existing watchdog/forensics/boot
suites all green; ruff clean.
2026-07-28 22:41:09 -07:00
Carl Taylor
3a4aa2f8e6 feat(gateway): streaming TTS adapter contract and consumer (#60671)
Add an opt-in streaming-audio adapter seam to BasePlatformAdapter so
voice-capable gateway platforms (LiveKit, Discord voice, future adapters)
can consume LLM output as streaming PCM audio before the full response
completes, dropping perceived voice latency from ~2-3.5s to ~500-800ms.

Adapter contract (gateway/platforms/base.py):
- AudioFormat dataclass: declared sample_rate, channels, sample_width
- StreamingTTSHandle: opaque handle with audible/aborted flags
- supports_streaming_tts / begin_streaming_tts / write_streaming_tts
  / finish_streaming_tts / abort_streaming_tts
- All default to unsupported/no-op so existing adapters are source-compatible
- Per-turn _streaming_tts_completed_chats set suppresses duplicate whole-file
  auto-TTS when streaming succeeded; cleared after turn completion

Gateway consumer (gateway/streaming_tts_consumer.py):
- StreamingTTSConsumer: bridges sync agent deltas to async adapter audio sink
- Uses existing SentenceChunker (no competing parser)
- Thread-safe bounded queue; on_delta never blocks the agent worker thread
- Resolves configured streaming provider via resolve_streaming_provider()
- Serialises clause playback in order; flushes tail on completion
- Pre-audio failure: completed=False (falls back to whole-file TTS)
- Post-audio failure: completed=True, partial=True (no replay from start)
- Abort is idempotent; late chunks silently dropped
- Per-turn state isolated across concurrent chats

Gateway integration (gateway/run.py):
- message_type parameter threaded through _run_agent -> _run_agent_inner
- StreamingTTSConsumer created when voice input + auto-TTS + provider active
- Delta callback teed to both text stream consumer and TTS consumer
- TTS-only delta callback installed when text streaming is off
- finish() called from executor; wait_complete() in async context after
- Barge-in aborts the consumer at all three interrupt detection points
- Runner-level _send_voice_reply suppressed when streaming TTS completed

Tests (tests/gateway/test_streaming_tts_consumer.py):
- 15 focused tests: adapter defaults, lifecycle, ordered chunks,
  unsupported/No-streamer fallback, abort idempotency, late-chunk drop,
  pre/post-audio failure, concurrent-turn isolation, think-block suppression,
  queue backpressure

Does not touch desktop/TUI code or add config flags. Plugin TTS provider
stream() metadata gap (#47896) is explicitly out of scope — built-in
ElevenLabs/OpenAI PCM streamers are the first consumers.

Refs: #60671, #47896
2026-07-28 22:31:40 -07:00
Nick Edson
201be3e546 fix(gateway): reserve retrying Photon listener ownership 2026-07-28 21:45:52 -07:00
Nick Edson
c608a6937a fix(gateway): release failed Photon listener claims 2026-07-28 21:45:52 -07:00
Nick Edson
a908c62d28 fix(gateway): guard Photon sidecar listener collisions 2026-07-28 21:45:52 -07:00
Nick Edson
cf19ac8ff3 fix(gateway): prevent duplicate Photon sidecar storms 2026-07-28 21:45:52 -07:00
Teknium
41233e19c6 fix(gateway): forward failure_reason through the empty-response return path
Sibling of #64686: _run_agent's empty-final_response early return dropped
failure_reason (and #64686 only fixed the non-empty path), so downstream
consumers (TUI billing surface, transient-failure persistence) lost the
structured reason exactly when a failed run produced no text.

Also hardens the two BasePlatformAdapter identity checks (edit_message /
delete_message) with getattr so duck-typed adapters without the attribute
mean 'capability absent', not AttributeError — this was crashing the
send_progress_messages path for minimal adapters and test fakes.
2026-07-28 21:45:44 -07:00