Model-picker audit follow-through — closes the remaining pieces of the
"switch one session, switches everywhere / can't tell whose session this
is" report class:
- tui_gateway: `config.set key=fast` with a session no longer writes the
global agent.service_tier to config.yaml (sibling of the earlier
`reasoning` scoping fix). It pins create_service_tier_override
("priority" / "" for explicit normal) so lazy builds and rebuilds keep
the choice; the desktop's per-model presets were rewriting the global
tier on every model pick. Fast-support validation now checks a draft's
picked model, and `config.get key=fast` reads the pre-build pin.
- desktop: owning-profile tag (initial chip + tooltip/aria label) on
pinned rows and search results in the All-profiles sidebar, and on the
chat header once a second profile exists (#66003).
- desktop: composer model pill shows a pin dot + tooltip when a manual
sticky pick is overriding the Settings default for new chats (#62055).
Closes#66003. Addresses #62055.
A cron job ("Daily Buzz Report") died with 'AIAgent' object has no
attribute '_claim_stream_writer'. The #65991 single-writer fence lives on
AIAgent (run_agent.py), but the streaming paths that use it live in other
modules — chat_completion_helpers (chat / anthropic / bedrock) and
codex_runtime (codex responses) — and called it directly as
agent._claim_stream_writer() / agent._stream_writer_is_current(). That makes
those modules hard-depend on the method being present on whatever object is
passed as agent.
The fence is an *additive* safety net that may only ever drop a provably
superseded stream, never the sole legitimate writer. But the direct calls
turned any agent that doesn't expose it — a version-skewed checkout (the
streaming helper module newer than run_agent), a hot-reloaded gateway mid
git-pull, a duck-typed agent, or a test double — into a fatal AttributeError
that aborts the whole turn (and, on cron, fails the job).
Route every cross-module claim/check through agent/stream_single_writer.py.
claim_stream_writer(agent) returns 0 when the fence is unavailable (or
raises), and stream_writer_is_current(agent, token) treats a 0 token or an
absent guard as "current" — so a guard-less agent degrades to "no fence"
instead of crashing, while a real AIAgent keeps the full single-writer
protection. Internal self.* uses inside run_agent are unchanged (self is
always a full AIAgent there).
* fix(credential-pool): throttle "no available entries" log to stop Windows log-lock storm
Credential selection runs on a hot path (every model call plus auxiliary
tasks), so an empty/exhausted pool logged "no available entries" at INFO on
*every* selection. On Windows, where multiple Hermes processes share one
rotating log guarded by concurrent-log-handler's cross-process lock, that
per-selection volume storms the lock (RuntimeError: Cannot acquire lock after
20 attempts), pegs a core, and stalls the asyncio event loop long enough that
the Desktop backend readiness probe times out ("Timed out connecting to Hermes
backend after 15000ms") even though the backend already announced
HERMES_BACKEND_READY.
Log the condition at most once per 60s window, re-arming on a successful
selection so recovery->re-exhaustion still surfaces promptly. Same fix class as
the warn-once dedup in #58265.
* test(credential-pool): cover no-available-entries log throttle
Assert the empty-pool INFO line logs at most once per throttle window, logs
again after the window elapses, and re-arms on a successful selection so a
recover->re-exhaust transition surfaces promptly. Uses a deterministic fake
monotonic clock (no sleeps, no network).
Complements the #65637 salvage (53d358838 + a9cc17fd8): the gateway
session store now retries transcript appends through its own queue, but
cron and CLI writers call SessionDB directly — a corrupt FTS index still
hard-failed their appends until the next process restart triggered the
offline repair.
_execute_write now detects the FTS-corruption error class (both the
generic 'database disk image is malformed' and newer SQLite's
'fts5: corrupt structure record' variant), performs a one-shot in-place
rebuild by delegating to the existing rebuild_fts(), and retries the
failed write. One-shot per instance so an unrecoverable database cannot
loop; lock/busy jitter-retry path untouched.
E2E-verified: corrupted messages_fts_data rejects appends; with this fix
the same append self-heals, persists, and FTS search works again.
Community feedback (@LSanapalli on X): the inline task-creation form is
cramped inside a ~280px column with no way to resize; board-level
workspace defaults can't be changed after board creation; and users
believe they must block a task, comment, then unblock just to talk to
a worker.
- Create-task dialog: replace the inline column form with a centered
modal (reuses hermes-kanban-dialog chrome, 36rem wide) with labeled
fields for title, assignee, priority, skills, workspace kind/path,
goal mode, and parent task. Same request shape; Enter/Escape behavior
preserved; submit disabled until a title is present.
- Board settings dialog: new Settings button in the board switcher opens
a modal to edit display name, description, and the board-level default
project directory (default_workdir). PATCH /boards/:slug now accepts
default_workdir (validated absolute existing dir; empty string clears;
omitted leaves unchanged) and returns the recomputed
default_workspace_kind so task-creation defaults follow immediately.
- Comment workflow hint: the task drawer's comment box now explains that
comments land on the thread immediately and reach the worker on its
next run/kanban_show() — no block/unblock dance needed — with a fuller
tooltip for when blocking IS the right tool.
- i18n: new keys optional in the kanban namespace with English fallbacks
in the bundle (established pattern; avoids churning 17 locale files).
- Docs: dashboard section updated for the dialog + Settings button.
A root-launched CLI session can leak /root into the terminal cwd state a
non-root gateway/cron process later resolves (#65583). os.path.isdir('/root')
is True for a non-root user — stat only needs search permission on / — so
_resolve_safe_cwd returned it and subprocess.Popen(cwd='/root') died with
PermissionError: [Errno 13], failing EVERY cron job's terminal/file/search
tool on every command until restart.
_resolve_safe_cwd now requires X_OK (new _cwd_usable helper) and climbs to
the nearest enterable ancestor, logging a WARNING that names the leak class
when an existing-but-denied cwd is skipped. Missing-cwd recovery (#17558)
behavior unchanged.
E2E-verified: LocalEnvironment constructed with an unenterable cwd now runs
commands from the fallback directory instead of raising.
The widened resume sites pass repair_alternation=True; the DB doubles in
tests/test_tui_gateway_server.py (separate from tests/tui_gateway/) needed
the same signature update as Frowtek's originals.
test_build_child_agent_ignores_acp_command_when_binary_missing patches
shutil.which globally and asserted the LAST call was which("copilot").
That is order-dependent: an unrelated which("uv") reached later in the same
process (which happens under some CI test-slice orderings) becomes the last
call, so assert_called_with("copilot") fails even though the copilot binary
was probed exactly as intended. Switch to assert_any_call("copilot"), which
verifies the actual intent and is robust to unrelated which() calls. The
behavioural assertions (provider, acp_command, acp_args) are unchanged.
The lazy session.resume path now calls
db.get_messages_as_conversation(target, repair_alternation=True), but the
fake _DB stubs in test_protocol.py still declared the pre-change signature,
so the resume raised "unexpected keyword argument 'repair_alternation'"
and the three session_resume_lazy tests failed.
Mirror the real get_messages_as_conversation signature in the stubs by
accepting (and ignoring) repair_alternation.
Follow-up to the restore-boundary alternation heal (#65492): get_messages_
as_conversation grew a repair_alternation flag, wired into gateway
load_transcript and the CLI startup resume. Three other LIVE-REPLAY
restore sites still loaded the transcript verbatim, so a durable
'user;user' violation there re-fires the pre-request defensive repair on
every request for the rest of the session (it only ever mutates the
per-request list, never the restored working conversation):
- acp_adapter/session.py::SessionManager._restore — the loaded history
becomes the resumed ACP (Zed) agent's SessionState.history.
- hermes_cli/cli_commands_mixin.py — the /resume slash command sets
self.conversation_history from the load (the startup resume was fixed,
this mid-session one was missed).
- tui_gateway/server.py — the resume handler feeds the load into the
deferred session record's working conversation.
Pass repair_alternation=True at all three so the wedge is healed once at
restore. Inspection/export consumers (trace upload, context guard,
api_server history, display_history) keep the verbatim default.
Adds an end-to-end regression test driving the ACP _restore path: a
seeded user;user session restores to an alternation-clean live history
with no user input lost.
Two more display gaps from #26541 grafted onto the merged bridge:
- webSearch: codex's built-in web search now produces a tool.started/
tool.completed bubble pair (query as preview + args). Previously the
item type wasn't in _CODEX_TOOL_ITEM_TYPES, so built-in searches
showed nothing.
- mcp.hermes-tools.* stripping: tools codex invokes through Hermes' own
hermes-tools MCP server display as their bare names (web_search,
browser_navigate) instead of mcp.hermes-tools.web_search. The inner
dispatch subprocess can't fire native progress events, so the
codex-level event is the display event — name it the way users know
the tool.
Credit: both behaviors designed and first implemented by @simpolism in
PR #26541 (May 15, earliest of the app-server display-bridge family).
The approval-drain loop in CodexAppServerSession.run_turn drains up to 8
pending notifications to keep per-turn state current before answering a
server-initiated approval request — but never forwarded them to the
on_event display hook. Tool bubbles for items drained alongside an
approval (e.g. the item/started for the very command awaiting approval)
silently disappeared.
Mirror the main notification path's on_event invocation in the drain
loop. Regression test demonstrates RED→GREEN.
Grafted from PR #26541 by @simpolism — the earliest submission of the
codex app-server display-bridge family (May 15). Confirmed independently
by #64698 and #65412.
The REFERENCE ONLY framing ('treat as background reference, NOT as active
instructions... Do NOT answer questions or fulfill requests') was observed
bleeding into general tool-use suppression: a production session went
narration-only for 7 consecutive turns immediately after a compression
event, describing edits instead of calling tools (#65848 report).
Fix is additive: one clause stating the note does not restrict HOW the
agent works — tools remain fully active for the active task. Every
anti-resumption protection stays intact; the previous prefix generation
is frozen into _HISTORICAL_SUMMARY_PREFIXES per the module contract so
persisted summaries still get the directive-strip on re-compaction.
The #65848 rewrite was not taken: dropping the 'Do NOT answer questions'
line and the four-heading discard directive risks re-opening the
stale-task-resumption class those clauses exist to prevent (the carveout
era regressions #41607/#38364/#42812 documented in this file).
Report and root-cause analysis: @yasserbousrih (#65848).
Widen the #65991 single-writer fence to run_codex_stream: each codex
attempt claims the delta sink before consuming events, and the consume
loop's interrupt_check now also stops the instant a newer attempt
supersedes this one. Parity with the chat_completions / anthropic /
bedrock paths from the salvaged fix.
Two regression tests: superseded codex stream is fenced mid-stream;
sole-writer codex stream delivers unchanged.
Assert that a superseded stream (older writer token, other thread) is fenced
from the delta sink, the active writer is never fenced, a non-claiming thread
is never treated as a writer, and the real consume loop stops the instant it is
superseded — so two streams can never interleave into one turn (#65991).
A hermes --tui session whose main thread wedges before app.run() returns
never executes the finally that calls _run_cleanup — the only place the
exit watchdog was armed — so a dead CLI lingered indefinitely (observed
~47 min at 4% CPU, the #65998 class).
Arm the backstop from the SIGTERM/SIGHUP handlers instead (both the
interactive and single-query paths), the earliest moment shutdown intent
is unambiguous. The signal-armed leash is 2x HERMES_EXIT_WATCHDOG_S so a
slow-but-progressing _run_cleanup (which still arms its own tighter timer)
is never cut short; the outer timer only wins when cleanup was never
reached. Idempotent across repeated signals; never raises from a handler.
Deliberately NOT armed at startup: the watchdog thread calls os._exit(0)
unconditionally after its sleep, so a startup-armed timer (the #65998
approach) would hard-kill every session that outlives the timeout.
Supersedes #65998; thanks @JeffStone69 for the report and root-cause gap
analysis.
Degrading models (observed with gpt-5.6 past ~350K input) emit tool-call
batches like 6 valid named calls + 1 blank-name call. Previously the
whole turn was voided — every valid call got 'Skipped: another tool call
in this turn used an invalid name' — and three such batches tripped the
3-strike stop, killing sessions that were still making progress.
Now a mixed batch error-results ONLY the invalid call(s) (terse
anti-priming error for blank names per #47967, catalog dump for typos)
and dispatches the valid subset for execution. The assistant message
keeps every emitted call so provider-side tool_call/result pairing stays
intact. The 3-strike counter only advances when a turn contains NO valid
call, so a fully-degenerate model still stops while a mostly-coherent
one keeps working. Broken JSON args on a never-executing invalid call no
longer trigger the whole-turn JSON retry loop.
Field evidence: July 2026 debug bundle showed gpt-5.6-sol emitting
6-call batches with one blank-name rider at 559K/384K-token context in
two separate sessions; 13 valid tool calls were discarded before the
session stopped as partial.
Follow-up fixes for salvaged PR #65637:
1. Clear _dirty_transcripts in rewrite_transcript + rewind_session —
stale pending messages were re-inserted after /retry, /undo, /compress.
2. Narrow _is_fts_corruption_error to specific SQLite error strings —
bare 'fts' substring matched 'shifts', 'gifts', etc.
3. Move DB write outside _transcript_retry_lock — holding the lock
during writes serialized all sessions' transcript appends and blocked
during FTS rebuild. Now the lock guards only the pending queue.
4. Push rebuild_fts() into SessionDB — SessionStore was reaching into
_conn/_lock private attrs. SessionDB.rebuild_fts() follows the same
pattern as optimize_fts().
5. Cap pending per session at 200 — prevents unbounded memory growth
when DB is persistently broken. Oldest messages dropped with warning.
Added 4 new tests: dirty-clear on rewrite/rewind, FTS matcher false
positives, pending cap enforcement.
Two changes to tests/agent/test_file_safety_session_state.py:
1. Drop the stale monkeypatch on tools.file_tools._get_live_tracking_cwd
— the helper was deleted in the cwd-tracking refactor (c80b244b5),
and monkeypatch.setattr on a missing attribute raises AttributeError,
breaking CI slice 4/8 on main for every PR. The patch was redundant:
the test writes an absolute path, so cwd resolution never engages.
2. Make the fixture stale-proof: instead of monkeypatching the private
_hermes_home_path/_hermes_root_path helpers (same failure class if
they're ever renamed), set HERMES_HOME to <root>/profiles/work and
let the real resolution chain (get_hermes_home /
get_default_hermes_root's profiles-parent rule) derive both paths.
The fixture now references zero private symbols and exercises the
production resolution path.
Port from anomalyco/opencode#35671: Z.AI / Zhipu GLM returns
'tokens in request more than max tokens allowed' (error code 1210) on
context overflow. This matched no pattern in _CONTEXT_OVERFLOW_PATTERNS,
so the error classified as unknown/retryable — the agent would retry the
oversized request instead of triggering context compression.
Proven live on main before the fix: classify_api_error() returned
FailoverReason.unknown for the exact Z.AI error shape; now returns
context_overflow.
Port from anomalyco/opencode#35439/#35500: preserve full MCP catalogs
across paginated tools/list responses.
The MCP spec allows servers to paginate tools/list, resources/list, and
prompts/list via an opaque nextCursor token. The Python SDK's
ClientSession.list_* methods fetch exactly one page per call, and hermes
never passed the cursor back — on a paginated server every tool,
resource, and prompt past page 1 was silently invisible to the agent.
Adds _paginate_full_list() (cursor-draining helper with a 50-page
runaway cap and spec-correct opaque-string cursor validation) and
applies it at all three discovery sites: _discover_tools(), the
tools/list_changed refresh handler, and the list_resources/list_prompts
utility handlers. The keepalive probe intentionally keeps its
single-page call (liveness only).
E2E: real stdio MCP server serving 3 pages (2/1/1 tools) — old code
discovered 2 tools, new code discovers all 4.
Port from Kilo-Org/kilocode#11955: Gemini validates every object schema's
required list strictly against the same node's properties and fails the
ENTIRE GenerateContentRequest with HTTP 400 'required[0]: property is not
defined' when a name has no matching property. MCP servers (e.g. the GitHub
remote MCP) routinely emit array item schemas carrying required without
properties, which made every request on the native Gemini path fail before
any model output.
sanitize_gemini_schema() now filters required to names present in the node's
properties and drops the keyword when nothing valid remains. Applies
recursively (properties / items / anyOf). Tool handlers still validate
required fields at execution time, so nothing the model could actually use
is lost.
Scoped to the Gemini-facing sanitizer only — the universal
tools/schema_sanitizer.py already prunes typed object nodes, and its
remaining gap (untyped nodes) is contested by open PR #20151.
Port from openai/codex#31494: user-visible history replay must strip CSI
sequences and control characters. Stored conversation history can carry
raw terminal escapes (pasted content, gateway-origin text, model output
echoing injected tool results). Replaying it via /resume's recap panel or
build_recap (/status on CLI + gateway) wrote those bytes straight to the
terminal — an injected message could clear the screen, retitle the window,
move the cursor, or restyle the recap UI. Rich's Text() does not neutralize
raw escape bytes.
- tools/ansi_strip.py: add sanitize_display_text() — strip_ansi() plus
bare C0/C1 control removal, preserving \n and \t, normalizing \r to \n
(adapted to Python from Codex's sanitize_user_text; reuses the existing
ECMA-48 stripper instead of transcribing their char-walk)
- hermes_cli/cli_agent_setup_mixin.py: sanitize user + assistant text in
_display_resumed_history() before building the Rich recap panel
- hermes_cli/session_recap.py: sanitize preview lines in build_recap()
(_truncate choke point) so /status recaps are clean on every platform
- tests: 10 new sanitize_display_text cases (incl. the exact codex#31494
fixture), recap + resume-display leak assertions
The previous commit fixes the canonical agent/skill_utils.parse_frontmatter.
Six more modules reimplement the '---' fence check locally and had the
same bug:
- tools/skill_manager_tool.py _validate_frontmatter — rejected BOM'd
skill_manage create/edit content outright
- tools/skills_hub.py GitHubSource._parse_frontmatter_quick and
OptionalSkillSource._parse_frontmatter — hub browse/install metadata
- hermes_cli/skills_hub.py — local skill install validation
- gateway/run.py — skill slug discovery for disabled-skill hints
- agent/prompt_builder.py _strip_yaml_frontmatter — BOM'd context files
(AGENTS.md) leaked raw frontmatter into the system prompt
- tools/blueprints.py _split_frontmatter — str.lstrip() does not strip
U+FEFF (not whitespace), so the existing lstrip never covered it
Sibling-surface regression tests added.
Bug class also fixed upstream in cline/cline#12218 (found by the weekly
Cline PR scout).
A UTF-8 BOM saved into a SKILL.md (e.g. Notepad or PowerShell `>`) is kept by
read_text(encoding="utf-8"), so the string handed to parse_frontmatter starts
with the BOM and the startswith("---") fence check fails. The whole frontmatter
is then silently dropped: the skill loads with no name/description, `platforms`
gating falls open (a macOS-only skill becomes visible everywhere), and
required_environment_variables / metadata.hermes.config setup never fires.
Strip a single leading BOM at the top of parse_frontmatter, the shared
chokepoint for every local skill-loading path (_parse_skill_file,
discover_all_skill_config_vars, DESCRIPTION.md parsing, _inject_skill_config,
and the tools/skills_tool._parse_frontmatter re-export), so the whole class is
covered, not just the reported site. Only the leading marker is removed; a BOM
mid-content is left as data. Mirrors the existing file-tools BOM handling
(#35278) and CONTRIBUTING.md "File encoding".
Adds tests: BOM'd frontmatter parses identically to plain, the body is
BOM-free, platforms gating and config-var extraction survive a BOM, and an
end-to-end BOM-write / plain-read round trip (mirroring _parse_skill_file).
test_auto_reset_does_not_recover_session_being_ended asserted the old
end_session('session_reset') call; auto-reset now writes through
promote_to_session_reset with the specific auditable reason
('suspended' for a suspended entry). Missed in the local targeted run
because the file wasn't in the touched-suite list; caught by CI shard 6.
Unifies the two gateway subsystems that were fighting each other: the
'never lose a session' recovery machinery (#54878 stale-route self-heal,
find_latest_gateway_session_for_peer reopening agent_close/ws_orphan_reap
rows) and the session reset/expiry machinery (expiry watcher, /new,
/resume, resume_pending freshness gate).
The unified contract:
- INTENTIONAL boundaries (expiry finalization, auto-reset, /new,
/resume switch) are recorded durably via promote_to_session_reset(),
which upgrades accidental recoverable end_reasons (agent_close,
ws_orphan_reap) to the explicit boundary while preserving other
explicit reasons (compression, etc.). Recovery then correctly refuses
to resurrect them.
- ACCIDENTAL ends (crash, cleanup bug, mistaken reaper) stay
recoverable — genuine crash recovery is untouched.
On top of the cherry-picked contributor commits:
- promote_to_session_reset widened to ws_orphan_reap + parameterized
reason so auto-reset paths stay auditable (idle/daily/suspended/
resume_pending_expired) (#61220, #61993, #63539)
- get_or_create_session auto-reset, reset_session (/new), and
switch_session (/resume) all write through the promote path — the
first-reason-wins end_session no-op could previously leave a reset
session resurrectable behind a stale agent_close row (#61993)
- resume_pending freshness gate now honors session_reset.mode=none:
explicit opt-out of automatic resets also opts out of the zombie
gate (#61052)
- resume recovery note extracted to build_resume_recovery_note() and
made adapter-aware via a new interactive_resume capability flag:
webhook/api_server auto-resume turns now CONTINUE the interrupted
task instead of emitting an unanswerable 'session restored'
acknowledgement that abandoned the work (#57056)
- tests updated to call the real note builder instead of mirroring it
E2E-validated against a real SessionDB + SessionStore in a temp
HERMES_HOME: expiry->agent_close->no-resurrection, /new promote,
crash recovery preserved, mode=none opt-out, routing-table flag sync.
When a gateway session with resume_pending=True is not recovered within the
auto-continue freshness window (e.g. because repeated API calls timed out on a
large context), get_or_create_session correctly creates a new session. However
two gaps existed:
1. The user received no notification — resume_pending_expired fell through the
generic "inactive for Xh" else-branch in run.py, which produces wrong wording
and (for session_reset.mode: none users) is gated on policy.notify that
evaluates to False.
2. The old session was ended in state.db with the hardcoded generic reason
"session_reset", making it impossible to distinguish from a normal idle/daily
reset in post-mortem analysis.
Fix:
- gateway/run.py: add an explicit resume_pending_expired case for the agent
context note ("gateway restart recovery timed out") and the user-facing
notification. Always notify for this reason — like suspended — because the
user had an active session that was silently replaced.
- gateway/session.py: pass auto_reset_reason as the DB end_reason instead of
the hardcoded "session_reset", so all auto-reset paths are auditable.
- tests: extend TestResumePendingExpiredAutoReset in test_session_reset_notify.py
with five new cases that cover the reason, activity flag, DB end_reason,
non-regression of the idle path, and freshness-disabled bypass.
Closes#58933
Co-authored-by: Cursor <cursoragent@cursor.com>
When the #54878 self-healing path drops a stale sessions.json entry, the
fix at gateway/session.py:1765 now checks _should_reset() before falling
through to DB recovery. This test covers the case where the stale entry's
session is overdue under an idle/daily reset policy — it must create a
fresh session, set auto-reset metadata, and NOT call reopen_session().
Address review feedback on #63068:
1. Replace unconditional reopen_session() + end_session() with a
conditional promote_to_session_reset() method in SessionDB.
The new method only promotes live rows or rows ended with
agent_close — explicit boundaries (compression, session_reset,
new_command) are preserved via first-writer-wins semantics.
2. Rewrite tests to use real SessionDB instead of MagicMock:
- 7 unit tests for promote_to_session_reset edge cases
- 3 integration tests verifying the actual recovery contract
in find_latest_gateway_session_for_peer after promotion
Port from anomalyco/opencode#36130: the Responses spec carries streaming
error details at the top level of the error frame, but the official OpenAI
SDK and several OpenAI-compatible proxies wrap them in an HTTP-style nested
envelope ({"type": "error", "error": {code, message, param}}).
_raise_stream_error only read top-level fields, so nested-envelope frames
collapsed to the generic 'stream emitted error event' placeholder with
code=None — the error classifier never saw the provider's real failure
reason, misrouting rate-limit / context-overflow / entitlement errors into
the generic retry path.
Top-level fields keep precedence; the envelope is a fallback. Null-tolerant
for spec-compliant frames with explicit nulls.
