Commit graph

1145 commits

Author SHA1 Message Date
Teknium
597615ade4
fix(ci): make tests, workflows, and attribution reliable under load (#66373)
* feat(attribution): conflict-free contributor mappings via contributors/emails/ directory

The AUTHOR_MAP dict in scripts/release.py was a merge-conflict magnet:
every concurrent salvage PR appended entries to the same lines of the
same file, so parallel PRs re-conflicted on every merge to main.

New system: one file per email under contributors/emails/ — filename is
the commit-author email, first non-comment line is the GitHub login.
File additions never conflict, so any number of PRs can add mappings
concurrently.

- scripts/release.py: AUTHOR_MAP is now LEGACY_AUTHOR_MAP (frozen)
  merged with the directory at import time (directory wins). All
  existing consumers (resolve_author, contributor_audit.py) unchanged.
- scripts/add_contributor.py: idempotent CLI to add a mapping; refuses
  conflicting reassignments (incl. against the legacy map), validates
  email/login shapes.
- contributor-check.yml: attribution gate now accepts a mapping file OR
  a legacy entry; failure message prints the exact add_contributor
  command. Also auto-resolves bare <login>@users.noreply.github.com
  emails is intentionally NOT added (kept id+login form only, matching
  previous behavior).
- contributor_audit.py: guidance now points at add_contributor.py.
- tests/scripts/test_contributor_map.py: 12 tests covering loader,
  merge precedence, CLI idempotency/conflict/validation, subprocess E2E.

* feat(ci): one-shot per-file flake retry in the parallel test runner

A failing test FILE is re-run once in a fresh subprocess. Pass-on-retry
counts as green but is loudly reported in a '⚠ FLAKY' summary section
(with both attempts' output preserved) so the flake gets fixed instead
of eating a full-run rerun. Deterministic failures fail both attempts —
regressions cannot be laundered green.

- --file-retries N / HERMES_TEST_FILE_RETRIES (default 1, 0 disables)
- E2E verified: simulated first-run-fail flake goes green with banner;
  deterministic failure still exits 1; retries=0 restores old behavior.

This converts the dominant CI failure mode (one timing-sensitive test
flaking a 4600-test shard, requiring a manual 10-minute rerun and an
agent triage loop) into a self-healing retry that costs one file's
runtime.

* test(approval): loosen wall-clock perf bounds 0.15s -> 2.0s

These guard against catastrophic regex backtracking (seconds-to-minutes
class), but 0.15s is within scheduler-stall noise on loaded shared CI
runners — test_max_accepted_separator_free_input_is_fast failed a CI
shard this week on runner load alone. 2.0s still catches the regression
class with zero flake surface.

* fix(ci): job timeouts everywhere + retries on all network installs

Reliability pass over every workflow:
- timeout-minutes on all 21 jobs that lacked one (a hung job previously
  burned the 6-hour default runner budget)
- ./.github/actions/retry wrapped around every network-fetching install
  that lacked it: pip installs (deploy-site, skills-index), npm ci
  (deploy-site website, upload_to_pypi web + ui-tui), uv sync (docker
  test deps). Deterministic build steps (npm run build) deliberately
  NOT retried — split into separate steps so a real build failure fails
  fast instead of retrying 3x.

* docs(agents): document the file-retry flake policy

* fix(ci): curl retries on deploy hook + skills-index probe

* fix(ci): kill the remaining transient-failure classes in workflows + Dockerfile

From the workflow reliability audit:
- tests.yml: duration-cache restore had NO restore-keys while saves use
  run_id-suffixed keys — the cache never matched once, so LPT slicing
  always ran blind and unbalanced slices pushed heavy files toward the
  per-file timeout. One-line restore-keys fixes slice balancing.
- Label gates (lint ci-reviewed, supply-chain mcp-catalog-reviewed):
  'gh pr view || true' turned an API blip into 'label absent' → false
  BLOCKING failure. Now 3x retry, and API failure is reported as an API
  failure instead of a missing label.
- detect-changes action: compare API retried before failing open (was
  silently running all lanes on any blip).
- uv-lockfile-check: 'uv lock --check' resolves against PyPI — retried
  so registry blips don't read as 'lockfile stale'.
- docker.yml merge job: imagetools create retried (Docker Hub eventual
  consistency on just-pushed digests).
- Dockerfile: apt-get Acquire::Retries=3; s6-overlay ADDs converted to
  curl --retry 3 (ADD cannot retry; checksums still enforced); npm
  --fetch-retries=5; playwright chromium fetch retried 3x.
- Advisory artifact uploads (per-slice durations, ci-timings report)
  get continue-on-error so an artifact-service blip can't fail a green
  test slice.

* fix(tests): kill the two root-cause flakes — leaking pre-warm timer + env-dependent provider list

- test_tui_gateway_server.py: session.create / non-eager session.resume
  arm a 50ms threading.Timer (_schedule_agent_build) that outlives its
  test and fires into the NEXT test's _make_agent mock, racily
  corrupting captured state (the recurring session_resume shard
  failures). Replaced the per-test whack-a-mole stub with a module-wide
  autouse fixture; the 3 worker-lifecycle tests that genuinely need the
  deferred build opt back in via @pytest.mark.real_agent_prewarm (new
  marker in pyproject).
- test_api_key_providers.py: PROVIDER_ENV_VARS is now derived from the
  live PROVIDER_REGISTRY instead of a hand-list that had drifted
  (missing HF_TOKEN / DEEPINFRA_API_KEY) — resolve_provider('auto')
  tests failed on any machine with HF_TOKEN exported. E2E-verified with
  HF_TOKEN/DEEPINFRA_API_KEY set: 42/42 pass.

* test: de-flake 30 timing-sensitive test files for loaded CI runners

Root-cause fixes from the flake audit (session-DB mining + repo sweep):

Event-based sync instead of sleep-sync:
- title_generator: mock sets threading.Event, wait(10) replaces
  sleep(0.3) hoping the daemon thread got scheduled
- docker zombie_reaping / profile_gateway: poll-for-state helpers
  replace fixed 1-3s sleeps (s6 transitions + SIGCHLD reaping are async)
- process_registry tree test: select()-bounded readline replaces an
  unbounded blocking read (parent wedge now fails THIS test with a clear
  message instead of an opaque rc=124 file kill); SIGTERM grace 1s->2s
  (the 1s partition window mid-interpreter-startup is how a child PID
  escaped the live-system guard in CI)

Timeout raises (loaded 8-way-sliced runners see ~5s scheduling floors;
all of these complete in ms-to-1s when healthy so the raises cost
nothing on green runs):
- subprocess/thread waits <= 2s raised to 10-15s across mcp_tool,
  mcp_circuit_breaker, mcp_reconnect_retry_reset, mcp_parked_self_probe,
  mcp_cancelled_error_propagation, registry, clarify_gateway, interrupt,
  voice_cli_integration, docker_environment, session_store_lock_io,
  planned_stop_watcher, cli_interrupt_subagent, thread_scoped_output
  (joins now also assert not is_alive() so stragglers fail loudly)
- wall-clock discrimination ceilings loosened where the guarded hang is
  10x larger: local_background_child_hang 4s->10s, interrupt_cleanup
  setup 5s->20s + pgid-exit 30s->60s, mcp_stability grandchild spinup
  5s->15s, protocol/gil-starvation fast-handler 0.5s->2s,
  iso_certify_seam 1.5s->5s, wait_for_mcp_discovery 0.1s->1s
- narrow assertion windows widened: honcho first-turn wait 0.4..0.65 ->
  0.25..2.0 (property is bounded-not-hung, not an exact wall-clock);
  compression fork-lock TTL 1s->3s (12 refresh chances per lease);
  compression-lock expiry margins symmetric (ttl 0.05->0.5, sleep 1.0)
- telegram hung-DNS bound 1.0->1.4 (fake hang is 1.5s — must stay under)

* fix(tests): repair indentation from de-flake batch edit

* fix(tests): harden env isolation and replace remaining sleep-sync races

The full 42k-test run and complete npm check surfaced three more classes:

- Environment isolation: local ~/.honcho defaultHost and SSH_* variables
  leaked into Python/TUI tests. Pin the default Honcho host in the
  hermetic fixture, isolate the one fallback test from ~/.honcho, and
  blank SSH_* around terminalSetup tests. This flipped 20 false failures
  back to deterministic behavior on developer machines.
- Background-thread sleep-sync: Honcho async writer tests patched
  time.sleep globally, then busy-polled with that same mocked sleep. Under
  full-suite load the poller could starve the writer. Each test now waits
  on an Event emitted by the exact flush/retry transition; 30/30 passed
  under 15-way contention.
- Desktop streaming: the test slept 80ms and assumed a 500ms timer could
  not fire before its assertion. A loaded runner descheduled the test for
  >500ms and both chunks arrived. Producer controls now gate second-chunk
  and completion transitions explicitly.

Also make file-retry observability complete: a self-healed flaky file now
prints BOTH attempts' full output in the FLAKY summary. Two behavioral
runner tests prove pass-on-retry is green+loud+traceback-preserving, while
a deterministic failure remains red.

* refactor(ci): use gh bot pat, better retries

refactor(ci): use retry action for PR label fetch
the retry action now captures stdout as a step output, so it can serve
double duty: retry + output capture for commands like 'gh pr view' whose
result must be consumed by later steps.

Retry action gains:
- 'stdout' output (heredoc-delimited to preserve newlines)
- tee to temp file so stdout still streams to the job log
- step id 'retry' for output reference

Both lint.yml and supply-chain-audit.yml now use the retry action
directly with 'command: gh pr view ...' and read
steps.<id>.outputs.stdout.

ci: use AUTOFIX_BOT_PAT for all gh CLI / GitHub API auth

Replace secrets.GITHUB_TOKEN and github.token with
secrets.AUTOFIX_BOT_PAT across all workflows and composite actions
that use the gh CLI or GitHub API. The PAT has consistent permissions
across fork PRs (where GITHUB_TOKEN is read-only), avoids API rate
limit sharing with the default token, and is already used by
js-autofix.yml for the same reasons.

19 sites swapped across 9 files:
- lint.yml (3): label fetch, comment post/edit, comment update
- supply-chain-audit.yml (5): scan, critical comment, unbounded dep
  comment, label fetch, mcp-catalog comment
- lockfile-diff.yml (1): PR comment post/update
- skills-index-freshness.yml (1): issue creation on degraded probe
- skills-index.yml (2): index build, trigger deploy workflow
- upload_to_pypi.yml (2): release view poll, release upload
- ci.yml (1): timings report
- deploy-site.yml (2): skills index crawl
- detect-changes/action.yml (1): compare API call

---------

Co-authored-by: ethernet <arilotter@gmail.com>
2026-07-17 20:55:24 +00:00
Teknium
61bbc39330 fix(codex): harden final cache-key boundaries
Fold #62349's broader provider-boundary handling into the header fix: bound top-level and xAI override keys again at preflight after middleware, preserve unrelated headers, and cover boundaries and collisions.

Co-authored-by: Nick Taylor <nicktaylor@TheWorldofNick-Lappy.local>
2026-07-17 13:48:41 -07:00
Teknium
81496a8925 test(codex): cover overlength cache-scope headers
Exercise the real transport path for long session ids, including stable hashing and bounded body/header cache keys.
2026-07-17 13:48:41 -07:00
Haider Sultan
18331b9bbd feat(codex): stream live app-server events to TUI/desktop tool cards
Extends the app-server event bridge (make_codex_app_server_event_bridge)
to fire the authoritative stable-ID tool_start_callback /
tool_complete_callback alongside the existing tool_progress_callback,
and route item/reasoning/summaryDelta through the reasoning channel.

Surfaces that render structured tool cards (TUI, desktop) — not just
progress bubbles — now correlate live cards with the projected history
entry after a resume, because the call ids mirror CodexEventProjector's
_deterministic_call_id. Guarded per-callback so a broken display
consumer can't tear down the codex turn loop.

Grafted from PR #65412 by @HaiderSultanArc onto the merged bridge (the
PR's parallel _codex_live_event implementation was reconciled into the
bridge's existing _fire_tool_started/_fire_tool_completed helpers).
2026-07-17 13:44:12 -07:00
brooklyn!
41fdcae688
fix(streaming): make the single-writer fence best-effort so a missing guard can't crash a turn (#66448)
A cron job ("Daily Buzz Report") died with 'AIAgent' object has no
attribute '_claim_stream_writer'. The #65991 single-writer fence lives on
AIAgent (run_agent.py), but the streaming paths that use it live in other
modules — chat_completion_helpers (chat / anthropic / bedrock) and
codex_runtime (codex responses) — and called it directly as
agent._claim_stream_writer() / agent._stream_writer_is_current(). That makes
those modules hard-depend on the method being present on whatever object is
passed as agent.

The fence is an *additive* safety net that may only ever drop a provably
superseded stream, never the sole legitimate writer. But the direct calls
turned any agent that doesn't expose it — a version-skewed checkout (the
streaming helper module newer than run_agent), a hot-reloaded gateway mid
git-pull, a duck-typed agent, or a test double — into a fatal AttributeError
that aborts the whole turn (and, on cron, fails the job).

Route every cross-module claim/check through agent/stream_single_writer.py.
claim_stream_writer(agent) returns 0 when the fence is unavailable (or
raises), and stream_writer_is_current(agent, token) treats a 0 token or an
absent guard as "current" — so a guard-less agent degrades to "no fence"
instead of crashing, while a real AIAgent keeps the full single-writer
protection. Internal self.* uses inside run_agent are unchanged (self is
always a full AIAgent there).
2026-07-17 18:31:09 +00:00
HexLab
594308d4bb
fix(credential-pool): throttle "no available entries" log to stop Windows log-lock storm (contributes to #62698) (#66338)
* fix(credential-pool): throttle "no available entries" log to stop Windows log-lock storm

Credential selection runs on a hot path (every model call plus auxiliary
tasks), so an empty/exhausted pool logged "no available entries" at INFO on
*every* selection. On Windows, where multiple Hermes processes share one
rotating log guarded by concurrent-log-handler's cross-process lock, that
per-selection volume storms the lock (RuntimeError: Cannot acquire lock after
20 attempts), pegs a core, and stalls the asyncio event loop long enough that
the Desktop backend readiness probe times out ("Timed out connecting to Hermes
backend after 15000ms") even though the backend already announced
HERMES_BACKEND_READY.

Log the condition at most once per 60s window, re-arming on a successful
selection so recovery->re-exhaustion still surfaces promptly. Same fix class as
the warn-once dedup in #58265.

* test(credential-pool): cover no-available-entries log throttle

Assert the empty-pool INFO line logs at most once per throttle window, logs
again after the window elapses, and re-arms on a successful selection so a
recover->re-exhaust transition surfaces promptly. Uses a deterministic fake
monotonic clock (no sleeps, no network).
2026-07-17 13:08:46 -04:00
Teknium
73057ed161 fix(auxiliary): scope runtime state to each turn 2026-07-17 09:08:30 -07:00
srojk34
bcce700783 fix(compression): reset failure cooldown on runtime switch 2026-07-17 09:08:30 -07:00
Krowd
c201b72f34 fix(auxiliary): sync runtime after fallback restoration 2026-07-17 09:08:30 -07:00
dfein38347g
fdc6c32d7d fix(auxiliary): isolate runtime cache by live context 2026-07-17 09:08:30 -07:00
Teknium
ec3d958425 feat(codex): webSearch bubbles + bare hermes-tools names in app-server bridge
Two more display gaps from #26541 grafted onto the merged bridge:

- webSearch: codex's built-in web search now produces a tool.started/
  tool.completed bubble pair (query as preview + args). Previously the
  item type wasn't in _CODEX_TOOL_ITEM_TYPES, so built-in searches
  showed nothing.
- mcp.hermes-tools.* stripping: tools codex invokes through Hermes' own
  hermes-tools MCP server display as their bare names (web_search,
  browser_navigate) instead of mcp.hermes-tools.web_search. The inner
  dispatch subprocess can't fire native progress events, so the
  codex-level event is the display event — name it the way users know
  the tool.

Credit: both behaviors designed and first implemented by @simpolism in
PR #26541 (May 15, earliest of the app-server display-bridge family).
2026-07-17 06:49:47 -07:00
snav
11a91a6d17 fix(codex): forward drained notifications to on_event during approval roundtrips
The approval-drain loop in CodexAppServerSession.run_turn drains up to 8
pending notifications to keep per-turn state current before answering a
server-initiated approval request — but never forwarded them to the
on_event display hook. Tool bubbles for items drained alongside an
approval (e.g. the item/started for the very command awaiting approval)
silently disappeared.

Mirror the main notification path's on_event invocation in the drain
loop. Regression test demonstrates RED→GREEN.

Grafted from PR #26541 by @simpolism — the earliest submission of the
codex app-server display-bridge family (May 15). Confirmed independently
by #64698 and #65412.
2026-07-17 06:49:47 -07:00
Teknium
c7205040c3
fix(compression): affirm tool use stays active in the compaction handoff prefix (#66291)
The REFERENCE ONLY framing ('treat as background reference, NOT as active
instructions... Do NOT answer questions or fulfill requests') was observed
bleeding into general tool-use suppression: a production session went
narration-only for 7 consecutive turns immediately after a compression
event, describing edits instead of calling tools (#65848 report).

Fix is additive: one clause stating the note does not restrict HOW the
agent works — tools remain fully active for the active task. Every
anti-resumption protection stays intact; the previous prefix generation
is frozen into _HISTORICAL_SUMMARY_PREFIXES per the module contract so
persisted summaries still get the directive-strip on re-compaction.

The #65848 rewrite was not taken: dropping the 'Do NOT answer questions'
line and the four-heading discard directive risks re-opening the
stale-task-resumption class those clauses exist to prevent (the carveout
era regressions #41607/#38364/#42812 documented in this file).

Report and root-cause analysis: @yasserbousrih (#65848).
2026-07-17 06:49:42 -07:00
HexLab98
35cbffd5c8 test(streaming): cover the single-writer invariant for superseded streams
Assert that a superseded stream (older writer token, other thread) is fenced
from the delta sink, the active writer is never fenced, a non-claiming thread
is never treated as a writer, and the real consume loop stops the instant it is
superseded — so two streams can never interleave into one turn (#65991).
2026-07-17 06:49:23 -07:00
Teknium
348e9912ff
fix(agent): execute valid tool calls in mixed batches with invalid names (#66317)
Degrading models (observed with gpt-5.6 past ~350K input) emit tool-call
batches like 6 valid named calls + 1 blank-name call. Previously the
whole turn was voided — every valid call got 'Skipped: another tool call
in this turn used an invalid name' — and three such batches tripped the
3-strike stop, killing sessions that were still making progress.

Now a mixed batch error-results ONLY the invalid call(s) (terse
anti-priming error for blank names per #47967, catalog dump for typos)
and dispatches the valid subset for execution. The assistant message
keeps every emitted call so provider-side tool_call/result pairing stays
intact. The 3-strike counter only advances when a turn contains NO valid
call, so a fully-degenerate model still stops while a mostly-coherent
one keeps working. Broken JSON args on a never-executing invalid call no
longer trigger the whole-turn JSON retry loop.

Field evidence: July 2026 debug bundle showed gpt-5.6-sol emitting
6-call batches with one blank-name rider at 559K/384K-token context in
two separate sessions; 13 valid tool calls were discarded before the
session stopped as partial.
2026-07-17 06:48:42 -07:00
Teknium
b41b4b3ec0
test(file-safety): unbreak session-snapshot suite; de-flake fixture to env-var resolution (#66293)
Two changes to tests/agent/test_file_safety_session_state.py:

1. Drop the stale monkeypatch on tools.file_tools._get_live_tracking_cwd
   — the helper was deleted in the cwd-tracking refactor (c80b244b5),
   and monkeypatch.setattr on a missing attribute raises AttributeError,
   breaking CI slice 4/8 on main for every PR. The patch was redundant:
   the test writes an absolute path, so cwd resolution never engages.

2. Make the fixture stale-proof: instead of monkeypatching the private
   _hermes_home_path/_hermes_root_path helpers (same failure class if
   they're ever renamed), set HERMES_HOME to <root>/profiles/work and
   let the real resolution chain (get_hermes_home /
   get_default_hermes_root's profiles-parent rule) derive both paths.
   The fixture now references zero private symbols and exercises the
   production resolution path.
2026-07-17 05:45:12 -07:00
Hermes Agent
174fc958ab fix(errors): classify Z.AI GLM token-limit message as context overflow
Port from anomalyco/opencode#35671: Z.AI / Zhipu GLM returns
'tokens in request more than max tokens allowed' (error code 1210) on
context overflow. This matched no pattern in _CONTEXT_OVERFLOW_PATTERNS,
so the error classified as unknown/retryable — the agent would retry the
oversized request instead of triggering context compression.

Proven live on main before the fix: classify_api_error() returned
FailoverReason.unknown for the exact Z.AI error shape; now returns
context_overflow.
2026-07-17 04:57:27 -07:00
teknium1
c356752b6b fix(memory): drain queued writes on shutdown 2026-07-17 04:55:58 -07:00
teknium1
b4221c6db2 Inspired by Claude Code: protect session transcripts 2026-07-17 04:54:34 -07:00
Teknium
b78ff50d8d fix(gemini): prune required entries missing from properties in tool schemas
Port from Kilo-Org/kilocode#11955: Gemini validates every object schema's
required list strictly against the same node's properties and fails the
ENTIRE GenerateContentRequest with HTTP 400 'required[0]: property is not
defined' when a name has no matching property. MCP servers (e.g. the GitHub
remote MCP) routinely emit array item schemas carrying required without
properties, which made every request on the native Gemini path fail before
any model output.

sanitize_gemini_schema() now filters required to names present in the node's
properties and drops the keyword when nothing valid remains. Applies
recursively (properties / items / anyOf). Tool handlers still validate
required fields at execution time, so nothing the model could actually use
is lost.

Scoped to the Gemini-facing sanitizer only — the universal
tools/schema_sanitizer.py already prunes typed object nodes, and its
remaining gap (untyped nodes) is contested by open PR #20151.
2026-07-17 04:54:06 -07:00
Teknium
780e098077 fix: widen UTF-8 BOM tolerance to all sibling frontmatter parsers
The previous commit fixes the canonical agent/skill_utils.parse_frontmatter.
Six more modules reimplement the '---' fence check locally and had the
same bug:

- tools/skill_manager_tool.py _validate_frontmatter — rejected BOM'd
  skill_manage create/edit content outright
- tools/skills_hub.py GitHubSource._parse_frontmatter_quick and
  OptionalSkillSource._parse_frontmatter — hub browse/install metadata
- hermes_cli/skills_hub.py — local skill install validation
- gateway/run.py — skill slug discovery for disabled-skill hints
- agent/prompt_builder.py _strip_yaml_frontmatter — BOM'd context files
  (AGENTS.md) leaked raw frontmatter into the system prompt
- tools/blueprints.py _split_frontmatter — str.lstrip() does not strip
  U+FEFF (not whitespace), so the existing lstrip never covered it

Sibling-surface regression tests added.
Bug class also fixed upstream in cline/cline#12218 (found by the weekly
Cline PR scout).
2026-07-17 04:52:02 -07:00
Que0x
a4ecb3da9a fix(skills): strip UTF-8 BOM before parsing SKILL.md frontmatter
A UTF-8 BOM saved into a SKILL.md (e.g. Notepad or PowerShell `>`) is kept by
read_text(encoding="utf-8"), so the string handed to parse_frontmatter starts
with the BOM and the startswith("---") fence check fails. The whole frontmatter
is then silently dropped: the skill loads with no name/description, `platforms`
gating falls open (a macOS-only skill becomes visible everywhere), and
required_environment_variables / metadata.hermes.config setup never fires.

Strip a single leading BOM at the top of parse_frontmatter, the shared
chokepoint for every local skill-loading path (_parse_skill_file,
discover_all_skill_config_vars, DESCRIPTION.md parsing, _inject_skill_config,
and the tools/skills_tool._parse_frontmatter re-export), so the whole class is
covered, not just the reported site. Only the leading marker is removed; a BOM
mid-content is left as data. Mirrors the existing file-tools BOM handling
(#35278) and CONTRIBUTING.md "File encoding".

Adds tests: BOM'd frontmatter parses identically to plain, the body is
BOM-free, platforms gating and config-var extraction survive a BOM, and an
end-to-end BOM-write / plain-read round trip (mirroring _parse_skill_file).
2026-07-17 04:52:02 -07:00
Teknium
14ea8de763 fix(agent): harden non-finite wait recovery
Only advertise finite watchdog deadlines that are still in the future, exercise the full MoA heartbeat path, and register the salvaged contributor attribution.
2026-07-17 04:46:59 -07:00
发飙的公牛
1a5d2a12d8 fix: handle infinite Codex wait deadlines 2026-07-17 04:46:59 -07:00
Teknium
60419dfb4b fix(codex): reconcile app-server bridge with #38835, gate commentary on show_commentary
Follow-ups on top of @xxxigm's salvaged bridge (#33294):

- Remove the now-dead narrow item/started-only mapper from #38835
  (_codex_note_to_tool_progress) — the full bridge supersedes it and
  keeps the same tool-name contract; its tests are repointed at the
  bridge helpers.
- Preserve main's request_routing/approval-bypass wiring on the
  CodexAppServerSession constructor (landed after the PR was filed).
- Gate agentMessage interim delivery on display.show_commentary so the
  app-server runtime honors the same toggle as the codex_responses
  commentary channel (tool progress is unaffected).
- Add json import (bridge helpers use json.dumps) and modernize the
  wiring test's stub agent for main's usage-accounting attributes.
2026-07-17 04:32:56 -07:00
xxxigm
68d5368f38 test(codex): regression coverage for app-server event bridge (#33200)
42 tests across five suites:

* ``TestCodexItemToToolName`` / ``TestCodexItemToArgs`` /
  ``TestCodexItemToPreview`` / ``TestCodexItemCompletionPayload`` —
  pin the per-type mapping so the synthetic tool name + args the
  UI sees match what ``CodexEventProjector`` writes into messages.
* ``TestStreamDeltaDispatch`` / ``TestToolProgressDispatch`` /
  ``TestAgentMessageInterimDispatch`` — drive each Codex
  notification shape through the bridge and assert the right
  agent callback fires with the right arguments (including the
  duration / is_error / result kwargs the gateway renders).
* ``TestBridgeRobustness`` — defensive paths: non-dict
  notifications, missing params, raising callbacks (must not
  tear down the codex turn loop), and agents without callbacks
  registered (cron / gateway-less contexts).
* ``TestBridgeWiredInRuntime`` — integration guard that
  ``run_codex_app_server_turn`` actually constructs the session
  with ``on_event=<bridge>``, preventing a future refactor from
  silently regressing live progress visibility again.
2026-07-17 04:32:56 -07:00
cresslank
34837597d2 fix(auth): make xAI OAuth pools multi-account resilient
Keep each xAI OAuth auth-add login as an independent manual device-code pool entry and recognize xAI personal-team spending-limit 403 responses as billing exhaustion. Preserve the structured top-level error message so the failed credential is quarantined and the next healthy account is selected without attempting a pointless token refresh.

Route direct xAI HTTP consumers through the credential pool as well. Proactive and 401-reactive refreshes update the exact issuing manual entry, preserve validated xAI base URL overrides, and serialize single-use refresh-token rotation across concurrent pool instances.
2026-07-17 11:37:25 +05:30
Thatgfsj
ef1c622105 fix(title): prevent stale background title generation from reloading unloaded Ollama models
Add a runtime_validator callback to generate_title() / auto_title_session()
/ maybe_auto_title(). Callers snapshot the session's model+provider when
spawning the background titler; the validator runs right before the LLM
request and skips it silently when the live runtime no longer matches —
so a stale title request can't reload a model that strict_single_load
already evicted after a user model switch. Fail-open: a raising validator
never disables titling.

Wired at all four call sites (cli, gateway, tui_gateway, acp_adapter).

Surgical reapply of PR #19137 (base was 8k+ commits stale; the original
patch predates the pinned-language prompts, the atomic-write helper, and
the moved TUI/ACP call sites). Original work by @Thatgfsj. Closes #19027.
2026-07-16 23:07:13 -07:00
Teknium
7facf63ae7 fix(title): reconcile atomic auto-title writes with collision dedup retry
Combines the two salvaged fixes so they compose instead of conflict:
_persist_session_title (#50575) now writes through set_auto_title_if_empty
(#51483) when the store provides it — the collision-dedup retry and the
manual-/title race protection apply together. Predicate failure (a manual
title landed while generation was in flight) returns None: nothing written,
no callback. Legacy stores without the atomic method keep the plain
set_session_title path, including the vanished-session RuntimeError.

Tests cover both store shapes plus the race-skip path; E2E verified against
a real SQLite SessionDB (collision -> 'Weekly Report #2', manual title
preserved, cron dedup, blank guard). AUTHOR_MAP entry for rasitakyol.
2026-07-16 22:39:47 -07:00
Trevor Gordon
9bf5822a2f fix(cron): robust session title generation (#50535, #50536, #50537) 2026-07-16 22:39:47 -07:00
Raşit Akyol
d05cd7c1ef fix(agent): make auto-title write atomic 2026-07-16 22:39:47 -07:00
Raşit Akyol
f725cf830f fix(agent): avoid overwriting manual session titles 2026-07-16 22:39:47 -07:00
Teknium
8222b16785 fix(title): follow-ups for salvaged #37349 — lazy config import, guard ordering, config example
- Make the config imports lazy inside _auto_title_enabled(), matching the
  existing _title_language() pattern (title_generator is imported from agent
  code paths where a module-level hermes_cli import risks circularity).
- Check the enabled flag after the cheap first-exchange guard in
  maybe_auto_title so config isn't read on every turn of a long session.
- Repoint the two new tests at the real import site.
- Document the key in cli-config.yaml.example and merge the enabled flag
  into the existing title_generation block in configuration.md.
- AUTHOR_MAP entry for the contributor.
2026-07-16 22:24:45 -07:00
Danny Huang
e20c3c1c29 fix: honor disabled title generation config 2026-07-16 22:24:45 -07:00
amanning3390
311a5b0a55 feat(kimi): discover K3 on coding endpoint 2026-07-16 13:33:02 -07:00
Erosika
8d1c96fd2f fix(memory): align external prefetch guard with fail-open contracts 2026-07-16 12:48:48 -07:00
LeonSGP43
d77c455d7d fix(memory): fail fast on stuck external prefetch 2026-07-16 12:48:48 -07:00
Teknium
31a3822b80
fix(title): contain auto-title thread exceptions instead of dumping tracebacks to the terminal (#65792)
auto_title_session runs as a bare daemon-thread target. Any exception
escaping it hits the default threading excepthook and sprays a raw
traceback into the user's terminal mid-session. The canonical trigger
is the post-'hermes update' stale-module window: the function's lazy
imports read NEW source from disk while already-imported modules
(agent.portal_tags) are still the OLD cached version, producing an
ImportError that repeats on every auto-title attempt until the
long-running process restarts (seen live after 9ce0e67f2 added
set_conversation_context).

The public entrypoint now wraps the body in a catch-all that logs one
WARNING naming the likely cause ('restart the running Hermes process'),
routes the exception through the existing failure_callback channel
(user-visible warning in CLI, debug-suppressed in gateway per #23246),
and never re-raises. This also makes the function honor its own
docstring contract ('silently skips if title generation fails').
2026-07-16 12:41:56 -07:00
briandevans
9078a838c7 fix(lmstudio): clamp max/ultra reasoning effort to LM Studio's ceiling
LM Studio's request vocabulary tops out at "xhigh", but Hermes' generic
effort ladder has since grown two stronger levels. "max" and "ultra" miss
the _LM_VALID_EFFORTS membership test, keep the initialized "medium"
default, and are thereby conflated with unparseable input -- so asking for
more reasoning yields less than "xhigh":

    high  -> 'high'     xhigh -> 'xhigh'
    max   -> 'medium'   ultra -> 'medium'

This is drift, not a design choice. The valid set was an exact mirror of
VALID_REASONING_EFFORTS when the file was authored; the ladder then grew
"max" and later "ultra", and the sweep that taught every other provider
about the new levels missed this module -- it has never been touched since
it was written.

Clamp the two stronger levels onto LM Studio's declared ceiling instead,
mirroring the ceiling clamp every other provider already applies. Widening
_LM_VALID_EFFORTS would instead assert that LM Studio accepts "max" on the
wire, which is a provider-side claim this repo cannot verify; clamping
consumes only the ceiling the file already declares for itself.

The clamp is kept separate from _LM_EFFORT_ALIASES because that mapping is
also applied to the model's published allowed_options, which must not be
rewritten. A clamped value stays subject to the allowed_options check, so a
model that does not publish "xhigh" still gets the field omitted and falls
back to its own default -- exactly how a directly-requested "xhigh" behaves.

The regression test asserts monotonicity over the canonical ladder rather
than the two values alone, so the next level added upstream cannot silently
reintroduce the inversion.
2026-07-16 07:57:51 -07:00
Teknium
03c0b00f45
fix(usage): read DeepSeek's native prompt_cache_hit_tokens cache field (#65678)
Some checks are pending
CI / Detect affected areas (push) Waiting to run
CI / Python tests (push) Blocked by required conditions
CI / Python lints (push) Blocked by required conditions
CI / JS & TS checks (push) Blocked by required conditions
CI / Docs Site (push) Blocked by required conditions
CI / Deny unrelated histories (push) Blocked by required conditions
CI / Check contributors (push) Blocked by required conditions
CI / Check uv.lock (push) Blocked by required conditions
CI / package-lock.json diff (push) Blocked by required conditions
CI / Lint Docker scripts (push) Blocked by required conditions
CI / Build&Test Docker image (push) Blocked by required conditions
CI / Supply-chain scan (push) Blocked by required conditions
CI / OSV scan (push) Waiting to run
CI / All required checks pass (push) Blocked by required conditions
CI / CI timing report (push) Blocked by required conditions
Deploy Site / deploy-vercel (push) Waiting to run
Deploy Site / deploy-docs (push) Waiting to run
auto-fix lint issues & formatting / Generate eslint --fix patch (push) Waiting to run
auto-fix lint issues & formatting / Apply patch (push) Blocked by required conditions
DeepSeek's own API (api.deepseek.com) reports context-cache hits as
top-level usage.prompt_cache_hit_tokens / prompt_cache_miss_tokens
(prompt_tokens = hit + miss), not the OpenAI nested
prompt_tokens_details.cached_tokens shape. Neither normalize_usage()
nor the chat_completions transport's extract_cache_stats() read those
fields, so direct DeepSeek sessions always showed 0 cache-hit tokens:
invisible in accounting, mis-billed at the full input rate, and 0%
cache display.

Both layers now fall back to prompt_cache_hit_tokens when the nested
shape is absent; the nested value wins when both are present (proxies).

Fixes #61871.
2026-07-16 07:29:53 -07:00
Teknium
adb647269a fix(auxiliary): apply review fixes to #36043 — guard named-custom routing, drop dead key assignment, tighten Palantir host match
Review follow-ups on the cherry-picked #36043 commit:

1. Guard the custom:<name> passthrough with a _get_named_custom_provider
   lookup. The PR unconditionally kept the full custom:<name> string, which
   broke config-less runtime custom providers (#34777 regression — entries
   that exist only in the live runtime, not config.yaml): the named arm
   found no entry and resolution fell through to Step 2. Now custom:<name>
   only takes the named arm when a config entry actually exists; otherwise
   it collapses to the anonymous-custom arm with the runtime endpoint,
   preserving pre-PR behavior.

2. Drop the dead 'explicit_api_key = runtime_api_key' assignment (and its
   misleading comment) in the named-entry branch. resolve_provider_client's
   named-custom arm derives the key exclusively from the entry's
   api_key/key_env and never reads explicit_api_key, so the assignment was
   a no-op. Wiring precedence in was not justified: for a named custom
   provider the runtime key IS the entry's key (set_runtime_main sources it
   from the same config), so deletion is the honest option.

3. Tighten the Palantir Bearer-auth check from a loose substring match
   ('palantirfoundry' in normalized) to a hostname match via
   base_url_host_matches(..., 'palantirfoundry.com'), so path segments or
   lookalike domains containing the string no longer trigger Bearer auth.

Tests: named-custom anthropic_messages end-to-end routing (full name kept,
AnthropicAuxiliaryClient at the original /anthropic URL, no /v1 rewrite)
plus Palantir Bearer-auth positive and substring-false-positive cases.
2026-07-16 07:28:07 -07:00
Dan Schnurbusch
679487b807 fix(auth): enforce complete fallback routes
Skip provider-only setup fallbacks, keep fallback selection explicit for resumed sessions, preserve configured primary identity for cron drift checks, and make the auth lost-update regression deterministic.
2026-07-16 06:14:56 -07:00
Dan Schnurbusch
f68fd80f41 fix(auth): preserve fallback routes and OAuth state
Switch provider and model together after setup-time auth failure. Serialize global auth-store merges under target-specific locks and preserve auth-to-shared lock ordering for profile OAuth refreshes.
2026-07-16 06:14:56 -07:00
Teknium
a61e29e879 fix(pricing): refresh full DeepSeek snapshot to 2026-07 rates
Widens the deepseek-v4-flash addition to the whole stale-snapshot class:
- deepseek-v4-pro: $1.74/$3.48 → $0.435/$0.87, cache-read $0.003625
  (DeepSeek's 2026-07 price cut; every pro session was over-reporting 4x)
- deepseek-chat / deepseek-reasoner: deprecated 2026-07-24, now alias
  v4-flash non-thinking/thinking modes — repriced to match flash
  (reasoner was $0.55/$2.19 with no cache rate)
- cache_read added to every row; pricing_version unified at
  deepseek-pricing-2026-07
- invariant tests: aliases price identically to flash; every deepseek
  row carries cache_read < input
2026-07-16 05:56:22 -07:00
shuangxinniao
97397a1ccc feat(pricing): add deepseek-v4-flash to official-docs pricing snapshot
DeepSeek's /models endpoint returns no pricing, so direct-provider routes fall back to the _OFFICIAL_DOCS_PRICING snapshot. The table included deepseek-v4-pro but not the newer deepseek-v4-flash, so flash sessions reported $0.00 with cost_source "none". Add the flash entry (values from DeepSeek's official pricing page, mirroring the v4-pro entry; DeepSeek bills no separate cache-write cost) plus two regression tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:56:22 -07:00
Regina
59787b9ada chore(agent): tripwire — warn when a turn starts before the previous turn's persist
Two turns interleaving on one session corrupt the durable transcript:
flushes race (user rows persist out of arrival order), the identity-marker
dedup over shared history dicts can swallow a row, and the second turn
runs on a history base that never saw the first turn's exchange. The
dispatch route that lets the second turn through the busy guard is not
yet identified.

Add note_turn_start (build_turn_context) / note_turn_persisted
(_persist_session funnel): one WARNING naming both turn_ids when a turn
starts before the previous turn's turn-end persist. Ownership transfer
keeps a crashed turn from warning more than once; the unconditional clear
makes the tripwire under-report rather than double-report under a real
overlap. Log-only, no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 04:56:10 -07:00
Teknium
244f70aae5 fix(agent): scope install-tree guard to fallback-picked cwds, allow cli/tui in-tree dev
Follow-up on the salvaged #64611 commit: the original guard blocked the
install tree unconditionally, which would have broken the legitimate
'developing Hermes from a source clone' CLI flow (launching hermes inside
the repo and getting its AGENTS.md as project context).

Refined policy:
- resolve_context_cwd(): validates configured paths (missing dir -> None +
  warning) but honors an EXPLICIT install-tree cwd verbatim — deliberate
  user choice.
- build_context_files_prompt(): blocks only the cwd=None -> os.getcwd()
  FALLBACK into the install tree, with a new allow_install_tree_fallback
  param. system_prompt.py passes it for platform cli/tui (launch dir is
  the user's real shell cwd there); desktop/gateway surfaces keep the
  guard (their fallback dir is self-spawned, never user-picked).
- Warning log names the resolved dir and the terminal.cwd remedy.

E2E-verified all five scenarios: desktop fallback blocked, in-tree CLI dev
keeps AGENTS.md, explicit install-tree cwd honored, invalid TERMINAL_CWD
falls to None then blocked, normal workspace loads.
2026-07-16 04:32:23 -07:00
Evelyn Bruce
33513991be fix(agent): never load the install-tree AGENTS.md as project context 2026-07-16 04:32:23 -07:00
Teknium
fe5c0cb6c3 feat(pricing): refresh Fireworks snapshot to 2026-07, cover full serverless catalog + cached picker pricing
- Refresh _OFFICIAL_DOCS_PRICING fireworks entries against current
  docs.fireworks.ai/serverless/pricing: qwen3p6-plus is gone (replaced
  by qwen3p7-plus); add glm-5p2/5p1, kimi-k2p7-code, deepseek-v4-flash,
  minimax-m3/m2p7, gpt-oss-120b/20b, and the routers/*-fast tiers with
  their distinct higher rates.
- Picker pricing via get_pricing_for_provider('fireworks'): pure dict
  transform over the shared models.dev in-memory/disk cache (1h TTL) +
  _pricing_cache memoization — no new network call on the picker path.
- Wire pricing display into the generic api-key-provider setup flow so
  Fireworks model pickers show $/M columns like OpenRouter/Nous do.
- Invariant tests: plugin fallback_models all priced, fast tiers price
  higher than standard, every row carries cache_read < input.
2026-07-16 04:24:14 -07:00
Brendan DeBeasi
365620ab28 feat(agent): add Fireworks pricing entries + routing branch
Fireworks-hosted sessions previously showed estimated_cost_usd = 0
because (a) _OFFICIAL_DOCS_PRICING had no Fireworks entries and (b)
resolve_billing_route() had no branch for provider="fireworks",
falling through to billing_mode="unknown".

Adds entries for the three Fireworks models hermes operators are
most likely to route through (Kimi K2.6, DeepSeek V4 Pro, Qwen3.6-Plus)
and a routing branch that triggers on either explicit
provider="fireworks" or api.fireworks.ai base_url match. Mirrors the
recently-merged MiniMax addition pattern; pricing snapshot sourced
from https://docs.fireworks.ai/serverless/pricing and the per-model
pages on fireworks.ai.

Tests cover: (a) full Fireworks model id resolves to the snapshot
entry, (b) base_url alone is sufficient to route, (c) end-to-end
estimate returns "estimated" status with the expected dollar amount.

A follow-up upstream issue is open proposing a dynamic pricing
source (e.g. litellm's pricing JSON) as a permanent fix to the
PR-per-model treadmill that this snapshot keeps adding to.
2026-07-16 04:24:14 -07:00