hermes-agent/agent
Ziliang Peng 17cfa0f0a5 fix(background_review): inherit parent's reasoning_config to preserve Anthropic cache namespace
PR #17276 painstakingly pinned `_cached_system_prompt`, `session_start`,
`session_id`, and the toolset config on the background-review fork so its
outbound request body would byte-match the parent's and hit Anthropic's
exact-prefix cache. The contributor measured a ~26% end-to-end cost
reduction on Sonnet 4.5.

That optimization is currently being silently undone by a missing
`reasoning_config` kwarg. The fork's `AIAgent(...)` call omits it, so the
fork's `reasoning_config` defaults to `None`. `anthropic_adapter.build_anthropic_kwargs`
(line ~2165) then short-circuits the `thinking` / `output_config` block,
and the fork's request body lands in a DIFFERENT Anthropic cache namespace
from the parent's.

Result on the wire: 0 `cache_read_input_tokens`, full `cache_creation_input_tokens`
of the entire parent prefix — every single background review.

7 days of midagent.db traffic from one host running stock Hermes against
Anthropic Sonnet:

```
Background-review FIRST calls (the moment a review fork is born):
  count = 68
  cache_write tokens = 7,004,297
  cache_read tokens  = 1,016,335

Cost on Sonnet ($3.75/M write vs $0.30/M read):
  Spent on these writes:                      $26.27
  Cost if they had hit parent cache instead:   $2.10
  WASTED:                                     $24.16 / week / user
```

That is from one user. Multiply by Hermes's installed base for the full
impact.

Tested against api.anthropic.com directly (see refs/api-tests/ in the
attached investigation repo if needed):

| pair                                        | cache_r | cache_w |
|---------------------------------------------|---------|---------|
| parent fresh                                |       0 |  24,047 |
| parent same again                           |  24,047 |       0 |
| fork: appends 2 new tail msgs, thinking ON  |  24,047 |      22 |
| fork: appends 2 new tail msgs, thinking OFF |       0 |  24,047 |

Same fork-shape request, only difference is `thinking`. With the fix,
the fork hits the parent's full prefix and only writes the delta
(the `Review the conversation above…` prompt block, ~3-5K tokens).

One line in `agent/background_review.py`: pass
`reasoning_config=getattr(agent, "reasoning_config", None)` to the
`AIAgent(...)` constructor of the review fork. A short comment block
above it explains why so the next person who reads this code doesn't
re-introduce the regression.

`tests/run_agent/test_background_review_cache_parity.py` already covers
the system-prompt / session-id / toolset-config parity contracts that
PR #17276 introduced. I added:

* a `reasoning_config` attribute to `_make_agent_stub` so the stub has
  a non-None parent value the test can verify is propagated.
* `test_review_fork_inherits_parent_reasoning_config()` — asserts the
  fork's `AIAgent(...)` kwargs carry the parent's `reasoning_config`.
  Pre-fix this test fails with `None vs expected {'enabled': True, 'effort': 'medium'}`;
  post-fix all 4 tests in the file pass.

```
$ python -m pytest tests/run_agent/test_background_review_cache_parity.py -v
test_review_fork_inherits_parent_cached_system_prompt    PASSED
test_review_fork_pins_session_start_and_session_id       PASSED
test_review_fork_inherits_parent_toolset_config          PASSED
test_review_fork_inherits_parent_reasoning_config        PASSED  ← new
```

Also runs against the broader background-review test suite:
`test_background_review.py` (4), `test_background_review_summary.py` (8),
`test_background_review_toolset_restriction.py` (3) — 19/19 pass.

`agent/curator.py:1691` has the same omission for the umbrella-curation
fork, but curator's prompt is "curate all skills" — it shares no prefix
with any user conversation, so cache-parity is a non-issue there. Worth
auditing if the curator ever takes a parent conversation as input, but
not part of this PR.

The `agent/auxiliary_client.py:1006` `reasoning_config=None` hardcode is
intentional (title/summary one-shots on short prompts — per-call cost
of namespace flip is negligible) and is also out of scope.
2026-07-14 17:19:46 +05:30
..
lsp refactor(setup): route dependency installs through the canonical uv→pip→ensurepip ladder 2026-07-07 04:09:35 -07:00
pet fix: remove dead f-string prefixes via ruff F541 (216 sites) (#52336) 2026-07-05 13:42:46 -07:00
secret_sources feat(secrets): adapt 1Password onto the SecretSource interface 2026-07-06 04:58:07 -07:00
transports feat(reasoning): add max and ultra effort levels (#62650) 2026-07-12 00:26:49 -07:00
__init__.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
account_usage.py feat(codex): redeem banked usage-limit resets via /usage reset (#64280) 2026-07-14 03:23:19 -07:00
agent_init.py fix(agent): validate credential pool after provider auto-detection (#63425) 2026-07-14 16:51:35 +05:30
agent_runtime_helpers.py fix(agent): restore primary credential pool after fallback (#62417) 2026-07-11 03:44:02 -07:00
anthropic_adapter.py feat(reasoning): add max and ultra effort levels (#62650) 2026-07-12 00:26:49 -07:00
async_utils.py fix(async): close unscheduled coroutines in all threadsafe bridges (#26584) 2026-05-15 14:00:01 -07:00
auxiliary_client.py feat(providers): Support DeepInfra as an LLM provider 2026-07-14 02:59:39 +05:30
azure_identity_adapter.py feat(azure-foundry): add Microsoft Entra ID auth 2026-05-18 10:14:38 -07:00
background_review.py fix(background_review): inherit parent's reasoning_config to preserve Anthropic cache namespace 2026-07-14 17:19:46 +05:30
bedrock_adapter.py fix(bedrock): check boto3 version >= 1.34.59 before using converse_stream 2026-06-15 05:25:17 -07:00
billing_view.py feat(billing): /billing terminal billing — interactive TUI + CLI client (#45449) 2026-06-19 01:53:32 +05:30
bounded_response.py fix(agents): bound streaming error-response body reads 2026-07-05 14:00:20 -07:00
browser_provider.py fix(browser): self-review pass — dead-import, log levels, future-proofing 2026-05-17 04:04:15 -07:00
browser_registry.py style: restore PEP8 blank-line separation after dead-code removal 2026-05-29 04:22:27 -07:00
chat_completion_helpers.py fix(cron): scope inline calls to reported transport 2026-07-11 11:07:34 +05:30
codex_responses_adapter.py fix(codex): enforce Copilot replay policy at dispatch 2026-07-11 12:09:27 +05:30
codex_runtime.py fix(context): persist fallback compaction breaker 2026-07-14 02:19:40 +05:30
coding_context.py test: deflake CI and dev-machine flaky tests in bulk (11 tests, 10 files) (#61816) 2026-07-09 20:03:11 -07:00
context_breakdown.py feat(desktop): add context usage breakdown popover 2026-06-29 09:18:10 -04:00
context_compressor.py fix(context): persist fallback compaction breaker 2026-07-14 02:19:40 +05:30
context_engine.py fix(context): clamp -1 post-compression sentinel in sibling status paths 2026-07-01 13:36:50 +05:30
context_references.py fix(security): anchor @file context refs to canonical read deny-list 2026-07-01 02:43:49 -07:00
conversation_compression.py fix(context): persist fallback compaction breaker 2026-07-14 02:19:40 +05:30
conversation_loop.py Merge pull request #64004 from kshitijk4poor/salvage/63274-cli-close-persist 2026-07-14 16:45:35 +05:30
copilot_acp_client.py fix(file-safety): distinguish safe-root write denial from credential blocks 2026-07-14 17:09:40 +05:30
credential_persistence.py feat(auth): make xAI Grok OAuth device-code-only, drop loopback login 2026-07-02 13:17:41 -07:00
credential_pool.py fix(auth): centralize pool auth normalization 2026-07-13 23:20:48 +05:30
credential_sources.py feat(auth): make xAI Grok OAuth device-code-only, drop loopback login 2026-07-02 13:17:41 -07:00
credits_tracker.py feat(billing): /credits command — balance + portal top-up handoff (#44776) 2026-06-12 08:51:10 +00:00
curator.py fix(runtime): preserve resolved fork metadata 2026-07-10 18:32:32 +05:30
curator_backup.py fix: remove dead f-string prefixes via ruff F541 (216 sites) (#52336) 2026-07-05 13:42:46 -07:00
display.py fix(display): harden fallback label formatting 2026-07-10 19:14:06 +05:30
error_classifier.py fix(xai): recover legacy encrypted replay failures (#62420) 2026-07-11 03:51:38 -07:00
errors.py fix(agent,gateway,doctor): add SSL CA cert bundle fail-fast guard 2026-06-13 21:14:32 -07:00
file_safety.py fix(file-safety): distinguish safe-root write denial from credential blocks 2026-07-14 17:09:40 +05:30
gemini_native_adapter.py feat(gemini): improve request context for support and compatibility 2026-07-13 22:26:19 -07:00
gemini_schema.py chore: remove unused imports and dead locals (ruff F401, F841) (#17010) 2026-04-28 06:46:45 -07:00
i18n.py fix(packaging): ship locales/ i18n catalogs in wheel, sdist, and Nix (#38383) 2026-06-03 12:00:27 -07:00
image_gen_provider.py feat(image-gen): add image-to-image / editing to image_generate (#48705) 2026-06-18 22:13:07 -07:00
image_gen_registry.py fix(plugins): filter resolution by is_available() in web + image_gen registries 2026-05-13 22:31:28 -07:00
image_routing.py Guard native image routing with file safety 2026-07-05 03:15:03 -07:00
insights.py fix(insights): harden per-route usage attribution 2026-07-11 05:59:42 -07:00
iteration_budget.py refactor(run_agent): extract OpenAI proxy, safe stdio, IterationBudget 2026-05-16 17:59:32 -07:00
jiter_preload.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
kanban_stop.py follow-up: integrate agent nudge + dispatcher retry docs and tests 2026-07-14 16:47:33 +05:30
learn_prompt.py fix(learn): honor requirements mixed with sources in /learn requests (#55956) 2026-06-30 16:56:01 -07:00
learning_graph.py fix(learning_graph): guard non-dict metadata so /journey can't crash 2026-07-01 16:25:48 -05:00
learning_graph_render.py fix: cover remaining GNU-only %-d strftime site in learning graph render 2026-07-05 00:59:35 -07:00
learning_mutations.py refactor(journey): route memory mutations through MemoryStore atomic I/O 2026-06-30 15:16:21 -05:00
lmstudio_reasoning.py feat(agent): add lmstudio integration 2026-04-28 12:27:36 -07:00
manual_compression_feedback.py fix(compression): include system prompt + tool schemas in token estimates (#18265) 2026-04-30 23:03:54 -07:00
markdown_tables.py fix(cli): vertical fallback for markdown tables wider than terminal (#23948) 2026-05-11 16:49:13 -07:00
memory_manager.py refactor: address Phase-2 review findings on /new boundary handoff 2026-07-09 03:21:54 +05:30
memory_provider.py fix(backup): capture memory-provider state stored outside HERMES_HOME (#50325) 2026-06-21 12:03:46 -07:00
message_content.py fix(openviking): preserve structured sync attribution 2026-06-19 15:23:41 +08:00
message_sanitization.py fix(agent): close tool-call sequence on all interrupt aborts, not just finalize_turn 2026-06-25 12:24:34 -05:00
moa_loop.py fix(moa): flatten structured message content in the advisory view (#64319) 2026-07-14 03:22:48 -07:00
moa_trace.py fix(moa): capture streamed aggregator output into full-turn traces (#56312) 2026-07-01 04:07:46 -07:00
model_metadata.py fix(deepinfra): restore provider-prefix aliases for model parsing 2026-07-14 03:34:25 +05:30
models_dev.py remove Vercel AI Gateway and Vercel Sandbox (#33067) 2026-05-27 00:43:32 -07:00
moonshot_schema.py fix(moonshot): handle union type arrays in tool schemas 2026-06-13 05:51:41 -07:00
nous_rate_guard.py codebase: add encoding='utf-8' to all bare open() calls (PLW1514) 2026-05-08 14:27:40 -07:00
onboarding.py fix(config): close unreadable-overwrite bug class at a single chokepoint 2026-07-05 23:00:34 +05:30
oneshot.py feat(agent): one-shot LLM helper + llm.oneshot gateway RPC (#51261) 2026-06-23 08:01:50 +00:00
plugin_llm.py feat(plugins): run any LLM call from inside a plugin via ctx.llm (#23194) 2026-05-10 07:09:28 -07:00
portal_tags.py feat(nous): unified client=hermes-client-v<version> tag on every Portal request (#24779) 2026-05-12 20:49:20 -07:00
process_bootstrap.py fix(agent): apply pool-level keepalive to the process_bootstrap sibling builder 2026-07-05 03:14:55 -07:00
prompt_builder.py perf(skills): speed up snapshot prompt builds 2026-07-09 02:41:24 +05:30
prompt_caching.py refactor(salvage): scope #40632 to the two live copy-on-write sites 2026-07-09 14:35:14 +05:30
rate_limit_tracker.py refactor: remove dead code — 1,784 lines across 77 files (#9180) 2026-04-13 16:32:04 -07:00
reactions.py feat(agent): core affection reaction detector + reaction_callback 2026-07-10 05:41:59 -05:00
reasoning_timeouts.py fix(reasoning): add deepseek-v4-flash and deepseek-v4-pro to reasoning timeout floor 2026-07-09 15:04:14 +05:30
redact.py fix(redact): skip env-lookup exception for JSON/YAML config field redaction 2026-07-05 13:58:19 -07:00
replay_cleanup.py fix(agent): preserve none vs unknown tool effects (#61783) 2026-07-11 05:41:58 -07:00
retry_utils.py refactor(retry): single-source Z.AI overload short-attempts + drop change-detector assert 2026-07-07 11:57:01 +05:30
runtime_cwd.py fix(desktop): stabilize project folder sessions (#37586) 2026-06-02 20:23:09 +00:00
secret_scope.py feat(gateway): multiplex phase 2 — fail-closed profile credential isolation (Workstream A) 2026-06-19 07:34:15 -07:00
shell_hooks.py fix(cli): safe mode also skips shell-hook registration 2026-07-07 02:32:32 -07:00
skill_bundles.py fix(gateway): apply platform-disabled skill gate to bundle invocations (#59156) 2026-07-05 14:48:11 -07:00
skill_commands.py fix(skills): guard skill slash commands against core-command and slug collisions 2026-07-14 16:41:21 +05:30
skill_preprocessing.py fix(windows): hide console-window flash on backend git/gh/wmic/bash subprocess spawns 2026-06-28 05:28:45 -07:00
skill_utils.py perf(skills): speed up snapshot prompt builds 2026-07-09 02:41:24 +05:30
ssl_guard.py fix(ssl): align guard docs and escape hatch 2026-06-13 21:14:32 -07:00
ssl_verify.py fix(agent): honor custom CA certs on aux client + harden TLS resolution 2026-07-02 04:51:56 +05:30
stream_diag.py feat(agent): buffer retry/fallback status, surface only on terminal failure (#33816) 2026-05-28 04:53:27 -07:00
subdirectory_hints.py fix(subdirectory_hints): catch RuntimeError from Path.expanduser() 2026-07-01 04:55:15 -07:00
system_prompt.py fix(agent): tag desktop chat sessions as desktop 2026-07-08 06:18:18 -07:00
think_scrubber.py fix(agent): stateful streaming scrubber for reasoning-block leaks (#17924) (#20184) 2026-05-05 04:33:38 -07:00
thinking_timeout_guidance.py fix(agent): detect thinking-timeout for reasoning models and surface actionable guidance instead of misleading file-write advice 2026-06-25 19:00:48 -07:00
thread_scoped_output.py fix(bg-review): scope stdout/stderr silencing to the worker thread (#55966) 2026-06-30 17:28:33 -07:00
title_generator.py fix(title_generator): strip think blocks from LLM output before extracting title 2026-07-01 04:18:48 -07:00
tool_dispatch_helpers.py fix(agent): preserve none vs unknown tool effects (#61783) 2026-07-11 05:41:58 -07:00
tool_executor.py fix(agent): preserve none vs unknown tool effects (#61783) 2026-07-11 05:41:58 -07:00
tool_guardrails.py fix: add recovery hints to loop guard warnings 2026-05-19 00:12:12 -07:00
tool_result_classification.py fix(agent): preserve none vs unknown tool effects (#61783) 2026-07-11 05:41:58 -07:00
trace_upload.py feat(sessions): trace export + HF upload via 'sessions export --format trace' (#60507) 2026-07-07 15:12:49 -07:00
trajectory.py
transcription_provider.py feat(stt): add register_transcription_provider() plugin hook 2026-05-25 01:41:19 -07:00
transcription_registry.py fix(deepinfra): harden multimodal provider routing 2026-07-14 02:59:39 +05:30
tts_provider.py feat(tts): add register_tts_provider() plugin hook (closes #30398) 2026-05-24 18:04:54 -07:00
tts_registry.py feat(providers): Support DeepInfra as an LLM provider 2026-07-14 02:59:39 +05:30
turn_context.py fix(session): preserve clean multimodal persistence override 2026-07-14 03:32:45 +05:30
turn_finalizer.py fix(session): restore clean API-local turn content 2026-07-14 03:32:46 +05:30
turn_retry_state.py feat(vertex): add Google Vertex AI provider for Gemini (OAuth2) 2026-07-01 05:25:33 -07:00
usage_pricing.py feat(openai): cover gpt-5.6 -pro variants (PR #61587 complement) 2026-07-10 00:47:51 +05:30
verification_evidence.py feat(agent): recognize focused ad-hoc verification scripts 2026-06-24 23:03:45 -05:00
verification_stop.py fix(approval): allow verifier temp cleanup 2026-07-12 04:32:52 -07:00
verify_hooks.py feat(agent): add pre_verify hook and verify-on-stop coding guidance 2026-06-30 00:59:29 -05:00
vertex_adapter.py security(vertex): route credential/project/region resolution through the profile secret scope 2026-07-02 06:07:56 +05:30
video_gen_provider.py feat(providers): Support DeepInfra as an LLM provider 2026-07-14 02:59:39 +05:30
video_gen_registry.py fix(deepinfra): harden multimodal provider routing 2026-07-14 02:59:39 +05:30
web_search_provider.py fix(web): widen config-aware env resolution to exa/parallel/tavily/brave-free providers 2026-07-06 02:42:24 -07:00
web_search_registry.py fix(web): correct 'disabled plugin' diagnosis for web backends (#59573) 2026-07-06 04:38:17 -07:00