_wait_for_callback still read the legacy module-level _oauth_port, so
with two concurrent OAuth flows, flow A's callback wait bound flow B's
port while A's redirect URI pointed at A's port — the callback-side
half of the cross-flow collision that #65622 fixed on the redirect
side. _make_callback_waiter(port) closes over each flow's resolved
port; both provider construction sites (build_oauth_auth and
MCPOAuthManager._build_provider) now wire per-flow waiters. The legacy
_wait_for_callback delegates for backwards compatibility.
Direction credit to @LeonSGP43 (#34280) and the #34260 analysis.
_wait_for_callback catches OSError on bind with a comment claiming the port is
held by a server build_oauth_auth started, and promising to fall back to polling
it. build_oauth_auth never starts a callback server (this is the only listener),
so there is nothing to poll: the branch just raised a misleading "OAuth callback
timed out" when the real cause is a busy port (a concurrent login, a leftover
listener, or a fixed oauth.redirect_port that collided).
Fix the stale comment and raise an accurate, actionable message (names the port,
suggests freeing it or setting a free oauth.redirect_port), chained from the
original OSError. Behavior is otherwise unchanged. Adds a regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A bare False from connect() on EADDRINUSE made the gateway reconnect
watcher treat a port conflict as transient and retry forever at the
backoff cap — 1568+ retries over 5 days in a multi-profile production
setup, filling errors.log and leaking 2 ResponseStore fds per retry.
Set a non-retryable fatal error (api_server_port_in_use) in the bind
OSError branch so the platform drops from the reconnect queue;
recover via /platform resume api_server after changing the port.
Re-implementation of #52132 by @msalles1 against the direct-bind path
from #65621 (the pre-probe block their patch targeted no longer
exists). Their production diagnosis and test scenario preserved.
_clean_reasoning_effort kept its own whitelist that stopped at 'max',
silently dropping 'ultra' from MoA slot configs. Route it through
hermes_constants.parse_reasoning_effort — the same one-source-of-truth
fix the salvaged commit applies to the gateway — so future effort
levels can't drift here either. Docs updated to list ultra.
Follow-up to salvaged PR #64012.
Switch provider and model together after setup-time auth failure. Serialize global auth-store merges under target-specific locks and preserve auth-to-shared lock ordering for profile OAuth refreshes.
- redirect_host tests: localhost swap, precedence of full redirect_uri,
empty-value fallback, client-metadata propagation
- The salvaged redirect-uri hint tests predate the #57836
OAuthNonInteractiveError guard; monkeypatch _is_interactive like the
sibling tests in TestRedirectHandlerSshHint
The PR added a configurable `redirect_uri` (proxy/Funnel callbacks) but
shipped without tests, and the loopback SSH-tunnel hint stayed hardcoded —
actively misleading the exact proxy user the feature targets.
- Extract `_resolve_redirect_uri(cfg, port)` so the client-metadata and
pre-registration paths derive an identical callback (a mismatch makes the
authorization server reject the redirect).
- Make `_redirect_handler` redirect_uri-aware: a configured proxy callback
reaches this machine on its own, so it no longer prints the `ssh -N -L`
loopback guidance. Wired via `functools.partial` — no new global state.
- Document `redirect_uri` in the config block.
- 14 new tests (red/green TDD): helper resolution + empty-string fallback,
metadata + pre-registration for configured/default, AnyUrl normalization,
no-client_id skip, client_secret combo, and both SSH-hint branches.
ruff clean · 81 passed (tests/tools/test_mcp_oauth.py) · ty baseline unchanged
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Widens the deepseek-v4-flash addition to the whole stale-snapshot class:
- deepseek-v4-pro: $1.74/$3.48 → $0.435/$0.87, cache-read $0.003625
(DeepSeek's 2026-07 price cut; every pro session was over-reporting 4x)
- deepseek-chat / deepseek-reasoner: deprecated 2026-07-24, now alias
v4-flash non-thinking/thinking modes — repriced to match flash
(reasoner was $0.55/$2.19 with no cache rate)
- cache_read added to every row; pricing_version unified at
deepseek-pricing-2026-07
- invariant tests: aliases price identically to flash; every deepseek
row carries cache_read < input
DeepSeek's /models endpoint returns no pricing, so direct-provider routes fall back to the _OFFICIAL_DOCS_PRICING snapshot. The table included deepseek-v4-pro but not the newer deepseek-v4-flash, so flash sessions reported $0.00 with cost_source "none". Add the flash entry (values from DeepSeek's official pricing page, mirroring the v4-pro entry; DeepSeek bills no separate cache-write cost) plus two regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-ups on top of the cherry-picked cluster commits:
- slack: scope-authoritative app-token read — get_secret() with a
narrow UnscopedSecretError fallback to os.getenv. Keeps @kohoj's
correct semantics (scoped profile can never silently inherit the
default profile's Socket Mode app) while fixing the regression where
the default-profile startup loop and background reconnect rebuild,
which call connect() unscoped under multiplex, would raise and
fail-loop. Supersedes the 'or os.getenv' variant from #64461 which
reintroduced the cross-profile fallback leak.
- test: unscoped-multiplex fallback regression test for connect().
- run.py: convert the last legacy self.adapters.get(source.platform)
site (_rename_discord_auto_thread) to _adapter_for_source(source)
so profile-routed Discord sources rename threads on the right
adapter (from #57417's sweep).
- AUTHOR_MAP entry for @aguung.
Partial cherry-pick of a7ffbbff7 from PR #63256: secondary-profile
adapter creation errors no longer abort the whole secondary startup
(try/except around _create_adapter + loud warning on None return), and
Home Assistant's check_ha_requirements() becomes dep-only with the
credential moved to a new validate_ha_config() so secondary profiles
whose HASS_TOKEN lives in the profile secret scope are not silently
dropped by the registry gate.
Telegram diagnostic hunks and profile-label stamping dropped: the
regression they targeted does not exist on current main and they
conflict with the connect() teardown fence.
_find_free_port() closed its probe socket before HTTPServer re-bound
the port minutes later, leaving a window where another process could
steal it (#22161 by @amathxbt). _reserve_callback_port() now keeps the
selected socket bound (bounded FIFO pool) until _wait_for_callback
adopts it via bind_and_activate=False. Also sets allow_reuse_address
BEFORE binding — the cherry-picked #44872 set it after the constructor
had already bound, where it is a no-op.
Also updates the three #57836 non-interactive-guard tests to the
closure-factory API from #44872.
Two related OAuth fixes:
1. Replace module-level _redirect_handler with _make_redirect_handler()
closure factory that closes over the resolved port. This prevents
cross-server state pollution when multiple MCP servers run OAuth
concurrently (#44588).
2. Set server.allow_reuse_address = True on the ephemeral callback
HTTPServer so the socket doesn't stay in TIME_WAIT after the flow
completes. This prevents 'Address already in use' errors on the
next OAuth flow for the same port (#44590).
Fixes#44588Fixes#44590
The single-family pre-probe (_port_is_available) raced the real bind and
reported a lingering TIME_WAIT socket as 'in use', failing gateway
restarts for up to ~60s (#10297). Port the webhook adapter's bind
mechanics (#63711/#65482): delete the probe, bind directly with clean
OSError handling and runner teardown, and scope reuse_address=False to
macOS only so Linux restarts rebind past TIME_WAIT instantly.
Credit to @lrawnsley (#10297) for identifying the TIME_WAIT restart
failure.
The overview's total_input/output/cache token counts summed only the
sessions counters (main-loop usage), while the per-model breakdown
already included auxiliary usage rows (task dimension from #65537) and
reconciled residuals. Result: hermes insights top-line totals
undercounted aux spend (compression summarizer, vision, titles) and
disagreed with the per-model table below them — the symptom reported
in #58592 and requested in #9979.
When the per-model breakdown is available, derive the overview token
totals from it (same pattern total_cost already used). Verified no
double-count across incremental CLI deltas, gateway absolute
overwrites, and aux rows.
Two turns interleaving on one session corrupt the durable transcript:
flushes race (user rows persist out of arrival order), the identity-marker
dedup over shared history dicts can swallow a row, and the second turn
runs on a history base that never saw the first turn's exchange. The
dispatch route that lets the second turn through the busy guard is not
yet identified.
Add note_turn_start (build_turn_context) / note_turn_persisted
(_persist_session funnel): one WARNING naming both turn_ids when a turn
starts before the previous turn's turn-end persist. Ownership transfer
keeps a crashed turn from warning more than once; the unconditional clear
makes the tripwire under-report rather than double-report under a real
overlap. Log-only, no behavior change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a telegram.free_response_topics config list of '<chat_id>:<thread_id>'
entries (plus the TELEGRAM_FREE_RESPONSE_TOPICS env bridge) so a single
forum topic can be free-response — the bot replies without a mention —
without opening the whole chat via free_response_chats. A missing
message_thread_id is normalized to the General topic ('1') via
_effective_message_thread_id.
Re-ported from PR #36049 (by @wesleion): the original patched
gateway/platforms/telegram.py, which has since moved to
plugins/platforms/telegram/adapter.py with a second gating site
(_should_observe_unmentioned_group_message) and plugin-hook config
bridging (_apply_yaml_config). Both gating sites now honor
free_response_topics.
Salvaged-from: #36049
load_transcript is now a live-replay restore site that heals alternation
violations on load (#64934), so the old all-user 120-row fixture was
merged into a single message and the precondition len==120 failed. The
fixture was never a valid conversation shape; alternate user/assistant
so it exercises the same bloat scenario without tripping the repair.
A turn that persists a user row with no assistant row (suppressed reply,
or two concurrent turns interleaving their flushes) leaves a user;user
pair in state.db. The defensive pre-request repair_message_sequence then
re-fires on EVERY request for the rest of the session's life — it mutates
only the per-request list, never the stored transcript.
Add repair_alternation (default False) to get_messages_as_conversation
and pass it from the three live-replay restore sites (gateway
load_transcript, CLI session resume x2). Inspection/export consumers
(trace upload, context guard, api_server history) keep the verbatim
default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Telegram only auto-derives a voice/audio clip's duration from container
metadata for short recordings; clips longer than ~4:50 are delivered with
duration 0 and render as 0:00 in the player. Probe the length locally
(stdlib wave -> mutagen -> ffprobe) and pass duration explicitly to
sendVoice/sendAudio. Best-effort: when nothing can read the file we omit
duration and fall back to Telegram's prior behavior.
Extracts and hardens the Telegram-only part of the stale, Piper-bundled
PR #7815 (ffprobe-only, predates the send_voice retry/anchor refactor);
relates to #8508.
When an agent is running, the gateway runner's running-agent block routes
"session-level toggles that are safe to run mid-agent" through a membership
set before the catch-all that rejects everything else with
"Agent is running — /<cmd> can't run mid-turn".
A dedicated /footer dispatch branch already sat inside that guard, but the
set listed only {"yolo", "verbose"}, so the footer branch was unreachable:
/footer fell through to the catch-all and was rejected, forcing users to
/stop a running agent just to toggle the runtime-metadata footer. /footer
is a pure display toggle like its sibling /verbose — it only writes
display.runtime_footer.enabled and returns a status string — so it belongs
in the same set.
Add "footer" to the set so the existing dispatch branch becomes reachable.
Regression test (tests/gateway/test_footer_command_mid_run.py): asserts
/footer and /footer <arg> dispatch to _handle_footer_command while an agent
is running, with a /verbose parity guard. Verified failing before the fix
(handler awaited 0 times) and passing after.
Follow-up on the salvaged #64611 commit: the original guard blocked the
install tree unconditionally, which would have broken the legitimate
'developing Hermes from a source clone' CLI flow (launching hermes inside
the repo and getting its AGENTS.md as project context).
Refined policy:
- resolve_context_cwd(): validates configured paths (missing dir -> None +
warning) but honors an EXPLICIT install-tree cwd verbatim — deliberate
user choice.
- build_context_files_prompt(): blocks only the cwd=None -> os.getcwd()
FALLBACK into the install tree, with a new allow_install_tree_fallback
param. system_prompt.py passes it for platform cli/tui (launch dir is
the user's real shell cwd there); desktop/gateway surfaces keep the
guard (their fallback dir is self-spawned, never user-picked).
- Warning log names the resolved dir and the terminal.cwd remedy.
E2E-verified all five scenarios: desktop fallback blocked, in-tree CLI dev
keeps AGENTS.md, explicit install-tree cwd honored, invalid TERMINAL_CWD
falls to None then blocked, normal workspace loads.
_extract_parallel_scope_path used Path.cwd() (process cwd) instead of the
tool's actual execution cwd, and os.path.abspath() instead of os.path.realpath(),
so symlink aliases and relative/absolute path pairs that resolve to the same
physical file were treated as distinct targets and placed in the same parallel
segment. On case-insensitive platforms (Windows) os.path.normcase() was also
absent, allowing Foo.txt and foo.txt to race.
Changes:
- agent/tool_dispatch_helpers.py: introduce _canonical_path(raw_path,
execution_cwd) applying expanduser->abspath->realpath->normcase; thread
execution_cwd through _extract_parallel_scope_path and
_plan_tool_batch_segments
- agent/tool_executor.py: pass get_active_env(effective_task_id).cwd as
execution_cwd to _plan_tool_batch_segments; add pathlib.Path import
- run_agent.py: pass active env cwd to _plan_tool_batch_segments at the
second call site inside _execute_tool_calls
- tests/run_agent/test_tool_batch_segmentation.py: add 5 regression tests
covering relative/absolute same target, symlink alias, execution_cwd vs
process cwd, symlink parent + nonexistent write target, and Windows
case-insensitive alias (skipped on non-Windows)
Fixes a file-corruption / lost-update race introduced by the mixed
tool-batch segmentation feature (perf commit #64460).
When gateway.multiplex_profiles is on, the default-profile GatewayRunner
used to call load_gateway_config() unscoped. Platform tokens that lived
only in a profile .env (often a secondary profile) never reached the
primary Telegram adapter, producing "No bot token configured" and an
infinite reconnect watcher loop (#64674).
- Load primary config under the default profile secret scope when multiplex
is enabled (same path secondary adapters already use).
- Skip starting token platforms on the default profile when no credential
is present under multiplex; secondary profiles still connect with their
scoped tokens.
- Drop empty-token configs from the reconnect queue so they cannot spin
forever.
Regression coverage in tests/gateway/test_64674_multiplex_primary_token_scope.py.
Extends @AlexFucuson9's 3-site fix (#58594) across the full adapter:
every logger call and SendResult.error that interpolates a raw PTB
exception now routes through _redact_telegram_error_text(). Covers
polling conflict/retry/network ladders, overflow-split edits, draft
sends, prompt/approval/clarify/picker sends, media send fallbacks,
media cache failures, reactions, and chat-info lookups (48 additional
sites). Telegram Bot API exceptions embed the token in the request URL
(/bot<TOKEN>/<method>), so any raw str(exc) is a leak surface.
Adds regression tests for SendResult.error redaction (update prompt,
clarify) and delete_message debug-log redaction.
test_run_prompt_submit_requeues_all_unstarted_notifications_with_real_threading
asserted strict FIFO order of the requeued events. The completion_queue is
process-global, and notification pollers leaked by earlier session.init tests
in the same file legitimately steal-and-requeue foreign-session events
(_notification_poller_loop's belongs-elsewhere branch), rotating the queue —
CI slice 6/8 saw [batch_3, batch_2] and failed on ordering alone.
Assert the actual requeue contract instead: batch_1 is consumed while
batch_2 and batch_3 both remain queued — membership via a deadline drain
(an event can be transiently held by a poller mid-cycle), not order.
Verified: single test 10/10 green, full file 337/337 across 3 consecutive
CI-parity runs.
- Refresh _OFFICIAL_DOCS_PRICING fireworks entries against current
docs.fireworks.ai/serverless/pricing: qwen3p6-plus is gone (replaced
by qwen3p7-plus); add glm-5p2/5p1, kimi-k2p7-code, deepseek-v4-flash,
minimax-m3/m2p7, gpt-oss-120b/20b, and the routers/*-fast tiers with
their distinct higher rates.
- Picker pricing via get_pricing_for_provider('fireworks'): pure dict
transform over the shared models.dev in-memory/disk cache (1h TTL) +
_pricing_cache memoization — no new network call on the picker path.
- Wire pricing display into the generic api-key-provider setup flow so
Fireworks model pickers show $/M columns like OpenRouter/Nous do.
- Invariant tests: plugin fallback_models all priced, fast tiers price
higher than standard, every row carries cache_read < input.
Fireworks-hosted sessions previously showed estimated_cost_usd = 0
because (a) _OFFICIAL_DOCS_PRICING had no Fireworks entries and (b)
resolve_billing_route() had no branch for provider="fireworks",
falling through to billing_mode="unknown".
Adds entries for the three Fireworks models hermes operators are
most likely to route through (Kimi K2.6, DeepSeek V4 Pro, Qwen3.6-Plus)
and a routing branch that triggers on either explicit
provider="fireworks" or api.fireworks.ai base_url match. Mirrors the
recently-merged MiniMax addition pattern; pricing snapshot sourced
from https://docs.fireworks.ai/serverless/pricing and the per-model
pages on fireworks.ai.
Tests cover: (a) full Fireworks model id resolves to the snapshot
entry, (b) base_url alone is sufficient to route, (c) end-to-end
estimate returns "estimated" status with the expected dollar amount.
A follow-up upstream issue is open proposing a dynamic pricing
source (e.g. litellm's pricing JSON) as a permanent fix to the
PR-per-model treadmill that this snapshot keeps adding to.
_clamp_duration returned durations[0] for all families when duration=None,
causing pixverse-v6, seedance-2.0, and kling-v3-4k to always send their
minimum value (1s, 4s, 3s respectively) instead of omitting the field and
letting the FAL endpoint apply its own default.
Range families are now detected via the existing _is_duration_range
heuristic and return None (field omitted) when no duration is requested.
Enum families like veo3.1 keep sending their first entry as the default.
Follow-up for salvaged PRs #56055 + #56803, adapted to the per-session
cwd record store (PR #65213): the record (live cd state) is rung 1,
the registered session.cwd.set override rung 2, TERMINAL_CWD rung 3 —
the same ladder file tools and the terminal resolve against. Covers
record-over-override, record-only, and stale-record fall-through.
MEDIA: tags whose path had an unknown extension (.py, .log, .toml,
.weirdext, ...) fell between both extraction passes: the anchored
extension allowlist (MEDIA_TAG_CLEANUP_RE) did not match them, and the
extension-less pass explicitly skipped any path that HAD a suffix. The
file was never delivered even though the intended design (universal
ingress/egress) says any non-credential file should ship.
Widen _path_lacks_deliverable_extension() so the validated delivery
pass (MEDIA_EXTENSIONLESS_TAG_RE + validate_media_delivery_path)
covers every path the extension allowlist does not — unknown
extensions and extension-less files alike. Security posture is
unchanged: unknown-extension paths only deliver after full validation
(exists, symlinks resolved, credential/system denylist, strict-mode
allowlist+recency), and unvalidated tags stay visible in the text
instead of being silently dropped. Known extensions keep their
unconditional pre-existing behavior.
Because extract_media, _strip_media_tag_directives (non-streaming
dispatch), and strip_media_directives_for_display (streaming) all
share the same two regexes + predicate, all delivery paths pick up the
widened behavior with no per-site changes. Dispatch partition in
gateway/run.py already routes non-image/video extensions through
send_document.
Closes the gap reported in PR #36060; supersedes the allowlist-append
approach there (an extension allowlist can never enumerate every file
type a user asks the agent to produce).
Co-authored-by: Randimt <randimt@users.noreply.github.com>
* feat(analytics): record auxiliary model usage per task in session accounting
Auxiliary LLM calls (vision, compression, title_generation, web_extract,
session_search, ...) discarded their token usage, leaving dashboard
analytics blind to aux model spend (issue #23270).
- hermes_state.py: session_model_usage gains a task PK dimension
(''=main loop) via v22 table-rebuild migration (SQLite can't alter a
PK); record_auxiliary_usage() writes per-(model,provider,task) deltas
WITHOUT touching sessions counters (gateway overwrites those with
absolute main-loop totals — folding aux in would double-count or be
clobbered). Aux rows never inherit the session's main-loop route.
- agent/aux_accounting.py: ContextVar ambient accounting context
(mirrors the portal_tags conversation context); record_aux_usage()
normalizes usage via usage_pricing.normalize_usage, estimates cost,
and is strictly best-effort. moa_reference/moa_aggregator excluded —
conversation_loop already folds MoA usage+cost into the main delta.
- agent/auxiliary_client.py: _validate_llm_response is the recording
chokepoint — every successful non-streaming aux response passes
through it exactly once, sync and async, including fallback paths
(model read from the response itself stays accurate across
fallbacks).
- run_agent.py: run_conversation publishes/resets the accounting
context; agent/title_generator.py republishes on its bare thread.
- hermes_cli/web_server.py: /api/analytics/usage folds aux rows into
by_model (aux-only models finally appear) and adds a by_task
summary; /api/analytics/models surfaces aux rows on the Models page.
Design per review of PR #62850 by @eeksock (thread-local + separate
auxiliary_usage table): rebuilt on ContextVar (async-safe — thread-local
cross-attributes concurrent coroutines on one event loop) and the
existing session_model_usage table instead of a parallel accounting
path, extended beyond vision to every aux task, and wired the analytics
endpoints so the dashboard actually shows it. Credit to @eeksock for
the approach and @tboatman for the detailed root-cause analysis.
* test(moa): match _validate_llm_response mock to new accounting-hint signature
* test(aux): accept accounting-hint kwargs in remaining _validate_llm_response mocks
