Two related OAuth fixes:
1. Replace module-level _redirect_handler with _make_redirect_handler()
closure factory that closes over the resolved port. This prevents
cross-server state pollution when multiple MCP servers run OAuth
concurrently (#44588).
2. Set server.allow_reuse_address = True on the ephemeral callback
HTTPServer so the socket doesn't stay in TIME_WAIT after the flow
completes. This prevents 'Address already in use' errors on the
next OAuth flow for the same port (#44590).
Fixes#44588Fixes#44590
The single-family pre-probe (_port_is_available) raced the real bind and
reported a lingering TIME_WAIT socket as 'in use', failing gateway
restarts for up to ~60s (#10297). Port the webhook adapter's bind
mechanics (#63711/#65482): delete the probe, bind directly with clean
OSError handling and runner teardown, and scope reuse_address=False to
macOS only so Linux restarts rebind past TIME_WAIT instantly.
Credit to @lrawnsley (#10297) for identifying the TIME_WAIT restart
failure.
The overview's total_input/output/cache token counts summed only the
sessions counters (main-loop usage), while the per-model breakdown
already included auxiliary usage rows (task dimension from #65537) and
reconciled residuals. Result: hermes insights top-line totals
undercounted aux spend (compression summarizer, vision, titles) and
disagreed with the per-model table below them — the symptom reported
in #58592 and requested in #9979.
When the per-model breakdown is available, derive the overview token
totals from it (same pattern total_cost already used). Verified no
double-count across incremental CLI deltas, gateway absolute
overwrites, and aux rows.
Guard staged node-pty ASAR path rewrites so already-unpacked paths are
not rewritten twice. Normalize spawn-helper to mode 0755 in both the
prebuild and locally compiled build/Release staging paths.
Add behavioral coverage for both unpacked path forms and both helper
layouts.
Co-authored-by: zhouwei <zwcf5200@163.com>
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Two turns interleaving on one session corrupt the durable transcript:
flushes race (user rows persist out of arrival order), the identity-marker
dedup over shared history dicts can swallow a row, and the second turn
runs on a history base that never saw the first turn's exchange. The
dispatch route that lets the second turn through the busy guard is not
yet identified.
Add note_turn_start (build_turn_context) / note_turn_persisted
(_persist_session funnel): one WARNING naming both turn_ids when a turn
starts before the previous turn's turn-end persist. Ownership transfer
keeps a crashed turn from warning more than once; the unconditional clear
makes the tripwire under-report rather than double-report under a real
overlap. Log-only, no behavior change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a telegram.free_response_topics config list of '<chat_id>:<thread_id>'
entries (plus the TELEGRAM_FREE_RESPONSE_TOPICS env bridge) so a single
forum topic can be free-response — the bot replies without a mention —
without opening the whole chat via free_response_chats. A missing
message_thread_id is normalized to the General topic ('1') via
_effective_message_thread_id.
Re-ported from PR #36049 (by @wesleion): the original patched
gateway/platforms/telegram.py, which has since moved to
plugins/platforms/telegram/adapter.py with a second gating site
(_should_observe_unmentioned_group_message) and plugin-hook config
bridging (_apply_yaml_config). Both gating sites now honor
free_response_topics.
Salvaged-from: #36049
PR #36051's values went stale since May 31: session-store SCHEMA_VERSION
is now 21 (PR said 14), and the dashboard ships 8 built-in themes
(PR said 7). Also document the v16/v18/v20 data migrations added since.
Cross-checked website/docs against the source at main HEAD and corrected
documented commands, env vars, config keys, headers, and default values
that don't match the code. Docs-only; no behavioral changes.
Refs #36048
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
load_transcript is now a live-replay restore site that heals alternation
violations on load (#64934), so the old all-user 120-row fixture was
merged into a single message and the precondition len==120 failed. The
fixture was never a valid conversation shape; alternate user/assistant
so it exercises the same bloat scenario without tripping the repair.
A turn that persists a user row with no assistant row (suppressed reply,
or two concurrent turns interleaving their flushes) leaves a user;user
pair in state.db. The defensive pre-request repair_message_sequence then
re-fires on EVERY request for the rest of the session's life — it mutates
only the per-request list, never the stored transcript.
Add repair_alternation (default False) to get_messages_as_conversation
and pass it from the three live-replay restore sites (gateway
load_transcript, CLI session resume x2). Inspection/export consumers
(trace upload, context guard, api_server history) keep the verbatim
default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Telegram only auto-derives a voice/audio clip's duration from container
metadata for short recordings; clips longer than ~4:50 are delivered with
duration 0 and render as 0:00 in the player. Probe the length locally
(stdlib wave -> mutagen -> ffprobe) and pass duration explicitly to
sendVoice/sendAudio. Best-effort: when nothing can read the file we omit
duration and fall back to Telegram's prior behavior.
Extracts and hardens the Telegram-only part of the stale, Piper-bundled
PR #7815 (ffprobe-only, predates the send_voice retry/anchor refactor);
relates to #8508.
When an agent is running, the gateway runner's running-agent block routes
"session-level toggles that are safe to run mid-agent" through a membership
set before the catch-all that rejects everything else with
"Agent is running — /<cmd> can't run mid-turn".
A dedicated /footer dispatch branch already sat inside that guard, but the
set listed only {"yolo", "verbose"}, so the footer branch was unreachable:
/footer fell through to the catch-all and was rejected, forcing users to
/stop a running agent just to toggle the runtime-metadata footer. /footer
is a pure display toggle like its sibling /verbose — it only writes
display.runtime_footer.enabled and returns a status string — so it belongs
in the same set.
Add "footer" to the set so the existing dispatch branch becomes reachable.
Regression test (tests/gateway/test_footer_command_mid_run.py): asserts
/footer and /footer <arg> dispatch to _handle_footer_command while an agent
is running, with a /verbose parity guard. Verified failing before the fix
(handler awaited 0 times) and passing after.
Follow-up on the salvaged #64611 commit: the original guard blocked the
install tree unconditionally, which would have broken the legitimate
'developing Hermes from a source clone' CLI flow (launching hermes inside
the repo and getting its AGENTS.md as project context).
Refined policy:
- resolve_context_cwd(): validates configured paths (missing dir -> None +
warning) but honors an EXPLICIT install-tree cwd verbatim — deliberate
user choice.
- build_context_files_prompt(): blocks only the cwd=None -> os.getcwd()
FALLBACK into the install tree, with a new allow_install_tree_fallback
param. system_prompt.py passes it for platform cli/tui (launch dir is
the user's real shell cwd there); desktop/gateway surfaces keep the
guard (their fallback dir is self-spawned, never user-picked).
- Warning log names the resolved dir and the terminal.cwd remedy.
E2E-verified all five scenarios: desktop fallback blocked, in-tree CLI dev
keeps AGENTS.md, explicit install-tree cwd honored, invalid TERMINAL_CWD
falls to None then blocked, normal workspace loads.
_extract_parallel_scope_path used Path.cwd() (process cwd) instead of the
tool's actual execution cwd, and os.path.abspath() instead of os.path.realpath(),
so symlink aliases and relative/absolute path pairs that resolve to the same
physical file were treated as distinct targets and placed in the same parallel
segment. On case-insensitive platforms (Windows) os.path.normcase() was also
absent, allowing Foo.txt and foo.txt to race.
Changes:
- agent/tool_dispatch_helpers.py: introduce _canonical_path(raw_path,
execution_cwd) applying expanduser->abspath->realpath->normcase; thread
execution_cwd through _extract_parallel_scope_path and
_plan_tool_batch_segments
- agent/tool_executor.py: pass get_active_env(effective_task_id).cwd as
execution_cwd to _plan_tool_batch_segments; add pathlib.Path import
- run_agent.py: pass active env cwd to _plan_tool_batch_segments at the
second call site inside _execute_tool_calls
- tests/run_agent/test_tool_batch_segmentation.py: add 5 regression tests
covering relative/absolute same target, symlink alias, execution_cwd vs
process cwd, symlink parent + nonexistent write target, and Windows
case-insensitive alias (skipped on non-Windows)
Fixes a file-corruption / lost-update race introduced by the mixed
tool-batch segmentation feature (perf commit #64460).
Follow-ups to @SAMBAS123's #64986 salvage:
- Replace the hardcoded token-platform set in _platform_has_bot_credential
with PLATFORM_TOKEN_ENV_NAMES, a shared canonical map in gateway/config.py
also used by the empty-token validation warning — one source of truth, so
future token platforms can't silently bypass the gate or drift between
the two sites.
- After secondary-profile startup, warn loudly for any platform skipped on
the primary that no secondary profile ended up serving: an enabled
platform with no credential anywhere is a config error, not a silent
no-op.
- AUTHOR_MAP entry for the salvaged commit's author email.
When gateway.multiplex_profiles is on, the default-profile GatewayRunner
used to call load_gateway_config() unscoped. Platform tokens that lived
only in a profile .env (often a secondary profile) never reached the
primary Telegram adapter, producing "No bot token configured" and an
infinite reconnect watcher loop (#64674).
- Load primary config under the default profile secret scope when multiplex
is enabled (same path secondary adapters already use).
- Skip starting token platforms on the default profile when no credential
is present under multiplex; secondary profiles still connect with their
scoped tokens.
- Drop empty-token configs from the reconnect queue so they cannot spin
forever.
Regression coverage in tests/gateway/test_64674_multiplex_primary_token_scope.py.
Extends @AlexFucuson9's 3-site fix (#58594) across the full adapter:
every logger call and SendResult.error that interpolates a raw PTB
exception now routes through _redact_telegram_error_text(). Covers
polling conflict/retry/network ladders, overflow-split edits, draft
sends, prompt/approval/clarify/picker sends, media send fallbacks,
media cache failures, reactions, and chat-info lookups (48 additional
sites). Telegram Bot API exceptions embed the token in the request URL
(/bot<TOKEN>/<method>), so any raw str(exc) is a leak surface.
Adds regression tests for SendResult.error redaction (update prompt,
clarify) and delete_message debug-log redaction.
Telegram Bot API URLs carry credentials in the path as
/bot<TOKEN>/<method>. Three error-handling paths logged raw exception
text that could include these URLs:
- sendRichMessage fallback (line 1603)
- editMessageText fallback (line 1709)
- polling reconnect warning (line 1902)
Replace raw / with which
uses the existing redact_sensitive_text(force=True) pipeline. This
matches the pattern already used by transient send failures, retry
errors, and legacy edit paths.
Fixes#58376
mark_awaiting_text is the 'Other (type answer)' mode-flip, not a send-time
setup call — invoking it in send_clarify forces the user's next message to
be captured as the clarify response, racing the button-click path and
bypassing the buttons entirely. Telegram calls it only in the 'other'
callback branch; do the same here.
test_run_prompt_submit_requeues_all_unstarted_notifications_with_real_threading
asserted strict FIFO order of the requeued events. The completion_queue is
process-global, and notification pollers leaked by earlier session.init tests
in the same file legitimately steal-and-requeue foreign-session events
(_notification_poller_loop's belongs-elsewhere branch), rotating the queue —
CI slice 6/8 saw [batch_3, batch_2] and failed on ordering alone.
Assert the actual requeue contract instead: batch_1 is consumed while
batch_2 and batch_3 both remain queued — membership via a deadline drain
(an event can be transiently held by a poller mid-cycle), not order.
Verified: single test 10/10 green, full file 337/337 across 3 consecutive
CI-parity runs.
- Refresh _OFFICIAL_DOCS_PRICING fireworks entries against current
docs.fireworks.ai/serverless/pricing: qwen3p6-plus is gone (replaced
by qwen3p7-plus); add glm-5p2/5p1, kimi-k2p7-code, deepseek-v4-flash,
minimax-m3/m2p7, gpt-oss-120b/20b, and the routers/*-fast tiers with
their distinct higher rates.
- Picker pricing via get_pricing_for_provider('fireworks'): pure dict
transform over the shared models.dev in-memory/disk cache (1h TTL) +
_pricing_cache memoization — no new network call on the picker path.
- Wire pricing display into the generic api-key-provider setup flow so
Fireworks model pickers show $/M columns like OpenRouter/Nous do.
- Invariant tests: plugin fallback_models all priced, fast tiers price
higher than standard, every row carries cache_read < input.
Fireworks-hosted sessions previously showed estimated_cost_usd = 0
because (a) _OFFICIAL_DOCS_PRICING had no Fireworks entries and (b)
resolve_billing_route() had no branch for provider="fireworks",
falling through to billing_mode="unknown".
Adds entries for the three Fireworks models hermes operators are
most likely to route through (Kimi K2.6, DeepSeek V4 Pro, Qwen3.6-Plus)
and a routing branch that triggers on either explicit
provider="fireworks" or api.fireworks.ai base_url match. Mirrors the
recently-merged MiniMax addition pattern; pricing snapshot sourced
from https://docs.fireworks.ai/serverless/pricing and the per-model
pages on fireworks.ai.
Tests cover: (a) full Fireworks model id resolves to the snapshot
entry, (b) base_url alone is sufficient to route, (c) end-to-end
estimate returns "estimated" status with the expected dollar amount.
A follow-up upstream issue is open proposing a dynamic pricing
source (e.g. litellm's pricing JSON) as a permanent fix to the
PR-per-model treadmill that this snapshot keeps adding to.
_clamp_duration returned durations[0] for all families when duration=None,
causing pixverse-v6, seedance-2.0, and kling-v3-4k to always send their
minimum value (1s, 4s, 3s respectively) instead of omitting the field and
letting the FAL endpoint apply its own default.
Range families are now detected via the existing _is_duration_range
heuristic and return None (field omitted) when no duration is requested.
Enum families like veo3.1 keep sending their first entry as the default.
Follow-up for salvaged PRs #56055 + #56803, adapted to the per-session
cwd record store (PR #65213): the record (live cd state) is rung 1,
the registered session.cwd.set override rung 2, TERMINAL_CWD rung 3 —
the same ladder file tools and the terminal resolve against. Covers
record-over-override, record-only, and stale-record fall-through.
execute_code's _resolve_child_cwd() only checked the process-global
TERMINAL_CWD env var and os.getcwd(), ignoring the per-session cwd
override registered via session.cwd.set → register_task_env_overrides.
This caused execute_code to write to the process launch directory
while sibling tools (write_file, read_file, patch, terminal) correctly
resolved the session workspace — two file-writing paths in one turn
silently disagreed on the working directory.
Fix: pass task_id to _resolve_child_cwd() and check
_registered_task_cwd_override(task_id) before falling back to
TERMINAL_CWD and os.getcwd(), matching the lookup order used by
file_tools._resolve_base_dir and terminal_tool._resolve_command_cwd.
MEDIA: tags whose path had an unknown extension (.py, .log, .toml,
.weirdext, ...) fell between both extraction passes: the anchored
extension allowlist (MEDIA_TAG_CLEANUP_RE) did not match them, and the
extension-less pass explicitly skipped any path that HAD a suffix. The
file was never delivered even though the intended design (universal
ingress/egress) says any non-credential file should ship.
Widen _path_lacks_deliverable_extension() so the validated delivery
pass (MEDIA_EXTENSIONLESS_TAG_RE + validate_media_delivery_path)
covers every path the extension allowlist does not — unknown
extensions and extension-less files alike. Security posture is
unchanged: unknown-extension paths only deliver after full validation
(exists, symlinks resolved, credential/system denylist, strict-mode
allowlist+recency), and unvalidated tags stay visible in the text
instead of being silently dropped. Known extensions keep their
unconditional pre-existing behavior.
Because extract_media, _strip_media_tag_directives (non-streaming
dispatch), and strip_media_directives_for_display (streaming) all
share the same two regexes + predicate, all delivery paths pick up the
widened behavior with no per-site changes. Dispatch partition in
gateway/run.py already routes non-image/video extensions through
send_document.
Closes the gap reported in PR #36060; supersedes the allowlist-append
approach there (an extension allowlist can never enumerate every file
type a user asks the agent to produce).
Co-authored-by: Randimt <randimt@users.noreply.github.com>
* feat(analytics): record auxiliary model usage per task in session accounting
Auxiliary LLM calls (vision, compression, title_generation, web_extract,
session_search, ...) discarded their token usage, leaving dashboard
analytics blind to aux model spend (issue #23270).
- hermes_state.py: session_model_usage gains a task PK dimension
(''=main loop) via v22 table-rebuild migration (SQLite can't alter a
PK); record_auxiliary_usage() writes per-(model,provider,task) deltas
WITHOUT touching sessions counters (gateway overwrites those with
absolute main-loop totals — folding aux in would double-count or be
clobbered). Aux rows never inherit the session's main-loop route.
- agent/aux_accounting.py: ContextVar ambient accounting context
(mirrors the portal_tags conversation context); record_aux_usage()
normalizes usage via usage_pricing.normalize_usage, estimates cost,
and is strictly best-effort. moa_reference/moa_aggregator excluded —
conversation_loop already folds MoA usage+cost into the main delta.
- agent/auxiliary_client.py: _validate_llm_response is the recording
chokepoint — every successful non-streaming aux response passes
through it exactly once, sync and async, including fallback paths
(model read from the response itself stays accurate across
fallbacks).
- run_agent.py: run_conversation publishes/resets the accounting
context; agent/title_generator.py republishes on its bare thread.
- hermes_cli/web_server.py: /api/analytics/usage folds aux rows into
by_model (aux-only models finally appear) and adds a by_task
summary; /api/analytics/models surfaces aux rows on the Models page.
Design per review of PR #62850 by @eeksock (thread-local + separate
auxiliary_usage table): rebuilt on ContextVar (async-safe — thread-local
cross-attributes concurrent coroutines on one event loop) and the
existing session_model_usage table instead of a parallel accounting
path, extended beyond vision to every aux task, and wired the analytics
endpoints so the dashboard actually shows it. Credit to @eeksock for
the approach and @tboatman for the detailed root-cause analysis.
* test(moa): match _validate_llm_response mock to new accounting-hint signature
* test(aux): accept accounting-hint kwargs in remaining _validate_llm_response mocks
Docs and MultiplexConfigError already promise secondary profiles are
served through the shared listener's /p/<profile>/ prefix, but only the
webhook adapter registered those routes — api_server returned 404.
Mirror every HTTP route, validate the prefix, and scope agent runs /
session DB / model listing to the target profile.
