Fixes#50051 by preserving nested gateway.multiplex_profiles and routing gateway config env reads through the active profile secret scope when present.
This keeps secondary profile adapter startup from inheriting default-profile platform tokens or port-binding enables while preserving legacy single-profile behavior outside a scope.
Constraint: latest upstream main f57ff7aef1 still reproduced both nested-config loss and cross-profile env leakage
Rejected: special-casing API_SERVER_* only | left other profile-scoped tokens vulnerable to the same leak
Confidence: high
Scope-risk: moderate
Directive: keep future gateway/config env reads on the scoped helper path unless a variable is explicitly process-global
Tested: pytest -q tests/gateway/test_multiplex_phase0.py tests/gateway/test_multiplex_credential_isolation.py tests/gateway/test_config.py -k 'multiplex or scope or getenv or api_server or relay'
Not-tested: full gateway startup across live platform adapters
test_group_new_keeps_existing_reset_semantics_when_dm_topic_mode_enabled
asserts 'parallel work' not in the /new reply — but /new appends a
random tip from hermes_cli.tips (380 entries), and one tip's text
contains exactly that phrase (the delegate_task concurrency tip). CI
failed on PR #59331 slice 2 when the dice landed on it. Pin
get_random_tip in the test.
The routing sweep sends these paths through _adapter_for_source, which
reads source.profile. A bare MagicMock auto-attribute is truthy, so the
fixtures looked like stamped secondary profiles and hit the new
fail-closed branch. Real SessionSource.profile is None or str
(AGENTS.md pitfall #17).
Follow-up to the routing sweep: when a stamped secondary profile has no
_profile_adapters entry (adapter failed to connect / was refused), return
None instead of falling back to the default profile's adapter — the
fallback sends replies out the wrong bot, which is the exact leak class
this cluster fixes. Also restores main's deliberate fail-fast on
port-binding platforms in secondary profiles (the cherry-picked commit
had softened it to silent force-disable).
Co-authored-by: ManniBr <m888.braun@hotmail.com>
Replace 53 instances of self.adapters.get(source.platform) with
self._adapter_for_source(source) in gateway/run.py.
self.adapters is the default profile's adapter map. In multiplex mode,
secondary profiles (lars, kira, jonas, caro) have their adapters in
_profile_adapters[profile]. _adapter_for_source() (from authz_mixin.py)
correctly resolves through _profile_adapters when source.profile is set.
Without this fix, ALL response paths for secondary profiles — streaming,
sending, media delivery, voice, typing indicators, queue operations,
startup restore, and platform notices — route through the default
profile's bot token instead of the profile's own token.
Fixes: Multiplex profiles responding with wrong bot token on Telegram,
Discord, and all other platforms.
register_mcp_servers now nudges cached entries whose session is None
via _signal_reconnect, so a new agent session recovers a parked server
immediately instead of waiting up to _PARKED_RETRY_INTERVAL for the
next self-probe (#50170). Gate-check idea credit: @izumi0uu (#50184),
@LeonSGP43 (#37772), @Tranquil-Flow (#37899).
The dead-session half-open test drives _signal_reconnect with
session=None; the salvaged _ReconnectAdapter assumed a live old
session. Also count set() calls explicitly instead of relying on
MagicMock introspection.
_wait_for_server_session_ready used a time.monotonic deadline; the
circuit-breaker tests freeze monotonic, turning the loop into an
infinite spin (300s SIGKILL in CI-parity runs). Bound by iteration
count instead.
Four independent pre-request stalls sat on the critical path between
prompt submission and the first streamed token, measured with cProfile
against a live process:
1. Discord capability detection (~2.0s, worst 5s): get_tool_definitions
-> _get_dynamic_schema made a BLOCKING https call to discord.com
inside AIAgent.__init__ for any user with DISCORD_BOT_TOKEN set, on
every platform, every cold process. Now non-blocking: memory cache ->
24h disk cache -> permissive default + one background detection that
seeds the disk cache for the next process. The permissive default is
pinned per-process so tool schemas never flip mid-conversation
(prompt-cache safety); it mirrors the existing detection-failure
fallback (all actions exposed, 403s enriched at call time).
2. Ollama /api/show probe (~0.3s): get_model_context_length step 5e
POSTed to <base_url>/api/show for KNOWN providers (openrouter etc.),
got a 404, and never cached the miss - so every fresh process paid a
full HTTP round-trip. Known non-Ollama providers now skip the probe;
local/custom/unknown endpoints keep the exact previous behavior.
3. env_probe subprocess sweep (~0.5s): the Python-toolchain probe ran
4-8 subprocess calls inside the FIRST system prompt build. Now warmed
off-thread during agent init; the prompt build hits the cache (same
lock, so a mid-flight warm just joins instead of recomputing).
4. tools.mcp_tool import (~0.4s): the between-turns MCP refresh in
build_turn_context imported the whole mcp package even with zero MCP
servers configured. MCP tools can only exist if tools.mcp_tool was
already imported (discovery/reload paths), so gate the import on
sys.modules membership - no behavior change for MCP users.
CLI additionally pre-imports run_agent + openai off-thread during the
idle banner window (same pattern as the /model picker prewarm), hiding
the remaining ~1.5s of module imports while the user types. Fixes 1-4
apply to every interaction layer (CLI, gateway, TUI, desktop, cron).
Measured cold first turn (submit -> request dispatched, openrouter,
discord token set): 4.3s before -> 0.9s after CLI prewarm (~80%); the
agent-side non-import cost drops 2.9s -> 0.36s (init) + 0.27s (turn
prologue).
DeepSeek v4 (and other strict OpenAI-compatible providers) reject an
assistant message carrying ``tool_calls: []`` with HTTP 400 "Invalid
'messages[N].tool_calls': empty array. Expected an array with minimum
length 1, but got an empty array instead." Once it hits, every retry on
the session returns the same 400 and the conversation is stuck.
An empty array is semantically identical to "no tool calls", but it
reaches the wire from session resume, host-fed histories, and the
consecutive-assistant merge in repair_message_sequence (which preserves
a pre-existing []). None of the existing sanitizer passes touch it —
they all short-circuit on ``if not tcs`` / ``if msg.get("tool_calls")``.
Fix it at sanitize_api_messages, the final pre-API chokepoint, per the
#56980 review guidance: normalize on the per-call copy (shallow-copy the
message, drop the key) rather than in repair_message_sequence, which
would destructively rewrite the persisted trajectory and prompt cache.
The salvaged commit rewrote update-marker.cjs and its test with CRLF
line endings (Windows editor artifact); restore LF so the diff shows
only the substantive change.
Some MCP servers (e.g. Spring Boot apps with a React SPA) serve their
frontend on any unmatched GET route. The MCP endpoint works perfectly
via POST (JSON-RPC), but a GET to /mcp falls through to the SPA
controller and returns text/html. Hermes's preflight content-type probe
sees HTML instead of application/json or text/event-stream and refuses
to connect.
This adds a per-server config option that
bypasses the content-type probe, letting the SDK connect directly via
POST where it works fine.
```yaml
mcp_servers:
stirling-pdf:
url: http://localhost:8090/mcp
headers:
X-API-KEY: <key>
skip_preflight: true
```
Related: #52460 (OAuth redirect preflight), #51600 (skip probe on mcp add),
#40366 (skip probe on reconnect — already merged).
Some MCP servers (e.g. DocuSeal) serve their web UI on HEAD/GET but
speak Streamable HTTP only via POST. The preflight probe now tries a
lightweight JSON-RPC `initialize` POST before rejecting endpoints
whose HEAD/GET returns a non-MCP content type (e.g. `text/html`).
If the POST returns `application/json` or `text/event-stream` with a
2xx status, the endpoint is accepted. Otherwise the original rejection
behaviour is preserved.
Adds 5 new test cases covering the POST probe path:
- POST rescues HTML HEAD with JSON response
- POST rescues HTML HEAD with event-stream response
- POST still rejects when it also returns HTML
- POST still rejects on non-2xx status
- POST not attempted when HEAD already returns valid MCP content type
Follow-up to #9006/#58899. The gateway routing index (session_key ->
SessionEntry) now lives in a new gateway_routing table in state.db as the
primary store; sessions.json is demoted to an optional legacy mirror.
- hermes_state.py: schema v19 — gateway_routing table (scope + session_key
PK; scope = resolved sessions_dir so multiple stores sharing one state.db
never cross-contaminate) with save/replace/load/delete methods
- gateway/session.py: _save() writes the whole index atomically to the DB
(mirrors the old full-file JSON rewrite semantics) and only falls back to
JSON when the DB write fails; _ensure_loaded reads the DB first and folds
in legacy sessions.json entries for keys the DB lacks (pre-migration
import; DB entries win over stale JSON)
- gateway/config.py + hermes_cli/config.py: new write_sessions_json flag
(default true for compat/downgrade safety); gateway.write_sessions_json:
false stops producing the file entirely
- sessions.json _README updated to say it's a legacy mirror + how to
disable it
Rehydration is now lossless across restarts even with sessions.json deleted:
suspended/resume_pending/model_override/token state all round-trip through
the DB (the old sessions-table recovery only rebuilt the bare key mapping).
Follow-up to the #45545 salvage: the cherry-picked fix duplicated the
~35-line header-shaping block (kimi UA, copilot headers, nvidia NIM,
provider-profile defaults, query params) from the explicit_base_url
branch. Route the main_runtime case through the same block instead —
one client-build path, no drift risk. Also uses _create_openai_client
like the sibling branch instead of constructing OpenAI directly.
When the main agent uses a named custom provider (custom:<name>),
resolve_runtime_provider correctly resolves the base_url and api_key.
But the auxiliary client re-resolves from the bare 'custom' provider
name, losing the provider identity. The bare 'custom' falls back to
OpenRouter, which _resolve_custom_runtime() then rejects — leaving all
auxiliary tasks (title gen, compression, vision, session search, etc.)
with no credentials.
Fix: when resolve_provider_client receives a main_runtime dict
containing concrete base_url + api_key, use it directly instead of
re-resolving. The main agent already solved provider resolution;
the auxiliary client just needs to reuse its answer.
Closes#45472
A Windows venv broken mid-update (e.g. python-dotenv missing after a partial
pip install) still has python.exe + Scripts\hermes.exe on disk.
unwrapWindowsVenvHermesCommand() returned that interpreter with no probe --
bypassing even the caller's --version smoke test -- so every recovery action
(Retry, Repair install, Use local gateway) re-resolved the same dead backend:
ModuleNotFoundError: No module named 'dotenv', same overlay, forever.
- unwrapWindowsVenvHermesCommand now runs canImportHermesCli() on the venv
python (checkout on PYTHONPATH, mirroring isActiveRuntimeUsable) and
returns null on failure so the resolver falls through to the bootstrap
installer, which actually repairs the venv.
- hermesRuntimeImportProbe() adds 'import dotenv' -- the first third-party
import on the CLI boot path (hermes_cli/env_loader.py) -- so a venv missing
python-dotenv fails the probe everywhere it's used (isActiveRuntimeUsable,
system-python rung, and the new unwrap gate).
- Regression tests: probe content + source assertion that the unwrap path
probes and falls through.
Regression tests from PR #51586: the inspection agent must receive the
platform-resolved enabled_toolsets and agent.disabled_toolsets, and a
Blank Slate profile's prompt-size must count exactly the 6 file/terminal
tool schemas.
The `hermes prompt-size` command now uses `_get_platform_tools()` to resolve
platform-specific toolsets the same way the gateway does, and also honors
`agent.disabled_toolsets` from config. This fixes the discrepancy where
`prompt-size` reported more tools than actually available in real sessions
for a given platform.
Fixes#41445.
Two pre-existing tests awaited run() to return after initial-connect
retry exhaustion; with #57477's parking that await hangs (CI: 300s
SIGKILL on slices 4 and 6). Assert the new contract instead: the task
stays alive (parked) and exits on shutdown.
Parking deregisters the server's tools, which removes the only paths
that could ever set _reconnect_event (circuit-breaker half-open probe
and _signal_reconnect both live inside registered tool handlers). A
parked server was therefore unrevivable short of a manual /mcp reload —
the park comment's promised breaker wake could never fire.
Make the parked wait a timed wait: every _PARKED_RETRY_INTERVAL (300s)
the run task wakes and attempts one revival probe, re-parking on
failure instead of burning the full 5-retry budget each cycle. Explicit
reconnect requests still wake it immediately. Idea credit: @Hellbayne
(PR #38881, earliest never-abandon proposal), reconciled with the
park design from #53599.
_discover_tools only filled self._tools; registry registration happened
only in _discover_and_register_server (initial start) and _refresh_tools.
After parking deregistered a server's tools, a revival rebuilt the
transport but published zero tools — a phantom recovery.
Register freshly discovered tools whenever _ready is set and the
registry entry list is empty. Extracted from PR #54139 by @nicha16
(the remainder of that PR reverses the park design and is not taken).
The local retries variable in MCPServerTask.run() accumulated across
transient disconnections — each transport exception incremented it, but
only clean transport returns (auth recovery / manual refresh) or
park-wake reset it. Five transient blips over a long-uptime gateway
would permanently park the MCP server.
Promote retries to instance attribute _reconnect_retries and reset it
at all 4 session-establishment sites in _run_stdio / _run_http, so only
consecutive failures without successful reconnection count toward the
parking budget.
Fixes#57604
Raise the CLI login floor from 180s to 315s (OAuth callback window 300s
+ headroom, matching web_server's existing constant), and let the GUI
re-auth path honor a configured connect_timeout larger than 315s.
Persists as the server's connect_timeout in config, which the probe
now honors. CLI-flag portion of PR #54494; the probe-wrapper portion
was superseded by resolving connect_timeout inside _probe_single_server.
_reauth_oauth_server (hermes mcp login / reauth) called
_probe_single_server without a timeout, so it always used the 30s
probe default — far too short for a human browser OAuth round-trip
(open → sign in → consent → loopback redirect). The server-level
connect_timeout in config.yaml was silently ignored, so login timed
out at ~40s no matter what the user configured.
Pass the server's configured connect_timeout through, with a 180s
floor for the interactive login path. Update the two TestMcpLogin
probe mocks for the new kwarg and assert the login path propagates a
>=180s timeout.
Follow-up to the salvaged #54938: the bounded reader gives a proper 413 +
anomaly telemetry for oversized chunked bodies; client_max_size makes
aiohttp enforce the same 1 MiB cap on every other read path
(#58536/#58902/#59180 pattern). Test fixture's fake Application now
accepts kwargs.
Follow-up to the salvaged #54944: before this, aiohttp's implicit 1 MiB
default client_max_size tripped BEFORE the intended 3 MB Meta cap could
apply on read() paths — the explicit value makes the documented limit
real while the bounded reader keeps chunked bodies from buffering past
3 MB (#58536/#58902/#59180 pattern).
The salvaged #25296 fixture's _FakeRequest.read() calls json.dumps but the
test module never imported json — the NameError was swallowed by the
handler's generic except → 400, failing 10 payload tests.
Follow-up to the salvaged #54620: the post-read length check bounds
processing but a chunked body is still buffered by aiohttp first.
client_max_size enforces the same 64 KiB cap mid-read on every path
(#58536/#58902/#59180 pattern).
_handle_webhook() called request.read() with no size guard. Since the
endpoint is publicly reachable, an attacker can send an arbitrarily large
POST body to exhaust gateway memory.
Add _TWILIO_WEBHOOK_MAX_BODY_BYTES (64 KiB — well above any real Twilio
payload) and gate on both Content-Length and actual read size, returning
HTTP 413 with an empty TwiML Response on oversized requests. Mirrors the
guard already present in the Raft adapter.
Follow-up to the salvaged #57943: a wedged npm (dead registry, network
blackhole) ran unbounded inside asyncio.to_thread, holding the photon
connect path hostage. Cap npm ci / npm install at 600s; on timeout, log
and leave the stale deps in place so the readiness check reports the
real error and the next reconnect tick retries.
