mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-18 04:41:56 +00:00
fix(install): use --extra all not --all-extras; drop lazy-covered extras from [all] (#24515)
* fix(install): use `--extra all` not `--all-extras`; drop lazy-covered extras from [all]
Two coupled fixes for the Windows install hang where uv sync built
python-olm from sdist and failed on missing make.
# Root cause: --all-extras vs --extra all (credit: ethernet)
`uv sync --all-extras` installs every key in [project.optional-
dependencies], bypassing the curated [all] extra entirely. So even
when [all] excluded [matrix], [rl], [yc-bench], etc., the installer
pulled them anyway because they were still defined as extras. On
Windows that meant python-olm (no wheel, needs make to build from
sdist) and the install died there.
The right flag is `--extra all` — install just the [all] extra's
contents, respecting curation. Empirically verified via dry-run:
--all-extras: pulls python-olm, mautrix, ctranslate2, onnxruntime,
atroposlib, tinker, wandb, modal, daytona, vercel,
python-telegram-bot, discord.py, slack-bolt,
dingtalk-stream, lark-oapi, anthropic, boto3,
edge-tts, elevenlabs, exa-py, fal-client, faster-
whisper, firecrawl-py, honcho-ai, parallel-web
--extra all: pulls none of those — just [all]'s curated set
Dockerfile already uses `--extra all` (with comment explaining the
gotcha) — knowledge existed; the gap was install.sh / install.ps1 /
setup-hermes.sh.
Sites fixed: scripts/install.sh L1118, scripts/install.ps1 L809,
setup-hermes.sh L245.
# Companion fix: drop lazy-covered extras from [all]
`tools/lazy_deps.py` already covers anthropic, bedrock, exa,
firecrawl, parallel-web, fal, edge-tts, elevenlabs, modal, daytona,
vercel, all messaging platforms (telegram/discord/slack/matrix/
dingtalk/feishu), honcho, and faster-whisper. They were ALSO in
[all], which defeats the whole point of lazy-install — fresh
installs eager-pulled them and inherited whatever was broken
upstream (the matrix → python-olm → no Windows wheel chain being
the proximate symptom).
[all] now contains only what genuinely can't be lazy-installed:
cron, cli, dev, pty, mcp, homeassistant, sms, acp, google, web,
youtube. Same trim applied to [termux-all]. New regression test
asserts the contract: every extra in LAZY_DEPS must NOT also appear
in [all].
# Companion fix: surface uv progress + errors
setup-hermes.sh's hash-verified path swallowed uv's stderr to a
tempfile, identical to the install.sh bug fixed in PR #24504. Same
fix applied: stream stderr through directly so users see live
progress instead of staring at a frozen prompt.
# Files
- pyproject.toml: trim [all] and [termux-all] to non-lazy extras only.
- scripts/install.sh: --all-extras → --extra all; trim _ALL_EXTRAS /
_PYPI_EXTRAS to match.
- scripts/install.ps1: --all-extras → --extra all; trim $allExtras /
$pypiExtras to match.
- setup-hermes.sh: --all-extras → --extra all; stream stderr.
- tests/test_project_metadata.py: invert matrix-in-[all] assertion;
add lazy-coverage contract test.
- uv.lock: regenerated.
# Validation
5/5 metadata tests pass. 37/37 in update_autostash + tool_token_
estimation. `uv lock --check` passes. Empirical dry-run confirms
`--extra all` excludes python-olm + RL chain on the new lockfile.
* fix(install): parse [all] from pyproject.toml instead of mirroring it
ethernet's review point: the previous patch left two hand-mirrored
copies of [all]'s contents (in install.sh's $_ALL_EXTRAS and
install.ps1's $allExtras). That guarantees future drift the next
time pyproject.toml's [all] changes.
Now both scripts parse pyproject.toml at install time using stdlib
tomllib (Python 3.11+, which the bootstrap step already requires).
Single source of truth. The only purpose of the parsed list is to
build the 'Tier 2: [all] minus broken extras' fallback spec — so we
parse, filter against $brokenExtras, and rebuild the .[a,b,c] spec.
Also: removed redundant fallback tiers.
Before: Tier 1 [all]
Tier 2 [all] minus broken
Tier 3 PyPI-only extras (no git deps)
Tier 4 [web,mcp,cron,cli,messaging,dev]
Tier 5 .
After: Tier 1 [all]
Tier 2 [all] minus broken
Tier 3 .
Tier 3 (PyPI-only) and Tier 4 (dashboard+core) used to dodge the [rl]
git+sdist deps and the [matrix] python-olm build. Both are no longer
in [all] post-2026-05-12 lazy-install migration, so the carve-out
tiers had no remaining content. Tier 4 also referenced [messaging],
which is now lazy-installed — the hardcoded fallback was actually
inconsistent with the new policy.
Defensive fallback: if tomllib parse fails (corrupted pyproject,
unexpected schema), Tier 2 collapses to '.[all]' (same as Tier 1) so
the broken-extras path becomes a no-op rather than crashing.
* fix(gateway): hide Matrix from setup picker on Windows
Matrix is the one messaging platform that has no working install path
on Windows: [matrix] -> mautrix[encryption] -> python-olm, which has
Linux-only wheels and needs make + libolm to build from sdist. The
[all] cleanup in this PR keeps mautrix out of fresh installs, but a
user who picked Matrix in 'hermes setup gateway' would still walk
into the same sdist build failure when the wizard tried to install
the extra.
Hide the option at the picker so users never get the chance to try.
The gate lives in _all_platforms() — single source of truth for the
setup wizard, the curses gateway-config menu, and any future picker.
Adapter loading at runtime is intentionally NOT gated: users who
already have MATRIX_* env vars set (e.g. config copied from a Linux
install) keep working if they somehow have python-olm available.
This is the lowest-friction fix — picker visibility only.
Tests cover linux/darwin/win32 and verify other platforms aren't
collateral damage.
This commit is contained in:
Teknium
GitHub
parent
4bb0a82a2b
commit
3955aefced
8 changed files with 296 additions and 218 deletions
|
|
@ -806,7 +806,14 @@ function Install-Dependencies {
|
|||
# current extras spec, NOT because they're equivalent in posture.
|
||||
if (Test-Path "uv.lock") {
|
||||
Write-Info "Trying tier: hash-verified (uv.lock) ..."
|
||||
& $UvCmd sync --all-extras --locked
|
||||
# Critical flag choice: `--extra all`, NOT `--all-extras`.
|
||||
# --all-extras = every [project.optional-dependencies] key,
|
||||
# bypassing the curated [all] extra. On Windows
|
||||
# that means [matrix] -> python-olm (no wheel,
|
||||
# needs `make` to build from sdist) and the
|
||||
# install fails.
|
||||
# --extra all = just the [all] extra's contents (curated).
|
||||
& $UvCmd sync --extra all --locked
|
||||
if ($LASTEXITCODE -eq 0) {
|
||||
Write-Success "Main package installed (hash-verified via uv.lock)"
|
||||
$script:InstalledTier = "hash-verified (uv.lock)"
|
||||
|
|
@ -822,53 +829,59 @@ function Install-Dependencies {
|
|||
$skipPipFallback = $false
|
||||
}
|
||||
|
||||
# Install main package. Tiered fallback so a single flaky git+https dep
|
||||
# (atroposlib / tinker in the [rl] extra) doesn't silently drop
|
||||
# dashboard/MCP/cron/messaging extras. Each tier's stdout/stderr is
|
||||
# Install main package. Tiered fallback so a single flaky transitive
|
||||
# doesn't silently drop everything. Each tier's stdout/stderr is
|
||||
# preserved — no Out-Null swallowing — so the user can see what failed.
|
||||
#
|
||||
# Tier 1: [all] — everything, including RL git+https deps (best case).
|
||||
# Tier 2: [all] minus a small list of currently-broken extras. The
|
||||
# broken list is centralised in $brokenExtras below — when
|
||||
# a package gets quarantined / yanked / pulled, add it here
|
||||
# and the resolver no longer chokes on it. This is what saves
|
||||
# the user from silently losing 10+ unrelated extras every
|
||||
# time one upstream package breaks.
|
||||
# Tier 3: [core-extras] synthesised locally — all PyPI-only extras we
|
||||
# ship, also minus $brokenExtras. Drops [rl] and [matrix]
|
||||
# (linux-only) which are the usual failure culprits.
|
||||
# Tier 4: [web,mcp,cron,cli,messaging,dev] — the minimum we strongly
|
||||
# believe a user expects `hermes dashboard` / slash commands /
|
||||
# cron / messaging platforms to work out of the box.
|
||||
# Tier 5: bare `.` — last-resort so at least the core CLI launches.
|
||||
# Tier 1: [all] — the curated extra in pyproject.toml.
|
||||
# Tier 2: [all] minus the currently-broken extras list ($brokenExtras).
|
||||
# Edit $brokenExtras below when something on PyPI breaks; this
|
||||
# lets users keep the rest of [all] when one transitive is
|
||||
# unavailable. The list of [all]'s contents is parsed from
|
||||
# pyproject.toml at runtime — there is NO hand-mirrored copy
|
||||
# to drift out of sync.
|
||||
# Tier 3: bare `.` — last-resort so at least the core CLI launches.
|
||||
|
||||
# Currently-broken extras. Edit this list when an upstream package
|
||||
# gets quarantined / yanked / breaks resolution. Empty means everything
|
||||
# in [all] should be installable; populate with the names of extras
|
||||
# whose deps are temporarily unavailable to keep installs working
|
||||
# for users.
|
||||
# whose deps are temporarily unavailable.
|
||||
$brokenExtras = @()
|
||||
|
||||
$allExtras = @(
|
||||
"modal","daytona","vercel","messaging","matrix","cron","cli","dev",
|
||||
"tts-premium","slack","pty","honcho","mcp","homeassistant","sms",
|
||||
"acp","voice","dingtalk","feishu","google","bedrock","web",
|
||||
"youtube"
|
||||
)
|
||||
$pypiExtras = @(
|
||||
"web","mcp","cron","cli","voice","messaging","slack","dev","acp",
|
||||
"pty","homeassistant","sms","tts-premium","honcho","google",
|
||||
"bedrock","dingtalk","feishu","modal","daytona","vercel","youtube"
|
||||
)
|
||||
$safeAll = ($allExtras | Where-Object { $brokenExtras -notcontains $_ }) -join ","
|
||||
$safePypi = ($pypiExtras | Where-Object { $brokenExtras -notcontains $_ }) -join ","
|
||||
# Parse [project.optional-dependencies].all from pyproject.toml.
|
||||
# tomllib is stdlib on Python 3.11+ which the bootstrap guarantees.
|
||||
$pythonExeForParse = if (-not $NoVenv) { "$InstallDir\venv\Scripts\python.exe" } else { (& $UvCmd python find $PythonVersion) }
|
||||
$allExtras = @()
|
||||
if (Test-Path $pythonExeForParse) {
|
||||
$parsed = & $pythonExeForParse -c @"
|
||||
import re, sys, tomllib
|
||||
try:
|
||||
with open('pyproject.toml', 'rb') as fh:
|
||||
data = tomllib.load(fh)
|
||||
specs = data['project']['optional-dependencies']['all']
|
||||
out = []
|
||||
for s in specs:
|
||||
m = re.search(r'hermes-agent\[([\w-]+)\]', s)
|
||||
if m: out.append(m.group(1))
|
||||
print(','.join(out))
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
"@ 2>$null
|
||||
if ($LASTEXITCODE -eq 0 -and $parsed) {
|
||||
$allExtras = $parsed.Trim().Split(',')
|
||||
}
|
||||
}
|
||||
if (-not $allExtras -or $allExtras.Count -eq 0) {
|
||||
Write-Warn "Could not parse [all] from pyproject.toml; Tier 2 will be a no-op."
|
||||
$safeAll = "all"
|
||||
} else {
|
||||
$safeAll = ($allExtras | Where-Object { $brokenExtras -notcontains $_ }) -join ","
|
||||
}
|
||||
$brokenLabel = if ($brokenExtras) { ($brokenExtras -join ", ") } else { "none" }
|
||||
|
||||
$installTiers = @(
|
||||
@{ Name = "all (with RL/matrix extras)"; Spec = ".[all]" },
|
||||
@{ Name = "all"; Spec = ".[all]" },
|
||||
@{ Name = "all minus known-broken ($brokenLabel)"; Spec = ".[$safeAll]" },
|
||||
@{ Name = "PyPI-only extras (no git deps)"; Spec = ".[$safePypi]" },
|
||||
@{ Name = "dashboard + core platforms"; Spec = ".[web,mcp,cron,cli,messaging,dev]" },
|
||||
@{ Name = "core only (no extras)"; Spec = "." }
|
||||
)
|
||||
$installed = $skipPipFallback
|
||||
|
|
|
|||
|
|
@ -1100,22 +1100,30 @@ install_deps() {
|
|||
# extras spec, NOT because they're equivalent in posture.
|
||||
if [ -f "uv.lock" ]; then
|
||||
log_info "Trying tier: hash-verified (uv.lock) ..."
|
||||
log_info "(this resolves + downloads ~50 packages — first run on a fresh"
|
||||
log_info " venv can take 1-5 minutes; uv prints progress below)"
|
||||
log_info "(this resolves + downloads the curated [all] set — first run on a"
|
||||
log_info " fresh venv can take 1-5 minutes; uv prints progress below)"
|
||||
# Stream uv's progress directly to the user instead of swallowing
|
||||
# it with `2>"$(mktemp)"`. Two reasons:
|
||||
# 1. `--all-extras --locked` against a fresh venv has to pull
|
||||
# every transitive (torch-class deps included) — silencing
|
||||
# stderr makes the install look frozen for minutes on slow
|
||||
# networks. Users see "Trying tier: hash-verified ..." and
|
||||
# assume it's hung.
|
||||
# 1. `--extra all --locked` against a fresh venv has to pull
|
||||
# every transitive — silencing stderr makes the install
|
||||
# look frozen for minutes on slow networks. Users see
|
||||
# "Trying tier: hash-verified ..." and assume it's hung.
|
||||
# 2. The previous `2>"$(mktemp)"` substituted the path at
|
||||
# command-build time but never saved it, so on failure the
|
||||
# uv error message was unreachable — the user just got the
|
||||
# generic "lockfile may be stale" warning.
|
||||
#
|
||||
# Critical flag choice: `--extra all`, NOT `--all-extras`.
|
||||
# --all-extras = every [project.optional-dependencies] key.
|
||||
# This bypasses the curated `[all]` extra
|
||||
# entirely and pulls e.g. [matrix] (which
|
||||
# needs python-olm + make on Windows) and
|
||||
# [rl] (git+https deps that fail offline).
|
||||
# --extra all = install just the `[all]` extra's contents.
|
||||
# This respects the curation in pyproject.toml.
|
||||
# uv's own progress UI handles TTY detection and downgrades
|
||||
# gracefully when stdout/stderr aren't terminals.
|
||||
if UV_PROJECT_ENVIRONMENT="$INSTALL_DIR/venv" $UV_CMD sync --all-extras --locked; then
|
||||
if UV_PROJECT_ENVIRONMENT="$INSTALL_DIR/venv" $UV_CMD sync --extra all --locked; then
|
||||
log_success "Main package installed (hash-verified via uv.lock)"
|
||||
log_success "All dependencies installed"
|
||||
return 0
|
||||
|
|
@ -1131,57 +1139,63 @@ install_deps() {
|
|||
# fresh install all the way down to "core only" — the user should keep
|
||||
# everything else they signed up for.
|
||||
#
|
||||
# Tier 1: [all] — everything, including RL git+https deps (best case).
|
||||
# Tier 2: [all] minus the currently-broken extras list. Edit
|
||||
# _BROKEN_EXTRAS below when something on PyPI breaks; this lets
|
||||
# users keep voice/honcho/google/slack/matrix/etc. even when
|
||||
# one transitive is unavailable. List the extras here as bare
|
||||
# names from pyproject.toml [project.optional-dependencies] —
|
||||
# the script translates them to `[a,b,c]` form below.
|
||||
# Tier 3: PyPI-only extras (no git deps) — drops [rl] / [yc-bench]
|
||||
# which are git+https and may fail in restricted networks.
|
||||
# Tier 4: dashboard + core platforms — minimum viable interactive set.
|
||||
# Tier 5: bare `.` — last-resort so at least the core CLI launches.
|
||||
#
|
||||
# Each tier's stderr is captured to a tempfile so we can show the user
|
||||
# WHY the higher tier failed instead of silently dropping support.
|
||||
# Tier 1: [all] — the curated extra in pyproject.toml.
|
||||
# Tier 2: [all] minus the currently-broken extras list (_BROKEN_EXTRAS).
|
||||
# Edit _BROKEN_EXTRAS below when something on PyPI breaks; this
|
||||
# lets users keep the rest of [all] when one transitive is
|
||||
# unavailable. The list of [all]'s contents is parsed from
|
||||
# pyproject.toml at runtime — there is NO hand-mirrored copy
|
||||
# to drift out of sync. If you want to change what [all]
|
||||
# contains, edit pyproject.toml only.
|
||||
# Tier 3: bare `.` — last-resort so at least the core CLI launches.
|
||||
# Skipped tiers like "PyPI-only extras (no git deps)" used to
|
||||
# exist to dodge [rl] / [matrix] git+sdist deps; those are no
|
||||
# longer in [all] post-2026-05-12 lazy-install migration, so
|
||||
# a separate PyPI-only tier had no remaining content.
|
||||
local _BROKEN_EXTRAS=() # populate when an extra becomes unresolvable
|
||||
local _ALL_EXTRAS=(
|
||||
modal daytona vercel messaging matrix cron cli dev tts-premium slack
|
||||
pty honcho mcp homeassistant sms acp voice dingtalk feishu google
|
||||
bedrock web youtube
|
||||
)
|
||||
# Tier 2: all extras minus _BROKEN_EXTRAS
|
||||
local _SAFE_EXTRAS=()
|
||||
local _e _b _skip
|
||||
for _e in "${_ALL_EXTRAS[@]}"; do
|
||||
_skip=false
|
||||
for _b in "${_BROKEN_EXTRAS[@]}"; do
|
||||
if [ "$_e" = "$_b" ]; then _skip=true; break; fi
|
||||
|
||||
# Parse [project.optional-dependencies].all from pyproject.toml.
|
||||
# tomllib is stdlib on Python 3.11+ which uv's bootstrap guarantees.
|
||||
# Falls back to a hand list if parse fails — defensive only.
|
||||
local _ALL_EXTRAS_CSV
|
||||
_ALL_EXTRAS_CSV="$(
|
||||
"$PYTHON_PATH" - <<'PY' 2>/dev/null
|
||||
import re, sys, tomllib
|
||||
try:
|
||||
with open("pyproject.toml", "rb") as fh:
|
||||
data = tomllib.load(fh)
|
||||
specs = data["project"]["optional-dependencies"]["all"]
|
||||
extras = []
|
||||
for s in specs:
|
||||
m = re.search(r"hermes-agent\[([\w-]+)\]", s)
|
||||
if m:
|
||||
extras.append(m.group(1))
|
||||
print(",".join(extras))
|
||||
except Exception as e:
|
||||
print("", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
PY
|
||||
)"
|
||||
if [ -z "$_ALL_EXTRAS_CSV" ]; then
|
||||
log_warn "Could not parse [all] from pyproject.toml; falling back to .[all] only."
|
||||
_ALL_EXTRAS_CSV=""
|
||||
fi
|
||||
|
||||
# Build "[all] minus broken" spec by filtering the parsed list.
|
||||
local _SAFE_SPEC=".[all]"
|
||||
if [ -n "$_ALL_EXTRAS_CSV" ] && [ "${#_BROKEN_EXTRAS[@]}" -gt 0 ]; then
|
||||
local _SAFE_EXTRAS=()
|
||||
local _e _b _skip
|
||||
IFS=',' read -ra _ALL_EXTRAS_ARR <<< "$_ALL_EXTRAS_CSV"
|
||||
for _e in "${_ALL_EXTRAS_ARR[@]}"; do
|
||||
_skip=false
|
||||
for _b in "${_BROKEN_EXTRAS[@]}"; do
|
||||
if [ "$_e" = "$_b" ]; then _skip=true; break; fi
|
||||
done
|
||||
if [ "$_skip" = false ]; then _SAFE_EXTRAS+=("$_e"); fi
|
||||
done
|
||||
if [ "$_skip" = false ]; then _SAFE_EXTRAS+=("$_e"); fi
|
||||
done
|
||||
local _SAFE_SPEC
|
||||
_SAFE_SPEC=".[$(IFS=,; echo "${_SAFE_EXTRAS[*]}")]"
|
||||
# Tier 3: PyPI-only extras (no git deps), still skipping broken ones.
|
||||
# Mirrors the install.ps1 list but excludes [rl] / [yc-bench] / [matrix]
|
||||
# (matrix needs python-olm which fails to build on some hosts).
|
||||
local _PYPI_EXTRAS=(
|
||||
web mcp cron cli voice messaging slack dev acp pty homeassistant sms
|
||||
tts-premium honcho google bedrock dingtalk feishu modal daytona vercel
|
||||
youtube
|
||||
)
|
||||
local _PYPI_SAFE=()
|
||||
for _e in "${_PYPI_EXTRAS[@]}"; do
|
||||
_skip=false
|
||||
for _b in "${_BROKEN_EXTRAS[@]}"; do
|
||||
if [ "$_e" = "$_b" ]; then _skip=true; break; fi
|
||||
done
|
||||
if [ "$_skip" = false ]; then _PYPI_SAFE+=("$_e"); fi
|
||||
done
|
||||
local _PYPI_SPEC
|
||||
_PYPI_SPEC=".[$(IFS=,; echo "${_PYPI_SAFE[*]}")]"
|
||||
local _TIER4_SPEC=".[web,mcp,cron,cli,messaging,dev]"
|
||||
_SAFE_SPEC=".[$(IFS=,; echo "${_SAFE_EXTRAS[*]}")]"
|
||||
fi
|
||||
|
||||
ALL_INSTALL_LOG=$(mktemp)
|
||||
local _installed=false
|
||||
|
|
@ -1201,10 +1215,8 @@ install_deps() {
|
|||
return 1
|
||||
}
|
||||
|
||||
install_tier "all (with RL/matrix extras)" ".[all]" \
|
||||
install_tier "all" ".[all]" \
|
||||
|| install_tier "all minus known-broken (${_BROKEN_EXTRAS[*]:-none})" "$_SAFE_SPEC" \
|
||||
|| install_tier "PyPI-only extras (no git deps)" "$_PYPI_SPEC" \
|
||||
|| install_tier "dashboard + core platforms" "$_TIER4_SPEC" \
|
||||
|| install_tier "core only (no extras)" "."
|
||||
|
||||
rm -f "$ALL_INSTALL_LOG"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue