hermes-agent/setup-hermes.sh
Teknium 3955aefced
fix(install): use --extra all not --all-extras; drop lazy-covered extras from [all] (#24515)
* fix(install): use `--extra all` not `--all-extras`; drop lazy-covered extras from [all]

Two coupled fixes for the Windows install hang where uv sync built
python-olm from sdist and failed on missing make.

# Root cause: --all-extras vs --extra all (credit: ethernet)

`uv sync --all-extras` installs every key in [project.optional-
dependencies], bypassing the curated [all] extra entirely. So even
when [all] excluded [matrix], [rl], [yc-bench], etc., the installer
pulled them anyway because they were still defined as extras. On
Windows that meant python-olm (no wheel, needs make to build from
sdist) and the install died there.

The right flag is `--extra all` — install just the [all] extra's
contents, respecting curation. Empirically verified via dry-run:

  --all-extras: pulls python-olm, mautrix, ctranslate2, onnxruntime,
                atroposlib, tinker, wandb, modal, daytona, vercel,
                python-telegram-bot, discord.py, slack-bolt,
                dingtalk-stream, lark-oapi, anthropic, boto3,
                edge-tts, elevenlabs, exa-py, fal-client, faster-
                whisper, firecrawl-py, honcho-ai, parallel-web
  --extra all:  pulls none of those — just [all]'s curated set

Dockerfile already uses `--extra all` (with comment explaining the
gotcha) — knowledge existed; the gap was install.sh / install.ps1 /
setup-hermes.sh.

Sites fixed: scripts/install.sh L1118, scripts/install.ps1 L809,
setup-hermes.sh L245.

# Companion fix: drop lazy-covered extras from [all]

`tools/lazy_deps.py` already covers anthropic, bedrock, exa,
firecrawl, parallel-web, fal, edge-tts, elevenlabs, modal, daytona,
vercel, all messaging platforms (telegram/discord/slack/matrix/
dingtalk/feishu), honcho, and faster-whisper. They were ALSO in
[all], which defeats the whole point of lazy-install — fresh
installs eager-pulled them and inherited whatever was broken
upstream (the matrix → python-olm → no Windows wheel chain being
the proximate symptom).

[all] now contains only what genuinely can't be lazy-installed:
cron, cli, dev, pty, mcp, homeassistant, sms, acp, google, web,
youtube. Same trim applied to [termux-all]. New regression test
asserts the contract: every extra in LAZY_DEPS must NOT also appear
in [all].

# Companion fix: surface uv progress + errors

setup-hermes.sh's hash-verified path swallowed uv's stderr to a
tempfile, identical to the install.sh bug fixed in PR #24504. Same
fix applied: stream stderr through directly so users see live
progress instead of staring at a frozen prompt.

# Files

- pyproject.toml: trim [all] and [termux-all] to non-lazy extras only.
- scripts/install.sh: --all-extras → --extra all; trim _ALL_EXTRAS /
  _PYPI_EXTRAS to match.
- scripts/install.ps1: --all-extras → --extra all; trim $allExtras /
  $pypiExtras to match.
- setup-hermes.sh: --all-extras → --extra all; stream stderr.
- tests/test_project_metadata.py: invert matrix-in-[all] assertion;
  add lazy-coverage contract test.
- uv.lock: regenerated.

# Validation

5/5 metadata tests pass. 37/37 in update_autostash + tool_token_
estimation. `uv lock --check` passes. Empirical dry-run confirms
`--extra all` excludes python-olm + RL chain on the new lockfile.

* fix(install): parse [all] from pyproject.toml instead of mirroring it

ethernet's review point: the previous patch left two hand-mirrored
copies of [all]'s contents (in install.sh's $_ALL_EXTRAS and
install.ps1's $allExtras). That guarantees future drift the next
time pyproject.toml's [all] changes.

Now both scripts parse pyproject.toml at install time using stdlib
tomllib (Python 3.11+, which the bootstrap step already requires).
Single source of truth. The only purpose of the parsed list is to
build the 'Tier 2: [all] minus broken extras' fallback spec — so we
parse, filter against $brokenExtras, and rebuild the .[a,b,c] spec.

Also: removed redundant fallback tiers.

  Before:   Tier 1 [all]
            Tier 2 [all] minus broken
            Tier 3 PyPI-only extras (no git deps)
            Tier 4 [web,mcp,cron,cli,messaging,dev]
            Tier 5 .

  After:    Tier 1 [all]
            Tier 2 [all] minus broken
            Tier 3 .

Tier 3 (PyPI-only) and Tier 4 (dashboard+core) used to dodge the [rl]
git+sdist deps and the [matrix] python-olm build. Both are no longer
in [all] post-2026-05-12 lazy-install migration, so the carve-out
tiers had no remaining content. Tier 4 also referenced [messaging],
which is now lazy-installed — the hardcoded fallback was actually
inconsistent with the new policy.

Defensive fallback: if tomllib parse fails (corrupted pyproject,
unexpected schema), Tier 2 collapses to '.[all]' (same as Tier 1) so
the broken-extras path becomes a no-op rather than crashing.

* fix(gateway): hide Matrix from setup picker on Windows

Matrix is the one messaging platform that has no working install path
on Windows: [matrix] -> mautrix[encryption] -> python-olm, which has
Linux-only wheels and needs make + libolm to build from sdist. The
[all] cleanup in this PR keeps mautrix out of fresh installs, but a
user who picked Matrix in 'hermes setup gateway' would still walk
into the same sdist build failure when the wizard tried to install
the extra.

Hide the option at the picker so users never get the chance to try.
The gate lives in _all_platforms() — single source of truth for the
setup wizard, the curses gateway-config menu, and any future picker.

Adapter loading at runtime is intentionally NOT gated: users who
already have MATRIX_* env vars set (e.g. config copied from a Linux
install) keep working if they somehow have python-olm available.
This is the lowest-friction fix — picker visibility only.

Tests cover linux/darwin/win32 and verify other platforms aren't
collateral damage.
2026-05-12 15:06:25 -07:00

472 lines
18 KiB
Bash
Executable file

#!/bin/bash
# ============================================================================
# Hermes Agent Setup Script
# ============================================================================
# Quick setup for developers who cloned the repo manually.
# Uses uv for desktop/server setup and Python's stdlib venv + pip on Termux.
#
# Usage:
# ./setup-hermes.sh
#
# This script:
# 1. Detects desktop/server vs Android/Termux setup path
# 2. Creates a Python 3.11 virtual environment
# 3. Installs the appropriate dependency set for the platform
# 4. Creates .env from template (if not exists)
# 5. Symlinks the 'hermes' CLI command into a user-facing bin dir
# 6. Runs the setup wizard (optional)
# ============================================================================
set -e
# Colors
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
CYAN='\033[0;36m'
RED='\033[0;31m'
NC='\033[0m'
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
# Prevent uv from discovering config files (uv.toml, pyproject.toml) from the
# wrong user's home directory when running under sudo -u <user>. See #21269.
export UV_NO_CONFIG=1
PYTHON_VERSION="3.11"
is_termux() {
[ -n "${TERMUX_VERSION:-}" ] || [[ "${PREFIX:-}" == *"com.termux/files/usr"* ]]
}
get_command_link_dir() {
if is_termux && [ -n "${PREFIX:-}" ]; then
echo "$PREFIX/bin"
else
echo "$HOME/.local/bin"
fi
}
get_command_link_display_dir() {
if is_termux && [ -n "${PREFIX:-}" ]; then
echo '$PREFIX/bin'
else
echo '~/.local/bin'
fi
}
echo ""
echo -e "${CYAN}⚕ Hermes Agent Setup${NC}"
echo ""
# ============================================================================
# Install / locate uv
# ============================================================================
echo -e "${CYAN}${NC} Checking for uv..."
UV_CMD=""
if is_termux; then
echo -e "${CYAN}${NC} Termux detected — using Python's stdlib venv + pip instead of uv"
else
if command -v uv &> /dev/null; then
UV_CMD="uv"
elif [ -x "$HOME/.local/bin/uv" ]; then
UV_CMD="$HOME/.local/bin/uv"
elif [ -x "$HOME/.cargo/bin/uv" ]; then
UV_CMD="$HOME/.cargo/bin/uv"
fi
if [ -n "$UV_CMD" ]; then
UV_VERSION=$($UV_CMD --version 2>/dev/null)
echo -e "${GREEN}${NC} uv found ($UV_VERSION)"
else
echo -e "${CYAN}${NC} Installing uv..."
# Capture installer output so a failure shows the user WHY
# (network, glibc mismatch on old distros, missing curl, disk
# full, etc.) instead of "✗ Failed to install uv" with zero
# diagnostic. Two-stage to avoid `curl | sh` masking curl
# failures (sh exits 0 on empty stdin under no pipefail).
_uv_log="$(mktemp 2>/dev/null || echo "/tmp/hermes-uv-install.$$.log")"
_uv_installer="$(mktemp 2>/dev/null || echo "/tmp/hermes-uv-installer.$$.sh")"
if ! curl -LsSf https://astral.sh/uv/install.sh -o "$_uv_installer" 2>"$_uv_log"; then
echo -e "${RED}${NC} Failed to download uv installer."
sed 's/^/ /' "$_uv_log" >&2
echo -e "${CYAN}${NC} Install manually: https://docs.astral.sh/uv/"
rm -f "$_uv_log" "$_uv_installer"
exit 1
fi
if sh "$_uv_installer" >>"$_uv_log" 2>&1; then
rm -f "$_uv_installer"
if [ -x "$HOME/.local/bin/uv" ]; then
UV_CMD="$HOME/.local/bin/uv"
elif [ -x "$HOME/.cargo/bin/uv" ]; then
UV_CMD="$HOME/.cargo/bin/uv"
fi
if [ -n "$UV_CMD" ]; then
rm -f "$_uv_log"
UV_VERSION=$($UV_CMD --version 2>/dev/null)
echo -e "${GREEN}${NC} uv installed ($UV_VERSION)"
else
echo -e "${RED}${NC} uv installer reported success but binary not found. Add ~/.local/bin to PATH and retry."
echo -e "${CYAN}${NC} Installer output:"
sed 's/^/ /' "$_uv_log" >&2
rm -f "$_uv_log"
exit 1
fi
else
echo -e "${RED}${NC} Failed to install uv."
echo -e "${CYAN}${NC} Installer output:"
sed 's/^/ /' "$_uv_log" >&2
echo -e "${CYAN}${NC} Install manually: https://docs.astral.sh/uv/"
rm -f "$_uv_log" "$_uv_installer"
exit 1
fi
fi
fi
# ============================================================================
# Python check (uv can provision it automatically)
# ============================================================================
echo -e "${CYAN}${NC} Checking Python $PYTHON_VERSION..."
if is_termux; then
if command -v python >/dev/null 2>&1; then
PYTHON_PATH="$(command -v python)"
if "$PYTHON_PATH" -c 'import sys; raise SystemExit(0 if sys.version_info >= (3, 11) else 1)' 2>/dev/null; then
PYTHON_FOUND_VERSION=$($PYTHON_PATH --version 2>/dev/null)
echo -e "${GREEN}${NC} $PYTHON_FOUND_VERSION found"
else
echo -e "${RED}${NC} Termux Python must be 3.11+"
echo " Run: pkg install python"
exit 1
fi
else
echo -e "${RED}${NC} Python not found in Termux"
echo " Run: pkg install python"
exit 1
fi
else
if $UV_CMD python find "$PYTHON_VERSION" &> /dev/null; then
PYTHON_PATH=$($UV_CMD python find "$PYTHON_VERSION")
PYTHON_FOUND_VERSION=$($PYTHON_PATH --version 2>/dev/null)
echo -e "${GREEN}${NC} $PYTHON_FOUND_VERSION found"
else
echo -e "${CYAN}${NC} Python $PYTHON_VERSION not found, installing via uv..."
$UV_CMD python install "$PYTHON_VERSION"
PYTHON_PATH=$($UV_CMD python find "$PYTHON_VERSION")
PYTHON_FOUND_VERSION=$($PYTHON_PATH --version 2>/dev/null)
echo -e "${GREEN}${NC} $PYTHON_FOUND_VERSION installed"
fi
fi
# ============================================================================
# Virtual environment
# ============================================================================
echo -e "${CYAN}${NC} Setting up virtual environment..."
if [ -d "venv" ]; then
echo -e "${CYAN}${NC} Removing old venv..."
rm -rf venv
fi
if is_termux; then
"$PYTHON_PATH" -m venv venv
echo -e "${GREEN}${NC} venv created with stdlib venv"
else
$UV_CMD venv venv --python "$PYTHON_VERSION"
echo -e "${GREEN}${NC} venv created (Python $PYTHON_VERSION)"
fi
export VIRTUAL_ENV="$SCRIPT_DIR/venv"
SETUP_PYTHON="$SCRIPT_DIR/venv/bin/python"
# ============================================================================
# Dependencies
# ============================================================================
echo -e "${CYAN}${NC} Installing dependencies..."
if is_termux; then
export ANDROID_API_LEVEL="$(getprop ro.build.version.sdk 2>/dev/null || printf '%s' "${ANDROID_API_LEVEL:-}")"
echo -e "${CYAN}${NC} Termux detected — installing the tested Android bundle"
"$SETUP_PYTHON" -m pip install --upgrade pip setuptools wheel
if [ -f "constraints-termux.txt" ]; then
"$SETUP_PYTHON" -m pip install -e ".[termux]" -c constraints-termux.txt || {
echo -e "${YELLOW}${NC} Termux bundle install failed, falling back to base install..."
"$SETUP_PYTHON" -m pip install -e "." -c constraints-termux.txt
}
else
"$SETUP_PYTHON" -m pip install -e ".[termux]" || "$SETUP_PYTHON" -m pip install -e "."
fi
echo -e "${GREEN}${NC} Dependencies installed"
else
# Prefer uv sync with lockfile (hash-verified installs) when available,
# fall back to pip install for compatibility or when lockfile is stale.
#
# Multi-tier pip fallback. Goal: ONE compromised PyPI package
# (mistralai 2.4.6 in May 2026 → quarantined) shouldn't silently demote
# a fresh setup to "core only". Edit _BROKEN_EXTRAS when a transitive
# breaks; users keep voice / honcho / google / slack / matrix etc. even
# if mistral can't resolve.
_BROKEN_EXTRAS=() # populate when an extra becomes unresolvable
_ALL_EXTRAS=(
modal daytona vercel messaging matrix cron cli dev tts-premium slack
pty honcho mcp homeassistant sms acp voice dingtalk feishu google
bedrock web youtube
)
_SAFE_EXTRAS=()
for _e in "${_ALL_EXTRAS[@]}"; do
_skip=false
for _b in "${_BROKEN_EXTRAS[@]}"; do
[ "$_e" = "$_b" ] && _skip=true && break
done
[ "$_skip" = false ] && _SAFE_EXTRAS+=("$_e")
done
_SAFE_SPEC=".[$(IFS=,; echo "${_SAFE_EXTRAS[*]}")]"
_try_install() {
$UV_CMD pip install -e ".[all]" \
|| $UV_CMD pip install -e "$_SAFE_SPEC" \
|| $UV_CMD pip install -e "."
}
if [ -f "uv.lock" ]; then
# Hash-verified install (preferred). The lockfile records SHA256
# hashes for every transitive — a compromised transitive would have
# a different hash and be REJECTED by uv. This is the only path
# that protects against transitive-package supply-chain attacks
# (the direct deps in pyproject.toml are exact-pinned, but
# `uv pip install` re-resolves transitives fresh from PyPI).
echo -e "${CYAN}${NC} Using uv.lock for hash-verified installation..."
echo -e "${CYAN}${NC} (first run on a fresh venv can take 1-5 minutes; uv prints progress below)"
# Critical flag choice: `--extra all`, NOT `--all-extras`. The
# latter installs every [project.optional-dependencies] key,
# bypassing the curated [all] extra and pulling backends like
# [matrix] (python-olm needs make on Windows) and [rl] (git+https
# deps that fail offline). See pyproject.toml's [all] for the
# curated set, and tools/lazy_deps.py for backends that install
# at first use.
# Also: stream stderr through directly so the user sees uv's
# progress UI instead of staring at a frozen prompt.
if UV_PROJECT_ENVIRONMENT="$SCRIPT_DIR/venv" $UV_CMD sync --extra all --locked; then
echo -e "${GREEN}${NC} Dependencies installed (hash-verified via uv.lock)"
else
echo -e "${YELLOW}${NC} Lockfile sync failed (see uv output above)."
echo -e "${YELLOW}${NC} Falling back to PyPI resolve — transitives will NOT be hash-verified."
_try_install
echo -e "${GREEN}${NC} Dependencies installed (transitives re-resolved, not hash-verified)"
fi
else
echo -e "${YELLOW}${NC} uv.lock not found — installing without hash verification of transitives."
_try_install
echo -e "${GREEN}${NC} Dependencies installed (transitives re-resolved, not hash-verified)"
fi
fi
# ============================================================================
# Submodules (terminal backend + RL training)
# ============================================================================
echo -e "${CYAN}${NC} Installing optional submodules..."
# tinker-atropos (RL training backend)
if is_termux; then
echo -e "${CYAN}${NC} Skipping tinker-atropos on Termux (not part of the tested Android path)"
elif [ -d "tinker-atropos" ] && [ -f "tinker-atropos/pyproject.toml" ]; then
$UV_CMD pip install -e "./tinker-atropos" && \
echo -e "${GREEN}${NC} tinker-atropos installed" || \
echo -e "${YELLOW}${NC} tinker-atropos install failed (RL tools may not work)"
else
echo -e "${YELLOW}${NC} tinker-atropos not found (run: git submodule update --init --recursive)"
fi
# ============================================================================
# Optional: ripgrep (for faster file search)
# ============================================================================
echo -e "${CYAN}${NC} Checking ripgrep (optional, for faster search)..."
if command -v rg &> /dev/null; then
echo -e "${GREEN}${NC} ripgrep found"
else
echo -e "${YELLOW}${NC} ripgrep not found (file search will use grep fallback)"
read -p "Install ripgrep for faster search? [Y/n] " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]] || [[ -z $REPLY ]]; then
INSTALLED=false
if is_termux; then
pkg install -y ripgrep && INSTALLED=true
else
# Check if sudo is available
if command -v sudo &> /dev/null && sudo -n true 2>/dev/null; then
if command -v apt &> /dev/null; then
sudo apt install -y ripgrep && INSTALLED=true
elif command -v dnf &> /dev/null; then
sudo dnf install -y ripgrep && INSTALLED=true
fi
fi
# Try brew (no sudo needed)
if [ "$INSTALLED" = false ] && command -v brew &> /dev/null; then
brew install ripgrep && INSTALLED=true
fi
# Try cargo (no sudo needed)
if [ "$INSTALLED" = false ] && command -v cargo &> /dev/null; then
echo -e "${CYAN}${NC} Trying cargo install (no sudo required)..."
cargo install ripgrep && INSTALLED=true
fi
fi
if [ "$INSTALLED" = true ]; then
echo -e "${GREEN}${NC} ripgrep installed"
else
echo -e "${YELLOW}${NC} Auto-install failed. Install options:"
if is_termux; then
echo " pkg install ripgrep # Termux / Android"
else
echo " sudo apt install ripgrep # Debian/Ubuntu"
echo " brew install ripgrep # macOS"
echo " cargo install ripgrep # With Rust (no sudo)"
fi
echo " https://github.com/BurntSushi/ripgrep#installation"
fi
fi
fi
# ============================================================================
# Environment file
# ============================================================================
if [ ! -f ".env" ]; then
if [ -f ".env.example" ]; then
cp .env.example .env
echo -e "${GREEN}${NC} Created .env from template"
fi
else
echo -e "${GREEN}${NC} .env exists"
fi
# ============================================================================
# PATH setup — symlink hermes into a user-facing bin dir
# ============================================================================
echo -e "${CYAN}${NC} Setting up hermes command..."
HERMES_BIN="$SCRIPT_DIR/venv/bin/hermes"
COMMAND_LINK_DIR="$(get_command_link_dir)"
COMMAND_LINK_DISPLAY_DIR="$(get_command_link_display_dir)"
mkdir -p "$COMMAND_LINK_DIR"
ln -sf "$HERMES_BIN" "$COMMAND_LINK_DIR/hermes"
echo -e "${GREEN}${NC} Symlinked hermes → $COMMAND_LINK_DISPLAY_DIR/hermes"
if is_termux; then
export PATH="$COMMAND_LINK_DIR:$PATH"
echo -e "${GREEN}${NC} $COMMAND_LINK_DISPLAY_DIR is already on PATH in Termux"
else
# Determine the appropriate shell config file
SHELL_CONFIG=""
if [[ "$SHELL" == *"zsh"* ]]; then
SHELL_CONFIG="$HOME/.zshrc"
elif [[ "$SHELL" == *"bash"* ]]; then
SHELL_CONFIG="$HOME/.bashrc"
[ ! -f "$SHELL_CONFIG" ] && SHELL_CONFIG="$HOME/.bash_profile"
else
# Fallback to checking existing files
if [ -f "$HOME/.zshrc" ]; then
SHELL_CONFIG="$HOME/.zshrc"
elif [ -f "$HOME/.bashrc" ]; then
SHELL_CONFIG="$HOME/.bashrc"
elif [ -f "$HOME/.bash_profile" ]; then
SHELL_CONFIG="$HOME/.bash_profile"
fi
fi
if [ -n "$SHELL_CONFIG" ]; then
# Touch the file just in case it doesn't exist yet but was selected
touch "$SHELL_CONFIG" 2>/dev/null || true
if ! echo "$PATH" | tr ':' '\n' | grep -q "^$HOME/.local/bin$"; then
if ! grep -q '\.local/bin' "$SHELL_CONFIG" 2>/dev/null; then
echo "" >> "$SHELL_CONFIG"
echo "# Hermes Agent — ensure ~/.local/bin is on PATH" >> "$SHELL_CONFIG"
echo 'export PATH="$HOME/.local/bin:$PATH"' >> "$SHELL_CONFIG"
echo -e "${GREEN}${NC} Added ~/.local/bin to PATH in $SHELL_CONFIG"
else
echo -e "${GREEN}${NC} ~/.local/bin already in $SHELL_CONFIG"
fi
else
echo -e "${GREEN}${NC} ~/.local/bin already on PATH"
fi
fi
fi
# ============================================================================
# Seed bundled skills into ~/.hermes/skills/
# ============================================================================
HERMES_SKILLS_DIR="${HERMES_HOME:-$HOME/.hermes}/skills"
mkdir -p "$HERMES_SKILLS_DIR"
echo ""
echo "Syncing bundled skills to ~/.hermes/skills/ ..."
if "$SCRIPT_DIR/venv/bin/python" "$SCRIPT_DIR/tools/skills_sync.py" 2>/dev/null; then
echo -e "${GREEN}${NC} Skills synced"
else
# Fallback: copy if sync script fails (missing deps, etc.)
if [ -d "$SCRIPT_DIR/skills" ]; then
cp -rn "$SCRIPT_DIR/skills/"* "$HERMES_SKILLS_DIR/" 2>/dev/null || true
echo -e "${GREEN}${NC} Skills copied"
fi
fi
# ============================================================================
# Done
# ============================================================================
echo ""
echo -e "${GREEN}✓ Setup complete!${NC}"
echo ""
echo "Next steps:"
echo ""
if is_termux; then
echo " 1. Run the setup wizard to configure API keys:"
echo " hermes setup"
echo ""
echo " 2. Start chatting:"
echo " hermes"
echo ""
else
echo " 1. Reload your shell:"
echo " source $SHELL_CONFIG"
echo ""
echo " 2. Run the setup wizard to configure API keys:"
echo " hermes setup"
echo ""
echo " 3. Start chatting:"
echo " hermes"
echo ""
fi
echo "Other commands:"
echo " hermes status # Check configuration"
if is_termux; then
echo " hermes gateway # Run gateway in foreground"
else
echo " hermes gateway install # Install gateway service (messaging + cron)"
fi
echo " hermes cron list # View scheduled jobs"
echo " hermes doctor # Diagnose issues"
echo ""
# Ask if they want to run setup wizard now
read -p "Would you like to run the setup wizard now? [Y/n] " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]] || [[ -z $REPLY ]]; then
echo ""
# Run directly with venv Python (no activation needed)
"$SCRIPT_DIR/venv/bin/python" -m hermes_cli.main setup
fi