24e9ed73c2
Cron jobs run through cron/scheduler.py's own ThreadPoolExecutor via a
standalone AIAgent (run_job/run_one_job), entirely outside
GatewayRunner._running_agents -- the dict _drain_active_agents() and
every other active-work check on that class reads. A gateway shutdown
(/update, /restart, and SIGUSR1 all funnel through the same stop())
could log active_at_start=0 and immediately kill tool subprocesses
while a cron job's terminal command was still running, with no wait
and no indication anything was interrupted.
Real-world impact (from the issue): a scheduled daily briefing cron
job was in flight during /update, its tool subprocess got killed
by the unconditional shutdown cleanup, and the job was never marked
failed -- it simply never completed or delivered, with no error
surfaced anywhere. A repro with a 30-minute `sleep` cron job in flight
during /update reproduced the same pattern: subprocess killed at
+0.22s of drain (active_at_start=0), the job's agent thread continued
in-process and produced a plausible-looking final response from the
truncated tool output, and the scheduler marked the run successful.
Root cause is layered, not a single line:
1. GatewayRunner._drain_active_agents() only waits on _running_agents.
Cron work was invisible to it, so drain returned instantly whenever
the only active work was a cron job.
2. Even with visibility, the shutdown's final tool-subprocess kill
(process_registry.kill_all()) is a global, unconditional sweep with
no per-job targeting -- a long-running cron job that outlives the
drain timeout still gets its subprocess killed.
3. cron/scheduler.py had no way to detect that a job's tool subprocess
was killed out from under it mid-run; the agent thread kept going
and its eventual (often degraded but plausible-looking) response
got reported as a normal successful completion.
Fix, three parts:
- cron/scheduler.py: expose get_running_job_ids() (thread-safe
snapshot of the existing _running_job_ids set, already used to
prevent double-dispatch) so the gateway can read cron's in-flight
state without reaching into private module internals.
- gateway/run.py: GatewayRunner._active_cron_job_count() reads that
snapshot. _drain_active_agents() now waits on
(_running_agents OR active cron jobs), so a cron-only workload gets
the same bounded wait chat sessions already get instead of an
instant active_at_start=0. Shutdown drain logging gains
cron_active_at_start/cron_active_now fields alongside the existing
ones (unchanged, for compat).
- cron/scheduler.py: mark_running_jobs_interrupted(reason), called by
gateway/run.py's _kill_tool_subprocesses() right after
process_registry.kill_all(), marks every job still in
_running_job_ids at that instant as failed/interrupted via the
existing mark_job_run() -- and records the job IDs in
_interrupted_job_ids BEFORE writing, so run_one_job()'s own
eventual completion for the same run (racing in its own thread)
checks that flag and skips its normal write instead of clobbering
the interrupted status with a false "ok" produced from the
now-truncated tool output. This does not attempt to correlate a
killed PID to a specific job ID (process_registry tracks PIDs, not
job IDs) -- any job still dispatched at the moment of a forced kill
is treated as interrupted, matching the existing coarser precedent
set by _interrupt_running_agents(), which interrupts every entry in
_running_agents on a drain timeout without per-agent correlation
either.
Deliberately out of scope (flagged in the issue as a separate,
lower-priority concern): startup-time reconciliation of cron runs that
started but never reached a terminal status.
Testing:
- tests/cron/test_shutdown_interrupt.py (12 tests): get_running_job_ids
snapshot semantics, mark_running_jobs_interrupted marking/no-op/
partial-failure behavior, and -- the core race guard -- run_one_job
skipping its own last_status write (both the success path and the
exception path) when the shutdown path already marked the run
interrupted, with a control test proving ordinary un-interrupted
completions are unaffected.
- tests/gateway/test_cron_active_work_drain.py (9 tests):
_active_cron_job_count reading cron state and failing closed (0) if
the cron module is unavailable; _drain_active_agents waiting for an
in-flight cron job the same way it waits for chat sessions, timing
out if the job outruns the window, and leaving existing chat-session
drain behavior unchanged; a full runner.stop() integration test
(drain-timeout path) proving mark_running_jobs_interrupted actually
fires with the right job ID when a tool subprocess is force-killed,
plus a no-op control when nothing cron-related is in flight.
- tests/gateway/test_shutdown_cache_cleanup.py: added
_active_cron_job_count() to that file's hand-rolled _FakeGateway test
double, which stop() now calls -- without it those 8 pre-existing
tests AttributeError (caught by fail-then-pass below, not a
production bug).
Fail-then-pass: reverted gateway/run.py + cron/scheduler.py, all 21
new tests fail (fixture/attribute errors -- the feature doesn't exist
yet); restored, all 21 pass.
Regression check: ran the full plausibly-affected surface --
tests/gateway/{test_gateway_shutdown,test_restart_drain,
test_restart_notification,test_restart_redelivery_dedup,
test_restart_resume_pending,test_restart_service_detection,
test_shutdown_cache_cleanup,test_stuck_loop,test_clean_shutdown_marker,
test_external_drain_control,test_session_state_cleanup,
test_update_command,test_update_streaming}.py plus tests/cron/ (944
tests) -- against a clean upstream/main checkout and against this
branch. Diffed the two FAILED lists: identical, 20 pre-existing
failures on both sides (Windows-locale/cp1252 file-encoding issues and
Unix-permission-bit assertions that don't apply on this Windows dev
box), zero new failures, zero fixed-by-accident. The 8
test_shutdown_cache_cleanup.py failures found mid-development were
from the _FakeGateway gap above, fixed in the same commit and
confirmed clean on the final rerun (diff against baseline: exit 0).
Fixes #60432
|
||
|---|---|---|
| .github | ||
| .plans | ||
| acp_adapter | ||
| acp_registry | ||
| agent | ||
| apps | ||
| assets | ||
| cron | ||
| datagen-config-examples | ||
| docker | ||
| docs | ||
| gateway | ||
| hermes_cli | ||
| infographic | ||
| locales | ||
| nix | ||
| optional-mcps | ||
| optional-skills | ||
| packaging/homebrew | ||
| plugins | ||
| providers | ||
| scripts | ||
| skills | ||
| tests | ||
| tools | ||
| tui_gateway | ||
| ui-tui | ||
| web | ||
| website | ||
| .dockerignore | ||
| .env.example | ||
| .envrc | ||
| .gitattributes | ||
| .gitignore | ||
| .hadolint.yaml | ||
| .mailmap | ||
| AGENTS.md | ||
| batch_runner.py | ||
| cli-config.yaml.example | ||
| cli.py | ||
| constraints-termux.txt | ||
| CONTRIBUTING.es.md | ||
| CONTRIBUTING.md | ||
| docker-compose.windows.yml | ||
| docker-compose.yml | ||
| Dockerfile | ||
| flake.lock | ||
| flake.nix | ||
| hermes | ||
| hermes-already-has-routines.md | ||
| hermes_bootstrap.py | ||
| hermes_constants.py | ||
| hermes_logging.py | ||
| hermes_state.py | ||
| hermes_time.py | ||
| LICENSE | ||
| MANIFEST.in | ||
| mcp_serve.py | ||
| mini_swe_runner.py | ||
| model_tools.py | ||
| package-lock.json | ||
| package.json | ||
| pyproject.toml | ||
| README.es.md | ||
| README.md | ||
| README.ur-pk.md | ||
| README.zh-CN.md | ||
| run_agent.py | ||
| SECURITY.es.md | ||
| SECURITY.md | ||
| setup-hermes.sh | ||
| setup.py | ||
| toolset_distributions.py | ||
| toolsets.py | ||
| trajectory_compressor.py | ||
| utils.py | ||
| uv.lock | ||
Hermes Agent ☤
The self-improving AI agent built by Nous Research. It's the only agent with a built-in learning loop — it creates skills from experience, improves them during use, nudges itself to persist knowledge, searches its own past conversations, and builds a deepening model of who you are across sessions. Run it on a $5 VPS, a GPU cluster, or serverless infrastructure that costs nearly nothing when idle. It's not tied to your laptop — talk to it from Telegram while it works on a cloud VM.
Use any model you want — Nous Portal, OpenRouter, OpenAI, your own endpoint, and many others. Switch with hermes model — no code changes, no lock-in.
| A real terminal interface | Full TUI with multiline editing, slash-command autocomplete, conversation history, interrupt-and-redirect, and streaming tool output. |
| Lives where you do | Telegram, Discord, Slack, WhatsApp, Signal, and CLI — all from a single gateway process. Voice memo transcription, cross-platform conversation continuity. |
| A closed learning loop | Agent-curated memory with periodic nudges. Autonomous skill creation after complex tasks. Skills self-improve during use. FTS5 session search with LLM summarization for cross-session recall. Honcho dialectic user modeling. Compatible with the agentskills.io open standard. |
| Scheduled automations | Built-in cron scheduler with delivery to any platform. Daily reports, nightly backups, weekly audits — all in natural language, running unattended. |
| Delegates and parallelizes | Spawn isolated subagents for parallel workstreams. Write Python scripts that call tools via RPC, collapsing multi-step pipelines into zero-context-cost turns. |
| Runs anywhere, not just your laptop | Six terminal backends — local, Docker, SSH, Singularity, Modal, and Daytona. Daytona and Modal offer serverless persistence — your agent's environment hibernates when idle and wakes on demand, costing nearly nothing between sessions. Run it on a $5 VPS or a GPU cluster. |
| Research-ready | Batch trajectory generation, trajectory compression for training the next generation of tool-calling models. |
Quick Install
Linux, macOS, WSL2, Termux
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
Windows (native, PowerShell)
Heads up: Native Windows runs Hermes without WSL — CLI, gateway, TUI, and tools all work natively. If you'd rather use WSL2, the Linux/macOS one-liner above works there too. Found a bug? Please file issues.
Run this in PowerShell:
iex (irm https://hermes-agent.nousresearch.com/install.ps1)
The installer handles everything: uv, Python 3.11, Node.js, ripgrep, ffmpeg, and a portable Git Bash (MinGit, unpacked to %LOCALAPPDATA%\hermes\git — no admin required, completely isolated from any system Git install). Hermes uses this bundled Git Bash to run shell commands.
If you already have Git installed, the installer detects it and uses that instead. Otherwise a ~45MB MinGit download is all you need — it won't touch or interfere with any system Git.
Android / Termux: The tested manual path is documented in the Termux guide. On Termux, Hermes installs a curated
.[termux]extra because the full.[all]extra currently pulls Android-incompatible voice dependencies.Windows: Native Windows is fully supported — the PowerShell one-liner above installs everything. If you'd rather use WSL2, the Linux command works there too. Native Windows install lives under
%LOCALAPPDATA%\hermes; WSL2 installs under~/.hermesas on Linux.
After installation:
source ~/.bashrc # reload shell (or: source ~/.zshrc)
hermes # start chatting!
Troubleshooting
Windows Defender or antivirus flags uv.exe as malware
If your antivirus (Bitdefender, Windows Defender, etc.) quarantines uv.exe from the Hermes bin folder (%LOCALAPPDATA%\hermes\bin\uv.exe), this is a false positive. The file is Astral's uv — the Rust Python package manager Hermes bundles to manage its Python environment. ML-based antivirus engines commonly flag unsigned Rust binaries that download and install packages.
To verify your copy is authentic:
# Install GitHub CLI if needed
winget install --id GitHub.cli
# Login to GitHub
gh auth login
# Run verification
$uv = "$env:LOCALAPPDATA\hermes\bin\uv.exe"
$ver = (& $uv --version).Split(' ')[1]
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$zip = "$env:TEMP\uv.zip"
Invoke-WebRequest "https://github.com/astral-sh/uv/releases/download/$ver/uv-x86_64-pc-windows-msvc.zip" -OutFile $zip -UseBasicParsing
gh attestation verify $zip --repo astral-sh/uv
Expand-Archive $zip "$env:TEMP\uv_x" -Force
(Get-FileHash "$env:TEMP\uv_x\uv.exe").Hash -eq (Get-FileHash $uv).Hash
If attestation says "Verification succeeded" and the last line prints True, you're good.
To whitelist Hermes:
- Windows Defender: Run PowerShell as Admin →
Add-MpPreference -ExclusionPath "$env:LOCALAPPDATA\hermes\bin" - Bitdefender: Add an exception in the Bitdefender console (Protection > Antivirus > Settings > Manage Exceptions)
- Whitelist the folder, not the file hash — Hermes updates
uvand the hash changes every version
For more context, see the upstream Astral reports: astral-sh/uv#13553, astral-sh/uv#15011, astral-sh/uv#10079.
Getting Started
hermes # Interactive CLI — start a conversation
hermes model # Choose your LLM provider and model
hermes tools # Configure which tools are enabled
hermes config set # Set individual config values
hermes gateway # Start the messaging gateway (Telegram, Discord, etc.)
hermes setup # Run the full setup wizard (configures everything at once)
hermes claw migrate # Migrate from OpenClaw (if coming from OpenClaw)
hermes update # Update to the latest version
hermes doctor # Diagnose any issues
Skip the API-key collection — Nous Portal
Hermes works with whatever provider you want — that's not changing. But if you'd rather not collect five separate API keys for the model, web search, image generation, TTS, and a cloud browser, Nous Portal covers all of them under one subscription:
- 300+ models — pick any of them with
/model <name> - Tool Gateway — web search (Firecrawl), image generation (FAL), text-to-speech (OpenAI), cloud browser (Browser Use), all routed through your sub. No extra accounts.
One command from a fresh install:
hermes setup --portal
That logs you in via OAuth, sets Nous as your provider, and turns on the Tool Gateway. Check what's wired up any time with hermes portal info. Full details on the Tool Gateway docs page.
You can still bring your own keys per-tool whenever you want — the gateway is per-backend, not all-or-nothing.
CLI vs Messaging Quick Reference
Hermes has two entry points: start the terminal UI with hermes, or run the gateway and talk to it from Telegram, Discord, Slack, WhatsApp, Signal, or Email. Once you're in a conversation, many slash commands are shared across both interfaces.
| Action | CLI | Messaging platforms |
|---|---|---|
| Start chatting | hermes |
Run hermes gateway setup + hermes gateway start, then send the bot a message |
| Start fresh conversation | /new or /reset |
/new or /reset |
| Change model | /model [provider:model] |
/model [provider:model] |
| Set a personality | /personality [name] |
/personality [name] |
| Retry or undo the last turn | /retry, /undo |
/retry, /undo |
| Compress context / check usage | /compress, /usage, /insights [--days N] |
/compress, /usage, /insights [days] |
| Browse skills | /skills or /<skill-name> |
/<skill-name> |
| Interrupt current work | Ctrl+C or send a new message |
/stop or send a new message |
| Platform-specific status | /platforms |
/status, /sethome |
For the full command lists, see the CLI guide and the Messaging Gateway guide.
Documentation
All documentation lives at hermes-agent.nousresearch.com/docs:
| Section | What's Covered |
|---|---|
| Quickstart | Install → setup → first conversation in 2 minutes |
| CLI Usage | Commands, keybindings, personalities, sessions |
| Configuration | Config file, providers, models, all options |
| Messaging Gateway | Telegram, Discord, Slack, WhatsApp, Signal, Home Assistant |
| Security | Command approval, DM pairing, container isolation |
| Tools & Toolsets | 40+ tools, toolset system, terminal backends |
| Skills System | Procedural memory, Skills Hub, creating skills |
| Memory | Persistent memory, user profiles, best practices |
| MCP Integration | Connect any MCP server for extended capabilities |
| Cron Scheduling | Scheduled tasks with platform delivery |
| Context Files | Project context that shapes every conversation |
| Architecture | Project structure, agent loop, key classes |
| Contributing | Development setup, PR process, code style |
| CLI Reference | All commands and flags |
| Environment Variables | Complete env var reference |
Migrating from OpenClaw
If you're coming from OpenClaw, Hermes can automatically import your settings, memories, skills, and API keys.
During first-time setup: The setup wizard (hermes setup) automatically detects ~/.openclaw and offers to migrate before configuration begins.
Anytime after install:
hermes claw migrate # Interactive migration (full preset)
hermes claw migrate --dry-run # Preview what would be migrated
hermes claw migrate --preset user-data # Migrate without secrets
hermes claw migrate --overwrite # Overwrite existing conflicts
What gets imported:
- SOUL.md — persona file
- Memories — MEMORY.md and USER.md entries
- Skills — user-created skills →
~/.hermes/skills/openclaw-imports/ - Command allowlist — approval patterns
- Messaging settings — platform configs, allowed users, working directory
- API keys — allowlisted secrets (Telegram, OpenRouter, OpenAI, Anthropic, ElevenLabs)
- TTS assets — workspace audio files
- Workspace instructions — AGENTS.md (with
--workspace-target)
See hermes claw migrate --help for all options, or use the openclaw-migration skill for an interactive agent-guided migration with dry-run previews.
Contributing
We welcome contributions! See the Contributing Guide for development setup, code style, and PR process.
Quick start for contributors — use the standard installer, then work from the
full git checkout it creates at $HERMES_HOME/hermes-agent (usually
~/.hermes/hermes-agent). This matches the layout used by hermes update, the
managed venv, lazy dependencies, gateway, and docs tooling.
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
cd "${HERMES_HOME:-$HOME/.hermes}/hermes-agent"
uv pip install -e ".[all,dev]"
scripts/run_tests.sh
Manual clone fallback (for throwaway clones/CI where you intentionally do not want the managed install layout):
Create the venv outside the cloned source tree — a venv inside the directory the agent operates from can be wiped by a relative-path command the agent runs against its own checkout, destroying the running runtime mid-session.
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv ~/.hermes/venvs/hermes-dev --python 3.11
source ~/.hermes/venvs/hermes-dev/bin/activate
uv pip install -e ".[all,dev]"
scripts/run_tests.sh
Community
- 💬 Discord
- 📚 Skills Hub
- 🐛 Issues
- 🔌 computer-use-linux — Linux desktop-control MCP server for Hermes and other MCP hosts, with AT-SPI accessibility trees, Wayland/X11 input, screenshots, and compositor window targeting.
- 🔌 HermesClaw — Community WeChat bridge: Run Hermes Agent and OpenClaw on the same WeChat account.
License
MIT — see LICENSE.
Built by Nous Research.