Addresses GodsBoy's May 30 follow-up review on PR #30179.
P0 — Linux sandboxes could not reach the proxy:
- _default_http_listen now binds the docker bridge gateway on Linux
(host.docker.internal resolves to the bridge gateway there; the old
loopback-only bind was unreachable from containers). Loopback stays
the default on Docker Desktop platforms; bridge-less Linux falls back
to loopback with a warning.
- Live-testing the fix against the real v0.39 binary surfaced a second
latent bug the host-side E2E had masked: v0.39's http_listen does NOT
terminate CONNECT — tunnel_listen does. HTTPS_PROXY traffic through
http_listen got 400s from upstream. build_proxy_config now binds
tunnel_listen (CONNECT/MITM) on tunnel_port and http_listen
(plain-HTTP forwards) on tunnel_port+1; docker.py points HTTPS_PROXY
at tunnel_port and HTTP_PROXY at tunnel_port+1.
- Liveness probes (start_proxy poll loop, get_status) now probe the
CONFIGURED bind host via _read_http_listen_from_config() instead of
hardcoded loopback, which would have killed a healthy bridge-bound
daemon as 'never came up'.
P2 — allow_env_fallback dead on the partial-secret path:
- The missing-secret branch in _build_proxy_subprocess_env now honors
proxy.allow_env_fallback exactly as its own error message promises.
- cmd_start refuses (or warns, with the fallback flag) when
credential_source=bitwarden but secrets.bitwarden is disabled/missing
— closing the silent-degrade-to-host-env hole.
P2 — audit.log decoy on v0.39:
- Wizard pre-create failure downgraded to a warning (file is
non-load-bearing until the version bump); success line qualified as
'reserved'; user + dev docs stop telling operators to wire monitoring
to the path today.
P3 — metrics comment no longer claims 'hermes egress status' surfaces
the ephemeral metrics port (it can't; :0 is random and unrecorded).
Validation: 180 unit tests pass (9 new), gated E2E passes, and a live
end-to-end run against the real binary verified CONNECT-MITM with
Authorization swap on tunnel_listen, plain-HTTP swap on tunnel_port+1,
403 on non-allowlisted hosts, and bind-host-aware status probing.
Docs gain a Linux firewall troubleshooting section (container→docker0
INPUT drops, e.g. ufw default-deny).
When Hermes runs in TUI mode, the gateway child process communicates with
the Node.js parent over a JSON-RPC protocol on stdin. Subprocess calls that
inherit this stdin fd can trigger a race condition where the child's stdin
read returns EOF, causing the gateway to exit cleanly (exit code 0) mid-tool-
execution.
This is the same root cause as issue #14036 (byterover plugin) and PR #39257
(SSH environment backend). This commit applies the fix — stdin=subprocess.DEVNULL
— to all 85 subprocess.run() and subprocess.Popen() calls that execute inside
the TUI gateway child process.
Scope: TUI-context code only (agent/, tools/, plugins/, tui_gateway/server.py).
CLI code (cli.py, hermes_cli/), tests, scripts, and gateway process management
are excluded — they don't run inside the TUI child and inherit the terminal's
stdin, not the JSON-RPC pipe.
85 call sites across 28 files. All files pass syntax check.
Salvage of #36631 (@annguyenNous), rebased onto current main with
regression tests added. Fixes#36266.
When a persistent Docker sandbox container is removed out-of-band (idle
reaper, `docker prune`, OOM kill, daemon restart), the gateway kept
issuing `docker exec` against the dead container ID, returning
"No such container" on every subsequent tool call — the agent was
permanently blocked until the gateway process restarted.
DockerEnvironment.execute() now detects the "No such container" /
"is not running" error after a non-zero exit (gated on
persist_across_processes) and calls _recreate_container(): it tries
label-based reuse first, falls back to a fresh container replaying the
same image + full all_run_args set, re-runs init_session(), and retries
the command once. A genuine non-zero exit is NOT misclassified as
container-gone.
Differs from #36631 as submitted: adds the tests the original lacked.
tests/tools/test_docker_environment.py covers _is_container_gone pattern
matching (incl. the negative/control case), the recover-and-retry path,
the persist_across_processes=False opt-out (no recovery), and the
ordinary-failure passthrough (no spurious recreation). _make_dummy_env
now forwards persist_across_processes.
Verified:
- Unit: 67/67 in test_docker_environment.py (4 new + existing).
- Live E2E against the real docker daemon: started a persistent
container, `docker rm -f`'d it out-of-band, and the next execute()
transparently recreated a fresh container and succeeded; a follow-up
command worked in the recovered container; a real `exit N` passed
through without triggering recovery.
Co-authored-by: annguyenNous <annguyenNous@users.noreply.github.com>
When `docker run -d` fails after Docker has already created the container
object (e.g. exit 125 when the daemon isn't ready, or a timeout mid image
pull), the code raised before `self._container_id` was set — so the
container leaked permanently in "Created" state. Reported in #7439:
110+ orphaned containers accumulated over 3 days from hourly cron-
scheduled gateway sessions hitting a Docker Desktop startup race.
The orphan reaper added in #33645 (reap_orphan_containers) does NOT cover
this case: it filters `status=exited`, but a failed-create container is in
`Created` state, so it slips through and is never reaped.
Wrap the `docker run -d` call in try/except and `docker rm -f` the
container by its known name before re-raising.
Salvages #7440 by @Tranquil-Flow. Their branch predated the cross-process
reuse + labels rework on `main`, so a cherry-pick conflicted; reconstructed
the same intent (plus their two regression tests, adapted to mock the new
reuse `docker ps` probe) against current `main`.
Verified adversarially: reverted just the product change to origin/main's
`docker.py`, ran the two new tests -> both FAIL with
`assert 0 == 1 ("docker rm should be called once")`. With the fix applied,
both pass; full test_docker_environment.py is 65/65 green.
Closes#7440. Fixes#7439.
Co-authored-by: Evi Nova <66773372+Tranquil-Flow@users.noreply.github.com>
s6-overlay images (e.g. hermes-agent:latest) use /init as PID 1 and exec
/run/s6/basedir/bin/init during stage0 startup. The Docker terminal backend
unconditionally added Docker --init and mounted /run as noexec, which broke
those images in two ways: --init created a second competing PID-1 init, and
the noexec /run made s6 stage0 fail with "exec: /run/s6/basedir/bin/init:
Permission denied" (exit 126), so the container died and terminal commands
reported a generic "container is not running" error.
Detect images whose entrypoint is /init via 'docker image inspect' and, for
those images only, skip Docker --init and mount /run with exec. All other
images keep the hardened --init + noexec defaults. Detection is best-effort:
any inspect failure falls back to the safe defaults.
The docker_forward_env build loop only consulted the ~/.hermes/.env disk
fallback when a key was unset (value is None), not when it was present
but empty (""). A transient empty value in os.environ was therefore
forwarded into the sandbox container as `-e KEY=`, clobbering the correct
value on disk. Sandboxed workloads then read a zero-length secret and
failed auth (observed as intermittent Linear API 401s) with no gateway
restart and no .env rewrite.
Treat empty-string like unset (`if not value:` on the fallback) and never
forward a blank secret (`if value:` on the guard).
Fixes#35580
Salvages #24490 by @liuhao1024 against current main.
The Docker daemon will silently auto-create a directory at the host
path of any `-v <host>:<container>` bind mount when the host path
doesn't exist. In Docker-in-Docker setups (where the outer host's
real credential file isn't visible inside the agent's parent
container), this leaves a directory at the credential mount source —
and the inner `docker run` then refuses to mount a directory over a
file destination with exit 125.
Add defensive shape guards to all three mount loops in
DockerEnvironment.__init__:
* credentials (expected: file) — skip + warn on directory or missing
* skills (expected: dir) — skip + warn when not a directory
* cache (expected: dir) — skip + warn when not a directory
Failed mounts surface as WARN logs rather than crashing the container
start. Existing well-formed sources mount unchanged.
The original PR's branch was on a pre-container-reuse-rework base
(May 12) and conflicted with the post-May-28 driver work (label
tagging, container reuse, orphan reaper). Reconstructed the same
intent on current main; the three guard blocks slot cleanly into
`tools/environments/docker.py` around the existing mount loops.
Three new tests pinned in `tests/tools/test_docker_environment.py`:
directory-source skip, missing-source skip, valid-file mounts. Test-
first regression verification: reverted just the production code to
`origin/main` and confirmed the new tests fail with
`'deleted_token.json' is contained here: /root/.hermes/...` — the
fixed code makes them pass. Full file passes (54/54).
Closes#24490
Co-authored-by: liuhao1024 <11816344+liuhao1024@users.noreply.github.com>
The first iteration of this PR did docker stop on every cleanup in
persist mode (only skipping docker rm). Ben caught this as
contradicting the documented "ONE long-lived container shared across
sessions" semantics: stopping the container on every Hermes /quit kills
any background processes inside (npm watchers, pytest watchers,
long-running scripts) — exactly the case persist mode is supposed to
protect.
This commit splits the cleanup paths cleanly:
* **Persist mode (default)** — cleanup() is a NO-OP for the
container. Container stays running, processes survive, next Hermes
process attaches via the existing label probe in ~ms instead of
waiting for docker start. Resource reclamation happens via the
orphan reaper at next startup (2 × lifetime_seconds threshold), which
covers the SIGKILL / OOM / abandoned-laptop cases.
* **Opt-out mode (persist_across_processes=False)** — unchanged:
docker stop + docker rm -f on cleanup as before.
* **Explicit teardown** — new cleanup(force_remove=True) kwarg
overrides persist mode and tears the container down unconditionally.
cleanup_vm(task_id) now defaults to force_remove=True since
it's the user-driven reset path (called from AIAgent.close(),
/reset-style flows, and the idle reaper's per-turn cleanup).
The idle reaper in _cleanup_inactive_envs calls env.cleanup()
directly with no kwargs, so idle persist-mode envs are no-op'd — the
container survives the in-process pop and the next tool call re-probes
via labels. No state leak: _container_id is still cleared on the
in-process handle.
E2E verified against real Docker:
✓ Container is still running after cleanup()
✓ Background process (sleep loop) survived cleanup()
✓ Filesystem state preserved across cleanup()
✓ In-process container_id cleared (next __init__ will re-probe)
✓ Background process visible from reused env (no docker start happened)
✓ force_remove=True removed the container even in persist mode
✓ cleanup_vm() removed the container (defaults to force_remove=True)
Test changes:
* Replaces `test_cleanup_with_persist_only_stops_no_rm` with
`test_cleanup_with_persist_is_noop_for_container` — asserts neither
stop nor rm runs in persist mode, and the in-process handle is
cleared so re-probe works.
* Adds `test_cleanup_force_remove_stops_and_rms_even_in_persist_mode`
— covers the new kwarg.
* Updates `test_cleanup_uses_subprocess_run_not_detached_shell` and
`test_wait_for_cleanup_after_cleanup_returns_true` to pass
`force_remove=True` so they actually exercise the docker code path
(default no-op would trivially pass).
cleanup_vm() forwards `force_remove` only to backends whose cleanup()
accepts the kwarg (currently just DockerEnvironment) via runtime
signature inspection — Modal/Daytona/SSH `cleanup()` signatures are
unchanged.
Refs #20561
The cleanup-fix in the previous commit handles the graceful-exit leak: a
Hermes process that runs ``atexit`` will now actually wait on the docker
stop/rm worker thread, so containers either survive (persist mode) or are
fully removed (opt-out mode) by the time the interpreter exits.
But ``atexit`` doesn't fire on SIGKILL, OOM-kill, or terminal-window
close. Containers from those exits stay parked with no surviving Python
process to reuse or remove them, so they accumulate until the operator
intervenes with ``docker rm -f``. The cleanup-fix doesn't help this class
— there's no live cleanup() to fix.
This commit adds the safety net: a startup orphan reaper that runs once
per Hermes process and removes long-Exited hermes-labeled containers
that the prior commit couldn't reach.
Implementation:
* New ``reap_orphan_containers()`` in ``tools/environments/docker.py``.
Filters: ``label=hermes-agent=1`` + ``status=exited`` + (optional)
``label=hermes-profile=<current>``. Per-container ``docker inspect``
parses ``State.FinishedAt`` (with nanosecond-precision trimming for
Python's microsecond-bound ``fromisoformat``); containers older than
the threshold get ``docker rm -f``'d. The ``status=exited`` filter is
load-bearing — a running container may belong to a sibling Hermes
process whose reuse path will pick it up; killing it would crash the
sibling mid-command. Single-container failures are logged and the
sweep continues to the next candidate.
* New ``_maybe_reap_docker_orphans()`` helper in
``tools/terminal_tool.py``. Wired into ``_create_environment()`` for
``env_type == "docker"``. Gated by:
- ``terminal.docker_orphan_reaper: true`` (default; opt-out for
operators running multiple Hermes processes in the same profile
who don't trust the conservative defaults)
- ``_docker_orphan_reaper_ran`` module flag with double-checked
locking — parallel subagents and RL rollouts don't trigger N
concurrent docker ps storms
- Age threshold = ``2 × TERMINAL_LIFETIME_SECONDS`` with a 60s floor
(so ``TERMINAL_LIFETIME_SECONDS=0`` doesn't race the user's own
setup)
- Profile scoping — a research profile NEVER reaps the default
profile's stragglers
- Exception swallow — a janitor failure must never block container
creation
* New config ``terminal.docker_orphan_reaper`` wired through all four
config-bridge sites (cli.py, gateway/run.py, hermes_cli/config.py,
tests/conftest.py) and pinned by
``test_docker_orphan_reaper_is_bridged_everywhere``.
Coverage:
* 9 new unit tests in test_docker_environment.py — happy path, recent-
container sparing, profile scoping, unparseable-timestamp safety,
docker-ps-failure handling, partial-failure continuation, nanosecond
timestamp parsing, zero-value FinishedAt rejection.
* 6 new integration tests in test_docker_orphan_reaper_integration.py
— once-per-process gate, disable-flag respected, lifetime doubling
with 60s floor, current-profile filter wiring, exception swallow.
* 1 new bridge-invariant regression test.
Closes#20561 (combined with the two prior commits on this branch).
The Docker backend docs claim "Single persistent container — ONE long-
lived container shared across sessions, /new, /reset, and delegate_task
subagents. Stopped/removed on shutdown." In practice the code only
honored that contract within a single Python process via the in-memory
\`_active_environments[task_id]\` cache. Every \`hermes chat\` invocation
spawned a fresh \`hermes-<hex>\` container; older containers piled up in
\`Exited\` state and accumulated until manual \`docker rm\` (issue #20561).
Three root causes, all addressed by this commit:
1. No cross-process container discovery.
2. \`cleanup()\` used fire-and-forget \`subprocess.Popen("... &", shell=True)\`
which raced with parent-process exit — when Python exited promptly the
detached shell child got killed mid-\`docker stop\`, leaving stopped
containers behind.
3. The \`docker rm\` step in cleanup was gated on \`not self._persistent\`
(the bind-mount-persistence flag). Default config sets
\`container_persistent: true\`, so the default happy path skipped \`rm\`
entirely — even when the user explicitly didn't want cross-process
reuse, containers leaked.
Fix:
* Add \`DockerEnvironment.__init__(persist_across_processes=True)\`. When
true, init probes
\`docker ps -a --filter label=hermes-agent=1
--filter label=hermes-task-id=<task>
--filter label=hermes-profile=<profile>\`
and reuses a matching container (running → attach; stopped →
\`docker start\` → attach; \`docker start\` failure → fall through to a
fresh \`docker run\`). Multiple matches prefer the running one, with the
stragglers left for the orphan reaper (next commit) to clean up.
* Rewrite \`cleanup()\`. Uses \`subprocess.run(..., timeout=30)\` on a
daemon \`threading.Thread\`, not the racy \`Popen(... &)\`. The
\`_persistent\` guard is dropped on the \`rm\` step — \`rm\` now runs
whenever \`persist_across_processes\` is false, regardless of the
bind-mount-persistence setting. The leak class is gone in all
combinations.
* Add \`wait_for_cleanup(timeout)\`. \`tools/terminal_tool.py\`'s atexit
hook calls this on every active env, blocking up to 15s for the
cleanup thread before interpreter exit. Without this, \`hermes /quit\`
raced the daemon-thread teardown and dropped the stop/rm work.
* New config \`terminal.docker_persist_across_processes\` (default
\`true\` — restores the documented contract). Set \`false\` for hard
per-process isolation. Wired through all four config-bridge sites
(cli.py env_mappings, gateway/run.py _terminal_env_map,
hermes_cli/config.py _config_to_env_sync, tests/conftest.py env-strip
list); regression-pinned by
\`test_docker_persist_across_processes_is_bridged_everywhere\` matching
the existing pattern for docker_run_as_host_user / docker_env.
Reuse intentionally does NOT compare image / mounts / resources — only
the labels. Operators changing those settings should set
\`docker_persist_across_processes: false\` (or \`docker rm -f\` the
labeled container) to force a fresh start. This keeps the probe cheap
and the failure mode obvious.
Coverage: 12 new unit tests in tests/tools/test_docker_environment.py
covering reuse paths (running, stopped, fallback, opt-out, duplicate
preference) and cleanup behavior (persist-mode no-rm, opt-out always-rm,
no-Popen, wait_for_cleanup semantics, partial-init safety). Plus one
config-bridge regression pin.
Refs #20561
Issue #20561 (Docker containers accumulate) needs a way to identify
hermes-created containers from the outside — both for the orphan reaper
(a follow-up commit) and for operators triaging `docker ps -a | grep
hermes-` after a SIGKILL leaves stragglers. The previous `hermes-<hex>`
name prefix was the only signal, which broke down under cross-process
reuse (planned) and against any custom `--name` someone might pass via
`docker_extra_args`.
This commit adds three labels at `docker run` time:
--label hermes-agent=1 # global sweep target
--label hermes-task-id=<sanitized> # per-task reuse key
--label hermes-profile=<sanitized> # per-profile isolation key
Values are sanitized to `[A-Za-z0-9_.-]` and truncated to 63 chars so the
label round-trips cleanly through `docker ps --filter label=key=value`.
Empty or non-string inputs collapse to "unknown" rather than producing
an unqueryable empty value.
No behavior change: the labels are pure metadata. The follow-up commits
in this PR (cleanup-fix + orphan reaper) are what use them.
Refs #20561
Single content conflict in hermes_cli/config.py — kept BOTH the
paste_collapse_threshold knobs from main and the proxy section from
this branch (they're independent additions to DEFAULT_CONFIG).
All 187 tests in test_iron_proxy.py + test_iron_proxy_cli.py +
test_config.py pass post-merge.
The s6-overlay migration replaced every runtime use of gosu with
s6-setuidgid (in stage2-hook.sh, main-wrapper.sh, per-service run
scripts, and cont-init.d hooks), but the gosu binary itself was still
being copied into the image from tianon/gosu, and several comments
across the repo still pointed to it.
Image changes:
- Drop the FROM tianon/gosu:1.19-trixie AS gosu_source stage
- Drop the COPY --from=gosu_source /gosu /usr/local/bin/ layer
- Net: one fewer base-image pull, ~12-15 MB layer eliminated
Documentation/comment refresh (no behavior change):
- Dockerfile: update root-user rationale comment + cont-init.d comment
- docker/main-wrapper.sh: drop "pre-s6 contract (gosu drop)" reference
- docker-compose.yml: update UID/GID remap comment
- .hadolint.yaml: update DL3002 ignore rationale
- website/docs/user-guide/docker.md: privilege-drop helper is s6-setuidgid now
- hermes_cli/config.py: docker_run_as_host_user docstring
tools/environments/docker.py runs *arbitrary user images* via the
terminal backend, not the bundled Hermes image. It still needs SETUID/
SETGID caps so user images that use gosu/su/s6-setuidgid all work.
Renamed the cap-list constant _GOSU_CAP_ARGS → _PRIVDROP_CAP_ARGS and
updated comments to list s6-setuidgid alongside the others as examples.
The matching test (test_security_args_include_setuid_setgid_for_gosu_drop
→ test_security_args_include_setuid_setgid_for_privdrop) was renamed
and its docstring updated; behavior is unchanged.
Verification:
- hadolint clean against .hadolint.yaml
- shellcheck clean against all docker/ shell scripts
- Image rebuilt successfully (sha 1a090924ccea)
- Docker harness: 19 passed in 41.87s (every Phase 0 test + Phase 4
per-profile-gateway lifecycle + container-restart reconciliation)
- tests/tools/test_docker_environment.py: 23 passed (rename did not
break test discovery; pre-existing unrelated mock warning)
The plan document (docs/plans/2026-05-07-s6-overlay-dynamic-subagent-gateways.md)
intentionally retains its historical references to gosu — it describes
the pre-s6 entrypoint as background for understanding the migration.
GodsBoy 2nd-round P1 (all 4 addressed):
- _detect_docker_bridge_ip: replace `ip.count('.') == 3` heuristic with
ipaddress.IPv4Address validation + reject unspecified/loopback/multicast/
reserved/link-local/global addresses. Hostile `ip` shim on PATH used to
be able to inject 0.0.0.0 here and re-open INADDR_ANY binding.
- cmd_setup credential_source preservation: re-running `hermes egress
setup` without --from-bitwarden no longer silently downgrades a previous
bitwarden config back to env. Require --no-bitwarden to switch
explicitly; otherwise preserve the existing mode and surface the
decision.
- fail_on_uncovered_providers docstring/default mismatch: docstring used
to claim default=True; behavior was default=False. Resolved by
truth-in-advertising — docstring now correctly states default=False —
AND splitting providers into a strict LLM-specific tier
(_LLM_SPECIFIC_NON_BEARER_PROVIDERS, used by start blocking) and a
generic uncovered tier (used by wizard warnings). Generic cloud creds
(AWS_*, GOOGLE_APPLICATION_CREDENTIALS) no longer trip refuse-start
for operators using terraform/gcloud alongside Hermes. New
discover_blocked_providers() returns the strict subset.
- start_proxy poll-loop must verify listening before pidfile:
previously fell through deadline-expired as success and wrote a
pidfile for a non-listening daemon. Refactored into a do-while
shape, require `listening=True` for success, kill the child + unlink
the pidfile on failure paths.
GodsBoy 2nd-round P2 (the worth-keeping subset):
- O_NOFOLLOW + 0o600 + st_uid check on iron-proxy.log open (symmetric
with the pidfile and audit-log paths the same PR hardens).
- pidfile O_EXCL: refactored pidfile-write into _write_pidfile_safely
which uses O_EXCL to detect concurrent starts. EEXIST with a live
pid means "another start in progress" — refuse with actionable
message; EEXIST with a dead pid means "stale crash" — unlink and
retry once. Discriminates rather than racing.
- _VERSION_CACHE: invalidate on install_iron_proxy success;
don't cache empty stdout (would poison `hermes egress status` for
the lifetime of the process if first probe hit a corrupt binary).
- ensure_audit_log now RAISES on OSError instead of swallowing it as
a warning. Previous behavior let the daemon create the file under
the default umask, exactly the world-readable scenario the helper
was built to prevent. cmd_setup catches the new RuntimeError and
surfaces "✗" with the actionable message.
- SIGINT/SIGTERM handler scoped around the start_proxy poll loop:
Ctrl-C while waiting for `hermes egress start` no longer leaks an
orphan daemon with the port bound. Handler kills the child +
unlinks the pidfile before re-raising.
- pidfile written IMMEDIATELY after Popen, BEFORE the listening
verification. Parent dying during the poll loop now leaves a
pidfile pointing at the orphan so the next `hermes egress stop` can
clean up. Failure paths in the poll loop explicitly unlink.
- _DEFAULT_UPSTREAM_DENY_CIDRS: add ::ffff:0:0/96 (IPv4-mapped IPv6 —
closes the v6-resolved IMDS bypass), 100.64.0.0/10 (CGNAT / cloud
overlays / K8s pod networks), 198.18.0.0/15 (RFC2544 benchmark).
- _NON_BEARER_PROVIDERS split into LLM-specific (Anthropic / Azure /
Gemini — block when strict) vs generic-cloud (AWS_*, GCP appdefault
— warn-only).
- docker.py except narrowing: load_config can raise yaml.YAMLError on
a malformed config.yaml, not just ImportError. Two callsites
(collision check + precedence resolution) now catch yaml.YAMLError
via a sentinel `import yaml` and fail-safe to enforced mode.
GodsBoy 2nd-round P3:
- _reset_for_tests: was a no-op claiming symmetry with bitwarden;
now actually clears _VERSION_CACHE and _proxy_nonce so in-process
callers (notebooks, pytest -p no:xdist) don't see state leakage.
- tests/test_iron_proxy_cli.py: replaced hardcoded Path("/tmp/...")
with hermes_home/-derived fixtures. Matches the same cleanup we
did for test_iron_proxy.py in the previous round.
- --rotate-tokens confirmation gate: when there are existing tokens,
prompt for "rotate" confirmation (skipped when stdin isn't a tty
so CI/scripted use still works) AND back up the mappings to a
timestamped sibling before overwriting. Surface a no-op note when
rotate is requested with no existing tokens.
stephenschoettler (runtime-boundary review):
- #1 BWS silent degrade at proxy start: when credential_source=bitwarden
but the BWS access token or project_id is missing OR the fetch
returns no values for mapped providers, raise instead of silently
falling back to host env. cmd_start also pre-checks at the wizard
layer for actionable error messages. Opt-in escape hatch via new
`proxy.allow_env_fallback: true` config for migration scenarios.
- #2 docker_env collision detection extended: `docker_env:
{OPENROUTER_API_KEY: sk-real}` in config.yaml with enforce_on_docker:
true now raises just like an HTTPS_PROXY collision would. The
collision check pulls mapped provider names from load_mappings() at
call time.
- #3 PID nonce persisted to disk: cross-CLI-invocation stale-pidfile
defense now works. start_proxy writes the nonce next to the pidfile
(sibling 0o600), stop_proxy reads it back via _read_persisted_nonce()
and uses it as a _pid_alive signal in the new process. Falls back
to argv0 basename matching when the file is missing (legacy install).
arshkumarsingh:
- #1 NODE_OPTIONS append-merge: egress dict no longer sets NODE_OPTIONS
directly (would clobber the operator's --max-old-space-size etc.).
Carry the egress flag in a sentinel key
_HERMES_EGRESS_NODE_OPTIONS_APPEND; DockerEnvironment merges into the
existing NODE_OPTIONS in env_args computation with de-duplication.
- #2 docs: structured per-request audit log is at audit.log, not
iron-proxy.log (the latter is daemon stdout/stderr). Diagram and
step-7 text corrected; both file roles are now documented separately.
Tests
- Added 12 new tests in test_iron_proxy.py covering bridge-IP rejection
(parametrized over 8 dangerous inputs), default deny-list adjacency
(IPv4-mapped-v6 + CGNAT), blocked-providers strict-subset property,
_pid_proc_starttime parser with paren-containing comm,
stop_proxy SIGKILL suppression on starttime drift, _reset_for_tests
clear behavior, iron_proxy_version don't-cache-empty, NODE_OPTIONS
sentinel verification, ensure_audit_log raise-on-OSError, and
persisted-nonce roundtrip.
- Added 1 new test in test_iron_proxy_cli.py covering cmd_start
BWS-token-missing fail-loud.
- All 100 tests in test_iron_proxy + test_iron_proxy_cli pass; all 78
tests in test_docker_environment + test_config still pass.
Acknowledged but not addressed:
- GodsBoy P3 dead-code `extra_env` kwarg: kept (removing is a breaking
change for any out-of-tree caller; the kwarg is documented and works).
- Residual risks GodsBoy called out: iron-proxy in-memory secret
zeroisation (Go-binary territory, out of scope); _PROXY_SUBPROCESS_ENV
_ALLOWLIST cosmetic gaps (RUST_LOG, GOMAXPROCS); follow-up.
P0 — must-fix
- iron_proxy: emit default upstream_deny_cidrs (loopback, IMDS
169.254.0.0/16, RFC1918) when caller passes None. Honours the docs
promise that cloud-metadata IPs are refused regardless of allowlist.
- iron_proxy: bind 127.0.0.1 (+ docker0 bridge IP on Linux) instead of
INADDR_ANY (':9090'). LAN peers with a leaked sandbox token could
otherwise spend the operator's API quota against any allowlisted
upstream.
- ensure_ca_cert: write the CA private key via os.open(..., 0o600)
instead of shutil.copy2+os.chmod — closes the TOCTOU window where
the key existed under the default umask.
- discover_uncovered_providers + proxy.fail_on_uncovered_providers
config: refuse to start (when strict) if env vars for non-bearer
providers (Anthropic native x-api-key, AWS SigV4, Azure OpenAI,
etc.) are present. Surfaces a wizard warning in non-strict mode.
P1 — should-fix
- start_proxy: build a minimal subprocess env (PATH/HOME/locale +
only the env names referenced by mappings) instead of os.environ
.copy(). Strips proxy-recursion vars (HTTPS_PROXY etc.). Stops
the proxy's /proc/<pid>/environ from leaking every host secret
to same-uid local processes.
- start_proxy: optional Bitwarden refresh path
(refresh_secrets_from_bitwarden=True, bitwarden_config=...).
When credential_source=bitwarden, cmd_start wires it in — that's
what delivers the rotation guarantee the docs make.
- build_proxy_config: wire audit_log into the rendered yaml
(log.audit_path). Parameter was accepted but never used.
- ensure_audit_log: pre-create the audit log with 0o600 perms so
iron-proxy inherits tight permissions instead of relying on umask.
- Rename 'hermes proxy ...' → 'hermes egress ...' in user-facing
strings (docstring, RuntimeError messages, post-setup banner).
- start_proxy: open log file with 0o600 perms and close the parent
fd immediately after Popen — fixes the per-restart fd leak.
- DockerEnvironment: detect collisions between docker_env and the
egress-controlling env vars (HTTPS_PROXY, SSL_CERT_FILE, etc.).
When enforce_on_docker=true, fail loud rather than silently
inverting the isolation; when false, warn and let docker_env win.
- proxy_cli: merge_mappings preserves existing tokens on re-setup;
--rotate-tokens flag re-mints all of them. Stops re-running
`hermes egress setup` from invalidating tokens baked into
already-running sandboxes.
- proxy_cli: --from-bitwarden fail-loud on disabled BW config,
missing access token, or empty vault. Previously fell through to
the env path while still writing credential_source: bitwarden.
- docker.py: narrow `except Exception` → `except ImportError`;
iron_proxy._read_tunnel_port_from_config: same. Bare excepts
were masking real config-load bugs.
- start_proxy: write pidfile via os.open with O_NOFOLLOW + 0o600
+ st_uid check. Refuses to follow a pre-existing symlink at the
pidfile path.
- mint_proxy_token docstring: document the 128-bit suffix entropy
explicitly (sha256 truncated to 32 hex chars).
P2 — follow-up
- start_proxy: poll-with-timeout (100ms cadence on _port_listening)
instead of an unconditional 5s sleep. Saves several seconds per
Docker container create when enforce_on_docker=true.
- docker.py: apply enforce_on_docker semantics when CA file vanishes
between status.configured check and CA mount. Previously returned
empty args silently.
- docker.py: refuse to mount when mappings.json is empty/corrupt
(was indistinguishable from upstream outage from inside the
sandbox).
- install_iron_proxy: tarfile.extract(..., filter='data') to silence
the PEP 706 deprecation and opt into the 3.14+ default.
- _proxy_state_dir: chmod 0o700 unconditionally; add
_proxy_state_dir_ro() so read-only callers don't create the dir.
- stop_proxy: re-verify pid before SIGKILL via /proc/<pid>/stat
starttime AND _pid_alive. Prevents SIGKILL'ing a recycled pid.
- _pid_alive: tightened cmdline check — basename match on argv[0]
plus an in-process nonce env var ('iron-proxy' in cmdline matched
'tail iron-proxy.log' and editors with the log open).
- docker.py: NODE_OPTIONS=--use-openssl-ca so Node.js routes through
the OpenSSL CA store SSL_CERT_FILE controls, narrowing the
Python/curl-replace vs Node-add asymmetry waefrebeorn flagged.
P3 — polish
- proxy_cli: dest='egress_command' (was 'proxy_command' which
collided lexically with the inbound OAuth subparser).
- iron_proxy_version: cache by binary path — get_status is called
per Docker container create, version is constant per binary.
- Drop unused `import sys` from iron_proxy.
- proxy_cli: `is not None` check on --tunnel-port (was treating 0
as falsy and silently substituting the default).
- proxy_cli cmd_disable: use get_status().pid instead of reaching
into ip._read_pid() (stale pidfile from a crashed run would have
fired a spurious "still running" warning).
- Tests: replace hardcoded /tmp/ca.* paths with tmp_path-derived
fixtures so tests are hermetic across hosts.
CI
- Windows footguns scanner: os.kill(pid, 0) is now gated behind
platform.system() != 'Windows' with a windows-footgun: ok marker;
signal.SIGKILL falls back to SIGTERM on Windows via
getattr(signal, 'SIGKILL', signal.SIGTERM).
- docs MDX compilation: replace bare `<https://…>` URLs with
`[text](url)` syntax (MDX-jsx parser rejects the angle-bracket
form).
Tests
- 32 new tests covering default deny CIDRs, bind policy, audit log
wiring, subprocess env minimization, CA TOCTOU 0o600, state dir
0o700, empty-mappings refusal, CA-vanished refusal, docker_env
collision detection, token preservation/rotate, uncovered provider
detection, and the proxy_cli command handlers + argparse wiring.
- All 156 tests in test_iron_proxy + test_iron_proxy_cli +
test_docker_environment + test_config pass locally.
Acknowledged but not addressed in this revision
- E2E test for HTTPS CONNECT + TLS-MITM path: existing E2E exercises
plain HTTP; full MITM coverage needs separate CI infra (real iron-
proxy binary + curl with custom CA). Tracked as follow-up.
- Cosign-style supply-chain verification for the binary checksum:
upstream iron-proxy doesn't sign releases yet. Accepted pattern
(same as Bitwarden integration); tracked as follow-up.
- CA rotation CLI (`hermes egress rotate-ca`): scope-cut to a
follow-up.
Reviewers: @annguyenNous @waefrebeorn @GodsBoy @erhnysr
Adds a TLS-intercepting egress proxy for remote terminal sandboxes (Docker
v1; Modal/SSH to follow). When enabled, the sandbox holds opaque proxy
tokens; iron-proxy swaps them for real provider API keys at the egress
boundary. Compromising the sandbox leaks tokens that only work from behind
the proxy.
Wraps ironsh/iron-proxy (Apache-2.0, Go binary). Same lazy-install pattern
as the recently merged Bitwarden Secrets Manager integration — pinned
version, SHA-256 verified download into ~/.hermes/bin/iron-proxy, no apt
or sudo required.
Disabled by default. Run `hermes egress setup` to mint tokens and
`hermes egress start` to launch. The Docker backend then automatically
mounts the CA, sets HTTPS_PROXY + CA-bundle env vars, and adds the
host-gateway hostmap.
New surfaces:
hermes egress install — download the pinned iron-proxy binary
hermes egress setup — interactive wizard (supports --from-bitwarden)
hermes egress start — spawn the managed proxy daemon
hermes egress stop — SIGTERM (+SIGKILL after 5s grace)
hermes egress status — binary + config + pid + listening + mappings
hermes egress disable — flip proxy.enabled = false
hermes egress config — print the path to the generated proxy.yaml
Optional Bitwarden integration: `--from-bitwarden` sources the real
upstream credentials from a BSM project at proxy startup, so rotating a
key in the Bitwarden web app propagates to sandboxes on the next proxy
start without touching .env.
Hermes-side scope (v1):
agent/proxy_sources/iron_proxy.py — install + CA + config + lifecycle
hermes_cli/proxy_cli.py — `hermes egress` subcommand tree
hermes_cli/config.py — "proxy:" section in DEFAULT_CONFIG
hermes_cli/main.py — argparse wiring (uses 'egress'
because 'proxy' is the existing
inbound OAuth reverse proxy)
tools/environments/docker.py — CA mount, HTTPS_PROXY, CA-bundle
env vars, --add-host wiring
Hermetic tests cover the full lifecycle: token mint, mapping discovery,
config + mappings I/O, install pipeline (HTTP + tar + checksum all mocked),
subprocess lifecycle (Popen mocked), Docker backend arg builder.
A live E2E test (gated on HERMES_RUN_E2E=1) downloads the real iron-proxy
binary, spawns it, routes a curl request through it against a local fake
upstream, and verifies the Authorization header was swapped from the proxy
token to the real secret value (and the proxy token did NOT leak through
to upstream).
Failures (binary missing, port collision, bad token) never block agent
startup — they emit a warning and continue. The Docker backend refuses to
start a sandbox when proxy.enabled=true but the daemon is dead, unless
proxy.enforce_on_docker is explicitly set to false.
Docs: website/docs/user-guide/egress/{index,iron-proxy}.md
Tests: tests/test_iron_proxy.py (35), tests/test_iron_proxy_e2e.py (1)
Two independent opt-in QoL toggles, both off by default.
terminal.docker_extra_args:
- List of extra flags appended verbatim to docker run after security
defaults. Useful for adding capabilities (e.g. --cap-add SETUID) or
other docker run options not exposed by existing config keys.
- Non-string entries are logged and skipped.
- Also available via TERMINAL_DOCKER_EXTRA_ARGS='[...]' env var.
display.timestamps:
- Appends [HH:MM] to user input bullet and the assistant response box
header. Single hub in _format_submitted_user_message_preview()
covers both single-line and multi-line user previews; assistant
response label gets the timestamp at box-open time.
Closes#1569 (timestamps).
Co-authored-by: Mibayy <Mibayy@users.noreply.github.com>
Add opt-in terminal.docker_run_as_host_user config flag that passes
--user $(id -u):$(id -g) to the Docker backend so files written into
bind-mounted directories (/workspace, /root, docker_volumes entries) are
owned by the host user instead of root.
When enabled on POSIX platforms, also drops SETUID/SETGID caps since the
container no longer needs gosu/su to switch users. Falls back cleanly on
platforms without os.getuid (e.g. native Windows Docker) with a warning.
Wired through all three config.yaml -> TERMINAL_* env-var bridges:
- cli.py env_mappings (CLI + TUI startup)
- gateway/run.py _terminal_env_map (gateway / messaging platforms)
- hermes_cli/config.py _config_to_env_sync (`hermes config set`)
Also fixes docker_mount_cwd_to_workspace silently failing in gateway
mode -- it was missing from gateway/run.py's _terminal_env_map.
Adds tests/tools/test_terminal_config_env_sync.py to guard against
future drift between the three bridges (same bug class shipped twice
in one month).
Bundled Hermes image won't work with this flag since its entrypoint
expects to start as root for the usermod/gosu hermes flow; works with
the default nikolaik/python-nodejs image and plain Debian/Ubuntu.
The Docker terminal backend runs containers with `--cap-drop ALL`
and re-adds only DAC_OVERRIDE, CHOWN, FOWNER. Since commit fee0e0d3
("run as non-root user, use virtualenv") the image entrypoint drops
from root to the `hermes` user via `gosu`, which requires CAP_SETUID
and CAP_SETGID. Without them every sandbox container exits
immediately with:
Dropping root privileges
error: failed switching to 'hermes': operation not permitted
Breaking every terminal/file tool invocation in `terminal.backend: docker`
mode.
Fix: add SETUID and SETGID to the cap-add list. The `no-new-privileges`
security-opt is kept, so gosu still cannot escalate back to root after
the one-way drop — the hardening posture is preserved.
Reproduction
------------
With any image whose ENTRYPOINT calls `gosu <user>`, the container
exits immediately under the pre-fix cap set. Post-fix, the drop
succeeds and the container proceeds normally.
docker run --rm \
--cap-drop ALL \
--cap-add DAC_OVERRIDE --cap-add CHOWN --cap-add FOWNER \
--security-opt no-new-privileges \
--entrypoint /usr/local/bin/gosu \
hermes-claude:latest hermes id
# -> error: failed switching to 'hermes': operation not permitted
# Same command with SETUID+SETGID added:
# -> uid=10000(hermes) gid=10000(hermes) groups=10000(hermes)
Tests
-----
Added `test_security_args_include_setuid_setgid_for_gosu_drop` that
asserts both caps are present and the overall hardening posture
(cap-drop ALL + no-new-privileges) is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- find_docker() now checks HERMES_DOCKER_BINARY env var first, then
docker on PATH, then podman on PATH, then macOS known locations
- Entrypoint respects HERMES_HOME env var (was hardcoded to /opt/data)
- Entrypoint uses groupmod -o to tolerate non-unique GIDs (fixes macOS
GID 20 conflict with Debian's dialout group)
- Entrypoint makes chown best-effort so rootless Podman continues
instead of failing with 'Operation not permitted'
- 5 new tests covering env var override, podman fallback, precedence
Based on work by alanjds (PR #3996) and malaiwah (PR #8115).
Closes#4084.
Two issues with sandbox container spawning:
1. PID 1 was `sleep 2h` which doesn't call wait() — every background
process that exited became a zombie (<defunct>), and the process
tool reported them as "running" because zombie PIDs still exist in
the process table. Fix: add --init to docker run, which uses
tini (Docker) or catatonit (Podman) as PID 1 to reap children
automatically. Both runtimes support --init natively.
2. The fixed 2-hour lifetime was arbitrary and sometimes too short
for long agent sessions. Fix: replace 'sleep 2h' with
'sleep infinity'. The idle reaper (_cleanup_inactive_envs, gated
by terminal.lifetime_seconds, default 300s) already handles
cleanup based on last activity timestamp — there's no need for
the container itself to have a fixed death timer.
Fixes#6908.
Automated dead code audit using vulture + coverage.py + ast-grep intersection,
confirmed by Opus deep verification pass. Every symbol verified to have zero
production callers (test imports excluded from reachability analysis).
Removes ~1,534 lines of dead production code across 46 files and ~1,382 lines
of stale test code. 3 entire files deleted (agent/builtin_memory_provider.py,
hermes_cli/checklist.py, tests/hermes_cli/test_setup_model_selection.py).
Co-authored-by: alt-glitch <balyan.sid@gmail.com>
Fixes 9 test failures on current main, incorporating ideas from PR stack
#6219-#6222 by xinbenlv with corrections:
- model_metadata: sync HF context length key casing
(minimaxai/minimax-m2.5 → MiniMaxAI/MiniMax-M2.5)
- cli.py: route quick command error output through self.console
instead of creating a new ChatConsole() instance
- docker.py: explicit docker_forward_env entries now bypass the
Hermes secret blocklist (intentional opt-in wins over generic filter)
- auxiliary_client: revert _read_main_provider() to simple
provider.strip().lower() — the _normalize_aux_provider() call
introduced in 5c03f2e7 stripped the custom: prefix, breaking
named custom provider resolution
- auxiliary_client: flip vision auto-detection order to
active provider → OpenRouter → Nous → stop (was OR → Nous → active)
- test: update vision priority test to match new order
Based on PR #6219-#6222 by xinbenlv.
Shell injection via unquoted workdir interpolation in docker, singularity,
and SSH backends. When workdir contained shell metacharacters (e.g.
~/;id), arbitrary commands could execute.
Changes:
- Add shlex.quote() at each interpolation point in docker.py,
singularity.py, and ssh.py with tilde-aware quoting (keep ~
unquoted for shell expansion, quote only the subpath)
- Add _validate_workdir() allowlist in terminal_tool.py as
defense-in-depth before workdir reaches any backend
Original work by Mariano A. Nicolini (PR #5620). Salvaged with fixes
for tilde expansion (shlex.quote breaks cd ~/path) and replaced
incomplete deny-list with strict character allowlist.
Co-authored-by: Mariano A. Nicolini <entropidelic@users.noreply.github.com>
Add docker_env option to terminal config — a dict of key-value pairs that
get set inside Docker containers via -e flags at both container creation
(docker run) and per-command execution (docker exec) time.
This complements docker_forward_env (which reads values dynamically from
the host process environment). docker_env is useful when Hermes runs as a
systemd service without access to the user's shell environment — e.g.
setting SSH_AUTH_SOCK or GNUPGHOME to known stable paths for SSH/GPG
agent socket forwarding.
Precedence: docker_env provides baseline values; docker_forward_env
overrides for the same key.
Config example:
terminal:
docker_env:
SSH_AUTH_SOCK: /run/user/1000/ssh-agent.sock
GNUPGHOME: /root/.gnupg
docker_volumes:
- /run/user/1000/ssh-agent.sock:/run/user/1000/ssh-agent.sock
- /run/user/1000/gnupg/S.gpg-agent:/root/.gnupg/S.gpg-agent
The config key skills.external_dirs and core resolution (get_all_skills_dirs,
get_external_skills_dirs in agent/skill_utils.py) already existed but several
code paths still only scanned SKILLS_DIR. Now external dirs are respected
everywhere:
- skills_categories(): scan all dirs for category discovery
- _get_category_from_path(): resolve categories against any skills root
- skill_manager_tool._find_skill(): search all dirs for edit/patch/delete
- credential_files.get_skills_directory_mount(): mount all dirs into
Docker/Singularity containers (external dirs at external_skills/<idx>)
- credential_files.iter_skills_files(): list files from all dirs for
Modal/Daytona upload
- tools/environments/ssh.py: rsync all skill dirs to remote hosts
- gateway _check_unavailable_skill(): check disabled skills across all dirs
Usage in config.yaml:
skills:
external_dirs:
- ~/repos/agent-skills/hermes
- /shared/team-skills
- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
credential_files.py for host-side cache directory passthrough
(documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
of host layout.
This replaces the need for a dedicated extract_archive tool (PR #4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.
Closes: related to PR #4819 by kshitijk4poor
Skills with scripts/, templates/, and references/ subdirectories need
those files available inside sandboxed execution environments. Previously
the skills directory was missing entirely from remote backends.
Live sync — files stay current as credentials refresh and skills update:
- Docker/Singularity: bind mounts are inherently live (host changes
visible immediately)
- Modal: _sync_files() runs before each command with mtime+size caching,
pushing only changed credential and skill files (~13μs no-op overhead)
- SSH: rsync --safe-links before each command (naturally incremental)
- Daytona: _upload_if_changed() with mtime+size caching before each command
Security — symlink filtering:
- Docker/Singularity: sanitized temp copy when symlinks detected
- Modal/Daytona: iter_skills_files() skips symlinks
- SSH: rsync --safe-links skips symlinks pointing outside source tree
- Temp dir cleanup via atexit + reuse across calls
Non-root user support:
- SSH: detects remote home via echo $HOME, syncs to $HOME/.hermes/
- Daytona: detects sandbox home before sync, uploads to $HOME/.hermes/
- Docker/Modal/Singularity: run as root, /root/.hermes/ is correct
Also:
- credential_files.py: fix name/path key fallback in required_credential_files
- Singularity, SSH, Daytona: gained credential file support
- 14 tests covering symlink filtering, name/path fallback, iter_skills_files
Two related fixes for remote terminal backends (Modal/Docker):
1. NEW: Credential file mounting system
Skills declare required_credential_files in frontmatter. Files are
mounted into Docker (read-only bind mounts) and Modal (mounts at
creation + sync via exec on each command for mid-session changes).
Google Workspace skill updated with the new field.
2. FIX: Docker backend now includes env_passthrough vars
Skills that declare required_environment_variables (e.g. Notion with
NOTION_API_KEY) register vars in the env_passthrough system. The
local backend checked this, but Docker's forward_env was a separate
disconnected list. Now Docker exec merges both sources, so
skill-declared env vars are forwarded into containers automatically.
This fixes the reported issue where NOTION_API_KEY in ~/.hermes/.env
wasn't reaching the Docker container despite being registered via
the Notion skill's prerequisites.
Closes#3665
Drop the mini-swe-agent git submodule. All terminal backends now use
hermes-agent's own environment implementations directly.
Docker backend:
- Inline the `docker run -d` container startup (was 15 lines in
minisweagent's DockerEnvironment). Our wrapper already handled
execute(), cleanup(), security hardening, volumes, and resource limits.
Modal backend:
- Import swe-rex's ModalDeployment directly instead of going through
minisweagent's 90-line passthrough wrapper.
- Bake the _AsyncWorker pattern (from environments/patches.py) directly
into ModalEnvironment for Atropos compatibility without monkey-patching.
Cleanup:
- Remove minisweagent_path.py (submodule path resolution helper)
- Remove submodule init/install from install.sh and setup-hermes.sh
- Remove mini-swe-agent from .gitmodules
- environments/patches.py is now a no-op (kept for backward compat)
- terminal_tool.py no longer does sys.path hacking for minisweagent
- mini_swe_runner.py guards imports (optional, for RL training only)
- Update all affected tests to mock the new direct subprocess calls
- Update README.md, CONTRIBUTING.md
No functionality change — all Docker, Modal, local, SSH, Singularity,
and Daytona backends behave identically. 6093 tests pass.
When container_persistent=false, the inner mini-swe-agent cleanup only
runs 'docker stop' in the background, leaving containers in Exited state.
Now cleanup() also runs 'docker rm -f' to fully remove the container.
Also fixes pre-existing test failures in model_metadata (gpt-4.1 1M context),
setup tests (TTS provider step), and adds MockInnerDocker.cleanup().
Original fix by crazywriter1. Cherry-picked and adapted for current main.
Fixes#1679
Docker terminal sessions are secret-dark by default. This adds
terminal.docker_forward_env as an explicit allowlist for env vars
that may be forwarded into Docker containers.
Values resolve from the current shell first, then fall back to
~/.hermes/.env. Only variables the user explicitly lists are
forwarded — nothing is auto-exposed.
Cherry-picked from PR #1449 by @teknium1, conflict-resolved onto
current main.
Fixes#1436
Supersedes #1439
Keep Docker sandboxes isolated by default. Add an explicit terminal.docker_mount_cwd_to_workspace opt-in, thread it through terminal/file environment creation, and document the security tradeoff and config.yaml workflow clearly.
Fixes#1445 — When using Docker backend, the user's current working
directory is now automatically bind-mounted to /workspace inside the
container. This allows users to run `cd my-project && hermes` and have
their project files accessible to the agent without manual volume config.
Changes:
- Add host_cwd and auto_mount_cwd parameters to DockerEnvironment
- Capture original host CWD in _get_env_config() before container fallback
- Pass host_cwd through _create_environment() to Docker backend
- Add TERMINAL_DOCKER_NO_AUTO_MOUNT env var to disable if needed
- Skip auto-mount when /workspace is already explicitly mounted
- Add tests for auto-mount behavior
- Add documentation for the new feature
The auto-mount is skipped when:
1. TERMINAL_DOCKER_NO_AUTO_MOUNT=true is set
2. User configured docker_volumes with :/workspace
3. persistent_filesystem=true (persistent sandbox mode)
This makes the Docker backend behave more intuitively — the agent
operates on the user's actual project directory by default.
* feat: improve context compaction handoff summaries
Adapt PR #916 onto current main by replacing the old context summary marker
with a clearer handoff wrapper, updating the summarization prompt for
resume-oriented summaries, and preserving the current call_llm-based
compression path.
* fix: clearer error when docker backend is unavailable
* fix: preserve docker discovery in backend preflight
Follow up on salvaged PR #940 by reusing find_docker() during the new
availability check so non-PATH Docker Desktop installs still work. Add
a regression test covering the resolved executable path.
---------
Co-authored-by: aydnOktay <xaydinoktay@gmail.com>
On macOS, Docker Desktop installs the CLI to /usr/local/bin/docker, but
when Hermes runs as a gateway service (launchd) or in other non-login
contexts, /usr/local/bin is often not in PATH. This causes the Docker
requirements check to fail with 'No such file or directory: docker' even
though docker works fine from the user's terminal.
Add find_docker() helper that uses shutil.which() first, then probes
common Docker Desktop install paths on macOS (/usr/local/bin,
/opt/homebrew/bin, Docker.app bundle). The resolved path is cached and
passed to mini-swe-agent via its 'executable' parameter.
- tools/environments/docker.py: add find_docker(), use it in
_storage_opt_supported() and pass to _Docker(executable=...)
- tools/terminal_tool.py: use find_docker() in requirements check
- tests/tools/test_docker_find.py: 4 tests (PATH, fallback, not found, cache)
2877 tests pass.
cap-drop ALL removes DAC_OVERRIDE, which root needs to write to
bind-mounted directories owned by the host user (uid 1000). This
broke persistent Docker sandboxes — the container couldn't write
to /workspace or /root.
Add back the minimum capabilities needed:
- DAC_OVERRIDE: root can write to bind-mounted dirs owned by host user
- CHOWN: package managers (pip, npm, apt) need to set file ownership
- FOWNER: needed for operations on files owned by other users
Still drops all other capabilities (NET_RAW, SYS_ADMIN, etc.) and
keeps no-new-privileges. Security boundary is the container itself.
Verified end-to-end: create files → destroy container → new container
with same task_id → files persist on host and are accessible in the
new container.
The Docker sandbox previously used --read-only on the root filesystem and
noexec on /tmp. This broke 30+ skills that need to install packages:
- npm install -g (codex, claude-code, mcporter, powerpoint)
- pip install (20+ mlops/media/productivity skills)
- apt install (minecraft-modpack-server, ml-paper-writing)
- Build tools that compile in /tmp (pip wheels, node-gyp)
The container is already fully isolated from the host. Industry standard
(E2B, Docker Sandboxes, OpenAI Codex) does not use --read-only — the
container itself is the security boundary.
Retained security hardening:
- --cap-drop ALL (zero capabilities)
- --security-opt no-new-privileges (no escalation)
- --pids-limit 256 (no fork bombs)
- Size-limited tmpfs for /tmp, /var/tmp, /run
- nosuid on all tmpfs mounts
- noexec on /var/tmp and /run (rarely need exec there)
- Resource limits (CPU, memory, disk)
- Ephemeral containers (destroyed after use)
Fixes#189.
- Updated the installation script to check for necessary build tools on Debian/Ubuntu systems and prompt the user to install them if missing.
- Improved user interaction by redirecting input from /dev/tty for prompts, ensuring compatibility when the script is piped from curl.
- Added checks to verify the successful installation of the main package and provide guidance if installation fails.
- Enhanced the handling of shell configuration files to ensure ~/.local/bin is added to PATH for various shell types.
- Introduced a static method to verify if the Docker storage driver supports the --storage-opt size= option.
- Enhanced resource argument handling in DockerEnvironment to conditionally include storage options based on the support check.
- Added caching for the support check result to optimize performance across instances.
Three issues prevented the Docker terminal backend from working:
1. `effective_image` was referenced but never defined — only the Modal
backend sets this variable. Use `image` directly instead.
2. `--storage-opt size=N` is unsupported on Docker Desktop for Mac
(requires overlay2 with xfs backing). Skip the flag on Darwin.
3. Docker requires absolute paths for `-w` (working directory) but the
default cwd was `~`, which Docker does not expand. Default to `/root`
and translate any `~` passed in from callers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added a new section in the README for Inference Providers, detailing setup instructions for Nous Portal, OpenRouter, and Custom Endpoints, improving user guidance for LLM connections.
- Updated messaging platform setup instructions to include Slack and WhatsApp, providing clearer steps for configuration.
- Introduced a new environment variable, TERMINAL_SANDBOX_DIR, to allow users to customize the sandbox storage location for Docker and Singularity environments.
- Refactored the Docker and Singularity environment classes to utilize the new sandbox directory for persistent workspaces, enhancing organization and usability.
- Improved handling of working directories across various environments, ensuring compatibility and clarity in execution paths.
- Introduced a shared interrupt signaling mechanism to allow tools to check for user interrupts during long-running operations.
- Updated the AIAgent to handle interrupts more effectively, ensuring in-progress tool calls are canceled and multiple interrupt messages are combined into one prompt.
- Enhanced the CLI configuration to include container resource limits (CPU, memory, disk) and persistence options for Docker, Singularity, and Modal environments.
- Improved documentation to clarify interrupt behaviors and container resource settings, providing users with better guidance on configuration and usage.
