hermes-agent/tools/environments
Siddharth Balyan 27eeea0555
perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive (#8014)
* perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive

SSH: symlink-staging + tar -ch piped over SSH in a single TCP stream.
Eliminates per-file scp round-trips. Handles timeout (kills both
processes), SSH Popen failure (kills tar), and tar create failure.

Modal: in-memory gzipped tar archive, base64-encoded, decoded+extracted
in one exec call. Checks exit code and raises on failure.

Both backends use shared helpers extracted into file_sync.py:
- quoted_mkdir_command() — mirrors existing quoted_rm_command()
- unique_parent_dirs() — deduplicates parent dirs from file pairs

Migrates _ensure_remote_dirs to use the new helpers.

28 new tests (21 SSH + 7 Modal), all passing.

Closes #7465
Closes #7467

* fix(modal): pipe stdin to avoid ARG_MAX, clean up review findings

- Modal bulk upload: stream base64 payload through proc.stdin in 1MB
  chunks instead of embedding in command string (Modal SDK enforces
  64KB ARG_MAX_BYTES — typical payloads are ~4.3MB)
- Modal single-file upload: same stdin fix, add exit code checking
- Remove what-narrating comments in ssh.py and modal.py (keep WHY
  comments: symlink staging rationale, SIGPIPE, deadlock avoidance)
- Remove unnecessary `sandbox = self._sandbox` alias in modal bulk
- Daytona: use shared helpers (unique_parent_dirs, quoted_mkdir_command)
  instead of inlined duplicates

---------

Co-authored-by: kshitijk4poor <82637225+kshitijk4poor@users.noreply.github.com>
2026-04-12 06:18:05 +05:30
..
__init__.py feat(environments): add Daytona cloud sandbox backend 2026-03-05 10:02:21 -08:00
base.py fix: prevent agent from stopping mid-task — compression floor, budget overhaul, activity tracking 2026-04-11 16:18:57 -07:00
daytona.py perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive (#8014) 2026-04-12 06:18:05 +05:30
docker.py fix(docker): --init for zombie reaping + sleep infinity for idle-based lifetime 2026-04-10 15:42:30 -07:00
file_sync.py perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive (#8014) 2026-04-12 06:18:05 +05:30
local.py fix: per-profile subprocess HOME isolation (#4426) (#7357) 2026-04-10 13:37:45 -07:00
managed_modal.py feat(environments): unified spawn-per-call execution layer 2026-04-08 17:23:15 -07:00
modal.py perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive (#8014) 2026-04-12 06:18:05 +05:30
modal_utils.py feat(environments): unified spawn-per-call execution layer 2026-04-08 17:23:15 -07:00
singularity.py feat(environments): unified spawn-per-call execution layer 2026-04-08 17:23:15 -07:00
ssh.py perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive (#8014) 2026-04-12 06:18:05 +05:30