Follow-up for salvaged PR #39946: file_sync.py already aliases time.sleep
as _sleep specifically to avoid tests mutating the shared stdlib module
object. Apply the same convention to the rate-limit clock (_monotonic)
and point the new regression test at it.
FileSyncManager.sync() is rate-limited to once per _sync_interval via
_last_sync_time, and its docstring promises that on failure "state rolls
back so the next cycle retries everything". But the except handler also
set _last_sync_time = time.monotonic() on failure, so the next non-forced
sync() within the interval hit the rate-limit guard and returned early —
suppressing the retry the rollback had just prepared.
Because the non-forced sync() runs before every command on the SSH, Modal
and Daytona backends, a single transient upload failure (network blip,
dropped channel) left the remote with stale files for the next command
(up to _sync_interval, default 5s). Forced syncs bypass the guard, which
is why it was intermittent.
Remove the failure-path timestamp bump so the clock only advances on a
successful or no-op cycle, matching the documented contract. Add a
regression test that fails before this change and passes after.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cover the three key behaviors:
- bulk_upload_fn is called instead of per-file upload_fn
- Fallback to upload_fn when bulk_upload_fn is None
- Rollback on bulk upload failure retries all files
Replace per-backend ad-hoc file sync with a shared FileSyncManager
that handles mtime-based change detection, remote deletion of
locally-removed files, and transactional state updates.
- New FileSyncManager class (tools/environments/file_sync.py)
with callbacks for upload/delete, rate limiting, and rollback
- Shared iter_sync_files() eliminates 3 duplicate implementations
- SSH: replace unconditional rsync with scp + mtime skip
- Modal/Daytona: replace inline _synced_files dict with manager
- All 3 backends now sync credentials + skills + cache uniformly
- Remote deletion: files removed locally are cleaned from remote
- HERMES_FORCE_FILE_SYNC=1 env var for debugging
- Base class _before_execute() simplified to empty hook
- 12 unit tests covering mtime skip, deletion, rollback, rate limiting
