Commit graph

2 commits

Author SHA1 Message Date
Ben
41b9b7e719 test(lazy-deps): make durable-target tests network-free
CI test shard has no PyPI egress: the real 'pip install packaging==20.9'
in test_core_package_is_not_shadowed failed (the pypi.org reachability
probe passed but the actual install didn't), failing slice 2/6.

- Prove the anti-shadow invariant deterministically: synthesize a fake
  'packaging' in the durable target with a sentinel and assert the import
  still resolves to the core copy (TestCoreNeverShadowed). No network.
- Cover the install wire offline: stub subprocess and assert --target +
  --constraint are built in durable mode and absent in venv-scoped mode
  (TestInstallArgConstruction).
- Gate the genuine PyPI install behind HERMES_RUN_NETWORK_TESTS=1 (opt-in,
  skipped in CI) instead of a flaky reachability probe that doesn't predict
  install success.
2026-06-25 09:20:13 +10:00
Ben
cbd6ba1bdd fix(docker): redirect lazy installs to a durable target so opt-in backends work in the immutable image (#51136)
The published Docker image seals the agent venv (root-owned, read-only
/opt/hermes) and sets HERMES_DISABLE_LAZY_INSTALLS=1 so a runtime install
can't mutate and brick the core. But opt-in backends (Firecrawl web search,
Exa, Feishu, ...) deliberately keep their SDKs in tools/lazy_deps.py and out
of [all] (pyproject policy 2026-05-12: one quarantined release must not break
every install). The two policies collided: the SDK isn't baked in AND can't
lazy-install, so the default Firecrawl web_search/web_extract fail out of the
box in Docker (#51136), as do Exa (#49445) and Feishu (#50205).

Fix the whole class instead of baking in one backend: when
HERMES_LAZY_INSTALL_TARGET is set, lazy installs are redirected to a writable
dir on the durable /opt/data volume via `pip/uv install --target`, and that
dir is APPENDED to the end of sys.path. Because the core venv always wins
name collisions, a package installed this way can only ADD new modules — it
can never shadow, downgrade, or break a module the core ships. The worst a
bad/incompatible backend package can do is fail to import and report itself
unavailable; the agent core stays healthy. That structural guarantee is what
made it safe to seal the venv, and it is preserved here even with installs
re-enabled.

- tools/lazy_deps.py: durable-target mode — `--target` install + core-pinned
  `--constraint` file (shared deps resolve to core's versions, conflicts fail
  loudly at install time), append-only sys.path activation, ABI/Python-version
  stamp that wipes the store if an image rebuild bumps the interpreter, and a
  reworked gate so HERMES_DISABLE_LAZY_INSTALLS=1 redirects (rather than hard-
  blocks) when a target is set. security.allow_lazy_installs=false still
  disables installs in every mode.
- hermes_bootstrap.py: activate the durable target on sys.path at first import
  (before any backend imports its SDK) so packages installed on a previous run
  are importable on this run.
- Dockerfile: set HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages.
- docker/stage2-hook.sh: seed + chown the dir on the data volume.
- tests: real-install E2E proving installs land in the target, import cleanly,
  don't leak into the sealed venv, and that a core package is never shadowed;
  ABI-stamp wipe/preserve; gate matrix; Dockerfile/stage2 contract test.

Fixes #51136
2026-06-25 09:20:13 +10:00