fix(update): prevent and self-heal half-updated venvs on Windows (#57659)

Root-causes the July 2026 Windows incident chain (locked _brotlicffi.pyd /
_sodium.pyd during install, then 'No module named annotated_doc' with
'hermes update' insisting 'Already up to date!'):

- hermes update: probe venv core imports even when the checkout is current;
  a half-updated venv (dep sync killed mid-flight by a locked .pyd) is now
  detected and repaired instead of being reported as up to date
- hermes update (Windows): after pausing gateways, refuse to mutate the venv
  while other processes run from the venv interpreter (the Desktop backend
  runs as python.exe so the hermes.exe shim guard never saw it); --force
  keeps the old behavior
- install.ps1 venv stage: disarm gateway autostart Scheduled Tasks before
  the kill sweep (they respawn the gateway inside the kill->delete window),
  make the sweep a bounded loop requiring 3 clean passes, and rename-then-
  delete the old venv (a rename succeeds even with mapped DLLs) with stale-
  dir cleanup on the next run
- desktop updater: 'venv shim still locked after 15s' now ABORTS the update
  hand-off (restarting our backend, surfacing the holder to the user)
  instead of 'proceeding anyway (force)' into guaranteed venv corruption;
  the unlock wait also re-kills respawned backends each poll tick
This commit is contained in:
Teknium 2026-07-03 03:24:08 -07:00 committed by GitHub
parent 741bd9ba42
commit b14d75f8af
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 507 additions and 20 deletions

View file

@ -2168,9 +2168,25 @@ async function releaseBackendLock(updateRoot, tag) {
rememberLog(`[${tag}] venv shim unlocked; safe to proceed`)
return { unlocked: true }
}
// A supervised backend can respawn between kill and check (grandchildren,
// pool entries registered mid-teardown). Re-collect and re-kill each pass
// instead of trusting the initial sweep.
const stragglers = []
if (hermesProcess && Number.isInteger(hermesProcess.pid)) stragglers.push(hermesProcess.pid)
for (const entry of backendPool.values()) {
if (entry.process && Number.isInteger(entry.process.pid)) stragglers.push(entry.process.pid)
}
for (const pid of stragglers) forceKillProcessTree(pid)
await new Promise(r => setTimeout(r, 300))
}
rememberLog(`[${tag}] venv shim still locked after 15s; proceeding anyway (force)`)
// Do NOT proceed past a held lock: handing off to the updater while another
// process (a second desktop window, a user terminal, an unkillable child)
// still maps the venv's files guarantees a half-updated venv — the updater's
// dependency sync dies on access-denied partway through uninstalls, leaving
// imports broken (the July 2026 brotlicffi/_sodium.pyd incidents). Failing
// the update loudly and keeping the app running is strictly better than a
// bricked install that needs manual venv surgery.
rememberLog(`[${tag}] venv shim still locked after 15s; aborting hand-off (something outside this app holds the venv)`)
return { unlocked: false }
}
@ -2250,7 +2266,20 @@ async function applyUpdates(opts = {}) {
// spawn the updater. Without this the updater races a still-locked
// hermes.exe (held by the backend child / its grandchildren) and the update
// bricks. See releaseBackendLockForUpdate for the full failure analysis.
await releaseBackendLockForUpdate(updateRoot)
const lock = await releaseBackendLockForUpdate(updateRoot)
if (!lock.unlocked) {
// Something OUTSIDE this app holds the venv (a second window, a user
// terminal running hermes, an unkillable child). Handing off anyway
// guarantees a half-updated venv — abort loudly instead and let the
// user close the holder and retry. Restart our own backend so the app
// keeps working after the failed attempt.
const message =
'Update aborted: another process is holding the Hermes install open ' +
'(a second Hermes window or a terminal running hermes?). Close it and retry.'
emitUpdateProgress({ stage: 'error', message, percent: null })
startHermes().catch(() => {})
return { ok: false, error: message }
}
// Detached so the updater outlives this process — it needs us GONE before
// `hermes update` will run (the venv shim is locked while we live).