Diagnosis from the profiler + runner logs: with the cgroup-aware
default (-j 16 on the 8-CPU pods) all 25 files run concurrently against
the single dind daemon. Every file stretches to ~900s wall (P50 892s,
CPU-wall 14549s vs 918s wall) and teardown docker-rm calls exceed their
10s timeout — the job dies on teardown errors while tests themselves
pass 53/53. The arm64 lane (2 CPU → -j 4) went green for exactly this
reason. Pin the amd64 lane to the same effective width.
- doctor: 'inside a container' branch rewrote terminal_env to local for
EVERY non-docker backend, so TERMINAL_ENV=vercel_sandbox diagnostics
vanished when doctor ran inside a container (CI runner pods). Scope
the informational skip to the implicit local case only; remote
backends keep their real diagnostics. Fixes
test_doctor_reports_vercel_backend_diagnostics on ARC runners — and
for actual users running doctor in the Docker distribution with a
remote terminal backend configured.
- termux audio test: detect_audio_environment() probes the real host
for containment; pin is_container=False (a Termux device is never a
container) so the containerized runner doesn't flip available=False.
os.cpu_count() reports the HOST cores. In an ARC runner pod
(limit 8 CPU on a 22-core node) the runner spawned -j 44 workers on
8 usable CPUs — ~5x oversubscription. Every 'timing flake' family on
the self-hosted runners (docker rm teardown TimeoutExpired x111,
compression fork, termux probe, pty reaper, session hygiene) is CPU
starvation from that oversubscription, not real test bugs.
Read cgroup v2 cpu.max (v1 cfs_quota fallback) and clamp to host count.
Verified: --cpus=8 container reports 8, bare host unchanged.
HERMES_TEST_WORKERS override still wins.
- test_supports_systemd_services_returns_true_when_systemctl_present:
pin is_container=False (host contract; CI runner pods are containers)
- test_systemd_restart_gracefully_restarts_running_service_and_waits:
stub _preflight_user_systemd — no user D-Bus in runner pods; the test
asserts restart choreography, not D-Bus reachability
Test fixes for ARC runner pods (containers) vs GHA ubuntu-latest VMs:
- test_gateway_wsl / test_copilot_acp_client: pin is_container=False on
host-behavior tests — runner pods ARE containers and the prod code
intentionally behaves differently there. Also drop inherited
HERMES_REAL_HOME so nix dev shells don't leak into the assert.
- honcho memo + skill-utils external-dirs cache: add st_size to the
mtime_ns cache keys. overlayfs (runner pods) coalesces rapid writes
into one mtime tick, so same-tick edits were served stale (3 honcho
pin tests + skill cache invalidation test).
Verified with KUBERNETES_SERVICE_HOST set to simulate the pod env.
Swap all `runs-on: ubuntu-latest` to `runs-on: arc-runner-set` all jobs.
The ARM docker build job in docker.yml uses `${{ matrix.runner }}`
and is left untouched since the GKE runner pool is x86_64 only.
Runners are backed by ARC (Actions Runner Controller) on a GKE cluster
with a spot preemptible node pool that scales based on job demand.
Use the baked Electron dependencies for the desktop E2E job.
The repo's .npmrc sets engine-strict=true and package.json pins
engines.npm, so an npm outside that range aborts every npm ci /
npm install we run inside the checkout:
npm error code EBADENGINE
npm error notsup Required: {"npm":"<11.10.0 || >=12.0.0"}
npm error notsup Actual: {"npm":"11.10.0"}
Our callers made that worse: _run_npm_install_deterministic sees
`npm ci` fail and falls through to `npm install`, which fails
identically, so the user got a buried EBADENGINE and no remedy.
React to the failure instead of predicting it. npm states the
required range in its own error, so there is no need for a version
probe on the happy path or a semver range matcher — the recovery
reads the constraint out of the output it just produced, upgrades,
and retries once.
Scope is deliberately narrow. Hermes only upgrades an npm inside its
own managed Node tree ($HERMES_HOME/node), installing with --prefix
so bin/npm keeps resolving to the upgraded lib/node_modules/npm; a
managed install writes prefix=~/.local into node/etc/npmrc, so
without the override the "upgrade" would land elsewhere while the
managed npm stayed stale. A system / nvm / brew / Nix npm belongs to
the user, so that case prints the exact command and lets the original
failure stand.
The upgrade runs from a temp cwd with npm_config_min_release_age=0,
otherwise the checkout's own min-release-age gate would refuse the
npm release we need.
_run_npm_install_deterministic's capture_output=False callers (the
desktop install) streamed npm output and returned stderr=None, which
would leave the recovery nothing to read — stderr is now teed, so
live output is unchanged and the text stays inspectable.
Verified end to end against real npm binaries on copies of a managed
tree: managed npm 11.10.0 -> EBADENGINE -> upgraded to 12.0.2 ->
retry exits 0; a foreign npm 11.10.0 hard-fails with the manual
command and is left untouched.
pin brace-expansion to 5.0.8
update concurrently to 10.0.4
update electron-builder to 26.15.3
update eslint to 10.8.0
update eslint-plugin-perfectionist to 5.10.0
update @assistant-ui/react to 0.15.0
update @assistant-ui/react-streamdown to 0.3.8
update radix-ui to 1.6.7
update react-router-dom to react-router@8.3.0 - react-router-dom is no longer a standalone package, it just reexports react-router
remove @radix-ui/react-slot: we import this from `radix-ui`
remove eslint-plugin-react: we imported it, but never actually used it!
The statusbar is now opt-in. Existing users with a stored preference
keep their choice; new users get a clean bottom edge. The way back is
the view.toggleStatusbar keybind or the ⌘K row, unchanged.
Dated snapshot of deepseek-v4-flash, live on both provider endpoints
(verified via OpenRouter /api/v1/models and Nous portal /v1/models).
Context (1M via deepseek-v4-flash prefix match), reasoning stale-timeout
floor (600s), and pricing (official_models_api live billing on both
routes) all resolve without new entries. model-catalog.json regenerated
via scripts/build_model_catalog.py.
The chronos cron-fire verifier constructed PyJWKClient without explicit
headers, so its JWKS fetch to the NAS portal hit the same WAF 403 the
dashboard-auth providers already guard against. It reaches the same
portal issuer, so it's the same bug class — mirror the fix here and add
a constructor-contract regression test.
Co-authored-by: James Hodgkinson <james@terminaloutcomes.com>
The Nous PyJWKClient was constructed without explicit headers, while the
self_hosted provider already sends Accept + User-Agent. Without them the
Portal WAF can block the JWKS fetch, so the same failure mode remained for
the Nous dashboard-auth route. Mirror the self_hosted fix and add a
constructor-contract regression test.
