bench(desktop): trustworthy cold-start measurement (code-splitting is not the lever) (#67720)

* bench(desktop): measure the full picture — prod build, cold-start, first-token

Stop drip-feeding scenarios: extend the harness to cover the latencies that
actually dominate perceived speed, and measure them on a REAL production build.

- --prod: build a production renderer with the probe included (VITE_PERF_PROBE=1,
  off in normal builds) and launch it from dist/. Measures minified React, so
  numbers are representative shipped figures instead of ~3x-inflated dev ones.
- cold-start scenario (tier "cold"): launch → CDP → driver → first paint, via a
  fresh isolated spawn per run. Captures spawn_to_cdp_ms, spawn_to_driver_ms, fcp_ms.
- first-token scenario (backend tier): Enter → first assistant token painted —
  the TTFT latency an agent app is uniquely judged on.
- run.mjs gained --prod (build once), cold-start fresh-spawn loop, and gates
  ci+cold tiers against the baseline.

Baseline re-captured on a PRODUCTION build (median of 5), darwin-arm64 — all
green. Representative numbers:
  cold-start  spawn→interactive ~1.6s, FCP ~0.5s
  stream      frame p95 22ms, 1 longtask
  keystroke   p50 2ms, p95 8.7ms
  transcript  mount 145ms, 82ms longtask (400-msg open)

The prod build also settled the open question from the dev numbers: the
transcript-mount "lead" (221ms longtask in dev) is only ~72-82ms in prod — not
actionable. Measurement did its job.

* bench(desktop): trustworthy cold-start measurement (code-splitting is NOT the lever)

Investigated code-splitting the ~22MB renderer bundle to cut cold start. It is
the wrong fix on both counts:

1. Intentional design: vite.config disables codeSplitting because Shiki emits
   thousands of dynamic chunks and electron-builder OOMs scanning them — a
   packaging/installer constraint, not an oversight.
2. The data says it wouldn't help. Fixing the cold-start measurement to be
   trustworthy and reading the boot composition (prod build):
     spawn → interactive ~1.5s
     renderer nav → DOMInteractive ~0.8s, → DOMContentLoaded ~1.06s
   so the whole 22MB bundle EVAL is only ~0.27s (DCL − DOMInteractive) of the
   ~1.5s. The dominant costs are Electron/window startup and React app mount —
   neither touched by splitting.

The measurement fixes (the real content of this PR — no app change, since the
optimization was rejected):
- Drop HERMES_DESKTOP_BOOT_FAKE from spawned instances — it injected artificial
  per-phase boot-overlay sleeps that inflated cold-start (and slowed every run).
- Unique debug/dev port per cold-start run — a just-killed instance can hold
  :9222 briefly, so reusing it made CDP attach to the DYING instance and report
  garbage (spawn_to_cdp of ~4ms). Stepping the port per run fixes the race.
- Richer boot marks (dom_interactive, dom_content_loaded, main-script size) so
  cold-start composition is visible, not just a single number.
- Forward all numeric boot marks from the cold-start loop.
- Re-baseline cold-start with the clean numbers.

A real cold-start win would target Electron startup / app-mount (e.g. V8 code
cache, deferred non-critical mount) — a future pass, now that it's measurable.

apps/desktop/scripts/perf/baseline.json

@@ -1,9 +1,9 @@
 {
   "_meta": {
-    "note": "Median of 5 runs, darwin-arm64, `npm run perf -- cold-start stream keystroke transcript --spawn --prod` — a PRODUCTION renderer (minified React), so these are representative shipped numbers, not dev-inflated. Re-baseline per device with the same command + `--update-baseline`. Tolerances are loose to absorb cross-machine variance; cold-start especially varies with disk/OS state.",
+    "note": "Median of 5 runs, darwin-arm64, `--spawn --prod` (PRODUCTION minified renderer, real boot — no fake-boot). Representative shipped numbers, not dev-inflated. cold-start uses a fresh unique-port spawn per run and its marks are process-spawn wall clock (spawn_to_*) or renderer nav-relative (dom_*). Re-baseline per device with `--update-baseline`; tolerances are loose for cross-machine/disk variance.",
     "platform": "darwin-arm64",
     "node": "v24.11.0",
-    "updated": "2026-07-19T21:38:19.701Z"
+    "updated": "2026-07-19T22:59:21.605Z"
   },
   "scenarios": {
     "stream": {
@@ -49,9 +49,11 @@
         "tolAbs": 150
       },
       "metrics": {
-        "spawn_to_cdp_ms": 326,
-        "spawn_to_driver_ms": 1647,
-        "fcp_ms": 500
+        "spawn_to_cdp_ms": 1098,
+        "spawn_to_driver_ms": 1482,
+        "dom_interactive_ms": 794,
+        "dom_content_loaded_ms": 1057,
+        "nav_to_read_ms": 1209
       }
     }
   }

apps/desktop/scripts/perf/lib/launch.mjs

@@ -170,7 +170,6 @@ export async function startIsolatedInstance({
   hermesHome,
   userDataDir,
   seedConfig = true,
-  bootFakeStepMs = 120,
   settleMs = 2500,
   connectTimeoutMs = 90000
 } = {}) {
@@ -232,11 +231,13 @@ export async function startIsolatedInstance({
     // Isolated Electron: own --user-data-dir (single-instance lock scope) + own
     // HERMES_HOME (backend + sessions). No DEV_SERVER env in prod → dist load.
     const electronBin = require('electron')
+    // NB: do NOT set HERMES_DESKTOP_BOOT_FAKE here — it injects artificial
+    // per-phase sleeps into the boot overlay, which inflates cold-start timing
+    // (and adds pointless startup latency to the steady-state runs). We want the
+    // real boot sequence.
     const env = {
       ...process.env,
       HERMES_HOME: home,
-      HERMES_DESKTOP_BOOT_FAKE: '1',
-      HERMES_DESKTOP_BOOT_FAKE_STEP_MS: String(bootFakeStepMs),
       XCURSOR_SIZE: '24'
     }
 
@@ -332,17 +333,26 @@ async function readBootMarks(cdp) {
     return await cdp.eval(`(() => {
       const paints = performance.getEntriesByType('paint')
       const fcp = paints.find(p => p.name === 'first-contentful-paint')
+      const nav = performance.getEntriesByType('navigation')[0]
       const composer = document.querySelector('[data-slot="composer-rich-input"]')
+      // Largest script resource ≈ the (intentionally single) renderer bundle.
+      // responseEnd → the script's own decode; the eval cost shows up as the gap
+      // between the bundle's responseEnd and domInteractive.
+      const scripts = performance.getEntriesByType('resource').filter(r => r.initiatorType === 'script')
+      const mainScript = scripts.sort((a, b) => (b.encodedBodySize || 0) - (a.encodedBodySize || 0))[0]
+      const round = n => (typeof n === 'number' ? Math.round(n) : null)
       return {
-        fcp_ms: fcp ? Math.round(fcp.startTime) : null,
-        // performance.now() at read time ≈ time since nav start; only meaningful
-        // right after boot (cold-start reads it immediately).
-        nav_to_read_ms: Math.round(performance.now()),
+        fcp_ms: fcp ? round(fcp.startTime) : null,
+        dom_interactive_ms: nav ? round(nav.domInteractive) : null,
+        dom_content_loaded_ms: nav ? round(nav.domContentLoadedEventEnd) : null,
+        main_script_kb: mainScript ? round((mainScript.encodedBodySize || 0) / 1024) : null,
+        main_script_response_end_ms: mainScript ? round(mainScript.responseEnd) : null,
+        nav_to_read_ms: round(performance.now()),
         composer_present: !!composer
       }
     })()`)
   } catch {
-    return { fcp_ms: null, nav_to_read_ms: null, composer_present: false }
+    return { fcp_ms: null, dom_interactive_ms: null, composer_present: false }
   }
 }
 

apps/desktop/scripts/perf/run.mjs

@@ -152,9 +152,13 @@ async function main() {
     const perRun = []
 
     for (let i = 0; i < runs; i++) {
-      const inst = await startIsolatedInstance({ port, devPort, prod, coldStart: true })
-      const t = inst.timings
-      perRun.push({ spawn_to_cdp_ms: t.spawn_to_cdp_ms, spawn_to_driver_ms: t.spawn_to_driver_ms, fcp_ms: t.fcp_ms })
+      // Unique debug + dev port per run: a just-killed instance can keep :9222
+      // held for a beat, and reusing it makes the next CDP.connect attach to the
+      // dying instance (garbage timings). Stepping the port sidesteps the race.
+      const inst = await startIsolatedInstance({ port: port + i, devPort: devPort + i, prod, coldStart: true })
+      // Forward every numeric boot mark; only the baseline keys are gated, the
+      // rest (dom_interactive, main_script_kb, …) are reported for composition.
+      perRun.push(Object.fromEntries(Object.entries(inst.timings).filter(([, v]) => typeof v === 'number')))
       inst.teardown()
     }