hermes-agent/apps/desktop/scripts/perf/baseline.json
brooklyn! 8142331616
bench(desktop): measure representative (warm-cache) cold start (#67733)
Profiling the boot answered "is there a real cold-start win?": no wasteful
hotspot — the renderer does only ~tens of ms of work at mount, no heavy library
(shiki/mermaid/katex/d3/motion) initializes at startup; the rest is Electron
runtime + waiting, near the Electron floor.

It also exposed that the cold-start number was pessimistic: a fresh
--user-data-dir per run means a COLD V8 code cache and worst-case bundle
recompile every launch. Real users reuse their profile. Measured delta:
  fresh (cold cache):  spawn→interactive ~1.48s
  reused (warm cache): ~1.0s
So representative launch is ~1.0s; only first-launch-after-install pays ~+400ms.

- coldStartSamples() reuses one profile (run 0 warms the cache, discarded;
  runs 1..N are warm samples), stepping ports + pausing so the single-instance
  lock releases. `--cold-fresh` measures the first-launch worst case.
- Re-baselined cold-start with the representative warm numbers.

Net: nothing high-ROI left to optimize. The only lever is shipping a pre-warmed
V8 code cache to make first launch match warm (~400ms, once per update) — real
packaging complexity for a marginal win, deliberately not pursued.
2026-07-19 23:21:45 +00:00

60 lines
1.7 KiB
JSON

{
"_meta": {
"note": "Median of 5 runs, darwin-arm64, `--spawn --prod` (PRODUCTION minified renderer, real boot — no fake-boot). Representative shipped numbers, not dev-inflated. cold-start reuses one profile so the V8 code cache is WARM (what users get after first launch, ~1.0s); a fresh-profile first launch is ~+400ms (measure with `--cold-fresh`). Marks are process-spawn wall clock (spawn_to_*) or renderer nav-relative (dom_*). Re-baseline per device with `--update-baseline`; tolerances loose for cross-machine/disk variance.",
"platform": "darwin-arm64",
"node": "v24.11.0",
"updated": "2026-07-19T23:16:01.227Z"
},
"scenarios": {
"stream": {
"tolerance": {
"tolFrac": 0.6,
"tolAbs": 5
},
"metrics": {
"longtasks_n": 1,
"longtask_max_ms": 67,
"frame_p95_ms": 22,
"frame_p99_ms": 23.7,
"slow_frames_33": 1,
"intermut_p95_ms": 36.1
}
},
"keystroke": {
"tolerance": {
"tolFrac": 0.6,
"tolAbs": 4
},
"metrics": {
"keystroke_p50_ms": 2.1,
"keystroke_p95_ms": 8.7,
"keystroke_p99_ms": 16.9,
"keystroke_slow_16": 2
}
},
"transcript": {
"tolerance": {
"tolFrac": 0.75,
"tolAbs": 40
},
"metrics": {
"transcript_mount_ms": 145,
"transcript_longtask_ms": 82,
"transcript_longtask_max_ms": 82
}
},
"cold-start": {
"tolerance": {
"tolFrac": 0.6,
"tolAbs": 150
},
"metrics": {
"spawn_to_cdp_ms": 606,
"spawn_to_driver_ms": 984,
"dom_interactive_ms": 324,
"dom_content_loaded_ms": 574,
"nav_to_read_ms": 721
}
}
}
}