perf(ci): sparse+blobless checkout for the test-slice generate job

The "Generate slices" job spent ~90% of its wall time in actions/checkout
(4-27s across the last 10 runs, of a 10-32s job) pulling a ~212MB working
tree. Everything in the test matrix waits on it.

It doesn't need those files. `--generate-slices` returns before
`_approximately_count_tests`, so it only ever uses test file *paths* plus
the cached durations — it never opens a test file.

The blocker was discovery: `_discover_files` rglobs the filesystem, which
finds nothing under a sparse checkout. So add `--discover-from-git`, which
lists paths via `git ls-files`. Sparse checkout only clears the worktree
(entries are marked skip-worktree), so the index still carries every path
and enumerates exactly the same set. Skip-part filtering and the
root-override rule are duplicated to match `_discover_files` semantics.

Measured on a real clone of this repo:

    full depth=1 clone:       7s   212M
    blobless+sparse clone:    3s   3.4M   (still sees all 2510 test paths)

Both discovery paths produce byte-identical slice JSON over the full 2472
test files, so slice assignment is unchanged.

Tests assert the properties that make this safe: the two discovery paths
agree on a full checkout, the git path still works when the files are
absent from disk, and the skip-part override behaves the same either way.
This commit is contained in:
ethernet 2026-07-31 13:26:27 -04:00
parent f702bba63c
commit 4e3768ea36
3 changed files with 176 additions and 2 deletions

View file

@ -26,6 +26,17 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# This job only needs test file *paths* — it computes the LPT
# distribution and never opens a test file. So skip the blobs
# (~212MB working tree -> ~3MB) and materialize only the script
# itself. `--discover-from-git` below lists the test paths from
# the git index, which a sparse checkout leaves fully populated.
# Checkout was ~90% of this job's wall time, and every test slice
# waits on it.
filter: blob:none
sparse-checkout: scripts/run_tests_parallel.py
sparse-checkout-cone-mode: false
- name: Restore duration cache
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@ -42,7 +53,7 @@ jobs:
- name: Generate test slices
id: matrix
run: |
MATRIX=$(python3 scripts/run_tests_parallel.py --generate-slices ${{ inputs.slice_count }})
MATRIX=$(python3 scripts/run_tests_parallel.py --generate-slices ${{ inputs.slice_count }} --discover-from-git)
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
test: