next.js

c3dd28fd - [ci] Split up large cache-components-dev-warmup test suite (#95553)

[ci] Split up large cache-components-dev-warmup test suite (#95553) This splits two of our slowest test suites into smaller pieces that can be run in parallel and retried in isolation. ## Context Jest treats test suites as the smallest test execution unit. Slow test suites can lead to situations where a test shard is using very little CPU waiting for one last test suite to finish, and when tests fail and have to be retried, we have to retry the entire suite. Here's a particularly bad example: <img width="3116" height="2634" alt="Screenshot 2026-06-08 at 20-00-05 CI Telemetry — turbopack production tests (1)" src="https://github.com/user-attachments/assets/b5534200-4b77-40d5-a219-4b6fc552ed77" /> I instructed Claude to fetch test timing information from GitHub Actions Artifact, and to report the slowest test suites. > How timings work today > > - The `fetch-test-timings` job runs `node run-tests.js --timings --write-timings`, which pulls per-file durations from a Vercel KV store and uploads them as the test-timings artifact (test-timings.json, 5-day retention). > - Each sharded job downloads that artifact and run-tests.js -g N/M greedily bin-packs whole test files into groups by predicted duration (run-tests.js:528-549). > - I pulled the artifact from the July 7 canary run (28843797348): 1684 test files, ~61,500s total, median 20s, but max 1023s. > > Since the file is the unit of scheduling, a shard can never finish faster than its largest file. The job durations from that run confirm the imbalance this causes: test dev 4/10 took 28 min while siblings took 10–17; test cache components dev 6/6 took 13 min vs 5–6 for siblings; test turbopack production 5/7 took 20 min vs 10–11. > The slowest suites ``` ┌────────────────────┬────────────────────────────────────────────────────────────────────────────┐ │ Time │ Suite │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 1023s + 933s │ test/development/app-dir/cache-components-dev-warmup/ (both variants) │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 942s │ test/production/create-next-app/templates/matrix.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 788s + 432s + 232s │ test/e2e/app-dir/cache-components-errors/cache-components-errors.*.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 779s │ test/development/acceptance-app/rsc-build-errors.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 470s │ test/e2e/opentelemetry/instrumentation/opentelemetry.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 457s │ test/e2e/app-dir/actions/app-action-node-middleware.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 451s │ test/e2e/app-dir/next-after-app/index.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 447s │ test/e2e/app-dir/css-order/css-order.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 426s │ test/e2e/telemetry/config.test.ts │ ├────────────────────┼────────────────────────────────────────────────────────────────────────────┤ │ 402s │ test/production/debug-build-path/debug-build-paths.test.ts │ └────────────────────┴────────────────────────────────────────────────────────────────────────────┘ ``` > Reading these files, almost every one is slow for the same structural reason: a `describe.each(...)` over fixtures/modes/runtimes where each iteration boots its own `nextTestSetup` (dev server or full production build), run serially within one Jest worker: > - **cache-components-dev-warmup:** `describe.each` over 2 fixture dirs, and it restarts the dev server (stop/clean/start) before every one of ~17 tests. It's slow by design (cache-warmup isolation), so per-test cost can't shrink — but the file can be split. > - **create-next-app matrix:** `describe.each(app|pages)` × `it.each` over a ~16–48-combination flag matrix, each doing a full CNA scaffold (+ dev-server cycle for pages), fully serial. > - cache-components-errors.test.ts: 7490 lines, 129 tests, with per-describe `next build --experimental-build-mode compile` and per-path generate builds in prod mode. > - **css-order:** three sibling `describe.each` blocks over 4–6 chunking modes, each mode booting its own server, ~3 blocks × modes × page-pair orderings. > - **next-after-app, opentelemetry:** `describe.each` over runtimes/configs, again one server per iteration.