next.js

eca564f4 - turbopack: intern merge bitmaps in compute_merged_modules (#100236)

turbopack: intern merge bitmaps in compute_merged_modules (#100236) ### What? Interns the merge bitmaps in `compute_merged_modules` (turbopack-core, `module_graph/merged_modules.rs`). Each distinct `RoaringBitmapWrapper` gets a dense `u32` id once, after the fixed-point traversal. The `map chunk groups` and `exposed computation` traversals then key `chunk_lists` by that id and compare ids instead of bitmaps. This is the first of three stacked PRs that speed up `compute_merged_modules`: 1. intern bitmaps (this PR) 2. parallel merge of the chunk group lists (#100237) 3. reconciliation shortcuts (#100238) ### Why? `map chunk groups` visits every module of every chunk group (about 19.8 M visits on the bench app below). On each visit, the old code: - looked up `chunk_lists: FxHashMap<&RoaringBitmapWrapper, _>`, where `RoaringBitmapWrapper::hash` serializes the whole bitmap into the hasher; - compared the parent's and the child's bitmaps. In a CPU profile, `serialize_into` under that hash accounted for about two thirds of the phase's samples. ### How? - **`intern_bitmaps`:** a small private helper that maps each key to the id of its bitmap. Equal bitmaps (by `RoaringBitmapWrapper`'s `Eq`) get the same id. - **Equivalence:** id equality is exactly bitmap equality, so the lists, their order and the output are unchanged. - **Tracing:** an `intern bitmaps` span shows the cost of interning (~35 ms on the bench app). ### Results Bench app `bench/nested-deps-app-router-many-pages` at 32 CPUs: - **Builds:** cold `next build` with `taskset 0-31` and `TURBO_TASKS_AVAILABLE_PARALLELISM=32`, using release binaries built at the base commit (`6dda4371e`) with and without this change. - **Rounds:** 1 warm-up per arm, then 4 rounds in alternating order. - **Phase times:** read from the `overview` light trace. | Metric (s) | canary median | this PR median | Median paired change | Faster in | |---|---:|---:|---:|---:| | Wall | 54.72 | 53.13 | −4.7% | 4/4 | | "Compiled in" | 34.45 | 31.45 | −10.0% | 4/4 | | `compute merged modules` | 6.64 | 4.10 | −38.8% | 4/4 | | `map chunk groups` | 3.14 | 0.49 | −84.3% | 4/4 | | `merging chunk group lists` | 1.73 | 1.79 | +0.5% | 2/4 | | `reconciliation` | 0.93 | 0.94 | +0.6% | 2/4 | <details><summary>Per round (canary → this PR)</summary> | Round | Order | Wall | "Compiled in" | `compute merged modules` | `map chunk groups` | `merging chunk group lists` | `reconciliation` | |---|---|---:|---:|---:|---:|---:|---:| | 1 | canary first | 53.70 → 50.20 | 33.80 → 29.80 | 6.56 → 4.09 | 3.20 → 0.54 | 1.62 → 1.78 | 0.92 → 0.93 | | 2 | this PR first | 54.55 → 52.93 | 35.10 → 30.90 | 6.72 → 4.10 | 3.08 → 0.46 | 1.81 → 1.79 | 0.96 → 0.96 | | 3 | canary first | 54.89 → 53.33 | 33.30 → 32.00 | 6.48 → 3.88 | 3.01 → 0.48 | 1.65 → 1.67 | 0.92 → 0.91 | | 4 | this PR first | 58.91 → 54.70 | 35.70 → 32.80 | 7.08 → 4.35 | 3.27 → 0.50 | 1.91 → 1.91 | 0.94 → 0.95 | </details> - **Limits:** one VM, one app, 4 rounds, and no significance test. The light trace adds some overhead to every arm. - **Output:** the `.next` output (71,117 files) is identical to canary's. Only the per-build random preview-mode and server-action keys differ. ### Verification - New unit test `intern_bitmaps_gives_equal_bitmaps_equal_ids` (equal bitmaps built in a different order, different bitmaps, empty bitmaps, dense ids). - `cargo test -p turbopack-core` - `cargo test -p turbopack-tests --test snapshot --test execution -- scope_hoisting` (2 snapshot and 52 execution fixtures, with unchanged snapshots) - `cargo clippy -p turbopack-core --all-targets -- -D warnings` <!-- NEXT_JS_LLM --> <!-- fleet d8fee2f9-18db-4936-b59a-f8417b8f4bbc --> Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>