next.js
7ea59208 - Turbopack: add max_chunk_size to production chunking (#100478)

Commit
20 hours ago
Turbopack: add max_chunk_size to production chunking (#100478) ### What? Adds `max_chunk_size` to Turbopack's production chunking config as a best-effort upper bound on JavaScript chunk size. Merging respects the limit; chunks already oversized on their own are split deterministically. Next.js enables separate defaults: - **Client:** 1,000,000 bytes, overridable with `experimental.turbopackChunking.maxChunkSize`. - **Server (Node.js and edge):** 10,000,000 bytes, overridable with `experimental.turbopackChunking.maxServerChunkSize`. With minification and `turbopackMinifyBeforeChunking` enabled, the defaults scale to **400,000 / 4,000,000 bytes** to match upstream minified sizing. Explicit settings are not scaled. Each option affects only its own side; `0` disables it. ### Why? Production chunking can produce arbitrarily large chunks from shared dependencies or merging. These take longer to load and parse, and changing one module invalidates the whole chunk. A merge-aware limit and deterministic splits keep chunks manageable while reducing cache churn. ### How? **Deterministic grouping**, based on webpack's `deterministicGrouping`, simplified to a scalar size: - When splitting is needed, extract items sized at least the maximum into standalone groups, then sort the rest by module identifier. - Recursively cut where adjacent identifiers share the shortest prefix. Break ties by balanced size, then the leftmost cut. - Keep both sides at least `min_chunk_size` where possible. If that conflicts with the maximum, prefer smaller chunks over oversized ones. - Preserve input order on disabled/no-split paths. Use `SmallVec` for the outer result and consume the original partition buffers without recollecting payloads. **Production integration:** - Oversized chunks never become merge candidates; pair merges exceeding the maximum are skipped. - If leftover candidates together exceed the limit, group them deterministically as whole candidates, keyed by their smallest module identifier. Their components stay intact. - Only original, unmerged oversized chunks need item-level splitting, after expanding batches. Split parts carry no batch groups. - Generate component chunks in `make_chunk`, preserving component output for merged and remainder chunks. - Staying under the maximum takes priority over `min_chunk_size` and `max_chunk_count_per_group`. - CSS and development chunking are unchanged. Disabling the limit preserves previous output. **Normalization:** when the limit is enabled, raise `max_chunk_size` to at least `3 * min_chunk_size`, then cap `max_merge_chunk_size` at that maximum (including an unlimited merge cap). Next.js warns with option names and original/adjusted values. Node.js uses the configured `minServerChunkSize` and `maxMergeServerChunkSize`; their defaults scale with the sizing mode. Edge uses an internal 20 KB minimum (8 KB when minifying before chunking) and the effective maximum as an implicit merge cap. Both Node and edge context variants resolve defaults before normalization. The maximum uses Turbopack's existing uncompressed size estimate: unminified by default, or minified when minify-before-chunking applies. It does not cap emitted file bytes. A single indivisible module or scope-hoisted item can still exceed it. ### Tests - Grouping tests cover constraints, max-first fallback, exact-limit extraction, oversized/zero-size items, ordering, prefix cuts, tie-breaking, non-ASCII identifiers, locality and inline empty/single-group storage. Config tests cover normalization and the sizing-mode/explicit-override matrix. - Execution tests replace the four max-size snapshots, checking emitted byte bounds with a framing allowance and module membership: package boundaries, no-merge splitting, capped merging despite a one-chunk goal, and candidate-level remainder grouping. Each fixture fails when the maximum is disabled. Existing execution coverage checks top-level-await and dynamic imports across split chunks. - The local CLI suite covers client/server isolation, normalization warnings, finite/unlimited merge caps, component chunks and minified-mode defaults/overrides: **19 tests pass** both normally and with Cache Components. - A deploy-active e2e fixture covers both minify-before-chunking modes: direct rendering with explicit client/server limits, lazy client hydration, controlled client navigation without a document reload, and fetching public JS URLs to verify payload separation. **8 tests pass locally** both normally and with Cache Components; dev smoke passes 6 runtime checks. All 15 hosted changed-test deployment jobs passed. Logs confirm the new fixture executed in real deploy mode in adapter, non-adapter and Cache Components variants. - Fresh native/framework build, root/Next type checks, formatting, ESLint, strict Clippy, 67 Bitset tests, 256 core tests (2 ignored), 89 Next-core tests, 629 ECMAScript tests, 26 snapshots and 451 execution tests pass. <!-- fleet 28d4c923-3384-4a31-9b95-e4b68ec76808 --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Author
Parents
Loading