Turbopack: add max_chunk_size to production chunking (#100478)
### What?
Adds `max_chunk_size` to Turbopack's production chunking config as a
best-effort upper bound on JavaScript chunk size. Merging respects the
limit; chunks already oversized on their own are split
deterministically.
Next.js enables separate defaults:
- **Client:** 1,000,000 bytes, overridable with
`experimental.turbopackChunking.maxChunkSize`.
- **Server (Node.js and edge):** 10,000,000 bytes, overridable with
`experimental.turbopackChunking.maxServerChunkSize`.
With minification and `turbopackMinifyBeforeChunking` enabled, the
defaults scale to **400,000 / 4,000,000 bytes** to match upstream
minified sizing. Explicit settings are not scaled. Each option affects
only its own side; `0` disables it.
### Why?
Production chunking can produce arbitrarily large chunks from shared
dependencies or merging. These take longer to load and parse, and
changing one module invalidates the whole chunk. A merge-aware limit and
deterministic splits keep chunks manageable while reducing cache churn.
### How?
**Deterministic grouping**, based on webpack's `deterministicGrouping`,
simplified to a scalar size:
- When splitting is needed, extract items sized at least the maximum
into standalone groups, then sort the rest by module identifier.
- Recursively cut where adjacent identifiers share the shortest prefix.
Break ties by balanced size, then the leftmost cut.
- Keep both sides at least `min_chunk_size` where possible. If that
conflicts with the maximum, prefer smaller chunks over oversized ones.
- Preserve input order on disabled/no-split paths. Use `SmallVec` for
the outer result and consume the original partition buffers without
recollecting payloads.
**Production integration:**
- Oversized chunks never become merge candidates; pair merges exceeding
the maximum are skipped.
- If leftover candidates together exceed the limit, group them
deterministically as whole candidates, keyed by their smallest module
identifier. Their components stay intact.
- Only original, unmerged oversized chunks need item-level splitting,
after expanding batches. Split parts carry no batch groups.
- Generate component chunks in `make_chunk`, preserving component output
for merged and remainder chunks.
- Staying under the maximum takes priority over `min_chunk_size` and
`max_chunk_count_per_group`.
- CSS and development chunking are unchanged. Disabling the limit
preserves previous output.
**Normalization:** when the limit is enabled, raise `max_chunk_size` to
at least `3 * min_chunk_size`, then cap `max_merge_chunk_size` at that
maximum (including an unlimited merge cap). Next.js warns with option
names and original/adjusted values. Node.js uses the configured
`minServerChunkSize` and `maxMergeServerChunkSize`; their defaults scale
with the sizing mode. Edge uses an internal 20 KB minimum (8 KB when
minifying before chunking) and the effective maximum as an implicit
merge cap. Both Node and edge context variants resolve defaults before
normalization.
The maximum uses Turbopack's existing uncompressed size estimate:
unminified by default, or minified when minify-before-chunking applies.
It does not cap emitted file bytes. A single indivisible module or
scope-hoisted item can still exceed it.
### Tests
- Grouping tests cover constraints, max-first fallback, exact-limit
extraction, oversized/zero-size items, ordering, prefix cuts,
tie-breaking, non-ASCII identifiers, locality and inline
empty/single-group storage. Config tests cover normalization and the
sizing-mode/explicit-override matrix.
- Execution tests replace the four max-size snapshots, checking emitted
byte bounds with a framing allowance and module membership: package
boundaries, no-merge splitting, capped merging despite a one-chunk goal,
and candidate-level remainder grouping. Each fixture fails when the
maximum is disabled. Existing execution coverage checks top-level-await
and dynamic imports across split chunks.
- The local CLI suite covers client/server isolation, normalization
warnings, finite/unlimited merge caps, component chunks and
minified-mode defaults/overrides: **19 tests pass** both normally and
with Cache Components.
- A deploy-active e2e fixture covers both minify-before-chunking modes:
direct rendering with explicit client/server limits, lazy client
hydration, controlled client navigation without a document reload, and
fetching public JS URLs to verify payload separation. **8 tests pass
locally** both normally and with Cache Components; dev smoke passes 6
runtime checks. All 15 hosted changed-test deployment jobs passed. Logs
confirm the new fixture executed in real deploy mode in adapter,
non-adapter and Cache Components variants.
- Fresh native/framework build, root/Next type checks, formatting,
ESLint, strict Clippy, 67 Bitset tests, 256 core tests (2 ignored), 89
Next-core tests, 629 ECMAScript tests, 26 snapshots and 451 execution
tests pass.
<!-- fleet 28d4c923-3384-4a31-9b95-e4b68ec76808 -->
---------
Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>