[turbopack] Sort modules in chunks to reduce duplicates (#94961)
# What
When building a chunk, lay out chunk_items in sorted order.
# Why
There are a number of ways turbopack can create chunk groups with
identical sets of chunk items, but different chunk parents or different
availability info. This means we cannot trivially deduplicate these
chunks. Heuristics inside of `make_production_chunks` means we also do
not always observe 'trivial merge opportunities' there either.
By canonically sorting these chunks we can ensure that we produce the
same content and thus select the same filenames.
See https://vercel.slack.com/archives/C09R44U5HQW/p1781814781879439, on
https://vercel.com/academy when switching to Turbopack, we started
shipping duplicate chunks that only differed by order. Now, module
ordering is fixed in chunks so that the file name hashes are consistent
and we don't ship duplicates.
For vercel-academy,
before:
| | Client JS | Client CSS | Client Source Maps | Server Bundled JS |
Server Unbundled | Server Source Maps |
| --------- | ------------------ | ------------------- |
------------------ | -------------------- | --------------------- |
-------------------- |
| **Total** | 41 files / 7.24 MB | 5 files / 580.57 KB | - | 286 files /
18.30 MB | 1700 files / 48.26 MB | 286 files / 78.63 MB |[4:54
PM]to:[4:54 PM]## Totals
after:
| | Client JS | Client CSS | Client Source Maps | Server Bundled JS |
Server Unbundled | Server Source Maps |
| --------- | ------------------ | ------------------- |
------------------ | -------------------- | --------------------- |
-------------------- |
| **Total** | 36 files / 4.43 MB | 5 files / 580.57 KB | - | 286 files /
18.30 MB | 1700 files / 48.26 MB | 286 files / 78.63 MB |