Turbopack: take next/dynamic loader availability from chunking for the loadable manifest (#101172)
## Summary
Fixes #100293. Fixes #99149.
With Turbopack, a route's `react-loadable-manifest.json` could name
`next/dynamic` chunks that the client build never emits. The SSR HTML
then preloads them (`<link rel="preload" as="script">`) and every SSR of
that route requests a 404. The page still works, but the `next/dynamic`
island loads later than it should.
**Why:** the manifest didn't read the async loaders that chunking
emitted. `collect_next_dynamic_chunks` rebuilt each loader for the
manifest and re-derived the availability it should be created with,
duplicating logic in `make_chunk_group`. Two known drifts:
- **#100293 (production, App Router):** a client component with
`next/dynamic` imports is rendered by a layout and by the page, and the
page makes some of the target's modules available. The manifest used the
page's chunk group availability, but the loader that is actually emitted
is created by the layout's chunk group.
- **#99149 (dev):** `make_chunk_group` creates loaders with
`in_async_module()` availability when nested async availability is off,
but the manifest didn't apply that flag. #99150 fixes this by applying
the flag in the manifest too.
Both are the same class of bug. This PR removes the re-derivation
instead of mirroring each rule.
**How:**
- `make_chunk_group` records, for every async loader it creates, the
canonical availability it passes to `async_loader_chunk_item`. The
record is a keyed cell (`AsyncLoaderAvailabilityMap`) read per module.
- `ChunkGroupResult` carries one record per chunk group, in load order.
`concatenate` keeps them all, so a page's concatenated group includes
the loaders its layouts created. The first group in load order that
created a loader wins. Usually later groups see it as available and
create none; when several create one, they share a module id and the
runtime keeps the first factory registered.
- `collect_next_dynamic_chunks` builds the eager loader the manifest
needs from that recorded availability. It no longer applies
`in_async_module()`, canonicalizes, or selects a segment. App Router
reads the parent client reference's concatenated group; Pages Router
reads the page's and `/_app`'s actual chunk groups (see #100091).
- **Imports behind a plain `import()`:** a `next/dynamic` import inside
a module loaded with a plain `import()` gets its loader from that
module's async chunk group, not the parent's. The collector records the
plain `import()` path as a parent-linked tree, not as a vector cloned
for each visited module or import. Each import stores only its innermost
plain `import()`. The resolver follows that path, starting from the
parent's result:
- It takes each hop's recorded availability and gets that hop's chunk
group from `AsyncLoaderModule::chunk_group_result`. That method is now
shared with the emitted loader, including its "already available" check,
so it returns the same cached result.
- It resolves each hop's chunk group at most once per manifest
computation, then reuses it for all imports beneath that hop.
- It searches the groups on the path outermost-first, matching runtime
load order. Each lookup can therefore read the path's ancestors, but it
no longer rescans the same prefix for every hop.
- If no record exists for an import, the build fails with a diagnostic
naming the module and the number of chunk groups searched. It does not
guess an availability; guessing was shown to reproduce the bad
filenames.
**Prior work:** #88775 preserved a module → async loader mapping on
`ChunkGroupResult` for the same reason. It was reverted in #89073 after
a build failed with `Dynamic entry not found in async loaders`; #89121
is the re-apply draft. That failure was reportedly Linux-only and **has
not been reproduced here**, so this PR does not claim to fix it. Its
main difference from #88775 is that it records the loader availability
rather than the loader itself, and stores it in keyed cells. A missing
record still fails the build, with a diagnostic. Additional validation
against a large app is still pending.
Tests are ported from #88775 and #99150, thanks to both.
## Verification
- `test/e2e/app-dir/next-dynamic-shared-client-reference` (new): a
layout and the page render the same client component with an SSR and an
`ssr: false` `next/dynamic` import. Fails without the fix in Turbopack
`next start` (a 404 preload and two manifest files that don't exist).
- `test/e2e/app-dir/next-dynamic-behind-async-import` (new):
`next/dynamic` imports, SSR and `ssr: false`, in a module loaded through
`React.lazy(() => import(...))`. It covers:
- App Router: one plain `import()`, two of them, a loader reused from an
outer chunk group, and an outer module the layout already loads.
- Pages Router: one plain `import()`.
The first version of this PR failed this suite with a build error;
canary and this version pass it. A fixture for the reused outer loader
also failed an intermediate version that only searched the innermost
group.
- `test/e2e/app-dir/dynamic` async-import cases (from #99150), App and
Pages Router: the nested async import now shares a module with its
`next/dynamic` target, so availability affects the emitted chunks. Both
fail without the fix in Turbopack dev.
- `test/e2e/next-dynamic-app` (from #99150) and
`test/development/nested-dynamic-components` (from #88775; the "skip if
no manifest" branch is now an assertion): these already pass on current
canary and act as regression guards.
- With a locally built native binary on the rebased branch, all five
suites pass in Turbopack dev and start and in webpack dev and start.
- Existing suites pass in Turbopack dev and start:
`app-dir/dynamic-css`, `next-dynamic`, `next-dynamic-css`,
`next-dynamic-lazy-compilation`, `pages-app-next-dynamic-preload`.
- A larger reproduction (a third-party library loaded with `import()` in
the targets, `cacheComponents: true`): every manifest file is emitted
and every preload returns 200.
- Negative controls (local only): bypassing the recorded availability
makes the build fail with the new diagnostic; bypassing it with a
guessed fallback reproduces the 404; skipping the plain `import()` hops
makes the new suite fail with the diagnostic.
- `cargo clippy` on `turbopack-core`, `turbopack-ecmascript`,
`turbopack-browser`, `turbopack-nodejs`, `next-core`, `next-api`; `cargo
test -p turbopack-core --lib`.
- Not run: deploy tests (the new HTTP and hydration assertions run there
in CI).
<!-- NEXT_JS_LLM -->
<!-- fleet 0836f4bf-2736-4809-be86-8f840a4fbb93 -->
---------
Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>