Improve deduping of concurrent `'use cache'` invocations (#91830)
With #71286, we implemented deduping of cache entries under certain
circumstances. For example, the following constructed example was fixed
with the PR:
```js
async function getCachedRandom() {
'use cache'
return Math.random()
}
const rand1 = await getCachedRandom()
const rand2 = await getCachedRandom()
assert(rand1 === rand2)
```
However, this implementation relied on awaiting the two calls
sequentially. When rendering components, this can usually not be
guaranteed. E.g. the following example was not properly deduped:
```jsx
async function Cached() {
'use cache'
return <p>{Math.random()}</p>
}
export default function Page() {
return (
<>
<Cached />
<Cached />
</>
)
}
```
This did render the same value, but only because we triggered two render
passes, and the last cached value was used for both elements in the
final render pass. But during the first render pass, the `Cached`
function was called twice, and the cache entry was also set twice.
With #75786, we also fixed the render scenario by wrapping the cached
function in `React.cache`. This however did not work for route handlers.
E.g. the first example rewritten as follows, and used in a route
handler, still wouldn't be deduped:
```js
const [rand1, rand2] = await Promise.all([
getCachedRandom(),
getCachedRandom(),
])
```
Furthermore, with this solution, nested cached functions could not be
deduped across different outer cache scopes. This is because each cache
scope creates its own `React.cache` scope. Example:
```jsx
async function Inner() {
'use cache'
return <p>{Math.random()}</p>
}
async function Outer1() {
'use cache'
return <Inner />
}
async function Outer2() {
'use cache'
return <Inner />
}
export default function Page() {
return (
<>
<Outer1 />
<Outer2 />
</>
)
}
```
This PR introduces two-layer invocation tracking that deduplicates these
cases without changing how cache handlers are implemented. Only the
first invocation (the "leader") performs the cache handler lookup and
generation. Subsequent invocations ("joiners") tee the leader's result
stream instead.
The deduplication scope starts after the Resume Data Cache (RDC) lookup
and before the cache handler `get`. The RDC phase is excluded because it
throws synchronous errors (dynamic usage errors) that need individual
stack traces per call site, and RDC lookups are local with no network
savings from deduplication.
Intra-request deduplication is stored on the `WorkStore` and keyed by
`serializedCacheKey` (the coarse key, which is safe because root params
are identical within a single request). Cross-request deduplication is
stored in a module-scope map keyed by `cacheHandlerKey`, which may
include root params on the warm path. Cross-request joiners must await
metadata for root param verification before forking the stream. If the
key mismatches (different root params), the joiner retries with a
recomputed key. Because metadata is checked before `fork()` is called, a
mismatched joiner never consumes a stream from the wrong entry.
Stream tee-ing is lazy via the `SharedCacheEntry` class: the leader
calls `fork()` to get its copy, and each joiner calls `fork()` on
demand. If no joiners exist, only one tee occurs. The
`SharedCacheResult` discriminated union wraps either a
`SharedCacheEntry` (for the cached case) or a hanging promise (for
`prerender-dynamic`).
Each invocation still decodes the stream independently via
`createFromReadableStream` with its own `temporaryReferences`, because
sharing the decoded result would cause cache poisoning when components
receive different non-serializable props (e.g. `children`).
Admittedly, this adds significant complexity to the `cache()` function.
Two classes help keep it manageable:
- `SharedCacheEntry` encapsulates stream ownership and lazy tee-ing so
that call sites don't need to reason about which streams have been
consumed or need cloning.
- `ResolvableSharedCacheResult` manages the deferred promise, map
registration, and lazy cleanup. Entries stay in the dedup maps until
collection completes (so late-arriving invocations can join while the
leader streams), then clean up automatically on resolve or reject.
A follow-up refactoring to extract the cache handler lookup and
generation into a separate function would help break up the function's
size.
closes #78703
next.js
1066fbf7 - Improve deduping of concurrent `'use cache'` invocations (#91830)
