migrate turbo-tasks to scattered collect (#94503)
# What
This uses `scatter_collect` from @mmastrac to register all values,
traits and value impls of traits. now all data can be consistently
accessed when constructing the VALUEs registry, eliminating a number of
racy access opportunities.
# Why
Scatter collect uses link sections to gather registries at link time,
this resolve some inherent brittleness in constructor functions since
the relative execution order is **not guaranteed** and it is also not
even possible to _detect_ execution that is too early (early access to
an `inventory` collection will miss some entries silently)
We have a single report that appears to be due to a race like this
```
web:dev: thread 'tokio-rt-worker' (41574444) panicked at turbopack/crates/turbo-tasks/src/macro_helpers.rs:274:13:
web:dev: no trait impl registered for value type turbopack_core::module_graph::ModuleGraphImportTracer
web:dev: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
I do not entirely understand how this is possible since constructors
should complete prior to the first napi entry point (other than
`napi::module_init` which is itself a constructor).
Our initialization logic is fundamentally brittle since it relies on
* an `inventory` of all value types
* an `inventory `of all trait types
* a set of ctor functions to register trait implementations (basically
(value, trait) pairs)
All of these need to be initialized/invoked before the `VALUES` registry
is accessed, but the above panic implies that the ctor function for
`ModuleGraphImportTracer` to register its implementation of
`ImportTracer` hasn't occurred yet.
By moving the registies to link time we completely eliminate all startup
races since the linker does all the 'gathering', now the only possible
concern is that we attempt to downcast a trait prior the the `VALUES`
registry completing which does not appear to be possible.
# Binary size impact
Measured on `next-swc.darwin-arm64.node` built with `pnpm --filter
@next/swc build-native-release` (release profile), this branch vs
`canary`. Stripped = `strip -x`; gzip = `gzip -9` of the stripped
binary.
| Metric | branch | canary | Δ | Δ% |
|---|---:|---:|---:|---:|
| **raw** | 122,879,888 B | 124,184,976 B | **−1,305,088 B** |
**−1.051%** |
| **stripped** | 83,505,264 B | 84,127,232 B | **−621,968 B** |
**−0.739%** |
| **gzip+stripped** | 28,800,547 B | 28,925,679 B | **−125,132 B** |
**−0.433%** |
The addon shrinks on every metric. Replacing the generated per-impl
registration functions (and their `ctor`/`inventory` machinery) with
link-time scattered collection removes real code/data — ~622 KB even
after stripping — and laying the registries out contiguously at link
time compresses better, so the gzip win holds up too.
<!-- NEXT_JS_LLM_PR -->
