turbo-tasks-backend: stop checking keyed dependencies once a dependent is stale (#101736)
## What?
When a cell changes, stop checking a dependent task's remaining keyed
dependencies once it has already been marked stale (and therefore
dirty).
## Why?
#101489 (flat `EdgeSet` for task edges) reduced edge-storage memory and
made initial keyed registration faster, but completed tasks now keep
their forward edges in a flat `EdgeSet`. `invalidate_cell_dependents`
checks that set once per changed key, and each `EdgeSet::contains` is a
linear scan. Changing M keys that one task reads can therefore take
O(M²) comparisons.
On the `keyed_deps` benchmark (1,000 reader tasks × 10,000 keys),
changing all keys went from about 5 s to about 34 s.
### Attribution
Release builds of the `keyed_deps` benchmark, 3 interleaved runs per
case and thread count. The benchmark source is identical in all builds.
Both read modes, shared and disjoint keys, and the initial / change all
/ change unread / change shift phases were run with correctness
assertions and a memory measurement after each phase.
| Commit | Meaning |
|---|---|
| `c893228b` | Immediately before #101489 |
| `9f6975f7` | #101489 (immediately before #101591) |
| `cad154f1` | #101591; benchmark binary **byte-identical** to
`9f6975f7` |
#101591 (Next.js config defaults and docs) does not affect this
benchmark; identical binaries are not counted as separate samples.
"Change all" phase:
| Case | Threads | Before #101489 | At #101489 |
|---|---:|---:|---:|
| Keyed reader, shared keys | 16 | 5.00 s | 33.23 s |
| Keyed reader, shared keys | 32 | 4.67 s | 34.08 s |
| Per-key `get`, shared keys | 16 | 15.37 s | 40.23 s |
| Per-key `get`, shared keys | 32 | 14.97 s | 42.44 s |
| Keyed reader, disjoint keys | 16 | 8.50 s | 37.89 s |
| Keyed reader, disjoint keys | 32 | 8.17 s | 38.50 s |
| Per-key `get`, disjoint keys | 16 | 18.77 s | 44.25 s |
| Per-key `get`, disjoint keys | 32 | 20.03 s | 45.17 s |
Profile (`perf record -F 499`, keyed reader, shared keys, 16 threads,
initial + change all):
- At #101489: **91.7% of self samples in `update_cell`**, vs **6.2%**
before it.
- The hot loop resolves through `slice_contains` → `EdgeSet::contains`
(`edge_set.rs:182`) → `cell_dependencies_keyed_contains` →
`invalidate_cell_dependents` (`update_cell.rs:276`). The scan dominates
the profile, and invalidation is effectively serial.
- #101489 also makes initial keyed-reader registration ~2.9–3.3× faster
and lowers peak RSS by ~0.3 GiB. This PR keeps that representation
instead of reverting those gains.
## How?
In `invalidate_cell_dependents`, break out of the per-dependent key loop
as soon as `make_stale = true`.
Why this is correct:
- Stale already implies dirty, so checking the remaining keys cannot
change the outcome for that task.
- Keys that match only an outdated dependency don't set `make_stale`, so
they still continue through the loop.
- With `task_dirty_cause` enabled, the dirty cause is still built from
**all** changed keys, not only the ones inspected before the break.
## Results
Base: canary `dcb536d0`. Same workload and phases, 3 interleaved
before/after runs for each case at 16 and 32 threads, memory collected
after each timed phase. Times are medians; speedups are the median of
**paired** ratios, so they need not exactly equal the ratio of the
displayed medians. A machine restart happened between complete runs,
never inside a before/after pair.
"Change all" phase:
| Case | Threads | Canary | This PR | Paired speedup [min–max] |
|---|---:|---:|---:|---:|
| Keyed reader, shared keys | 16 | 34.17 s | 0.685 s | **50.31×
[47.02–51.49]** |
| Keyed reader, shared keys | 32 | 34.12 s | 0.656 s | **52.05×
[50.77–53.98]** |
| Per-key `get`, shared keys | 16 | 41.47 s | 6.21 s | **6.66×
[6.53–6.75]** |
| Per-key `get`, shared keys | 32 | 41.57 s | 7.73 s | **5.23×
[5.20–6.12]** |
| Keyed reader, disjoint keys | 16 | 38.08 s | 4.21 s | **9.11×
[7.62–9.92]** |
| Keyed reader, disjoint keys | 32 | 37.21 s | 3.70 s | **10.06×
[9.95–10.14]** |
| Per-key `get`, disjoint keys | 16 | 43.33 s | 9.28 s | **4.71×
[4.64–4.94]** |
| Per-key `get`, disjoint keys | 32 | 44.24 s | 10.19 s | **4.31×
[3.91–4.37]** |
<details>
<summary>Other phases: paired speedups (16 threads / 32
threads)</summary>
| Case | Initial | Change all (2nd) | Change unread | Change shift |
Change shift (2nd) |
|---|---|---|---|---|---|
| Keyed reader, shared keys | 0.97× / 0.98× | 51.19× / 51.66× | 1.01× /
1.01× | 1.02× / 1.22× | 1.02× / 1.16× |
| Per-key `get`, shared keys | 1.04× / 1.09× | 6.54× / 5.42× | 1.01× /
1.02× | 1.01× / 1.12× | 1.03× / 1.16× |
| Keyed reader, disjoint keys | 0.99× / 0.95× | 8.41× / 10.43× | **0.83×
/ 1.01×** | 0.99× / 1.02× | 1.03× / 1.10× |
| Per-key `get`, disjoint keys | 1.05× / 1.13× | 4.74× / 3.82× | 1.00× /
0.95× | 1.05× / 0.96× | 1.09× / 0.99× |
The 16-thread "keyed reader, disjoint keys" change-unread result is
slower in all three pairs (0.76–0.88×), even though that case never
takes the new early exit. The cause is not established, so no blanket
speedup is claimed.
</details>
### Memory
Mimalloc **committed** (not live) bytes per edge; median of 3 samples
after the second "change shift". Peak RSS is the process high-water
mark.
| Case | Threads | Canary B/edge | This PR B/edge | Canary peak RSS |
This PR peak RSS |
|---|---:|---:|---:|---:|---:|
| Keyed reader, shared keys | 16 | 95.3 | 95.2 | 0.79 GiB | 0.78 GiB |
| Keyed reader, shared keys | 32 | 122.0 | 122.2 | 0.91 GiB | 0.92 GiB |
| Per-key `get`, shared keys | 16 | 95.4 | 95.5 | 0.78 GiB | 0.78 GiB |
| Per-key `get`, shared keys | 32 | 121.6 | 121.5 | 0.89 GiB | 0.89 GiB
|
| Keyed reader, disjoint keys | 16 | 135.1 | 135.8 | 1.19 GiB | 1.19 GiB
|
| Keyed reader, disjoint keys | 32 | 142.9 | **157.3** | 1.34 GiB | 1.33
GiB |
| Per-key `get`, disjoint keys | 16 | 140.3 | 136.9 | 1.20 GiB | 1.19
GiB |
| Per-key `get`, disjoint keys | 32 | 156.1 | 151.3 | 1.29 GiB | 1.29
GiB |
The higher retained commit for 32-thread "keyed reader, disjoint keys"
is not attributed. This change adds no data structure, but
timing-dependent allocator or scheduling effects have not been ruled in
or out.
## Tests
- Two new backend integration tests: a task that reads many changing
keys gets the new values and re-executes exactly once, for both the
keyed reader and per-key `get`.
- These are correctness tests; removing the early exit keeps them
passing. Moving the break before `make_stale = true` makes both fail.
- fmt and clippy `-D warnings`, including with `task_dirty_cause`.
- `turbo-tasks` + `turbo-tasks-backend`: 623 tests passed.
- The benchmark asserts outputs and reader re-execution counts in every
measured phase.
- Not measured here: real builds, persistent-cache/snapshot costs, and
exhaustive concurrent-GC interleavings.
<!-- NEXT_JS_LLM -->
<!-- fleet 86487b82-3b2f-4464-84da-b3be3db57b3a -->
Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>