turbo-tasks-fs: skip compare-before-write in directories created by the filesystem (#101813)
## Summary
Writing output files is the effect **apply** phase. In a cold `next
build` of `bench/large-app-router` (5k routes, about 175k write effects,
about 211k output files), it keeps only about 3.5–4 CPUs busy for
several seconds on 16 and 32 CPU machines.
**Why.** The bottleneck is in the kernel, not in Turbopack's concurrency
limits:
- About 108k of the files land in one directory,
`.next/server/chunks/ssr`.
- On Linux, `open(O_CREAT)` takes the parent directory's inode lock
exclusively. On XFS it also takes the allocation-group lock.
- Before each write, `DiskFileSystem` compares the new content with the
existing file. For a missing file that compare is an `open(O_RDONLY)`
that takes the same directory lock in shared mode, so it contends with
the concurrent creates.
- Off-CPU sampling of the Tokio workers during apply found:
- about 30% of context switches waiting on the directory rwsem from the
create;
- about 30% waiting on it from the compare-read;
- about 30% in `xfs_create → xfs_read_agi`.
- The write semaphore (#101519), the apply concurrency limit and the
blocking threads were not the limit.
- A C microbenchmark of 100k files in one directory with 16 threads:
create-only takes 2.0 s, compare-read then create takes 3.7–4.6 s.
**Fix.**
- `DiskFileSystem` records the directories it created itself in this
session, in a new `CreatedDirs` (`created_dirs.rs`). It is a
`DashSet<PathBuf>`, with no platform-specific code.
- A write whose parent directory is in that set skips the compare-read.
That check is just a set lookup, with no syscall.
- Skipping is safe because files in such a directory are only written by
this filesystem, and the per-key effect state already dedupes identical
writes before apply.
- `create_dir_all` on the existing `NotFound` retry path is replaced by
an equivalent level-by-level version that records what it created. A
directory deleted during the session is re-created and recorded again by
the next write into it.
- A full filesystem invalidation clears the set.
The set is only a performance hint. It assumes nobody else modifies
these directories while the `DiskFileSystem` is attached. If someone
does, the worst outcome is rewriting a file that already has identical
content. Content is never incorrect.
**Why warm builds improve too.** "Warm" here reuses the persistent
cache, but `next build` still cleans the dist dir before emitting. So
the output directories are re-created in every build, and the warm apply
phase takes the same create-heavy path. If the dist dir is no longer
cleaned up front (as #101521 proposes), most of the warm gain would go
away, while the cold gain stays.
## Benchmarks
Setup:
- Linux, XFS. CPU count limited via `taskset` and
`TURBO_TASKS_AVAILABLE_PARALLELISM`.
- Base is canary `8d1f4ed9`. Both binaries were built with the same
toolchain and profile, plus identical apply start/end markers for
measurement only (not part of this PR).
- Builds are interleaved base/PR. Results are mean [min–max].
- Warm cells use a shared `TURBO_ENGINE_VERSION`, so both binaries read
the same persistent cache. The two binaries have different git versions,
which would otherwise make the PR's first "warm" build a cold one.
- The host was noticeably slower and noisier during this matrix than in
earlier runs (cold large-app-router builds took about 105–120 s), so
compare within a row, not against older numbers.
| app · mode · CPUs | n | apply phase (s) | compile (s) | wall (s) |
main VmHWM (MiB) | tree peak RSS (MiB) |
|---|---|---|---|---|---|---|
| large · cold · 16 | 5 | 4.44 [4.04–4.79] → **3.28 [3.03–3.67] (−26%)**
| 80.8 → 79.0 | 121.5 → 120.7 | 12153 → 12174 | 17511 → 17633 |
| large · cold · 32 | 10¹ | 5.65 [4.79–6.68] → 3.98 [3.43–5.09] (−30%) |
66.5 → 66.6 | 108.1 → 108.2 | 12493 → 12553 | 21480 → 19490² |
| large · warm · 16 | 3 | 7.06 [5.11–9.29] → **4.40 [3.76–4.97] (−38%)**
| 10.4 [8.3–13.0] → 7.6 [6.7–8.4] (−27%) | 65.9 → 63.1 | 3196 → 3199 |
12453 → 12346 |
| large · warm · 32 | 3 | 11.27 [9.55–13.37] → **4.82 [4.35–5.44]
(−57%)** | 15.0 [12.8–17.6] → **8.3 [7.9–8.9] (−45%)** | 62.5 → **53.4**
| 3280 → 3276 | 18153 → 18034 |
| nested-deps · cold · 16 | 3 | – (60 effects) | 4.80 → 4.97 | 6.3 → 6.5
| 1031 → 1026 | 1266 → 1263 |
| nested-deps · cold · 32 | 3 | – | 4.83 → 4.73 | 6.3 → 6.1 | 1183 →
1188 | 1390 → 1411 |
| nested-deps · warm · 16 | 10³ | – | 0.16 → 0.16 | 1.4 → 1.4 | 430
[417–438] → 422 [412–432] | 618 → 620 |
| nested-deps · warm · 32 | 3 | – | 0.20 → 0.21 | 1.6 → 1.5 | 482 → 476
| 655 → 645 |
Notes:
- Bold marks changes whose min–max ranges don't overlap. No metric
regresses with non-overlapping ranges.
- Cold compile/wall changes are within noise; the apply phase is a few
seconds of a 60–80 s compile.
- ¹ At n=5, the large · cold · 32 apply ranges touched (base min 4.51 =
PR max 4.51), so I re-ran it once at n=10. Sorted apply times:
- base: 4.79 4.90 4.97 5.28 5.56 5.84 6.01 6.18 6.27 6.68
- PR: 3.43 3.65 3.74 3.76 3.80 3.89 4.01 4.01 4.45 5.09
So 9 of the 10 PR runs are faster than the fastest base run, and one
outlier overlaps.
- ² Tree peak RSS at large · cold · 32 is bimodal in both arms (about 18
GB or about 25 GB, depending on static-worker timing). The ranges
overlap.
- ³ At n=3, nested-deps · warm · 16 main VmHWM came out +1.9% with
non-overlapping ranges (435 → 443 MiB), even though only 60 effects are
applied. A single interleaved re-run at n=10 (shown in the table) showed
−1.7% with overlapping ranges.
- Disk: every measured build started with ≥15 GB free, except the warm
large-app-router runs, which started with 11.7–14.1 GB. The warm `.next`
with its persistent cache is about 7.6 GB, on a 32 GB disk.
Busy CPUs of the main process during the apply phase only (cold, 32
CPUs, n=5; per-run 250 ms means aligned to the apply start; zero after a
run has finished applying):
```mermaid
xychart-beta
title "cold, 32 CPUs: busy CPUs during apply"
x-axis "seconds since apply start" ["0.00", "0.25", "0.50", "0.75", "1.00", "1.25", "1.50", "1.75", "2.00", "2.25", "2.50", "2.75", "3.00", "3.25", "3.50", "3.75", "4.00", "4.25", "4.50", "4.75", "5.00", "5.25", "5.50", "5.75"]
y-axis "busy CPUs" 0 --> 8
line "canary" [4.22, 5.34, 5.06, 4.48, 4.09, 4, 3.92, 4.02, 3.93, 3.97, 3.91, 3.81, 3.89, 3.8, 3.67, 3.58, 3.85, 3.75, 2.85, 2.85, 3.1, 2.35, 0, 0]
line "this PR" [4.63, 5.59, 5.96, 6.16, 6.26, 6.24, 6.28, 6.22, 6.22, 6.21, 6.2, 5.91, 6.23, 6.09, 4.75, 4.81, 2.39, 1.12, 0, 0, 0, 0, 0, 0]
```
The apply phase stays bound by per-directory serialization of file
creation. Further gains would need the chunk output spread over
subdirectories, which changes the output layout and is out of scope
here.
Also measured and rejected: a per-directory async create lock on top of
this change. It was slower, with apply at 7.7 s, because lock handoff
serialized the creates.
## Verification
- New tests in `turbo-tasks-fs`, in `created_dirs.rs` and `disk.rs`
(`created_dirs_tests`). They run on all platforms and cover:
- `create_dir_all` records only the directories it created, and fails on
files;
- new content is written into created directories;
- pre-existing directories are still compared, and identical writes are
skipped;
- a directory deleted during the session is re-created by the next
write, and the content is correct;
- full invalidation clears the set.
- `cargo test -p turbo-tasks-fs`, clippy (`-D warnings`) and rustfmt.
Tested locally on Linux only; Windows is covered by CI.
- Output equivalence was checked on the previous revision of this PR,
which made the same skip decisions for directories created by this
filesystem, plus an extra directory-identity check. Cold builds, canary
vs PR, were compared file by file after replacing only the build ID and
per-build keys (preview props, server-action encryption key):
- nested-deps: 94 entries, 0 differences;
- large-app-router, with static generation serialized for this
comparison (`experimental.cpus: 1`, `staticGenerationMaxConcurrency:
1`): 211,210 files, 0 differences;
- with default static-generation concurrency, prerendered RSC/HTML row
ordering differs between builds even for canary vs canary. All chunks
are byte-identical.
<!-- NEXT_JS_LLM -->
<!-- fleet c034b90f-2e81-4204-9abc-8dbe8917d3e7 -->
---------
Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Vercel Fleet <fleet@vercel.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>