next.js

ba894dd1 - Turbopack: release freed memory right after project shutdown (#101529)

Turbopack: release freed memory right after project shutdown (#101529) ## What? After `project.shutdown()` resolves in a Turbopack `next build`, return the memory that was freed by dropping the turbo-tasks storage to the OS right away. Until now it stayed resident for another 6–10s. Part of #100250. ## Why? `TurboTasksBackend::stop` already drops the task storage (`storage.drop_contents()`). Even so, on a large generated App Router app (5000 routes, 32 CPUs) the main process stayed at **~12.5–13 GB RSS** once shutdown resolved. It only fell to ~1 GB in steps over the next 6–10s, while the static generation workers were already loading their page bundles. That overlap directly raises the peak memory of the build's process tree. Root cause, found with RSS timelines, mimalloc commit stats and thread counts: - mimalloc keeps each page in the heap of the thread that allocated it. A free from another thread only queues the block on that page, and the owning thread has to collect its heap before the page becomes free. - Tokio scheduler workers collect on every park (`on_thread_park` → `TurboMalloc::thread_park`). **Blocking-pool threads have no such hook.** - Many blocking-pool threads used to be scheduler workers that did compilation and persistence work. They handed their worker role to a new thread in `block_in_place` (used by `scope_bounded`) and own a large share of the heap. During shutdown the process grows from 44 to 75 threads. - Those threads only collect when they exit after Tokio's 10s idle keep-alive. RSS dropped in step with the thread count (75 → 44). - A single `TurboMalloc::collect(true)` on one thread released only ~7.5 GB of the ~13 GB, because it cannot process pages owned by other live threads. - JS-held napi references were not the cause. The `Project` External is garbage-collected ~1.5s after shutdown, and dropping it doesn't change RSS. ## How? - New `turbo_tasks::memory::release_freed_memory()`: - It reads Tokio's `num_idle_blocking_threads` metric. This needs `tokio_unstable`, which `.cargo/config.toml` already enables for all targets. - It runs one `spawn_blocking` task per idle blocking thread. A bounded rendezvous makes the tasks land on distinct threads, and each one calls `TurboMalloc::collect(false)` to process its own heap. - A single `collect(true)` on a blocking thread then purges the freed pages of all threads to the OS. - `TurboMalloc::collect(true)` now initializes the calling thread's mimalloc heap (`mi_thread_init()`) first. - mimalloc's `mi_collect` skips everything, including the process-wide purge, when the calling thread's heap is uninitialized. - That is the case on a fresh blocking-pool thread that never allocated. - It rarely happens on canary, where many initialized blocking threads are idle. With fewer blocking threads (e.g. without `block_in_place`), the purge was frequently a no-op and ~12 GB stayed resident. - `mi_thread_init()` is idempotent, and `collect(false)` (the park hook) is unchanged. - It is best effort and cannot hang: the rendezvous is bounded (1s), and any thread it misses still collects when it exits. - `project_shutdown` calls it (traced as `release freed memory`) after `stop_and_wait` and the exit handlers. Persistence and compilation-event delivery are therefore already done. The JS thread is not blocked, and the dev server's `project_on_exit` path is unchanged. Alternatives evaluated for a more general mid-build/dev variant: - Tokio has no idle hook for blocking-pool threads. - Collecting at the end of `scope_bounded` / `block_in_place` is too early. It can't process frees that arrive after the thread has gone idle, and the new integration test demonstrates this. - Shortening the blocking-pool keep-alive to 1s still left 5.4 GB at shutdown resolution and would add thread churn. ### Tests - `turbopack/crates/turbo-tasks/src/memory.rs` has unit tests: - each idle blocking thread is visited exactly once and reused rather than new threads being spawned; - zero visits spawn no threads; - a capped blocking pool times out instead of hanging. - `turbopack/crates/turbo-tasks/tests/release_freed_memory.rs` is a Linux test in its own binary with mimalloc as the global allocator: - An idle blocking thread owns 256 MB of half-used pages. They are freed from another thread, and the test checks they stay resident even after a forced collect on both threads. - `release_freed_memory()` then returns them to the OS. - The test fails without the release call. - `turbopack/crates/turbo-tasks-malloc/tests/forced_collect.rs` is a Linux test in its own binary: - 512 MB is freed on a thread that has since exited. - `collect(true)` runs on a fresh thread that never allocated, and the test checks the memory is returned. - Without the `mi_thread_init()` fix it fails: 557 MB stays resident. ## Benchmarks > **Update after rebasing onto canary f4ea7a70 (mimalloc v3.6.1, #101518):** the tables below were measured on the previous allocator. > - **Why this PR alone no longer reaches ~1 GB:** v3.6.1 defaults `allow_thp` to 2. On Linux with THP available, its minimal purge size becomes 2 MiB, so the small free ranges of the fragmented heap are never purged, not even by the forced collect here. Details: https://github.com/vercel/next.js/pull/101518#issuecomment-6819311577 > - **What this PR alone does on v3.6.1** (smoke, 32 CPUs): main RSS at resolve 12.24 GB, vs 13.84 GB on canary. > - **The fix is #101623**, at the top of the stack #101529 → #101589 → #101623. It lowers mimalloc's purge minimum only around this PR's final collect. > - **Full stack vs canary** (`bench/large-app-router`, cold, n=3): main RSS at resolve 1.05 ± 0.04 GB (16 CPUs) and 1.17 ± 0.01 GB (32 CPUs), vs 13.15 / 14.49 GB. Peak tree RSS −5.74 / −13.24 GB. > - The `mi_thread_init` fix here is still needed on v3.6.1: `tests/forced_collect.rs` still fails without it. Setup: - Same machine (32 vCPU = 16 physical cores × 2 SMT, 64 GB RAM), cold builds (`.next` removed), `TURBO_ENGINE_IGNORE_DIRTY=1`. - next-swc built natively in release mode for both base and change, on the same canary commit (4d51724b7c). - Base and change interleaved within each iteration, with the order alternating between iterations. - 16 CPUs = `taskset -c 0,2,…,30` (one per physical core); 32 CPUs = `0-31`. - RSS sampled from `/proc` every 100 ms. "Tree" is main + all workers. - Values are mean ± stddev. ### `bench/large-app-router` (generated, 5000 routes; n=6 per cell) | CPUs | | wall (s) | peak tree RSS (GB) | main RSS at `shutdown()` resolve (GB) | +1s | +3s | peak tree RSS after shutdown (GB) | | --- | --- | --- | --- | --- | --- | --- | --- | | 16 | base | 148.4 ± 25.9 | 24.08 ± 1.77 | 12.32 ± 1.16 | 12.32 ± 1.16 | 12.32 ± 1.16 | 23.09 ± 1.28 | | 16 | **new** | 141.9 ± 26.8 | **22.11 ± 2.42** | **0.92 ± 0.07** | 0.92 ± 0.07 | 0.92 ± 0.07 | **13.07 ± 0.12** | | 32 | base | 126.2 ± 17.6 | 23.11 ± 4.82 | 12.56 ± 0.80 | 9.03 ± 2.76 | 5.24 ± 0.68 | 18.89 ± 0.30 | | 32 | **new** | 118.3 ± 17.7 | **21.55 ± 4.09** | **2.37 ± 3.42**¹ | 1.32 ± 0.84 | 0.97 ± 0.08 | 18.80 ± 0.13 | ¹ 0.8–1.1 GB in 5 of 6 runs. In one run, memory freed after the sweep took until +2.6s to be purged. Machine speed drifted between iterations, so absolute walls vary a lot. The paired (same-iteration) new − base deltas are more reliable: | app | CPUs | Δ wall (s) | Δ peak tree RSS (GB) | | --- | --- | --- | --- | | large-app-router | 16 | −6.6 ± 8.7 | −2.0 ± 3.0 | | large-app-router | 32 | −7.8 ± 2.5 | −1.6 ± 6.3 | | nested-deps-app-router-many-pages | 16 | −0.2 ± 2.4 | −3.3 ± 1.2 | | nested-deps-app-router-many-pages | 32 | +0.1 ± 1.7 | −1.7 ± 0.6 | ### `bench/nested-deps-app-router-many-pages` (n=4 per cell) | CPUs | | wall (s) | peak tree RSS (GB) | main RSS at `shutdown()` resolve (GB) | peak tree RSS after shutdown (GB) | | --- | --- | --- | --- | --- | --- | | 16 | base | 37.4 ± 1.6 | 14.56 ± 1.18 | 3.72 ± 1.36 | 14.56 ± 1.18 | | 16 | **new** | 37.3 ± 1.0 | **11.29 ± 0.12** | **0.41 ± 0.01** | 11.29 ± 0.12 | | 32 | base | 32.9 ± 0.7 | 16.99 ± 0.57 | 2.57 ± 0.64 | 16.99 ± 0.57 | | 32 | **new** | 33.1 ± 1.4 | **15.27 ± 0.03** | **0.47 ± 0.00** | 15.27 ± 0.03 | The tables above were measured before the fresh-thread `collect(true)` fix. A release-build sanity run with the fix (large-app-router, 32 CPUs) measured: - wall: 96.6 s - peak tree RSS: 19.0 GB - main RSS at `shutdown()` resolve: 0.98 GB Peak main-process RSS during compilation is unchanged (14.0–14.2 GB on large-app-router, ~4.5 GB on nested-deps). No configuration showed a peak RSS regression. ### Note for work that starts static workers earlier (e.g. a worker fork server) - This only releases memory **after** `project.shutdown()` resolves: from about 13 GB to about 1 GB within ~1.3s of the storage drop. - `project.shutdown()` itself takes 22–30s on large-app-router, almost entirely writing the filesystem cache. - Page-data collection already runs concurrently with that write. The main process necessarily holds its full ~13.6 GB heap until the write finishes. - The bimodal ~27 GB peak tree RSS on large-app-router always occurs inside that window, before shutdown resolves, in both base and new. - Workers that load bundles during that window will still overlap with the full Turbopack heap. Reducing memory during the final persist is separate work. ## Verification - `cargo test -p turbo-tasks --lib memory` - `cargo test -p turbo-tasks --test release_freed_memory` - `cargo test -p turbo-tasks-malloc` - `cargo clippy -p turbo-tasks -p turbo-tasks-malloc -p next-napi-bindings --all-targets` - `cargo check -p turbo-tasks -p turbo-tasks-malloc --lib --target wasm32-wasip1-threads` - `pnpm test-start-turbo test/e2e/filesystem-cache/build-cache-default.test.ts` (persistent build cache still written and read back) - Manual: a warm rebuild of nested-deps reuses the filesystem cache (compile 16.4s → 5.5s). - Not run: the browser-based `filesystem-cache-app-changes` tests (no Playwright browser available in my environment). <!-- NEXT_JS_LLM --> <!-- fleet 0e159516-3c7f-4582-a520-97d8b272ee82 --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>