next.js

e2e8ba9a - turbo-trace-server: reduce allocation overhead when loading large trace files (#92896)

turbo-trace-server: reduce allocation overhead when loading large trace files (#92896) ### What? Reduces allocation overhead in `turbo-trace-server` when reading large trace files. Measured improvement: **114 MB/s → 156 MB/s** (~37% throughput increase) on a 16 GB trace with 46 million spans. ### Why? Profiling showed the reader thread spending significant time in heap allocation and deallocation during the initial bulk-load phase. The hot spots were: 1. `to_string()` / `into_owned()` in `process_internal_row_queue` allocating a new `String` for every span name, category, and arg key/value — even when the same string (e.g. `"turbo_tasks::function"`) appeared in millions of spans. 2. `Vec<SelfTimeEntry>::push` growing one-at-a-time, triggering many `realloc` calls. 3. `OnceLock<Box<SpanTimeData>>` — inherent to the lazy-init design; mitigated by the allocator switch. 4. `Vec<TraceRow>::push` in the deserialization loop re-growing during the initial read. 5. `process_internal_row` allocating a fresh `Vec` queue on every call (one per row). 6. `Vec<Span>::push` and hash map rehashing in `Store::add_span`. 7. `into_static()` cloning forcing borrowed `Cow` strings to heap for queued rows. ### How? **1. mimalloc as global allocator (`main.rs`, `Cargo.toml`)** Added `turbo-tasks-malloc` as a dependency with a default-on `custom_allocator` feature (matching the pattern in `turbopack-cli` and `turbopack-nft`). Set `#[global_allocator] static ALLOC: TurboMalloc = TurboMalloc`. Also wired `TurboMalloc::thread_stop()` via rayon's `exit_handler` so each worker thread returns its mimalloc thread-local heap to the global pool on exit. **2. `RcStrInterning` in `turbo-rcstr`** New `RcStrInterning` struct backed by `FxHashSet<RcStr>`. Methods: `intern(&str)`, `intern_cow(Cow<str>)`, `intern_display(&Display)`. Strings shorter than the inline threshold (7 bytes on 64-bit) are already zero-allocation inline atoms and skip the set entirely. **3. `String` → `RcStr` for span fields** Changed `Span.name`, `Span.category`, and `Span.args` key/value types from `String`/`Vec<(String, String)>` to `RcStr`/`Vec<(RcStr, RcStr)>`. Same for `SpanNames` and `SpanBottomUpBuilder`. All `to_string()` / `into_owned()` calls in `process_internal_row_queue` are replaced with `interner.intern()` / `intern_cow()` / `intern_display()` calls on a per-`TurbopackFormat` `RcStrInterning`. Repeated names like `"turbo_tasks::function"` now share a single allocation via refcount bump instead of a fresh heap allocation each time. The `into_static()` path for queued rows also benefits: cloning an already-interned `RcStr` is a refcount increment, not a string copy. **4. Capacity hints** Pre-allocated major collections to avoid repeated reallocation during the initial bulk load: - `Store::spans`: `Vec::with_capacity(131_072)` - `TurbopackFormat::id_mapping`: `with_capacity(131_072)` - `TurbopackFormat::outdated_spans`: `with_capacity(8_192)` - `TurbopackFormat::queued_rows`: `with_capacity(1_024)` - `TurbopackFormat::thread_stacks`, `thread_allocation_counters`: `with_capacity(64)` - `TurbopackFormat::self_time_started`: `with_capacity(256)` **5. Reused queue Vec (hot spot #8)** `TurbopackFormat` now holds `row_queue: Vec<InternalRow<'static>>` as a field. `process_internal_row` takes it via `take` and puts it back after the loop, retaining the allocated capacity across calls instead of allocating a new `Vec::new()` for every row processed. **6. `TraceFormat::create_reused()` hook (hot spot #4)** Added an overridable `create_reused()` method to the `TraceFormat` trait. `TurbopackFormat` overrides it to return `Vec::with_capacity(4_096)`, giving the row deserialization buffer reasonable initial capacity to avoid repeated doubling during the initial bulk read. **7. `SelfTimeTree` leaf capacity (hot spot #2)** New child nodes created at split time get `Vec::with_capacity(SPLIT_COUNT / 2)` = 64 entries, matching the expected post-split occupancy and avoiding incremental doubling after every split. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>