next.js

292a3fba - Report OS memory pressure in TurboMalloc and trace samples (#93333)

Report OS memory pressure in TurboMalloc and trace samples (#93333) ### What? Adds a new `TurboMalloc::memory_pressure()` method that returns a normalized OS-level memory pressure value in the range `0..=100` as `Option<u8>`. That value is attached to every memory sample in the tracing layer (`TraceRow::MemorySample`) and propagated through the trace-server so that span queries return a `memory_pressure_samples` vector next to the existing `memory_samples`. ### Why? Our current tracing only records the in-process allocator usage (`TurboMalloc::memory_usage()`), which does not tell us when the *operating system* is actually under memory pressure. We want that signal in the trace output to: 1. Surface real OS memory pressure in trace dashboards alongside our own allocation totals. 2. Eventually use it as input to task-eviction decisions in `turbo-tasks` (see branch description). This PR lands the plumbing; the eviction heuristic is not part of this change. ### How? **`TurboMalloc::memory_pressure() -> Option<u8>`** — new, in `turbopack/crates/turbo-tasks-malloc/`. Values are normalized so that `0` = no pressure, `100` = maximum pressure. Platform-specific backends: | Platform | Source | Notes | |---|---|---| | Linux | `/proc/pressure/memory` (`some` `avg10`), fallback to `(MemTotal - MemAvailable) / MemTotal` from `/proc/meminfo` | PSI is not available on all kernels (< 4.20, without `CONFIG_PSI`, restricted containers). The meminfo fallback keeps the signal meaningful on any standard Linux system and matches the semantics of Windows' `dwMemoryLoad`. | | macOS | `kern.memorystatus_level` sysctl (% free memory) | Pressure = `100 - level`, read via `libc::sysctlbyname`. | | Windows | `MEMORYSTATUSEX::dwMemoryLoad` via `GlobalMemoryStatusEx` (`windows-sys`) | Already a 0–100 percentage of physical memory in use. | | Other / wasm | — | Returns `None`. | All runtime failures (missing file, sysctl error, failed API call, unparseable content) silently yield `None` rather than panicking. **Wiring into tracing:** - `TraceRow::MemorySample` gains a `memory_pressure: u8` field. `0` is used when `memory_pressure()` returns `None` on unsupported platforms. - `RawTraceLayer::maybe_report_memory_sample` populates it on every sample (sampling cadence unchanged). - This is a breaking change to the postcard wire format of `MemorySample`; old trace files cannot be read by the new `turbopack-trace-server`. Given the dev-only nature of this data that seemed acceptable — let me know if a migration is desired. **Wiring into the trace-server:** - `Store::memory_samples` is now `Vec<(Timestamp, u64, u8)>` and `add_memory_sample(ts, memory, memory_pressure)`. - A new `Store::memory_pressure_samples_for_range(start, end) -> Vec<u8>` mirrors `memory_samples_for_range`: same `MAX_MEMORY_SAMPLES = 200` cap and same group-and-max downsampling, so both vectors align index-by-index for a given span query. - `ServerToClientMessage::QueryResult` gains a `memory_pressure_samples: Vec<u8>` field next to `memory_samples`. **Dependencies:** `libc` (macOS only, `cfg`-gated), `windows-sys` with the `Win32_System_SystemInformation` feature (Windows only, `cfg`-gated). No new deps on Linux. ### Verification - `cargo build -p turbo-tasks-malloc -p turbopack-trace-utils -p turbopack-trace-server` - `cargo clippy -p turbo-tasks-malloc -p turbopack-trace-utils -p turbopack-trace-server --all-targets -- -D warnings` - `cargo test -p turbo-tasks-malloc` — 7 tests pass, including: - `memory_pressure_is_in_range`: asserts `Some(_)` and `≤ 100` on Linux, macOS and Windows (via `cfg`-gated `.expect()`), and allows `None` elsewhere. - Parser tests for both PSI and `/proc/meminfo` code paths (typical content, malformed input, clamping). - Runtime sanity check on the Linux sandbox: `/proc/pressure/memory` is absent (kernel 5.10 without `CONFIG_PSI`); the `/proc/meminfo` fallback returned `Some(3)` as expected. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>