turbo
22b6577a - perf: Move synchronous cache archiving off Tokio runtime workers (#14013)

Commit
11 days ago
perf: Move synchronous cache archiving off Tokio runtime workers (#14013) ## Summary Closes TURBO-6051. `AsyncCache` spawned ordinary Tokio tasks whose `CacheMultiplexer::put` ran synchronous archive I/O and zstd compression inline: `FSCache::put` is fully synchronous, and the HTTP path's archive construction was synchronous inside an `async` fn. Large artifacts occupying runtime workers starved unrelated process/output/network/timer work. Restores had the same problem on the fetch path, which runs inline on the calling task rather than on the cache worker pool. ## Changes - `CacheMultiplexer::put` / `fetch`: local-cache reads, writes, restores, and shared-archive installs now run inside `tokio::task::spawn_blocking` (sub-caches are `Arc`-shared). Local fetch errors still fall through to the remote cache exactly as before. - `HTTPCache::put`: artifact body construction (read + compress + spool, shared with #14012 as `ArtifactBody::from_files`) runs on the blocking pool; the upload itself stays async. The combined local+remote write path in the multiplexer builds the shared archive on the blocking pool as well. - `HTTPCache::fetch` / `fetch_with_archive`: tar extraction runs on the blocking pool; the verified spool is dropped (and its temp file cleaned up) there. Concurrency bounds are unchanged: the existing `AsyncCache` worker semaphore still limits concurrent cache jobs, so blocking-pool fanout stays bounded by the configured cache workers on the write path. Error propagation and shutdown draining (workers are awaited to completion) are preserved; join failures surface as a new `CacheError::BlockingTask` variant. ## Benchmarks 4 concurrent 64 MiB incompressible artifact puts (local + remote) against the repo's mock server as a separate process, while a 5 ms heartbeat task ticks on the same runtime. Debug build, macOS. Temporary harness, not committed. | Tokio runtime workers | Before: total / worst heartbeat gap | After: total / worst heartbeat gap | | --- | --- | --- | | 1 | 2.26 s / **1.1 s** | 1.35 s / **6.3 ms** | | 2 | 2.33 s / **1.4 s** | 1.38 s / **6.1 ms** | | 8 | 2.72 s / 6.1 ms | 1.30 s / 6.1 ms | Before: with 1–2 workers the heartbeat stalled for over a second (5 ms target cadence). After: worst gap stays at ~6 ms regardless of worker count, and total cache-save time improves even at 8 workers (archiving no longer competes with upload progress/runtime work). ## Verification - `cargo test -p turborepo-cache`: 159 passed, 0 failed (includes the worker/shutdown draining tests in `async_cache.rs` and the shared-archive round-trips from #14012). - `cargo clippy -p turborepo-cache --all-targets`: clean. - Rebased onto `main` after #14012 merged; conflicts resolved by composing the two changes (shared `ArtifactBody` + blocking-pool execution).
Author
Parents
Loading