perf: Move synchronous cache archiving off Tokio runtime workers (#14013)
## Summary
Closes TURBO-6051.
`AsyncCache` spawned ordinary Tokio tasks whose `CacheMultiplexer::put`
ran synchronous archive I/O and zstd compression inline: `FSCache::put`
is fully synchronous, and the HTTP path's archive construction was
synchronous inside an `async` fn. Large artifacts occupying runtime
workers starved unrelated process/output/network/timer work. Restores
had the same problem on the fetch path, which runs inline on the calling
task rather than on the cache worker pool.
## Changes
- `CacheMultiplexer::put` / `fetch`: local-cache reads, writes,
restores, and shared-archive installs now run inside
`tokio::task::spawn_blocking` (sub-caches are `Arc`-shared). Local fetch
errors still fall through to the remote cache exactly as before.
- `HTTPCache::put`: artifact body construction (read + compress + spool,
shared with #14012 as `ArtifactBody::from_files`) runs on the blocking
pool; the upload itself stays async. The combined local+remote write
path in the multiplexer builds the shared archive on the blocking pool
as well.
- `HTTPCache::fetch` / `fetch_with_archive`: tar extraction runs on the
blocking pool; the verified spool is dropped (and its temp file cleaned
up) there.
Concurrency bounds are unchanged: the existing `AsyncCache` worker
semaphore still limits concurrent cache jobs, so blocking-pool fanout
stays bounded by the configured cache workers on the write path. Error
propagation and shutdown draining (workers are awaited to completion)
are preserved; join failures surface as a new `CacheError::BlockingTask`
variant.
## Benchmarks
4 concurrent 64 MiB incompressible artifact puts (local + remote)
against the repo's mock server as a separate process, while a 5 ms
heartbeat task ticks on the same runtime. Debug build, macOS. Temporary
harness, not committed.
| Tokio runtime workers | Before: total / worst heartbeat gap | After:
total / worst heartbeat gap |
| --- | --- | --- |
| 1 | 2.26 s / **1.1 s** | 1.35 s / **6.3 ms** |
| 2 | 2.33 s / **1.4 s** | 1.38 s / **6.1 ms** |
| 8 | 2.72 s / 6.1 ms | 1.30 s / 6.1 ms |
Before: with 1–2 workers the heartbeat stalled for over a second (5 ms
target cadence). After: worst gap stays at ~6 ms regardless of worker
count, and total cache-save time improves even at 8 workers (archiving
no longer competes with upload progress/runtime work).
## Verification
- `cargo test -p turborepo-cache`: 159 passed, 0 failed (includes the
worker/shutdown draining tests in `async_cache.rs` and the
shared-archive round-trips from #14012).
- `cargo clippy -p turborepo-cache --all-targets`: clean.
- Rebased onto `main` after #14012 merged; conflicts resolved by
composing the two changes (shared `ArtifactBody` + blocking-pool
execution).