turbo

f0a0d775 - perf: Build cache archives once for local and remote writes (#14012)

perf: Build cache archives once for local and remote writes (#14012) ## Summary Closes TURBO-5986. Builds on the spooled-artifact transfer work merged in #14011. With both local and remote cache writes enabled, output files were opened, read, and compressed twice: `FSCache::put` built its own archive while `HTTPCache::put` built a second copy for upload. The fetch direction had the mirror problem: a remote hit restored files and then `FSCache::put` re-read and re-compressed every restored output to populate the local cache. ## Changes - New shared `ArtifactBody` (in `artifact_body.rs`): the canonical compressed archive, held in memory below 8 MiB and spooled to an anonymous temp file above it. Built once via `ArtifactBody::from_files`; every consumer gets a fresh positioned reader or bounded stream over identical bytes. - `CacheMultiplexer::put` with local+remote writes now builds the archive once, atomically installs it into the local cache (temp + rename, 1 MiB-buffered copy, same publication semantics as `CacheWriter::create`), and uploads the same bytes. Single-destination paths are unchanged. - `CacheMultiplexer::fetch` on a remote hit with local writes installs the *downloaded* archive bytes (signature verification gates any restore or local install) instead of re-encoding the restored files. A rejected download can never become a local hit. - `HTTPCache::put` splits into archive construction and `put_body`; `fetch_with_archive` returns the verified body alongside restored files. Signing is unchanged: the tag covers exactly the uploaded bytes. ## Benchmarks End-to-end through `AsyncCache` against the repo's mock remote as a separate process; debug build; macOS. Five interleaved runs per scenario (order alternated), medians reported. Harness was temporary and is not committed. Debug-build disk I/O on this machine is noisy; the many-small-files case is the most deterministic and the multi-hundred-MiB case is dominated by page-cache variance, so phase timings from an instrumented run are included for the large case. | Scenario | Phase | Before | After | Change | | --- | --- | --- | --- | --- | | 10,000 × 100 B files | put (local+remote) | 1,980 ms | 916 ms | −54% | | 10,000 × 100 B files | remote-hit fetch + local install | 2,125 ms | 1,395 ms | −34% | | 100 × 1 MiB files | put (local+remote) | 207 ms | 148–263 ms (noisy) | ~wash to −29% | | 100 × 1 MiB files | remote-hit fetch + local install | 254 ms | 197 ms | −22% | | 1 × 512 MiB file | put (local+remote) | 3,460 ms | 2,498 ms | −28% | | 1 × 512 MiB file | remote-hit fetch + local install | 4,156–4,530 ms | 3,222–3,672 ms | −14 to −25% | Instrumented phase breakdown, 512 MiB remote-hit fetch: local archive install (buffered copy of verified bytes) 338–641 ms, replacing a full read + zstd re-encode of restored outputs (~1.1–1.2 s in the same environment). Restore time dominates and is identical in both paths. ## Correctness preserved - Local publication stays atomic (temp file + rename); metadata/manifest sidecars are written exactly as before. - Signing still covers exactly the uploaded bytes; retries after token refresh resend byte-identical content. - New tests: local install and remote upload are byte-identical after a combined put; a remote-hit fetch installs byte-identical archive bytes locally. - `cargo test -p turborepo-cache`: 159 passed, 0 failed. `cargo clippy -p turborepo-cache`: clean.