next.js
72a62492 - turbo-persistence: validate sst_inspect block reads with BlockOffsets (#100313)

Commit
14 hours ago
turbo-persistence: validate sst_inspect block reads with BlockOffsets (#100313) ## Summary #100288 moved SST block-directory validation into `BlockOffsets`: the whole directory is read with `pread` and validated once when a file is opened, and every block read goes through `BlockOffsets::block_range`. That covers the runtime readers this PR originally hardened. What's left here is the `sst_inspect` tool, which still read SST files with unchecked arithmetic, plus end-to-end tests. - **`sst_inspect` reuses the runtime validation.** `BlockOffsets` is now public, with `read` and `block_range` (both documented). `sst_inspect` builds its block directory with `BlockOffsets::read` and locates every block through `block_range`. Before this change it panicked on several kinds of corrupt file: | Corrupt file | Before | | --- | --- | | File shorter than its block directory | subtract with overflow | | Zero blocks | `block_count - 1` overflow | | Out-of-range block offset | slice panic | | Out-of-range block index | slice panic | | Index block ending inside the directory | reported as a checksum mismatch | All of these now return a descriptive error. - **Zero-block files.** `BlockOffsets::read` accepts an empty directory, and the runtime rejects it separately when reading the index block. `sst_inspect` now rejects zero blocks explicitly before it looks for the index block. - **Decompressed-length mismatch.** In `sst_inspect` this returns an error instead of failing an `assert_eq!`. - **End-to-end open tests (coverage only).** These open corrupt files through `StaticSortedFile::open` in every lookup access mode and through the file-backed `StaticSortedFileIter::open` (iteration uses `pread` since #100336): - empty and truncated files; - zero blocks; - an index block that is past the end, overlaps the directory, is too short, ends before it starts, or is header-only; - a real SST with each non-index directory entry corrupted. Current canary already rejects all of these, so they only guard against regressions. Runtime reader behavior is unchanged. ## Verification - `cargo test -p turbo-persistence --all-targets`, plus runs with `--features strict_checks` and with `--no-default-features`. - `cargo clippy -p turbo-persistence --all-targets` in all three configurations, and `cargo doc` with `-D warnings`. - On canary without this change: - the new `sst_inspect` tests fail as listed above; - the runtime open tests pass when only the test code is applied. <!-- NEXT_JS_LLM --> <!-- fleet 011b2031-27d3-4027-861a-603404820e49 --> Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Author
Parents
Loading