next.js
5102fd07 - turbo-persistence: StreamingSstWriter performance and readability improvements (#90692)

Commit
8 days ago
turbo-persistence: StreamingSstWriter performance and readability improvements (#90692) ## Summary Performance and readability improvements to `StreamingSstWriter` in turbo-persistence, based on a performance audit of the streaming SST file builder. ### Pre-allocate Vec fields (33eb3e8f) - Pre-allocate `compress_buffer`, `pending_small_value_block`, `key_buffer`, `block_offsets`, and `key_block_boundaries` with capacity estimates derived from `max_entry_count` - Eliminates ~10-15 reallocations during early writes (e.g. for 1M-entry compaction files, the pre-allocation is ~89 KB across all Vecs) ### Readability cleanup (3de7a45a) - Extract `KeyBlockAccumulator` struct from four loose `current_key_block_*` fields, with `should_flush`/`add`/`reset` methods - Add architectural diagram comment for `pending_keys` queue layout showing resolved vs. unresolved regions - Name magic constant `BLOCK_INDEX_CAPACITY_BUFFER` (was bare `16`) - Simplify `flush_remaining_key_blocks()` using local `KeyBlockAccumulator` and remove dead state resets after `close()` - Remove `try_flush_key_block()` — its logic is now split between `KeyBlockAccumulator::should_flush()` and inline flush calls, which is clearer - Deduplicate `TestEntry` constructors via shared `new()` helper ### Derive value block estimate from constants (580e5d4c) - Replace magic number `40` (entries per value block) with `MIN_SMALL_VALUE_BLOCK_SIZE / AVG_SMALL_VALUE_SIZE` (= 128) - Add `AVG_SMALL_VALUE_SIZE = 64` constant ## Test Plan - All 14 existing `static_sorted_file_builder` tests pass: `cargo test -p turbo-persistence --lib -- static_sorted_file_builder` - No behavioral changes — all changes are internal to the writer
Author
Parents
Loading