next.js
11823f84 - Add support for multi-valued tables (#89728)

Commit
7 days ago
Add support for multi-valued tables (#89728) ## What Add support for multi-valued tables in `turbo-persistence`. A multi-valued table allows multiple distinct values to be associated with a single key. Each family is independently configured as `SingleValue` (existing behavior) or `MultiValue` via the new `FamilyKind` enum. ## Why This will support the `TaskCache` table (implemented in #88904), where keys will change to be _hashes_ instead of full `TaskType` values. This greatly decreases DB size and speeds up queries due to smaller key sizes, at the cost of hash collisions requiring multiple values per key. ## How ### API - New `FamilyKind` enum (`SingleValue` / `MultiValue`) and per-family `FamilyConfig` in `DbConfig` - `get()` for single-valued families (panics if called on multi-valued) - `get_multiple()` for multi-valued families, returns `SmallVec<[ArcBytes; 1]>` — stack-allocated for the common 0–1 result case, heap-scales when needed - `put()` and `delete()` are unchanged — the family kind controls dedup/compaction behavior ### Write path & compaction - **Single-valued** (unchanged): last-write-wins per key - **Multi-valued**: all values are maintained, deletions 'shadow' old values. - Deletion inserts a tombstone that shadows all older values for that key across SST layers. Values written *after* the tombstone in the same batch are retained. - To avoid extra buffering logic the `MergeIter` semantics were changed so it produces 'newest' entries first * for SingleValues families this is no different since we only keep one value * for MultiValued families this makes dealing with tombstones trivial, but does mean compaction will reverse the order of the set. For this reason we make no guarantees about ordering. ### Read path - Controlled by a `FIND_ALL` const generic on the internal lookup methods - **Single-valued** (`FIND_ALL=false`): binary search, return _last_ match, stop * This fixes a bug where we might return Deleted when there is a value in the SST depending on what the search algorithm found first - **Multi-valued** (`FIND_ALL=true`): scan all matching entries in the SST block, then continue to older SSTs. If a tombstone is found, stop searching older layers. --------- Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
Author
Parents
Loading