next.js
2ff2a7aa - Turbopack: Add `turbo-frozenmap` crate with `FrozenMap` and `FrozenSet` implementations (#87042)

Commit
128 days ago
Turbopack: Add `turbo-frozenmap` crate with `FrozenMap` and `FrozenSet` implementations (#87042) This implements a `FrozenMap` type as a flat sorted boxed slice of `(K, V)` tuples. This is intended to be used with `TaskInput`s and `turbo_task::Value`s, where the data is stored immutably, and where we already try to call `shrink_to_fit`. This map is smaller than `HashMap` or `BTreeMap`, and provides `O(log n)` lookups with excellent cache locality (so it's probably faster than `HashMap` in most cases). Since the map is sorted, equality doesn't care about order, so this is a better implementation of `Eq` for our use-cases than what `IndexMap` provides. It has a variety of ways it can be constructed: A `HashMap`, a `Vec<(K, V)>`, a `BTreeMap`, etc. There are pros and cons of all of these: - `Vec<(K, V)>` is good if you don't need lookups until after freezing, and you don't expect many duplicates, since sorting at the end is cheaper than maintaining a map. - A `BTreeMap` is good if you need a real map, because we can avoid a sort at the end. - `HashMap`/`IndexMap` are okay if you're getting this data structure from somewhere else. `IndexSet` is also provided as a thin wrapper around `IndexMap`. ## Remaining Issues This can't be used with `ResolvedVc` as keys, because `ResolvedVc` does not implement `Ord`. I think it should be okay to do that because it's no worse than our current implementations of `Eq` and `Hash`, but I know this is contentious, so I'm leaving it out of this PR. ## Implementation Strategy I used Claude Code to write most of this: - Provided it rust's stdlib implementation of BTreeMap. - Asked it to extract the public interface into a separate file, excluding nightly-only features. - Asked it to remove all `&mut` methods. - Asked it to implement `FrozenMap` with the same API. Then I did a bunch of manual cleanup: - Replaced a bunch of manual trait implementations with derives. - Added bincode and serde traits. - Added various `turbo-tasks` trait implementations. - Added some methods/impls for getting the underlying slice. - Optimized a bunch of the constructors. Then I asked Claude Code to implement the Set version, and did another round of cleanup: - Removed bunch of unsafe code where it was trying to transmute `(T, ())` to `T`. The memory layout of a tuple is technically undefined, so this wasn't safe, and the APIs it was trying to create doing this weren't really needed either. - Replaced some of the iterator newtype wrappers with simpler type aliases. The newtype was overkill for our use-case.
Author
bgw bgw
Parents
Loading