uv
81029801 - puffin-resolver: make VersionMap construction lazy

Commit
1 year ago
puffin-resolver: make VersionMap construction lazy That is, a `PrioritizedDistribution` for a specific version of a package is not actually materialized in memory until a corresponding `VersionMap::get` call is made for that version. Similarly, iteration lazily materializes distributions as it moves through the map. It specifically does not materialize everything first. The main reason why this is effective is that an `OwnedArchive<SimpleMetadata>` represents a zero-copy (other than reading the source file) version of `SimpleMetadata` that is really just a `Vec<u8>` internally. The problem with `VersionMap` construction previously is that it had to eagerly materialize a `SimpleMetadata` in memory before anything else, which defeats a large part of the purpose of zero-copy deserialization. By making more of `VersionMap` construction itself lazy, we permit doing some parts of resolution without necessarily fully deserializing a `SimpleMetadata` into memory. Indeed, with this commit, in the warm cached case, a `SimpleMetadata` is itself never materialized fully in memory. This does not completely and totally fully realize the benefits of zero-copy deserialization. For example, we are likely still building lots of distributions in memory that we don't actually need in some cases. Perhaps in cases where no resolution exists, or when one needs to iterate over large portions of the total versions published for a package.
Author
Committer
Parents
Loading