huggingface_hub
ae3c4a02 - Framework-agnostic `split_state_dict_into_shards` helper (#1938)

Commit
1 year ago
Framework-agnostic `split_state_dict_into_shards` helper (#1938) * Draft for a framework-agnostic 'split_state_dict_into_shards' helper * refactor * mee * Fix storage id stuff + add tests * add example * docstrings * fix tf * deterministic tests * 5GB shards * import torch * rename to split_state_dict_into_shards_factory * fix * docs/source/en/package_reference/serialization.md * with docs
Author
Parents
Loading