huggingface_hub
Framework-agnostic `split_state_dict_into_shards` helper
#1938
Merged
Merged
Framework-agnostic `split_state_dict_into_shards` helper #1938
Wauplin
merged
19 commits into
main
from
add-helper-to-shard-model
Draft for a framework-agnostic 'split_state_dict_into_shards' helper
777b2a8c
muellerzr
commented
on 2023-12-22
Merge branch 'main' into add-helper-to-shard-model
7ddfe725
Merge branch 'main' into add-helper-to-shard-model
dd3a9d2e
refactor
351a5eec
Wauplin
changed the title
mee
ec50ac9a
Merge branch 'main' into add-helper-to-shard-model
56b3a2be
Fix storage id stuff + add tests
b58aa6c5
add example
8f5f7a59
docstrings
5861f894
fix tf
7217f80d
Merge branch 'main' into add-helper-to-shard-model
8160d13f
deterministic tests
bb8f593f
5GB shards
f4d94bce
import torch
2f384909
rename to split_state_dict_into_shards_factory
b76583d2
fix
586b8d82
docs/source/en/package_reference/serialization.md
f9c5057f
with docs
d77c1cf3
Merge branch 'main' into add-helper-to-shard-model
2c33aaa9
julien-c
commented
on 2024-02-23
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub