transformers
Enhance Model Loading By Providing Parallelism, Uses Optional Env Flag
#36835
Merged
Merged
Enhance Model Loading By Providing Parallelism, Uses Optional Env Flag #36835
Cyrilvallez
merged
34 commits into
huggingface:main
from
inf3rnus:03-18-25-parallel-model-loading
Get parallel loader working. Include tests.
8fb9b187
Update the tests for parallel loading
27f36f24
Merge branch 'main' into 03-18-25-parallel-model-loading
7e5ecd89
Rename env variables.
e7c3ea52
Add docs for parallel model weight loading.
7599fe25
Touch up parallel model loading docs.
065e1022
Touch up parallel model loading docs again.
d31594a7
Edit comment in test_modeling_utils_parallel_loading.py
33b3e0f0
Merge branch 'main' into 03-18-25-parallel-model-loading
3fb6b659
Make sure HF_PARALLEL_LOADING_WORKERS is spelled correctly in modelin…
0e22c047
Correct times for parallelized loading, previous times were for a "ho…
904bdaf6
Update parallel model loading so the spawn method is encapsulated. DR…
7e37ba4b
Update docs on model loading parallelism so that details on setting t…
a203f6a1
Fix style on model loading parallelism changes.
14e9eefb
Merge remote-tracking branch 'upstream/main' into 03-18-25-parallel-m…
fe1fc0c5
Merge latest version of master's modeling_utils.
d5637e8b
Removed unused variable.
e0d37bb0
Fix argument packing for the parallel loader.
9b4165c1
Fix state dict being undefined in the parallel model loader.
1085461d
Merge main.
82ab2ec5
Rename variables used in parallel model loading for clarity. Use get_…
7ae3db61
Switch to the use of threads for parallel model loading.
8d04325e
Update docs for parallel loading.
674ec373
Remove the use of json.loads when evaluating HF_ENABLE_PARALLEL_LOADI…
b8a1470f
Move parallelized shard loading into its own function.
efb6605b
Remove use of is_true(). Favor checking env var true values for HF_EN…
c66daef8
Update copyright to 2025 in readme for paralell model loading.
4566c5cc
Remove garbage collection line in load_shard_file, implicit garbage c…
610c5e31
Run formatter on modeling_utils.py
a9cb54b5
Merge branch 'main' into 03-18-25-parallel-model-loading
fc76fbbe
Apply style fixes
16f3751a
Cyrilvallez
commented
on 2025-05-22
Merge main.
cd0f42ee
Delete tests/utils/test_modeling_utils_parallel_loading.py
3b9f4587
Merge branch 'main' into 03-18-25-parallel-model-loading
b6bf4213
Cyrilvallez
approved these changes
on 2025-05-23
Reviewers
Assignees
No one assigned
Labels
Core: Modeling
from_pretrained
Milestone
No milestone
Login to write a write a comment.
Login via GitHub