enhancements and fixes for FSDP and DeepSpeed #532
pacman100
merged
19 commits into
huggingface:main
from
pacman100:smangrul/saving-and-loading-utilities
checkpointing enhancements and fixes for FSDP and DeepSpeed
0639d1c6
resolving comments
e3ee960c
fixes
a59223f8
resolving comments
2109a2da
adding FSDP for all the collective operations
94967ba1
pacman100
changed the title
sgugger
approved these changes
on 2022-07-20
adding deepspeed and fsdp tests
44d7e987
Merge branch 'main' into smangrul/saving-and-loading-utilities
b7515d9b
reverting `mocked_dataloader` changes
3d25ac51
adding FSDP tests
0645a09c
data files revert
ce4aa1e5
excluding fsdp tests from `tests_core`
0bbb981f
try 2
35301bca
adding time delay to avoid `torchrun` from crashing at times leading …
6d85654e
reducing the time of tests
9a8103bd
fixes
77d6a775
fix
f42a6e0f
fixes and reduce time further
b5851090
reduce time further and minor fixes
d159a53a
adding a deepspeed basic e2e test for single gpu setup
436e06be
sgugger
approved these changes
on 2022-07-26
Login to write a write a comment.
Login via GitHub