DeepSpeed
Enable ZeRO3 allgather for multiple dtypes
#4647
Merged

Enable ZeRO3 allgather for multiple dtypes #4647

tjruwase merged 6 commits into master from tohtana/z3_multi_dtypes
tohtana
enable z3 allgather for multiple dtyps
cb16dd98
tohtana tohtana requested a review from jeffra jeffra 2 years ago
tohtana tohtana requested a review from tjruwase tjruwase 2 years ago
tohtana tohtana requested a review from mrwyattii mrwyattii 2 years ago
tjruwase
tjruwase approved these changes on 2023-11-07
tjruwase Merge branch 'master' into tohtana/z3_multi_dtypes
d96985a6
loadams Merge branch 'master' into tohtana/z3_multi_dtypes
67d744f7
add import
8bffe3e0
Merge branch 'tohtana/z3_multi_dtypes' of github.com:microsoft/DeepSp…
cec732c3
tjruwase Merge branch 'master' into tohtana/z3_multi_dtypes
e58254d5
tjruwase tjruwase merged b8e16642 into master 2 years ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone