DeepSpeed
Make reduce scatter optional for ZeRO-1 as workaround
#971
Merged
Overview
Commits
5
Changes
View On GitHub

Make reduce scatter optional for ZeRO-1 as workaround #971

jeffra merged 5 commits into master from olruwase/zero_optional_reduce_scatter
tjruwase
tjruwase Make reduce scatter optional for ZeRO-1 as workaround
4ff2cfe8
tjruwase tjruwase requested a review from jeffra jeffra 4 years ago
tjruwase tjruwase requested a review from arashashari arashashari 4 years ago
tjruwase tjruwase requested a review from awan-10 awan-10 4 years ago
tjruwase tjruwase requested a review from cli99 cli99 4 years ago
tjruwase tjruwase requested a review from conglongli conglongli 4 years ago
tjruwase tjruwase requested a review from eltonzheng eltonzheng 4 years ago
tjruwase tjruwase requested a review from minjiaz minjiaz 4 years ago
tjruwase tjruwase requested a review from niumanar niumanar 4 years ago
tjruwase tjruwase requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
tjruwase tjruwase requested a review from samyam samyam 4 years ago
tjruwase tjruwase requested a review from ShadenSmith ShadenSmith 4 years ago
tjruwase Make allreduce default for ZeRO 1
ff581f35
tjruwase Merge branch 'master' into olruwase/zero_optional_reduce_scatter
a43e0fb5
jeffra Merge branch 'master' into olruwase/zero_optional_reduce_scatter
e4144359
jeffra Merge branch 'master' into olruwase/zero_optional_reduce_scatter
980fdeda
jeffra jeffra merged 0b80ad06 into master 4 years ago
jianzi123
tjruwase

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum