DeepSpeed
Zero3 Fix allreduce optimization for extra large tensor
#3832

Merged

Zero3 Fix allreduce optimization for extra large tensor #3832

tjruwase merged 2 commits into deepspeedai:master from BacharL:zero3_grads_fix

BacharL requested a review from

jeffra 2 years ago

BacharL requested a review from

tjruwase 2 years ago

BacharL requested a review from

samyam 2 years ago

BacharL requested a review from

mrwyattii 2 years ago

tjruwase commented on 2023-06-28

Zero3 Fix allreduce optimization for extra large tensor

9b3a233b

BacharL force pushed from d2040670 to 9b3a233b 2 years ago

Merge branch 'master' into zero3_grads_fix

7fa15089

tjruwase approved these changes on 2023-07-02

tjruwase merged d229ff17 into master 2 years ago

Reviewers

tjruwase

jeffra

samyam

mrwyattii

Assignees

No one assigned

Labels

None yet

Milestone

No milestone