3 multi-backward
#7992

Open

Add engine.coalesce_grad_reduction() for ZeRO 1/2/3 multi-backward #7992

roycho96 wants to merge 3 commits into deepspeedai:master from roycho96:feat/zero-coalesce-grad-reduction

roycho96 requested a review from

tjruwase 8 days ago

roycho96 requested a review from

tohtana 8 days ago

roycho96 requested a review from

loadams 8 days ago

Add engine.coalesce_grad_reduction() for ZeRO 1/2/3 multi-backward

d4e71b71

roycho96 force pushed from 4ea6fc26 to d4e71b71 8 days ago

Merge branch 'master' into feat/zero-coalesce-grad-reduction

ccca4710

Merge branch 'master' into feat/zero-coalesce-grad-reduction

98bd6b4d

Reviewers

tjruwase

tohtana

loadams

Assignees

No one assigned

Labels

None yet

Milestone

No milestone