PR #1273 ZeRO-1 empty grads fix + tests

ZeRO-1 empty grads fix + tests #1273

jeffra merged 3 commits into master from jeffra/zero-fixes

fix empty grad zero tests

c64d0cfc

dont clear grads in stage 1 code path

3546315c

prevent none grads from being reduced

e249c125

jeffra requested a review from

awan-10 5 years ago

jeffra requested a review from

cli99 5 years ago

jeffra requested a review from

conglongli 5 years ago

jeffra requested a review from

eltonzheng 5 years ago

jeffra requested a review from

minjiaz 5 years ago

jeffra requested a review from

niumanar 5 years ago

jeffra requested a review from

RezaYazdaniAminabadi 5 years ago

jeffra requested a review from

samyam 5 years ago

jeffra requested a review from

ShadenSmith 5 years ago

jeffra requested a review from

tjruwase 5 years ago

tjruwase approved these changes on 2021-08-02

jeffra merged adc21a4d into master 5 years ago

jeffra deleted the jeffra/zero-fixes branch 5 years ago

Reviewers

tjruwase

awan-10

cli99

conglongli

eltonzheng

minjiaz

niumanar

RezaYazdaniAminabadi

samyam

ShadenSmith

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed ZeRO-1 empty grads fix + tests #1273 Merged

ZeRO-1 empty grads fix + tests #1273

DeepSpeed
ZeRO-1 empty grads fix + tests
#1273

Merged