PR #2289 Fix for Zero3 when MP>1

Fix for Zero3 when MP>1 #2289

jeffra merged 3 commits into deepspeedai:master from Quentin-Anthony:zero3-mp-fix

Fix for Zero3 when MP>1 and at least one batch param undefined

d2bea227

Quentin-Anthony requested a review from

jeffra 3 years ago

Quentin-Anthony requested a review from

samyam 3 years ago

Quentin-Anthony requested a review from

tjruwase 3 years ago

Quentin-Anthony requested a review from

ShadenSmith 3 years ago

Quentin-Anthony requested a review from

conglongli 3 years ago

Quentin-Anthony requested a review from

awan-10 3 years ago

Quentin-Anthony requested a review from

cli99 3 years ago

Quentin-Anthony requested a review from

eltonzheng 3 years ago

Quentin-Anthony requested a review from

minjiaz 3 years ago

Quentin-Anthony requested a review from

RezaYazdaniAminabadi 3 years ago

Quentin-Anthony requested a review from

duli2012 3 years ago

Quentin-Anthony requested a review from

mrwyattii 3 years ago

Quentin-Anthony requested a review from

yaozhewei 3 years ago

Quentin-Anthony requested a review from

arashb 3 years ago

Quentin-Anthony requested a review from

xiaoxiawu-microsoft 3 years ago

Quentin-Anthony requested a review from

samadejacobs 3 years ago

Quentin-Anthony requested a review from

cmikeh2 3 years ago

tjruwase approved these changes on 2022-09-02

Merge branch 'master' into zero3-mp-fix

599535d2

Merge branch 'master' into zero3-mp-fix

d5ec6bd2

jeffra merged 4524b8db into master 3 years ago

Reviewers

tjruwase

jeffra

samyam

ShadenSmith

conglongli

awan-10

cli99

eltonzheng

minjiaz

RezaYazdaniAminabadi

duli2012

mrwyattii

yaozhewei

arashb

xiaoxiawu-microsoft

samadejacobs

cmikeh2

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Fix for Zero3 when MP>1 #2289 Merged

Fix for Zero3 when MP>1 #2289

DeepSpeed
Fix for Zero3 when MP>1
#2289

Merged