DeepSpeed
Add CPUAdam optimizer for zero-offload in deepspeed engine
#484

Merged

Add CPUAdam optimizer for zero-offload in deepspeed engine #484

jeffra merged 10 commits into master from reyazda/cpu-offload-optimizer

add adamW to CPU-ADAM implementation

6faa75b1

supporting cpu-adam optimizer for zero-offload on deepspeed side

283da002

RezaYazdaniAminabadi requested a review from

arashashari 5 years ago

RezaYazdaniAminabadi requested a review from

awan-10 5 years ago

RezaYazdaniAminabadi requested a review from

cli99 5 years ago

RezaYazdaniAminabadi requested a review from

conglongli 5 years ago

RezaYazdaniAminabadi requested a review from

eltonzheng 5 years ago

RezaYazdaniAminabadi requested a review from

jeffra 5 years ago

RezaYazdaniAminabadi requested a review from

minjiaz 5 years ago

RezaYazdaniAminabadi requested a review from

niumanar 5 years ago

RezaYazdaniAminabadi requested a review from

samyam 5 years ago

RezaYazdaniAminabadi requested a review from

ShadenSmith 5 years ago

RezaYazdaniAminabadi requested a review from

tjruwase 5 years ago

add adamW and deepspeed-adamw mode

e2f204a3

jeffra approved these changes on 2020-10-24

address comment about bug

65f71413

jeffra commented on 2020-10-29

update with agreed upon cpu-adam engine logic

d9df2391

remove tabs

e5d50e1c

add doc-string for the cpu-adam optimizer

cf16a64d

fixing typo

26a28fde

updating config json documentation

7f858901

bump DSE to match cpu-adam updates

e0ca4433

jeffra merged f5aa2547 into master 5 years ago

mrwyattii deleted the reyazda/cpu-offload-optimizer branch 2 years ago

Reviewers

jeffra

arashashari

awan-10

cli99

conglongli

eltonzheng

minjiaz

niumanar

samyam

ShadenSmith

tjruwase

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Add CPUAdam optimizer for zero-offload in deepspeed engine #484 Merged

Add CPUAdam optimizer for zero-offload in deepspeed engine #484

DeepSpeed
Add CPUAdam optimizer for zero-offload in deepspeed engine
#484

Merged