DeepSpeed
set adamw_mode default true (follows FusedAdam and < 0.3.11 logic)
#844

Merged

set adamw_mode default true (follows FusedAdam and < 0.3.11 logic) #844

jeffra merged 4 commits into master from jeffra/adamw_default_true

set adamw_mode default true (follows FusedAdam and < 0.3.11 logic)

37e4e74f

jeffra requested a review from

arashashari 4 years ago

jeffra requested a review from

awan-10 4 years ago

jeffra requested a review from

cli99 4 years ago

jeffra requested a review from

conglongli 4 years ago

jeffra requested a review from

eltonzheng 4 years ago

jeffra requested a review from

minjiaz 4 years ago

jeffra requested a review from

niumanar 4 years ago

jeffra requested a review from

RezaYazdaniAminabadi 4 years ago

jeffra requested a review from

samyam 4 years ago

jeffra requested a review from

ShadenSmith 4 years ago

jeffra requested a review from

tjruwase 4 years ago

refactor of adam(w) logic, still same effect as before but hopefully …

c4df508a

conglongli approved these changes on 2021-03-09

samyam commented on 2021-03-09

jeffra marked this pull request as draft 4 years ago

add unit tests sweeping the space of adam/adamw

a23891aa

jeffra marked this pull request as ready for review 4 years ago

Merge branch 'master' into jeffra/adamw_default_true

2e391d23

samyam approved these changes on 2021-03-11

ShadenSmith approved these changes on 2021-03-11

jeffra merged dd03cff2 into master 4 years ago

jeffra deleted the jeffra/adamw_default_true branch 4 years ago

Reviewers

ShadenSmith

samyam

conglongli

arashashari

awan-10

cli99

eltonzheng

minjiaz

niumanar

RezaYazdaniAminabadi

tjruwase

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed set adamw_mode default true (follows FusedAdam and < 0.3.11 logic) #844 Merged

set adamw_mode default true (follows FusedAdam and < 0.3.11 logic) #844

DeepSpeed
set adamw_mode default true (follows FusedAdam and < 0.3.11 logic)
#844

Merged