PR #1790 01 adam optimizer

add 0/1 Adam implementation

EugeneLYC committed 4 years ago

add pytest for 0/1 Adam and fix loading issue

EugeneLYC committed 4 years ago

Merge branch 'master' into 01Adam

conglongli committed 4 years ago

fix formatting issue and a type

EugeneLYC committed 4 years ago

Merge branch 'master' into 01Adam

jeffra committed 4 years ago

Merge branch 'master' into 01Adam

jeffra committed 4 years ago

Merge branch 'master' into 01Adam

conglongli committed 4 years ago

Merge branch 'master' into 01Adam

conglongli committed 4 years ago

fix mask/warning issues and add 0/1 Adam docs/tutorial

EugeneLYC committed 4 years ago

fix formatting and add more details to 0/1 Adam tutorial

EugeneLYC committed 4 years ago

Merge branch 'master' into 01Adam

jeffra committed 4 years ago

Merge branch 'master' into 01Adam

conglongli committed 4 years ago

mark new tests as forced sequential

jeffra committed 4 years ago

disable new tests (testing hang issue in CI)

jeffra committed 4 years ago

Revert "disable new tests (testing hang issue in CI)"

conglongli committed 4 years ago

fix naive all reduce hanging issue (still need testing)

conglongli committed 4 years ago

remove the scaling after naive allreduce

conglongli committed 4 years ago

mention 0/1 Adam in 1-bit Adam tutorial

conglongli committed 4 years ago

Merge branch 'master' into 01Adam

conglongli committed 4 years ago

remove pytest.mark.sequential

conglongli committed 4 years ago

Merge branch '01Adam' of github.com:EugeneLYC/DeepSpeed-1 into 01Adam

conglongli committed 4 years ago

fix the comm volume saving number with FP16

EugeneLYC committed 4 years ago

Using GPT results (FP16) to update comm volume saving number

EugeneLYC committed 4 years ago

fail fast during tests

jeffra committed 4 years ago

DeepSpeed 01 adam optimizer #1790 Merged

DeepSpeed
01 adam optimizer
#1790

Merged