01 adam optimizer #1790
jeffra
merged
24 commits into
deepspeedai:master
from
01Adam
add 0/1 Adam implementation
fcfe2637
add pytest for 0/1 Adam and fix loading issue
8885646e
Merge branch 'master' into 01Adam
4e7e9bbc
fix formatting issue and a type
c17741be
Merge branch 'master' into 01Adam
cd69b55d
Merge branch 'master' into 01Adam
adb71510
Merge branch 'master' into 01Adam
6c9bdc1f
Merge branch 'master' into 01Adam
6ddaf2e9
fix mask/warning issues and add 0/1 Adam docs/tutorial
6c1946fd
fix formatting and add more details to 0/1 Adam tutorial
99d35a26
Merge branch 'master' into 01Adam
db6f5b8e
Merge branch 'master' into 01Adam
75d09d97
mark new tests as forced sequential
dfd6cf66
conglongli
approved these changes
on 2022-03-08
disable new tests (testing hang issue in CI)
3750c2c1
Revert "disable new tests (testing hang issue in CI)"
f90a5d76
fix naive all reduce hanging issue (still need testing)
727a043d
remove the scaling after naive allreduce
419486e7
mention 0/1 Adam in 1-bit Adam tutorial
7190a990
Merge branch 'master' into 01Adam
900cdf20
remove pytest.mark.sequential
a4a86f84
Merge branch '01Adam' of github.com:EugeneLYC/DeepSpeed-1 into 01Adam
ac08717b
fix the comm volume saving number with FP16
a1e1615a
Using GPT results (FP16) to update comm volume saving number
93680e7b
fail fast during tests
a2f273e3
disabled auto-merge 3 years ago
Manually disabled by user
Manually disabled by user
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub