DeepSpeed
1-bit Adam v2
#817
Merged
Overview
Commits
55
Changes
View On GitHub

1-bit Adam v2 #817

conglongli merged 55 commits into master from staging-1bit-adam-v2
conglongli
awan-10 NCCL-based 1-bit Adam + Code Refactor for Comm. Backends (#594)
a6dba72a
conglongli Merge branch 'master' into staging-1bit-nccl-v2
78400850
conglongli Revert "Merge branch 'master' into staging-1bit-nccl-v2"
6dbdd985
conglongli Revert "Revert "Merge branch 'master' into staging-1bit-nccl-v2""
9712f10c
conglongli Merge branch 'master' into staging-1bit-nccl-v2
4d1f4f01
conglongli comm optimization + 1-bit lamb
d8a23c98
awan-10 Saving/debugging commit.
89e19362
conglongli finalizing 1-bit lamb
a1bbf781
conglongli finalizing 1-bit lamb
db0ca769
conglongli add momentum mask and chkpt handling for 1-bit adam
07deab84
awan-10 Merge remote-tracking branch 'origin/staging-1bit-nccl-v2' into stagi…
625f475f
awan-10 Cleanup and modify nccl test to be runnable with deepspeed launcher.
d55fddb5
awan-10 Merge branch 'master' into staging-1bit-nccl-v2
5b1cacb7
awan-10 Fix format.
8cbc212b
awan-10 fix formatting again.
ff8c871a
awan-10 make test runnable without mpi4py
c17041f3
awan-10 Add dist.alltoall and dist.allgather instead of custom functions.
5e01a30e
awan-10 remove debug prints.
97a55577
conglongli formatting and renaming
e3e1e39b
conglongli renaming
d5b9dcc8
conglongli renaming
3d66a8a2
conglongli add unit test, fix existing tests
b042467e
awan-10 Merge branch 'master' into staging-1bit-adam-v2
ab3521d1
conglongli skip unit test when torch < 1.8
9fa5166c
conglongli revert 1-bit lamb
65d7ec59
conglongli flatten momentum when dimension is more than 1
8376a404
conglongli add warning message for 1-bit adam under fp32
6a19f296
conglongli improve version check
819043da
conglongli add fp32 test
a6943be9
conglongli Merge remote-tracking branch 'origin' into staging-1bit-adam-v2
2042b299
conglongli 1-bit adam doc
66a8c930
conglongli conglongli requested a review from jeffra jeffra 5 years ago
conglongli conglongli requested a review from awan-10 awan-10 5 years ago
conglongli conglongli requested a review from arashashari arashashari 5 years ago
conglongli conglongli requested a review from cli99 cli99 5 years ago
conglongli conglongli requested a review from eltonzheng eltonzheng 5 years ago
conglongli conglongli requested a review from minjiaz minjiaz 5 years ago
conglongli conglongli requested a review from niumanar niumanar 5 years ago
conglongli conglongli requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 5 years ago
conglongli conglongli requested a review from samyam samyam 5 years ago
conglongli conglongli requested a review from ShadenSmith ShadenSmith 5 years ago
conglongli conglongli requested a review from tjruwase tjruwase 5 years ago
awan-10
awan-10 commented on 2021-03-04
awan-10
awan-10 commented on 2021-03-04
awan-10
conglongli fix file name
fb329a9c
awan-10
awan-10 commented on 2021-03-04
conglongli doc fix
0b3c1d76
conglongli torch 1.8 is released
0bffa9b1
conglongli doc fix
294c2d6f
conglongli fix tests
003981a4
conglongli Merge branch 'master' into staging-1bit-adam-v2
3f42b3a4
conglongli Merge branch 'master' into staging-1bit-adam-v2
bbd61436
conglongli update news
877f8d71
conglongli Merge branch 'master' into staging-1bit-adam-v2
f861465c
conglongli add doc for momentum mask
2ed029e3
conglongli Merge branch 'staging-1bit-adam-v2' of github.com:microsoft/DeepSpeed…
c6e7cf79
conglongli fix checkpoing handling, add unit test
3b53c90a
conglongli checkpoint handling doc
42407294
conglongli doc final cleanup
968a53fd
conglongli Merge branch 'master' into staging-1bit-adam-v2
bbec3007
jeffra Merge branch 'master' into staging-1bit-adam-v2
e28a99e9
jeffra
jeffra commented on 2021-03-16
jeffra
jeffra commented on 2021-03-16
jeffra
jeffra commented on 2021-03-16
jeffra
jeffra commented on 2021-03-16
jeffra
jeffra commented on 2021-03-16
jeffra
jeffra commented on 2021-03-16
jeffra
jeffra commented on 2021-03-16
jeffra
jeffra commented on 2021-03-16
conglongli bump dates
1221aec8
conglongli Merge branch 'staging-1bit-adam-v2' of github.com:microsoft/DeepSpeed…
535b5bad
jeffra
jeffra requested changes on 2021-03-16
conglongli update tests
8cfd2b78
conglongli url change
38ff08a9
conglongli doc fix
de036564
conglongli fix test
5957bce9
conglongli doc update
ef51ac69
conglongli
conglongli Merge branch 'master' into staging-1bit-adam-v2
7c08b34d
jeffra
jeffra approved these changes on 2021-03-16
conglongli conglongli merged 68c8481b into master 5 years ago
conglongli conglongli deleted the staging-1bit-adam-v2 branch 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum