microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into amawa/1bit-adam-nccl

awan-10 committed 5 years ago

0c5b18f1

Updates to comm. phase 2. nccl comm. passed the tests.

awan-10 committed 5 years ago

1bf1c275

Initial gather impl. Works intra-node.

awan-10 committed 5 years ago

a72049b6

[doc] typo fix and clarification (#563)

stas00 committed 5 years ago

Verified 17f36f1b

jeffra committed 5 years ago

c51fa65d

update manifest

jeffra committed 5 years ago

e4e20662

modify to test the nccl side code path

awan-10 committed 5 years ago

3e6974d1

Fix bugs. still does not work.

awan-10 committed 5 years ago

ebec1fee

bump to 0.3.6 and fix manifest to include reqs (#561)

jeffra committed 5 years ago

Verified 73c3262d

Adds long_description to setup.py (#560)

Shaden Smith committed 5 years ago

Verified 60097136

remove old code for comm.

awan-10 committed 5 years ago

57ab220a

Merge branch 'master' into amawa/1bit-adam-nccl

awan-10 committed 5 years ago

39b5949d

Deprecate client ability to disable gradient reduction (#552)

tjruwase committed 5 years ago

Verified 6e65c2cc

Update badges and CI name (#557)

jeffra committed 5 years ago

Verified 1ef5cd23

Switch to CI to GitHub Actions (#556)

jeffra committed 5 years ago

Verified 3347460e

Create main.yml

jeffra committed 5 years ago

Verified c18fb0de

Use dist collectives instead of mpi routines.

awan-10 committed 5 years ago

79f64049

temporary commit to save stuff.

awan-10 committed 5 years ago

567232be

add nccl 1-bit optim.

awan-10 committed 5 years ago

47788326

Bug fix for norm calculation in absence of model parallel group (#551)

samyam committed 5 years ago

Verified 00c3a254

jeffra committed 5 years ago

16313a96

Turn back on PP tests (#558)

jeffra committed 5 years ago

Verified eec44af1

Simplify dist init and only init if needed. (#553)

awan-10 committed 5 years ago

Verified 0e831e23

Adding static_loss_scale to unfused optimizer (#546)

samyam committed 5 years ago

Verified bcd56f97

Support non-tensor state in checkpoint (#548)

tjruwase committed 5 years ago

Verified 6021b702

Fix unbalanced gradients bug in ZeRO-2 gradient accumulation (#545)

tjruwase committed 5 years ago

Verified 0178e6cc

bump version 0.3.4

jeffra committed 5 years ago

6b28bc5d

Discover variables for NCCL backend on AML without mpi4py (#542)

awan-10 committed 5 years ago

Verified 1b45917c

Fix setup.py for cpu-only environment installation (#538)

harrydrippin committed 5 years ago

Verified d81cb26d

backwards compatability w. v020 ckpts, fix issue with zero-1 ckpts (#543)

jeffra committed 5 years ago

Verified dce054db

Older