microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'staging-zero-dual-v3' into reza/deepspeed_adam_merge_v3

Reza Yazdani committed 5 years ago

58c0741f

Reza Yazdani committed 5 years ago

4c257dd0

Reza Yazdani committed 5 years ago

d4c3b0a9

Reza Yazdani committed 5 years ago

7aaccf45

add casting kernel

Reza Yazdani committed 5 years ago

bb75df77

rename perf test

Reza Yazdani committed 5 years ago

36d5fde7

Reza Yazdani committed 5 years ago

37887241

ZeRO-Offload passing model functionality tests (#366)

tjruwase committed 5 years ago

Verified aa3f2893

fix array-sizes

Reza Yazdani committed 5 years ago

2d17a6d3

fix variance indexes

Reza Yazdani committed 5 years ago

ad8af384

Reza Yazdani committed 5 years ago

59ffc1a9

fixing adam copy fp16-param-add more compile flags for cpu_adam

Reza Yazdani committed 5 years ago

1d4b41fe

fixing the cpu_adam API and add deepspeed_adam flag in config.py (#365)

RezaYazdaniAminabadi committed 5 years ago

Verified 606543de

everything is working

samyam committed 5 years ago

9ba232aa

Generalize detection of ZeRO supported optimizers (#349)

tjruwase committed 5 years ago

Verified 253b0447

change gpt2 pretrain to have DeepSpeed adam (#361)

samyam committed 5 years ago

Verified 1a4a82b6

Allocating CPU memory directly on CPU without transfering them from GPU (#360)

samyam committed 5 years ago

Verified 7be128af

Merge branch 'staging-zero-dual-v2' of github.com:microsoft/DeepSpeed into staging-zero-dual-v2

tjruwase committed 5 years ago

077cfd47

Enable contiguous gradients for cpu_offload

tjruwase committed 5 years ago

ea5b9918

Import path fixes + conditional imports (#358)

jeffra committed 5 years ago

Verified 130dd70b

Samyamr/zero offload correctness (#359)

samyam committed 5 years ago

Verified af512117

make the adam unit test work with random params and grads and for more steps

Reza Yazdani committed 5 years ago

504a643b

add cpu adam optimizer (#356)

RezaYazdaniAminabadi committed 5 years ago

Verified 20c414d0

Correctness and perf fixes (#354)

tjruwase committed 5 years ago

Verified 6604a5da

Assert ZeRO-Offload+gradient accumulation (#347)

tjruwase committed 5 years ago

Verified 0159ebba

update DSE to staging for zero-dual

jeffra committed 5 years ago

75d70a9f

ZeRO-Offload v1 (squash) (#345)

jeffra committed 5 years ago

e45b5e4c

update DSE and rename SA tests

jeffra committed 5 years ago

1661e830

Update Dockerfile

jeffra committed 5 years ago

Verified 5518aae5

Update Dockerfile

jeffra committed 5 years ago

Verified 87165403

Older