microsoft/DeepSpeed

Pull Requests Commits

rename perf test

Reza Yazdani committed 4 years ago

36d5fde7

Reza Yazdani committed 4 years ago

37887241

fix array-sizes

Reza Yazdani committed 4 years ago

2d17a6d3

fix variance indexes

Reza Yazdani committed 4 years ago

ad8af384

Reza Yazdani committed 4 years ago

59ffc1a9

fixing adam copy fp16-param-add more compile flags for cpu_adam

Reza Yazdani committed 4 years ago

1d4b41fe

everything is working

samyam committed 4 years ago

9ba232aa

Generalize detection of ZeRO supported optimizers (#349)

tjruwase committed 4 years ago

Verified 253b0447

change gpt2 pretrain to have DeepSpeed adam (#361)

samyam committed 4 years ago

Verified 1a4a82b6

Allocating CPU memory directly on CPU without transfering them from GPU (#360)

samyam committed 4 years ago

Verified 7be128af

Merge branch 'staging-zero-dual-v2' of github.com:microsoft/DeepSpeed into staging-zero-dual-v2

tjruwase committed 4 years ago

077cfd47

Enable contiguous gradients for cpu_offload

tjruwase committed 4 years ago

ea5b9918

Import path fixes + conditional imports (#358)

jeffra committed 4 years ago

Verified 130dd70b

Samyamr/zero offload correctness (#359)

samyam committed 4 years ago

Verified af512117

make the adam unit test work with random params and grads and for more steps

Reza Yazdani committed 4 years ago

504a643b

add cpu adam optimizer (#356)

RezaYazdaniAminabadi committed 4 years ago

Verified 20c414d0

Correctness and perf fixes (#354)

tjruwase committed 4 years ago

Verified 6604a5da

Assert ZeRO-Offload+gradient accumulation (#347)

tjruwase committed 4 years ago

Verified 0159ebba

update DSE to staging for zero-dual

jeffra committed 4 years ago

75d70a9f

ZeRO-Offload v1 (squash) (#345)

jeffra committed 4 years ago

e45b5e4c

update DSE and rename SA tests

jeffra committed 4 years ago

1661e830

Update Dockerfile

jeffra committed 4 years ago

Verified 5518aae5

Update Dockerfile

jeffra committed 4 years ago

Verified 87165403

Sparse attn + ops/runtime refactor + v0.3.0 (#343)

jeffra committed 4 years ago

Verified e5bbc2e5

Switches BBS example to use mbsize=3 and gas=2 to fit in 16GB of memory. (#341)

Shaden Smith committed 4 years ago

Verified 838f53b7

Update .gitignore

jeffra committed 4 years ago

Verified e8dd47df

Update run_func_test.py

samyam committed 4 years ago

Verified f4726b75

Rename ds_config_func_bs8_zero0_gas10.json to ds_config_func_bs8_zero0_gas3.json

samyam committed 4 years ago

Verified 6122a74e

Rename ds_config_func_bs8_zero2_gas10.json to ds_config_func_bs8_zero2_gas3.json

samyam committed 4 years ago

Verified 7a356b28

Samyamr/grad acc stage2 (#338)

samyam committed 4 years ago

Verified 7240abf3

Older