microsoft/DeepSpeed

Pull Requests Commits

Increate bucket size scaler

tjruwase committed 4 years ago

d1164337

tjruwase committed 4 years ago

dbcebb83

Add ZeRO-Offload model tests

tjruwase committed 4 years ago

80b99c6a

fix adam perormance (#372)

RezaYazdaniAminabadi committed 4 years ago

Verified 75e9e320

fixing corner cases (#371)

RezaYazdaniAminabadi committed 4 years ago

Verified bae81311

ZeRO-Offload: Integration code fixes (#370)

tjruwase committed 4 years ago

Verified 19aac8ad

Reza Yazdani committed 4 years ago

edb770d2

Merge branch 'reza/deepspeed_adam_merge_v3' into staging-zero-dual-v3

Reza Yazdani committed 4 years ago

773e454c

Merge branch 'staging-zero-dual-v3' into reza/deepspeed_adam_merge_v3

Reza Yazdani committed 4 years ago

58c0741f

Reza Yazdani committed 4 years ago

4c257dd0

Reza Yazdani committed 4 years ago

d4c3b0a9

Reza Yazdani committed 4 years ago

7aaccf45

add casting kernel

Reza Yazdani committed 4 years ago

bb75df77

tjruwase committed 4 years ago

942ec905

tjruwase committed 4 years ago

f0c34d0c

fixing adam copy fp16-param and add more compile flags for cpu_adam (#367)

RezaYazdaniAminabadi committed 4 years ago

Verified d8ff56c3

rename perf test

Reza Yazdani committed 4 years ago

36d5fde7

Reza Yazdani committed 4 years ago

37887241

ZeRO-Offload passing model functionality tests (#366)

tjruwase committed 4 years ago

Verified aa3f2893

fix array-sizes

Reza Yazdani committed 4 years ago

2d17a6d3

fix variance indexes

Reza Yazdani committed 4 years ago

ad8af384

Reza Yazdani committed 4 years ago

59ffc1a9

fixing adam copy fp16-param-add more compile flags for cpu_adam

Reza Yazdani committed 4 years ago

1d4b41fe

fixing the cpu_adam API and add deepspeed_adam flag in config.py (#365)

RezaYazdaniAminabadi committed 4 years ago

Verified 606543de

everything is working

samyam committed 4 years ago

9ba232aa

Generalize detection of ZeRO supported optimizers (#349)

tjruwase committed 4 years ago

Verified 253b0447

change gpt2 pretrain to have DeepSpeed adam (#361)

samyam committed 4 years ago

Verified 1a4a82b6

Allocating CPU memory directly on CPU without transfering them from GPU (#360)

samyam committed 4 years ago

Verified 7be128af

Merge branch 'staging-zero-dual-v2' of github.com:microsoft/DeepSpeed into staging-zero-dual-v2

tjruwase committed 4 years ago

077cfd47

Enable contiguous gradients for cpu_offload

tjruwase committed 4 years ago

ea5b9918

Older