deepspeedai/DeepSpeed

Pull Requests Commits

Clean up debug statements

samadejacobs committed 3 years ago

33325bc7

Add zero_param_group_size as arg

samadejacobs committed 3 years ago

58c39cf7

Add forward flag to partitioned_param

samadejacobs committed 3 years ago

befe2ebf

Add profiler and set fw/bw flag for allgather

samadejacobs committed 3 years ago

4bf6d3fd

Add partitioned param profiler, authored by @olruwase

samadejacobs committed 3 years ago

ddd23da7

hpZeRO impl details in partitioned param

samadejacobs committed 3 years ago

fd401932

hpZeRO impl details in partitioned param

samadejacobs committed 3 years ago

81a988f3

hpZeRO impl details in partitioned param

samadejacobs committed 3 years ago

b6769c80

Add invalidate secondary partition post-step

samadejacobs committed 3 years ago

f553e2e6

Add zero param group partitioning to param offload

samadejacobs committed 3 years ago

ec112add

Add zero param group partitioning to stage 3

samadejacobs committed 3 years ago

29d7f5d1

Extends group comm to support hpZeRO

samadejacobs committed 3 years ago

ae3cad0c

Implements get all ranks in a given group

samadejacobs committed 3 years ago

36d9b8a6

Update README.md (#2192)

gasparitiago committed 3 years ago

Verified 0f5c2012

Update nv-lightning-v100.yml (#2190)

jeffra committed 3 years ago

Verified 1223b13c

[docs] add more models to adoption (#2189)

jeffra committed 3 years ago

Verified 33667e0e

fix missing import (#2175)

rahilbathwal5 committed 3 years ago

Verified ee5ce524

Update README to latest Composer version (#2177)

hanlint committed 3 years ago

Verified 80b5b925

Enable fused_lamb_cuda_kernel on ROCm (#2148)

rraminen committed 3 years ago

Verified 2e3769a1

Match compute and reduce dtype (#2145)

tjruwase committed 3 years ago

Verified e419f7cb

fixing model partitioning without injection (#2179)

RezaYazdaniAminabadi committed 3 years ago

Verified e7d99595

update offload docs to include stage 1 (#2178)

jeffra committed 3 years ago

Verified fad0a410

Update for AMD CI workflow (#2172)

mrwyattii committed 3 years ago

Verified d1cd18e5

[docs] adoption updates (#2173)

jeffra committed 3 years ago

Verified bb49dc73

Fix wrong unit of latency in flops-profiler (#2090) (#2095)

zionwu committed 3 years ago

Verified 6bfcf3c6

delay torch import for inference compatability check (#2167)

jeffra committed 3 years ago

Verified 776e3698

Fix for distributed tests on pytorch>=1.12 (#2141)

mrwyattii committed 3 years ago

Verified 1a71e77d

jeffra committed 3 years ago

b005db86

Tensor parallelism for Mixture of Experts (#2074)

siddharth9820 committed 3 years ago

Verified 5fe9d610

Release swap buffers for persisted params (#2089)

tjruwase committed 3 years ago

Verified 2210ebe7

Older