microsoft/DeepSpeed

Pull Requests Commits

remove the random-generator from context during inference

Reza Yazdani committed 2 years ago

6addd4ee

Fix the tensor-slicing copy for qkv parameters

Reza Yazdani committed 2 years ago

920e6be2

Update README.md (#2192)

gasparitiago committed 2 years ago

Verified 0f5c2012

Update nv-lightning-v100.yml (#2190)

jeffra committed 2 years ago

Verified 1223b13c

[docs] add more models to adoption (#2189)

jeffra committed 2 years ago

Verified 33667e0e

fix missing import (#2175)

rahilbathwal5 committed 2 years ago

Verified ee5ce524

Update README to latest Composer version (#2177)

hanlint committed 2 years ago

Verified 80b5b925

Enable fused_lamb_cuda_kernel on ROCm (#2148)

rraminen committed 2 years ago

Verified 2e3769a1

Match compute and reduce dtype (#2145)

tjruwase committed 2 years ago

Verified e419f7cb

fixing model partitioning without injection (#2179)

RezaYazdaniAminabadi committed 2 years ago

Verified e7d99595

update offload docs to include stage 1 (#2178)

jeffra committed 2 years ago

Verified fad0a410

Update for AMD CI workflow (#2172)

mrwyattii committed 2 years ago

Verified d1cd18e5

[docs] adoption updates (#2173)

jeffra committed 2 years ago

Verified bb49dc73

Fix wrong unit of latency in flops-profiler (#2090) (#2095)

zionwu committed 2 years ago

Verified 6bfcf3c6

delay torch import for inference compatability check (#2167)

jeffra committed 2 years ago

Verified 776e3698

Fix for distributed tests on pytorch>=1.12 (#2141)

mrwyattii committed 2 years ago

Verified 1a71e77d

jeffra committed 2 years ago

b005db86

Tensor parallelism for Mixture of Experts (#2074)

siddharth9820 committed 2 years ago

Verified 5fe9d610

Release swap buffers for persisted params (#2089)

tjruwase committed 2 years ago

Verified 2210ebe7

enable fp16 input autocasting (#2158)

jeffra committed 2 years ago

Verified a039e226

[zero-3] shutdown zero.Init from within ds.init (#2150)

jeffra committed 2 years ago

Verified 46401b38

prevent cuda 10 builds of inference kernels on ampere (#2157)

jeffra committed 2 years ago

Verified 63f470ee

Elastic Training support in DeepSpeed (#2153) (#2156)

aj-prime committed 2 years ago

Verified 1ed5aa96

Added retain_graph as a kwarg to the main engine backward function (#1149)

ncilfone committed 2 years ago

Verified ba67bd9a

Fix random token-generation issue + MP-checkpoint loading/saving (#2132)

RezaYazdaniAminabadi committed 2 years ago

Verified 556f0051

fix: fix BF16_Optimizer compatibility issue with optimizer state 0-dim tensor (#2152)

shjwudp committed 2 years ago

Verified 57140e8e

Graceful exit on failures for multi-node runs (#2008)

Jerry Mannil committed 2 years ago

Verified 66d29b0a

Trajepl/nebula ckpt engine (#2085)

trajepl committed 2 years ago

Verified e669aaf5

force newer datasets version (#2147)

jeffra committed 2 years ago

Verified a54661a0

formatting fix for #1962

jeffra committed 2 years ago

b442264d

Older