microsoft/DeepSpeed

Pull Requests Commits

fix all workflows and formatting

jeffra committed 3 years ago

cb1e299c

update workflows for mii

jeffra committed 3 years ago

10a48955

jeffra committed 3 years ago

e7d42a3d

add mii to ds install

jeffra committed 3 years ago

260aafec

jeffra committed 3 years ago

5bb25ad2

delay mii import

jeffra committed 3 years ago

f5024b02

get policies from mii

jeffra committed 3 years ago

b8c9c3e7

delay torch import for inference compatability check (#2167)

jeffra committed 3 years ago

Verified 776e3698

Fix for distributed tests on pytorch>=1.12 (#2141)

mrwyattii committed 3 years ago

Verified 1a71e77d

jeffra committed 3 years ago

b005db86

Tensor parallelism for Mixture of Experts (#2074)

siddharth9820 committed 3 years ago

Verified 5fe9d610

Release swap buffers for persisted params (#2089)

tjruwase committed 3 years ago

Verified 2210ebe7

enable fp16 input autocasting (#2158)

jeffra committed 3 years ago

Verified a039e226

[zero-3] shutdown zero.Init from within ds.init (#2150)

jeffra committed 3 years ago

Verified 46401b38

prevent cuda 10 builds of inference kernels on ampere (#2157)

jeffra committed 3 years ago

Verified 63f470ee

Elastic Training support in DeepSpeed (#2153) (#2156)

aj-prime committed 3 years ago

Verified 1ed5aa96

Added retain_graph as a kwarg to the main engine backward function (#1149)

ncilfone committed 3 years ago

Verified ba67bd9a

Fix random token-generation issue + MP-checkpoint loading/saving (#2132)

RezaYazdaniAminabadi committed 3 years ago

Verified 556f0051

fix: fix BF16_Optimizer compatibility issue with optimizer state 0-dim tensor (#2152)

shjwudp committed 3 years ago

Verified 57140e8e

Graceful exit on failures for multi-node runs (#2008)

Jerry Mannil committed 3 years ago

Verified 66d29b0a

Trajepl/nebula ckpt engine (#2085)

trajepl committed 3 years ago

Verified e669aaf5

force newer datasets version (#2147)

jeffra committed 3 years ago

Verified a54661a0

formatting fix for #1962

jeffra committed 3 years ago

b442264d

Add purely-local sliding window sparse attention config (#1962)

Quentin-Anthony committed 3 years ago

Verified be46ff6d

Refactor ZeRO configs to use Pydantic (#2004)

mrwyattii committed 3 years ago

Verified 59975896

fix broken links (#2144)

jimwu6 committed 3 years ago

Verified 6f7137cb

[docs] README formatting fix

jeffra committed 3 years ago

76a01d5d

[docs] update build pipeline badges

jeffra committed 3 years ago

Verified 56a0d186

[docs] add details about install reqs (#2143)

jeffra committed 3 years ago

Verified 84dbe7c6

Allow turning off loss scaling wrt GAS + update tput calculator (#2140)

jeffra committed 3 years ago

Verified 5bd09a8f

Older