microsoft/DeepSpeed

Pull Requests Commits

Complete reorg of pointwise operations, migrate padding kernels to new style

cmikeh2 committed 2 years ago

5870921a

Merge remote-tracking branch 'origin/master' into inference/add-bf16-support

cmikeh2 committed 2 years ago

c324a942

Disable kernel build based on data type

cmikeh2 committed 2 years ago

8ace956f

Change zero_grad() argument to match pytorch (#2741)

loadams committed 2 years ago

Verified 34a11688

WIP prototype of kernel changes. Does not compile.

cmikeh2 committed 2 years ago

f4e137ab

Inference Refactor (replace_with_policy, model_implementations) (#2554)

awan-10 committed 2 years ago

Verified 867da307

fix typo (#2718)

mrwyattii committed 2 years ago

Verified 8df50a26

BF16 optimizer for BF16+ZeRO Stage 1 (#2706)

jomayeri committed 2 years ago

Verified 8d87c89e

update for lm-eval==0.3.0 (#2713)

mrwyattii committed 2 years ago

Verified 23e5133c

[install] only add deepspeed pkg at install (#2714)

jeffra committed 2 years ago

Verified 0b549ad7

remove master branch from CI triggers (#2712)

mrwyattii committed 2 years ago

Verified df2495ca

CUDA optional deepspeed ops (#2507)

tjruwase committed 2 years ago

Verified 3f210c97

jeffra committed 2 years ago

7d0e4270

ZeRO3 handling frozen weights] (#2653)

tjruwase committed 2 years ago

Verified bf6b9802

remove print side effect from importing deepspeed (#2704)

jeffra committed 2 years ago

Verified 35575bce

non-MoE stage 1 requires CG disabled (#2703)

jeffra committed 2 years ago

Verified e4ba7222

using correct loss scale in zero step (#2695)

jomayeri committed 2 years ago

Verified fe728e3e

exclude benchmarks during install (#2698)

jeffra committed 2 years ago

Verified cd271a4a

fix for latest diffusers (#2699)

mrwyattii committed 2 years ago

Verified c9c6ab9e

[GatheredParameters] add support for any iterator (#2664)

stas00 committed 2 years ago

Verified 217cc07b

Extend quantization utils features (#2683)

lokoppakmsft committed 2 years ago

Verified aef8a856

Pass training flag to forward call from Eval (#2604)

lokoppakmsft committed 2 years ago

Verified e7c14026

fix import path to op_builder (#2687)

tohtana committed 2 years ago

Verified cf9e433f

Add mlflow logging for aml (#2495)

cassieesvelt committed 2 years ago

Verified a3d7f106

remove duplicated code in ZeRO (#2655)

wkcn committed 2 years ago

Verified 89da037e

real_accelerator validation check for both accelerator and deepspeed.accelerator path (#2685)

delock committed 2 years ago

Verified 62c071e0

[inference] ds-mlp refactor w.r.t. ops (#2668)

jeffra committed 2 years ago

Verified c702b64c

fix Tensor contiguous bug in model_compression (#2671)

xiaoxiawu-microsoft committed 2 years ago

Verified be6d19f0

[fp16] lower initial_scale_power (#2663)

stas00 committed 2 years ago

Verified f30a0308

[Bug Fixed] use torch.cuda.is_available() (#2661)

wkcn committed 2 years ago

Verified 323c266c

Older