microsoft/DeepSpeed

Pull Requests Commits

Matching fp16 with debugging codes

tjruwase committed 3 years ago

e48035b7

finish zero_stage 1 sharding

tjruwase committed 3 years ago

819abe2a

bf16+zero_stage_1 first cut

tjruwase committed 3 years ago

6f5ebc37

fp32 reduction; flattened tensors

tjruwase committed 3 years ago

a3d3576e

Got bf16 working

tjruwase committed 3 years ago

6eb4f1fa

jeffra committed 3 years ago

fb0dc00f

bf16 is supported w. zero 1, fix assert (#1779)

jeffra committed 3 years ago

Verified 8eef742f

Fix CPU-Offload: Send groups of parameter lists as the FP16 parameters (#1774)

RezaYazdaniAminabadi committed 3 years ago

Verified 5ca26277

[save_16bit_model] add missing prologue (#1741)

stas00 committed 3 years ago

Verified baef92e2

Prepare zero-3 optimizer for ckpt load/save (#1750)

tjruwase committed 3 years ago

Verified 5fe5b38e

unset torch arch list for JIT mode (#1765)

jeffra committed 3 years ago

Verified 674c7588

[Doc] Add async I/O op (#1754)

tjruwase committed 3 years ago

Verified 4f96ffd9

Load MoE checkpint at deepspeed inference-engine (#1759)

RezaYazdaniAminabadi committed 3 years ago

Verified 841f99d1

Fix autotuning config (#1660)

cli99 committed 3 years ago

Verified cccc5450

fixing the inference build path when pre-building the inference op (#1755)

RezaYazdaniAminabadi committed 3 years ago

Verified d3cad051

fix pytest issue (#1764)

jeffra committed 3 years ago

Verified 56eac382

fixing a bf16 support issue (#1760)

duli2012 committed 3 years ago

Verified 97f8a9eb

Loosen requirement on packaging dependency (#1758)

jeffra committed 3 years ago

Verified dbe8ee16

Improve how runner parses env var file (#1747)

liamcli committed 3 years ago

Verified dac9056e

Move param_shapes to model files (#1732)

tjruwase committed 3 years ago

Verified 135a6256

separate add and mul flops compute function (#1745)

cli99 committed 3 years ago

Verified ba9c4cc7

Fix the tensor-slicing with multi-GPU inference and kernel-injection (#1724)

RezaYazdaniAminabadi committed 3 years ago

Verified fa073576

[debug log] disable (#1736)

stas00 committed 3 years ago

Verified bea701a1

[distributed] print dist init only on rank 0 (#1737)

stas00 committed 3 years ago

Verified e26de20b

[zero3] remove debug print (#1733)

stas00 committed 3 years ago

Verified fdd59ca2

[config] fix assert message (#1734)

stas00 committed 3 years ago

Verified ed4bbe08

Multi-node save pid support + allow sparse-attn extra (#1728)

jeffra committed 3 years ago

Verified 9351266f

launcher save pid + require manual triton install for sparse-attn (#1727)

jeffra committed 3 years ago

Verified 171316fc

Add a very simple PyTorch Lightning test! (#1726)

Sean Naren committed 3 years ago

Verified df724e71

Add codespell to pre-commit checks (#1717)

aphedges committed 3 years ago

Verified 4cf970e6

Older