microsoft/DeepSpeed

Pull Requests Commits

Debugging bf16/master merge

tjruwase committed 3 years ago

850af62a

Reduce tied weight gradients

tjruwase committed 3 years ago

37011a92

Fix clip_grad key error

tjruwase committed 3 years ago

35ea3808

Enable loading fp16_zero_1 into bf16_zero_1 engine and vice versa

tjruwase committed 3 years ago

b81d862f

Fix grad norm scaling

tjruwase committed 3 years ago

0ad7c7d3

Unscale grad norm

tjruwase committed 3 years ago

f4977024

bf16 gradient clipping fix

tjruwase committed 3 years ago

27e5b956

Fix gradient clipping

tjruwase committed 3 years ago

15293139

Matching loss with fp16

tjruwase committed 3 years ago

82450539

Matching fp16 with debugging codes

tjruwase committed 3 years ago

e48035b7

finish zero_stage 1 sharding

tjruwase committed 3 years ago

819abe2a

bf16+zero_stage_1 first cut

tjruwase committed 3 years ago

6f5ebc37

fp32 reduction; flattened tensors

tjruwase committed 3 years ago

a3d3576e

Got bf16 working

tjruwase committed 3 years ago

6eb4f1fa

jeffra committed 3 years ago

fb0dc00f

bf16 is supported w. zero 1, fix assert (#1779)

jeffra committed 3 years ago

Verified 8eef742f

Fix CPU-Offload: Send groups of parameter lists as the FP16 parameters (#1774)

RezaYazdaniAminabadi committed 3 years ago

Verified 5ca26277

[save_16bit_model] add missing prologue (#1741)

stas00 committed 3 years ago

Verified baef92e2

Prepare zero-3 optimizer for ckpt load/save (#1750)

tjruwase committed 3 years ago

Verified 5fe5b38e

unset torch arch list for JIT mode (#1765)

jeffra committed 3 years ago

Verified 674c7588

[Doc] Add async I/O op (#1754)

tjruwase committed 3 years ago

Verified 4f96ffd9

Load MoE checkpint at deepspeed inference-engine (#1759)

RezaYazdaniAminabadi committed 3 years ago

Verified 841f99d1

Fix autotuning config (#1660)

cli99 committed 3 years ago

Verified cccc5450

fixing the inference build path when pre-building the inference op (#1755)

RezaYazdaniAminabadi committed 3 years ago

Verified d3cad051

fix pytest issue (#1764)

jeffra committed 3 years ago

Verified 56eac382

fixing a bf16 support issue (#1760)

duli2012 committed 3 years ago

Verified 97f8a9eb

Loosen requirement on packaging dependency (#1758)

jeffra committed 3 years ago

Verified dbe8ee16

Improve how runner parses env var file (#1747)

liamcli committed 3 years ago

Verified dac9056e

Move param_shapes to model files (#1732)

tjruwase committed 3 years ago

Verified 135a6256

separate add and mul flops compute function (#1745)

cli99 committed 3 years ago

Verified ba9c4cc7

Older