microsoft/DeepSpeed

Pull Requests Commits

Update sparse_attn.py

jeffra committed 3 years ago

Verified d9ddaeb2

Move param_shapes to model files (#1732)

tjruwase committed 3 years ago

Verified 135a6256

separate add and mul flops compute function (#1745)

cli99 committed 3 years ago

Verified ba9c4cc7

Fix the tensor-slicing with multi-GPU inference and kernel-injection (#1724)

RezaYazdaniAminabadi committed 3 years ago

Verified fa073576

[debug log] disable (#1736)

stas00 committed 3 years ago

Verified bea701a1

[distributed] print dist init only on rank 0 (#1737)

stas00 committed 3 years ago

Verified e26de20b

[zero3] remove debug print (#1733)

stas00 committed 3 years ago

Verified fdd59ca2

[config] fix assert message (#1734)

stas00 committed 3 years ago

Verified ed4bbe08

Multi-node save pid support + allow sparse-attn extra (#1728)

jeffra committed 3 years ago

Verified 9351266f

launcher save pid + require manual triton install for sparse-attn (#1727)

jeffra committed 3 years ago

Verified 171316fc

Add a very simple PyTorch Lightning test! (#1726)

Sean Naren committed 3 years ago

Verified df724e71

Add codespell to pre-commit checks (#1717)

aphedges committed 3 years ago

Verified 4cf970e6

Align bfloat16 docs (#1715)

manuelciosici committed 3 years ago

Verified 09c065b4

Fix checkpoint api (#1714)

tjruwase committed 3 years ago

Verified e40558de

Various ZeRO Stage3 Optimizations + Improvements (including bfloat16 support) (#1453)

jfc4050 committed 3 years ago

Verified 4912e0ad

preserve cuda visible devices order (#1712)

jeffra committed 3 years ago

Verified 2d51f617

Fix inference api & add more description on inference engine tutorial (#1711)

RezaYazdaniAminabadi committed 3 years ago

Verified 94de0229

add logo and move news (#1709)

jeffra committed 3 years ago

Verified 2662fded

Reorganize MoE news and tutorials. (#1708)

awan-10 committed 3 years ago

Verified af074de3

Add more context for the MoE Inference tutorial (#1707)

RezaYazdaniAminabadi committed 3 years ago

Verified e27a60a8

pr moe tutorial creation (#1704)

yaozhewei committed 3 years ago

Verified 53fdadfb

add moe-inference tutorial (#1706)

RezaYazdaniAminabadi committed 3 years ago

Verified 38e16c69

MoE inference + PR-MoE model support (#1705)

jeffra committed 3 years ago

Verified e46d808a

[ZeRO] Default disable elastic ckpt in stage 1+2 and reduce CPU memory overhead during ckpt load (#1525)

jeffra committed 3 years ago

Verified 3293cf72

force clear stashed tensors (#1698)

jeffra committed 3 years ago

Verified e4cf40d6

support module and no python args for launcher (#1690)

liamcli committed 3 years ago

Verified fead387f

add -lcurand to fix torch-nightly issue w. JIT (#1688)

jeffra committed 3 years ago

Verified a85dce07

[docs] switch to transparent dark logo

jeffra committed 3 years ago

3a4cb042

fix the half-precision version of rotary_pos_emb kernel (#1683)

RezaYazdaniAminabadi committed 3 years ago

Verified 762e697a

GPT-J inference support (#1670)

RezaYazdaniAminabadi committed 3 years ago

Verified 289c3f9b

Older