microsoft/DeepSpeed

Pull Requests Commits

fix softmax issue & curand_generator on A100

Reza Yazdani committed 3 years ago

70420bf3

add unit test for checking new kernel functionality

Reza Yazdani committed 3 years ago

ea87c94c

add the softmax-dropout kernel

Reza Yazdani committed 3 years ago

5537e8c8

jeffra committed 3 years ago

90ab62cf

add zero12 support

jeffra committed 3 years ago

2ac608e6

fix linear kwargs, fix tracer issue, support fs+z0

jeffra committed 3 years ago

9d042a42

temporarily fix zero-3 tracer bug.

awan-10 committed 3 years ago

4799ce87

add list support for bwd

jeffra committed 3 years ago

d2576c10

progress on latest master

jeffra committed 3 years ago

2e61a864

Align bfloat16 docs (#1715)

manuelciosici committed 3 years ago

Verified 09c065b4

Fix checkpoint api (#1714)

tjruwase committed 3 years ago

Verified e40558de

Various ZeRO Stage3 Optimizations + Improvements (including bfloat16 support) (#1453)

jfc4050 committed 3 years ago

Verified 4912e0ad

preserve cuda visible devices order (#1712)

jeffra committed 3 years ago

Verified 2d51f617

Fix inference api & add more description on inference engine tutorial (#1711)

RezaYazdaniAminabadi committed 3 years ago

Verified 94de0229

add logo and move news (#1709)

jeffra committed 3 years ago

Verified 2662fded

Reorganize MoE news and tutorials. (#1708)

awan-10 committed 3 years ago

Verified af074de3

Add more context for the MoE Inference tutorial (#1707)

RezaYazdaniAminabadi committed 3 years ago

Verified e27a60a8

pr moe tutorial creation (#1704)

yaozhewei committed 3 years ago

Verified 53fdadfb

add moe-inference tutorial (#1706)

RezaYazdaniAminabadi committed 3 years ago

Verified 38e16c69

MoE inference + PR-MoE model support (#1705)

jeffra committed 3 years ago

Verified e46d808a

[ZeRO] Default disable elastic ckpt in stage 1+2 and reduce CPU memory overhead during ckpt load (#1525)

jeffra committed 3 years ago

Verified 3293cf72

force clear stashed tensors (#1698)

jeffra committed 3 years ago

Verified e4cf40d6

support module and no python args for launcher (#1690)

liamcli committed 3 years ago

Verified fead387f

add -lcurand to fix torch-nightly issue w. JIT (#1688)

jeffra committed 3 years ago

Verified a85dce07

[docs] switch to transparent dark logo

jeffra committed 3 years ago

3a4cb042

fix the half-precision version of rotary_pos_emb kernel (#1683)

RezaYazdaniAminabadi committed 3 years ago

Verified 762e697a

GPT-J inference support (#1670)

RezaYazdaniAminabadi committed 3 years ago

Verified 289c3f9b

[docs] add gh-dark-mode logo

jeffra committed 3 years ago

7e857aab

[docs] add light-mode logo

jeffra committed 3 years ago

Verified 9c5cf3a5

add segfault guard for cpu-adam/adagrad (#1681)

jeffra committed 3 years ago

Verified 2422ec48

Older