microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into olruwase/restore_from_bit16_weights

tjruwase committed 3 years ago

Verified db207d87

Various ZeRO Stage3 Optimizations + Improvements (including bfloat16 support) (#1453)

jfc4050 committed 3 years ago

Verified 4912e0ad

Fix checkpoint api

tjruwase committed 3 years ago

6b9fee46

preserve cuda visible devices order (#1712)

jeffra committed 3 years ago

Verified 2d51f617

Fix inference api & add more description on inference engine tutorial (#1711)

RezaYazdaniAminabadi committed 3 years ago

Verified 94de0229

add logo and move news (#1709)

jeffra committed 3 years ago

Verified 2662fded

Reorganize MoE news and tutorials. (#1708)

awan-10 committed 3 years ago

Verified af074de3

Add more context for the MoE Inference tutorial (#1707)

RezaYazdaniAminabadi committed 3 years ago

Verified e27a60a8

pr moe tutorial creation (#1704)

yaozhewei committed 3 years ago

Verified 53fdadfb

add moe-inference tutorial (#1706)

RezaYazdaniAminabadi committed 3 years ago

Verified 38e16c69

MoE inference + PR-MoE model support (#1705)

jeffra committed 3 years ago

Verified e46d808a

[ZeRO] Default disable elastic ckpt in stage 1+2 and reduce CPU memory overhead during ckpt load (#1525)

jeffra committed 3 years ago

Verified 3293cf72

force clear stashed tensors (#1698)

jeffra committed 3 years ago

Verified e4cf40d6

support module and no python args for launcher (#1690)

liamcli committed 3 years ago

Verified fead387f

add -lcurand to fix torch-nightly issue w. JIT (#1688)

jeffra committed 3 years ago

Verified a85dce07

[docs] switch to transparent dark logo

jeffra committed 3 years ago

3a4cb042

fix the half-precision version of rotary_pos_emb kernel (#1683)

RezaYazdaniAminabadi committed 3 years ago

Verified 762e697a

GPT-J inference support (#1670)

RezaYazdaniAminabadi committed 3 years ago

Verified 289c3f9b

[docs] add gh-dark-mode logo

jeffra committed 3 years ago

7e857aab

[docs] add light-mode logo

jeffra committed 3 years ago

Verified 9c5cf3a5

add segfault guard for cpu-adam/adagrad (#1681)

jeffra committed 3 years ago

Verified 2422ec48

Copy grads to cpu in z1-offload (#1679)

tjruwase committed 3 years ago

Verified cef116f8

[docs] add logo (#1676)

jeffra committed 3 years ago

Verified c2735996

update results about public Pile dataset (#1675)

conglongli committed 3 years ago

Verified aca64799

Fix largest param numel calculation (#1623)

tjruwase committed 3 years ago

Verified 4354c3cc

support CPU Adam and Adagrad on Windows with SDK 10.0.22000 (#1634)

lostmsu committed 3 years ago

Verified 74493b2b

jeffra committed 3 years ago

b6f0ac97

Various small documentation text improvements (#1665)

manuelciosici committed 3 years ago

Verified d0ab7224

Convert the fp16_params to group of parameters (#1651)

RezaYazdaniAminabadi committed 3 years ago

Verified 559c4ce1

Replace calls to print() with calls to logger (#1664)

manuelciosici committed 3 years ago

Verified 40ce131c

Older