microsoft/DeepSpeed

Pull Requests Commits

Adding communication optimizatiosn for MoE

minjiaz committed 2 years ago

c61f5a6d

use HF NeoX (#2087)

mrwyattii committed 2 years ago

Verified ee7ea3b8

Dont overwrite hook handles in flop profiler (#2106)

Sanger2000 committed 2 years ago

Verified 9027f861

[ds-inference] checkpoint loading => tqdm (#2107)

stas00 committed 2 years ago

Verified 16699d83

Add Inference support for running the BigScience-BLOOM Architecture (#2083)

RezaYazdaniAminabadi committed 2 years ago

Verified aa88137b

jeffra committed 2 years ago

2feaf6d6

Improving memory utilization of Z2+MoE (#2079)

siddharth9820 committed 2 years ago

Verified c1af73f7

fixed "None type has no len()" (#2091)

xiazeyu committed 2 years ago

Verified b0523787

Add missing newline for ZeroOneAdam parameter table (#2088)

manuelciosici committed 2 years ago

Verified db3252b0

remove require grad in params count (#2065)

cli99 committed 2 years ago

Verified 0ad08608

Codeowner addendum and fix to small model debugging script (#2076)

samadejacobs committed 2 years ago

Verified 50a652e7

Fix partition id in the fp32->fp16 param copying step for z2+cpu-offload (#2059)

siddharth9820 committed 2 years ago

Verified b3388e14

Check for bf16 support only if CUDA is available (#2049)

aphedges committed 2 years ago

Verified 3540ce74

[docs] fix broken read-the-docs build (#2075)

jeffra committed 2 years ago

Verified 559fb8e5

Comments for better understanding of zero stage1_2 (#2027)

kisseternity committed 2 years ago

Verified 9305916d

add ds inference paper (#2072)

jeffra committed 2 years ago

Verified 9fc4e5f1

Comms Benchmarks (#2040)

Quentin-Anthony committed 2 years ago

Verified 9b70ce56

Fix missing import in replace_module.py (#2050)

aphedges committed 2 years ago

Verified 76ea0534

correct partition_id in fp32 param -> fp16 param for MoE+z2 (#2058)

siddharth9820 committed 2 years ago

Verified 38a00bee

Fix for AMD unit tests (#2047)

mrwyattii committed 2 years ago

Verified 7bae53d1

Fix the half-precision version of CPU-Adam (#2032)

RezaYazdaniAminabadi committed 2 years ago

Verified a04480e1

Add compression papers (#2042)

conglongli committed 2 years ago

Verified ff87c4e1

fixed print statement (#2038)

mrwyattii committed 2 years ago

Verified 52181779

Split parameter offload from z3 (#2009)

tjruwase committed 2 years ago

Verified 678c3fe3

Retain available params until last use (#2016)

tjruwase committed 2 years ago

Verified 2a1a4096

Fix inference unit test import error catching (#2024)

mrwyattii committed 2 years ago

Verified ec1ec204

fix import errors (#2026)

KMFODA committed 2 years ago

Verified 735406e5

Use partition size (#2011)

tjruwase committed 2 years ago

Verified d86a2de9

DeepSpeed Monitor Module (Master) (#2013)

Quentin-Anthony committed 2 years ago

Verified c87f6ee2

DataLoader Length Fix (#1718)

Sanger2000 committed 2 years ago

Verified ae198e20

Older