microsoft/DeepSpeed

Pull Requests Commits

jeffra committed 3 years ago

c1965126

Elasticity updates (squash) (#997)

jeffra committed 3 years ago

fb30b815

add moe to sidebar

jeffra committed 3 years ago

e070a09a

MoE read the docs update (#1312)

jeffra committed 3 years ago

Verified 9cb64a1f

bump 0.5.1, DSE, moe docs

jeffra committed 3 years ago

a1de767a

DeepSpeed MoE (#1310)

awan-10 committed 3 years ago

Verified f2843244

Curriculum learning (#1307)

conglongli committed 3 years ago

Verified b2b34ae3

jeffra committed 3 years ago

504893ae

Use correct input size for splits (#1284)

tjruwase committed 3 years ago

Verified c543a41b

Align gradient partitioning order (#1274)

tjruwase committed 3 years ago

Verified b1b41754

Update adam.py (#1278)

DT6A committed 3 years ago

Verified 00320a9b

ZeRO-1 empty grads fix + tests (#1273)

jeffra committed 3 years ago

Verified adc21a4d

jeffra committed 3 years ago

1ff25748

[Doc] round_robin_gradients (#1261)

tjruwase committed 3 years ago

Verified 40c381df

query for libaio package using known package managers (#1250)

adammoody committed 3 years ago

Verified e82060d0

Use correct default for round robin gradients (#1258)

tjruwase committed 3 years ago

Verified 97f7ed9e

Fix cudaErrorInvalidConfiguration in attn_softmax() for large seq_length*heads values (#1239)

dfyz committed 3 years ago

Verified bfe7f0db

Make round robin gradient partitioning configurable (default False) (#1256)

tjruwase committed 3 years ago

Verified 4d420df5

ignore overlap/contiguous_gradients if using zero 1 (#1246)

jeffra committed 3 years ago

Verified 6ae756c0

aio: note the reason for compiling with -O0 (#1249)

adammoody committed 3 years ago

Verified 10cc2c97

aio: test link against libaio using distutils (#1247)

adammoody committed 3 years ago

Verified 6ec84967

Fixing inference api for FP32 and non-masking GPT-based models (#1204)

RezaYazdaniAminabadi committed 3 years ago

Verified 6ba96289

[zero2] zero_param_shapes: switch to round_robin_fp16_groups (#1240)

stas00 committed 3 years ago

Verified 7392e459

[see_memory_usage] fix deprecation (#1234)

stas00 committed 3 years ago

Verified 32e85eda

enable async io op on powerpc architectures (#1224)

adammoody committed 3 years ago

Verified 89b0fb42

[zero3] fix reference counting in backward over multiple forwards (#1227)

stas00 committed 3 years ago

Verified 3fa24208

[zero3] release tmp memory when consolidating fp16 weights take2 (#1223)

stas00 committed 3 years ago

Verified 81039463

Bump addressable from 2.7.0 to 2.8.0 in /docs (#1225)

dependabot[bot] committed 3 years ago

Verified 54bed327

ZeRO2-Offload: Disable copy overlapping (#1219)

tjruwase committed 3 years ago

Verified 97207c8c

formatting fix for release script

jeffra committed 3 years ago

d1a7a55e

Older