microsoft/DeepSpeed

Pull Requests Commits

adding new policy for GPTJ model architecture

Reza Yazdani committed 4 years ago

770e3b09

update for cuda-11.4 (#1329)

stas00 committed 4 years ago

Verified b9ece257

Use clone to avoid checkpoint bloat (#1326)

tjruwase committed 4 years ago

Verified 336dd089

Activation checkpointing improvements (#1254)

tjruwase committed 4 years ago

Verified 85acf14c

use scalar cpu-adam in case of exception in builder (#1259)

RezaYazdaniAminabadi committed 4 years ago

Verified 9645e7bc

Add more synchronizations and barriers for the multi-gpu inference case (#1309)

RezaYazdaniAminabadi committed 4 years ago

Verified 0ec11daa

jeffra committed 4 years ago

c1b0a4e1

Reducing the memory-overhead of creating model for multi-GPU run (#1244)

RezaYazdaniAminabadi committed 4 years ago

Verified 49b6a632

Support Callable type for client optimizer and lr_scheduler (#1316)

tjruwase committed 4 years ago

Verified 274c375c

[zero] restore fp16 params if no zero ckpts available (#1322)

jeffra committed 4 years ago

Verified aa121291

Add issue templates

jeffra committed 4 years ago

10b48405

[docs] update moe features and news post

jeffra committed 4 years ago

6cd5f87b

add MoE press release links

jeffra committed 4 years ago

058ab819

Updating the torch version check to numeric (#1314)

pruthvistony committed 4 years ago

Verified e804f150

add moe to sidebar

jeffra committed 4 years ago

e070a09a

MoE read the docs update (#1312)

jeffra committed 4 years ago

Verified 9cb64a1f

bump 0.5.1, DSE, moe docs

jeffra committed 4 years ago

a1de767a

DeepSpeed MoE (#1310)

awan-10 committed 4 years ago

Verified f2843244

Curriculum learning (#1307)

conglongli committed 4 years ago

Verified b2b34ae3

jeffra committed 4 years ago

504893ae

Use correct input size for splits (#1284)

tjruwase committed 4 years ago

Verified c543a41b

Align gradient partitioning order (#1274)

tjruwase committed 4 years ago

Verified b1b41754

Update adam.py (#1278)

DT6A committed 4 years ago

Verified 00320a9b

ZeRO-1 empty grads fix + tests (#1273)

jeffra committed 4 years ago

Verified adc21a4d

jeffra committed 4 years ago

1ff25748

[Doc] round_robin_gradients (#1261)

tjruwase committed 4 years ago

Verified 40c381df

query for libaio package using known package managers (#1250)

adammoody committed 4 years ago

Verified e82060d0

Use correct default for round robin gradients (#1258)

tjruwase committed 4 years ago

Verified 97f7ed9e

Fix cudaErrorInvalidConfiguration in attn_softmax() for large seq_length*heads values (#1239)

dfyz committed 4 years ago

Verified bfe7f0db

Make round robin gradient partitioning configurable (default False) (#1256)

tjruwase committed 4 years ago

Verified 4d420df5

Older