microsoft/DeepSpeed

Pull Requests Commits

update for cuda-11.4 (#1329)

stas00 committed 4 years ago

c7f3bc51

Fix PP checkpoint bloat (#1324)

tjruwase committed 4 years ago

Verified 72ce55ab

[zero] restore fp16 params if no zero ckpts available (#1322)

jeffra committed 4 years ago

db2f8a03

turn excessive noise off (#1293)

stas00 committed 4 years ago

Verified e65e511b

API for obtaining global gradient norm (#1292)

tjruwase committed 4 years ago

Verified cce85b89

Use mpu in DeepSpeedConfig() call (#1271)

tjruwase committed 4 years ago

Verified 6b428821

fix empty grad zero tests

jeffra committed 4 years ago

bc17042a

prevent none grads from being reduced

jeffra committed 4 years ago

4b354096

dont clear grads in stage 1 code path

jeffra committed 4 years ago

e9b5dffa

Correctness fix PP+ZeRO for gradient accumulation + updates from master (#1263)

jeffra committed 4 years ago

Verified f93e22b3

Cleaning up tensor/pipe parallel accounting. (#1252)

Shaden Smith committed 4 years ago

Verified d6945dea

Adds engine.was_step_applied() (#1251)

Shaden Smith committed 4 years ago

Verified 9dbfdbd4

added shaden's set_train_batch_size patches, plus formatting

jeffra committed 4 years ago

d26c258b

Merge branch 'master' into megatron2.4-3d

jeffra committed 4 years ago

Verified 081ddb5f

clean up logging (#1190)

stas00 committed 4 years ago

Verified a0292398

less verbose load

Shaden Smith committed 4 years ago

bd9e9539

undo noise (#1191)

stas00 committed 4 years ago

Verified bc019a53

Fix bugs about non-contiguous tensor broadcasting (#1168)

hyunwoongko committed 4 years ago

Verified 429cbc89

introduce debug utils (#1136)

stas00 committed 4 years ago

Verified c0c4ebf1

ZeRO 2+3 memory estimators (#965)

stas00 committed 4 years ago

Verified 0c1802cc

zero_to_fp32: restore persistent buffers (#1146)

stas00 committed 4 years ago

Verified df8b1f88

jeffra committed 4 years ago

0982ecca

remove torchvision dependency (#1178)

jeffra committed 4 years ago

Verified 3b689844

Avoid partitioning small activations (#1154)

tjruwase committed 4 years ago

Verified b1669c0d

add assert to make sure zero isn't enabled with 1bit-* (#1169)

jeffra committed 4 years ago

Verified 1ba3e8e3

patch in ompi local rank if local_rank isn't set (#1164)

jeffra committed 4 years ago

Verified 4c114a27

Remove hard torch dependency at install (#1166)

jeffra committed 4 years ago

Verified da1fe2f8

[Doc] Fix steps_per_print description (#1163)

tjruwase committed 4 years ago

Verified fa7921e2

Samyamr/largest partitioned params calculation fix (#1150)

samyam committed 4 years ago

Verified 4eaf9106

Fix bugs in the tutorial documentation (#1157)

hyunwoongko committed 4 years ago

Verified aa16828c

Older