microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into ckpt-fix-unfused

awan-10 committed 4 years ago

Verified 775c2cae

Fix the copying of params for the finetuning case.

awan-10 committed 4 years ago

1b6a2150

Fix all Pipeline Module Parameters being sent to cuda:0 (#687)

sdtblck committed 4 years ago

Verified 669028f0

Make reduce scatter optional for ZeRO-1 as workaround (#971)

tjruwase committed 4 years ago

Verified 0b80ad06

Use odd shape tensor to represent parameter data in partitioned state (#981)

cli99 committed 4 years ago

Verified 894f21da

1-bit LAMB optimizer (#970)

conglongli committed 4 years ago

Verified 67a48aaa

remove debug prints: (#986)

stas00 committed 4 years ago

Verified eecef309

Update fused_optimizer.py (#984)

awan-10 committed 4 years ago

Verified a0b4c024

make bold+italic work without escaping _ (#775)

stas00 committed 4 years ago

Verified 835b4c87

only load dynamic loss scale if load_optim_states is true.

awan-10 committed 4 years ago

aeec9747

Merge branch 'master' into ckpt-fix-unfused

tjruwase committed 4 years ago

Verified 930997c9

[ZeRO Infinity] Allow Init to take a dict for the deepspeed config (#983)

Sean Naren committed 4 years ago

Verified 35251023

awan-10 committed 4 years ago

97db6c58

Update unfused_optimizer.py

awan-10 committed 4 years ago

Verified be29975a

add option to force multi-node launcher mode (#977)

jeffra committed 4 years ago

Verified 9e0dab40

assert no Z2/Z3 with pipeline and fix some docs links (#980)

Shaden Smith committed 4 years ago

Verified fbece50b

jeffra committed 4 years ago

2c2a7f31

[docs] zero-inf updates

jeffra committed 4 years ago

5f570bbc

ZeRO-Infinity docs (#979)

Shaden Smith committed 4 years ago

Verified 11279ae4

[docs] zero-inf updates

jeffra committed 4 years ago

598e50f9

[docs] add ZeRO-Inf news items

jeffra committed 4 years ago

1a741957

ZeRO-Infinity tutorial additions (#978)

Shaden Smith committed 4 years ago

Verified 3c47d09c

revert zero-inf change to launcher

jeffra committed 4 years ago

72a30c1e

ZeRO-Infinity (#976)

jeffra committed 4 years ago

Verified 0d4a54a0

Fix ZeRO-3 UnboundLocalError (#968)

tjruwase committed 4 years ago

Verified 2805c393

update lr scheduler doc for doing per step or epoch update (#913)

cli99 committed 4 years ago

Verified c83e49f9

[zero] faster flatten/unflatten (cpp version) (#910)

stas00 committed 4 years ago

Verified 8b8ed2a7

document how to override ~/.cache/torch_extensions (#959)

stas00 committed 4 years ago

Verified 7003d447

[config] turn exponential notation back on for config dump (#955)

stas00 committed 4 years ago

Verified c87118b0

fix double linear override; spelling (#954)

stas00 committed 4 years ago

Verified adac058a

Older