microsoft/DeepSpeed

Pull Requests Commits

Reduce CPU memory overhead during ZeRO checkpoint loading (#1524)

jeffra committed 3 years ago

Verified 165739a5

pass gas boundary from engine->zero and add support to fp16 w/o zero

jeffra committed 3 years ago

42278e1e

formatting + allow external gas

jeffra committed 3 years ago

a83ffe05

curriculum learning (#1393)

conglongli committed 3 years ago

Verified 6a40bc59

remove bad import from cherry-pick

jeffra committed 3 years ago

dab6dc86

API for obtaining global gradient norm (#1292)

tjruwase committed 3 years ago

8a4dc5f1

jeffra committed 3 years ago

a11ab513

support passing model load func

jeffra committed 3 years ago

fedec15d

restore fp16 params if no zero ckpts available

jeffra committed 3 years ago

115cb52b

scale_wrt_gas support and ensure ckpt global steps aligned

jeffra committed 3 years ago

60c9d0ac

jeffra committed 3 years ago

9629ce23

Merge branch 'jeffra/engine-xthru-v2' of github.com:microsoft/DeepSpeed into jeffra/engine-xthru-v2

jeffra committed 3 years ago

2a790647

log custom loss scale

jeffra committed 3 years ago

2cfff51c

jeffra committed 3 years ago

Verified 38dab24c

fix logging for custom loss scale

jeffra committed 3 years ago

5e824457

Merge branch 'master' of github.com:microsoft/DeepSpeed into jeffra/engine-xthru

jeffra committed 3 years ago

bd19394e

add rigid ckpt support to zero-2

jeffra committed 3 years ago

935d5ef5

ignore overlap/contiguous_gradients if using zero 1 (#1246)

jeffra committed 3 years ago

Verified 6ae756c0

aio: note the reason for compiling with -O0 (#1249)

adammoody committed 3 years ago

Verified 10cc2c97

aio: test link against libaio using distutils (#1247)

adammoody committed 3 years ago

Verified 6ec84967

Update utils.py

jeffra committed 3 years ago

Verified e5f0f8b8

lazy load fairseq

jeffra committed 3 years ago

f4d97b1e

jeffra committed 3 years ago

c2912732

jeffra committed 3 years ago

210c6c70

add fairseq adam to zero compatible optims

jeffra committed 3 years ago

f0f2aa82

client override loss scale

jeffra committed 3 years ago

96af261f

Fixing inference api for FP32 and non-masking GPT-based models (#1204)

RezaYazdaniAminabadi committed 3 years ago

Verified 6ba96289

[zero2] zero_param_shapes: switch to round_robin_fp16_groups (#1240)

stas00 committed 3 years ago

Verified 7392e459

jeffra committed 3 years ago

295ad53c

[see_memory_usage] fix deprecation (#1234)

stas00 committed 3 years ago

Verified 32e85eda

Older