microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'olruwase/round_robin_gradient_option' of github.com:microsoft/DeepSpeed into olruwase/round_robin_gradient_option

tjruwase committed 4 years ago

92d27e3c

Log config setting

tjruwase committed 4 years ago

cf4b34fc

Merge branch 'master' into olruwase/round_robin_gradient_option

tjruwase committed 4 years ago

Verified 8326d3b5

Fix cudaErrorInvalidConfiguration in attn_softmax() for large seq_length*heads values (#1239)

dfyz committed 4 years ago

Verified bfe7f0db

Merge with master

tjruwase committed 4 years ago

273b45f6

Use the correct default

tjruwase committed 4 years ago

88dc4486

Make round robin gradient partitioning configurable (default False) (#1256)

tjruwase committed 4 years ago

Verified 4d420df5

Make round robin gradient partitioning configurable (default False)

tjruwase committed 4 years ago

b9fe6bb8

ignore overlap/contiguous_gradients if using zero 1 (#1246)

jeffra committed 4 years ago

Verified 6ae756c0

aio: note the reason for compiling with -O0 (#1249)

adammoody committed 4 years ago

Verified 10cc2c97

aio: test link against libaio using distutils (#1247)

adammoody committed 4 years ago

Verified 6ec84967

Fixing inference api for FP32 and non-masking GPT-based models (#1204)

RezaYazdaniAminabadi committed 4 years ago

Verified 6ba96289

[zero2] zero_param_shapes: switch to round_robin_fp16_groups (#1240)

stas00 committed 4 years ago

Verified 7392e459

[see_memory_usage] fix deprecation (#1234)

stas00 committed 4 years ago

Verified 32e85eda

enable async io op on powerpc architectures (#1224)

adammoody committed 4 years ago

Verified 89b0fb42

[zero3] fix reference counting in backward over multiple forwards (#1227)

stas00 committed 4 years ago

Verified 3fa24208

[zero3] release tmp memory when consolidating fp16 weights take2 (#1223)

stas00 committed 4 years ago

Verified 81039463

Bump addressable from 2.7.0 to 2.8.0 in /docs (#1225)

dependabot[bot] committed 4 years ago

Verified 54bed327

ZeRO2-Offload: Disable copy overlapping (#1219)

tjruwase committed 4 years ago

Verified 97207c8c

formatting fix for release script

jeffra committed 4 years ago

d1a7a55e

bump to 0.4.4 + add release scripts

jeffra committed 4 years ago

dba0f431

revert part of #1220 (#1221)

stas00 committed 4 years ago

Verified bc451c09

[zero3] release tmp memory when consolidating fp16 weights (#1220)

stas00 committed 4 years ago

Verified 2660cc4d

[model weights] zero_to_fp32 multiple improvements (#1181)

stas00 committed 4 years ago

Verified 2a921069

enable cpu adam op on powerpc architectures (#1213)

adammoody committed 4 years ago

Verified f65ff908

json wrong format (#1210)

GeorgeSen committed 4 years ago

Verified 5652072e

improve debug (#1215)

stas00 committed 4 years ago

Verified 5127b2fa

[zero.Init] post_init partitining is to be run only by a child module (#1202)

stas00 committed 4 years ago

Verified 497b741f

[zero3] params_to_reduce isn't always there (#1214)

stas00 committed 4 years ago

Verified 91f58c06

Upgrade bundler to address CVE-2020-36327

jeffra committed 4 years ago

Verified aeea85b4

Older