deepspeedai/DeepSpeed

Pull Requests Commits

sfc-gh-truwase committed 1 year ago

b7cd78f0

Update email address (#7624)

sfc-gh-truwase committed 1 year ago

Verified 79caae1c

[TiledMLP] moe support (#7622)

stas00 committed 1 year ago

Verified 1b08325d

Clarify document of leaf module config (#7623)

tohtana committed 1 year ago

Verified 1ae1cdd8

Blog of zenflow binding study (#7614)

delock committed 1 year ago

Verified 2b68bbc5

Enable grad scaler for ZeRO-0 + torch.autocast path (#7619)

tohtana committed 1 year ago

Verified 71d077da

Super offload blog Chinese version (#7620)

delock committed 1 year ago

Verified 65322e10

add print_dist util (#7621)

stas00 committed 1 year ago

Verified 4eb37729

Improve leaf module interface (enable via config, relax matching criteria, add document, etc.) (#7604)

tohtana committed 1 year ago

Verified 7d9a2f2b

Show mismatching values when DeepCompile test fails (#7618)

tohtana committed 1 year ago

Verified 82a9db7e

DeepCompile: Use min_cut_rematerialization for partitioning joint graphs (#7609)

eternalNight committed 1 year ago

Verified 2a769889

[wall_clock_breakdown] always log stats when enabled (#7617)

stas00 committed 1 year ago

Verified 9cbd3edd

Fixed save_checkpoint race when consolidating NVMe offloaded tensors (#7613)

H1manshu21 committed 1 year ago

Verified e37c37ac

Fixed the issue that universal checkpoint cannot be loaded for stage3 when world size expansion. (#7599)

zhengchenyu committed 1 year ago

Verified 07e76bd4

Minor fix in the SuperOffload blog (#7612)

xylian86 committed 1 year ago

Verified 330f738c

DeepCompile: Fix IPG bucket clearing (#7610)

eternalNight committed 1 year ago

Verified aa90f544

Handle the case of DeepCompile's enabled but not activated (#7603)

tohtana committed 1 year ago

Verified e32e8173

Add venv to .gitignore (#7605)

zhengchenyu committed 1 year ago

Verified 177c25c9

Add blog for SuperOffload (#7594)

xylian86 committed 1 year ago

Verified 462d28c5

DeepCompile: Fuse allgather and downcast (#7588)

eternalNight committed 1 year ago

Verified 4efd7eca

DeepCompile: Specify tensor aliasing in C++ op schema (#7597)

eternalNight committed 1 year ago

Verified 6fcccfa2

Fixed the problem of loading universal checkpoint error in multi-machine mode. (#7601)

zhengchenyu committed 1 year ago

Verified 47b3fb5e

Change current_device() to current_device_name() (#7600)

delock committed 1 year ago

Verified 66c70312

Fix the universal checkpoint issue for stage3 when there are multiple subgroups. (#7585)

zhengchenyu committed 1 year ago

Verified 91d14527

Simplify leaf module hook (#7592)

tohtana committed 1 year ago

Verified 6ea345ae

disables ZeRO checkpoint loading path when stage=0 (#7586)

therealnaveenkamal committed 1 year ago

Verified b7565400

Include init file for superoffload folder (#7591)

nguyen599 committed 1 year ago

Verified 16c1bf42

SuperOffload Release (#7559)

xylian86 committed 1 year ago

Verified af56ed4d

Deepcompile: Make size of activation to free configurable (#7582)

eternalNight committed 1 year ago

Verified 17d80ce4

Broadcast fp16 overflow in Z1 (#7580)

sfc-gh-truwase committed 1 year ago

Verified bc9ed477

Older