microsoft/DeepSpeed

Pull Requests Commits

allow OMP_NUM_THREADS in superoffload

delock committed 123 days ago

e9227620

add comments for failure

delock committed 124 days ago

33b9dab9

delock committed 128 days ago

0d15c5f2

allow config grad convert on CPU

delock committed 128 days ago

1978617c

Provide option to not convert GPU grad to FP32 before copy to CPU

delock committed 128 days ago

018bb69f

sfc-gh-truwase committed 130 days ago

b7cd78f0

Update email address (#7624)

sfc-gh-truwase committed 130 days ago

Verified 79caae1c

[TiledMLP] moe support (#7622)

stas00 committed 130 days ago

Verified 1b08325d

Clarify document of leaf module config (#7623)

tohtana committed 130 days ago

Verified 1ae1cdd8

Blog of zenflow binding study (#7614)

delock committed 131 days ago

Verified 2b68bbc5

Enable grad scaler for ZeRO-0 + torch.autocast path (#7619)

tohtana committed 133 days ago

Verified 71d077da

Super offload blog Chinese version (#7620)

delock committed 133 days ago

Verified 65322e10

add print_dist util (#7621)

stas00 committed 134 days ago

Verified 4eb37729

Improve leaf module interface (enable via config, relax matching criteria, add document, etc.) (#7604)

tohtana committed 134 days ago

Verified 7d9a2f2b

Show mismatching values when DeepCompile test fails (#7618)

tohtana committed 134 days ago

Verified 82a9db7e

DeepCompile: Use min_cut_rematerialization for partitioning joint graphs (#7609)

eternalNight committed 134 days ago

Verified 2a769889

[wall_clock_breakdown] always log stats when enabled (#7617)

stas00 committed 135 days ago

Verified 9cbd3edd

Fixed save_checkpoint race when consolidating NVMe offloaded tensors (#7613)

H1manshu21 committed 136 days ago

Verified e37c37ac

Fixed the issue that universal checkpoint cannot be loaded for stage3 when world size expansion. (#7599)

zhengchenyu committed 136 days ago

Verified 07e76bd4

Minor fix in the SuperOffload blog (#7612)

xylian86 committed 136 days ago

Verified 330f738c

DeepCompile: Fix IPG bucket clearing (#7610)

eternalNight committed 136 days ago

Verified aa90f544

Handle the case of DeepCompile's enabled but not activated (#7603)

tohtana committed 137 days ago

Verified e32e8173

Add venv to .gitignore (#7605)

zhengchenyu committed 137 days ago

Verified 177c25c9

Add blog for SuperOffload (#7594)

xylian86 committed 137 days ago

Verified 462d28c5

DeepCompile: Fuse allgather and downcast (#7588)

eternalNight committed 138 days ago

Verified 4efd7eca

DeepCompile: Specify tensor aliasing in C++ op schema (#7597)

eternalNight committed 139 days ago

Verified 6fcccfa2

Fixed the problem of loading universal checkpoint error in multi-machine mode. (#7601)

zhengchenyu committed 139 days ago

Verified 47b3fb5e

Change current_device() to current_device_name() (#7600)

delock committed 139 days ago

Verified 66c70312

Fix the universal checkpoint issue for stage3 when there are multiple subgroups. (#7585)

zhengchenyu committed 140 days ago

Verified 91d14527

Simplify leaf module hook (#7592)

tohtana committed 140 days ago

Verified 6ea345ae

Older