microsoft/DeepSpeed

Pull Requests Commits

feat: Add Triton-optimized Muon optimizer implementation

delock committed 109 days ago

87f1979d

make torch.compile more adaptive to old pytorch version

delock committed 126 days ago

632ab6b2

apply torch.compile to Muon optimizer

delock committed 129 days ago

e02c0ec9

make muon optimizer totally running on GPU

delock committed 129 days ago

bbb4bfcf

Fix illegal memory access with multi_tensor_apply size above INT_MAX (#7639)

wangyan-mms committed 132 days ago

Verified 7af561c2

add support for tensor learning rate (vs scalar) (#7633)

NirSonnenschein committed 133 days ago

Verified 407708cd

Take **kwargs in __init__ of DeepSpeedZeroOptimizer subclasses (#7634)

eternalNight committed 133 days ago

Verified 2734a6a1

[XPU][CI] recover xpu-max1100 workflow (#7630)

Liangliang-Ma committed 140 days ago

Verified 69e03e52

Add ZenFlow code for Stage 3 (#7516)

JoshWoo2003 committed 140 days ago

Verified 7cb1b88e

sfc-gh-truwase committed 146 days ago

b7cd78f0

Update email address (#7624)

sfc-gh-truwase committed 146 days ago

Verified 79caae1c

[TiledMLP] moe support (#7622)

stas00 committed 146 days ago

Verified 1b08325d

Clarify document of leaf module config (#7623)

tohtana committed 146 days ago

Verified 1ae1cdd8

Blog of zenflow binding study (#7614)

delock committed 147 days ago

Verified 2b68bbc5

Enable grad scaler for ZeRO-0 + torch.autocast path (#7619)

tohtana committed 149 days ago

Verified 71d077da

Super offload blog Chinese version (#7620)

delock committed 149 days ago

Verified 65322e10

add print_dist util (#7621)

stas00 committed 149 days ago

Verified 4eb37729

Improve leaf module interface (enable via config, relax matching criteria, add document, etc.) (#7604)

tohtana committed 150 days ago

Verified 7d9a2f2b

Show mismatching values when DeepCompile test fails (#7618)

tohtana committed 150 days ago

Verified 82a9db7e

DeepCompile: Use min_cut_rematerialization for partitioning joint graphs (#7609)

eternalNight committed 150 days ago

Verified 2a769889

[wall_clock_breakdown] always log stats when enabled (#7617)

stas00 committed 151 days ago

Verified 9cbd3edd

Fixed save_checkpoint race when consolidating NVMe offloaded tensors (#7613)

H1manshu21 committed 152 days ago

Verified e37c37ac

Fixed the issue that universal checkpoint cannot be loaded for stage3 when world size expansion. (#7599)

zhengchenyu committed 152 days ago

Verified 07e76bd4

Minor fix in the SuperOffload blog (#7612)

xylian86 committed 152 days ago

Verified 330f738c

DeepCompile: Fix IPG bucket clearing (#7610)

eternalNight committed 152 days ago

Verified aa90f544

Handle the case of DeepCompile's enabled but not activated (#7603)

tohtana committed 153 days ago

Verified e32e8173

Add venv to .gitignore (#7605)

zhengchenyu committed 153 days ago

Verified 177c25c9

Add blog for SuperOffload (#7594)

xylian86 committed 153 days ago

Verified 462d28c5

DeepCompile: Fuse allgather and downcast (#7588)

eternalNight committed 154 days ago

Verified 4efd7eca

DeepCompile: Specify tensor aliasing in C++ op schema (#7597)

eternalNight committed 154 days ago

Verified 6fcccfa2

Older