deepspeedai/DeepSpeed

Pull Requests Commits

feat: Add Triton-optimized Muon optimizer implementation

delock committed 329 days ago

87f1979d

make torch.compile more adaptive to old pytorch version

delock committed 347 days ago

632ab6b2

apply torch.compile to Muon optimizer

delock committed 350 days ago

e02c0ec9

make muon optimizer totally running on GPU

delock committed 350 days ago

bbb4bfcf

Fix illegal memory access with multi_tensor_apply size above INT_MAX (#7639)

wangyan-mms committed 352 days ago

Verified 7af561c2

add support for tensor learning rate (vs scalar) (#7633)

NirSonnenschein committed 354 days ago

Verified 407708cd

Take **kwargs in __init__ of DeepSpeedZeroOptimizer subclasses (#7634)

eternalNight committed 354 days ago

Verified 2734a6a1

[XPU][CI] recover xpu-max1100 workflow (#7630)

Liangliang-Ma committed 361 days ago

Verified 69e03e52

Add ZenFlow code for Stage 3 (#7516)

JoshWoo2003 committed 361 days ago

Verified 7cb1b88e

sfc-gh-truwase committed 1 year ago

b7cd78f0

Update email address (#7624)

sfc-gh-truwase committed 1 year ago

Verified 79caae1c

[TiledMLP] moe support (#7622)

stas00 committed 1 year ago

Verified 1b08325d

Clarify document of leaf module config (#7623)

tohtana committed 1 year ago

Verified 1ae1cdd8

Blog of zenflow binding study (#7614)

delock committed 1 year ago

Verified 2b68bbc5

Enable grad scaler for ZeRO-0 + torch.autocast path (#7619)

tohtana committed 1 year ago

Verified 71d077da

Super offload blog Chinese version (#7620)

delock committed 1 year ago

Verified 65322e10

add print_dist util (#7621)

stas00 committed 1 year ago

Verified 4eb37729

Improve leaf module interface (enable via config, relax matching criteria, add document, etc.) (#7604)

tohtana committed 1 year ago

Verified 7d9a2f2b

Show mismatching values when DeepCompile test fails (#7618)

tohtana committed 1 year ago

Verified 82a9db7e

DeepCompile: Use min_cut_rematerialization for partitioning joint graphs (#7609)

eternalNight committed 1 year ago

Verified 2a769889

[wall_clock_breakdown] always log stats when enabled (#7617)

stas00 committed 1 year ago

Verified 9cbd3edd

Fixed save_checkpoint race when consolidating NVMe offloaded tensors (#7613)

H1manshu21 committed 1 year ago

Verified e37c37ac

Fixed the issue that universal checkpoint cannot be loaded for stage3 when world size expansion. (#7599)

zhengchenyu committed 1 year ago

Verified 07e76bd4

Minor fix in the SuperOffload blog (#7612)

xylian86 committed 1 year ago

Verified 330f738c

DeepCompile: Fix IPG bucket clearing (#7610)

eternalNight committed 1 year ago

Verified aa90f544

Handle the case of DeepCompile's enabled but not activated (#7603)

tohtana committed 1 year ago

Verified e32e8173

Add venv to .gitignore (#7605)

zhengchenyu committed 1 year ago

Verified 177c25c9

Add blog for SuperOffload (#7594)

xylian86 committed 1 year ago

Verified 462d28c5

DeepCompile: Fuse allgather and downcast (#7588)

eternalNight committed 1 year ago

Verified 4efd7eca

DeepCompile: Specify tensor aliasing in C++ op schema (#7597)

eternalNight committed 1 year ago

Verified 6fcccfa2

Older