microsoft/DeepSpeed

Pull Requests Commits

Remove eval loss curve from Muon blog

delock committed 4 days ago

454eb1a2

Add Peng Du (@pengdurice) to Muon blog contributors

delock committed 5 days ago

e7987967

Update Muon blog: convergence results, citation fixes, and DeepSeek-V4

delock committed 11 days ago

52c9c296

Merge branch 'master' into gma/muon_blog

sfc-gh-truwase committed 12 days ago

Verified 86e5bf05

Dynamic offload compatible with static optimizer offload (#7979)

sfc-gh-truwase committed 16 days ago

Verified f6d7ae48

Fix Adam subgroup inconsistency (#7982)

st-bang97 committed 17 days ago

Verified 44c51e34

deepcompile: Fix backward graph recompilation due to unbalanced forward/backward visits (#7980)

eternalNight committed 18 days ago

Verified 077bff55

Fix ZeRO-1/2 CPU-offloaded gradient loss with multiple backward() per step (#7981)

roycho96 committed 18 days ago

Verified aeb10bb1

Avoid CUDA reinit error in CI tests (#7977)

tohtana committed 23 days ago

Verified 83d43c35

Merge branch 'master' into gma/muon_blog

delock committed 23 days ago

Verified 28db3d4b

Rename dequantization template parameters (#7976)

Flamefire committed 25 days ago

Verified dc0fd295

fix(op_builder): avoid duplicate/wrong -gencode flags (#7974)

Cursx committed 25 days ago

Verified 893c6d20

fix(fp_quantizer): fix UB and negative shift warnings in fp_quantize_impl.cu (#7973)

Cursx committed 25 days ago

Verified 3ba3dc97

Enable PyTorch version selection for full test (#7968)

tohtana committed 28 days ago

Verified 0ba23529

Fix WarmupCosineLR multi-group initialization (#7969)

tohtana committed 28 days ago

Verified 3fd762cf

Fix DeepCompile+Z3 on PyTorch v2.9/2.10 (#7951)

tohtana committed 29 days ago

Verified ecb26a51

Fix overlap-comm buffer lifetimes (#7965)

tohtana committed 29 days ago

Verified dac1525b

Fix zero 1 and 2 CPU-offloaded gradient norm (#7967)

alek6kun committed 29 days ago

Verified 9d632f10

Refine Muon blog: convergence results, LR tuning guide, and formatting fixes

delock committed 30 days ago

79325337

Add Muon blog to Latest News in README and docs landing page

delock committed 31 days ago

a91018f0

Add GLM-5 as Muon adopter and fix What's Next roadmap

delock committed 31 days ago

5da5fad6

Revamp future plan into What's Next with active roadmap tone

delock committed 31 days ago

fee921db

Add Muon pretraining convergence advantage to What is Muon section

delock committed 31 days ago

379e6b82

Update Muon blog future plan: mark ZeRO stage 3 and Gram NS as done

delock committed 31 days ago

be3cb5d3

Update Muon blog with measured convergence and memory data

delock committed 31 days ago

7abc53a6

Add training configuration caption to convergence chart

delock committed 31 days ago

8d9433e2

Add convergence experiment result and fix typos in Muon blog

delock committed 31 days ago

7c57422e

delock committed 31 days ago

0890c495

delock committed 31 days ago

f2967a1d

remove memory data

delock committed 31 days ago

dc7c02bf

Older