microsoft/DeepSpeed

Pull Requests Commits

Share zero_parameters for post-init ZeRO-3 ds_* walks

sfc-gh-truwase committed 5 days ago

4f9979d3

Skip unpartitioned parameters where the module is walked for ds_numel and ds_tensor

alanhuangyoo committed 6 days ago

48c6920c

Leave zero-element parameters out of ZeRO-3 partitioning

alanhuangyoo committed 6 days ago

8b31dbd0

Skip fp16-config tests on accelerators without fp16 support (#8684)

delock committed 7 days ago

Verified df00e0ba

fix(rollout): forward top-k sampling configuration (#8676)

nathon-lee committed 7 days ago

Verified 729d5aeb

Fix Triton compatibility for FlashAttention and MatMul autotuning (#8679)

hwchen2017 committed 7 days ago

Verified 5a0edd15

Implement FusedLion, FusedLamb, and Quantizer ops for NPU accelerator (#8627)

VenusTZZ committed 7 days ago

Verified 2b14a066

Add `DualPipeV` (#8576)

phoeenniixx committed 7 days ago

Verified 7687a786

Skip ZeRO DeepCompile backward hooks for AutoSP (#8667)

jinyouzhi committed 7 days ago

Verified 016fb769

AutoEP: Add activation registry (#8650)

pengdurice committed 7 days ago

Verified c193f222

fix(rollout): reject shared prefill with continuous batching (#8583)

modelpath-dev committed 8 days ago

Verified aa49eaab

docs: add Simplified Chinese translation (README.zh.md) (#8498)

JasonYeYuhe committed 8 days ago

Verified 5d2e2c47

[AutoTP] Implement VocabParallelEmbedding and enhance tied model support (#8582)

jinyouzhi committed 8 days ago

Verified 5f9f48f0

Keep ZenFlow Adam step counters in float32 (#8496)

vineethsaivs committed 8 days ago

Verified 2f3632a3

Replay AutoEP's cached DeepEP dispatch in the layout it was built with (#8636)

yh0903 committed 9 days ago

Verified bc778b8c

Deprecate and remove graph harvesting (#8579)

hwchen2017 committed 10 days ago

Verified 5340ba59

Fix Muon aspect-ratio scale truncated when muon_update recompiles (#8641)

alanhuangyoo committed 10 days ago

Verified bc44b341

A zero-element parameter loses its shape when it is bound to a flat buffer (#8467)

alanhuangyoo committed 10 days ago

Verified ca7455ef

Stage Muon's momentum once per partition in ZeRO-1/2 (#8633)

alanhuangyoo committed 10 days ago

Verified 1355e9da

Avoid repeated full gradient clearing in ZeRO-3 subgroup updates (#8625)

aprylewu committed 10 days ago

Verified 581ac41b

fix: Remove the unused ROCm Apex clone, build, and PYTHONPATH setup. (#8617)

nathon-lee committed 10 days ago

Verified 67dbc35d

Fix SequenceTiledCompute backward for empty trailing shards (#8434)

taking-lying-flat committed 10 days ago

Verified 9fb97fc5

Add affine AutoEP checkpoint placement (#8544)

jinyouzhi committed 11 days ago

Verified 35710273

Loosen the late-iteration gradient tolerance in the checkpointing test (#8648)

delock committed 11 days ago

Verified a4490b2a

Fix Muon optimizer under ZeRO CPU offload and bound gather buffers (#8464)

jinyouzhi committed 12 days ago

Verified d099bc61

Fix 2 CPU failures in cpu-torch-latest (#8629)

jinyouzhi committed 12 days ago

Verified 873d07b2

Run Muon once per optimizer step under ZeRO-3, not once per micro-batch (#8600)

alanhuangyoo committed 12 days ago

Verified 4a5856c2

Fix pipe tests failures on CPU (#8611)

jinyouzhi committed 13 days ago

Verified ee1ce8f2

Add opt-in regional compilation around AutoEP layers (#8380)

yh0903 committed 13 days ago

Verified 6c2fae83

AutoEP: share one DeepEP buffer across MoE layers instead of one per layer (#8557)

yh0903 committed 13 days ago

Verified 26a49fd3

Older