microsoft/DeepSpeed

Pull Requests Commits

ZeRO-1/2: allocate CPU gradient accumulation lazily

stas00 committed 2 days ago

e5d68d99

stas00 committed 2 days ago

7e00e1a8

ZeRO-1/2: preserve gradients across request boundaries until step

seanbell committed 2 days ago

bfa3c3da

Check where column-split pieces write in a shard (#8922)

Achyuthan-S committed 2 days ago

Verified 8d4f29c7

Fix race between folded checkpoint load and metadata drop in AutoEP+AutoTP test (#8943)

jinyouzhi committed 2 days ago

Verified 1bdaaf73

[AutoEP] Ensure consistent order of partition groups in all-gather when ZeRO-3 (#8876)

jinyouzhi committed 2 days ago

Verified 12ffb990

Add a v1 multi-rank CPU suite (manual dispatch first) (#8926)

delock committed 3 days ago

Verified 8f9db2c6

[AffineIR] Affine storage maps for live ZeRO parameters and optimizer states (#8885)

jinyouzhi committed 3 days ago

Verified f8185049

Use nvtx's module-level API so NVTX domain ranges work on older nvtx releases (#8900)

hwchen2017 committed 3 days ago

Verified 3dd1293f

Fix performance regression caused by comms logger and bug in comms_logger.enabled (#8909)

hwchen2017 committed 4 days ago

Verified 322283b2

Skip accelerator sync for unprofiled comm ops (#8795)

guptaishaan committed 4 days ago

Verified b5829596

Keep fp32 masters on ZeRO-1 and ZeRO-2 when a resume skips optimizer state (#8891)

sfc-gh-truwase committed 4 days ago

Verified 2dd22dc1

AutoEP: fused Triton kernel for the clamped expert activations (#8652)

pengdurice committed 4 days ago

Verified 7501610e

Bump AWS capacity-fallback Python from 3.10.13 to 3.12.14 (#8921)

delock committed 4 days ago

Verified 8ae7921d

Strip ANSI color codes before parsing FAILED lines in cpu-multi-triage (#8882)

delock committed 4 days ago

Verified 507602ff

Honor fp16 hysteresis in the ZeRO stage 0 fp16 optimizers (#8647)

vineethsaivs committed 5 days ago

Verified 62656fe7

Fix scalar parameter byte-view broadcasts (#8642)

adenzhou1350 committed 5 days ago

Verified bf222a2a

Set TORCHINDUCTOR_COMPILE_THREADS=1 in modal CI sandbox to stop 300s test timeouts (#8914)

delock committed 5 days ago

Verified 1fcac823

[AffineIR] Plan expert transfers from the live runtime placement (#8800)

jinyouzhi committed 5 days ago

Verified 8a47f989

Bump modal-torch-latest CI Python from 3.10 to 3.12 (#8902)

delock committed 7 days ago

Verified fc7bc301

docs: fix remaining dead links in the Chinese README, tutorials and blog (#8894)

woodforestsen committed 8 days ago

Verified fb5e960b

docs: fix 30 dead links across README, docs pages, tutorials and posts (#8688)

pratikgx committed 9 days ago

Verified 9a31ccf4

[OPSD] Request last-position logits only in HybridEngine rollout prefill (#8839)

jinyouzhi committed 9 days ago

Verified ff34be65

Add RVV vectorized FP32 kernel to CPU Adam (#8817)

zhangjian29 committed 9 days ago

Verified 11ac9f86

Fix numerical stability in Triton masked softmax (#8782)

arcusbuilds committed 10 days ago

Verified 67ec1961

Add Triton kernels for vocab-parallel logprobs, entropy and KL (#8838)

jinyouzhi committed 10 days ago

Verified 0b0d4081

Expect AutoTP to keep a row-parallel output head tied to the embedding (#8855)

jinyouzhi committed 10 days ago

Verified 68faa67c

Keep Yuan's o_proj bias whole on every rank and describe it as replicated (#8857)

Achyuthan-S committed 11 days ago

Verified b39bf260

Skip the accelerator-only CPU optimizer tests on the cpu accelerator (#8836)

delock committed 11 days ago

Verified 127f89d9

Update to filter out version.txt from PR gates. (#8872)

loadams committed 11 days ago

Verified 0f8c9d7f

Older