microsoft/DeepSpeed

Pull Requests Commits

Fix pipe tests failures on CPU (#8611)

jinyouzhi committed 2 hours ago

Verified ee1ce8f2

Add opt-in regional compilation around AutoEP layers (#8380)

yh0903 committed 2 hours ago

Verified 6c2fae83

AutoEP: share one DeepEP buffer across MoE layers instead of one per layer (#8557)

yh0903 committed 4 hours ago

Verified 26a49fd3

Implement graph operations for NPU accelerator (#8605)

VenusTZZ committed 4 hours ago

Verified 478228b6

Skip fp16 ZeroPP tests on accelerators without fp16 support (#8610)

jinyouzhi committed 4 hours ago

Verified 2a4ee92b

Remove stale compression test from HPU CI (#8616)

nathon-lee committed 11 hours ago

Verified 1eb56d29

AutoEP: drop the redundant float32 copy in the default combine (#8621)

pengdurice committed 12 hours ago

Verified bbcfc8d5

Skip narrow dtypes in the mixed-precision cast (#8416)

sfc-gh-abkulkarni committed 14 hours ago

Verified 88d2f816

Wait for all async all-gathers in coalesced ZeRO-3 parameter gather (#8539)

delock committed 16 hours ago

Verified e71b8f33

Remove the older TiledLoss and UlyssesSPFwdLossBwdWithLogits (#8587)

vineethsaivs committed 17 hours ago

Verified d7a2ceff

Normalize Gram Newton-Schulz inputs before casting to fp16 (#8533)

vineethsaivs committed 20 hours ago

Verified 1663b5b5

Overlap AutoEP split planning with token packing (#8359)

yh0903 committed 21 hours ago

Verified 0d4aa274

[AutoTP] Add optional Liger vocabulary-parallel cross entropy (#8540)

0z5a committed 1 day ago

Verified 4bd251b6

Add opt-in AutoEP Python GC policy (#8451)

yh0903 committed 1 day ago

Verified 43c53e9b

Fix silent gradient-bucket drop on CPU in ZeRO-1/2 non-contiguous reduction (#8382)

delock committed 1 day ago

Verified 20a60f22

Fix memory leak and NULL-deref in shm_initialize (#8565)

tritsystem committed 1 day ago

Verified 1f5f9888

Release AutoEP router-logit caches on checkpoint replay exits (#8581)

yh0903 committed 1 day ago

Verified b16a3fab

[AutoTP] Support eager row-parallel output head training (#8538)

0z5a committed 2 days ago

Verified 468a8ad1

[muon] Per-head Muon for linear-attention layers: read head geometry from the owning module (#8436)

alanhuangyoo committed 2 days ago

Verified 141ee9f8

Fix HybridEngine OPT generation with modern cache interfaces (#8523)

0z5a committed 2 days ago

Verified 6093db0e

Describe the GPTBigCode and Yuan shared-QK layouts (#8575)

Achyuthan-S committed 2 days ago

Verified b2370ad9

Use int64 program ids in the SwiGLU and AutoEP fused restore Triton kernels (#8591)

pengdurice committed 3 days ago

Verified bb5c23e0

Remove elastic training (#8563)

pengdurice committed 3 days ago

Verified 215f5aa9

implement register_host_memory and unregister_host_memory for supa (#8537)

frozenleaves committed 3 days ago

Verified 4e394422

Register native pinned host memory with the CANN runtime for NPU (#8532)

VenusTZZ committed 4 days ago

Verified 0ca14e83

Support Muon in BF16_Optimizer with FP32 gradient accumulation (#8526)

0z5a committed 4 days ago

Verified 5880d38c

Remove Progressive Layer Dropping (PLD) support (#8562)

hwchen2017 committed 4 days ago

Verified 9943f39b

[AutoTP] Add support for vocab-parallel LM head and related configurations (#8309)

jinyouzhi committed 4 days ago

Verified ef7d23f4

Upgrade `yapf` to v0.43.0 to fix EOFError failure in CI (#8574)

jinyouzhi committed 4 days ago

Verified 9ce505d8

[muon] Per-head Newton-Schulz for attention projections (#8384)

alanhuangyoo committed 4 days ago

Verified fe8c4b10

Older