microsoft/DeepSpeed

Pull Requests Commits

test: add regression test for softmax_context_int8 alpha scale

delock committed 40 days ago

82ecffef

fix: attention_unfused alpha missing norm_factor squaring

delock committed 49 days ago

ceef30ff

Fix Muon optimizer under ZeRO CPU offload and bound gather buffers (#8464)

jinyouzhi committed 51 days ago

Verified d099bc61

Fix 2 CPU failures in cpu-torch-latest (#8629)

jinyouzhi committed 51 days ago

Verified 873d07b2

Run Muon once per optimizer step under ZeRO-3, not once per micro-batch (#8600)

alanhuangyoo committed 52 days ago

Verified 4a5856c2

Fix pipe tests failures on CPU (#8611)

jinyouzhi committed 52 days ago

Verified ee1ce8f2

Add opt-in regional compilation around AutoEP layers (#8380)

yh0903 committed 52 days ago

Verified 6c2fae83

AutoEP: share one DeepEP buffer across MoE layers instead of one per layer (#8557)

yh0903 committed 52 days ago

Verified 26a49fd3

Implement graph operations for NPU accelerator (#8605)

VenusTZZ committed 52 days ago

Verified 478228b6

Skip fp16 ZeroPP tests on accelerators without fp16 support (#8610)

jinyouzhi committed 52 days ago

Verified 2a4ee92b

Remove stale compression test from HPU CI (#8616)

nathon-lee committed 52 days ago

Verified 1eb56d29

AutoEP: drop the redundant float32 copy in the default combine (#8621)

pengdurice committed 52 days ago

Verified bbcfc8d5

Skip narrow dtypes in the mixed-precision cast (#8416)

sfc-gh-abkulkarni committed 53 days ago

Verified 88d2f816

Wait for all async all-gathers in coalesced ZeRO-3 parameter gather (#8539)

delock committed 53 days ago

Verified e71b8f33

Remove the older TiledLoss and UlyssesSPFwdLossBwdWithLogits (#8587)

vineethsaivs committed 53 days ago

Verified d7a2ceff

Normalize Gram Newton-Schulz inputs before casting to fp16 (#8533)

vineethsaivs committed 53 days ago

Verified 1663b5b5

Overlap AutoEP split planning with token packing (#8359)

yh0903 committed 53 days ago

Verified 0d4aa274

[AutoTP] Add optional Liger vocabulary-parallel cross entropy (#8540)

0z5a committed 53 days ago

Verified 4bd251b6

Add opt-in AutoEP Python GC policy (#8451)

yh0903 committed 53 days ago

Verified 43c53e9b

Fix silent gradient-bucket drop on CPU in ZeRO-1/2 non-contiguous reduction (#8382)

delock committed 53 days ago

Verified 20a60f22

Fix memory leak and NULL-deref in shm_initialize (#8565)

tritsystem committed 53 days ago

Verified 1f5f9888

Release AutoEP router-logit caches on checkpoint replay exits (#8581)

yh0903 committed 53 days ago

Verified b16a3fab

[AutoTP] Support eager row-parallel output head training (#8538)

0z5a committed 54 days ago

Verified 468a8ad1

[muon] Per-head Muon for linear-attention layers: read head geometry from the owning module (#8436)

alanhuangyoo committed 54 days ago

Verified 141ee9f8

Fix HybridEngine OPT generation with modern cache interfaces (#8523)

0z5a committed 54 days ago

Verified 6093db0e

Describe the GPTBigCode and Yuan shared-QK layouts (#8575)

Achyuthan-S committed 54 days ago

Verified b2370ad9

Use int64 program ids in the SwiGLU and AutoEP fused restore Triton kernels (#8591)

pengdurice committed 55 days ago

Verified bb5c23e0

Remove elastic training (#8563)

pengdurice committed 55 days ago

Verified 215f5aa9

implement register_host_memory and unregister_host_memory for supa (#8537)

frozenleaves committed 56 days ago

Verified 4e394422

Register native pinned host memory with the CANN runtime for NPU (#8532)

VenusTZZ committed 56 days ago

Verified 0ca14e83

Older