deepspeedai/DeepSpeed

Pull Requests Commits

Support flash-attn 2.7.0 in FPDT attention (#8022)

xbcReal committed 1 day ago

Verified 45429221

fix(zero): enable vmap on LinearFunctionForZeroStage3 (#8023)

roycho96 committed 1 day ago

Verified ae576f83

Sort and dedupe -gencode flags emitted by op_builder.builder (#8021)

adityasingh2400 committed 2 days ago

Verified 43166463

fix: add setup_context for torch.func compatibility (#7916)

roycho96 committed 4 days ago

Verified 67b9e212

fix gemma4 num attention head bugs (from #7975) (#7990)

delock committed 4 days ago

Verified bd71eed6

[Blog] Muon Optimizer Support in DeepSpeed (#7962)

delock committed 4 days ago

Verified de473091

[fix] fix test_zf.py hang bug (#8012)

xbcReal committed 4 days ago

Verified 26ee3856

Support bf16 optimizer states with CPU offload (#8010)

lucaspirola committed 4 days ago

Verified 3c337b54

Make GitHub Actions job names unique (#8014)

tohtana committed 4 days ago

Verified 9f95452e

Run FastFileWriter fd-close test outside pytest-forked (#8015)

tohtana committed 4 days ago

Verified d6d4b3d0

Add configurable torch-latest dependency versions (#8016)

tohtana committed 5 days ago

Verified b387166f

Remove stale step() docstring from DeepSpeedCPUAdam (#8011)

lucaspirola committed 5 days ago

Verified ae075141

Fix ZeRO-3 forward crash on modules with plain dict _parameters (#8009)

roycho96 committed 5 days ago

Verified d7a3972f

fix: use subprocess instead of os.system in data_analyzer.py (#7994)

orbisai0security committed 6 days ago

Verified 8cdf8651

Auto-detect CUTLASS for EvoformerAttention (#8000)

MaxTretikov committed 6 days ago

Verified d5356e07

fix(io): close aio_fd in FastFileWriter._fini to prevent fd leak (#8005)

jg-heo committed 8 days ago

Verified b01a0915

zero3: SDMA allgather via mori (sdma_allgather) (#7999)

inkcherry committed 11 days ago

Verified 66af8f03

Optimize singleton MoE collectives (#7997)

Tianyi-Franklin-Wang committed 11 days ago

Verified 55491413

Update topkgating probability-mask test expectation (#8007)

tohtana committed 12 days ago

Verified 5be4c70c

Add office hours times/link on the README (#8004)

loadams committed 13 days ago

Verified 883fefb2

Update version post-v0.19.0 release (#7996)

loadams committed 18 days ago

Verified a7811998

Update version.txt before 0.19.0 release (#7995)

loadams committed 18 days ago

Verified 98a2fbdf

[AutoSP] (Sequence Parallelism) support for Multimodal Models (ViT + LLM) (#7984)

nathon-lee committed 24 days ago

Verified 4e668fce

Add Gram Newton-Schulz orthogonalization for Muon optimizer (#7953)

delock committed 24 days ago

Verified 8a77f381

Add DeepSpeed NVTX domain support (#7988)

heurry committed 25 days ago

Verified 5b441027

fix: topkgating major bug (#7986)

excepshenal committed 25 days ago

Verified 853c9389

Fix BF16_Optimizer last-microbatch grad leak under ZeRO-1 (#7985)

maxyu1115 committed 26 days ago

Verified 5999fb06

Fix modal ci timeout (#7989)

sfc-gh-truwase committed 26 days ago

Verified 6e291bee

Dynamic offload compatible with static optimizer offload (#7979)

sfc-gh-truwase committed 31 days ago

Verified f6d7ae48

Fix Adam subgroup inconsistency (#7982)

st-bang97 committed 32 days ago

Verified 44c51e34

Older