deepspeedai/DeepSpeed

Pull Requests Commits

Merge remote-tracking branch 'origin/master' into jeffra/913-fixes-master-sync-r2

sfc-gh-jrasley committed 11 hours ago

5e3922e4

default on with z2 + cpu offload

sfc-gh-jrasley committed 11 hours ago

7a28b842

Run Muon once per optimizer step under ZeRO-3, not once per micro-batch (#8600)

alanhuangyoo committed 1 day ago

Verified 4a5856c2

Fix pipe tests failures on CPU (#8611)

jinyouzhi committed 1 day ago

Verified ee1ce8f2

Add opt-in regional compilation around AutoEP layers (#8380)

yh0903 committed 1 day ago

Verified 6c2fae83

AutoEP: share one DeepEP buffer across MoE layers instead of one per layer (#8557)

yh0903 committed 1 day ago

Verified 26a49fd3

Implement graph operations for NPU accelerator (#8605)

VenusTZZ committed 1 day ago

Verified 478228b6

Skip fp16 ZeroPP tests on accelerators without fp16 support (#8610)

jinyouzhi committed 1 day ago

Verified 2a4ee92b

Remove stale compression test from HPU CI (#8616)

nathon-lee committed 1 day ago

Verified 1eb56d29

AutoEP: drop the redundant float32 copy in the default combine (#8621)

pengdurice committed 1 day ago

Verified bbcfc8d5

Skip narrow dtypes in the mixed-precision cast (#8416)

sfc-gh-abkulkarni committed 1 day ago

Verified 88d2f816

Wait for all async all-gathers in coalesced ZeRO-3 parameter gather (#8539)

delock committed 1 day ago

Verified e71b8f33

Remove the older TiledLoss and UlyssesSPFwdLossBwdWithLogits (#8587)

vineethsaivs committed 1 day ago

Verified d7a2ceff

Normalize Gram Newton-Schulz inputs before casting to fp16 (#8533)

vineethsaivs committed 2 days ago

Verified 1663b5b5

Overlap AutoEP split planning with token packing (#8359)

yh0903 committed 2 days ago

Verified 0d4aa274

[AutoTP] Add optional Liger vocabulary-parallel cross entropy (#8540)

0z5a committed 2 days ago

Verified 4bd251b6

Add opt-in AutoEP Python GC policy (#8451)

yh0903 committed 2 days ago

Verified 43c53e9b

Fix silent gradient-bucket drop on CPU in ZeRO-1/2 non-contiguous reduction (#8382)

delock committed 2 days ago

Verified 20a60f22

Fix memory leak and NULL-deref in shm_initialize (#8565)

tritsystem committed 2 days ago

Verified 1f5f9888

Release AutoEP router-logit caches on checkpoint replay exits (#8581)

yh0903 committed 2 days ago

Verified b16a3fab

[AutoTP] Support eager row-parallel output head training (#8538)

0z5a committed 3 days ago

Verified 468a8ad1

[muon] Per-head Muon for linear-attention layers: read head geometry from the owning module (#8436)

alanhuangyoo committed 3 days ago

Verified 141ee9f8

Fix HybridEngine OPT generation with modern cache interfaces (#8523)

0z5a committed 3 days ago

Verified 6093db0e

Describe the GPTBigCode and Yuan shared-QK layouts (#8575)

Achyuthan-S committed 3 days ago

Verified b2370ad9

Use int64 program ids in the SwiGLU and AutoEP fused restore Triton kernels (#8591)

pengdurice committed 4 days ago

Verified bb5c23e0

Remove elastic training (#8563)

pengdurice committed 4 days ago

Verified 215f5aa9

use sensible defaults

sfc-gh-jrasley committed 4 days ago

f5310bde

implement register_host_memory and unregister_host_memory for supa (#8537)

frozenleaves committed 5 days ago

Verified 4e394422

Register native pinned host memory with the CANN runtime for NPU (#8532)

VenusTZZ committed 5 days ago

Verified 0ca14e83

Support Muon in BF16_Optimizer with FP32 gradient accumulation (#8526)

0z5a committed 5 days ago

Verified 5880d38c

Older