microsoft/DeepSpeed

Pull Requests Commits

[tiled mlp] reshape instead of view (#8348)

stas00 committed 6 hours ago

Verified 87d9ecd8

Fallback for unsupported Hybrid Engine policies (#8265)

LiRunGuo committed 20 hours ago

Verified a36f78e7

Count each module object once when aggregating flops profiler totals (#8320)

ebarkhordar committed 23 hours ago

Verified f4ee787d

Fix the seq-first Ulysses all2all output layout (#8317)

vineethsaivs committed 1 day ago

Verified cd5206af

Raise the per-tensor norms to norm_type when combining them (#8313)

vineethsaivs committed 1 day ago

Verified 37cf8f24

Add configurable dtype for ZeRO checkpoint export (#8318)

gaoxiaomo committed 2 days ago

Verified 92843ad7

Honor adam_w_mode in the CPU multi_tensor_adam binding (#8307)

PKUWZP committed 2 days ago

Verified c7cc64a9

Route ZeRO/SuperOffload pin sites through accelerator pin_memory (#8256)

sfc-gh-truwase committed 2 days ago

Verified eebe24a6

[AutoTP] Fix training lm_head routing (#8302)

gaoxiaomo committed 2 days ago

Verified 1b956e62

Fix ZeRO-3 synchronization during OPSD rollout (#8264)

LiRunGuo committed 2 days ago

Verified e40c681b

Isolate DeepCompile list-schedule test ops (#8319)

tohtana committed 2 days ago

Verified 28ed612e

Fix shared loss gradient accumulation (#8245)

tohtana committed 2 days ago

Verified 4ffb4e5c

stop allocating per-element temporaries in the overflow check (#8325)

stas00 committed 3 days ago

Verified 32e301ff

feat(opsd): share prompt prefill across rollout samples (#8296)

nathon-lee committed 3 days ago

Verified a9505fd6

Enable optimized Adam backend for MuonWithAuxAdam optimizer (#8278)

jinyouzhi committed 3 days ago

Verified d47e4f4d

[Apple Silicon Support Phase 1] Add Metal FusedAdam kernel and CPU Adam build for Apple Silicon (#8300)

PKUWZP committed 3 days ago

Verified 715965e0

Fix AutoTP metadata updates for unsharded modules (#8299)

zupengwang committed 4 days ago

Verified 7154a00f

Avoid redundant copies in MPS P2P staging (#8314)

FU-max-boop committed 4 days ago

Verified eba5d273

Reject invalid ZenFlow ratio and update interval boundaries (#8274)

tandede committed 4 days ago

Verified 6e3bd087

Import print_dist in auto_tp (#8311)

alanhuangyoo committed 4 days ago

Verified b753aec8

Fix device mismatch in test_gate_up_partition_covers_the_whole_weight (#8308)

alanhuangyoo committed 5 days ago

Verified 84fd92aa

Route isend/irecv to nonblocking backend methods and stage them as async on MPS (#8303)

PKUWZP committed 5 days ago

Verified 9311fd55

perf(rollout): add HybridEngine rollout profiling (#8295)

nathon-lee committed 5 days ago

Verified da3ca683

Skip zero-sized parameters in HP fragment mapping (#8298)

fwerkor committed 6 days ago

Verified 573ce488

Drop the documented grad_hooks ZeRO option, which does not exist (#8242)

vineethsaivs committed 6 days ago

Verified f76ab880

Register native pinned host memory with CUDA for GPU DMA (#8283)

sfc-gh-truwase committed 6 days ago

Verified cb260806

Fix zero reduce bucket size validation (#8266)

fwerkor committed 6 days ago

Verified e70b9035

[AutoTP] Convert HF embedding_rowwise tp_plan entries to SKIP specs (#8294)

ebarkhordar committed 6 days ago

Verified 20ea4543

fix(engine): resolve inference workspace attribute lookup (#8288)

nathon-lee committed 6 days ago

Verified fb57b819

Enable DeepSpeed support on Apple Silicon (MPS) with ZeRO Stage 1-3 (#8293)

PKUWZP committed 6 days ago

Verified 64fcec6b

Older