microsoft/DeepSpeed

Pull Requests Commits

Fix AutoEP ZeRO-1/2 universal conversion (#8198)

tohtana committed 9 hours ago

Verified da066407

fix(zero3): async grad offload + pinned offload buffers by default (#8207)

delock committed 16 hours ago

Verified 7dfa67ae

Fix ZeRO++ secondary shard copy for small params (#8210)

ZyEng-Art committed 21 hours ago

Verified 2d4488ef

Do not let one op builder's compatibility probe break importing deepspeed (#8216)

adityasingh2400 committed 21 hours ago

Verified 646e491c

Update actions versions for workflows. (#8228)

loadams committed 1 day ago

Verified c6e72794

Restore the custom DCO workflow (#8220)

tohtana committed 2 days ago

Verified 20328cf5

Update version.txt after 0.19.4 release (#8219)

loadams committed 2 days ago

Verified 14364cd1

Update version.txt pre-0.19.4 release (#8218)

loadams committed 2 days ago

Verified c4550314

Unmanaged gradient accumulation: ZeRO stage 3 support (#8217)

sfc-gh-truwase committed 2 days ago

Verified e75d1150

Fix ZeRO-3 all_reduce param fetch stride for padded parameters (#8158)

ebarkhordar committed 2 days ago

Verified 4e3acb1e

Unmanaged gradient accumulation: ZeRO stage 2 support (#8203)

sfc-gh-truwase committed 2 days ago

Verified 3c737a1c

Stop trim_mean sorting the caller's list in place (#8199)

vineethsaivs committed 2 days ago

Verified b8b44809

Share DeepNVMe pinned-tensor manager and route swap buffers through I/O handles (#8212)

sfc-gh-truwase committed 2 days ago

Verified e1d6b4fe

Make PipelineModule.set_checkpoint_interval actually change the interval (#8178)

vineethsaivs committed 3 days ago

Verified 87e4c8c8

Reduce redundant work in AutoEP token routing (#8209)

hwchen2017 committed 4 days ago

Verified 58aab420

Fix MLU_Accelerator conformance to the DeepSpeedAccelerator ABC (#8208)

ebarkhordar committed 4 days ago

Verified 437226bf

Consolidate LR scheduler tensor-LR tests into a contract suite (#8206)

delock committed 4 days ago

Verified 6498adc5

Fix DeepCompile ZeRO-3 gathered parameter ownership (#8157)

tohtana committed 4 days ago

Verified d8dd9e43

Wiring up offload_opt_states (#8186)

pengdurice committed 4 days ago

Verified 76928ff9

Add Triton grouped-GEMM for MoE experts on Ampere/Ada (#8180)

hwchen2017 committed 5 days ago

Verified df84f6d8

Preserve tensor learning rates in OneCycle (#8205)

tohtana committed 5 days ago

Verified f6a386a3

Remove redundant DCO workflow (#8188)

tohtana committed 5 days ago

Verified d2bd087c

Add keyword argument support to activation checkpointing (#8182)

zupengwang committed 5 days ago

Verified ebf1531e

MoE dispatch: fix silent Tutel + TP corruption, speed up native and Tutel paths (#8195)

jinyouzhi committed 5 days ago

Verified 619e3199

Add managed_gradient_accumulation for ZeRO stage 0/1 (#8184)

sfc-gh-truwase committed 5 days ago

Verified 7a31fe66

[AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP (#8168)

delock committed 5 days ago

Verified eec237ee

Preserve tensor learning rates across scheduler updates (#8202)

n33levo committed 5 days ago

Verified 3b641487

Clamp capacity to num_tokens in MoE gating drop branches (complete #5353) (#8155)

ebarkhordar committed 6 days ago

Verified c43cf21c

Skip CUDA transformer forward tests when TransformerBuilder is incompatible (#8200)

sfc-gh-truwase committed 7 days ago

Verified 7e07be2a

Enable ZeRO-3 linear wrapper for existing models (#8189)

tohtana committed 7 days ago

Verified 617061d6

Older