microsoft/DeepSpeed

Pull Requests Commits

Extend unmanaged gradient accumulation to ZeRO stage 3

sfc-gh-truwase committed 1 hour ago

80a83d8d

Clarify unmanaged offload rejection as optimizer state offload

sfc-gh-truwase committed 2 hours ago

94b67229

Merge branch 'master' into sfc-gh-truwase/gas_mgmt_zero2

stas00 committed 1 day ago

Verified a83b0aee

Reduce redundant work in AutoEP token routing (#8209)

hwchen2017 committed 1 day ago

Verified 58aab420

Fix MLU_Accelerator conformance to the DeepSpeedAccelerator ABC (#8208)

ebarkhordar committed 1 day ago

Verified 437226bf

Consolidate LR scheduler tensor-LR tests into a contract suite (#8206)

delock committed 1 day ago

Verified 6498adc5

Fix DeepCompile ZeRO-3 gathered parameter ownership (#8157)

tohtana committed 1 day ago

Verified d8dd9e43

Wiring up offload_opt_states (#8186)

pengdurice committed 1 day ago

Verified 76928ff9

Merge branch 'master' into sfc-gh-truwase/gas_mgmt_zero2

sfc-gh-truwase committed 2 days ago

Verified 9fbb207e

Add Triton grouped-GEMM for MoE experts on Ampere/Ada (#8180)

hwchen2017 committed 2 days ago

Verified df84f6d8

Preserve tensor learning rates in OneCycle (#8205)

tohtana committed 2 days ago

Verified f6a386a3

Remove redundant DCO workflow (#8188)

tohtana committed 2 days ago

Verified d2bd087c

Add keyword argument support to activation checkpointing (#8182)

zupengwang committed 2 days ago

Verified ebf1531e

MoE dispatch: fix silent Tutel + TP corruption, speed up native and Tutel paths (#8195)

jinyouzhi committed 2 days ago

Verified 619e3199

Add managed_gradient_accumulation for ZeRO stage 0/1 (#8184)

sfc-gh-truwase committed 2 days ago

Verified 7a31fe66

[AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP (#8168)

delock committed 2 days ago

Verified eec237ee

Preserve tensor learning rates across scheduler updates (#8202)

n33levo committed 3 days ago

Verified 3b641487

Clamp capacity to num_tokens in MoE gating drop branches (complete #5353) (#8155)

ebarkhordar committed 3 days ago

Verified c43cf21c

Reject coalesce_grad_reduction in unmanaged mode

sfc-gh-truwase committed 3 days ago

d480edba

Support ZeRO overlap_comm in unmanaged mode for stage 2

sfc-gh-truwase committed 3 days ago

c317cbe8

Extend unmanaged gradient accumulation to ZeRO stage 2

sfc-gh-truwase committed 3 days ago

252e5421

Merge branch 'master' into sfc-gh-truwase/gas_mgmt_zero01

sfc-gh-truwase committed 4 days ago

Verified 77fbb461

Skip CUDA transformer forward tests when TransformerBuilder is incompatible (#8200)

sfc-gh-truwase committed 4 days ago

Verified 7e07be2a

Track global_samples by actual micro-batch count in unmanaged mode

sfc-gh-truwase committed 4 days ago

43070642

Enable ZeRO-3 linear wrapper for existing models (#8189)

tohtana committed 4 days ago

Verified 617061d6

Reject ZeRO overlap_comm in unmanaged gradient accumulation

sfc-gh-truwase committed 4 days ago

6b3d3564

Release GIL in blocking AIO bindings (#8196)

tohtana committed 5 days ago

Verified e2aae1b0

Warn when zero.Init silently falls back to a single rank (#8084) (#8089)

akshansh47 committed 5 days ago

Verified a375b7f1

Derive AutoEP rank splits from the per-expert count exchange (#8190)

yh0903 committed 5 days ago

Verified 35d1c7d3

Tersify comments in unmanaged gradient accumulation tests

sfc-gh-truwase committed 6 days ago

0191153d

Older