DeepSpeed
[AutoEP] Ensure consistent order of partition groups in all-gather when ZeRO-3
#8876

Merged

[AutoEP] Ensure consistent order of partition groups in all-gather when ZeRO-3 #8876

delock merged 4 commits into deepspeedai:master from jinyouzhi:zero3-gather-group-order

ZeRO-3: gather partition groups in the same order on every rank

41168b17

jinyouzhi marked this pull request as ready for review 9 days ago

jinyouzhi requested a review from

tjruwase 9 days ago

jinyouzhi requested a review from

tohtana 9 days ago

jinyouzhi requested a review from

loadams 9 days ago

Reproduce the gather-order deadlock with two ranks

6c819a97

jinyouzhi closed this 9 days ago

jinyouzhi reopened this 9 days ago

pengdurice added will-review

Merge branch 'master' into zero3-gather-group-order

8aa3d722

pengdurice approved these changes on 2026-10-25

pengdurice enabled auto-merge 5 days ago

Merge branch 'master' into zero3-gather-group-order

4f8df9c6

pengdurice enabled auto-merge 3 days ago

delock merged 12ffb990 into master 2 days ago

Reviewers

pengdurice

tjruwase

tohtana

loadams

Assignees

No one assigned

Labels

will-review

Milestone

No milestone