PR #8670 AutoEP + AutoTP folding under ZeRO-3

AutoEP + AutoTP folding under ZeRO-3 #8670

sfc-gh-truwase merged 6 commits into deepspeedai:master from pengdurice:peng-add-autoep-autotp-zero3-v1

Support AutoEP + AutoTP folding under ZeRO-3; average the folded MoE …

6b0a4d68

pengdurice force pushed from 66143d4f to 6b0a4d68 14 days ago

Update two AutoEP unit tests for AutoTP under ZeRO-3

2e070640

Fix ZeRO-3 folded checkpoint save and keep tied weights tied under ze…

667cbd44

pengdurice marked this pull request as ready for review 12 days ago

pengdurice requested a review from

tjruwase 12 days ago

pengdurice requested a review from

tohtana 12 days ago

pengdurice requested a review from

hwchen2017 12 days ago

pengdurice requested a review from

loadams 12 days ago

tohtana requested changes on 2026-10-04

PKUWZP requested a review from

PKUWZP 10 days ago

fixed comments

19e42148

tohtana approved these changes on 2026-10-07

Merge branch 'master' into peng-add-autoep-autotp-zero3-v1

246381be

pengdurice enabled auto-merge 6 days ago

Merge branch 'master' into peng-add-autoep-autotp-zero3-v1

77311fce

sfc-gh-truwase merged 7d77213d into master 4 days ago

Reviewers

tohtana

tjruwase

hwchen2017

loadams

PKUWZP

Assignees

No one assigned

Labels

None yet

Milestone

No milestone