DeepSpeed
Add AutoEP + AutoTP parallel folding
#8064
Open
Overview
Commits
9
Changes
View On GitHub

Add AutoEP + AutoTP parallel folding #8064

tohtana wants to merge 9 commits into deepspeedai:master from tohtana:tohtana/autoep-autotp-parallel-folding-design
tohtana
tohtana Add AutoEP + AutoTP parallel folding
278c9194
tohtana tohtana requested a review from tjruwase tjruwase 17 days ago
tohtana tohtana requested a review from hwchen2017 hwchen2017 17 days ago
tohtana tohtana requested a review from loadams loadams 17 days ago
tohtana tohtana requested a review from GuanhuaWang GuanhuaWang 17 days ago
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-06-13
tohtana Fix folded TP gradient reductions
0d44a1d2
tohtana Normalize folded TP ZeRO gradients
8b1c0428
PKUWZP PKUWZP requested a review from PKUWZP PKUWZP 16 days ago
delock
delock commented on 2026-06-18
delock
delock commented on 2026-06-19
tohtana Fix AutoEP folded gradient strategy
7246b146
tohtana Document AutoEP folding gradient-reduction rationale
e5c1ba23
tohtana Generalize AutoEP+AutoTP folding to cross-lane expert parallelism
13dceecf
PKUWZP
tohtana Gate AutoEP folding routing validation
59fdca46
tohtana Apply AutoEP folding yapf formatting
7426b121
tohtana Merge upstream master into AutoEP folding branch
ef57cb43

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum