Add AutoEP + AutoTP parallel folding #8064
tohtana
wants to merge
18 commits into
deepspeedai:master
from
tohtana:tohtana/autoep-autotp-parallel-folding-design
Add AutoEP + AutoTP parallel folding
278c9194
Fix folded TP gradient reductions
0d44a1d2
Normalize folded TP ZeRO gradients
8b1c0428
delock
commented
on 2026-06-18
delock
commented
on 2026-06-19
Fix AutoEP folded gradient strategy
7246b146
Document AutoEP folding gradient-reduction rationale
e5c1ba23
Generalize AutoEP+AutoTP folding to cross-lane expert parallelism
13dceecf
Gate AutoEP folding routing validation
59fdca46
Apply AutoEP folding yapf formatting
7426b121
Merge upstream master into AutoEP folding branch
ef57cb43
delock
approved these changes
on 2026-07-01
PKUWZP
requested changes
on 2026-07-01
Merge upstream master into AutoEP folding branch
f2c7f731
Add folded DeepCompile rejection tests
e6239e4a
Reject folded AutoEP DeepCompile configs
3c1e91a8
Add BF16 folded HP grad correction test
3fcc33bd
Apply folded correction before BF16 HP grad accumulation
b485a75e
Add ZeRO-1 overlap folding correction tests
4906a3a6
Apply folded correction in ZeRO-1 overlap hooks
42c4148d
Handle DeviceMesh split fallback for AutoTP groups
96f3b3b7
Apply pre-commit formatting
bc28dd8a
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub