transformers
Expert-parallel implementations for `ep_dispatch_experts`: all-reduce at inference, all-to-all in training
#49429
Open
Open
Expert-parallel implementations for `ep_dispatch_experts`: all-reduce at inference, all-to-all in training #49429
IlyasMoutawwakil
wants to merge
43 commits into
main
from
ep-masked-when-shared-batch
build and expand once the tp/ep_plan
6440b726
unify
79b5fdb0
clean fsdp
bb63397e
reorganize to avoid breaking changes
f9158f8d
better distinction
64eb317a
Merge branch 'main' into unify-resolve-plan
61a4954d
revert
5fbf21c9
Merge branch 'unify-resolve-plan' of github.com:huggingface/transform…
ff70527c
add tp plan to deepseek v4 (but attention stays replicated)
3f1e4ab9
Revert "add tp plan to deepseek v4 (but attention stays replicated)"
ff3629c6
IlyasMoutawwakil
changed the title 
IlyasMoutawwakil
changed the title
IlyasMoutawwakil
commented
on 2026-10-14
3outeille
commented
on 2026-10-15
3outeille
commented
on 2026-10-15
IlyasMoutawwakil
commented
on 2026-10-16
IlyasMoutawwakil
changed the title 
Merge branch 'main' into unify-resolve-plan
b5da03f9
Fix the merge of main: restore moved imports and the empty tp_plan ov…
d527c29f
Merge remote-tracking branch 'origin/main' into unify-resolve-plan
523101d8
cleaner mixin
96332a1e
cleaner
615fe835
deprecation cycle
868647f2
cleaner
a121baea
revert tp plan
e1d83ef5
Merge remote-tracking branch 'origin/main' into unify-resolve-plan
2dbcb42c
Pass the expanded plan in the bare-module colwise_gather_output test
9ef53bbc
retrigger to verify fetcher
a1a055ba
the branch is now pointing to main and not to unify-resolve-plan
b5eb8827
Refactor expert parallelism registration and improve documentation
0de36597
comment
bc4b6e0a
some models missed the tests EP mixin
c47e5dce
fix copyright
f88b6167
Merge branch 'cleanup-ep' into unify-resolve-plan
7d3345d8
move replace_layer_number_by_wildcard and _get_parameter_plan to para…
da4eed4d
Merge branch 'main' into cleanup-ep
d7591e12
Merge branch 'cleanup-ep' into unify-resolve-plan
a75a5cb2
Merge branch 'main' into unify-resolve-plan
6247bd14
Merge remote-tracking branch 'origin/unify-resolve-plan' into unify-r…
f5824079
Expert-parallel implementations for ep_dispatch_experts: all-reduce a…
6ac4923f
Run the backward-only all-reduces only when a gradient can flow
44cd6786
Drop the config-side EP implementation resolver
0ce3b90c
Merge main: the plans are resolved once in the model (#49527)
6f72a0ea
Say what each EP implementation does in short sentences
1710f6c5
EP implementations: one forward instead of dispatch + combine
a3365657
EP implementations are callables; inline the backward all-reduce check
704f9114
EP: register implementations in a dict on EpDispatchExpertsParallel
d8763955
EP: inline the shared-batch check into EPImplementation
6eae29f4
EP: always take the TP mesh in EP implementations
389f9423
EP: helper for the gradient-only all-reduce, kwarg ep mesh, one toy e…
4a1cd28f
3outeille
approved these changes
on 2026-11-06
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub