transformers
[`distributed`] Default MoE `ep_plans` to token dispatch
#49160
Merged
Merged
[`distributed`] Default MoE `ep_plans` to token dispatch #49160
3outeille
merged
119 commits into
main
from
ep-dispatch-default
3outeille
changed the title
3outeille
commented
on 2026-09-28
ArthurZucker
approved these changes
on 2026-09-30
Add dense and expert device mesh views with a MeshManager
eecb6a8a
cleaning
29f51267
clean
40745266
update
578af0e7
clean doc
083e2532
Use ep_size in the Kimi K2.5 expert parallel example
2e682969
Enable expert parallelism in the Mega MoE example
29eb5a36
warn once
d6090dfd
Add dense and expert device mesh views with a MeshManager
76ff6f0e
cleaning
9a1b525f
update
ce8657f9
clean doc
ea3aacba
Update src/transformers/distributed/mixin.py
d498e802
style: ruff import block formatting in distributed config
ce3f5760
add more comments
f8415e73
Decouple tp_plan and ep_plan for expert parallelism.
a115da8e
Match expert paths with a regex and keep only expert rules under toke…
271de167
revert change on warning
cc5200f9
Apply suggestion from @ArthurZucker
17f7b659
cleaning
ef837266
Fix tied embeddings for models with an EP-only base plan
e7bfac9b
[distributed] Add expert-parallel token dispatch, default for Qwen3 MoE
4cc84e42
clean
b756980d
fix
4d5ef4cf
make check repo
556f4b9e
Drop the load-time ep_size == tp_size guard that rejects token dispatch
59578e01
remove all to all warning by using functional version
16fbcca9
use ep plan instead of group_gemm is_expert
60fd0822
use dtensor instead of manuall all_reduce
fc42fb9c
better comment
1478d288
[`distributed`]: Fix DeepSeek-V4 tied embeddings with combined TP and…
51472f93
linting
ea814ebb
Add dense and expert device mesh views with a MeshManager
925415d4
cleaning
0c227f1d
clean
a5958d4e
update
65719164
clean doc
90fbd095
Use ep_size in the Kimi K2.5 expert parallel example
e4561f09
Enable expert parallelism in the Mega MoE example
7f4951f9
warn once
e65e3eaf
Add dense and expert device mesh views with a MeshManager
03af9e8b
cleaning
69a145b2
update
caa76409
clean doc
a3de4dc2
Update src/transformers/distributed/mixin.py
84cb4bee
style: ruff import block formatting in distributed config
b5521952
add more comments
d1cf2079
Decouple tp_plan and ep_plan for expert parallelism.
4b655fbf
Match expert paths with a regex and keep only expert rules under toke…
a4f11e3a
revert change on warning
a5cd182b
Apply suggestion from @ArthurZucker
3d45692b
cleaning
04752eb4
Fix tied embeddings for models with an EP-only base plan
c41435b0
[distributed] Add expert-parallel token dispatch, default for Qwen3 MoE
8dc8333b
clean
865d5d96
fix
cac1b2b0
make check repo
c59446d9
Drop the load-time ep_size == tp_size guard that rejects token dispatch
abf336cc
remove all to all warning by using functional version
6244a1c3
use ep plan instead of group_gemm is_expert
c72bbf72
use dtensor instead of manuall all_reduce
c31a1b7a
better comment
00ee316a
linting
ec8278e1
Train models sharded at load time with the Trainer, token dispatch in…
39c7e413
Use ep-dispatch version of expert_parallelism.md
dfec8d1e
Sync replicated trainable parameters when training a model sharded at…
628da80a
Fix utf-8 encoding in expert parallel trainer tests
e72d719b
[distributed] Default MoE expert-parallel plans to token dispatch
e9bcdc57
remove
4497d747
fix ruff
9b41975a
use attribute instead
69ecbf13
Fix EP plans for youtu (dense, drop inherited plan) and zaya (token d…
0cb580c6
Train models sharded at load time with the Trainer, token dispatch in…
39c7e413
Use ep-dispatch version of expert_parallelism.md
dfec8d1e
Sync replicated trainable parameters when training a model sharded at…
628da80a
Fix utf-8 encoding in expert parallel trainer tests
e72d719b
[distributed] Default MoE expert-parallel plans to token dispatch
e9bcdc57
remove
4497d747
fix ruff
9b41975a
use attribute instead
69ecbf13
fix
bcbb851d
Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…
d51aca88
Merge branch 'ep-plans' of github.com:huggingface/transformers into e…
448315ed
revert
877afeea
Merge branch 'ep-dispatch' of github.com:huggingface/transformers int…
bec5a12b
Merge branch 'main' into ep-mesh-infrastructure
0ec824f1
Merge branch 'ep-plans' into ep-dispatch
b20486ce
Merge branch 'ep-mesh-infrastructure' into ep-plans
c85c8bca
revert
0d8d4f55
Merge branch 'ep-dispatch' into ep-trainer
a7a37bed
Merge branch 'ep-plans' into ep-dispatch
c9d64290
typo
cedde47c
Merge branch 'main' into ep-mesh-infrastructure
b62bbbcd
clearer
6a23a306
Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…
daac87a5
Merge branch 'main' into ep-mesh-infrastructure
6c73628e
clean
b4d81b5b
Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…
e3cc578e
Merge branch 'ep-mesh-infrastructure' into ep-plans
9bb2518a
Merge branch 'ep-plans' into ep-dispatch
38704ae8
cleaning
7b43249c
cleaning
e0f70125
Merge branch 'ep-plans' of github.com:huggingface/transformers into e…
926ee709
Merge branch 'main' into ep-mesh-infrastructure
746b2c53
Merge branch 'ep-mesh-infrastructure' into ep-plans
0eac0f5c
Merge branch 'ep-plans' into ep-dispatch
030900b0
dont torch cat for empty tokens
af793976
Merge branch 'ep-dispatch' of github.com:huggingface/transformers int…
43a8c2d6
Merge branch 'ep-dispatch' into ep-trainer
37089c18
Merge branch 'ep-trainer' into ep-dispatch-default
40c20a4e
check code quality
865aa963
ArthurZucker
approved these changes
on 2026-10-06
Update src/transformers/distributed/utils.py
7b57dd23
Merge branch 'main' into ep-mesh-infrastructure
ec77a598
Update src/transformers/distributed/configuration_utils.py
7af3cbc9
linting
f37bc0f4
Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…
11f0a3d5
fix comment
a4d06939
Merge branch 'ep-mesh-infrastructure' into ep-plans
6a504c18
Merge branch 'main' into ep-mesh-infrastructure
812c060c
Merge branch 'ep-plans' into ep-dispatch
13d2fc74
Merge branch 'ep-mesh-infrastructure' into ep-plans
430bb2e6
Merge branch 'ep-dispatch' into ep-trainer
e431d5cb
Merge branch 'ep-trainer' into ep-dispatch-default
50d2f6b8
Merge branch 'ep-plans' into ep-dispatch
42d06077
Merge branch 'ep-dispatch' into ep-trainer
b0fd271d
Merge branch 'ep-trainer' into ep-dispatch-default
31868196
Base automatically changed from
ep-dispatch
to
main
12 days ago
Merge branch 'main' into ep-dispatch-default
114cc5c0
disabled auto-merge 12 days ago
Manually disabled by user
Manually disabled by user
Login to write a write a comment.
Login via GitHub