Tp param level #46290
3outeille
merged
34 commits into
distributed
from
tp_param_level
[distributed] Add param-level MoE TP/EP styles and ep_router
f20c14c1
[distributed] Add param-level apply pass to apply_tensor_parallel
bcecab89
[distributed] Add MoE TP/EP plan tests and a two-sided sharding asser…
963777a0
[distributed] Migrate MoE configs to decomposed TP/SP expert plans
6a6dda75
[distributed] Document decomposed MoE TP/EP plans
6d73f841
[distributed] Rename MoE intra-expert TP styles to moe_tp_*
ec2212bd
handle sparse and dense sp plan for qwen3_moe
654fa22a
better tests coverage for sp & ep
57b5a6e3
linting
e72ee624
uniformize TP Api to avoid confusion with torch native ops
2ff38219
inline tp
f5493685
rename
40897616
cleaning
18bccfaf
inline
5da75494
cleaning
a08c6acf
cleaning
b78f2347
linting
45925992
fix ci ep_backward
229235d8
linting
8cdcd889
remove flag expert parallel
6d0e153b
fix
df59a892
add tp plan + ep_plan
9add3298
revert doc
4bc3980b
fix install_forward
8393d5a0
linting
36a3c379
add moe identity back
24d042e0
no need aymore
69e118d8
update tp_plan for ernie4_5_vl_moe
b5d0bbab
sp + ep training / tp + ep inference (#46292)
d736cb69
Merge branch 'distributed' into tp_param_level
45935922
Merge branch 'distributed' into tp_param_level
d47f10a1
fix merge conflicts
79f20e90
linting
82d06ba4
Merge branch 'distributed' into tp_param_level
f005633b
Login to write a write a comment.
Login via GitHub