PR #48857 [`distributed`] Add dense and expert device mesh

[`distributed`] Add dense and expert device mesh #48857

3outeille merged 48 commits into main from ep-mesh-infrastructure

3outeille force pushed from eddc0d95 to 0dcfe943 28 days ago

3outeille commented on 2026-09-16

3outeille force pushed from 60cba1b1 to 6ac0a63d 28 days ago

3outeille changed the title ~~Add dense and expert device mesh views with a MeshManager (EP part 1)~~ Add dense and expert device mesh 28 days ago

3outeille changed the title ~~Add dense and expert device mesh~~ [`distributed`] Add dense and expert device mesh 28 days ago

3outeille marked this pull request as ready for review 28 days ago

3outeille requested a review from

ArthurZucker 28 days ago

3outeille force pushed from bb6cbe94 to 6d99b226 28 days ago

ArthurZucker commented on 2026-09-17

3outeille force pushed from 6d99b226 to 624f591a 26 days ago

3outeille force pushed from a37c988e to a0032b08 26 days ago

3outeille force pushed from a0032b08 to 402f7659 22 days ago

3outeille requested a review from

ArthurZucker 19 days ago

3outeille force pushed from 402f7659 to b92b1eb0 16 days ago

3outeille force pushed from b92b1eb0 to 2de77321 16 days ago

3outeille force pushed from 2de77321 to e78e14d0 16 days ago

3outeille force pushed from e78e14d0 to bde3f81f 16 days ago

3outeille force pushed from bde3f81f to 401226a7 15 days ago

3outeille force pushed from 401226a7 to 52c2e588 15 days ago

ArthurZucker commented on 2026-09-30

3outeille force pushed from 52c2e588 to 7acf7fa5 14 days ago

3outeille force pushed from e7fa1f99 to 1eef5165 13 days ago

3outeille force pushed from 1eef5165 to b309a715 13 days ago

Add dense and expert device mesh views with a MeshManager

eecb6a8a

cleaning

29f51267

clean

40745266

update

578af0e7

clean doc

083e2532

Use ep_size in the Kimi K2.5 expert parallel example

2e682969

Enable expert parallelism in the Mega MoE example

29eb5a36

warn once

d6090dfd

Add dense and expert device mesh views with a MeshManager

76ff6f0e

cleaning

9a1b525f

update

ce8657f9

clean doc

ea3aacba

Update src/transformers/distributed/mixin.py

d498e802

style: ruff import block formatting in distributed config

ce3f5760

add more comments

f8415e73

3outeille force pushed from b309a715 to f8415e73 13 days ago

Add dense and expert device mesh views with a MeshManager

925415d4

cleaning

0c227f1d

clean

a5958d4e

update

65719164

clean doc

90fbd095

Use ep_size in the Kimi K2.5 expert parallel example

e4561f09

Enable expert parallelism in the Mega MoE example

7f4951f9

warn once

e65e3eaf

Add dense and expert device mesh views with a MeshManager

03af9e8b

cleaning

69a145b2

update

caa76409

clean doc

a3de4dc2

Update src/transformers/distributed/mixin.py

84cb4bee

style: ruff import block formatting in distributed config

b5521952

add more comments

d1cf2079

3outeille force pushed from f8415e73 to d1cf2079 9 days ago

fix

bcbb851d

Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…

d51aca88

Merge branch 'main' into ep-mesh-infrastructure

0ec824f1

typo

cedde47c

Merge branch 'main' into ep-mesh-infrastructure

b62bbbcd

clearer

6a23a306

Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…

daac87a5

Merge branch 'main' into ep-mesh-infrastructure

6c73628e

clean

b4d81b5b

Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…

e3cc578e

Merge branch 'main' into ep-mesh-infrastructure

746b2c53

ArthurZucker approved these changes on 2026-10-06

Update src/transformers/distributed/utils.py

7b57dd23

Merge branch 'main' into ep-mesh-infrastructure

ec77a598

Update src/transformers/distributed/configuration_utils.py

7af3cbc9

linting

f37bc0f4

Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/trans…

11f0a3d5

fix comment

a4d06939

3outeille enabled auto-merge 8 days ago

Merge branch 'main' into ep-mesh-infrastructure

812c060c

3outeille merged f4ae7620 into main 8 days ago

3outeille deleted the ep-mesh-infrastructure branch 8 days ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers [`distributed`] Add dense and expert device mesh #48857 Merged

[`distributed`] Add dense and expert device mesh #48857

transformers
[`distributed`] Add dense and expert device mesh
#48857

Merged