DeepSpeed
[AutoTP] Implement VocabParallelEmbedding and enhance tied model support
#8582

Merged

[AutoTP] Implement VocabParallelEmbedding and enhance tied model support #8582

tohtana merged 9 commits into deepspeedai:master from jinyouzhi:vocab_parallel_tied_embedding

[AutoTP] Add VocabParallelEmbedding and unify tied/untied lm_head pat…

3d02aaf1

[AutoTP] Honor HF tied embedding_rowwise plans

b793a694

jinyouzhi marked this pull request as ready for review 24 days ago

Merge branch 'master' into vocab_parallel_tied_embedding

518de364

jinyouzhi requested a review from

tjruwase 24 days ago

jinyouzhi requested a review from

tohtana 24 days ago

jinyouzhi requested a review from

hwchen2017 24 days ago

jinyouzhi requested a review from

loadams 24 days ago

jinyouzhi marked this pull request as draft 24 days ago

jinyouzhi marked this pull request as ready for review 24 days ago

Merge branch 'master' into vocab_parallel_tied_embedding

49035c3c

delock requested a review from

delock 23 days ago

Merge remote-tracking branch 'origin/master' into vocab_parallel_tied…

40291e61

Fix tied vocabulary-parallel embedding regressions

778f4197

Fix tied embedding aliases and automatic vocabulary sharding

990ac2ce

delock approved these changes on 2026-09-24

Initialize AutoTP normally in embedding metadata test

8ad41d5e

Merge branch 'master' into vocab_parallel_tied_embedding

de79a6a7

tohtana enabled auto-merge 16 days ago

tohtana merged 5f9f48f0 into master 16 days ago

Reviewers

delock

tjruwase

tohtana

hwchen2017

loadams

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed [AutoTP] Implement VocabParallelEmbedding and enhance tied model support #8582 Merged

[AutoTP] Implement VocabParallelEmbedding and enhance tied model support #8582

DeepSpeed
[AutoTP] Implement VocabParallelEmbedding and enhance tied model support
#8582

Merged