DeepSpeed
[AutoTP] Support tied embeddings for row-parallel output-head training
#8680

Merged

[AutoTP] Support tied embeddings for row-parallel output-head training #8680

delock merged 4 commits into deepspeedai:master from jinyouzhi:fix/autotp-row-head-tie-snapshot

jinyouzhi requested a review from

hwchen2017 18 days ago

jinyouzhi requested a review from

loadams 18 days ago

jinyouzhi requested a review from

tjruwase 18 days ago

jinyouzhi requested a review from

tohtana 18 days ago

[AutoTP] Detect tied weights before replacement for row-parallel outp…

53c67928

jinyouzhi force pushed from bf0a8b5d to f96d8bb6 18 days ago

jinyouzhi changed the title ~~[AutoTP] Detect tied weights before replacement for row-parallel output-head training~~ [AutoTP] Support tied embeddings for row-parallel output-head training 18 days ago

[AutoTP] Support tied embeddings for row-parallel output-head training

f96d8bb6

delock requested a review from

delock 17 days ago

pengdurice added will-review

delock commented on 2026-09-30

[AutoTP] Cover head-first visit order for tied row-parallel output head

12ae3143

Merge branch 'master' into fix/autotp-row-head-tie-snapshot

30ddbf4f

delock approved these changes on 2026-10-04

delock enabled auto-merge 11 days ago

delock merged 3c6a5ca9 into master 6 days ago

Reviewers

delock

hwchen2017

loadams

tjruwase

tohtana

Assignees

No one assigned

Labels

will-review

Milestone

No milestone

DeepSpeed [AutoTP] Support tied embeddings for row-parallel output-head training #8680 Merged

[AutoTP] Support tied embeddings for row-parallel output-head training #8680

DeepSpeed
[AutoTP] Support tied embeddings for row-parallel output-head training
#8680

Merged