TP dtensor #47194
3outeille
wants to merge
11 commits into
split/a-pr-5-fsdp-plans
from
split/a-pr-6-tp-training-inference
draft tp
9ff670ed
only cohere2_moe supports dtensor TP
9bcf7b08
revert for 3 models
27cd35a8
same as before
415decc3
cleaning
908c276f
better tp_size handle
8da0fea0
better dispatch between tp dtensor and old plain tp
56b7291c
use the latest tp_plan
b32c537a
linting
7290d575
linting
778b00c6
force even vocab_size
b6494bc1
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub