transformers
8e40ad7b - Add tensor parallel plan for Nemotron (#48780)

Commit
28 days ago
Add tensor parallel plan for Nemotron (#48780) Adds base_model_tp_plan to NemotronConfig (colwise q/k/v + up_proj, rowwise o_proj + down_proj) and a colwise_gather_output plan for lm_head, matching the Llama convention. Verified on a 2-rank CPU/gloo mesh: sharded forward matches the single-process forward (max abs diff 4e-7), 13/24 params sharded. test_tp_plan_matches_params passes. Related: #34789 Co-authored-by: Ferdinand Mom <47445085+3outeille@users.noreply.github.com>
Parents
Loading