transformers
aea39f09 - tp loading: use dist.scatter for sharded redistribute

Commit
160 days ago
tp loading: use dist.scatter for sharded redistribute Each rank owns ~num_mappings/world_size mappings via partition_mappings_ across_ranks; for its owned mappings it now pre-slices the materialized full tensor into one shard per rank and calls dist.scatter, so every other rank receives only its own shard. - cluster bandwidth drops from (N-1)*sizeof(full) to (N-1)/N*sizeof(full) - no more view-keeps-full-tensor-alive bug: each scatter recv buffer is a fresh allocation sized to the local shard, no clone needed on rx - replicated params (tp_layer is None) still broadcast — they're small - world_size==1 path stays fully local, no comms - ragged shards (non-divisible sharded dim) raise rather than silently miscompute; every model we ship divides cleanly under tp_plan=auto
Author
Parents
Loading