transformers
10f60888 - Merge branch 'main' into load-convert-before-shard

Commit

9 days ago

Merge branch 'main' into load-convert-before-shard

References

#48204 - [Umbrella] FSDP2 + expert-parallel token dispatch, with sharded-loading prefetch

#48519 - Loading: convert transposed and RoPE-permuted weights before taking the DTensor shard

Author

qgallouedec

qgallouedec

Parents

Loading