pytorch
8e32e62f - [TP] Validate TP mesh dim for 2D composition (#111001)

Commit
1 year ago
[TP] Validate TP mesh dim for 2D composition (#111001) Currently, we only support intranode TP when compositin TP with other parallelism. This PR adds additional check to validate the TP mesh dim in TP initialization when parent mesh exists. cc. @fegin, @fduwjj Pull Request resolved: https://github.com/pytorch/pytorch/pull/111001 Approved by: https://github.com/fduwjj
Author
Committer
Parents
Loading