pytorch

be354d81 - [shard] Add basic math ops to ShardedTensor and add ReplicatedTensor inter-op

[shard] Add basic math ops to ShardedTensor and add ReplicatedTensor inter-op Pull Request resolved: https://github.com/pytorch/pytorch/pull/73703 This PR add basic math ops to ShardedTensor (+-*/), and add ReplicatedTensor inter-op ShardedTensor to those math ops. This enables ShardedTensor (op) ReplicatedTensor to avoid communication in certain cases. Differential Revision: [D34560867](https://our.internmc.facebook.com/intern/diff/D34560867/) **NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D34560867/)! Approved by: https://github.com/pritamdamania87