Add ShardedTensorMetadata for ShardedTensor. (#61683)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61683
This PR adds a consolidated metadata field (ShardedTensorMetadata)
which has all the necessary global metadata for a ShardedTensor.
ghstack-source-id: 133847517
Test Plan:
1) unit tests
2) waitforbuildbot
Reviewed By: wanchaol
Differential Revision: D29703719
fbshipit-source-id: 567279e46c787a88ef3310e4dce6fd2ad7631c62
