ShardedTensor framework for ChunkedShardingSpec (#58517)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58517
Building upon the sharding specifications, this PR introduces the
intial skeleton of ShardedTensor and allows building a ShardedTensor by
specifying ChunkedShardingSpec.
In follow up PRs, I'll add further support for GenericShardingSpec.
ghstack-source-id: 129917841
Test Plan:
1) unit tests.
2) waitforbuildbot
Reviewed By: SciPioneer
Differential Revision: D28526012
fbshipit-source-id: 8e62847b58957d284e40f57a644302c171289138
