Revert "Alternate sharding (#119078)"
This reverts commit 861acda20577739d52dd0bcf09e162192f25020f.
Reverted https://github.com/pytorch/pytorch/pull/119078 on behalf of https://github.com/clee2000 due to failing https://hud.pytorch.org/pytorch/pytorch/commit/861acda20577739d52dd0bcf09e162192f25020f ([comment](https://github.com/pytorch/pytorch/pull/119078#issuecomment-1946583857))
