Convert Sampler back to lazily construction (#63646)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63646
Fixes #63609
Test Plan: Imported from OSS
Reviewed By: NivekT
Differential Revision: D30451774
Pulled By: ejguan
fbshipit-source-id: 550d77494326446d1a42b5da0559e0d384c47413
