Add FSDP tests to verify forward overlap and memory usage (#67117)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67117
Add FSDP tests to verify forward overlap and memory usage
ghstack-source-id: 141783871
Test Plan: unit tests
Reviewed By: mrshenli
Differential Revision: D31845629
fbshipit-source-id: b8b747e036925a9bb9164f0a5546000eece8442a
