Test pipeline parallelism works with DDP. (#48470)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/48470
Adding a unit test to test this works as expected. Although, this
doesn't work with other checkpointing modes of the pipe and checkpoint=never
needs to be set for this to work.
ghstack-source-id: 118820806
Test Plan: waitforbuildbot
Reviewed By: mrshenli
Differential Revision: D25182668
fbshipit-source-id: 85e69e338bf388c132a303ad93e29ec2cc4a0ed8
