Add unit test to verify DDP + RPC correctness. (#40139)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/40139
This unit test runs the same set of operations locally and then with
DDP + RPC to verify correctness.
ghstack-source-id: 106287490
Test Plan:
buck test mode/dev-nosan caffe2/test/distributed/:ddp_under_dist_autograd
I ran these to make sure I am workin on a clean git repo.
git submodule update --init --recursive
to get latest tensor pipe code, otherwise build will have error.
to record installed binaries and torch package wheels to system paths
with-proxy env BUILD_CAFFE2_OPS=0 USE_CUDA=0 USE_MKLDNN=0 USE_DISTRIBUTED=1 python setup.py install --record files.txt
remove binaries and torch package wheels from system paths.
xargs rm -rf < files.txt
build in develop mode
with-proxy env BUILD_CAFFE2_OPS=0 USE_CUDA=0 USE_MKLDNN=0 USE_DISTRIBUTED=1 python setup.py develop
pytest test/distributed/test_ddp_under_dist_autograd.py::TestDdpUnderDistAutograd -v
Differential Revision: D22084385
fbshipit-source-id: e1f57e86ceddd4c96920ed904898e1763b47e8f2
