[WIP][Grad Compression] Unittest to verify allreduce_hook parity (#50851)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50851
Improves upon the previous unittest to ensure allreduce_hook results in the same gradients as vanilla allreduce in DDP.
ghstack-source-id: 120229103
Test Plan:
buck build mode/dev-nosan //caffe2/test/distributed:distributed_nccl_fork --keep-going
BACKEND=nccl WORLD_SIZE=2 ~/fbcode/buck-out/dev/gen/caffe2/test/distributed/distributed_nccl_fork#binary.par -r test_ddp_hook_parity
Reviewed By: SciPioneer
Differential Revision: D25963654
fbshipit-source-id: d55eee0aee9cf1da52aa0c4ba1066718aa8fd9a4
