[c10d] Fix test_collective_hang flakiness (#60662)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60662
Fixes this flaky test. Basically, sometimes a rank can exit the test
early before rank 0 calls into allreduce. In this case Gloo will throw
connection reset error on all other ranks.
ghstack-source-id: 132363151
Test Plan: CI
Reviewed By: zhaojuanmao
Differential Revision: D29364806
fbshipit-source-id: ce0c292a2166edad57ea0dbb76df12cfd560a10d
