fix testSend and testRecv in ProcessGroupGlooTest (#32134)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/32134
These tests weren't written in the most correct way and were often
flaky. It was tricky to identify these tests as flaky until we moved this file
to use gtest.
The gist of the issue is that the test previously would not coordinate sends
and recvs properly. For example, we created a single thread to test an
abortRecv and a successful recv. A separate sender thread was used to send 2
messages. What could go wrong here is that the first send could successfully
complete, resulting in the receiving end processing the message before it gets
the abort signal. In this case we would have an error in the test.
ghstack-source-id: 96806879
Differential Revision: D19379395
fbshipit-source-id: 24782ccaf6e6ec6b445378b29d5f10f901e0dee6
