[RPC] Fix a few flaky RPC tsan tests (#71460)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71460
When running with TSAN, we use a larger RPC timeout: https://github.com/pytorch/pytorch/blob/master/torch/testing/_internal/dist_utils.py#L68. As a result, the assertions here are invalid.
Tried to fix this by just setting `self.rpc_backend_options.rpc_timeout` to the new timeout, but `rpc_backend_options` is reconstructed every time it is accessed, so this doesn't work:: https://github.com/pytorch/pytorch/blob/master/torch/testing/_internal/distributed/rpc/tensorpipe_rpc_agent_test_fixture.py#L15
Just removing the asserts should be fine as they don't really add value to what's being tested.
ghstack-source-id: 147208455
Test Plan: CI
Reviewed By: fduwjj
Differential Revision: D33648421
fbshipit-source-id: 9a5052b1c851fe7f838792d8bdf17d0563b4aa00
(cherry picked from commit 96ddab3433aff88961236d2d64f2b685de1ccc15)
