[TensorPipe] Do not mark future messages as complete after they have timed out (#38931)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/38931
When requests time out they are not aborted, so they could in fact still complete successfully but, when they do so, they try to mark an errored future as complete, which causes an error. I don't see any atomic way of doing future->markCompleteIfNeeded, so we implement it on top of it on our side.
ghstack-source-id: 104760689
Test Plan: Hit this error in the RPC test suite, and it disappeared after this fix.
Differential Revision: D21703015
fbshipit-source-id: af92f7819ed907efb9b068a4ca65420739fac8cc
