Remove workaround for TensorPipe failing to get device of CUDA ptr (#50580)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50580
Due to what looked like a bug in CUDA, TensorPipe was sometimes failing to auto-detect the device of a CUDA pointer. A workaround, on the PyTorch side, was to always initialize a CUDA context on device 0. Now that TensorPipe has fixed that we can undo the workaround.
Reviewed By: mrshenli
Differential Revision: D25952929
fbshipit-source-id: 57a5f73241f7371661855c767e44a64ca3b84a74
