Revert D25293474: [pytorch][PR] Server connects to its listen socket addr
Test Plan: revert-hammer
Differential Revision:
D25293474 (https://github.com/pytorch/pytorch/commit/7c9ba621305f939891bbb6219cf9151092d0e301)
Original commit changeset: 15f75dab48a4
fbshipit-source-id: 71ca136f2aa3204ad49f76c604f51c477cba270a
