Fix pin_memory_thread not exiting quickly (#23646)
Summary:
fixes https://github.com/pytorch/pytorch/issues/23642
Pull Request resolved: https://github.com/pytorch/pytorch/pull/23646
Differential Revision: D16600874
Pulled By: soumith
fbshipit-source-id: 50f0828d774a558d6f21e9dd21135906bd5be128
