[reland] Skip OpenMP thread when OMP_NUM_THREADS is set to 1 (#35353)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/35353
Pull Request resolved: https://github.com/pytorch/pytorch/pull/35324
When the OMP_NUM_THREADS is set to 1, we don't need to launch the parallel_for function on an OpenMP thread since there is no intra-op parallelism. By avoiding that, we can reduce the unnecessary context switches.
Test Plan: internal
Reviewed By: ilia-cher
Differential Revision: D20638734
fbshipit-source-id: 0d5a6537aa2fc35d8d0904c3b9e734e52585eee7
