[reland] Skip OpenMP Thread when OMP_NUM_THREADS is 1 (#35541)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/35541
When the OMP_NUM_THREADS is set to 1, we don't need to launch the parallel_for function on an OpenMP thread since there is no intra-op parallelism. By avoiding that, we can reduce the unnecessary context switches.
Test Plan: internal
Reviewed By: ilia-cher
Differential Revision: D20680465
fbshipit-source-id: 4476a810dfe7bf268fcd58fd00afb89ba61644cf
