Skip OpenMP thread when OMP_NUM_THREADS is set to 1 (#35324)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/35324
When the OMP_NUM_THREADS is set to 1, we don't need to launch the parallel_for function on an OpenMP thread since there is no intra-op parallelism. By avoiding that, we can reduce the unnecessary context switches.
Test Plan: internal
Reviewed By: ilia-cher
Differential Revision: D20630949
fbshipit-source-id: 0b6f1ba5b535dafedb16742145a70cc4bb4872a2
