Enable threadpool threads to greedily acquire new tasks if available. (#17808)
Summary:
This improves locality and affinity by keeping work on the same
threads preferentially to starting work on new ones, and reduces
contention on the threadpool lock more generally.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/17808
Differential Revision: D14391282
Pulled By: resistor
fbshipit-source-id: 3aec81656a50460a725aa4187c61864295d4f46e
