Define PSIMD_SOURCE_DIR when including FP16 (#41233)
Summary:
Avoids a superflous redownload when *NNPACK is not used (e.g. on Power)
Example: https://powerci.osuosl.org/job/pytorch-master-nightly-py3-linux-ppc64le/1128/consoleFull
Search for "Downloading PSimd"
See also https://github.com/pytorch/pytorch/issues/41178
Pull Request resolved: https://github.com/pytorch/pytorch/pull/41233
Differential Revision: D22488833
Pulled By: malfet
fbshipit-source-id: 637291419ddd3b2a8dc25e211a4ebbba955e5855
