Lower libtorch build parallelization to avoid OOM (#94548)
Memory usage increases after https://github.com/pytorch/pytorch/pull/88575. Docker crashes with exit code 137, clearly means out of memory
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94548
Approved by: https://github.com/seemethere
