pytorch
6ee41974 - Speedup Linux nightly builds (#44532)

Commit
4 years ago
Speedup Linux nightly builds (#44532) Summary: `stdbuf` affects not only the process it launches, but all of its subprocessed, which have a very negative effects on the IPC communication between nvcc and c++ preprocessor, which results in 2x slowdown, for example: ``` $ time /usr/local/cuda/bin/nvcc /pytorch/aten/src/THC/generated/THCTensorMathPointwiseByte.cu -c ... real 0m34.623s user 0m31.736s sys 0m2.825s ``` but ``` time stdbuf -i0 -o0 -e0 /usr/local/cuda/bin/nvcc /pytorch/aten/src/THC/generated/THCTensorMathPointwiseByte.cu -c ... real 1m14.113s user 0m37.989s sys 0m36.104s ``` because OS spends lots of time transferring preprocessed source back to nvcc byte by byte, as requested via stdbuf call Pull Request resolved: https://github.com/pytorch/pytorch/pull/44532 Reviewed By: ngimel Differential Revision: D23643411 Pulled By: malfet fbshipit-source-id: 9fdaf8b8a49574e6b281f68a5dd9ba9d33464dff
Author
Parents
Loading