Add `USE_MAGMA` build flag (#55994)
Summary:
Many model pipelines/workflows don't use MAGMA even though it is included in the build by default. Leaving MAGMA kernels out of the build can save 60+MB of GPU memory when loading `libtorch_cuda.so` (tested on V100, current upstream master).
A current sharp corner of this flag is that toggling it when rebuilding requires `torch/include/THC/THCGeneral.h` to be *manually* deleted by the user, as even running `make clean` or `setup.py` with `--cmake` does not properly regenerate it with the appropriate substitution for `#cmakedefine USE_MAGMA`. Is there a way to force the regeneration of the header during a rebuild?
CC malfet ptrblck
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55994
Reviewed By: mruberry
Differential Revision: D27766287
Pulled By: malfet
fbshipit-source-id: 93deca57befa0febb9c5b7875ecf0015c547d421
