Support compilation via Torchdynamo, AOT Autograd, NVFuser (#17308)
* Support compilation via Torchdynamo, AOT Autograd, NVFuser
* Address comments
* Lint
* Stas comments - missing quality test
* Lintere
* Quality test
* Doc lint
* Reset CUDA peak mem
* Add CustomTrainer
* require a single gpu
Co-authored-by: Stas Bekman <stas@stason.org>