[reland][quant] QuantizedCUDA implementation (#36936) (#37081)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/37081
Closes https://github.com/pytorch/pytorch/issues/30813
Relanding of https://github.com/pytorch/pytorch/pull/35463
1. Tensor quantization logic(quantize_*) is moved to the aten/native/quantized. Previously all logic for tensor quantization lived in the aten/quantized/Quantizer.cpp file, and started to become complicated and hard to read. This problem should be addressed in refactoring PR. Still, I reworked this partially because I had to add tensor quantization logic for CUDA, and it was native to move everything to the aten/native/quantized.
2. Requirements to run CUDA_tensor_apply* was eased to process any tenser that lives on the CUDA device(QuantizedCUDA included).
3. All quantized data types now have a default constructor. NVCC refuses to compile any gpu_kernel or CUDA_tensor_apply* without them.
4. Minor changes in many files to register QuantizedCUDA backend.
5. test_quantized_tensor is extended to process QuantizedCUDA backend where possible.
Test Plan: Imported from OSS
Differential Revision: D21206694
Pulled By: jerryzh168
fbshipit-source-id: c7433aad9c095a34c57e6dddd128b5c5d9292373
