pytorch
4668d09c - [quant][graphmode][fx] Quantize the output of statically quantized fp16 op in QuantizeHandler (#58445)

Commit
3 years ago
[quant][graphmode][fx] Quantize the output of statically quantized fp16 op in QuantizeHandler (#58445) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/58445 Previously the output of statically quantized fp16 operator is not quantized in QuantizeHandler, which is not consistent with the behavior of static int8 operators. Also it does not work well with reference functions, this PR changes the fp16 static QuantizeHandler to quantize (call to(torch.float16)) in the QuantizeHandler, this also makes the future support for reference functions easier. Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps Imported from OSS Reviewed By: vkuzo Differential Revision: D28495830 fbshipit-source-id: 2140eab8ab2dd08f6570d9e305485e3029e1f47d
Author
Parents
Loading