pytorch
5977d1d8 - FixedQParamsFakeQuantize: adjust default quant_min and quant_max (#47423)

Commit
4 years ago
FixedQParamsFakeQuantize: adjust default quant_min and quant_max (#47423) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/47423 Since the dtype of this fake_quant is `quint8`, the output range should be from 0 to 255. Fixing. This should address the numerical inaccuracies with sigmoid and hardsigmoid with `FixedQParamsFakeQuantize` attached compared to their quantized counterparts. In a future PR, might be safer to also make the activation functions using `FixedQParamsFakeQuantize` to explicitly specify their expected output range and zero_point. Leaving that for later, as this bugfix should be landed urgently. Test Plan: Manual script which gives low SQNR before this PR and high SQNR after this PR: https://gist.github.com/vkuzo/9906bae29223da72b10d6b6aafadba42 https://github.com/pytorch/pytorch/pull/47376, which can be landed after this, adds a proper test. Imported from OSS Reviewed By: ayush29feb, jerryzh168 Differential Revision: D24751497 fbshipit-source-id: 4c32e22a30116caaceeedb4cd47146d066054a89
Author
Parents
Loading