pytorch
4fc0d534 - [PyTorch][Fix] Improve numerical stability of HistogramObserver (#86522)

Commit
2 years ago
[PyTorch][Fix] Improve numerical stability of HistogramObserver (#86522) Summary: As titled, HistogramObserver may fail in a certain scenario. Specifically, we originally compute `hist_bin_width` as `(self.max_val - self.min_val) / (self.bins * upsample_rate)`. It's possible that the numerator part is close the the FP32 threshold (1.4e-45) and conducting the division will cause overflow. Bring some redundent computations to avoid such scenario. Test Plan: https://pxl.cl/2ggD4 (https://github.com/pytorch/pytorch/commit/04490e90ea59229355b2771893719fe8896e80f0) Differential Revision: D40149594 Pull Request resolved: https://github.com/pytorch/pytorch/pull/86522 Approved by: https://github.com/jerryzh168
Author
Committer
Parents
Loading