pytorch

e1165088 - [ao] fix incorrect integer cast on histogram observer bounds (#90355)

[ao] fix incorrect integer cast on histogram observer bounds (#90355) Summary: A cast to int was added in https://github.com/pytorch/pytorch/pull/45630 to make mypy not complain. However this leads to unexpected behavior where the histogram doesn't actually capture the full range of activation values. note1: the test_histogram_observer_against_reference test was secretly broken, on master. The random parameters that normally get run apparently don't cause a test failure but if you make a loop repeatedly run the test, it would eventually fail. This was due to in some cases sum(<tensor>)!=torch.sum(<tensor>).item(). I was not able to reproduce this with a toy example but running this test in a loop and editing either observer to print the calculation for 'total' would break the test and show different behaviors. Fixing this test was necessary to land this PR since the changing histogram bounds changed things enough that this test would error. note2: updating histogram observer breaks some BC tests unless I regenerate the model using the HistogramObserver from this PR Test Plan: python test/test_quantization.py TestHistogramObserver.test_histogram_observer_correct_numel python test/test_quantization -k histogram Reviewers: Subscribers: Tasks: Tags: Pull Request resolved: https://github.com/pytorch/pytorch/pull/90355 Approved by: https://github.com/vkuzo