Fix weight quantization in RNNs (#35961)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/35961
Weight quantization was done incorrectly for LSTMs, the statistics for all weights (across layers) were combined in the observer. This meant that weights for later layers in a LSTM would use sub-optimal scales impacting accuracy. The problem gets worse as the number of layers increases.
ghstack-source-id: 103511725
Test Plan: Will be updated
Differential Revision: D20842145
fbshipit-source-id: a622b012d393e0755970531583950b44f1964413
