[quant] avoid resize calls in observer/fake_quant (#60386)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60386
During QAT we sometimes encounter errors with scripted models
`RuntimeError: cannot resize variables that require grad`
For per-tensor cases we don't need to resize some buffers so this PR removes the extra resize ops where applicable
Test Plan: Imported from OSS
Reviewed By: vkuzo
Differential Revision: D29271905
fbshipit-source-id: 01a484a9559a3a4180490f9476d0cd3044ba0d1b
