[quant][graphmode][refactor] Move check for weight outside of insertObserverFor (#33276)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/33276
att
Test Plan:
.
Imported from OSS
Differential Revision: D20123593
fbshipit-source-id: 45dc8488ddf02225ba2c20374c9385edd77a4912
