pytorch

14273126 - Numeric Suite: Swap with shadow modules only for quantized part of model (#51052)

Numeric Suite: Swap with shadow modules only for quantized part of model (#51052) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/51052 Ensure that shadow modules are inserted only for quantized modules in a model. Removes redundant module insertion. ghstack-source-id: 121041113 Test Plan: buck test caffe2/test:quantization -- 'test_compare_model_stub_partial \(quantization\.test_numeric_suite\.TestEagerModeNumericSuite\)' Reviewed By: vkuzo Differential Revision: D26054016 fbshipit-source-id: 73fc2fd2f0239b0363f358c80e34566d06a0c7cb