Do not insert observers for empty sequential modules (#28384)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/28384
ghstack-source-id: 92340259
Test Plan:
buck test caffe2/test:quantization -- 'test_fusion_sequential_model_train \(test_quantization\.FusionTest\)' --print-passing-details
buck test caffe2/test:quantization -- 'test_fusion_sequential_model_eval \(test_quantization\.FusionTest\)' --print-passing-details
Differential Revision: D18047293
fbshipit-source-id: 7e18b1aa76cc0fd26e8ee48a70c3a45688e73549
