Remove the activation observer for default_qconfig (#24299)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/24299
As suggested in https://github.com/pytorch/pytorch/pull/24232, we will remove the activation observer for dynamic quantization path.
ghstack-source-id: 88287094
Differential Revision: D16798590
fbshipit-source-id: 07a245d5584b5b15c6895d9b09deef4a0605073a
