pytorch

31f311a8 - [PT2E][Quantization] Refactor Quantizer and QNNPACKQuantizer (#99063)

[PT2E][Quantization] Refactor Quantizer and QNNPACKQuantizer (#99063) This diff renames quantization spec/config and operator config. It moves these datastructures to base quantizer. Base quantizer API now has get_supported_operators that returns list of patterns that a quantizer quantizes. There are two choices being debated for how to convey to user what a particular quantizer will quantize. 1. Modules. We just convey what nn.Modules will be quantized. Of course that does not mean that equivalent functional variants wont be quantized, however for simplifity we just use nn.Module. If certain ops are quatnzied in fused manner then that will considered internal details. Pros and cons of this approach pros: - Simple. Only nn Modules are listed. - User does not have to see fusion patterns. Cons: - confusing perhaps because it is not clear if supported = nn.Conv2d also means that the quantizer supported functional.conv2d - Hiding fusion pattern means user has no say in not fusing. Meaning if conv2d + relu is fused and user configures to quantize only conv, quantizer will also quantize the following relu as if conv2d + relu are fused. 2. Patterns. Be explicit about what is supported and enumerate all possible compbinations. Pros: - it is very clear what quantizer will do. no surprises. Cons: - It is not simple to parse. - It can be argued taht fusion is internal detail of the quantizer. So some quantizer implementation may chose to expose fusion patterns, while others may not and may not even provide any configurability. One option is to move set_supported_operators/modules out of base quantizer and let each quantizer define its own way of communicating what is supported. Issue with this is that when we want to "Compose" multiple quantizers there is no way for user to define the order of composition if user does not know what a quantizer supports. For exampl quantizer A may quantizer conv + relu while B only conv, but B's implementation is fast. In that case you may compose (B, A) such B quantizes conv and A quantizes relu. Not knowning what A and B support, makes such composition harder Differential Revision: [D44895547](https://our.internmc.facebook.com/intern/diff/D44895547/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D44895547/)! Pull Request resolved: https://github.com/pytorch/pytorch/pull/99063 Approved by: https://github.com/jerryzh168