pytorch

3142ce20 - [quant][pt2e] Support quantizer API in prepare_pt2e_quantizer (#97994)

[quant][pt2e] Support quantizer API in prepare_pt2e_quantizer (#97994) Summary: This PR added a quantizer API to prepare_pt2e_quantizer, which enables user to annotate the nodes in the graph directly to configure quantization, instead of relying on QConfigMapping, please see test cases in test_quantize_pt2e.py for examples. Also added a prototype for QNNPackQuantizer, that will be modified later to fully support different quantization capabilities of QNNPack/XNNPack The goal for introducing quantizer is to add flexibility to the quantization API to allow modeling users and backend developers to express their quantization intentions programmably, which will free architecture optimization team from supporting different use cases in the core API in the future, as a concrete example, we used to have https://pytorch.org/docs/master/generated/torch.ao.quantization.qconfig_mapping.QConfigMapping.html#torch.ao.quantization.qconfig_mapping.QConfigMapping as the API for users to express their intent for quantization in fx graph mode quantization, and it has some fancy options like `set_module_name_regex` and `set_module_name_object_type_order`, this is not needed for all backends and adds burden of maintenance to AO team, in the quantizer API we will move these APIs to a backend specific `Quantizer` that needs this feature, and all the backends or even advanced modeling users can implement their own quantizer to express their intent for quantization through annotating the nodes, for example, to express the quantization intention of quantizing a convolution node, a user will find the convolution node in the graph and do: ``` operator_spec = qnnpack_quantizer.get_default_per_channel_symmetric_qnnpack_operator_spec() conv_node.meta["target_dtype_info"] = { "input_act_obs_or_fq_ctr": _get_act_obs_or_fq_ctr(operator_spec), "weight_obs_or_fq_ctr": _get_weight_obs_or_fq_ctr(operator_spec) "bias_obs_or_fq_ctr": _get_bias_obs_or_fq_ctr(operator_spec), "output_act_obs_or_fq_ctr": _get_act_obs_or_fq_ctr(operator_spec), # TODO: validation of weight_index must be set if weight_obs_or_fq_ctr is set "weight_index": 1, # TODO: validation of bias_index must be set if bias_obs_or_fq_ctr is set "bias_index": 2, } ``` each backend will introduce their own quantizer, e.g. QNNPackQuantizer, which may expose more convenient APIs for modeling users to configure the annotation, and different quantizer can compose with each other to annotate the graph correctly for quantization. Test Plan: python test/test_quantization.py TestQuantizePT2E.test_simple_quantizer python test/test_quantization.py TestQuantizePT2E.test_qnnpack_quantizer_conv Reviewers: Subscribers: Tasks: Tags: Pull Request resolved: https://github.com/pytorch/pytorch/pull/97994 Approved by: https://github.com/vkuzo