onnxruntime
3044aa87 - [Quant tool] Extend support for QDQ type conversion at graph output (#20841)

Commit
2 years ago
[Quant tool] Extend support for QDQ type conversion at graph output (#20841) ### Description Allows mixed-precision overrides that adds a QDQ quantization type conversion sequence at a graph output that **is not** consumed by other nodes. This is not a common use-case but should handle it instead of raising an error. #### Example Original model ![image](https://github.com/microsoft/onnxruntime/assets/19691973/4c9c3bb0-4ca1-4213-9259-9d0506ed22f2) mixed-precision overrides: ```python mixed_prec_overrides = { "input_0": [{"quant_type": QuantType.QUInt16}], "op_0_out": [ { "quant_type": QuantType.QUInt16, "convert": {"quant_type": QuantType.QUInt8}, } ], } quantize_static( float_model_path, qdq_model_path, data_reader, quant_format=QuantFormat.QDQ, activation_type=QuantType.QUInt8, op_types_to_quantize=[node.op_type for node in float_model.graph.node], extra_options={ "TensorQuantOverrides": mixed_prec_overrides, }, ) ``` QDQ model: ![image](https://github.com/microsoft/onnxruntime/assets/19691973/804fc89b-4a00-43bc-a4ff-21edd6f27e98) ### Motivation and Context This scenario is arising for certain quantization configurations. Should handle it gracefully.
Parents
Loading