onnxruntime
b40bdd80 - docs(quantization): clarify CPU parameter recommendations (#28415)

Commit
64 days ago
docs(quantization): clarify CPU parameter recommendations (#28415) ## Summary - Document recommended `quantize_static` parameters for CPU inference per target hardware (x64 non-VNNI, x64 VNNI, ARM). - Resolve the contradictory guidance previously present at the top of the `quantize_static` docstring. - Add a matching "Choosing parameters for CPU inference" section to the quantization tool's README. ## Motivation Fixes #26564. Users following the existing docstring with default arguments can end up with QDQ-quantized models that are slower than the FP32 baseline on x86/x64 CPUs, because the docs didn't surface the importance of `reduce_range=True` on non-VNNI hardware or the recommended `activation_type` / `weight_type` combination per platform. The previous opening of the `quantize_static` docstring also contradicted itself: it recommended `QuantType.QInt8` for both activations and weights in one sentence and asymmetric activations for CPU in the next. ## Changes - `onnxruntime/python/tools/quantization/quantize.py` - Rewrite the `quantize_static` opening paragraph to point at the new platform-keyed section instead of giving a single, contradictory recommendation. - Add a "Choosing parameters for CPU inference" section to the `quantize_static` docstring covering QDQ vs QOperator, x64 non-VNNI, x64 VNNI, ARM, and `per_channel`. - Add a short opening note to `StaticQuantConfig.__init__` cross-referencing the same guidance. - Clarify that this guidance is independent of `convert_onnx_models_to_ort --target_platform`. - `onnxruntime/python/tools/quantization/README.md` - Add a "Choosing parameters for CPU inference" section after "Quantization Arguments", with a short `python` example and the same platform-keyed bullets. No code, signatures, defaults, or behavior change. Documentation only. ## Test plan - `python -c "import ast; ast.parse(open('onnxruntime/python/tools/quantization/quantize.py').read())"` parses cleanly. - `lintrunner -a` reports no issues on the changed files. - Existing tests in `onnxruntime/test/python/quantization/` are unaffected by docs-only changes. ## Issue resolution Closes #26564.
Author
Parents
Loading