onnxruntime
1990bd5f - Add CudaQuantizer to onnxruntime.quantization (#29509)

Commit
100 days ago
Add CudaQuantizer to onnxruntime.quantization (#29509) ## Description Adds a CUDA-focused `CudaQuantizer` helper to ONNX Runtime's Python quantization tools and exports it from `onnxruntime.quantization`. The helper centralizes the CUDA quantization paths needed by ONNX Runtime tests and `onnxruntime-genai` model building, covering both per-channel QMoE weight packing and MatMulNBits blockwise formats without duplicating that logic in test code. This also updates the CUDA MoE/QMoE transformer tests to consume the shared helper, including full-range symmetric offset storage and CUTLASS mixed-GEMM prepacked layouts. ## Summary of Changes ### Quantization API | File | Change | |------|--------| | `onnxruntime/python/tools/quantization/cuda_quantizer.py` | Adds `CudaQuantizer` with per-channel INT4/INT8 quantization, CUDA mixed-GEMM prepacking, MatMulNBits blockwise quantization, CUTLASS prepacked blockwise quantization, and a pure-PyTorch symmetric blockwise reference path. | | `onnxruntime/python/tools/quantization/__init__.py` | Exports `CudaQuantizer` from `onnxruntime.quantization`. | | `onnxruntime/python/tools/quantization/qmoe_quantizer.py` | Replaced by the more general CUDA quantizer implementation. | ### Test Updates | File | Change | |------|--------| | `onnxruntime/test/python/transformers/test_moe_cuda.py` | Replaces duplicated blockwise quantization/prepacking logic with `CudaQuantizer` calls. | | `onnxruntime/test/python/transformers/test_qmoe_cuda.py` | Uses `CudaQuantizer` for per-channel and blockwise QMoE test setup, including full-range unsigned offset storage coverage and default prepacked layout smoke tests. | ## Testing - Built the CUDA Release configuration. - `pytest transformers/test_qmoe_cuda.py::TestQMoEIntPrePackSmoke` - 5 passed, 2 subtests passed. - `pytest transformers/test_qmoe_cuda.py` - 97 passed, 12 skipped, 2 subtests passed. - Validated without CUDA that importing the quantization module does not require `torch`, and that torch-backed methods raise a clear error only when used. - `python -m py_compile onnxruntime/python/tools/quantization/cuda_quantizer.py` ## Checklist - [x] Tests added/updated - [x] No breaking changes to released APIs - [ ] CI passes --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Author
Parents
Loading