onnxruntime
8585c64b - [CPU] Fix output saturation in int8 QLinearSoftmax (#29728)

Commit
68 days ago
[CPU] Fix output saturation in int8 QLinearSoftmax (#29728) ### Description Uses the correct int8 range to clamp/saturate the output of the int8 specialization of the QLinearSoftmax kernel. Previously there was a copy-paste error, and it was saturating to 255 like in the uint8 specialization. https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#com.microsoft.QLinearSoftmax This change also adds a unit test case that causes output saturation, and reproduced the bug. ### Motivation and Context When there are: 1. Softmax outputs very close to 1 (in the original float graph), and 2. a y_scale of 1.0 / 256.0 and y_zero_point of -128 (as set by https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/quantization/operators/softmax.py) then the output of int8 QLinearSoftmax will saturate. Incorrect clamping is causing this to wrap around to -128 instead, producing incorrect results. https://github.com/microsoft/onnxruntime/issues/29727
Author
Parents
Loading