onnxruntime
3cc4cef0 - Add FLOAT8E8M0 data type support in ONNX Runtime (#28381)

Commit
121 days ago
Add FLOAT8E8M0 data type support in ONNX Runtime (#28381) ### Description This PR adds support for the `FLOAT8E8M0` data type in ONNX Runtime, following the same pattern used for existing Float8 types (Float8E4M3FN, Float8E4M3FNUZ, Float8E5M2, Float8E5M2FNUZ). #### FLOAT8E8M0 Format FLOAT8E8M0 is an 8-bit floating point format with: - 8 exponent bits - 0 mantissa bits - No sign bit (all values are positive) - Values are powers of two: `2^(val - 127)` - 0xFF is reserved for NaN - No infinity, no negative values, no denormals This format is useful as a scaling factor type in microscaling (MX) quantization formats. #### Changes **Core Type Definition** - `include/onnxruntime/core/common/float8.h` - Added `Float8E8M0` struct with float conversion routines and `std::numeric_limits` specialization. The float→Float8E8M0 conversion correctly handles NaN (including negative NaN) and infinity by checking these special values before sign-dependent logic, consistent with how `Float8E4M3FN` handles them. **C API** - `include/onnxruntime/core/session/onnxruntime_c_api.h` - Added `ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E8M0` enum value at ordinal 24 to match the ONNX protobuf spec (`FLOAT8E8M0 = 24`). Note: `UINT2` and `INT2` shift to ordinals 25 and 26 respectively to align with the spec. Language bindings (Java, C#, JS) will need corresponding updates in a follow-up PR. **Framework Integration** - `onnxruntime/core/framework/element_type_lists.h` - Added to AllFixedSizeIRv9, AllIRv9, AllFloat8 type lists - `include/onnxruntime/core/framework/to_tensor_proto_element_type.h` - Added template specialization - `include/onnxruntime/core/framework/data_types_internal.h` - Added dispatch cases - `include/onnxruntime/core/framework/data_types.h` - Added to IsTensorContainedType and IsSparseTensorContainedType - `onnxruntime/core/framework/utils.h` - Added GetONNXTensorElementDataType specialization - `onnxruntime/core/framework/data_types.cc` - Full type registration (tensor, sparse, seq, optional, prim) - `onnxruntime/core/framework/tensorprotoutils.cc` - UnpackTensor, size calculation, serialization - `onnxruntime/core/framework/tensor_type_and_shape.cc` - Type mapping - `onnxruntime/core/framework/onnxruntime_map_type_info.cc` - Type mapping **Shared Library Provider Bridge** - `onnxruntime/core/providers/shared_library/provider_interfaces.h` - `onnxruntime/core/providers/shared_library/provider_bridge_provider.cc` - `onnxruntime/core/providers/shared_library/provider_wrappedtypes.h` - `onnxruntime/core/session/provider_bridge_ort.cc` **Tests** - `onnxruntime/test/framework/float8e8m0_test.cc` - Unit tests for Float8E8M0 type including conversion, rounding, NaN (positive and negative), infinity (positive and negative), negative values, operators, and batch conversion - `onnxruntime/test/onnx/tensorprotoutils.cc` - Test infrastructure updates - `onnxruntime/test/unittest_util/checkers.cc` - Test checker updates ### Motivation and Context ONNX has added the FLOAT8E8M0 data type. This PR adds the corresponding support in ONNX Runtime so that models using this data type can be loaded and processed correctly. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: tianleiwu <30328909+tianleiwu@users.noreply.github.com> Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
Author
Parents
Loading