onnxruntime
b5ef8555 - Support onnx data types (bfloat16, float8) in python I/O binding APIs (#22306)

Commit
1 year ago
Support onnx data types (bfloat16, float8) in python I/O binding APIs (#22306) ### Description (1) Support onnx data types in python APIs: * IOBinding.bind_input * IOBinding.bind_output * ortvalue_from_shape_and_type (2) Add unit tests, which serves an example of running BFloat16 or Float8 models in Python. Other minor changes: (3) replace deprecated NP_TYPE_TO_TENSOR_TYPE by helper API. (4) Rename ortvalue_from_numpy_with_onnxtype to ortvalue_from_numpy_with_onnx_type. The integer of onnx element type can be found in (https://onnx.ai/onnx/api/mapping.html). Note that FLOAT4E2M1 is not supported yet. ### Motivation and Context Current python API does not support Bfloat16 and float8 (FLOAT8E4M3FN, FLOAT8E4M3FNUZ, FLOAT8E5M2, FLOAT8E5M2FNUZ) types, and other new data types like INT4, UInt4 etc. This removes the limitation. https://github.com/microsoft/onnxruntime/issues/13001 https://github.com/microsoft/onnxruntime/issues/20481 https://github.com/microsoft/onnxruntime/issues/20578
Author
Parents
Loading