onnxruntime
8705c68c - Compile API: output model and initializer stream write functions (#25455)

Commit
157 days ago
Compile API: output model and initializer stream write functions (#25455) ### Description - Adds `ModelCompilationOptions_SetOutputModelWriteFunc` to the compile API to allow writing the output model ONNX bytes to a user-provided write function (i.e., for streaming). - Adds `ModelCompilationOptions_SetOutputModelHandleInitializerFunc` to the compile API to allow the user to write individual initializers to some destination. Also allows specifying if an initializer should be embedded within the ONNX model or written to a custom file. - Adds C++, Python, and C# bindings for the new APIs. A follow-up PR adds a write function for EPContext node binary data: https://github.com/microsoft/onnxruntime/pull/25471 ### Example `ModelCompilationOptions_SetOutputModelWriteFunc`: https://github.com/microsoft/onnxruntime/blob/c62ed23c328cbbfefd3083c1f7a6ced604772c19/onnxruntime/test/providers/qnn/qnn_ep_context_test.cc#L2075-L2131 `ModelCompilationOptions_SetOutputModelHandleInitializerFunc`: https://github.com/microsoft/onnxruntime/blob/c62ed23c328cbbfefd3083c1f7a6ced604772c19/onnxruntime/test/providers/qnn/qnn_ep_context_test.cc#L2160-L2292 ### Motivation and Context Add output streaming capabilities when saving compiled models.
Parents
Loading