onnxruntime
36017adc - [EP ABI] Add CreateCustomOpDomains() API for plugin EP to register custom ops (#27050)

Commit
191 days ago
[EP ABI] Add CreateCustomOpDomains() API for plugin EP to register custom ops (#27050) ### Description The newly added two APIs, `CreateCustomOpDomains()` and `GetNumCustomOpDomains`, are used when running inference on a model that contains EP-specific custom operations. Workflow: 1. The EP implements these functions to supply a list of `OrtCustomOpDomain` instances. 2. The application either 1) calls `SessionOptionsAppendExecutionProvider_V2()` with an `OrtEpDevice` containing the plugin EP's factory or 2) enables auto ep selection. 3. Then ORT either 1) `SessionOptionsAppendExecutionProvider_V2()` appends the provided OrtCustomOpDomains to the session options or 2) registers the OrtCustomOpDomains from the selected EP devices. As a result, any session created from these session options will have these custom op domains registered in ORT, ensuring that the custom ops are properly recognized and validated when the model is loaded. Plugin EPs can provide two types of custom ops: 1. A full OrtCustomOp with a concrete kernel implementation - This Example EP demonstrates this approach. - In GetCapability(), it calls EpGraphSupportInfo_AddSingleNode() to inform ORT that the custom node should NOT be fused or compiled. Instead, ORT should invoke the custom node's Compute() function at runtime. 2. A "placeholder" OrtCustomOp with an empty kernel implementation - A compile-based Plugin EP can supply an OrtCustomOp whose CustomKernel::Compute() does nothing. The purpose is to satisfy model validation during model loading by registering the custom op as a valid operator in the session. - In GetCapability(), the EP should call EpGraphSupportInfo_AddNodesToFuse() to notify ORT that this custom node should be fused and compiled by the EP. - In Compile(), the EP executes its compiled bits to perform inference for the fused custom node. ### Motivation and Context Currently, the provider-bridge TRT RTX EP and TRT EP supports registering custom op domain list in session option so that it can run model contains TRT specific custom ops. This PR adds the same feature for plugin EP.
Author
Parents
Loading