onnxruntime
af0588b4 - [EP ABI] Add API to select the best compiled model compatibility info from candidate strings (#28387)

Commit
8 days ago
[EP ABI] Add API to select the best compiled model compatibility info from candidate strings (#28387) ### Description This PR adds a new EP API, `SelectBestModelCandidate`, that selects the best model variant from a set of candidates described by key-value metadata. The EP evaluates each candidate's metadata against the given hardware device and optional session options, and returns the index of the best match. **Key design points:** - Each candidate is an `OrtKeyValuePairs` representing one model variant. This future-proofs the API — additional metadata keys can be added over time without changing the function signature. - **Single-model variants (simple case):** The KVP contains a single `ep_compatibility_info` key with the compatibility string from the ONNX model metadata. - **Multi-model variants:** When a variant bundles multiple sub-models (e.g., prefill + decode in a GenAI scenario), the KVP uses indexed keys so the EP can inspect each sub-model independently: - `num_models` — number of sub-models (e.g., "3") - `<i>.ep_compatibility_info` — compatibility string for sub-model *i* (required per sub-model) - `<i>.role` — role of sub-model *i*, e.g., "prefill", "decode" (optional) - `<i>.future_meaningful_info` — additional EP-meaningful metadata for sub-model i (optional) A basic EP implementation validates all `<i>.ep_compatibility_info` entries. An advanced implementation can also consider `role` or other metadata for smarter ranking. - This approach delegates variant selection entirely to the EP, which has the domain knowledge to handle structurally mismatched variants (different sub-model counts, different roles, etc.) without ORT needing to understand model roles or compute aggregated scores. ### Motivation and Context The existing `ValidateCompiledModelCompatibilityInfo()` alone is not sufficient for some EPs to determine the best compatible model when there are multiple candidates. For example, an EP may support multiple compilation modes (e.g., "speed optimized" vs "memory optimized") that produce different compatibility strings. The EP can implement this function to evaluate the candidate metadata and select the best compatible variant based on its own criteria and the target device. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Author
Parents
Loading