onnxruntime

d6c363c0 - [OVEP] OpenVINO EP 1.26.0 Development Release Updates (#28297)

[OVEP] OpenVINO EP 1.26.0 Development Release Updates (#28297) ## Summary Periodic upstream sync of Intel's OVEP branch (`ovep_1_26_release`) into ORT main. All changes are scoped to the OpenVINO EP and its tests. ### OpenVINO 2026.0 / 2026.1 support - Add `V_2026_0` / `V_2026_1` version enums; `capability.cc` default bumped to `V_2026_1`. - Register FLOAT8E4M3FN / FLOAT8E5M2 initializer types on CPU / GPU / NPU. - Disable OVEP-level QDQ-stripping on OV ≥ 2026.1 (OV handles it internally). - Add `ReduceSum` to no-dimension-supported ops. ### KV-cache / stateful CausalLM - Rename `ReorderKVCache` → `SetReorderKVCacheStatus` across backend interfaces. - Populate `src_idx` / `dst_idx` in `PreProcessInferRequest` with shape validation; clean state after inference and on `RewindKVCache`. - `FuseCacheReorder`: `beam_idx` and `src_idx`/`dst_idx` paths are now mutually exclusive; reject models that already carry reorder inputs. - **Behavior change:** `RewindKVCache(index > 0)` now throws when reorder is enabled (physical KV-cache eviction pass is a TODO). ### NPU / provider options - Force `disable_dynamic_shapes=true` on NPU unless `enable_causallm` is set. - Preserve user-supplied `NPU_COMPILATION_MODE_PARAMS`; skip it when importing precompiled blobs. - Preserve factory-level `device_type` when session options don't override it (fixes NPU mis-selection from Python). - **Behavior change:** removed the `ORT_OPENVINO_NPU_COMPILER_TYPE` env override — OV's default NPU compiler is used now. ### External initializers / weight sharing - Drop the 32 MB embed threshold — always externalize when multiple external initializers are in memory. - `DumpOpenVINOEPModel` rebuilds a self-contained proto when initializer data was stripped. - `AddExternalWeight` validates re-adds against existing offset/size/location (parity with ABI EP); fix race in device-tensor mapping. - `ov_bin_manager`: bounds-checked pointer view over mapped weights (fixes read-only blob import). - `qdq_stripping`: use `std::from_chars` so offsets/lengths > 4 GB parse correctly. ### Perf-count dump - New `ORT_OPENVINO_PERF_COUNT=<dir>` env var writes per-subgraph CSV (`Layer Name,Status,Layer Type,Real Time (us),Exec Type`), replacing the old stdout-only debug dump. Requires `ov::enable_profiling` on the compiled model; logs a warning and no-ops otherwise. ### Misc - **API:** `IBackend::Infer` is no longer `const` (needed for perf-dump bookkeeping). - Filter orphaned graph outputs from OVEP sub-graphs. - Better error message for "cannot export dynamically compiled model" (points to `reshape_input`). - Human-readable `ovep_exception::type` strings. - `ov::shutdown()` on DLL unload. ### Tests - Add `OVEP_ExtInit_DynamicEmbed_Tests` and `OVEP_ExtInit_EmptyRawData_Tests`; refactor setup into `SetUpTestSuite`. - Narrow OVEP exclusions in `embed_layer_norm`, `fused_matmul`, `matmul_4bits`, `quantize_linear` (skip only unsupported sub-cases). - `perftest`: reset outputs per run to support data-dependent output shapes (e.g. NonZero). ## Testing Validated against the OpenVINO versions this release targets (2025.3 – 2026.1) on CPU / GPU / NPU: - New OVEP tests pass: `OVEP_ExtInit_Tests`, `OVEP_ExtInit_DynamicEmbed_Tests`, `OVEP_ExtInit_EmptyRawData_Tests` - Narrowed contrib-op exclusions verified against EmbedLayerNorm, FusedMatMul, MatMulNBits, QuantizeLinear - Stateful CausalLM flow exercised for KV-cache reorder + rewind - `ORT_OPENVINO_PERF_COUNT=<dir>` verified to produce per-subgraph CSVs - 2+ GB external-initializers-in-memory model loads on CPU / GPU / NPU --------- Signed-off-by: Jonathan Clohessy <jonathan.clohessy@arm.com> Signed-off-by: bfilipek <bartlomiej.filipek@intel.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com> Co-authored-by: Ryan Metcalfe <107415876+RyanMetcalfeInt8@users.noreply.github.com> Co-authored-by: jatinwadhwa921 <110383850+jatinwadhwa921@users.noreply.github.com> Co-authored-by: Jaswanth Gannamaneni <jaswanth.gannamaneni@intel.com> Co-authored-by: Klimenko, Mikhail <mikhail.klimenko@intel.com> Co-authored-by: Vishnudas Thaniel S <vishnudas.thaniel.s@intel.com> Co-authored-by: n1harika <niharika.sathish@intel.com> Co-authored-by: TejalKhade28 <tejal.khade@intel.com> Co-authored-by: Preetha Veeramalai <preetha.veeramalai@intel.com> Co-authored-by: liang <gxgaoliang@126.com> Co-authored-by: Javier Martinez <javier.e.martinez@intel.com> Co-authored-by: MayureshV1 <47039074+MayureshV1@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: sfatimar <sahar.fatima@intel.com> Co-authored-by: Garth Long <garth.long@intel.com> Co-authored-by: Eric Crawford <eric.r.crawford@intel.com> Co-authored-by: derdeljan-msft <derdeljan@microsoft.com> Co-authored-by: Jonathan Clohessy <jonathan.clohessy@arm.com> Co-authored-by: Akshay Sonawane <111780983+apsonawane@users.noreply.github.com> Co-authored-by: Christopher Warrington <chwarr@microsoft.com> Co-authored-by: Ishwar Raut <iraut@nvidia.com> Co-authored-by: Gaurav Garg <gaugarg@nvidia.com> Co-authored-by: Xinpeng Dou <15529241576@163.com> Co-authored-by: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Co-authored-by: adrastogi <aditya.rastogi@microsoft.com> Co-authored-by: Aditya Rastogi <adityar@ntdev.microsoft.com> Co-authored-by: qti-hungjuiw <hungjuiw@qti.qualcomm.com> Co-authored-by: qti-yuduo <yuduow@qti.qualcomm.com> Co-authored-by: Pradeep Sakhamoori <psakhamoori@microsoft.com> Co-authored-by: Adam Pocock <adam.pocock@oracle.com> Co-authored-by: Changming Sun <chasun@microsoft.com> Co-authored-by: mingyue <131847423+mingyueliuh@users.noreply.github.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: Susanta Bhattacharjee <susanta.bhattacharjee@intel.com> Co-authored-by: jatinwadhwa921 <jatin.wadhwa@intel.com> Co-authored-by: Jozef Wludzik <jozef.wludzik@intel.com> Co-authored-by: Bartlomiej Filipek <bartlomiej.filipek@intel.com> Co-authored-by: Kotomi-Du <yaru.du@intel.com> Co-authored-by: Rajeev Sekar <rajeevsekar21@gmail.com> Co-authored-by: Mayuresh M Varerkar <mayuresh.m.varerkar@intel.com> Co-authored-by: Mikhail Dvoretckii <mikhail.dvoretckii@intel.com> Co-authored-by: bopeng1234 <bo.peng@intel.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: fs-eire <7679871+fs-eire@users.noreply.github.com> Co-authored-by: Wenqin Yang <wenqin.yang@intel.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xieofxie <xieofxie@126.com> Co-authored-by: hualxie <hualxie@microsoft.com> Co-authored-by: Jiajia Qin <jiajiaqin@microsoft.com> Co-authored-by: Joshua Lochner <admin@xenova.com> Co-authored-by: Christian Bourjau <cbourjau@users.noreply.github.com> Co-authored-by: Xiaofei Han <xiaofeihan@microsoft.com> Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com> Co-authored-by: chunghow-qti <chunghow@qti.qualcomm.com> Co-authored-by: Guenther Schmuelling <guschmue@microsoft.com> Co-authored-by: Jiawei Shao <jiawei.shao@intel.com> Co-authored-by: Tianlei Wu <tlwu@microsoft.com> Co-authored-by: czekun <chen.zekun@intel.com> Co-authored-by: Ryan Metcalfe <ryan.metcalfe@intel.com> Co-authored-by: Jaskaran Singh Nagi <jaskaran.singh.nagi@intel.com> Co-authored-by: ai-fw-intg <sys_ai_fw_intg@intel.com> Co-authored-by: Rajeev Sekar <rajeev.sekar@intel.com> Co-authored-by: RajeevSekar <117911837+RajeevSekar@users.noreply.github.com> Co-authored-by: Nazanin Beheshti <nazanin.beheshti@intel.com>