onnxruntime
49efb328 - Crypto support : App supply I/O callbacks to EP + callback and fallback helpers (#28624)

Commit
5 days ago
Crypto support : App supply I/O callbacks to EP + callback and fallback helpers (#28624) ## Summary This PR adds an opt-in mechanism that lets an application supply its own I/O callbacks for an execution provider's EPContext binary data, so the data can live somewhere other than a plain file on disk (for example, an encrypted store or an in-memory buffer). It introduces the callback APIs end-to-end and demonstrates their use with a sample helper in the AutoEP example plugin EP. When an EP compiles a model into an EPContext model, it may emit the compiled blob either embedded in the ONNX model or as a separate external payload. For the external case, ORT previously assumed the payload is a file. These callbacks let the application own that read/write instead, while ORT core stays policy-neutral and never imposes a storage format. ### What this PR adds - **Write callback (`OrtWriteNamedBufferFunc`) + setter `OrtCompileApi::ModelCompilationOptions_SetEpContextDataWriteFunc`.** Set on `OrtModelCompilationOptions`, because writing EPContext binary data happens only during **compilation**. Passing a NULL callback clears a previously set one. - **Read callback (`OrtReadNamedBufferFunc`) + setter `OrtApi::SessionOptions_SetEpContextDataReadFunc`.** Set on `OrtSessionOptions`, because reading external EPContext binary data happens during **session load / inference**. Passing a NULL callback clears a previously set one. - **EP-facing access via `OrtEpContextConfig`.** Both callbacks are surfaced to execution providers through a single unified handle, `OrtEpContextConfig`, obtained via `OrtEpApi::SessionOptions_GetEpContextConfig` (getters `EpContextConfig_GetEpContextDataReadFunc` / `EpContextConfig_GetEpContextDataWriteFunc`, released with `ReleaseEpContextConfig`). This keeps the application-facing setters scoped to the correct lifecycle while giving EPs one consistent place to retrieve both callbacks. Each setter's doc comment cross-references the other so the split is discoverable. - **Experimental API surface + C++ accessors.** These functions ship through ORT's experimental API mechanism (declared in `include/onnxruntime/core/session/onnxruntime_experimental_c_api.inc`), so they are reached via the generated `Ort::Experimental::Get_<name>_SinceV28_Fn(...)` / `...FnOrThrow(...)` accessors rather than fixed `OrtApi` slots. A move-only RAII wrapper, **`Ort::Experimental::EpContextConfig`** (in `onnxruntime_experimental_cxx_api.h`), owns an `OrtEpContextConfig` and exposes `GetReadFunc()` / `GetWriteFunc()`; it can be constructed directly from a C++ `SessionOptions` / `ConstSessionOptions`. - **Sample-only helper utilities** (`onnxruntime/test/autoep/library/ep_context_data_utils.h`) implementing callback-or-file fallback behavior: if a callback is supplied it is used, otherwise the helper falls back to direct file I/O. The AutoEP example plugin EP uses this helper for its external EPContext read/write paths. Because the names read on the load side originate from the untrusted EPContext model (`ep_cache_context` attribute), the helper validates them: it rejects absolute/rooted paths, `..` traversal, and directory-like names (`.` or a trailing separator), and confines model-relative names to the model directory (resolving `.`/`..` and symlinks via `std::filesystem::weakly_canonical`). It reports all failures via `OrtStatus*` (no exceptions) and lives outside the public C API / EP ABI, so it is purely illustrative and imposes no policy on ORT core; its doc comments note that production EPs should still apply their own sandboxing and payload size limits. The callback typedef names (`OrtReadNamedBufferFunc` / `OrtWriteNamedBufferFunc`) are intentionally generic. They are currently used for EPContext binary data, but the contract is deliberately storage-agnostic so future APIs can reuse the same callback shape for other named data payloads. ### Note on the Android workflow change `.github/workflows/android.yml` bumps the minimal-build binary-size threshold (`1436672` -> `1438720` bytes) to accommodate the small size increase from compiling the new experimental API into the Android minimal build. ## Testing - Built and tested in RelWithDebInfo: `python tools/ci_build/build.py --config RelWithDebInfo --build --parallel --test --build_dir build\Windows`. - Focused EPContext suites: - Public C/C++ API: `onnxruntime_shared_lib_test.exe --gtest_filter=EpContextDataApiTest.*` -> 9 passed. - AutoEP helper + compile/load end-to-end (callbacks and file fallback): `onnxruntime_autoep_test.exe --gtest_filter=*EpContext*` -> 17 passed, 1 skipped (`EpContextDataUtils_ResolvePathRejectsSymlinkEscape` requires the Windows "create symbolic link" privilege). - `clang-format` clean on touched C++ files; `git diff --check`: clean. Test layout: public EPContext API tests in `onnxruntime/test/shared_lib/test_ep_context_data_api.cc`; sample-helper unit tests in `onnxruntime/test/autoep/ep_context_data_utils_test.cc`; compile/load end-to-end tests in `onnxruntime/test/autoep/test_execution.cc`. --------- Co-authored-by: Gopalakrishnan Nallasamy <gnallasamy@microsoft.com> Co-authored-by: Gopalakrishnan Nallasamy <gopalakrishnan.nallasamy@microsoft.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Parents
Loading