onnxruntime

76936d30 - Expand model package with authoring tools and schema versioning (#28989)

Expand model package with authoring tools and schema versioning (#28989) ### Description Builds out the standalone `model_package/` C library so a single library covers the full lifecycle of an ONNX Runtime model package: inspection, authoring, content-addressed shared assets, commit, prune, and validation. The library remains free of any dependency on ONNX Runtime itself. **Public C API (`model_package/include/model_package.h`)** - Lifecycle: `ModelPackage_Open` / `ModelPackage_New` / `ModelPackage_Close`, with `ModelPackageOpenOptions` controlling external-path access, symlink following, and strict unknown-field handling. - Inspection: a POD `ModelPackageInfo` tree (`ModelPackage_Info`) plus by-name lookups for components, variants, and per-namespace `executor_info` entries, and round-trip JSON getters that preserve fields unknown to the current build. - Path resolution: `ModelPackage_ResolveStringRef` implements the package's resolution rules — relative paths anchored at a base directory, `sha256:<hex>[/sub/path]` for shared-asset content, and portable vs installed confinement (absolute paths and `..` segments only allowed under `layout: "installed"`). - Shared assets: SHA-256 directory hashing, `ModelPackage_AddSharedAsset` (with reproducible-build URI check and an optional `copy_in` staging mode), `ModelPackage_RemoveSharedAsset`, and `ModelPackage_ResolveAssetUri`. Assets under `<package_root>/shared_assets/` are auto-discovered at `Open`. - Authoring: inline/external component setters, variant upsert/remove, per-namespace executor_info setters (inline and external), and package-level metadata / layout / `additional_metadata` setters. Mutations invalidate cached pointers in the mutated scope and its descendants. - Commit / prune / validate: `ModelPackage_Commit` writes the in-memory model to disk either in place or to a fresh `dest_root` ("save as"), with `PRESERVE` or `DENSE` write modes; `ModelPackage_Prune` reclaims unreferenced files under `shared_assets/` and tracked orphan variant/component directories left behind by removals; and `ModelPackage_Validate` runs schema, path-reachability, asset-rehash, and unknown-field checks and returns a JSON report. - Errors: opaque `ModelPackageStatus*` with a stable additive `ModelPackageErrorCode` enum (IO, schema, version, path confinement, asset missing, asset hash mismatch, not found, invalid arg, state). **Internal layout** The implementation is split into focused translation units: `manifest_parser`, `model_package_impl`, `authoring`, `commit_prune_validate`, `path_resolver`, `asset_hasher`, and an in-tree `sha256`. Shared error plumbing lives in `status_impl.h`. **ONNX Runtime integration (`onnxruntime/core/session/model_package/`)** The ORT-side glue is wired onto the library through the C inspection and path-resolution entry points. `model_package_context` now translates the library's info tree into ORT-internal structs and parses the `executor_info["ort"]` payload (`model_file`, `external_data`, `session_options`, `provider_options`). When a variant declares `external_data`, `CreateSessionForModelPackage` loads the model from a memory mapping and passes the resolved folder to the session via `kOrtSessionOptionsModelExternalInitializersFileFolderPath`, so external initializers — including those backed by a shared asset — are picked up at `Initialize` time. The experimental `OrtModelPackageApi_*_SinceV28` C entries introduced in #28990 are unchanged. **Documentation and tests** - `model_package/README.md` documents the on-disk layout, manifest and component schemas, shared-asset rules, path resolution, the authoring flow, and commit / prune / validate semantics. - `onnxruntime/core/session/model_package/README.md` documents the ORT consumer-side glue: the `executor_info["ort"]` schema, the variant selection algorithm, the session-creation contract, and the registered experimental C entries. - New library tests cover inspection, authoring, asset hashing, and commit (`model_package/tests/`). The ORT integration tests in `onnxruntime/test/autoep/test_model_package.cc` are reworked against the current C API surface. ### Motivation and Context ORT needs a single library that owns the model package format end to end — not just reading it, but producing it, validating it, and maintaining it on disk with content-addressed shared assets. Consolidating this behind one C API lets ORT, publisher tooling, and third-party consumers share the same parser, path-resolution rules, and on-disk invariants without each reimplementing them, and keeps the library independent of the ORT session runtime. --------- Co-authored-by: jambayk <jambayk@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: jambayk <jambay@github.com>