onnxruntime
1d0712a8 - Enable Robust Symlink Support for External Data (HF Cache Support) (#27374)

Commit
3 days ago
Enable Robust Symlink Support for External Data (HF Cache Support) (#27374) ## Summary This PR enables ONNX Runtime to correctly load models with external data when they are stored in symlinked directory structures. This is a common scenario for models cached by the Hugging Face Hub, where both the model file and data files are symlinks pointing into a flat `blobs/` directory. ## The Problem Previously, ONNX Runtime's external data path validation would only check if the resolved data path was under the logical directory of the model. In symlinked structures (like Hugging Face's `snapshots/` and `blobs/` layout), the resolved data path often sits in a different physical directory than the logical model path, leading to a "path escapes model directory" error even when the data is safely associated with the model. ## The Fix I have updated `ValidateExternalDataPath` to implement a dual-check mechanism: 1. **Logical Check:** Verify if the resolved data path is under the provided `base_dir` (the directory where the model was loaded from). 2. **Physical Check:** If the logical check fails, verify if the resolved data path is under the parent directory of the **real/canonical** model path. This approach ensures that models can load external data from both their logical siblings and their physical siblings while maintaining security constraints. ## Changes - **Core Logic:** - `onnxruntime/core/framework/tensorprotoutils.cc/h`: Updated `ValidateExternalDataPath` to accept `model_path` and perform the dual-check. - `onnxruntime/core/graph/graph.cc`: Updated call site in `ConvertInitializersIntoOrtValues` to pass the `model_path`. - **Cleanup:** - Removed unused `ValidateExternalDataPath` from `provider_api.h`, `provider_interfaces.h`, and `provider_bridge_ort.cc`. These were redundant as path validation is handled by core during session initialization. Provider has no need to validate it. - **Testing:** - Added `onnxruntime/test/python/onnxruntime_test_python_symlink_data.py` to simulate the Hugging Face Hub symlink structure and verify the fix.
Author
Parents
Loading