onnxruntime
d3b86e7c - Canonicalize external data locations (#32135)

Commit
3 days ago
Canonicalize external data locations (#32135) This pull request improves the robustness and correctness of handling external data references in ONNX Runtime, particularly focusing on rejecting invalid or ambiguous "location" entries and ensuring in-memory reference tags are not treated as file paths. The changes also enhance path validation logic and add new unit tests to cover these scenarios. **Validation and error handling improvements:** - Added a check in `ExternalDataInfo::Create` to reject duplicate "location" entries in the external data info, ensuring only one "location" is allowed per tensor. [[1]](diffhunk://#diff-df6326d7af04c52e54c61249ff8b1980da9bac81192ee2e63c5b2280d7b91e05R32) [[2]](diffhunk://#diff-df6326d7af04c52e54c61249ff8b1980da9bac81192ee2e63c5b2280d7b91e05L42-R47) - Updated path validation logic in `ValidateExternalDataPathFromDir` and `ValidateExternalDataPath` to explicitly reject in-memory reference tags (used for internal memory mapping) as invalid file paths, alongside empty and absolute paths. [[1]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L410-R417) [[2]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L423-R437) [[3]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L438-R446) [[4]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L448-R457) [[5]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L469-R477) [[6]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L497-R509) **Functionality and code clarity:** - Refined `HasExternalDataInMemory` to correctly detect in-memory references even if there are multiple "location" entries, improving reliability. **Testing enhancements:** - Added new unit tests to verify that duplicate "location" entries are rejected and that in-memory reference tags are correctly handled and rejected by validation logic. [[1]](diffhunk://#diff-d75ec5db9cc4642f78b6ff568aff6d10398fc211b0fb7c862d3ec88738e3eda6R146-R188) [[2]](diffhunk://#diff-d75ec5db9cc4642f78b6ff568aff6d10398fc211b0fb7c862d3ec88738e3eda6R771-R782) These changes collectively strengthen the validation of external data references, prevent ambiguous or invalid configurations, and ensure that in-memory tags are never misinterpreted as file paths.
Author
Parents
Loading