onnxruntime
780c0c01 - Bind external data validation to opened files (#33063)

Commit
5 days ago
Bind external data validation to opened files (#33063) This pull request enhances the security and robustness of external data file handling in ONNX Runtime by improving how external files are accessed, validated, and mapped into memory. The changes introduce canonical path and memory-mapping support to the `RandomAccessFile` abstraction, update file access logic to use open file handles rather than paths, and add stricter validation to prevent directory traversal attacks. Additionally, platform-specific implementations and tests have been updated to support these improvements. **Security and Path Validation Enhancements:** - Added `ValidateOpenedExternalDataPath` to ensure that opened external data files do not escape the model directory, mitigating directory traversal vulnerabilities. This validation is now called after opening external data files and before reading or mapping their contents. [[1]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406R571-R598) [[2]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1808-R1839) [[3]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1888-R1923) **RandomAccessFile API Extensions:** - Extended the `RandomAccessFile` interface with `GetCanonicalPath` and `Map` methods, enabling platform-independent retrieval of the canonical file path and memory-mapping of file regions. - Implemented these methods for POSIX (`PosixRandomAccessFile`) and Windows (`WindowsRandomAccessFile`) platforms, including robust error handling and platform-specific path normalization. [[1]](diffhunk://#diff-198b6ccde9c2ad51179a562793bb22e08f5b3cd631669fa1d93b27474e0f07d3R144-R196) [[2]](diffhunk://#diff-18a797524bf995a8ceddadab1fbe3731d4a0754d38ad56cd8b3f1d4c609af763R381-R437) **Refactoring of File Access Logic:** - Updated all external data file reading and prepacked weight loading logic to use the `RandomAccessFile` abstraction for reads and memory mapping, instead of repeatedly opening files by path. This ensures consistency and prevents issues if the file path is changed after opening. [[1]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1627-R1665) [[2]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1649-R1676) [[3]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1676-R1702) [[4]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1695-R1720) [[5]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1820-R1849) [[6]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1850-R1878) [[7]](diffhunk://#diff-d31e9fbe0f5334fcd949833e035f2b25d5ae810dcd505c545f6b372b546b1406L1888-R1923) **Testing Improvements:** - Added a new test to verify that memory mapping and canonical path retrieval operate correctly even if the file path is replaced after the file is opened, ensuring the API behaves as documented. **Platform and Build Enhancements:** - Included necessary headers and improved platform compatibility for new features, such as memory mapping and canonical path retrieval. These changes collectively make external data handling more secure, reliable, and portable across platforms. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Author
Parents
Loading